JP2005172548A - Supervisory device and database construction method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To offer a supervisory device capable of dispensing with signals from many microphones, as well as complicated data processings and accumulation of a huge amount of data. <P>SOLUTION: A signal from a microphone 4 is inputted into a sound board 100 installed in a personal computer 2. The inputted sound data are sampled and are stored in a memory 102 as digital data. Distribution of the integrated intensity of the frequency spectrum for each range divided into a predetermined frequency band is defined as a phoneme by a CPU(central processing unit) 104 for the frequency spectrum of the sound collected. The phonemes are accumulated in the database; and if the coefficient of correlation between the stored phonemes in the database of sound and the phonemes of the newly collected sound is below a predetermined value, it is decided that the new sound is an abnormal sound, and a predetermined action is performed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a monitoring device for grasping the status of a region to be monitored and the state of a device that is operated continuously or unattended, for example. Furthermore, the present invention relates to a database construction method for constructing a sound database useful for such a monitoring device.

  Conventionally, a monitoring device for monitoring an abnormal situation such as illegal entry into a building or the occurrence of a fire has been used. Many of these monitoring apparatuses have used a monitoring camera, an infrared sensor, or the like for detection for grasping an abnormal situation.

  For example, in a surveillance camera, personnel for monitoring the image are required, or in order to monitor without an operator, a complicated image recognition process and a device for the same are required. In addition, the infrared sensor has a problem of reacting to a heat source such as a small animal.

  Furthermore, in order to detect illegal intrusion into a building, it was necessary to install a number of sensors at the opening of the building such as windows and doors. Such sensors can be found by intruders and can be consciously destroyed. Furthermore, if the intruder cuts off the supply of electricity to the building, these monitoring devices may not function.

  Japanese Patent Laid-Open No. 2000-268265 proposes a monitoring device using a microphone. In other words, it is related to “information about the sound” such as a microphone for collecting surrounding sounds, information indicating the sound emitted from various objects, and the sound source and the attribute of the sound. A "sound database" to be recorded, comparison means for comparing information indicating sound from the microphone with information indicating sound recorded in the "sound database", and information based on the information from the comparison means Determination means for determining whether or not there is an abnormality in the monitoring area or the possibility thereof, and warning means for giving a predetermined warning to the user based on information from the determination means. A monitoring device using a characteristic microphone ”.

  Further, Japanese Patent Laid-Open No. 6-160172 proposes the following as an “abnormality detection device”. That is, “a sound collection microphone that collects sound waves generated in a diagnostic part of the monitoring target in the abnormality detection device that analyzes sound waves generated by the monitoring target and detects an abnormality of the monitoring target; and the sound collection microphone A waveform analysis means for performing a power spectrum analysis of the sound waves collected by the apparatus and performing a time-series change rate analysis of the analysis value, and a feature pattern of the sound obtained by the waveform analysis means when the monitoring target is normal Is stored as a reference value for each operating condition of the monitoring target, an analysis value by the waveform analysis means, and a reference value in the database means corresponding to the operating condition of the monitoring target at that time And a comparison calculation means for determining presence / absence of abnormality, an abnormality notification based on a comparison calculation by the comparison calculation means, and a change in the analysis value A monitoring means for predicting the monitored life to manage the trend of the abnormal detector ", characterized in that it comprises an.

  Furthermore, Japanese Patent Laid-Open No. 9-166483 (Patent Document 3) proposes the following as “apparatus monitoring method and apparatus”. That is, “in the method of monitoring the presence or absence of abnormality of the monitoring target device by detecting the sound from the monitoring target with the acoustic sensor, the normal sound of the same monitoring target is stored in advance from the frequency spectrum of the sound detection signal at the time of monitoring. The frequency spectrum is subtracted, and the similarity between the frequency spectrum pattern obtained by the subtraction and the frequency spectrum pattern stored in advance for the known abnormal sound and disturbance sound is calculated, and the similarity is set in advance. A device monitoring method characterized in that if it is equal to or greater than the value, it is determined as a known abnormal sound or disturbance sound, and the disturbance sound is removed or the presence or absence of the abnormal sound is determined.

  Further, JP 2000-259222 A proposes the following as a device monitoring / preventive maintenance system. That is, “a management device that arranges a multi-dimensional sensor capable of secondarily capturing the state change in the device and realizes the state change and maintenance management of the device based on information from the sensor group. In accordance with the information from the sensor, a calculation means for diagnosing deterioration and speed prediction of the equipment provided in the management equipment is used to monitor the state of the equipment and predict the state change, Disclosed by the display means of the management device, the management device has a data server unit for storing and storing these data, and editing / addition / registration of this information by the input means provided in the management device It is an equipment monitoring and preventive maintenance system.

  In addition, the invention of claim 2 is characterized by comprising a calculation function (calculation means or algorithm) for creating a reference space for determining an eigenmode of a state change of the operation status of the device. Equipment monitoring / preventive maintenance system ”.

Furthermore, the invention of claim 4 is as follows: "... the reference space of the device is based on data of the sensor group
(1): Data for creating a space (reference space) of the eigenmode of the device is collected, and the number of data is at least equal to or more than the number of sensors attached to the device, but two to three times that number.
(2): The data of the item (1) of the device is normalized by the average value and standard deviation for each element (for each sensor).
(3): Create a two-dimensional correlation matrix determined only by the number of each element from the data in (2), and further process this matrix to obtain the Mahalanobis distance of the matrix, A device monitoring / preventive maintenance system characterized in that a distribution state determined by distance (the average value is 0 and the standard deviation is about 1) is used as the reference space of the device.

JP 2000-268265 A JP-A-6-160172 Japanese Patent Laid-Open No. 9-166483 JP 2000-259222 A

  Japanese Patent Laid-Open No. 2000-268265 discloses a monitoring device using a microphone as described above. However, according to the description of the embodiment of the invention, the “sound database” 5 has various enormous volumes. A great number and types of sounds are recorded as waveform data (digital data) ”. In addition, “the waveform data indicating each sound is“ related information about the sound ”such as the name of the sound source and the attribute information of the sound (information such as when the sound occurred) It is related to and recorded.

  That is, in actual monitoring work, it is necessary to construct such a huge amount of data together with related information as a “sound database”. The work burden on this is not small. An example of using a plurality of microphones is also shown regarding sound capturing.

  Further, in the “abnormality detection apparatus” described in Japanese Patent Laid-Open No. 6-160172, “a power spectrum analysis of collected sound waves is performed in order to“ determine abnormality determination reliably even if the monitored operating condition changes ”. In addition, it is necessary to perform a time-series change rate analysis of the analysis value.

  Furthermore, in the “apparatus monitoring method and apparatus thereof” described in Japanese Patent Application Laid-Open No. 9-166483, an “acoustic detection signal at the time of monitoring” is used in order to “monitor abnormalities by separating abnormal sounds or disturbance sounds from monitoring sounds”. Subtracting the frequency spectrum stored in advance for the normal sound of the same monitoring target from the frequency spectrum of the frequency spectrum, the frequency spectrum pattern obtained by subtraction, and the frequency spectrum pattern stored in advance for known abnormal sounds and disturbance sounds It is necessary to calculate the similarity of the pattern.

  Furthermore, in the “equipment monitoring / preventive maintenance system” described in Japanese Patent Laid-Open No. 2000-259222, “to improve the accuracy of a method / diagnosis in which a device detects an abnormality or deterioration due to an external factor or an internal factor”. First, a “reference space” is created, and then “a two-dimensional correlation matrix is created and further processed to obtain the Mahalanobis distance of the matrix”.

  However, the data for creating this “reference space” is “at least more than the number of sensors attached to the device, but two to three times that number”, and the “data for each element (each sensor)”. It is necessary to “normalize by the average value and standard deviation” and “create a two-dimensional correlation matrix determined only by the number of each element”. In other words, such a method requires a lot of sensors and a lot of data processing.

  Therefore, the present invention is a monitoring device that does not require signals from a large number of microphones, for example, does not require complicated data processing, and does not require the accumulation of enormous amounts of data as in the prior art described above, and also has a database thereof. The purpose is to provide a monitoring device that can be automatically constructed.

  Furthermore, it aims at providing the database construction method which can construct | assemble such a sound database simply.

The monitoring device according to the first aspect of the present invention is the invention according to claim 1,
For the collected frequency spectrum of the sound, a distribution of the integrated intensity of the frequency spectrum for each range divided into a predetermined frequency band is defined as a phoneme, and a database for storing the phoneme;
If the correlation coefficient between each phoneme stored in the database and the phoneme of the newly collected sound is equal to or less than a predetermined value for abnormality determination, the newly collected sound is an abnormal sound. A phoneme determination unit to determine;
The monitoring device includes an action execution unit that executes a predetermined action when the phoneme determination unit determines that the newly collected sound is an abnormal sound.

The invention described in claim 2
The monitoring device according to claim 1,
In the database, the occurrence frequency of accumulated phonemes is calculated,
Further, a correlation coefficient between the accumulated phonemes is obtained,
The stored phonemes are rearranged in the descending order of the correlation coefficient with the phoneme based on the phoneme having the high occurrence frequency,
A virtual three-dimensional space of phonemes is constructed with the phoneme type as the first axis, the reciprocal of the phoneme duration as the second axis, and the reciprocal of the phoneme occurrence frequency as the third axis. This is a characteristic monitoring device.

The invention according to claim 3
The monitoring device according to claim 2,
Image processing is performed on adjacent phonemes in the plane defined by the first axis and the second axis in the three-dimensional space in the database, and a feature value of the coupling relation with respect to the first axis is given. It is the monitoring apparatus characterized by having.

The invention according to claim 4
The monitoring device according to claim 2,
Constructing a filter in a three-dimensional space of phonemes constructed only from normal phonemes stored in the database,
A monitoring apparatus comprising: a determination unit that passes phoneme data of newly collected sound through the filter and determines that the newly collected sound is an abnormal sound based on an intensity ratio before and after the filter. It is.

Furthermore, the second aspect of the database construction method is the invention according to claim 5,
Obtaining a frequency spectrum of the sound collected every predetermined time;
Obtaining an integrated intensity of the frequency spectrum for each range divided into a predetermined frequency band, defining a distribution of the integrated intensity as a phoneme, and storing it in a database;
Obtaining the phonemes for newly collected sounds; and
Obtaining a correlation coefficient between the phoneme of the newly collected sound and each phoneme stored in the database;
If the correlation coefficient is equal to or less than a predetermined value for data storage, the step of determining that the phoneme of the newly collected sound is different from the stored phoneme is included. is there.

The invention described in claim 6
In the database construction method of Claim 5,
The database construction method includes a step of storing the newly collected phonemes in the database only when it is determined that the newly collected phonemes are different from the stored phonemes.

The invention described in claim 7
In the database construction method of Claim 5,
Determining the frequency of occurrence of the accumulated phonemes;
Obtaining a correlation coefficient between the accumulated phonemes;
Rearranging each of the accumulated phonemes in descending order of the correlation coefficient with the phoneme having a high frequency of occurrence as a reference;
Constructing a three-dimensional space of phonemes, wherein the phoneme type is the first axis, the reciprocal of the phoneme duration is the second axis, and the reciprocal of the phoneme occurrence frequency is the third axis. is there.

The invention according to claim 8 provides:
In the database construction method of Claim 7,
The database construction method includes a step of using the three-dimensional space of the phonemes as a filter, passing newly collected phoneme data of the sound through the filter, and performing the determination based on an intensity ratio before and after the filter.

The invention according to claim 9 is:
In the database construction method of Claim 7,
Performing image processing on adjacent phonemes in a plane defined by a first axis and a second axis in the three-dimensional space in the database;
A database construction method including a step of assigning a feature amount of a connection relationship related to the first axis.

The invention according to claim 10 is:
In the database construction method of Claim 5,
Using phoneme data obtained within a predetermined time, defining a Mahalanobis reference space with the type of the phoneme data as a vertical matrix and the time axis as a horizontal matrix;
Calculating the distance of the maharabinos relative to the reference space of the maharabinos for the newly collected sound;
And recognizing that the state of the sound source has changed when recognizing that the Maharabinos distance exceeds a predetermined value.

The invention according to claim 11
In the database construction method of Claim 5,
Obtaining a correlation coefficient between phonemes stored in the database;
Setting a predetermined value for data reduction that is smaller than the predetermined value for data storage, and determining that phonemes having correlation coefficients exceeding the predetermined value for data reduction are the same phoneme. Is the method.

The invention according to claim 12
In the database construction method of Claim 11,
A database construction method including a step of re-accumulating phonemes determined to be the same phoneme in the determination step as one phoneme.

The invention according to claim 13
In the database construction method of Claim 5,
A database construction method including a step of defining a phoneme string as a phoneme string and storing it as a phoneme string database when a phoneme of the newly collected sound changes within a predetermined time.

The invention according to claim 14
The sound database construction method according to claim 13,
The database construction method includes a step of leaving only the phoneme string data recognized to be generated in a predetermined time among the phoneme string data in the phoneme string database.

  According to the present invention, a monitoring device can be established with a simple configuration in which, for example, a microphone is connected to a personal computer without using a monitoring camera or an infrared sensor.

  The monitoring device according to the present invention does not require a large amount of data to be stored, and therefore does not require many hardware resources.

  In addition, automatic database construction is possible. For this reason, it is not necessary to construct a basic database in advance when performing the monitoring work.

  Note that if the monitoring apparatus according to the present invention is configured using a computer having an internal power supply such as a notebook personal computer, it can cope with a place where there is no external power supply or a power failure.

Hereinafter, the present invention will be described in detail using an example of a monitoring device.
First, FIG. 1 shows a schematic configuration diagram of a monitoring apparatus according to the present invention. In FIG. 1, a microphone 4 is connected to a personal computer 2 and software necessary for realizing a monitoring function is installed. 2 and 3 are functional block diagrams for explaining the monitoring function of the personal computer 2.

  A database construction method necessary for configuring the monitoring apparatus according to the present invention will be described with reference to FIGS.

(Acquisition of waveform)
First, the target waveform is captured. The case where the target waveform is sound will be described below. The sampling method may be that the sound is collected by the microphone 4, and if it is already an electric signal, it may be input directly to the monitoring device.

  With reference to FIG. 2, first, the sound collected by the microphone 4 is preferably converted into an electric signal and further amplified by the amplifier 6 as necessary. At this time, the amplification factor is controlled by the gain adjustment unit 8 of the amplifier. The amplification factor may be appropriately determined from a reference value obtained by measuring the surrounding noise level by the noise level acquisition unit 10. In addition, it is preferable that the ambient noise level is measured at any time, and that the noise level is a moving average.

  The amplification factor can be determined as follows. First, the electric signal from the microphone 4 is converted into a digital signal by the A / D converter 12 and the standard deviation calculator 14 calculates the standard deviation of the digital signal. This standard deviation is compared with a reference value from the noise level acquisition unit 10 in the gain adjustment unit 8. When the standard deviation is small, the amplification factor is increased. When the standard deviation is large, the amplification factor is decreased. By performing such control, it is possible to prevent an error caused by an input signal being extremely large or small compared to ambient noise.

  Next, the threshold determination unit 16 determines whether or not the electrical signal from the microphone 4 is an input to be measured. This determination is made by threshold determination as to whether or not the input electrical signal is equal to or greater than a predetermined value compared to the reference value from the noise level acquisition unit 10 indicating the moving average noise level. Good.

  In order to obtain the waveform described above, a module including the amplifier 6, the gain adjuster 8, and the A / D converter 12 may be provided. This module can be configured by hardware. Specifically, a sound board usually mounted on a personal computer can be used.

  Furthermore, the threshold value determination unit described above may be configured by a module that can execute acquisition of a noise level and determination of a threshold value. This module may be composed of software that can process a signal from a sound board, for example.

(Waveform analysis: definition of phonemes)
When the threshold value determination unit 16 determines that the input is to be measured, the sound data acquisition unit 18 first acquires digital data at predetermined time intervals. Subsequently, the spectrum analysis unit 20 analyzes the frequency component. The obtained frequency components are distributed to frequency bands that have been divided and aread in advance. Further, for each frequency band, the intensity distribution is calculated by the intensity distribution calculation unit 22, and the frequency intensity distribution is acquired (see FIG. 4). Here, this frequency intensity distribution is defined as “phoneme”.

  The waveform analysis described above can be performed by a module that can execute spectrum analysis, acquisition of frequency intensity distribution, and definition of phonemes. This module may be composed of software that can process a signal from a sound board, for example.

(Database construction)
Subsequently, in order to construct the monitoring apparatus of the present invention, a sound database is first constructed. Using the method described above, the sound from the sound source is acquired and analyzed, and sequentially stored in the phoneme database.

  As shown in FIG. 3, if a new phoneme is input, the phoneme acquisition unit 24 first obtains the phoneme, and the correlation coefficient calculation unit 25 compares the similarity with the phoneme data stored in the phoneme database 26. I do.

  If the correlation coefficient between the new phoneme and any phoneme in the database 26 exceeds a predetermined value in the phoneme determination unit 28, the new phoneme and any phoneme in the database 26 are the same phoneme. It can be judged. If it is determined that the new phoneme is the same phoneme, the new phoneme is not stored in the phoneme database. The predetermined value at this time is set as a predetermined value for data storage.

  On the other hand, if this correlation coefficient is equal to or less than a predetermined value for data storage, it can be determined that the new phoneme and any phoneme in the database are different phonemes. If it is determined that the new phoneme is different from any phoneme in the database, the new phoneme is stored as new data in the phoneme database 26. It should be noted that a numerical value of 0.7 or more, which is considered to have a strong correlation, may be applied as the predetermined value for data storage. In this way, a phoneme database is constructed.

  The construction of the phoneme database described above may be executed by a module that can execute phoneme input, database search, correlation coefficient calculation, phoneme similarity determination, and new registration. It is good to be comprised with the software which can process the signal from an analysis module. The database itself may be constructed on an external storage device such as an HDD.

  Using the phoneme database constructed in this way, it can be determined whether or not the phonemes of newly collected sounds are abnormal. If the correlation coefficient between the phoneme of the new sound and any phoneme in the database 26 is equal to or less than the predetermined value for abnormality determination, the phoneme of the newly collected sound is abnormal. Can be determined.

  At this time, the predetermined value for determining the abnormality may be appropriately selected depending on how much the abnormality is different from the constructed phoneme database. As this predetermined value, for example, a numerical value less than 0.4, which is a lower limit value of a correlation coefficient that has a weak correlation, may be selected. Specifically, it may be 0.3, 0.2 or 0.1. In the case where it is determined that even a slight difference is as described above, the predetermined value for determining abnormality may be 0.4 or more.

  Furthermore, when it is determined that the phoneme of the newly collected sound is abnormal, the monitoring device according to the present invention includes an action execution unit that executes a predetermined action (see FIG. 5). .

Specific actions include
-Display an alarm message on the display-Sound an alarm sound from a speaker-Send an e-mail notifying the occurrence of an abnormality to a predetermined e-mail address.

(Duration)
In general, sounds have various lengths of time, but the duration of a sound is one of the elements of phonemes. The duration is also measured by the duration measuring unit 30 as one of the feature quantities and stored in the phoneme database 26.

  Such processing related to the duration time may be performed by a module capable of executing the measurement of the duration time and the accumulation of the feature amount. This module may be configured by software capable of processing a signal from the waveform analysis module, for example.

(Frequency of occurrence)
In general, each sound is generated at a certain frequency. This frequency of occurrence is also one of the elements of phonemes. This occurrence frequency is also preferably measured by the occurrence frequency measurement unit 32 as one of the feature quantities and stored in the phoneme database 26.

  Such processing relating to the occurrence frequency may be performed by a module capable of executing occurrence frequency measurement and feature amount accumulation. This module may be configured by software capable of processing a signal from the waveform analysis module, for example.

(Organization of database)
Now, when a predetermined time has passed, it is preferable to check the similarity between the accumulated phonemes again and organize the database. The database organizing unit 34 obtains the correlation coefficient between the accumulated phonemes, and if it exceeds a predetermined value for data organizing, it is re-recognized as the same phoneme.

  If they are recognized as the same phoneme, one of them may be used as a representative phoneme and the other may be deleted. In this way, phonemes determined to be the same phoneme can be stored again as one phoneme.

  It is necessary to use a smaller value than the predetermined value for data storage as the predetermined value for data arrangement used here. Generally, it is assumed that there is a weak correlation when the correlation coefficient is in the range of 0.4 to 0.7. In this database organization, it is preferable to use a numerical value of 0.4 or more and less than 0.7 as a predetermined value for data organization.

  In the database construction method described above, when a new sound is input, a correlation with data already accumulated is obtained, and only phonemes of sounds determined as different sounds are accumulated. For this reason, it is not necessary to store a huge amount of data in the database. That is, many hardware resources are not required for data storage. Furthermore, if the database is organized, more hardware resources are not required. Such a method is also considered as a kind of data compression.

  The database organization described above may be performed by a module that can execute calculation of cross-correlation coefficients, re-recognition of phonemes, and integration of data made into the same phonemes. This module can be realized by software.

(Phoneme sequence)
Sounds that are continuously input may not be the same and often change in the middle. In this case, naturally, the phoneme also changes. The phoneme changing here is newly defined as a phoneme string.

  For this phoneme string, the phoneme string database 36 may be constructed in the same manner as the phoneme. The phoneme string acquisition unit 38 acquires the phoneme string, and the correlation coefficient calculation unit 39 compares the similarity with the phoneme string data stored in the phoneme database 36.

  If the correlation coefficient between the new phoneme sequence and any phoneme sequence in the database 36 is equal to or greater than a predetermined value, the phoneme sequence determination unit 40 can determine that the phoneme sequence is the same. If this new phoneme string is determined to be the same phoneme string as any one of the phoneme strings in the database 36, the new phoneme string is not stored in the phoneme string database 36.

  If the correlation coefficient is less than a predetermined value, it can be determined that the new phoneme string and any phoneme string in the database 36 are different phoneme strings. If it is determined that the new phoneme string is different from any of the phoneme strings in the database 36, the new phoneme string is stored as new data in the phoneme string database 36. In addition, as this predetermined value, it is good to apply the numerical value of 0.7 or more considered that there exists a strong correlation.

  Furthermore, it is preferable to organize the phoneme string database by a method similar to the above-described database arrangement, and to create a database of only phoneme strings having a predetermined frequency of occurrence.

  The phoneme string database only needs to be executed by a module that can execute acquisition of a phoneme string, database search, correlation coefficient calculation, similarity determination of phoneme strings, and new registration, such as a waveform analysis module. It may be configured by software that can process signals from The database itself may be constructed on an external storage device such as an HDD.

  Such an apparatus can be easily realized by a personal computer to which a microphone is connected and software that can execute the above-described database construction method is installed.

  By continuously performing the processing as described above, it is possible to construct a sound database necessary for recognizing sound generated in a region to be monitored, for example.

  As described above, according to the database construction method of the present invention, the construction of the sound database can be automatically performed only by placing a device having such a function in an environment to be monitored, for example.

  As described above, according to the present invention, there is a great feature that it is not necessary to place a heavy burden on the operator when constructing a sound database.

(Phoneme space)
Whether the newly input sound is normal or abnormal can be determined using the phoneme space. Here, the phoneme space is defined as a space formed by three dimensions of phoneme type, reverse duration, and reverse occurrence frequency, as shown in FIG.

  Here, with respect to the feature quantity of duration, the masking effect of sound can be taken into consideration by taking the reciprocal of duration. Also, regarding the feature quantity of occurrence frequency, taking the reciprocal thereof also takes into account the sound masking effect. In general, it is known that when a sound is heard continuously or when it is heard frequently, human sensitivity to the sound gradually decreases. This is a phenomenon called a masking effect.

  Note that the reverse duration time is not handled as a continuous amount, but a predetermined segment may be provided and assigned to the segment.

  FIG. 7 is a functional block diagram for explaining the creation of the phoneme space. Such a phoneme space can be formed by a module capable of executing phoneme axis construction, segmentation of reverse duration time, and calculation of reverse occurrence frequency. This module can be realized, for example, by processing data using data stored in a phoneme database. Hereinafter, a description will be given with reference to FIG.

  First, the correlation coefficient of the phoneme constituting the phoneme database 26 is calculated by the correlation coefficient calculation unit 50, then the reciprocal of the occurrence frequency is calculated by the occurrence frequency reciprocalization unit 52, and then the duration of the phoneme is calculated. The reciprocal is calculated by the duration reciprocalization unit 54.

  Further, the phoneme is rearranged in the order of the correlation coefficient with the sorting unit 56 using the phoneme having the highest occurrence frequency as a reference.

  On the other hand, as described above in the segmentation unit 58, the calculated reverse duration time is assigned to a predetermined segment to make a region.

  The sorted phonemes, reverse duration, and reverse occurrence frequency are three-dimensionalized by the three-dimensionalization unit 60 to create a phoneme space.

(Sound recognition: Image processing of phoneme space)
For example, in the technology related to speech recognition, since vowels and consonants are processed separately as phonemes, it is necessary to perform very complicated pattern matching processing. However, one of the features of the present invention is that the sorting is performed by the correlation coefficient in the above phoneme space. That is, in the phoneme data constructing the phoneme database, the correlation coefficient is obtained, and the phonemes are rearranged in descending order of the correlation coefficient with the most frequently occurring phoneme as a reference.

  That is, the phoneme database can be handled as one piece of image information by performing such processing. In this case, the reverse occurrence frequency may be set as the image intensity.

  In addition, the three-dimensionalization unit 60 can apply coupling information between phonemes by performing a smoothing process on the surface formed by the sorted phoneme type axis and reverse duration axis.

  FIG. 8 shows a two-dimensional image that has been subjected to a smoothing process. This shows a state in which the reverse occurrence frequency is represented as the image intensity on the image plane defined by the sorted phoneme type axis and the reverse duration axis. Here, as the smoothing process, the density of the entire image is averaged. This can be compared to the coordination between events in the memory of the brain that help fix the memory.

  Furthermore, in addition to the smoothing process, it is also possible to give coupling information between the phonemes by a process of differentiation or image enhancement.

  Such image processing of the phoneme space may be configured by a module that can perform sorting based on the correlation coefficient in the phoneme space and smoothing processing on the plane formed by the phoneme type axis and the inverse duration time axis. This module may be composed of software that can process data stored in a phoneme database, for example.

  By continuously performing such processing, it is possible to accumulate sound knowledge in a phoneme database. Here, for example, only normal sounds are input to the monitoring device to obtain a phoneme space. This phoneme space can be defined as a normal phoneme space. This normal phoneme space can be a filter used when the judgment filter creation unit 62 judges whether the newly input sound is a normal sound or an abnormal sound.

(Filtering process)
The filtering process using this filter can be performed in the filtering processing unit 64 as follows. First, the intensity of the image in the phoneme space is read as light transmittance. Next, the new sound is mapped in the phoneme space with the phoneme type and reverse duration value obtained from the new sound. Subsequently, the intensity ratio before and after filtering by the normal phoneme space can be calculated by multiplying the transmittance at the mapped position. When this intensity ratio is larger than a predetermined value, that is, when a new sound is transmitted well, it can be determined that the new sound is different from the phoneme accumulated in the normal phoneme space.

  As described above, the present invention does not require complicated pattern matching processing, and can determine whether the input sound is normal or abnormal by simple processing.

  Further, this filtering process can be realized not only by software but also by hardware. Specifically, as shown in FIG. 9, a two-dimensional light emitting diode array 70 for displaying a newly input sound, a liquid crystal panel 72 as a filter, and a two-dimensional photodetector array for detecting transmitted light. 74 and the two-dimensional microlens arrays 76 and 78 can be used. By defining the phoneme space in this way, filtering processing can also be performed by hardware.

  The phonemes in the newly input sound are subjected to the above-described image processing and converted into electrical signals, and are displayed as image information by the light-emitting diode array 70 arranged two-dimensionally. The displayed phonemes are duplicated into a plurality of images by the microlens array 76 constituting the one-to-many optical system, and irradiated onto the liquid crystal panel 72.

  At this time, the liquid crystal panel 72 functions as a filter. That is, on the liquid crystal panel, filter information including normal phoneme spaces in a plurality of monitoring areas is displayed by changing the transmittance pattern. The light that has passed through the filter passes through the microlens array 78 and irradiates the photodetector array 74 corresponding to each filter. The light irradiated to the photodetector array is converted into an electrical signal.

  From the signal intensity ratio before and after filtering thus obtained, it is possible to instantaneously determine whether the newly input sound is normal or abnormal. For example, when the signal intensity ratio is larger than −3 dB, it can be determined that the newly input sound is an abnormal sound. For example, “when larger than −3 dB” means that the light of the image by the newly input sound is well transmitted through a filter constituted by a normal phoneme space.

  Furthermore, if the above-described filter processing unit is configured by hardware, a plurality of processing units can be constructed on the same plane. According to such a configuration, a plurality of determinations can be processed simultaneously and instantaneously.

(Change in situation)
In the area to be monitored, the situation changes with time. In other words, for example, in an area where people go in and out, the situation changes during the day and at night, and the criteria for judging whether it is normal or abnormal will also change. Here, if a determination is made based on a certain standard value, a correct determination cannot be made. Naturally, there will be differences between weekdays and holidays, and seasonal changes in the situation. Therefore, it is advisable to construct some normal phoneme spaces based on clock information and calendar information.

(Maharabinos space and distance)
Furthermore, the present invention is characterized by introducing the concept of a maharabinos space and a maharabinos distance in order to make an accurate determination according to a change in the situation over time. The maharabinos space is a space in which information recognized as normal is formed into a matrix and the non-dispersed correlation information is extracted. In this matrix, for example, a feature item may be used as a vertical matrix component, and a time axis or a space axis may be used as a horizontal matrix component.

  As described above, the maharabinos space can be constructed by forming a matrix from the feature items and information on the time axis and the space axis.

  The Mahalanobis distance refers to the distance from the normal center position in the Mahalanobis space to the coordinates where the information to be inspected is mapped. When this distance becomes larger than the predetermined value, it can be determined that the distance is away from the normal state. Usually, a numerical value of 4 or more is applied to this predetermined value.

  The construction of a normal space in which the concept of such maharabinos space and maharabinos distance is introduced will be described with reference to the functional block diagram shown in FIG.

(Judgment of situation change)
In the present invention, phonemes sorted into vertical matrix components are assigned time axes to horizontal matrix components. First, the maharabinos reference sequence generation unit 80 generates a maharabinos reference sequence by using phoneme data collected from a current predetermined time. Next, the maharabinos distance calculation unit 82 calculates the maharabinos distance between this reference sequence and a new phoneme. The situation conversion determination unit 84 determines that the situation has changed if the obtained Mahalanobis distance is greater than or equal to a predetermined value and continues for a predetermined time or longer.

  Thus, the determination of the situation change may be performed by a module that can execute the creation of the Maharabinos reference sequence and the calculation / determination of the Maharabinos distance. This module can be realized, for example, by using data stored in the phoneme database 26 and processing with software.

  In this way, it is preferable to construct a phoneme database while automatically recognizing a change in the situation, and providing divisions such as daytime, midnight, weekdays, and holidays. That is, the normal phoneme space in each situation is constructed.

(Special phoneme learning)
At this time, although the Mahalanobis distance is far from normal phoneme space, such as a phone ringtone or a chime sound that tells you to visit, a special phoneme is used to prevent normal sounds from being considered abnormal. It is good to register as. In this way, if the phoneme is registered as a special phoneme, it will be possible to recognize the next incoming call tone or chime tone.

  Furthermore, if it is registered in this monitoring device by meaning, for example, a word that instructs something by an operator's voice, voice control can be realized using this monitoring device.

(Basic configuration example)
Hereinafter, a basic configuration example in which the present invention is realized on a personal computer will be described.

  FIG. 11 shows a basic hardware configuration in the personal computer 2. A signal from the microphone 4 is input to the sound board 100 installed in the personal computer 2. The input sound data is sampled by a multimedia API attached to Windows (registered trademark) and stored in the memory 102 as digital data. Also, gain control can be realized by software by controlling the mixer. One microphone was used.

  By executing a multimedia API command by a CPU (Central Processing Unit) 104, sound data can be acquired from a microphone for a predetermined time. When fast Fourier transform (FFT) is performed on the acquired data, spectrum analysis can be performed. Further, instead of the fast Fourier transform, Maximum Entropy Method (MEM) may be applied.

  As an example of sampling conditions, the number of data is 1024, the sampling rate is 22.05 kHz, 8 bits, and monaural. At this time, it takes 46.4 mS to acquire one sample of data.

  The phoneme database and the phoneme string database may be composed of, for example, a C language structure. Modules related to database construction were coded in C ++. The timer function is realized by an interrupt timer, and an execution procedure is preferably created in a function called there.

  The phoneme space can be created in a linear memory space by first securing a global memory. Here, it is preferable to secure an amount of space necessary for situation determination.

  Measurements over a certain period of time may be controlled by using an interrupt timer or by creating a separate thread.

  The maharabinos space is calculated by the inverse matrix operation using the sweep-out method. If inverse matrix operation is not possible, use Schmitt expansion.

  Therefore, first, the sound of the environment to be monitored is acquired and analyzed by the above-described method, stored in the phoneme database, and the sound database is constructed. Furthermore, the phoneme database is constructed by measuring the duration and organizing the database. The phoneme string database is also constructed as described above.

  In order to determine whether the newly input sound is normal or abnormal, first, a phoneme space is defined, and then it is determined from the signal intensity ratio before and after filtering whether the newly input sound is normal or abnormal.

  When it is determined that the newly input sound is abnormal, a predetermined action is executed. This action is, for example, issuing an alarm or sending an e-mail from the speaker 106 or the like. When such an action is performed, a module necessary for the control is added. For example, when sending an e-mail, a control module for connecting to the POP server may be added.

  In FIG. 11, only the speaker 106 is shown as hardware for executing the action. In the figure, reference numeral 108 denotes a display.

  In this way, according to the present invention, a monitoring device that can grasp the state of the environment with only a signal from one microphone 4 can be configured on the personal computer 2. In the monitoring apparatus according to the present invention, it is preferable to use one microphone for a simple configuration.

(Specific example 1)
As described in the basic configuration example, the monitoring apparatus of the present invention can be configured on a personal computer. Therefore, in this specific example 1, the monitoring device of the present invention is configured by a notebook personal computer. FIG. 12 shows a monitoring device configured by connecting a microphone 4 to a notebook personal computer 112.

  Under a normal environment, a program for controlling the monitoring device is executed to measure a noise level first and store phoneme data and phoneme string data. When data is accumulated to some extent, a Maharabinos reference matrix is created, and the maharabinos distance is calculated for phoneme data that is input as needed, and the data is classified while grasping changes in the situation over time. . Then create a phoneme space. In this case, all accumulated data is recorded as sound in a normal situation.

  When a sound database is constructed in this way, it is possible to determine whether the newly input sound is normal or abnormal. Further, the operator may be requested to answer whether the judgment is correct or incorrect through the user interface, so that the operator can learn more accurately. If it is determined as abnormal, some predetermined action is executed.

  As described above, when the monitoring device is configured on the laptop personal computer 112 and installed in the room, it does not stand out like a normal monitoring device, and the indoor atmosphere is not destroyed.

  In this specific example 1, the monitoring device is configured on a notebook personal computer. For this reason, it can be used in a place without an external power supply. In addition, when installed in a place with an external power supply, even if the power supply is cut off due to a power failure, etc., it can operate for at least several hours with the built-in battery, detect an abnormality, and take action such as notification be able to.

(Specific example 2)
Although the control in the monitoring apparatus described above is configured by software, it is not always necessary. For example, it can be realized by a system using an FPGA (Field Programmable Gate Array), an ASIC (Application Specific IC), or a DSP (Digital Signal Processor). That is, it can be incorporated into pet robots, home appliances and mobile phones.

  As described above, when the monitoring device is built in and configured in the pet robot or the home appliance, the indoor atmosphere is not broken and can be preferably used. If it is built into a mobile phone, actions such as reporting can be performed even if an unexpected situation occurs.

(Application example 1)
Next, the case where the monitoring apparatus according to the present invention is applied to a factory will be described. For example, the operating sound of a continuously operating apparatus can be monitored by the monitoring apparatus according to the present invention, an alarm can be issued when a singular sound is detected, and an action for preventing the expansion of trouble can be performed.

  The present invention is not limited to sound, and any waveform data that can be digitally sampled can be monitored. Further, in order to perform failure analysis, a monitoring device that learns a transient response waveform can be used.

(Application example 2)
Furthermore, the case where the monitoring apparatus according to the present invention is applied to home monitoring will be described. As described in the first specific example, the monitoring apparatus of the present invention is configured on a notebook personal computer.

  As described above, when the monitoring device according to the present invention is operated, it is determined whether the newly input sound is normal or abnormal while grasping the situation such as whether there is a person in the room or being out of the office. can do. In addition, if it is possible to recognize a ringtone of a telephone call or a chime sound that tells a visit, it is possible to leave a history of incoming telephone calls or presence / absence of visitors.

  Furthermore, by adding a function that can respond to inquiries by e-mail from the outside, it is possible to grasp the situation of the home away from home.

  In addition, since the monitoring device according to the present invention can grasp the situation of the monitoring target region, for example, when an abnormality occurs to a sick person, a baby or a person who needs care, etc. You can also take actions such as performing.

(Example of phoneme data)
In Application Example 2, an indoor situation is measured for half a day, and phonemes sorted from the obtained data are displayed on a personal computer screen, and an example is shown in FIG.

  This phoneme pattern is a two-dimensional image corresponding to FIG. 8, and the reverse occurrence frequency is represented as image intensity on the image plane represented by the sorted phoneme type axis and reverse duration axis.

  An example of a normal phoneme space is shown in FIG. FIG. 14 shows only a normal phoneme space in FIG.

It is a figure explaining the example which implement | achieved the monitoring apparatus by this invention on the personal computer. It is a functional block diagram explaining the function of the monitoring apparatus shown in FIG. It is a functional block diagram explaining the function of the monitoring apparatus shown in FIG. It is a figure explaining the result of having calculated | required frequency intensity distribution from the input waveform. It is a functional block diagram explaining the function of the monitoring apparatus shown in FIG. It is a figure which illustrates phoneme space notionally. It is a figure explaining the example which sorted the phoneme in phoneme space. It is a figure explaining the phoneme space to which the smoothing process was performed. It is a figure which illustrates the mode of filtering processing typically. It is a block diagram explaining the structure of the normal phoneme space which introduce | transduced the concept of the maharabinos space and maharabinos distance. It is a figure which shows the hardware constitutions in a personal computer. It is a figure which shows the example which implement | achieved the monitoring apparatus of this invention with the notebook type personal computer. It is a figure which shows the example which displayed the sorted phoneme on the personal computer screen. It is the figure which displayed normal phoneme space on the personal computer screen.

Explanation of symbols

DESCRIPTION OF SYMBOLS 2 Personal computer 4 Microphone 6 Amplifier 8 Gain adjustment part 10 Noise level acquisition part 12 A / D converter 14 Standard deviation calculation part 16 Threshold value calculation part 18 Sound data acquisition part 20 Spectrum analysis part 22 Intensity distribution calculation part 24 Phoneme acquisition part 26 phoneme database 28 phoneme determination unit 29 action execution unit 30 duration measurement unit 32 occurrence frequency measurement unit 34 database organization unit 36 phoneme sequence database 38 phoneme sequence acquisition unit 39 correlation calculation unit 40 phoneme sequence determination unit 42 database organization unit 50 calculation Unit 52 occurrence frequency reciprocal unit 54 duration reciprocal unit 58 segmentation unit 60 three-dimensional unit 62 decision filter creation unit 64 filtering processing unit 70 two-dimensional light-emitting diode array 72 two-dimensional liquid crystal panel 74 two-dimensional photo detector array Lee, 78 two-dimensional microlens array 80 Maharabinosu reference sequence generator 82 Maharabinosu distance calculator 84 situation determination unit 100 soundboard 102 Memory 104 CPU
106 speaker 108 display 112 notebook personal computer

Claims (14)

For the collected frequency spectrum of the sound, a distribution of the integrated intensity of the frequency spectrum for each range divided into a predetermined frequency band is defined as a phoneme, and a database for storing the phoneme;
If the correlation coefficient between each phoneme stored in the database and the phoneme of the newly collected sound is equal to or less than a predetermined value for abnormality determination, the newly collected sound is an abnormal sound. A phoneme determination unit to determine;
A monitoring apparatus comprising: an action execution unit that executes a predetermined action when the phoneme determination unit determines that the newly collected sound is an abnormal sound. The monitoring device according to claim 1,
In the database, the occurrence frequency of accumulated phonemes is calculated,
Furthermore, a correlation coefficient between the accumulated phonemes is obtained,
The accumulated phonemes are rearranged in the descending order of the correlation coefficient with the phoneme based on the phoneme having the high occurrence frequency,
A phoneme virtual three-dimensional space is constructed with the phoneme type as the first axis, the reciprocal of the phoneme duration as the second axis, and the reciprocal of the phoneme occurrence frequency as the third axis. A monitoring device characterized. The monitoring device according to claim 2,
Image processing is performed on adjacent phonemes in the plane defined by the first axis and the second axis in the three-dimensional space in the database, and a feature value of the coupling relation with respect to the first axis is given. A monitoring device. The monitoring device according to claim 2,
Constructing a filter in a three-dimensional space of phonemes constructed only from normal phonemes stored in the database,
A monitoring apparatus comprising: a determination unit that passes phoneme data of newly collected sound through the filter and determines that the newly collected sound is an abnormal sound based on an intensity ratio before and after the filter. . Obtaining a frequency spectrum of the sound collected every predetermined time;
Obtaining an integrated intensity of the frequency spectrum for each range divided into a predetermined frequency band, defining a distribution of the integrated intensity as a phoneme, and storing it in a database;
Obtaining the phonemes for newly collected sounds; and
Obtaining a correlation coefficient between the phoneme of the newly collected sound and each phoneme stored in the database;
And a step of determining that the newly collected phoneme is different from the stored phoneme if the correlation coefficient is equal to or less than a predetermined value for data storage. In the database construction method of Claim 5,
A database construction method including a step of storing the newly collected phonemes in the database only when it is determined that the newly collected phonemes are different from the stored phonemes. In the database construction method of Claim 5,
Determining the frequency of occurrence of the accumulated phonemes;
Obtaining a correlation coefficient between the accumulated phonemes;
Rearranging each of the accumulated phonemes in descending order of the correlation coefficient with the phoneme having a high frequency of occurrence as a reference;
Constructing a three-dimensional phoneme space with the phoneme type as the first axis, the reciprocal of the phoneme duration as the second axis, and the reciprocal of the phoneme occurrence frequency as the third axis. In the database construction method of Claim 7,
A database construction method including the step of using the three-dimensional space of the phonemes as a filter, passing the phoneme data of newly collected sound through the filter, and making the determination based on an intensity ratio before and after the filter. In the database construction method of Claim 7,
Performing image processing on adjacent phonemes in a plane defined by a first axis and a second axis in the three-dimensional space in the database;
Providing a feature amount of a connection relationship with respect to the first axis. In the database construction method of Claim 5,
Using phoneme data obtained within a predetermined time, defining a Mahalanobis reference space with the type of the phoneme data as a vertical matrix and the time axis as a horizontal matrix;
Calculating the distance of the maharabinos relative to the reference space of the maharabinos for the newly collected sound;
And recognizing that the state of the sound source has changed when recognizing that the maharabinos distance exceeds a predetermined value. In the database construction method of Claim 5,
Obtaining a correlation coefficient between phonemes stored in the database;
Setting a predetermined value for data reduction that is smaller than the predetermined value for data storage, and determining that phonemes having correlation coefficients exceeding the predetermined value for data reduction are the same phoneme. Method. In the database construction method of Claim 11,
A database construction method including a step of re-accumulating phonemes determined to be the same phoneme in the determination step as one phoneme. In the database construction method of Claim 5,
A database construction method including a step of defining a group of phonemes to be changed as a phoneme string and storing it as a phoneme string database when a phoneme of the newly collected sound changes within a predetermined time. The sound database construction method according to claim 13,
A database construction method including a step of leaving only the phoneme string data recognized to be generated in a predetermined time among the phoneme string data in the phoneme string database.