JP6467915B2 - Feature sound extraction method, feature sound extraction device, computer program, distribution system - Google Patents

Description

  The present invention relates to a technique for extracting a characteristic part from a living sound.

  In recent years, a service for monitoring elderly people by using living sound sensing / presentation and a communication service have been studied. As an elemental technology common to these services, there is a life sound summarization technology for understanding the situation without listening to all sounds. The life sound summarization technique is a technique for extracting and presenting characteristic portions such as door opening / closing sounds and human laughter from collected life sounds.

  Conventionally, as a technique for extracting a characteristic portion of a signal, there is a method of acquiring abnormal data and detecting anomalies based on a deviation from the steady state (for example, Patent Document 1).

JP 2004-295861 A

  However, the living sound has a property of including a plurality of characteristic portions having different characteristic degrees, such as a sound of a fan of an air conditioner, a sound of water, and a sound of coughing. Therefore, in the method of judging the degree of deviation from the steady state with a threshold and determining whether or not to present, there is a problem that all the characteristic parts are presented or parts where the characteristic degree is weak are not presented at all. .

  In one aspect, there is provided a feature sound extraction method or the like that identifies the most characteristic part and extracts the sound related to the specific part even when a plurality of characteristic parts may exist in the living sound. Objective.

In one aspect, the feature sound extraction method executed by a computer divides sound data at predetermined time intervals, calculates a feature amount including a frequency component of sound data for each divided period, and calculates the sound data calculated for each period. For one or more component values x of the feature quantity including the frequency component of the data, the function differentiated or sub-differentiated by x in the section x ≧ a _b (a _b ≧ 0) is monotonically decreased. The lower bound T of the function value to satisfy satisfies the function f (x) existing in the section 0 ≦ x ≦ _ab , and based on the result of the function, the feature value for each period is obtained, and the obtained feature quantity for each period is obtained. Based on this, characteristic sound data is extracted.

  According to one aspect of the method, it is possible to extract the most characteristic part from the living sound.

It is a block diagram which shows the hardware constitutions of a characteristic sound extraction apparatus. It is a block diagram which shows the function structural example of a characteristic sound extraction apparatus. It is explanatory drawing which shows an example of the record layout of life sound DB. It is explanatory drawing which shows an example of the record layout of sound feature DB. It is explanatory drawing which shows an example of the record layout of sound cluster DB. It is explanatory drawing which shows an example of the record layout of digest display DB. It is explanatory drawing which shows an example of the record layout of characteristic sound DB. It is a flowchart which shows an example of the procedure of a main process. It is a flowchart which shows an example of the procedure of a feature score calculation process. It is a block diagram which shows an example of a feature calculation process. It is a flowchart which shows an example of the procedure of a feature calculation process. It is a graph which shows an example of the input-output relationship of a filter. It is a graph which shows an example of a feature-value. It is a flowchart which shows an example of the procedure of an output suppression process. It is a flowchart which shows an example of the procedure of a presentation process. It is explanatory drawing which shows the specific example of an output suppression process and a presentation process. It is a flowchart which shows an example of the procedure of a presence / absence determination process. It is explanatory drawing which shows the specific example of a presence / absence determination process. It is explanatory drawing which shows an example of a digest display. It is a flowchart which shows an example of the procedure of an occurrence frequency calculation process. It is a flowchart which shows an example of the procedure of an output suppression process. It is explanatory drawing which shows the specific example of an output suppression process. It is explanatory drawing which shows the other example of an output suppression process. It is a graph which shows an example of the input / output relationship of a filter. It is explanatory drawing which shows an example of a structure of a delivery system.

Embodiment 1
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 shows a feature sound extraction apparatus 1
It is a block diagram which shows the hardware constitutions. The feature sound extraction apparatus 1 is a general-purpose computer, a workstation, a desktop PC (personal computer), a notebook PC, or the like. The feature sound extraction device 1 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, a mass storage device 14, an input unit 15, an output unit 16, and a communication unit 17 (transmission). Part) and a reading part 18. Each component is connected by a bus.

  The CPU 11 controls each part of the hardware according to the control program 1P stored in the ROM 13. The RAM 12 is, for example, SRAM (Static RAM), DRAM (Dynamic RAM), or flash memory. The RAM 12 temporarily stores data generated when the CPU 11 executes the program.

  The mass storage device 14 is, for example, a hard disk, an SSD (Solid State Drive), or the like. The large-capacity storage device 14 stores various databases described later. Further, the control program 1P may be stored in the mass storage device 14.

The input unit 15 includes a keyboard, a mouse, and the like for inputting data to the feature sound extraction apparatus 1. Further, for example, a microphone 15 a is connected to collect life sounds, and the life sounds collected from the microphone 15 a are converted into electric signals and input to the input unit 15.
In this specification, “sound” is not limited to “sound” in a narrow sense in which vibration in the air is acquired by a microphone. For example, “vibration” that propagates in air, in a substance, or in a liquid is, for example, a microphone. This is a broad concept including the case of measuring with a measuring device such as a piezo element or a laser micro displacement meter.

  The output unit 16 is for outputting the image of the characteristic sound extraction device 1 to the display device 16a and outputting the sound to a speaker or the like.

  The communication unit 17 communicates with other computers via a network. The reading unit 18 reads a portable storage medium 1a including a CD (Compact Disk) -ROM and a DVD (Digital Versatile Disc) -ROM. The CPU 11 may read the control program 1P from the portable storage medium 1a via the reading unit 18 and store it in the large-capacity storage device 14. Alternatively, the CPU 11 may download the control program 1P from another computer via a network and store it in the mass storage device 14. Furthermore, the CPU 11 may read the control program 1P from the semiconductor memory 1b.

  FIG. 2 is a block diagram illustrating a functional configuration example of the feature sound extraction apparatus 1. Each functional unit of the feature sound extraction device 1 is realized by the control program 1P stored in the mass storage device 14 and hardware resources such as the CPU 11 and the RAM 12 operating in cooperation.

  The feature sound extraction apparatus 1 includes an input unit 110, a feature score calculation unit 120, a clustering unit 130, a digest display generation unit 140, an output suppression unit 150, and a presentation unit 160, a living sound DB 14a, a sound feature DB 14b, a sound cluster DB 14c, It includes a digest display DB 14d and a feature sound DB 14e.

  The input unit 110 includes a living sound input unit 111. The input unit 110 is a functional unit corresponding to the input unit 15 that is hardware. The feature score calculation unit 120 includes a sound feature calculation unit 121 (calculation unit, filter unit) and a sound cluster matching / score calculation unit 122 (feature amount calculation unit). The clustering unit 130 includes a clustering processing unit 131 and a cluster occurrence frequency calculation unit 132. The digest display generation unit 140 includes a presence / absence determination unit 141 (counting unit, determination unit). The output suppression unit 150 includes a feature location output suppression unit 151 (extraction unit). The presentation unit 160 includes a GUI display unit 161 and a voice presentation unit 162.

  The living sound DB 14a, the sound feature DB 14b, the sound cluster DB 14c, the digest display DB 14d, and the feature sound DB 14e are stored in the mass storage device 14.

  FIG. 3 is an explanatory diagram showing an example of a record layout of the living sound DB 14a. The living sound DB 14a includes columns of time stamps and sound file names. The time stamp stores the time when the life sound was acquired. The time as the time stamp is the beginning time or the end time of the life sound stored as the sound file. In the sound file name, the file name is stored.

  FIG. 4 is an explanatory diagram showing an example of the record layout of the sound feature DB 14b. The sound feature DB 14b includes columns of time stamps and feature amounts. The time stamp stores the time stamp of the sound data. The feature value stores the value of the feature value of the sound data.

  FIG. 5 is an explanatory diagram showing an example of a record layout of the sound cluster DB 14c. The sound cluster DB 14c includes columns of cluster ID, feature amount, and occurrence frequency. The cluster ID stores an ID that identifies each cluster. The feature value stores a feature value of each cluster, that is, a representative value of the cluster such as a center coordinate of each cluster or a median value of data included in the cluster. The occurrence frequency stores the occurrence frequency of each cluster.

  FIG. 6 is an explanatory diagram showing an example of a record layout of the digest display DB 14d. The digest display DB 14d includes columns of start time, end time, and digest ID. As the start time and end time, the start and end times of a time zone indicating the presence or absence of a person are stored. The digest ID stores an ID indicating the presence or absence of a person. For example, ID = 1 indicates presence and ID = 0 indicates absence.

  FIG. 7 is an explanatory diagram showing an example of a record layout of the feature sound DB 14e. The feature sound DB 14e includes columns of time stamp, score, and cluster ID. The time stamp stores the date and time when the sound data was observed. The score stores the frequency of sound data generation. The cluster ID stores the ID of the cluster to which the characteristic sound belongs.

  Next, an outline of the operation of the feature sound extraction device 1 will be described. The living sound input unit 111 stores the sound collected from the microphone 15a as data (sound data) in the living sound DB 14a. The format of the audio data stored in the living sound DB 14a may be an uncompressed format such as WAV (RIFF waveform Audio Format) or AIFF (Audio Interchange File Format), MP3 (MPEG-1 Audio Layer-3), WMA (WindowMs). (Registered trademark) Audio) may be used. Further, the living sound input unit 111 passes the sound data to the sound feature calculation unit 121.

  The sound feature calculation unit 121 divides the audio data by a time window and calculates a feature amount for each divided time. The calculated feature amount is stored in the sound feature DB 14b. The clustering processing unit 131 clusters the feature values stored in the sound feature DB 14b at a timing such as every time the sound feature DB 14b is updated. The cluster occurrence frequency calculation unit 132 calculates the occurrence frequency of each cluster and stores it in the sound cluster DB 14c. The sound feature calculation unit 121 passes the calculated feature amount to the sound cluster matching score calculation unit 122.

  The sound cluster matching score calculation unit 122 performs matching between the feature amount received from the sound feature calculation unit 121 and the feature amount of each cluster stored in the sound cluster DB 14c, and the sound to be processed belongs. Determine the cluster to be used. The generation frequency of the cluster to which the user belongs should be, for example, the generation frequency of the divided audio data, or a weighted sum of the generation frequencies of the clusters existing in the vicinity of the feature amount may be used. As a result of the above processing, the sound cluster matching score calculation unit 122 stores, in the feature sound DB 14e, the ID of the cluster to which the sound belongs, the frequency of occurrence, and the time stamp of the separated sound for each separated sound data.

  The feature location output suppression unit 151 extracts data for a predetermined time, for example, 30 minutes from the data stored in the feature sound DB 14e, specifies the sound data with the lowest occurrence frequency, and outputs the time stamp of the specified sound data To do.

  The presence / absence determination unit 141 extracts data for a certain period of time from the data stored in the temporary storage area, and uses it as input data. Also, the presence / absence determination unit 141 creates a background sound table (not shown) from the sound cluster DB 14c. The non-background sound table is obtained by adding a background sound or a non-background sound type to each sound cluster. The distinction between background sounds and non-background sounds is determined based on the frequency of occurrence. The presence / absence determination unit 141 obtains the number of appearances of the non-background sound included in the input data, determines that there is a person if the number of appearances exceeds a predetermined threshold, and if the number of appearances is equal to or less than the threshold. It is determined that there were no people. The presence / absence determination unit 141 stores the determination result in the digest display DB 14d.

  The voice presentation unit 162 presents only the sound data score output by the feature location output suppression unit 151 that exceeds a predetermined threshold. The score here may be, for example, a score that is inversely proportional to the occurrence frequency.

  The GUI display unit 161 causes the display device 16a to display a display screen indicating the presence / absence determination result based on the digest display DB 14d.

  Next, the operation of the feature sound extraction device 1 will be described in detail. FIG. 8 is a flowchart showing an example of the procedure of the main process. The CPU 11 of the feature sound extraction device 1 performs feature score calculation processing (step S1). Subsequently, the CPU 11 performs output suppression processing (step S2). Finally, the CPU 11 performs a presentation process (step S3).

  FIG. 9 is a flowchart illustrating an example of the procedure of the feature score calculation process. The CPU 11 of the feature sound extraction apparatus 1 performs feature amount calculation (step S11). FIG. 10 is a block diagram showing an example of the feature amount calculation process. FIG. 10 shows an outline of the feature amount calculation process. The feature amount calculation processing may be performed in the order of, for example, high-frequency emphasis processing, FFT (Fast Fourier Transform) processing, noise removal processing, mel filter processing, and spectral component power processing. A feature of the present embodiment is that the spectral component is subjected to a power process after calculating a feature amount (for example, a mel spectrum) including a frequency component calculated from sound data. Details of the feature amount calculation processing will be described with reference to FIG. In the following example, a mel spectrum is used as a feature amount including a frequency component calculated from sound data.

  FIG. 11 is a flowchart illustrating an example of the procedure of the feature amount calculation process. The CPU 11 of the feature sound extraction device 1 divides the input sound data every predetermined time and performs processing for each divided period. That is, the CPU 11 sets a time window, processes data in the time window, and when the process is completed, the CPU 11 shifts the time window and repeats the same process. The CPU 11 acquires data within the time window (step S21). The CPU 11 performs high frequency enhancement (step S22), FFT (step S23), amplitude calculation (step S24), noise removal (step S25), and mel spectrum extraction (step S26) on the data in the time window. Do. Since these processes are known techniques, the description thereof is omitted.

  Subsequently, the CPU 11 applies a filter to the mel spectrum obtained in step S26 (step S27). CPU11 outputs the mel spectrum which applied the filter as a feature-value (step S28). The CPU 11 returns the processing to the caller.

Here, the filter is a power function in which the multiplier p is less than the first power, and is represented by the following expression, for example.

  FIG. 12 is a graph showing an example of the input / output relationship of the filter. The horizontal axis is input and the vertical axis is output. Both the horizontal and vertical axes are dimensionless. A graph f1 having a multiplier p value of 1 is displayed for reference and is not used as a filter. The graph f2 is when the multiplier p is 0.5, and the graph f3 is when the multiplier p is 0.25. As shown in FIG. 12, the power filter having a multiplier of less than the first power has an effect of suppressing the output when it has a value of 1 or more, and always has a value of 0 or more for an input of 1 or less. Guaranteed to have. Therefore, the problem that the feature amount shape is greatly different at a fine sound volume that occurs when the mel spectrum feature is used and the problem that the value diverges with an output of 1 or less that occurs when the log filter is used are solved. Furthermore, since the volume and the frequency component are considered simultaneously in the feature amount, it is not necessary to handle the volume and frequency components in separate processing, and the processing is easy.

  FIG. 13 is a graph showing an example of the feature amount. The horizontal axis is frequency and the unit is KHz. The vertical axis is a spectrum value and is a dimensionless number. FIG. 13A is a coughing voice, and FIG. 13B is a feature amount obtained from a fan sound. As is apparent from the comparison between the two figures, the feature amount shape (spectrum value) is greatly different between the two. Therefore, it can be said that the feature amount is suitable for classifying the non-background sound (coughing voice) and the background sound (fan sound).

  Returning to FIG. 9, the CPU 11 stores the sound feature (feature amount) obtained by the feature amount calculation in the sound feature DB 14b (step S12). Subsequently, the CPU 11 performs matching between the feature amount and the feature amount of each cluster stored in the sound cluster DB 14c (step S13). The CPU 11 outputs the matched sound cluster ID and occurrence frequency (step S14). The CPU 11 returns the processing to the caller. Returning to FIG. 8, the CPU 11 performs an output suppression process (step S <b> 2).

  The output suppression process will be described. The output suppression process in the first embodiment is premised on a batch process in which sound data for a certain period of time that has already been accumulated is targeted for processing. The sound data to be processed includes a sound cluster ID, an occurrence frequency score, and a time stamp. FIG. 14 is a flowchart illustrating an example of the procedure of the output suppression process. The CPU 11 of the feature sound extraction apparatus 1 sets the suppression flag of each sound data to False and sets the feature location list F to an empty set (step S31). The suppression flag is a flag indicating whether or not to suppress the corresponding sound. If the suppression flag is True, it means output suppression and no sound is output. If the suppression flag is False, it means that the output is not suppressed, so a sound is output. The feature location list F is a list of locations containing feature sounds. The characteristic location is, for example, a time stamp of the characteristic sound.

  The CPU 11 refers to the sound cluster DB 14c and sorts the sound clusters in ascending order of occurrence frequency (step S32). The CPU 11 stores the permutation of IDs obtained from the sorted results as a sound cluster list in the RAM 12 or the like. The CPU 11 selects the first cluster in the sound cluster list as the processing target cluster (step S33). The CPU 11 acquires sound data having the same cluster ID as the selected class among the sound data to be processed, and stores it in the list L (step S34).

  The CPU 11 acquires the head sound data s from the list L (step S35). The CPU 11 determines whether or not the suppression flag for s is False and the occurrence frequency score exceeds the threshold value (step S36). If the CPU 11 determines that the suppression flag is False and the occurrence frequency score exceeds the threshold (YES in step S36), the CPU 11 adds the time stamp of the sound data s to the feature location list F (step S37). It is assumed that the threshold is predetermined. The CPU 11 sets the value of the suppression flag around the sound data s (before and after) for a certain time as True (step S38). The CPU 11 determines whether there is unprocessed sound data in the list L (step S39). If the CPU 11 determines that the suppression flag is True or the occurrence frequency score is equal to or less than the threshold (NO in step S36), the process proceeds to step S39. If the CPU 11 determines that there is unprocessed sound data in the list L (YES in step S39), the process returns to step S35. If the CPU 11 determines that there is no unprocessed sound data in the list L (NO in step S39), the CPU 11 determines whether there is an unprocessed cluster in the sound cluster DB 14c (step S40). If the CUP 11 determines that there is an unprocessed cluster (YES in step S40), the process returns to step S33. When determining that there is no unprocessed cluster (NO in step S40), the CPU 11 outputs a feature location list (step S41). The CPU 11 ends the output suppression process and returns the process to the caller. Returning to FIG. 8 again, the CPU 11 executes the presentation process (step S3) and ends the main process.

  The presentation process will be described. FIG. 15 is a flowchart illustrating an example of the procedure of the presentation process. The CPU 11 of the feature sound extraction device 1 acquires sound data (feature sound data) corresponding to the feature location from the feature sound DB 14e based on the feature location list obtained by the output suppression process (step S51). The CPU 11 determines whether or not the score of the acquired characteristic sound data exceeds the threshold value Ti (step S52). The threshold Ti may be determined in advance or may be determined when the presentation process is executed. The threshold value Ti may be a small value when many characteristic sounds are to be confirmed, and may be a large value otherwise. If the CPU 11 determines that the score exceeds the threshold (YES in step S52), the CPU 11 reproduces the characteristic sound (step S53). The CPU 11 determines whether there is unprocessed feature sound data (step S54). If the CPU 11 determines that there is unprocessed characteristic sound data (YES in step S54), the process returns to step S51. If the CPU 11 determines that there is no unprocessed feature sound data (NO in step S54), the CPU 11 ends the presentation process and returns the process to the caller. If the CPU 11 determines that the score is equal to or less than the threshold (NO in step S52), the process proceeds to step S54.

  Next, specific examples of output suppression processing and presentation processing will be shown. FIG. 16 is an explanatory diagram illustrating a specific example of output suppression processing and presentation processing. In FIG. 16, the waveform of the audio signal is shown with time on the horizontal axis and amplitude on the vertical axis. A rectangle that overlaps the waveform indicates a period divided by a time window. Shown above the waveform is a score of the feature level for each period. FIG. 16 shows two examples. In both examples, seven periods (K1 to K7, K11 to K17) separated by a time window are shown, and output suppression processing is performed in five periods. The feature degree score shown in the example of FIG. 16 is a score that is inversely proportional to the occurrence frequency, and is a score that increases as the occurrence frequency decreases. In the example on the left side of FIG. 16, the score for the period K4 is 0.5, and the scores for the other periods K1 to K3 and K5 to K7 are all 0.1. In the periods K2 to K6 to be subjected to the output suppression process, the period K4 is 0.5 and the score is larger than the other periods, so the sound of the period K4 is presented. In the example on the right side of FIG. 16, the score for the period K15 is 1.0, and the scores for the other periods K11 to K14 and K16 to K17 are all 0.5. In the periods K12 to K16 targeted for the output suppression process, the period K15 is 1.0, and the score is larger than the other periods, so the sound of the period K15 is presented. In this way, in the output suppression process, the characteristic sound is determined and presented during a plurality of periods, so even if the score is 0.5, the sound may be presented or may not be presented. In other words, it is possible to prevent a small number of characteristic parts from being extracted and being output as a whole or not as in the case of a uniform determination based on a threshold value. Become.

  Next, the presence / absence determination of a person will be described. FIG. 17 is a flowchart illustrating an example of the procedure of presence / absence determination processing. The CPU 11 of the feature sound extraction device 1 acquires sound data extracted in the past certain time from the feature sound DB 14e (step S61). The CPU 11 calculates the number C of sound data not included in the background sound in the acquired sound data (step S62). The CPU 11 determines whether or not the number C exceeds a threshold value (step S63). When the CPU 11 determines that the number C exceeds the threshold (YES in step S63), the CPU 11 determines that the return value is present (step S64). When the CPU 11 determines that the number C is equal to or less than the threshold (NO in step S63), the CPU 11 determines that the return value is absent (step S65). The CPU 11 ends the presence / absence determination process. The CPU 11 creates the digest display DB 14d by repeatedly executing the presence / absence determination process an appropriate number of times. Note that the background sound table is created before the presence / absence determination process is executed.

  FIG. 18 is an explanatory diagram showing a specific example of the presence / absence determination process. The waveforms shown in FIG. 18 are the same as those in FIG. The left side of FIG. 18 shows an example of sound data. The right side of FIG. 18 shows a conceptual representation of the background sound table. The sound data is data for 10 periods from the period K21 to K30. The sound data includes three kinds of sounds, a coughing sound, a television sound, and a fan sound. Further, according to the background sound table, coughing sounds and television sounds are defined as non-background sounds, and fan sounds are defined as background sounds. In the sound data of FIG. 18, the periods K21, K22, and K24 are fan sounds, the periods K23 and K29 are coughing sounds, and the periods K25 to K28 and K30 are television sounds. Therefore, the number of non-background sounds (circled in FIG. 18) is 7, and the number of background sounds (crossed in FIG. 18) is 3. For example, if the threshold is 6, it is determined that there was a person in the time period from the period K21 to K30. Note that the sound data, background sound data, and the like defined in FIG. 18 are examples, and another sound may be defined as the background sound due to environmental differences.

  FIG. 19 is an explanatory diagram showing an example of a digest display. In the digest display, the past 24 hours are displayed. In the example shown in FIG. 19, the hatched period represents the person's absence period 19 a, and the non-hatched period represents the person's residency period 19 b. In the present period 19b, a point 19c where a characteristic sound is further generated is indicated by an arrow. Further, the current time is indicated by an arrow 19d.

  As described above, by the presence / absence determination process, it is possible to show a tendency for a relatively long time together with detailed information on the moment.

  FIG. 20 is a flowchart illustrating an example of the procedure of occurrence frequency calculation processing. The occurrence frequency calculation process is a process performed by the clustering unit 130 and is a process for updating the sound cluster DB 14c. CPU11 acquires feature-value data from sound feature DB14b (step S71). The CPU 11 performs a clustering process on the feature data (step S72). Here, as the clustering processing, for example, a hard clustering method that fixes the number of clusters such as the k-means method, for example, a hierarchical clustering method such as the Ward method or a cluster number such as the DP-means method is dynamically determined. A hard clustering method, for example, a soft clustering method such as a method based on the optimization of the mixed Gaussian model by the EM method or a method based on the hierarchical Bayesian method by the Markov chain Monte Carlo method may be used.

  The CPU 11 calculates the frequency of occurrence of the obtained cluster (step S73). The CPU 11 stores the processing result, cluster ID, cluster occurrence frequency, and feature amount (for example, cluster center coordinates) in the sound cluster DB 14c (step S74). The CPU 11 ends the occurrence frequency calculation process.

  The occurrence frequency calculation process may be performed every time a predetermined amount of sound data is obtained, or may be performed every time a feature amount is calculated for each time window.

  As described above, in the first embodiment, by employing a power function having a multiplier of 1 or less as a filter, a problem that a feature amount shape is greatly different at a fine volume, which occurs when a mel spectrum feature is used, The problem that the value diverges with an output of 1 or less, which occurs when the log filter is used, is solved. In addition, a small number of characteristic parts are extracted by the output suppression process, and as in the case of uniform determination based on a threshold value, all output is performed or all output is not performed. It becomes possible to prevent. That is, the most characteristic part can be extracted from the life sounds. Furthermore, the presence / absence determination process makes it possible to show a tendency for a relatively long time together with detailed information on the moment.

Embodiment 2
In the second embodiment, the output suppression process is performed online. In the second embodiment, the configuration of the feature sound extraction apparatus 1 is the same as that of the first embodiment, and thus the description thereof is omitted. Since the process performed by the feature sound extraction apparatus 1 is the same as that of the first embodiment except for the output suppression process, the following description will mainly focus on the differences from the first embodiment.

  FIG. 21 is a flowchart illustrating an example of the procedure of the output suppression process. The CPU 11 of the feature sound extraction apparatus 1 acquires the sound information and the input sound information R separated by the newly input time window (step S81). The input sound information R includes a sound cluster ID, an occurrence frequency score, and a time stamp. The CPU 11 extracts the index element indicated by the ring buffer and sets it in the structure E (step S82). The CPU 11 determines whether E is the same as the maximum value element (step S83). The maximum value element may be an element having a largest score that is inversely proportional to the occurrence frequency.

  If the CPU 11 determines that E is the same as the maximum value element (YES in step S83), the CPU 11 stores E in the characteristic sound DB 14e (step S84). The CPU 11 clears the maximum value element, that is, sets it to NULL (step S85). The CPU 11 registers the input sound information R in the ring buffer (step S86). If the CPU 11 determines that E is not the same as the maximum value element (NO in step S83), the process proceeds to step S86.

  The CPU 11 determines whether the maximum element is cleared or whether the score of the maximum value element is larger than the score of the input sound information R (step S87). When the maximum element is cleared or the score of the maximum value element is larger than the score of the input sound information R (YES in step S87), the CPU 11 sets the maximum value element to R (step S88). The CPU 11 ends the output suppression process. When the maximum value element is not NULL and the score of the maximum value element is equal to or lower than the score of the input sound information R (NO in step S87), the CPU 11 ends the output suppression process.

  In the presentation process, one piece of data having the same score as the maximum value element is stored in the ring buffer, and sound data corresponding to data that has not been presented yet is reproduced.

  FIG. 22 is an explanatory diagram showing a specific example of output suppression processing. The waveforms shown in the upper part of FIG. 22 are the same as those in FIG. The middle and lower stages of FIG. 22 show the state of the ring buffer Ri. In the example shown in FIG. 22, the ring buffer Ri is composed of three buffers R1, R2, and R3. An upward arrow indicates the position of the index. The values arranged vertically indicate the values stored in each buffer. In order from the top, the order is score, cluster ID, and time stamp. FIG. 22 shows a case where data of period K36 is input as input sound information R. The R score S is 1.0, the sound cluster ID is 01, and the time stamp is 12:33. As for the maximum value element, the score S is 2.0, the sound cluster ID is 02, and the time stamp is 12:30. As shown in the middle part of FIG. 22, since the element of index I indicated by the ring buffer Ri is the same as the maximum value element, the maximum value element is once cleared and updated to a value based on the input sound information R. (See the lower part of FIG. 22).

  In the second embodiment, the feature sound can be presented almost in real time by performing online processing.

Embodiment 3
In the third embodiment, the output suppression process is different from that in the first embodiment. In the third embodiment, the configuration of the feature sound extraction apparatus 1 is the same as that of the first embodiment, and thus the description thereof is omitted. Since the process performed by the feature sound extraction apparatus 1 is the same as that in the first embodiment except for the output suppression process, the following description will mainly focus on the differences from the first embodiment.

  FIG. 23 is an explanatory diagram showing another example of output suppression processing. The waveforms and the like shown in FIG. 23 are the same as those in FIG. In the third embodiment, instead of simply selecting one location that has the maximum score, the sum of the evaluation scores in the feature locations is selected to be the maximum. In the example shown in FIG. 23, the sum of evaluation scores is obtained with three periods as one group. As shown on the right side of FIG. 23, there are three types of sound data in this example: a coughing sound, a television sound, and a fan sound. The evaluation score values are 2.0, 1.0, and 0.0, respectively. Since the group g1 has one coughing sound, no TV sound, and one fan sound, the sum S (W) of the evaluation values is calculated as follows. S (W) = 2.0 × 1 + 1.0 × 0 + 0.0 × 1 = 2.0

  When calculated in the same manner, the evaluation score of the group g2 is 1.0, and the evaluation score of the group g3 is 3.0. Therefore, the group g3 is a presentation candidate.

  In Embodiment 3, since a plurality of locations are grouped and the output suppression process is performed for each group, it is possible to present as many types of sounds as possible.

Examples of filters other than the power filter In the above description, the filter is not limited to the power filter. Any filter that satisfies the following two requirements can be used as a filter. The requirement of 1 is to take the minimum value T below a certain threshold _{ab of} 0 or more. This is because the value does not diverge when x is small. Another requirement is that the value differentiated or sub-differentiated by x is monotonously decreased in the range exceeding the threshold _ab . This is to reduce the influence of a strong sound.

  FIG. 24 is a graph showing an example of the input / output relationship of the filter. The horizontal axis is input, the vertical axis is output, and both the horizontal and vertical axes are dimensionless. FIG. 24A shows a power function of 1st power or less. FIG. 24B shows a filter function whose minimum value is not zero. FIG. 24C shows a filter function that does not have a minimum value when x = 0. FIG. 24D shows a filter function whose value greatly fluctuates locally. Since all the filter functions satisfy the above-described conditions, in the first to third embodiments, the filter shown in FIG. 24 may be employed instead of the power filter. In addition, what is shown in FIG. 24 is an example of a filter, It is not restricted to these.

Embodiment 4
It is also possible to configure a distribution system in which the characteristic sound extraction device 1 of the first to third embodiments described above is linked with a place server (distribution device). FIG. 25 is an explanatory diagram showing an example of the configuration of the distribution system. The distribution system includes a feature sound extraction device 1, a place server 2, a terminal 3, and a router 4. The place server 2 and the terminal 3 are respectively a general-purpose computer, a workstation, a desktop PC (personal computer), a notebook PC, a tablet PC, a smartphone, and the like. The feature sound extraction device 1 and the terminal 3 are connected to the place server 2 via the router 4 and the network N. The router 4 is not an essential configuration, and the feature sound extraction device 1 and the terminal 3 may be directly connected to the network N. The number of terminals 3 may be determined as appropriate. In the example shown in FIG. 25, the feature sound extraction device 1, the terminal 3, and the router 4 are installed in the same space SP.

  The place server 2 includes a control unit 21, a storage unit 22, and a communication unit 23 (reception unit, distribution unit). The control unit 21 includes a CPU, a RAM, a ROM, and the like, and controls each part of the hardware. The storage unit 22 stores a computer program distributed to the feature sound extraction device 1 and the terminal 3. The communication unit 23 communicates with the feature sound extraction device 1 and the terminal 3 via the network N and the router 4.

  The control unit 21 of the place server 2 receives the presence / absence determination result of the person from the presence / absence determination unit 141 (determination unit) of the feature sound extraction device 1 via the communication unit 23. Based on the received determination result, the control unit 21 distributes the computer program to the feature sound extraction device 1 via the communication unit 23. The characteristic sound extraction apparatus 1 further distributes the received computer program to the terminal device 3. The terminal 3 executes the received computer program. Accordingly, the terminal 3 performs an operation according to the presence / absence of a person.

  An example of the distribution system is use in CAI (Computer-Assisted Instruction or Computer-Aided Instruction). A space SP in which the feature sound extraction device 1, the terminal 3, and the router 4 are installed is taken as one classroom. When the feature sound extraction apparatus 1 determines that no person is present in the space SP, the terminal 3 is caused to execute a computer program that maintains the sleep state, thereby saving power consumption. When the characteristic sound extraction apparatus 1 determines that there is a person in the space SP, the computer program is distributed to the terminal 3 so that the terminal 3 can be used. When the space SP is a school computer room, information on the timetable may be stored in the place server 2 and an educational program for an appropriate subject may be distributed according to the time zone.

  In the fourth embodiment, since the operation of the terminal 3 can be changed depending on the presence / absence of a person, the user or the administrator of the terminal 3 does not need to perform an operation change operation. Play.

The technical features (components) described in each embodiment can be combined with each other, and new technical features can be formed by combining them.
The embodiments disclosed herein are illustrative in all respects and should not be considered as restrictive. The scope of the present invention is defined not by the above-mentioned meaning but by the scope of the claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of the claims.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
Divide the sound data every predetermined time,
Calculate the feature value including the frequency component of the sound data for each divided period,
For one or more component values x of feature quantities including frequency components of sound data calculated for each period,
In the interval a _b ≦ x ≦ a _t (0 ≦ a _b <a _t ≦ ∞), the function f which is monotonically decreased in the function differentiated or sub-differentiated by x and has a lower bound T of the function value satisfying the following formula: (X) acts,
Based on the effected result, the feature amount for each period is obtained,
A feature sound extraction method in which a process of extracting feature sound data is executed by a computer based on the obtained feature amount for each period.

(Appendix 2)
The characteristic sound extraction method according to claim 1, wherein the function f (x) is a power function having a multiplier p represented by the following expression of less than 1.

(Appendix 3)
Clustering the feature values for each period to classify the sound data,
The feature sound extraction method according to appendix 1 or 2, wherein feature sound data is extracted based on an occurrence frequency of each sound type.

(Appendix 4)
An evaluation index for each type of sound data is calculated from the occurrence frequency, the sound data of the type having the maximum evaluation index in the plurality of periods is determined as the characteristic sound data, and the determined characteristic sound data is extracted. 4. The characteristic sound extraction method according to 3.

(Appendix 5)
The feature sound extraction method according to claim 4, wherein the feature sound data is not extracted when the evaluation index value of the sound data determined as the feature sound data is smaller than a threshold value.

(Appendix 6)
An evaluation index for each type of sound data is calculated from the occurrence frequency,
Add an evaluation index of sound data included in each evaluation section including a predetermined number of the period,
Identify the evaluation interval that maximizes the added evaluation index,
The feature sound extraction method according to appendix 4 or 5, wherein the sound data included in the identified evaluation section is extracted as feature sound data.

(Appendix 7)
The sound data is collected in a space where humans can exist,
Count the number of appearances of the characteristic sound data for a predetermined time width,
The feature sound extraction method according to any one of appendices 4 to 6, wherein if the number of appearances is equal to or greater than a predetermined value, it is determined that there is a person in the space, otherwise it does not exist.

(Appendix 8)
A calculation unit that divides the sound data every predetermined time, and calculates a feature amount including a frequency component of the sound data for each divided period;
With respect to one or more component values x of feature quantities including frequency components of sound data calculated for each period, differentiation is performed with x in a section a _b ≦ x ≦ a _t (0 ≦ a _b <a _t ≦ ∞). Alternatively, a filter unit that applies a function f (x) in which a sub-differentiated function is monotonically decreasing and a lower bound T of a function value that satisfies the following equation exists,
A feature amount calculation unit that calculates a feature amount for each period based on the effected results;
A feature sound extraction apparatus comprising: an extraction unit that extracts feature sound data based on the obtained feature amount for each period.

(Appendix 9)
Divide the sound data every predetermined time,
Calculate the feature value including the frequency component of the sound data for each divided period,
For one or more component values x of feature quantities including frequency components of sound data calculated for each period,
In the interval a _b ≦ x ≦ a _t (0 ≦ a _b <a _t ≦ ∞), the function f which is monotonically decreased in the function differentiated or sub-differentiated by x and has a lower bound T of the function value satisfying the following formula: (X) acts,
Based on the effected result, the feature amount for each period is obtained,
A computer program that causes a computer to execute a process of extracting feature sound data based on a calculated feature amount for each period.

(Appendix 10)
A calculation unit that divides sound data collected in a space where a person may exist at predetermined time intervals, and calculates a feature amount including a frequency component of the sound data for each divided period;
With respect to one or more component values x of feature quantities including frequency components of sound data calculated for each period, differentiation is performed with x in a section a _b ≦ x ≦ a _t (0 ≦ a _b <a _t ≦ ∞). Alternatively, a filter unit that applies a function f (x) in which a sub-differentiated function is monotonically decreasing and a lower bound T of a function value that satisfies the following equation exists,
A feature amount calculation unit that calculates a feature amount for each period based on the effected results;
An extraction unit that extracts feature sound data based on the obtained feature amount for each period;
A counting unit that counts the number of appearances of the extracted characteristic sound data for a predetermined time width;
A determination unit that determines that a person is present in the space if the number of appearances is equal to or greater than a predetermined number;
A characteristic sound extraction device having a transmitter for transmitting the determined result, and a receiver for receiving the determined result;
A distribution system comprising: a distribution device having a distribution unit that distributes a predetermined computer program to the characteristic sound extraction device based on a received result.

1 Feature Sound Extractor 11 CPU
12 RAM
13 ROM
14 Mass storage device 14a Living sound DB
14b Sound feature DB
14c Sound cluster DB
14d Digest display DB
14e Feature sound DB
DESCRIPTION OF SYMBOLS 15 Input part 15a Microphone 16 Output part 16a Display apparatus 17 Communication part 18 Reading part 1P Control program 1a Portable storage medium 1b Semiconductor memory 110 Input part 111 Living sound input part 120 Feature score calculation part 121 Sound feature calculation part 122 Sound cluster matching Score calculation unit 130 Clustering unit 131 Clustering processing unit 132 Cluster occurrence frequency calculation unit 140 Digest display generation unit 141 Presence / absence determination unit 150 Output suppression unit 151 Feature location output suppression unit 160 Presentation unit 161 GUI display unit 162 Voice presentation unit 2 Place server 3 Terminal 4 Router N Network

Claims (5)

Divide the sound data every predetermined time,
Calculate the feature value including the frequency component of the sound data for each divided period,
For one or more component values x of feature quantities including frequency components of sound data calculated for each period,
In the section _{x ≧ a b (a b ≧} 0), the function obtained by differentiating or subderivative at x is monotonously decreasing function lower bound T function values which satisfy the following formula is present in the interval 0 ≦ x ≦ a _b f (X) acts,
Based on the effected result, the feature amount for each period is obtained,
A feature sound extraction method in which a process of extracting feature sound data is executed by a computer based on the obtained feature amount for each period.

The characteristic sound extraction method according to claim 1, wherein the function f (x) is a power function in which a multiplier p represented by the following expression is less than one.

A calculation unit that divides the sound data every predetermined time, and calculates a feature amount including a frequency component of the sound data for each divided period;
A function that is differentiated or sub-differentiated with x in a section x ≧ a _b (a _b ≧ 0) with respect to one or more component values x of feature quantities including frequency components of sound data calculated for each period is monotonously decreased. , and the filter unit that applies the function f (x) lower bound T function values which satisfy the following formula is present in the interval 0 ≦ x ≦ a _b,
A feature amount calculation unit that calculates a feature amount for each period based on the effected results;
A feature sound extraction apparatus comprising: an extraction unit that extracts feature sound data based on the obtained feature amount for each period.

Divide the sound data every predetermined time,
Calculate the feature value including the frequency component of the sound data for each divided period,
For one or more component values x of feature quantities including frequency components of sound data calculated for each period,
In the section _{x ≧ a b (a b ≧} 0), the function obtained by differentiating or subderivative at x is monotonously decreasing function lower bound T function values which satisfy the following formula is present in the interval 0 ≦ x ≦ a _b f (X) acts,
Based on the effected result, the feature amount for each period is obtained,
A computer program that causes a computer to execute a process of extracting feature sound data based on a calculated feature amount for each period.

A calculation unit that divides sound data collected in a space where a person may exist at predetermined time intervals, and calculates a feature amount including a frequency component of the sound data for each divided period;
A function that is differentiated or sub-differentiated with x in a section x ≧ a _b (a _b ≧ 0) with respect to one or more component values x of feature quantities including frequency components of sound data calculated for each period is monotonously decreased. , and the filter unit that applies the function f (x) lower bound T function values which satisfy the following formula is present in the interval 0 ≦ x ≦ a _b,
A feature amount calculation unit that calculates a feature amount for each period based on the effected results;
An extraction unit that extracts feature sound data based on the obtained feature amount for each period;
A counting unit that counts the number of appearances of the extracted characteristic sound data for a predetermined time width;
A determination unit that determines that a person is present in the space if the number of appearances is equal to or greater than a predetermined number;
A characteristic sound extraction device having a transmitter for transmitting the determined result, and a receiver for receiving the determined result;
A distribution system comprising: a distribution device having a distribution unit that distributes a predetermined computer program to the characteristic sound extraction device based on a received result.

Images