JP6959420B1 - Signal processing device and signal processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To satisfactorily separate an input physical quantity into a noise component and an extracted signal. A signal processing device (10) includes a learning unit (21) and a separation unit (40). The learning unit 21 uses the neural network 94 to generate a mask α for removing the noise component by adding the phase from the input physical quantity (sound near the engine 90) including the noise component (noise 91b). The phase of the extracted signal (extracted knocking sound 91a) from which the noise component is removed from the input physical quantity is set to the estimated signal (estimated knocking cylinder internal pressure 92) having the same dimension (unit) as the teacher signal (observed knocking cylinder internal pressure 93). The transfer function H for addition and conversion is learned. The separation unit 40 separates the input physical quantity into a noise component and an extraction signal. [Selection diagram] FIG. 3A

Description

The present invention relates to a signal processing device that satisfactorily separates an input physical quantity into a noise component and an extracted signal from which the noise component has been removed, and a signal processing method.

For example, the ignition timing of an internal combustion engine such as a gasoline engine is generally advanced as much as possible within the range of the crank angle at which knocking does not occur for the purpose of improving the output torque. Therefore, in the process of adjusting the ignition timing, the tester or the knocking determination device determines whether or not knocking has occurred. An example of such a knocking determination device is described in Patent Document 1.

The knocking determination device described in Patent Document 1 is a condition in which a determination signal for determining the presence or absence of knocking and a target signal to be compared are determined by a relationship with the determination signal (for example, a temporal relationship or a relationship in operating conditions). Select based on.

With the knocking determination device described in Patent Document 1, only the determination result of the presence or absence of knocking can be presented to the outside, and it is difficult to support the determination result. Therefore, the inventors of the present invention have proposed the device described in Patent Document 2 or Patent Document 3 as a device capable of estimating a desired physical quantity that supports the determination result.

The device described in Patent Document 2 is "a learning device for removing a noise component contained in an input physical quantity and estimating a desired physical quantity from the input physical quantity from which the noise component has been removed, and the noise component is a learning device. It is noise other than the knocking sound generated in the internal combustion engine, the input physical quantity is the sound pressure of the internal combustion engine including the noise and the knocking sound, and the desired physical quantity is the sound pressure of the internal combustion engine when knocking occurs. It is an in-cylinder pressure, the weight of the network that removes the noise by the neural network and generates a mask for extracting the knocking sound, and / or the internal combustion engine of the knocking sound extracted by the mask at the time of the knocking occurrence. A learning device characterized by including a learning unit that learns a transmission function that converts an in-cylinder pressure into an engine. " The apparatus described in Patent Document 2 uses a neural network to generate a mask that removes noise components contained in the input physical quantity, and / or sets the input physical quantity from which noise components are removed by the mask to a desired physical quantity. A learning unit for learning the transfer function to be converted is provided, a noise component included in the input physical quantity is removed, a desired physical quantity is estimated from the input physical quantity from which the noise component has been removed, and the presence or absence of knocking is determined.

The device described in Patent Document 3 is "a knocking determination device that removes noise other than the knocking sound generated in an internal combustion engine and estimates the knocking sound from which the noise has been removed, and removes the noise by a neural network. A learning unit that learns the weight of the network that generates the mask that extracts the knocking sound, and the transmission function that converts the knocking sound extracted by the mask into the in-cylinder pressure of the internal combustion engine when knocking occurs. A knocking determination device comprising a second estimation unit that estimates a knocking sound in which the noise is removed from a knocking sound containing the noise by using the mask by a neural network. " be. The device described in Patent Document 3 estimates a knocking sound in which noise is removed from a knocking sound containing noise by using a mask by a neural network.

Japanese Unexamined Patent Publication No. 2017-44148 Japanese Patent No. 6605170 Japanese Patent No. 6651040

However, the prior art described in Patent Documents 2 and 3 has a configuration in which the input physical quantity is separated into a noise component and an extraction signal without considering the phase component related to the input physical quantity. Therefore, in the prior art, even if the input physical quantity is separated into the noise component and the extraction signal, the extraction signal is mixed in the noise component, and the extraction signal cannot be separated accurately.

The present invention has been made to solve the above-mentioned problems, and a signal processing device that satisfactorily separates an input physical quantity into a noise component and an extracted signal by realizing a configuration in consideration of a phase component, and a signal processing device. The main purpose is to provide a signal processing method.

To solve the above problems, the present invention is a signal processing device, by the neural network, the weights of the network for generating a mask for removing an input physical quantity or found before Symbol noise component that contains the noise component as well as learning, a learning unit that learns a transfer function for convert the extracted signal in which the noise component is removed from the input physical quantity in the same dimension of the estimation signal and the teacher signal, and the input physical quantity the noise component The learning unit includes a separation unit that separates from the extracted signal, and the learning unit learns the weight of the network that generates the mask by adding a phase component related to the input physical quantity, and is related to the input physical quantity. The transfer function is learned by adding the phase component to be used.

Further, the present invention is a signal processing method, by the neural network, with learning the weights of the network for generating a mask for removing an input physical quantity or found before Symbol noise component that contains the noise component, the a learning step for learning a transfer function for convert the extracted signal in which the noise component is removed from the input physical quantity in the same dimension of the estimation signal and the teacher signal, the input physical quantity and the extraction signal and the noise component look including a separation step of separating, in said learning step, in consideration of the phase components associated with the input physical quantity, as well as learning the weights of the network for generating the mask, the phase components associated with the input physical quantity In addition, the configuration is such that the transfer function is learned.
Other means will be described later.

According to the present invention, the input physical quantity can be satisfactorily separated into a noise component and an extraction signal.

It is a block diagram which shows the whole structure of the signal processing system including the signal processing apparatus (estimating apparatus) which concerns on 1st Embodiment. It is a block diagram which shows the structure of the signal processing apparatus (estimation apparatus) which concerns on 1st Embodiment. It is explanatory drawing of the learning mode. It is operation explanatory drawing in the learning mode of the signal processing apparatus (estimation apparatus) which concerns on 1st Embodiment. It is explanatory drawing of the threshold value calculation mode. It is operation explanatory figure in the threshold value calculation mode of the signal processing apparatus (estimation apparatus) which concerns on 1st Embodiment. It is explanatory drawing of the determination mode. It is operation explanatory figure in the determination mode of the signal processing apparatus (estimation apparatus) which concerns on 1st Embodiment. It is explanatory drawing of the separation mode. It is operation explanatory drawing in the separation mode of the signal processing apparatus (estimation apparatus) which concerns on 1st Embodiment. It is explanatory drawing of the sensory test mode. It is operation explanatory drawing in the sensory test mode of the signal processing apparatus (estimating apparatus) which concerns on 1st Embodiment. It is explanatory drawing which shows the relationship between a noise component and a teacher signal at the time of learning. It is explanatory drawing which shows the relationship between the extraction signal and a teacher signal at the time of learning. It is explanatory drawing which shows the relationship between the extraction signal and the estimation signal at the time of learning. It is explanatory drawing of the neural network which learns the weight of the network which generates a mask in 1st Embodiment. It is explanatory drawing of the calculation example when the phase component such as pressure, vibration, and sound is not considered. It is explanatory drawing of the calculation example when the phase component such as pressure, vibration, and sound is taken into consideration. It is explanatory drawing (1) of signal processing in a sensory test. It is explanatory drawing (2) of the signal processing in a sensory test. It is explanatory drawing (3) of the signal processing in a sensory test. It is explanatory drawing (4) of the signal processing in a sensory test. It is explanatory drawing (5) of signal processing in a sensory test. It is explanatory drawing (6) of signal processing in a sensory test. It is a flowchart which shows the data collection processing of the signal processing apparatus (estimating apparatus) in 1st Embodiment. It is a flowchart which shows the learning process of the signal processing apparatus (estimating apparatus) in 1st Embodiment. It is a flowchart which shows the subroutine of learning processing. It is a flowchart which shows the modification example of the subroutine of learning processing. It is a flowchart which shows the threshold value calculation process of the signal processing apparatus (estimating apparatus) in 1st Embodiment. In the first embodiment, it is a flowchart which shows the threshold value calculation process of the signal processing apparatus (estimation apparatus) performed after the separation process of FIG. 17 and the sensory test process of FIG. It is a flowchart which shows the determination process of the signal processing apparatus (estimating apparatus) in 1st Embodiment. It is a flowchart which shows the separation process of the signal processing apparatus (estimating apparatus) in 1st Embodiment. It is a flowchart which shows the sensory test processing of the signal processing apparatus (estimating apparatus) in 1st Embodiment. It is a block diagram which shows the structure of the signal processing apparatus (estimation apparatus) which concerns on 2nd Embodiment. It is explanatory drawing of the 1st modification. In the first modification, it is explanatory drawing of learning of the weight and the transfer function of the network which generates a mask. It is explanatory drawing of the 2nd modification. In the second modification, it is explanatory drawing of learning of the weight and the transfer function of the network which generates a mask. In the third modification, it is explanatory drawing of learning of the weight and the transfer function of the network which generates a mask.

Hereinafter, embodiments of the present invention (hereinafter, referred to as “the present embodiment”) will be described in detail with reference to the drawings. It should be noted that each figure is merely schematically shown to the extent that the present invention can be fully understood. Therefore, the present invention is not limited to the illustrated examples. Further, in each figure, common components and similar components are designated by the same reference numerals, and duplicate description thereof will be omitted.

[First Embodiment]
<Overall configuration of signal processing system including signal processing device (estimating device)>
Hereinafter, the overall configuration of the signal processing system 100 including the signal processing device 10 (estimating device) according to the first embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing an overall configuration of a signal processing system 100 including a signal processing device 10 (estimating device).

The signal processing device 10 is a device that performs various types of processing on a signal. In the present embodiment, the signal processing device 10 will be described as functioning as an estimation device that estimates an estimation signal of the same dimension (unit) as the teacher signal by multiplying the extraction signal from which the noise component has been removed by a transfer function. .. Further, in the present embodiment, the signal processing device 10 will be described as being used for the purpose of verifying whether or not knocking has occurred in an internal combustion engine such as a gasoline engine. However, the signal processing device 10 is not limited to such applications, and can be used for various purposes.

As shown in FIG. 1, the signal processing system 100 determines whether or not the engine 1 to be tested is knocked, and includes a sound pressure sensor 4, an in-cylinder pressure sensor 5, a data collecting device 6, and a monitor 7. The headphone 8, the level designation unit 9, and the signal processing device 10 are provided.

As shown in FIG. 1, the engine 1 to be tested is mounted on the vehicle 3. The engine 1 to be tested may be used in a state where it is not mounted on the vehicle 3, for example, in a single state. An engine ECU (Electronic Control Unit) 2 that controls the drive of the engine 1 is connected to the engine 1. The engine ECU 2 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random access memory), and other storage devices. The engine ECU 2 controls the drive of the engine 1 while acquiring various information necessary for the drive control of the engine 1 from the outside of the engine ECU 2 by arithmetically processing the program stored in the ROM or the storage device by the CPU.

The engine ECU 2 outputs angle information representing the current rotation angle of the engine 1 to the data collection device 6. The angle information includes, for example, a rotation pulse and a crank angle pulse. The rotation pulse is a signal that outputs an absolute angle on the crank shaft. For example, one pulse is output for each rotation of the crank shaft. The crank angle pulse is a signal that is output every time the rotation angle of the crank shaft advances by a unit angle. For example, when one pulse is output every 1 °, the four strokes of intake, compression, combustion, and exhaust are 1. In a 4-stroke engine as a cycle, 720 pulses are output during two rotations in which the crank shaft rotates in one cycle. If the angle information is input to the data collecting device 6, the angle from the origin sensor that detects the origin of the rotation angle of the crank shaft and the angle sensor that detects the rotation angle of the crank shaft without going through the engine ECU 2. Information may be input to the data collection device 6.

A sound pressure sensor 4 is installed near the engine 1. The sound pressure sensor 4 detects a sound generated from the engine 1 and outputs a sound pressure signal based on the detected sound to the data collection device 6. More specifically, the sound pressure sensor 4 detects the sound pressure, which is an example of the physical amount based on the in-cylinder pressure fluctuation generated in the engine 1, and generates a sound pressure signal indicating the magnitude of the detected sound pressure. Therefore, when knocking does not occur in the engine 1, the sound pressure signal output from the sound pressure sensor 4 does not include sounds that correlate with knocking. On the other hand, when knocking occurs in the engine 1, the sound pressure signal output from the sound pressure sensor 4 includes a sound correlating with the knocking.

An in-cylinder pressure sensor 5 for detecting the in-cylinder pressure of the engine 1 is attached to the engine 1. The in-cylinder pressure sensor 5 outputs an in-cylinder pressure signal including a waveform component corresponding to the vibration of the combustion gas in the cylinder to the data collecting device 6. Here, when knocking does not occur in the engine 1, the in-cylinder pressure signal output from the in-cylinder pressure sensor 5 does not include a component that correlates with knocking. On the other hand, when knocking occurs in the engine 1, the in-cylinder pressure signal output from the in-cylinder pressure sensor 5 contains a component that correlates with knocking. The in-cylinder pressure sensor 5 may be integrated with the spark plug or may be configured separately from the spark plug.

The data collection device 6 inputs the sound pressure signal from the sound pressure sensor 4 and performs A / D conversion. Further, the data collection device 6 acquires the current angle information from the engine ECU 2 at the timing of inputting the sound pressure signal. Then, the data collection device 6 acquires the sound pressure signal for one cycle of the engine 1 based on the angle information. Therefore, taking a 4-stroke single-cylinder engine as an example, the data collection device 6 has a number corresponding to the rotation speed of the engine 1 per unit time, for example, if the rotation speed is 3000 [r / min], 1500 per minute. Generates individual sound pressure signals. Further, the data collection device 6 associates angle information with a part or all of the sound pressure signals. Then, the data collection device 6 outputs a sound pressure signal associated with the angle information to the signal processing device 10. The data collecting device 6 may temporarily hold the sound pressure signal or temporarily store the sound pressure signal and then output the sound pressure signal to the signal processing device 10. Further, the data collection device 6 may associate the time information with the sound pressure signal.

The monitor 7 displays the in-cylinder pressure of the engine 1 estimated by the signal processing device 10 and the determination result of the presence or absence of knocking by the signal processing device 10. As an example of the monitor 7, there is a general flat panel display.

The headphone 8 is a sound emitting unit that emits sound. The headphones 8 are worn on the head of the operator of the signal processing device 10 when inspecting the state of the test object (engine 1 in this embodiment) in the sensory test mode described later.

The level designation unit 9 is an input unit that receives level designation information for designating an ascending level or a descending level for a signal. The level designation unit 9 is composed of, for example, a touch panel display, a numeric keypad, a dedicated switch, and the like. In the present embodiment, when the level of the extraction signal (extraction knocking sound 91a (FIG. 11B)) is changed to generate the level change extraction signal (level change knocking sound 91aa (FIG. 12A)), the rising level of the extraction signal or In order to specify (input) the descending level, the level designating unit 9 is operated by the operator of the signal processing device 10 or a person around it.

The signal processing device 10 learns the weight W (FIG. 3B) and the transfer function H (FIG. 3A) of the network that generates the mask α. Here, the mask α is a matrix of real numbers or complex numbers for removing the noise component from the input physical quantity including the noise component. The mask α changes according to the input physical quantity. Further, the transfer function H is a complex number weight vector, and is interpreted as the reciprocal of the structural attenuation correction amount of the engine 1. The structural damping correction amount is a transmission characteristic until the vibration caused by the in-cylinder pressure during engine combustion passes through the engine 1 and becomes a sound to reach the sound pressure sensor. The combustion noise level of the engine 1 is obtained by multiplying the frequency component of the in-cylinder pressure of the engine 1 by the structural damping correction amount.

Therefore, the in-cylinder pressure of the engine 1 can be obtained by multiplying the combustion noise included in the noise near the engine 1, which is the sound pressure, by the reciprocal of the structural attenuation correction amount. Therefore, the in-cylinder pressure of the engine 1 can be estimated from the noise in the vicinity of the engine 1.

The principle of estimating the in-cylinder pressure of the engine 1 from the near sound of the engine 1 is described in, for example, Patent Document 2 described above. For example, according to the above-mentioned Patent Document 2, "First, the proximity sound y of the engine 1 and the measured in-cylinder pressure x of the engine 1 are collected as teacher data (teacher signal) .... The neighborhood sound of the engine 1. Using y and the measured in-cylinder pressure x of the engine 1, the weight and transmission function H of the network that generates the unknown mask α are learned by the neural network, and the mask α and the transmission function H generated by the learned network are learned. The in-cylinder pressure x of the engine 1 is estimated from the near sound y of the engine 1 measured at the time of the test. "

The signal processing device 10 extracts a knocking sound from the near sound of the engine 1 by using the mask α and the transmission function H generated by the learned network, estimates the knocking cylinder internal pressure of the engine 1, and estimates the extracted knocking sound and the estimation. The presence or absence of knocking is determined based on the knocking cylinder internal pressure of the engine 1. Then, the signal processing device 10 displays the extracted knocking sound, the estimated knocking cylinder internal pressure of the engine 1, and the determination result of the presence or absence of knocking on the monitor 7.

Here, the signal processing device 10 is composed of a CPU, a ROM, a RAM, another storage device, and the like. The signal processing device 10 performs arithmetic processing on the program stored in the ROM or the storage device by the CPU. The signal processing device 10 may be a personal computer (PC) or the like having a program that executes the following processing.

In the present embodiment, the signal processing device 10 operates in five operation modes: a learning mode, a threshold value calculation mode, a determination mode, a separation mode, and a sensory test mode. The first learning mode is an operation mode for learning the network weight W (FIG. 3B) and the transfer function H (FIG. 3A) that generate the mask α (FIG. 3A). The second threshold value calculation mode is an operation mode for calculating the threshold value when determining the presence or absence of knocking after learning the weight W and the transfer function H of the network that generates the mask α. The third determination mode uses a mask α generated by inputting an input physical quantity into the learned network to extract a signal (extracted in the present embodiment) from the input physical quantity (engine near sound 90 in the present embodiment). This is an operation mode in which the knocking sound 91a) is extracted and the presence or absence of knocking is determined. The fourth separation mode is an operation mode in which the input physical quantity is separated into an extraction signal and a noise component (noise 91b in this embodiment). The fifth sensory test mode is for listening to the processed sound described later by the inspector and determining the data to be used in the threshold value calculation mode in the target sound (knocking sound in this embodiment) to be inspected by the inspector's hearing. Operation mode. In the fifth sensory test mode, for example, the threshold value is calculated from the sounds that are out of the permissible range.

These five operation modes can be arbitrarily switched. For example, the operation mode of the signal processing device 10 can be switched via the CAN (Controller Area Network) by a management device (not shown). Further, the operation mode of the signal processing device 10 may be switched by using an operating means such as a mouse or a keyboard (not shown).

In the learning mode, the engine 1 is operated under the operating conditions where knocking occurs and the operating conditions where knocking does not occur, respectively, and the data collection device 6 collects the in-cylinder pressure signal as teacher data (teacher signal). At this time, the data collection device 6 inputs the in-cylinder pressure signal from the in-cylinder pressure sensor 5, performs A / D conversion, and associates this with the sound pressure signal.

In the threshold value calculation mode, the engine 1 is operated under operating conditions where knocking does not occur, and the data collection device 6 collects the sound pressure signal.

In the determination mode, the extracted knocking sound 91a (extracted signal) is extracted from the engine proximity sound 90 (input physical quantity) using the mask α generated by the learned network, and the signal processing device 10 knocks based on the threshold value. Determine the presence or absence.

In the threshold calculation mode or the determination mode, the data collection device 6 does not need to collect the in-cylinder pressure signal. Further, in the learning mode, both the sound pressure sensor 4 and the in-cylinder pressure sensor 5 operate, but in the determination mode, only the sound pressure sensor 4 operates.

In the separation mode, the engine 1 is operated, the signal processing device 10 inputs the engine proximity sound 90 (input physical quantity) into the learned network to generate a mask α, and the noise 91b (noise component) and the extraction knocking sound 91a are generated. Separated into (extracted signal). At that time, the signal processing device 10 stores the noise 91b (noise component) in the noise component storage unit 26c and the extraction knocking sound 91a (extraction signal) in the extraction signal storage unit 26d. In the present invention, the separation performance is improved by learning the weight W and the transfer function H of the network that generates the mask α in consideration of the amplitude and phase related to the input physical quantity. The mask α is implemented as a real number or a complex number, and the transfer function H is implemented as a complex number. As a result, the signal processing device 10 can satisfactorily separate the input physical quantity into the noise component and the extracted signal.

In the sensory test mode, the signal processing device 10 receives the level designation information from the level designation unit 9, and processes using the noise 91b (noise component) and the extraction knocking sound 91a (extraction signal) based on the level designation information. Generate sound.

Even if the amplitude spectrum is the same, the signal waveform changes greatly depending on the phase spectrum, which greatly affects the audible impression. Therefore, in order to generate a high-quality processed sound, it is important to acquire the extraction signal (extract knocking sound 91a in the present embodiment) used for the processed sound in consideration of the phase. In other words, the signal processing device 10 improves the separation performance by learning the weight W and the transfer function H of the network that generates the mask α in consideration of the amplitude and phase related to the input physical quantity, and performs high-quality processing. It is possible to generate a sound (a processed sound in which the knocking sound is natural in terms of audibility).

<Configuration of signal processing device (estimation device)>
Hereinafter, the configuration of the signal processing device 10 (estimation device) will be described with reference to FIG. FIG. 2 is a block diagram showing the configuration of the signal processing device 10 (estimating device). As shown in FIG. 2, the signal processing device 10 includes a signal cutting unit 11, a spectrogram calculation unit 12, a signal storage unit 13, a switch 14, a learning processing unit 20 (learning device), and a determination processing unit 30. And a separation unit 40 and a signal synthesis unit 50. Here, the signal processing device 10 receives the sound pressure signal and the angle information associated with the sound pressure signal from the data collecting device 6. Further, in the learning mode, the signal processing device 10 receives an in-cylinder pressure signal from the data collecting device 6.

The signal cutting unit 11 cuts out a sound pressure signal in a predetermined cutting angle range from the input sound pressure signal based on the angle information input from the data collecting device 6. For example, the cutout angle range is an angle range of about −10 to 90 ° of ATDC (After Top Dead Center). In the present embodiment, since the cutting is performed with reference to the TDC (Top Dead Center), the cutting angle range remains fixed even if the ignition timing is changed, but the cutting angle range is changed according to the change in the ignition timing. May be changed. When the signal cutting unit 11 cuts out the sound pressure signal, the signal cutting unit 11 outputs the cut out sound pressure signal to the spectrogram calculation unit 12.

The spectrogram calculation unit 12 performs a short-time Fourier transform (STFT) on the sound pressure signal cut out by the signal cutting unit 11 to calculate a spectrogram of the sound pressure signal. The short-time Fourier transform is performed, for example, by a fast Fourier transform (FFT) that calculates a discrete Fourier transform at high speed.
After that, the spectrogram calculation unit 12 writes the spectrogram of the sound pressure signal into the signal storage unit 13.

The signal storage unit 13 is a storage device such as a memory, HDD (Hard Disk Drive), SSD (Solid State Drive), etc. that stores the spectrogram of the sound pressure signal converted by the spectrogram calculation unit 12. In the learning mode, the signal storage unit 13 stores the in-cylinder pressure signal (observation knocking in-cylinder pressure 93 (FIG. 3A)) input from the data collection device 6 as teacher data (teacher signal). Here, the "knocking in-cylinder pressure" represents a knocking component superimposed on the in-cylinder pressure signal. The knocking in-cylinder pressure is obtained by performing a high-pass filter process in which the in-cylinder pressure is passed through a frequency band equal to or higher than the knocking frequency component.

The switch 14 is attached to the learning mode connection unit M1, the threshold value calculation mode connection unit M2, the determination mode connection unit M3, the separation mode connection unit M4, and the sensory test mode connection unit M5 corresponding to the above-mentioned five operation modes. It can be switched arbitrarily. The signal processing device 10 can output the signal stored in the signal storage unit 13 to an arbitrary output destination by switching the switch 14 according to the operation mode of the signal processing device 10.

For example, in the learning mode, the signal processing device 10 connects the switch 14 to the learning mode connection unit M1 and the learning unit 21 which describes the spectrogram of the sound pressure signal and the in-cylinder pressure signal stored in the signal storage unit 13. Output to. Further, for example, in the case of the threshold value calculation mode, the signal processing device 10 connects the switch 14 to the threshold value calculation mode connection unit M2, and the first estimation unit described later describes the spectrogram of the sound pressure signal stored in the signal storage unit 13. Output to 23. Further, for example, in the case of the determination mode, the signal processing device 10 connects the switch 14 to the determination mode connection unit M3, and the second estimation unit 31 below describes the spectrogram of the sound pressure signal stored in the signal storage unit 13. Output to (extract signal estimation unit). Further, for example, in the case of the separation mode, the signal processing device 10 connects the switch 14 to the separation mode connection unit M4 and outputs the spectrogram of the sound pressure signal stored in the signal storage unit 13 to the separation unit 40 described later. do. Further, for example, in the case of the sensory test mode, the signal processing device 10 connects the switch 14 to the sensory test mode connection unit M5 and outputs the execution instruction signal of the sensory test mode to the signal adjusting unit 51 described later.

<Learning processing department>
The learning processing unit 20 performs learning processing (FIG. 14A) for learning the weight W of the network that generates the mask α and the transmission function H in the learning mode, and determines the threshold value of the knocking sound extracted by the engine 1 in the threshold value calculation mode. A threshold value calculation process (FIG. 15A) for calculating the threshold value used for is performed. As shown in FIG. 2, the learning processing unit 20 includes a learning unit 21, a learned parameter storage unit 22, a first estimation unit 23, a threshold calculation unit 24, a threshold storage unit 25, and a teacher signal storage unit 26a. The estimation signal storage unit 26b, the noise component storage unit 26c, and the extraction signal storage unit 26d are provided.

In the learning mode, the learning unit 21 generates a mask α that removes the noise 91b from the engine near sound 90 (input physical quantity) and extracts the extraction knocking sound 91a (extraction signal) by the neural network 94 (FIG. 3A). The weight W of the network to be generated and the transmission function H that converts the extracted knocking sound 91a (extracted signal) extracted by the generated mask α into the estimated knocking in-cylinder pressure 92 of the engine 1 when knocking occurs are learned. In the present embodiment, the learning unit 21 learns the weight W and the transfer function H of the network that generates the mask α by using the spectrogram of the sound pressure signal input from the signal storage unit 13 and the in-cylinder pressure signal. After that, the learning unit 21 writes the weight W and the transfer function H of the network that generates the learned mask α into the learned parameter storage unit 22.

The learned parameter storage unit 22 is a storage device for a memory, an HDD, an SSD, or the like that stores the learned parameters (the weight W of the network that generates the mask α and the transfer function H).

The first estimation unit 23 extracts the knocking sound (extracted signal) of the engine 1 from the spectrogram of the sound pressure signal by using the mask α generated by the neural network 94 in the threshold value calculation mode. The knocking sound (extracted signal) extracted by the first estimation unit 23 is used when calculating the threshold value described later. Specifically, the first estimation unit 23 is a spectrogram of the sound pressure signal input from the signal storage unit 13 to the neural network 94 in which the weight W of the network that generates the mask α of the learned parameter storage unit 22 is reflected. Enter. Then, the mask α generated by the learned network extracts the knocking sound (extracted signal) from the spectrogram of the sound pressure signal. After that, the first estimation unit 23 outputs the knocking sound (extracted signal) to the threshold value calculation unit 24.

The threshold value calculation unit 24 calculates the threshold value based on the knocking sound (extracted signal) of the engine 1 extracted by the first estimation unit 23. Specifically, the threshold value calculation unit 24 sums the absolute values of the spectrograms of the knocking sounds (extracted signals) for each predetermined time (for example, one cycle of the engine 1). For example, in the engine 1 having a plurality of cylinders, when the knocking sounds (extracted signals) are summed, one score is obtained for each cylinder. Subsequently, the threshold value calculation unit 24 calculates the median value of the knocking sounds (extracted signals) summed up at predetermined time intervals. For example, the threshold value calculation unit 24 calculates the median value of the total sum of knocking sounds (extracted signals) for all cycles. At this time, the threshold value calculation unit 24 adds a margin preset with an arbitrary value to the median value to obtain a threshold value. The threshold value calculation unit 24 may sum the knocking sounds (extracted signals) for each cylinder to obtain the median value, and calculate the threshold value for each cylinder. On the other hand, the threshold value calculation unit 24 may sum the knocking sounds (extracted signals) in each cylinder, obtain the median value in all cylinders, and calculate a common threshold value in all cylinders. Further, the sum of the knocking sounds (extracted signals) of the processed sounds 91c judged to be unacceptable by the inspector in the sensory test described later may be obtained, and an arbitrary value equal to or less than the total value may be used as the threshold value. After that, the threshold value calculation unit 24 writes the calculated threshold value in the threshold value storage unit 25.

The threshold value storage unit 25 is a storage device such as a memory, HDD, SSD, etc. that stores the threshold value calculated by the threshold value calculation unit 24.

The teacher signal storage unit 26a is a storage device such as a memory, HDD, SSD, etc. that stores a teacher signal (in this embodiment, the observation knocking cylinder internal pressure 93 (FIG. 3A)).

The estimated signal storage unit 26b is a storage device for a memory, an HDD, an SSD, or the like that stores an estimated signal (in this embodiment, an estimated knocking cylinder internal pressure 92 (FIG. 3A)).

The noise component storage unit 26c is a storage device for a memory, HDD, SSD, or the like that stores a noise component (noise 91b (FIG. 6A) in the present embodiment) included in the input physical quantity.

The extraction signal storage unit 26d stores a memory, an HDD, an SSD, or the like that stores an extraction signal (in this embodiment, the extraction knocking sound 91a (FIG. 6A)) in which the noise component is removed from the input physical quantity containing the noise component. It is a device.

<Judgment processing unit>
In the determination mode, the determination processing unit 30 extracts the knocking sound of the engine 1 from the spectrogram of the sound pressure signal, and performs determination processing (FIG. 16) for determining the presence or absence of knocking. As shown in FIG. 2, the determination processing unit 30 includes a second estimation unit 31 (extraction signal estimation unit) and a threshold value determination unit 32.

The second estimation unit 31 (extraction signal estimation unit) extracts the extraction knocking sound 91a (extraction signal) from the spectrogram of the sound pressure signal by using the mask α generated by the neural network 94 in the determination mode. The extracted knocking sound 91a (extracted signal) extracted by the second estimation unit 31 is used for the threshold value determination described later. Since the processing content of the second estimation unit 31 is the same as that of the first estimation unit 23, the description thereof will be omitted.
After that, the second estimation unit 31 outputs the extracted extraction knocking sound 91a (extraction signal) to the threshold value determination unit 32.

In the determination mode, the threshold value determination unit 32 determines the presence or absence of knocking by determining the threshold value stored in the threshold value storage unit 25 and the knocking sound (extracted signal) extracted by the second estimation unit 31. Specifically, the threshold value determination unit 32 sums the absolute values of the spectrograms of the knocking sounds (extracted signals) at predetermined time intervals, as in the threshold value calculation unit 24. Then, the threshold value determination unit 32 compares the totaled knocking sound (extracted signal) with the threshold value, and if the totaled knocking sound (extracted signal) exceeds the threshold value, determines that there is knocking, and the totaled knocking sound (extracted). If the signal) is less than or equal to the threshold value, it is determined that there is no knocking. After that, the threshold value determination unit 32 outputs the knocking presence / absence determination result and the knocking sound (extracted signal) input from the second estimation unit 31 to the monitor 7 (FIG. 1).

<Separation part>
The separation unit 40 separates the input physical quantity into a noise component and an extraction signal. In the present embodiment, as shown in FIG. 6A, the separation unit 40 separates the engine vicinity sound 90 as an input physical quantity into a noise 91b which is a noise component and an extraction knocking sound 91a which is an extraction signal.

<Signal synthesizer>
The signal synthesis unit 50 synthesizes various signals to generate a processed sound. In the present embodiment, as shown in FIGS. 11A to 12C, the signal synthesizer 50 changes the level of the extraction signal (extraction knocking sound 91a) to change the noise component separated from the input physical quantity (engine proximity sound 90). A processed sound is generated by combining with (noise 91b). The signal synthesis unit 50 includes a signal adjustment unit 51 and a signal output unit 52. The signal adjusting unit 51 changes the signal level according to the rising level or falling level of the signal designated (input) by the level designating unit 9, and combines the level changing signal and the noise component to generate a processed sound. .. The signal output unit 52 outputs the processed sound (signal) to the sound emitting unit (headphones 8), and causes the sound emitting unit to emit the processed sound.

<Operation in learning mode>
Hereinafter, the operation of the signal processing device 10 in the learning mode will be described with reference to FIGS. 3A, 3B, 8A, 8B, 8C, and 9. FIG. 3A is an explanatory diagram of the learning mode. FIG. 3B is an operation explanatory view of the signal processing device 10 in the learning mode. FIG. 8A is an explanatory diagram showing the relationship between the noise component and the teacher signal during learning. FIG. 8B is an explanatory diagram showing the relationship between the extracted signal and the teacher signal during learning. FIG. 8C is an explanatory diagram showing the relationship between the extracted signal and the estimated signal during learning. FIG. 9 is an explanatory diagram of the neural network 94 that learns the weight W of the network that generates the mask α in the first embodiment.

As shown in FIG. 3A, in the learning mode, the learning unit 21 of the signal processing device 10 separates the engine vicinity sound 90 (input physical quantity) into the noise 91b (noise component) and the extraction knocking sound 91a (extraction signal). .. At that time, the learning unit 21 acquires the extraction knocking sound 91a (extraction signal) by multiplying the engine proximity sound 90 (input physical quantity) by the mask α. Further, the learning unit 21 acquires the noise 91b (noise component) by subtracting the extraction knocking sound 91a (extraction signal) from the engine vicinity sound 90 (input physical quantity). In the present embodiment, the mask α represents the ratio of the knocking sound included in the engine near sound 90 (input physical quantity) and the phase component (phase correction amount). The phase component will be described later with reference to FIGS. 10A and 10B.

The learning unit 21 of the signal processing device 10 observes the weight W of the network that generates the mask α and the extracted knocking sound 91a (extracted signal). The estimated knocking cylinder having the same dimension (unit) as the knocking cylinder internal pressure 93 (teacher signal). The transfer function H for converting the internal pressure 92 (estimated signal) by adding the phase is learned. In other words, when learning the network weight W and the transfer function H, the learning unit 21 has an amplitude and a phase related to the input physical quantity (engine near sound 90) with respect to the mask α and the transfer function H. Learn by adding ingredients. In the present embodiment, the learning unit 21 performs an inverse short-time Fourier transform (ISTFT) and a fast Fourier transform (FFT) on the extraction knocking sound 91a (extract signal), multiplies the transfer function H, and reverse-fast Fourier transform. By performing the transform (IFFT) and the short-time Fourier transform (SFTFT), the extracted knocking sound 91a (extracted signal) is converted into the estimated knocking in-cylinder pressure 92 (estimated signal). In the present embodiment, the transfer function H is an amplitude (gain) and a phase component for converting the extracted knocking sound 91a (extracted signal) into the estimated knocking cylinder pressure 92 (estimated signal). The phase component will be described later with reference to FIGS. 10A and 10B.

In the present embodiment, as shown in FIG. 3A, the learning unit 21 combines a signal waveform obtained by inverse short-time Fourier transform (ISTFT) of a noise component (noise 91b) and a teacher signal (observation knocking cylinder pressure 93). Mask α so that the coherence becomes smaller and the coherence between the signal waveform obtained by inverse short-time Fourier transform (ISTFT) of the extracted signal (extracted knocking sound 91a) and the teacher signal (observed knocking cylinder pressure 93) becomes larger. The weight W of the network that generates the above and the transfer function H are learned. As a result, the signal processing device 10 can remove the sound caused by the in-cylinder pressure from the noise 91b (noise component).

FIG. 3B shows the operation of the signal processing device 10 in the learning mode. The thick frame in FIG. 3B shows the operating components in the learning mode. The thick arrow in FIG. 3B indicates a signal output in the learning mode.

As shown in FIG. 3B, in the learning mode, the signal processing device 10 connects the switch 14 to the learning mode connection unit M1 to generate the engine proximity sound 90 (input physical quantity) stored in the signal storage unit 13. The spectrogram and the observation knocking cylinder pressure 93 (teacher signal) are output to the learning unit 21.

In response to this, the learning unit 21 generates a network weight W that generates a mask α that removes the noise 91b (noise component) from the engine near sound 90 (input physical quantity) by the neural network 94 (FIG. 3A). The transmission function H that converts the extracted knocking sound 91a (extracted signal) extracted by the mask α into the estimated knocking cylinder pressure 92 (estimated signal) of the engine 1 when knocking occurs is learned. At this time, the learning unit 21 generates a mask α corresponding to the engine near sound 90 (input physical quantity) every time a new engine near sound 90 (input physical quantity) is input.

Then, the learning unit 21 stores the learned parameters (the weight W of the network that generates the mask α and the transfer function H) in the learned parameter storage unit 22. Further, the learning unit 21 stores the observation knocking cylinder internal pressure 93 (teacher signal) in the teacher signal storage unit 26a, and stores the estimated knocking cylinder internal pressure 92 (estimated signal) in the estimated signal storage unit 26b.

<Operation in threshold calculation mode>
Hereinafter, the operation of the signal processing device 10 in the threshold value calculation mode will be described with reference to FIGS. 4A and 4B. FIG. 4A is an explanatory diagram of the threshold value calculation mode. FIG. 4B is an operation explanatory diagram of the signal processing device 10 in the threshold value calculation mode.

As shown in FIG. 4B, in the threshold value calculation mode, unlike the learning mode in FIG. 3B, the learning unit 21 of the signal processing device 10 is in a stopped state. Instead, as shown in FIG. 4B, the first estimation unit 23 and the threshold value calculation unit 24 of the learning processing unit 20 operate to calculate the threshold value T used for determining the threshold value of the extraction knocking sound (extraction signal) of the engine 1. do. The thick frame in FIG. 4B shows the operating components in the threshold calculation mode. The thick arrow in FIG. 4B indicates a signal output in the threshold value calculation mode.

As shown in FIG. 4B, in the threshold value calculation mode, the signal processing device 10 connects the switch 14 to the threshold value calculation mode connection unit M2, and the engine proximity sound 90 (input physical quantity) stored in the signal storage unit 13. ) Is output to the first estimation unit 23.

In response to this, the first estimation unit 23 acquires the weight W of the network that generates the mask α from the learned parameter storage unit 22. Then, the first estimation unit 23 extracts the extraction knocking sound 91a (extraction signal) of the engine 1 from the engine proximity sound 90 (input physical quantity) by using the mask α generated by the neural network 94 (FIG. 4A). FIG. 4A shows an outline of the operation of the first estimation unit 23 at this time. After that, the first estimation unit 23 outputs the extracted extraction knocking sound 91a (extraction signal) to the threshold value calculation unit 24.

In response to this, the threshold value calculation unit 24 calculates the threshold value T by summing the absolute values of the spectrograms of the extraction knocking sound 91a (extraction signal) in each cycle of the engine 1 and adding a preset margin. Then, the threshold value calculation unit 24 stores the threshold value T in the threshold value storage unit 25.

<Operation in judgment mode>
Hereinafter, the operation of the signal processing device 10 in the determination mode will be described with reference to FIGS. 5A and 5B. FIG. 5A is an explanatory diagram of the determination mode. FIG. 5B is an operation explanatory view of the signal processing device 10 in the determination mode.

As shown in FIG. 5B, in the determination mode, unlike the learning mode in FIG. 3B, the learning unit 21 of the signal processing device 10 is in a stopped state. Instead, as shown in FIG. 5B, the second estimation unit 31 (extraction signal estimation unit) and the threshold value determination unit 32 of the determination processing unit 30 operate to determine the presence or absence of knocking. The thick frame in FIG. 5B shows the operating components in the determination mode. The thick arrow in FIG. 5B indicates a signal output in the determination mode.

As shown in FIG. 5B, in the determination mode, the signal processing device 10 connects the switch 14 to the determination mode connection unit M3, and the engine proximity sound 90 (input physical quantity) stored in the signal storage unit 13 The spectrogram is output to the second estimation unit 31 (extraction signal estimation unit).

In response to this, the second estimation unit 31 acquires the weight W of the network that generates the mask α from the learned parameter storage unit 22. Then, the second estimation unit 31 extracts the extracted knocking sound 91a (extracted signal) from the engine vicinity sound 90 (input physical quantity) by using the mask α generated by the neural network 94 (FIG. 5A). FIG. 5A shows an outline of the operation of the second estimation unit 31 at this time. After that, the second estimation unit 31 outputs the extracted extraction knocking sound 91a (extraction signal) to the threshold value determination unit 32.

In response to this, the threshold value determination unit 32 acquires the threshold value T from the threshold value storage unit 25, compares the sum of the absolute values of the extracted knocking sounds 91a (extracted signal) with the threshold value T, and determines the presence or absence of knocking. Then, the threshold value determination unit 32 outputs, for example, a determination result of the presence or absence of knocking, a waveform diagram showing the relationship between the extraction knocking sound 91a (extraction signal) and the threshold value T, and the like to the monitor 7 and displays them.

<Operation in separation mode>
Hereinafter, the operation of the signal processing device 10 in the separation mode will be described with reference to FIGS. 6A and 6B. FIG. 6A is an explanatory diagram of the separation mode. FIG. 6B is an operation explanatory view of the signal processing device 10 in the separation mode.

As shown in FIG. 6B, in the separation mode, unlike the learning mode in FIG. 3B, the learning unit 21 of the signal processing device 10 is in a stopped state. Instead, as shown in FIG. 6B, the separation unit 40 operates to separate the engine vicinity sound 90 (input physical quantity) into the extraction knocking sound 91a (extraction signal) and the noise 91b (noise component). The thick frame in FIG. 6B shows the operating components in the separation mode. The thick arrow in FIG. 6B indicates a signal output in the separation mode.

As shown in FIG. 6B, in the separation mode, the signal processing device 10 connects the switch 14 to the separation mode connection unit M4, and the engine proximity sound 90 (input physical quantity) stored in the signal storage unit 13. The spectrogram is output to the separation unit 40.

In response to this, the separation unit 40 acquires the weight W of the network that generates the mask α from the learned parameter storage unit 22. Then, the separation unit 40 separates the engine vicinity sound 90 (input physical quantity) into the extraction knocking sound 91a (extraction signal) and the noise 91b (noise component) by using the mask α generated by the neural network 94 (FIG. 6A). do. FIG. 6A shows an outline of the operation of the separation unit 40 at this time. After that, the separation unit 40 stores the extraction knocking sound 91a (extraction signal) in the extraction signal storage unit 26d, and stores the noise 91b (noise component) in the noise component storage unit 26c.

<Operation in sensory test mode>
Hereinafter, the operation of the signal processing device 10 in the sensory test mode will be described with reference to FIGS. 7A and 7B. FIG. 7A is an explanatory diagram of the sensory test mode. FIG. 7B is an operation explanatory view of the signal processing device 10 in the sensory test mode. The sensory test mode is a mode for calculating the threshold value T based on the auditory sense of the examiner.

As shown in FIG. 7B, in the sensory test mode, unlike the learning mode in FIG. 3B, the learning unit 21 of the signal processing device 10 is in a stopped state. Instead, as shown in FIG. 7B, the signal adjustment unit 51 and the signal output unit 52 of the signal synthesis unit 50, and the first estimation unit 23 and the threshold value calculation unit 24 of the learning processing unit 20 are activated to operate the inspector. The threshold value T based on the audibility of is calculated. The thick frame in FIG. 7B shows the operating components in the sensory test mode. The thick arrow in FIG. 7B indicates a signal output in the sensory test mode.

As shown in FIG. 7B, in the sensory test mode, the signal processing device 10 outputs the execution instruction signal of the sensory test mode to the signal adjusting unit 51 by connecting the switch 14 to the sensory test mode connection unit M5. ..

In response to this, the signal adjusting unit 51 receives the level designation information from the level designation unit 9, and stores the extraction knocking sound 91a (extract signal) and the noise component storage unit 26c stored in the extraction signal storage unit 26d. The noise 91b (noise component) that has been generated is acquired. Then, the signal adjusting unit 51 generates the processed sound 91c by using the extracted knocking sound 91a (extracted signal) and the noise 91b (noise component) based on the level designation information. At this time, the signal adjusting unit 51 raises or lowers the level (magnitude) of the extracted knocking sound 91a (extracted signal) by the amount specified by the level designation information, and then synthesizes it with the noise 91b (noise component). Generates the processing sound 91c and outputs it to the signal output unit 52. The signal output unit 52 outputs the processed sound 91c to the headphones 8 (sound emitting unit) to emit the processed sound 91c.

Further, the signal adjusting unit 51 outputs the processed sound 91c to the first estimation unit 23. The first estimation unit 23 acquires the weight W of the network that generates the mask α from the learned parameter storage unit 22. Then, the first estimation unit 23 extracts the extraction knocking sound 91a (extraction signal) of the engine 1 from the processing sound 91c by using the mask α generated by the neural network 94 (FIG. 7A). FIG. 7A shows an outline of the operation of the first estimation unit 23 at this time. After that, the first estimation unit 23 outputs the extracted extraction knocking sound 91a (extraction signal) to the threshold value calculation unit 24. The threshold value calculation unit 24 sums the absolute values of the spectrograms of the extracted knocking sound 91a (extracted signal) in each cycle of the engine 1, and sets an arbitrary value equal to or less than the summed value as the threshold value T. Then, the threshold value calculation unit 24 stores the threshold value T in the threshold value storage unit 25.

By the way, in the prior art described in Patent Document 2 and Patent Document 3, since the objective function at the time of learning the neural network does not include the function for measuring the degree of sound separation, it is removed from the sound near the engine (input physical quantity). There was a possibility that the knocking sound (target sound) was mixed in the noise (sound other than the knocking sound (background sound)). That is, since the prior art described in Patent Document 2 and Patent Document 3 only minimizes the square error between the estimated in-cylinder pressure and the actually measured in-cylinder pressure (teacher data) at the time of learning, "noise has been removed. It was not monitored whether or not the "engine sound" was the one in which only noise (sound other than knocking sound (background sound)) was satisfactorily removed. For example, in the prior art described in Patent Document 3, "engine sound from which noise has been removed" is obtained by applying a mask α that does not consider the phase component related to engine near sound (input physical quantity) to engine near sound. That is, the knocking sound (corresponding to the "extract signal" of the present embodiment) is extracted. At that time, since the conventional technique described in Patent Document 3 does not use a function for measuring the degree of sound separation, the knocking sound (sound other than the knocking sound (background sound)) and the knocking sound that should not be removed (the knocking sound (background sound)) There was a possibility that the target sound) would be removed from the sound near the engine. Therefore, the prior art described in Patent Document 3 has a possibility of removing a knocking sound component that should not be removed from the sound near the engine. Therefore, the prior art described in Patent Document 3 may reduce the evaluation performance of the presence or absence of knocking.

On the other hand, in the signal processing device 10 according to the present embodiment, the knocking sound (target sound) is included in the noise (sound other than the knocking sound (background sound)) removed from the sound near the engine (input physical quantity) during learning. ) Is mixed in or not. As a configuration for that purpose, the signal processing device 10 according to the present embodiment has coherence between the signal waveform obtained by inverse short-time Fourier transform (ISTFT) of the noise component (noise 91b) and the teacher signal (observation knocking cylinder pressure 93). It is configured to learn so that Such a signal processing device 10 according to the present embodiment can improve the evaluation performance of the presence or absence of knocking as compared with the conventional techniques described in Patent Documents 2 and 3.

FIG. 8A shows the spectrogram F11 of the noise 91b (noise component), the signal waveform F12 obtained by performing the inverse short-time Fourier transform (ISTFT) on the spectrogram F11, and the signal of the observation knocking cylinder pressure 93 (teacher signal). The waveform F93 and the like are shown. Further, FIG. 8A shows the coherence F13 minimized in the process of learning between the signal waveform F12 and the signal waveform F93 of the observation knocking cylinder pressure 93 (teacher signal).

FIG. 8B shows the spectrogram F21 of the extracted knocking sound 91a (extracted signal), the signal waveform F22 obtained by performing an inverse short-time Fourier transform (ISTFT) on the spectrogram F21, and the observed knocking in-cylinder pressure 93 (teacher signal). The signal waveform F93 of is shown. Further, FIG. 8B shows the coherence F23 maximized in the process of learning between the signal waveform F22 and the signal waveform F93 of the observation knocking cylinder pressure 93 (teacher signal).

Coherence is defined by the following coherence function γ ^2. The coherence function γ ² indicates the degree of association between the input and output of the system. The coherence function γ ² is obtained by dividing the square of the absolute value of the cross spectrum by the power spectra of the measurement input and the output of the system.

Here, Wxy means the average value of the cross spectrum, Wxx means the average value of the power spectrum of x, and Wyy means the average value of the power spectrum of y. The coherence function γ ² takes a value between 0 and 1. When γ ² (f) is 1, it indicates that all the outputs of the system are caused by the measurement input at the frequency f. Further, when γ ² (f) is 0, it indicates that the output of the system has nothing to do with the measurement input at the frequency f. Further, when 0 <γ ² (f) <1, it is considered that there is a signal unrelated to the measurement, noise generated inside the system, non-linearity of the system, and the like.

In addition, coherent output power is a method for evaluating whether or not a knocking sound (target sound) is mixed in the noise (sound other than the knocking sound (background sound)) removed from the sound near the engine (input physical quantity). May be used. The learning unit 21 reduces the coherence and / or coherent output power between the signal waveform obtained by inverse short-time Fourier transform (ISTFT) of the noise component (noise 91b) and the teacher signal (observation knocking cylinder internal pressure 93). , The coherence and / or coherent output power of the signal waveform obtained by inverse short-time Fourier transform (ISTFT) of the extracted signal (extracted knocking sound 91a) and the teacher signal (observation knocking cylinder internal pressure 93) is increased. The weight W of the network that generates the mask α and the transfer function H are learned. As a result, the signal processing device 10 can further improve the evaluation performance of the presence or absence of knocking.

The coherent output power is ^{defined by the product of the coherence function γ 2} and the average value Wyy of the power spectrum of y. The coherent output power indicates the power caused by the input included in the output of the system.

Further, as shown in FIG. 3A, the learning unit 21 transfers the spectrogram obtained by performing a short-time Fourier transform (STFT) on the observed knocking cylinder internal pressure 93 (teacher signal) and the estimated knocking cylinder internal pressure 92 (estimated signal). The neural network 94 learns the weight W of the network that generates the mask α and the transfer function H so that the error is minimized. As a result, the signal processing device 10 can improve the learning accuracy of the weight W of the network that generates the mask α and the transfer function H.

Further, the learning unit 21 performs inverse short-time Fourier transform (ISTFT) and fast Fourier transform (FFT) on the observation knocking cylinder internal pressure 93 (teacher signal) and the extraction signal (extraction knocking sound 91a) and transmits them. A network that generates a mask α by the neural network 94 so that the error from the signal waveform of the estimated knocking in-cylinder pressure 92 (estimated signal) obtained by multiplying the function H and performing the inverse fast Fourier transform (IFFT) is minimized. The weight W and the transfer function H may be learned. As a result, the signal processing device 10 can further improve the learning accuracy of the weight W of the network that generates the mask α and the transfer function H.

FIG. 8C shows an example of converting the extracted knocking sound 91a (extracted signal) into the estimated knocking cylinder pressure 92 (estimated signal). FIG. 8C shows the spectrogram F31 of the extraction knocking sound 91a (extract signal), the signal waveform F32 obtained by performing the inverse short-time Fourier transform (ISTFT) on the spectrogram F31, and further performing the fast Fourier transform (FFT). The spectrum F33 obtained by the above is shown. Further, FIG. 8C shows the spectrum F35 of the estimated knocking cylinder pressure obtained by multiplying the signal of the spectrum F33 by the transfer function H. In FIG. 8C, as an example of the transfer function H, the frequency response characteristic F34 showing the correspondence between the frequency and the coefficient is shown. Further, FIG. 8C shows a signal waveform F36 obtained by performing an inverse fast Fourier transform (IFFT) on the spectrum F35, and an estimated knocking in-cylinder pressure 92 (estimated) obtained by further performing a short-time Fourier transform (STFT). Signal) spectrogram F37.

In the present embodiment, the learning unit 21 learns the weight W of the network that generates the mask α by the convolutional neural network (CNN). Further, the learning unit 21 learns the weight to be multiplied by the spectrum F33 of the extracted signal as the transfer function H.

As shown in FIG. 9, U-Net95 is an example of a neural network 94 that learns the weight W of the network that generates the mask α. This U-Net95 is a kind of Encoder-Decoder model, and is a deep learning method used for image recognition and sound separation. In sound separation, in U-Net95, convolution (stride is 2 or more) is performed with a downward path 96 (Encoder), and sound characteristics are extracted as the layer 97 becomes deeper. On the other hand, in the upward path 98 (Decoder), the mask α is generated by performing deconvolution and UP sampling (expansion) from the characteristics of the extracted sound. Up to this point, it is the configuration of a general Encoder-Decoder model. Further, in U-Net95, the output 99 from the convolution layer of each Encoder is merged with the convolution layer of the Decoder. As a result, the U-Net95 can generate a mask α with higher accuracy than the general Encoder-Decoder model. The signal processing device 10 can acquire an extraction signal (extraction knocking sound 91a) by generating a mask α and multiplying it by a sound near the engine 90.

In this way, since the learning unit 21 learns the weight W of the network that generates the mask α in the U-Net 95, the learned network generates an appropriate mask α according to the input near sound of the engine 1. Become. As a result, the signal processing device 10 can accurately estimate the knocking cylinder internal pressure of the engine 1.

<Effect of phase component>
Hereinafter, the influence of the phase component will be described with reference to FIGS. 10A and 10B. FIG. 10A is an explanatory diagram of a calculation example when phase components such as pressure, vibration, and sound are not taken into consideration. FIG. 10B is an explanatory diagram of a calculation example when phase components such as pressure, vibration, and sound are taken into consideration. In FIGS. 10A and 10B, the arrows in the circle represent phase components contained in pressure, vibration, sound, and the like.

In both the examples shown in FIGS. 10A and 10B, the following conditions are assumed.
-The system is a linear time-invariant system (a system in which the transfer characteristics between the input and output are linear and do not change with time).

In the example shown in FIG. 10A, when the engine 1 is driven, the observation knocking in-cylinder pressure 93 is observed by the in-cylinder pressure sensor 5 (FIG. 1). The knocking sound 82 is emitted by the observation knocking cylinder internal pressure 93 propagating in the engine 1. Further, when the knocking sound 82 is combined with the mechanical noise 83 (noise component) which is the background sound, the sound pressure sensor 4 observes the engine vicinity sound 90 (input physical quantity).

FIG. 10A shows an example of a schematic calculation of the observed knocking cylinder internal pressure 93, the knocking vibration 81, the knocking sound 82, the mechanical noise 83, and the engine vicinity sound 90 as shown below, and the phase φ does not change. I'm assuming.
Observation knocking cylinder pressure 93 = ASin (Ωt + φ)
Knocking vibration 81 = ABSin (Ωt + φ)
Knocking sound 82 = ABCSin (Ωt + φ)
Mechanical noise 83 = NSin (Ωt + φN)
Engine neighborhood sound 90 = (ABC + N) Sin (Ωt + φ)

On the other hand, FIG. 10B is an example in which the phase is taken into consideration. As shown below, an example of a schematic calculation of the observed knocking cylinder internal pressure 93, knocking vibration 81, knocking sound 82, mechanical noise 83, and engine proximity sound 90 is shown, and the phase changes each time the dimension (unit) changes. Considering that φ changes.
Observation knocking cylinder pressure 93 = ASin (Ωt + φA)
Knocking vibration 81 = ABSin (Ωt + φA + φB)
Knocking sound 82 = ABCSin (Ωt + φA + φB + φC)
Mechanical noise 83 = NSin (Ωt + φN)
Engine neighborhood sound 90 = ABCSin (Ωt + φA + φB + φC) + NSin (Ωt + φN)

In the example shown in FIG. 10B, since the change in the phase component when the pressure becomes vibration and is radiated as sound is taken into consideration, the signal can be analyzed better than the example shown in FIG. 10A. Therefore, as shown in FIG. 3A, the signal processing device 10 has a configuration in which the weight W of the network that generates the mask α and the transfer function H are learned in consideration of the phase. Such a signal processing device 10 can satisfactorily separate the engine vicinity sound 90 (input physical quantity) into the extraction knocking sound 91a (extraction signal) and the noise 91b (noise component). That is, the signal processing device 10 extracts the engine vicinity sound 90 (input physical quantity) so that the knocking sound (target sound) is not mixed in the noise (sound other than the knocking sound (background sound)). Knocking sound 91a (extracted). It can be separated into a signal) and a noise 91b (noise component). Such a signal processing device 10 can improve the evaluation performance of the presence or absence of knocking as compared with the conventional techniques described in Patent Documents 2 and 3. In addition, such a signal processing device 10 can perform a good sensory test.

(Outline of sensory test)
Hereinafter, the outline of the sensory test will be described with reference to FIGS. 11A to 12C. 11A to 12C are explanatory views of signal processing in the sensory test, respectively.

As shown in FIG. 2, each of the pressure and sound signals is collected by the data collecting device 6 and output to the signal processing device 10. The signal processing device 10 cuts out each signal by the signal cutting unit 11, calculates the spectrogram by the spectrogram calculation unit 12, and stores the spectrogram in the signal storage unit 13.

The signal processing device 10 according to the present embodiment can execute the above-mentioned separation mode and the above-mentioned sensory test mode. In the separation mode, the signal processing device 10 separates the input physical quantity (engine proximity sound 90) into an extraction signal (extraction knocking sound 91a) and a noise component (noise 91b). Then, in the sensory test mode, the signal processing device 10 changes the level of the extraction signal (extraction knocking sound 91a) to generate a level change extraction signal (level change knocking sound 91aa (FIG. 12A)), and generates a noise component (noise). The processing sound 91c (FIG. 12C) is generated by combining with 91b).

In the sensory test mode, the inspector wears the headphones 8 (FIG. 1) on the head and stands by in a state in which the level designation unit 9 can be operated. Then, the operator declares an acceptable level to the surroundings, operates the level designation unit 9, and inputs the level designation information to the signal processing device 10. The signal processing device 10 changes the level of the extraction signal (extraction knocking sound 91a) based on the input level designation information.

FIG. 11A shows an input physical quantity (engine proximity sound 90), and FIGS. 11B and 11C show an extraction signal (extraction knocking sound 91a) and a noise component separated from the input physical quantity (engine proximity sound 90) in the separation mode. (Noise 91b). FIG. 12A shows a level change extraction signal (level change knocking sound 91aa) generated by changing the level of the extraction signal (extraction knocking sound 91a) by 3db in the sensory test mode, and FIG. 12B shows. The noise component (noise 91b) synthesized with the level change extraction signal (level change knocking sound 91aa) is shown, and FIG. 12C shows the level change extraction signal (level change knocking sound 91aa) and the noise component (noise 91b). The synthesized processing sound 91c is shown. By generating the processing sound 91c, the signal processing device 10 can make it easy to distinguish the target sound (knocking sound) to be inspected. Such a signal processing device 10 can make the inspector grasp the presence or absence of the target sound (knocking sound) with high accuracy, and can improve the inspection performance. Further, the signal processing device 10 obtains the sum of the absolute values of the spectrograms of the processed sound 91c determined to be unacceptable by the inspector, and writes an arbitrary value equal to or less than the sum of values into the threshold storage unit 25. As a result, the threshold value T becomes a value suitable for the inspector's sensuality.

The signal processing device 10 multiplies the level change extraction signal (level change knocking sound 91aa) by the transfer function He (not shown) from the measurement target to the listener's ear position in the sensory test mode, depending on the operation. Then, the noise component at the ear position of the listener may be synthesized to generate the processed sound.

<Operation of signal processing device (estimating device)>
Hereinafter, the operation of the signal processing device 10 will be described with reference to FIGS. 13 to 18. FIG. 13 is a flowchart showing a data collection process of the signal processing device 10 (estimating device). FIG. 14A is a flowchart showing the learning process of the signal processing device 10 (estimating device). FIG. 14B is a flowchart showing a subroutine of learning processing. FIG. 14C is a flowchart showing a modification example of the subroutine of the learning process. FIG. 15A is a flowchart showing a threshold value calculation process of the signal processing device 10 (estimating device). FIG. 15B is a flowchart showing a threshold value calculation process performed after the separation process of FIG. 17 and the sensory test process of FIG. 18 of the signal processing device 10 (estimating device). FIG. 16 is a flowchart showing a determination process of the signal processing device 10 (estimating device). FIG. 17 is a flowchart showing the separation process of the signal processing device 10 (estimating device). FIG. 18 is a flowchart showing a sensory test process of the signal processing device 10 (estimating device).

In the learning mode, the signal processing device 10 performs the learning process of FIGS. 14A and 14B after performing the data collection process of FIG. 13. Further, in the threshold value calculation mode, the signal processing device 10 performs the threshold value calculation process of FIG. 15A after performing the data collection process of FIG. 13. Further, when the separation process is performed in the separation mode and the sensory test process is performed in the sensory test mode, the signal processing device 10 performs the threshold value calculation process of FIG. 15B after the separation process and the sensory test process. Further, in the determination mode, the signal processing device 10 performs the determination process of FIG. 16 after performing the data collection process of FIG. 13. However, the signal processing device 10 may perform the determination process of FIG. 16 in real time while performing the data collection process of FIG. 13. Further, in the case of the separation mode, the signal processing device 10 performs the data collection process of FIG. 13 and then the separation process of FIG. Further, in the sensory test mode, the signal processing device 10 performs the sensory test process of FIG. 18 before performing the threshold value calculation process of FIG. 15B.

(Data collection process)
Hereinafter, the data collection process will be described with reference to FIG.
As shown in FIG. 13, in step S20, the data collection device 6 inputs the sound pressure signal, the in-cylinder pressure signal (teacher signal), and the angle information to the signal cutting unit 11. In the case of the separation mode, the threshold value calculation mode, or the determination mode, it is not necessary to input the in-cylinder pressure signal in step S20.
In step S21, the signal cutting unit 11 calculates the combustion stroke timing of each cylinder from the angle information.

In step S22, the signal cutting unit 11 cuts out the sound pressure signal and the in-cylinder pressure signal in the vicinity of the TDC in accordance with the combustion stroke timing calculated in step S21.
In step S23, the spectrogram calculation unit 12 performs a short-time Fourier transform (STFT) on the sound pressure signal cut out in step S22 to calculate the spectrogram of the sound pressure signal.
In step S24, the spectrogram calculation unit 12 writes the spectrogram of the sound pressure signal into the signal storage unit 13. Further, the signal cutting unit 11 writes the in-cylinder pressure signal (teacher signal) to the signal storage unit 13.

(Learning process)
Hereinafter, the learning process executed in the learning mode will be described with reference to FIGS. 14A to 14C.
As shown in FIG. 14A, in step S30, the signal processing device 10 reads the spectrogram of the sound pressure signal and the in-cylinder pressure signal from the signal storage unit 13 and inputs them to the learning unit 21.
In step S31, the learning unit 21 uses the spectrum of the sound pressure signal and the in-cylinder pressure signal input in step S30 to generate the mask α by adding the amplitude and phase components related to the input physical quantity in the neural network 94. The weight W of the network to be used and the transfer function H are learned. The learning unit 21 learns the weight W of the network that generates the mask α by the convolutional neural network (for example, U-Net), and learns the weight to be multiplied by the spectrum of the extracted signal as the transfer function H.

In step S32, the learning unit 21 writes the weight W and the transfer function H of the network that generates the learned mask α into the learned parameter storage unit 22.

The process of step S31 is performed by, for example, a series of routine processes shown in FIG. 14B.
As shown in FIG. 14B, in step S101, the learning unit 21 uses a mask generated by inputting an input physical quantity into a convolutional neural network whose weight W is initialized with a random number in advance to convert the input physical quantity into a noise component and an extraction signal. Separate into.

In step S102, the learning unit 21 multiplies the transfer function H to convert the extracted signal (extracted knocking sound 91a) into an estimated signal (estimated knocking in-cylinder pressure 92).
In step S103, the learning unit 21 updates the weight W of the network that generates the mask α and the transfer function H. In step S103, the learning unit 21 reduces the coherence between the signal waveform of the noise component (noise 91b) obtained by the inverse short-time Fourier transform (ISTFT) and the teacher signal (observation knocking cylinder internal pressure 93), and also reduces the inverse short-time Fourier transform (ISTFT). To increase the coherence between the signal waveform of the extracted signal (extracted knocking sound 91a) obtained by Fourier transform (ISTFT) and the teacher signal (observed knocking cylinder pressure 93), and to increase the coherence with the estimated signal (estimated knocking cylinder pressure 92). The weight W of the network that generates the mask α and the transfer function H are updated so that the error from the spectrum obtained by performing short-time Fourier transform (STFT) on the teacher signal (observation knocking cylinder pressure 93) becomes small. do.

In step S104, the learning unit 21 determines whether or not the coherence and the error have converged, and if it is determined that the coherence and the error have converged (in the case of “Yes”), the process of step S31 is terminated, while the learning unit 21 converges. If it is determined that the case is not present (in the case of “No”), the processing of step S101 and subsequent steps is repeated. In step S104, the state in which the coherence and the error have converged is the coherence between the teacher signal (observation knocking cylinder internal pressure 93) and the signal waveform obtained by inverse short-time Fourier transform (ISTFT) of the noise component (noise 91b). Is a value close to the value "0", and the coherence between the teacher signal (observation knocking cylinder pressure 93) and the signal waveform obtained by inverse short-time Fourier transform (ISTFT) of the extraction signal (extraction knocking sound 91a) is ". The error of the estimated signal (estimated knocking in-cylinder pressure 92) with respect to the spectrogram obtained by performing short-time Fourier transform (STFT) on the teacher signal (observed knocking in-cylinder pressure 93) with a value close to "1" becomes "0". It is in a state of converging to a close value.

The process of step S31 may be performed, for example, as shown in FIG. 14C, instead of FIG. 14B.
The process shown in FIG. 14C is different from the process shown in FIG. 14B in that the process of step S103a is performed instead of the process of step S103.
In step S103a, in addition to the conditions of step S103, the learning unit 21 further performs an inverse short-time Fourier transform (ISTFT) on the estimated signal (estimated knocking cylinder pressure 92) to obtain the maximum value of the signal waveform and the maximum value of the signal waveform. The difference between the maximum value and the minimum value (knocking intensity), the maximum value of the teacher signal (observed knocking cylinder internal pressure 93), and the difference between the maximum value and the minimum value (knocking intensity) are calculated, and the former maximum value is calculated. The error between the value and the maximum value of the latter, and the difference between the maximum value and the minimum value of the former (knocking intensity) and the difference between the maximum value and the minimum value of the latter (knocking intensity) are reduced. The weight W of the network that generates the mask α and the transfer function H are updated so as to satisfy the condition of. That is, in step S103a, the learning unit 21 reduces the coherence between the signal waveform of the noise component (noise 91b) obtained by the inverse short-time Fourier transform (ISTFT) and the teacher signal (observation knocking cylinder internal pressure 93), and reverses it. The coherence between the signal waveform of the extracted signal (extracted knocking sound 91a) obtained by the short-time Fourier transform (ISTFT) and the teacher signal (observed knocking cylinder pressure 93) is increased, and the estimated signal (estimated knocking cylinder pressure 92) is increased. ) And the spectrogram obtained by performing a short-time Fourier transform (STFT) on the teacher signal (observed knocking cylinder pressure 93), and further inversely with respect to the estimated signal (estimated knocking cylinder pressure 92). The maximum value of the signal waveform obtained by short-time Fourier transform (ISTFT), the difference between the maximum value and the minimum value (knocking intensity), the maximum value of the teacher signal (observed knocking cylinder pressure 93), and the maximum value. The difference between the value and the minimum value (knocking intensity) is calculated, the error between the former maximum value and the latter maximum value, and the difference between the former maximum value and the minimum value (knocking intensity) and the latter maximum. The weight W of the network that generates the mask α and the transfer function H are updated so that the error in the difference between the value and the minimum value (knocking intensity) becomes small.

(Threshold calculation process)
Hereinafter, the threshold value calculation process executed in the threshold value calculation mode will be described with reference to FIG. 15A.
As shown in FIG. 15A, in step S40, the first estimation unit 23 extracts the knocking sound (extract signal) of the engine 1 from the spectrogram of the sound pressure signal by using the mask α generated by the neural network 94.
In step S41, the threshold value calculation unit 24 sums the absolute values of the spectrograms of the knocking sounds (extracted signals) in each cycle of the engine 1.

In step S42, the threshold value calculation unit 24 calculates the median value of the spectrogram of the knocking sound (extracted signal) in the entire cycle of the engine 1.
In step S43, the threshold value calculation unit 24 adds an arbitrary margin to the median value to obtain a threshold value.
In step S44, the threshold value calculation unit 24 writes the calculated threshold value in the threshold value storage unit 25.

The signal processing device 10 has a function of executing the threshold value calculation process of FIG. 15B after the separation process of FIG. 17 and the sensory test process of FIG. Hereinafter, the threshold value calculation process performed after the separation process of FIG. 17 and the sensory test process of FIG. 18 will be described with reference to FIG. 15B.
As shown in FIG. 15B, in step S140, the signal adjusting unit 51 acquires a signal that is unacceptable in the sensory test from the noise component storage unit 26c and the extraction signal storage unit 26d, and supplies the signal to the first estimation unit 23. do.
In step S141, the first estimation unit 23 acquires the spectrogram of the knocking sound (extracted signal) from the spectrogram of the signal that is unacceptable in the sensory test by using the mask α generated by the neural network 94, and the threshold calculation unit 23. Supply to 24.
In step S142, the threshold value calculation unit 24 sums the absolute values of the spectrograms of the knocking sounds (extracted signals) supplied from the first estimation unit 23.

In step S143, the threshold value calculation unit 24 sets an arbitrary value equal to or less than the total value as the threshold value.
In step S144, the threshold value calculation unit 24 writes the calculated threshold value in the threshold value storage unit 25.

(Determination process)
Hereinafter, the determination process executed in the determination mode will be described with reference to FIG.
As shown in FIG. 16, in step S50, the second estimation unit 31 uses the mask α generated by the neural network 94 to knock the engine 1 knocking sound (from the spectrogram of the sound pressure signal input from the signal storage unit 13). Extraction signal) is extracted.

In step S51, the threshold value determination unit 32 sums the absolute values of the spectrograms of the knocking sounds (extracted signals) in each cycle of the engine 1.
In step S52, the threshold value determination unit 32 compares the threshold value stored in the threshold value storage unit 25 with the total knocking sound (extracted signal), and determines whether or not the knocking sound (extracted signal) exceeds the threshold value. do.
When the total knocking sound (extracted signal) exceeds the threshold value (Yes in step S52), the threshold value determination unit 32 determines that knocking is present (step S53).

When the total knocking sound (extracted signal) is equal to or less than the threshold value (No in step S52), the threshold value determination unit 32 determines that there is no knocking (step S54).
In step S55, the threshold value determination unit 32 outputs the threshold value determination result and the knocking sound (extracted signal) to the monitor 7.

(Separation process)
Hereinafter, the separation process executed in the separation mode will be described with reference to FIG.
As shown in FIG. 17, in step S60, the separation unit 40 separates the input physical quantity into the noise component and the extraction signal by using the mask α generated by the neural network 94.
In step S61, the learning processing unit 20 stores the separated noise component and the extracted signal in the corresponding noise component storage unit 26c and the extracted signal storage unit 26d, respectively.

(Sensory test processing)
Hereinafter, the sensory test process performed in the sensory test mode will be described with reference to FIG.
As shown in FIG. 18, in step S70, the signal adjustment unit 51 of the signal synthesis unit 50 receives the input of the level designation information from the level designation unit 9.
In step S71, the signal adjusting unit 51 acquires the extracted signal from the extracted signal storage unit 26d, acquires the noise component from the noise component storage unit 26c, and changes the level of the extracted signal according to the level designation information to level. Generate a change extraction signal.

In step S72, the signal adjusting unit 51 synthesizes the level change extraction signal and the noise component to generate a processed sound.
In step S73, the signal output unit 52 of the signal synthesis unit 50 outputs the processed sound to the sound emitting unit (headphones 8), and emits the processed sound from the sound emitting unit.
In step S74, the signal synthesizing unit 50 determines whether or not there is an end instruction from the level designating unit 9, and if it is determined that the end instruction has been given (in the case of “Yes”), the processing sound is generated. 1 Output to the estimation unit 23, the threshold value is calculated by the threshold value calculation unit 24, and then the threshold value is written to the threshold value storage unit 25 to end the process of FIG. On the other hand, when it is determined that there is no end instruction (in the case of "No"), in step S75, the signal adjustment unit 51 of the signal synthesis unit 50 accepts the change of the level designation information from the level designation unit 9. The processing after step S71 is repeated.

After the separation process of FIG. 17 and the sensory test process of FIG. 18, the signal processing device 10 performs the threshold value calculation process shown in FIG. 15B.

<Main features of signal processing device (estimation device)>
(1) As shown in FIG. 3B, the signal processing device 10 (estimating device) according to the present embodiment includes a learning unit 21 and a separation unit 40. The learning unit 21 uses the neural network 94 to generate a mask α for removing the noise component by adding the phase from the input physical quantity (sound near the engine 90) including the noise component (noise 91b). Estimated signal (estimated knocking) of the same dimension (unit) as the teacher signal (observation knocking cylinder internal pressure 93) is the extracted signal (extracted knocking sound 91a) from which the noise component is removed from W and the input physical quantity (sound near the engine 90). The transfer function H for converting the in-cylinder pressure 92) by adding the phase is learned. The separation unit 40 separates the input physical quantity (engine proximity sound 90) into a noise component and an extraction signal (extraction knocking sound 91a).

The signal processing device 10 (estimation device) according to the present embodiment can satisfactorily separate the input physical quantity into the noise component and the extracted signal. In particular, the signal processing device 10 (estimating device) can separate the noise component into a signal suitable for level change in the sensory test. Such a signal processing device 10 (estimating device) can easily evaluate whether or not a target sound (knocking sound) is mixed in the background sound. Therefore, the signal processing device 10 (estimating device) can improve the evaluation performance of the presence or absence of knocking as compared with the conventional techniques described in, for example, Patent Document 2 and Patent Document 3. In addition, the signal processing device 10 can perform a good sensory test, and by determining the threshold value based on the data that is unacceptable in the sensory test, it is possible to make a judgment close to that of an inspector.

(2) As shown in FIG. 3A, the learning unit 21 of the signal processing device 10 (estimating device) according to the present embodiment learns the weight W of the network that generates the mask α and the transfer function H. Learning is performed by taking into account the amplitude and phase components related to the input physical quantity (sound near the engine 90).

The signal processing device 10 (estimation device) according to the present embodiment has a sound near the engine 90 (a sound near the engine (target sound)) so that the knocking sound (target sound) is not mixed in the noise (sound other than the knocking sound (background sound)). The input physical quantity) can be separated into an extraction knocking sound 91a (extraction signal) and a noise 91b (noise component). Such a signal processing device 10 can improve the evaluation performance of the presence or absence of knocking as compared with the conventional techniques described in Patent Documents 2 and 3. In addition, such a signal processing device 10 can perform a good sensory test.

(3) In the learning unit 21 of the signal processing device 10 (estimating device) according to the present embodiment, the relationship between the noise component (noise 91b) and the teacher signal (observation knocking cylinder internal pressure 93) becomes small, and the extraction signal (extract signal (3)) The network weight W and the transmission function H are learned so that the relationship between the extracted knocking sound 91a) and the teacher signal (observation knocking cylinder internal pressure 93) becomes large. Specifically, as shown in FIG. 3A, the learning unit 21 of the signal processing device 10 (estimation device) according to the present embodiment obtained the noise component (noise 91b) by inverse short-time Fourier transform (ISTFT). The coherence (and / or coherent output power) between the signal waveform and the teacher signal (observation knocking cylinder pressure 93) became smaller, and the extraction signal (extraction knocking sound 91a) was obtained by inverse short-time Fourier transform (ISTFT). The network weight W and the transfer function H are learned so that the coherence (and / or coherent output power) between the signal waveform and the teacher signal (observation knocking cylinder pressure 93) becomes large.

The signal processing device 10 (estimation device) according to the present embodiment can exclude the sound caused by the knocking cylinder internal pressure from the noise 91b (noise component).

(4) As shown in FIG. 3A, the learning unit 21 of the signal processing device 10 (estimating device) according to the present embodiment learns the train weight W and the transmission function H when learning the teacher signal (observation knocking). Learning is performed so that the error of the estimated signal (estimated knocking in-cylinder pressure 92) with respect to the in-cylinder pressure 93) becomes small. Specifically, the learning unit 21 of the signal processing device 10 (estimation device) performs a short-time Fourier transform (STFT) on the teacher signal (observation knocking cylinder internal pressure 93) to obtain an estimated signal (estimated knocking cylinder) for the spectrogram. Learning is performed so that the error of the internal pressure 92) becomes small. Alternatively, the learning unit 21 of the signal processing device 10 (estimating device) receives an inverse short-time Fourier transform (ISTFT) and a fast Fourier transform (ISTFT) with respect to the teacher signal (observation knocking cylinder internal pressure 93) and the extraction signal (extraction knocking sound 91a). The transformation (FFT) is performed, the transfer function H is multiplied, and the inverse fast Fourier transform (IFFT) is performed to obtain the estimated signal (estimated knocking in-cylinder pressure 92) so that the error with the signal waveform becomes small.

The signal processing device 10 (estimation device) according to the present embodiment as described above performs an estimated signal (estimated knocking cylinder pressure) for the spectrogram obtained by performing a short-time Fourier transform (STFT) on the teacher signal (observation knocking cylinder pressure 93). The network weight W and the transfer function H can be learned so that the error of 92) becomes small. Alternatively, the signal processing device 10 (estimation device) performs an inverse short-time Fourier transform (ISTFT) and a fast Fourier transform (FFT) with respect to the teacher signal (observation knocking cylinder internal pressure 93) and the extraction signal (extraction knocking sound 91a). And the network weight W and the transmission so that the error from the signal waveform of the estimated signal (estimated knocking cylinder pressure 92) obtained by performing the inverse fast Fourier transform (IFFT) by multiplying the transfer function H by The function H can be learned.

(5) As shown in FIG. 2, the signal processing device 10 (estimation device) according to the present embodiment changes the level of the extraction signal (extraction knocking sound 91a) and separates it from the input physical quantity (engine neighborhood sound 90). It is provided with a signal synthesizing unit 50 that generates a processed sound by synthesizing with the generated noise component.

The signal processing device 10 (estimating device) according to the present embodiment can make it easy to distinguish the target sound (knocking sound) to be inspected. Such a signal processing device 10 can make the inspector grasp the presence or absence of the target sound (knocking sound) with high accuracy, and determines the threshold value based on the data that is unacceptable in the sensory test to perform the inspection. You can make a judgment close to that of a person.

(6) As shown in FIG. 2, the signal synthesis unit 50 of the signal processing device 10 (estimation device) according to the present embodiment is a signal adjustment unit that receives the level designation of the extraction signal (extraction knocking sound 91a) by the operator. Has 51.

The signal processing device 10 (estimation device) according to the present embodiment can arbitrarily and finely change the level of the extraction signal (extraction knocking sound 91a).

(7) As shown in FIGS. 14A and 17, the signal processing device 10 (estimating device) according to the present embodiment can realize the following signal processing method. That is, the signal processing method according to the present embodiment includes a learning step (step S30 to step S32 in FIG. 14A) and a separation step (step S60 to step S61 in FIG. 17). In the learning step, the weight W of the network that generates the mask α for removing the noise component by adding the phase from the input physical quantity (sound near the engine 90) including the noise component (noise 91b) by the neural network 94. , And the extraction signal (extraction knocking sound 91a) from which the noise component is removed from the input physical quantity (engine neighborhood sound 90) is the estimated signal (estimated knocking cylinder) of the same dimension (unit) as the teacher signal (observation knocking cylinder internal pressure 93). The transfer function H for converting the internal pressure 92) by adding the phase is learned. In the separation step, the input physical quantity (engine near sound 90) is separated into a noise component and an extraction signal (extraction knocking sound 91a).

In such a signal processing method according to the present embodiment, the input physical quantity can be satisfactorily separated into a noise component and an extracted signal. In particular, it can be separated into a noise component and a signal suitable for level change in a sensory test.

(8) As shown in FIG. 3A, the signal processing device 10 (estimation device) according to the present embodiment has an extraction signal (extraction knocking sound 91a) obtained by removing a noise component (noise 91b) from an input physical quantity (engine proximity sound 90). ) Is multiplied by the transmission function H to estimate an estimated signal (estimated knocking in-cylinder pressure 92) having the same dimension (unit) as the teacher signal (observed knocking in-cylinder pressure 93). The signal processing device 10 (estimation device) according to the present embodiment includes the learning unit 21 and the separation unit 40 described above. The learning unit 21 uses the neural network 94 to remove the noise component from the input physical quantity (sound near the engine 90) and generate a mask α for extracting the extraction signal (extraction knocking sound 91a). Learn the transfer function H. The extraction signal estimation unit (second estimation unit 31) acquires an extraction signal (extraction knocking sound 91a) from which noise components have been removed by using the mask α. When learning the network weight W and the transfer function H, the learning unit 21 adds the amplitude and phase components related to the input physical quantity (engine proximity sound 90) to the mask α and the transfer function H. And learn.

Such a signal processing device 10 (estimation device) according to the present embodiment has an input physical quantity (engine) with respect to the mask α and the transfer function H when learning the network weight W and the transfer function H. It is possible to learn by adding the amplitude and phase components related to the neighboring sound 90).

(9) The signal processing device 10 (estimation device) according to the present embodiment can realize the following estimation method. That is, in the estimation method according to the present embodiment, the teacher signal (observation) is obtained by multiplying the extraction signal (extraction knocking sound 91a) obtained by removing the noise component (noise 91b) from the input physical quantity (engine vicinity sound 90) by the transmission function H. This is a method of estimating an estimated signal (estimated knocking cylinder pressure 92) having the same dimension (unit) as the knocking cylinder pressure 93). As shown in FIG. 14B or FIG. 14C, the estimation method according to the present embodiment includes a learning step (step S103 step) and an estimation signal estimation step (step S102 step). In the learning step (step S103), the neural network 94 removes the noise component from the input physical quantity (engine neighborhood sound 90) and generates a mask α for extracting the extraction signal (extraction knocking sound 91a). The weight W and the transfer function H are learned. In the estimation signal estimation step (step S102), the neural network 94 is used to acquire the extraction signal (extraction knocking sound 91a) from which the noise component has been removed by using the mask α, and the extraction is extracted by multiplying the transmission function H. The signal (extracted knocking sound 91a) is converted into an estimated signal. In the estimation method according to the present embodiment, when learning the network weight W and the transfer function H in the learning step (step S103), the input physical quantity (with respect to the mask α and the transfer function H) ( Learning is performed by taking into account the amplitude and phase components related to the engine vicinity sound 90).

Such an estimation method according to the present embodiment is related to the input physical quantity (engine proximity sound 90) with respect to the mask α and the transfer function H when learning the network weight W and the transfer function H. It is possible to learn by adding the amplitude and phase component to be performed.

As described above, according to the signal processing device 10 (estimating device) according to the first embodiment, the input physical quantity can be satisfactorily separated into the noise component and the extracted signal. In particular, the signal processing device 10 (estimating device) can separate the noise component into a signal suitable for level change in the sensory test. Such a signal processing device 10 (estimating device) can easily evaluate whether or not a target sound (knocking sound) is mixed in the background sound. Therefore, the signal processing device 10 (estimating device) can improve the evaluation performance of the presence or absence of knocking. In addition, the signal processing device 10 can perform a good sensory test.

[Second Embodiment]
Hereinafter, the configuration of the signal processing device 10A (estimation device) according to the second embodiment will be described with reference to FIG. FIG. 19 is a block diagram showing a configuration of the signal processing device 10A (estimation device) according to the second embodiment.

As shown in FIG. 19, the signal processing device 10A (estimation device) according to the second embodiment is a signal synthesizer as compared with the signal processing device 10 (estimation device) (see FIG. 2) according to the first embodiment. The difference is that 50 has the following functions. That is, the signal synthesis unit 50 has a function of multiplying the teacher signal (observation knocking cylinder internal pressure 93) by the reciprocal of the transfer function H and combining it with the noise component separated from the input physical quantity (engine proximity sound 90) to generate a processed sound. Has.

Similar to the signal processing device 10 according to the first embodiment, the signal processing device 10A (estimating device) according to the second embodiment grasps the presence or absence of the target sound (knocking sound) by the inspector with high accuracy. It is possible to improve the inspection performance.

The signal synthesis unit 50 multiplies the level-changed teacher signal whose level (magnitude) of the teacher signal (observation knocking cylinder pressure 93) is changed by the inverse of the transfer function H, and is separated from the input physical quantity (engine neighborhood sound 90). It may have a function of generating a processed sound by synthesizing it with a noise component.

Further, the signal synthesizer 50 is not a teacher signal (observation knocking cylinder pressure 93) but a transmission function to an arbitrary signal (for example, an arbitrary knocking cylinder pressure signal) having the same dimension as the teacher signal (observation knocking cylinder pressure 93). It may have a function of multiplying by the inverse number of H and synthesizing it with a noise component separated from the input physical quantity (engine proximity sound 90) to generate a processed sound.

Further, the signal synthesis unit 50 multiplies the teacher signal (observation knocking cylinder internal pressure 93) by the inverse of the change transfer function Hc in which the value of the transfer function H is changed, and sets the noise component separated from the input physical quantity (engine vicinity sound 90). It may be synthesized to generate a processed sound.

Further, the signal synthesis unit 50 multiplies the level change teacher signal whose level (magnitude) of the teacher signal (observation knocking cylinder pressure 93) is changed by the inverse of the change transfer function Hc whose value of the transfer function H is changed, and inputs the physical quantity. It may have a function of generating a processed sound by synthesizing it with a noise component separated from (sound near the engine 90).

Here, for example, the modified transfer function Hc is obtained by increasing or decreasing the amplitude and / or phase corresponding to a certain frequency band of the transfer function H.

Further, the signal synthesis unit 50 applies a transmission function He (not shown) from the measurement target measured in the acoustic vibration experiment to the ear position of the listener to the extraction signal, and measures the ear position of the listener when the vehicle is running. It may have a function of synthesizing noise components to generate a processed sound.

Further, the signal synthesis unit 50 obtains the estimated knocking sound by multiplying the teacher signal (observation knocking cylinder internal pressure 93) by the inverse number of the transmission function H estimated by the neural network 94, and further obtains the measurement by acoustic excitation or simulation. Multiply the transmission function He (not shown) from the target to the ear position of the listener (in the case of a vehicle, the passenger (especially the driver)), and synthesize and process the noise component of the listener's ear position measured while driving the actual vehicle. It may have a function of generating sound. In such a signal synthesis unit 50, for example, the knocking sound at the ear position of the listener (vehicle occupant (particularly driver)) obtained from the estimated knocking sound has a noise component observed at the listener's ear position during actual running. (Actual running sound) can be combined to simulate the knocking sound during actual driving.

The present invention is not limited to the above-described embodiment, and various modifications and modifications can be made without departing from the gist of the present invention.

For example, the above-described embodiment has been described in detail in order to explain the gist of the present invention in an easy-to-understand manner. Therefore, the present invention is not necessarily limited to those including all the components described above. In addition, the present invention can add other components to a certain component, or change some components to other components. In addition, the present invention can also delete some components.

For example, as described below, the signal processing device 10 (estimating device) can change the test target to any article. Further, the signal processing device 10 (estimating device) can change the input physical quantity and the teacher signal to any signal.

(1) For example, the signal processing device 10 can be changed and used in the environment shown in FIG. 20 as in the first modification shown in FIG. FIG. 20 is an explanatory diagram of the first modification. FIG. 21 is an explanatory diagram of learning of the weight and transfer function of the network that generates the mask in the first modification.

FIG. 20 shows an example of evaluating the vibration sound of the instrument panel 3b (instrument panel) detected at the ear position of the listener (passenger (particularly driver) of the vehicle) in the passenger compartment 3a of the vehicle 3. There is. In the example shown in FIG. 20, the listener's ear position running sound 190 (the vehicle interior sound heard at the listener's ear position) is the input physical quantity, and the observation instrument panel vibration acceleration 193 is the teacher signal. As shown in FIG. 21, the learning unit 21 of the signal processing device 10 uses the mask α to extract an input physical quantity (listener ear position running sound 190) as an extraction signal (extraction instrument panel vibration sound 191a) and a noise component (noise 191b). Separate into and. The mask α and the transfer function H include amplitude and phase components related to the input physical quantity.

Further, as shown in FIG. 21, the learning unit 21 of the signal processing device 10 obtains a signal waveform and a teacher signal (observation instrument panel vibration acceleration 193) obtained by performing an inverse short-time Fourier transform (ISTFT) on a noise component (noise 191b). The coherence between the signal waveform and the teacher signal (observation instrument panel vibration acceleration 193) obtained by inverse short-time Fourier transform (ISTFT) of the extracted signal (extracted instrument panel vibration sound 191a) becomes larger as the coherence with the signal becomes smaller. , The network weight W, and the transfer function H are learned. As a result, the signal processing device 10 has an input physical quantity (listener) so that the target sound (sound caused by the vibration of the instrument panel 3b) is not mixed in the noise (sound not caused by the vibration of the instrument panel 3b (background sound)). The ear position running sound 190) can be separated into an extraction signal (extraction instrument panel vibration sound 191a) and a noise component (noise 191b).

Further, as shown in FIG. 21, the learning unit 21 of the signal processing device 10 performs an inverse short-time Fourier transform (ISTFT) and a fast Fourier transform (FFT) on the extracted signal (extracted instrument panel vibration sound 191a). , By multiplying the transfer function H and performing inverse fast Fourier transform (IFFT) and short-time Fourier transform (STFT), the extracted signal (extracted instrument panel vibration sound 191a) is converted into an estimated signal (estimated instrument panel vibration acceleration 192). ing. Then, the learning unit 21 performs a short-time Fourier transform (STFT) on the estimated signal (estimated instrument panel vibration acceleration 192) and the teacher signal (observation instrument panel vibration acceleration 193) so that the error between the spectrogram obtained is minimized. , The neural network 94 learns the weight W of the network that generates the mask α and the transfer function H.

In such a signal processing device 10, for example, during a sensory test, the inspector examines the loudness of the sound caused by the vibration of the instrument panel 3b among the listener's ear position running sound 190 (vehicle interior sound heard at the listener's ear position). It can be made easier to grasp. For example, when the sound caused by the vibration of the instrument panel 3b is relatively loud, the signal processing device 10 generates a processed sound by the signal adjusting unit 51 of the signal synthesis unit 50, and performs subjective evaluation based on the auditory sense of the inspector. Therefore, the target value for vibration reduction of the instrument panel 3b can be set.

(2) Further, the signal processing device 10 can be changed and used in the environment shown in FIG. 22 as in the second modification shown in FIG. 23. FIG. 22 is an explanatory diagram of the second modification. FIG. 23 is an explanatory diagram of learning of the weight and transfer function of the network that generates the mask in the second modification.

FIG. 22 shows an example of evaluating the vibration sound of the compressor in the vicinity of the refrigerator 203. In the example shown in FIG. 22, the refrigerator sound 290 heard at the listener's ear position is the input physical quantity, and the observation compressor vibration acceleration 293 is the teacher signal. As shown in FIG. 23, the learning unit 21 of the signal processing device 10 uses the mask α to separate the input physical quantity (refrigerator sound 290) into an extraction signal (extraction compressor sound 291a) and a noise component (noise 291b). The mask α and the transfer function H include amplitude and phase components related to the input physical quantity.

Further, as shown in FIG. 23, the learning unit 21 of the signal processing device 10 obtains a signal waveform and a teacher signal (observation compressor vibration acceleration 293) obtained by performing an inverse short-time Fourier transform (ISTFT) on a noise component (noise 291b). The coherence between the signal waveform and the teacher signal (observation compressor vibration acceleration 293) obtained by inverse short-time Fourier transform (ISTFT) of the extracted signal (extracted compressor sound 291a) becomes larger as the coherence with the signal becomes smaller. The weight W of the network and the transfer function H are learned. As a result, the signal processing device 10 has an input physical quantity (refrigerator sound 290) so that the target sound (sound caused by the vibration of the compressor) is not mixed in the noise (sound not caused by the vibration of the compressor (background sound)). Can be separated into an extraction signal (extraction compressor sound 291a) and a noise component (noise 291b).

Further, as shown in FIG. 23, the learning unit 21 of the signal processing device 10 performs an inverse short-time Fourier transform (ISTFT) and a fast Fourier transform (FFT) on the extracted signal (extracting compressor sound 291a). By multiplying the transfer function H and performing the inverse fast Fourier transform (IFFT) and the short-time Fourier transform (STFT), the extraction signal (extraction compressor sound 291a) is converted into the estimation signal (estimated compressor vibration acceleration 292). .. Then, the learning unit 21 performs a short-time Fourier transform (STFT) on the estimated signal (estimated compressor vibration acceleration 292) and the teacher signal (observation compressor vibration acceleration 293) so that the error between the spectrogram obtained is minimized. , The neural network 94 learns the weight W of the network that generates the mask α and the transfer function H.

Such a signal processing device 10 can make it easier for the inspector to grasp the loudness of the refrigerator sound 290 that can be heard at the listener's ear position due to the vibration of the compressor, for example, during a sensory test. For example, when the sound caused by the vibration of the compressor is relatively loud, the signal processing device 10 generates a processed sound by the signal adjusting unit 51 of the signal synthesizing unit 50, and performs subjective evaluation by the inspector's hearing. , The target value of the vibration reduction of the compressor can be set.

(3) Further, the signal processing device 10 can be modified and used as in the third modification shown in FIG. 24. FIG. 24 is an explanatory diagram of learning of the weight and transfer function of the network that generates the mask in the third modification.

In the example shown in FIG. 24, in the passenger compartment 3a of the vehicle 3, the combustion sound included in the radiated sound of the engine 1 (FIG. 1) detected at the ear position of the listener (passenger (particularly driver) of the vehicle) is evaluated. An example of the case is shown. In the example shown in FIG. 24, the in-cylinder pressure 390 as a teacher signal of the engine 1 (FIG. 1), the radiated sound 390a and the vibration 390b as input physical quantities are observed, and the mask and the transfer function are learned based on each. Is shown.

The signal processing device 10 acquires the combustion sound 391aa (extracted signal) from the radiated sound 390a (input physical quantity) by using the mask α1 and also acquires the noise 391ab (noise component). The noise 391ab (noise component) is acquired by removing the combustion sound 391aa (extracted signal) from the radiated sound 390a (input physical quantity).

In the learning unit 21 of the signal processing device 10, the coherence between the noise component (noise 391ab) and the teacher signal (in-cylinder pressure 390) is reduced, and the coherence between the extraction signal (combustion sound 391aa) and the teacher signal (in-cylinder pressure 390) is reduced. The neural The network 94 learns the weight W1 (not shown) of the network that generates the mask α1 and the transfer function H1. The mask α1 represents the ratio of the combustion sound contained in the radiated sound 390a (input physical quantity) and the phase component (phase correction amount). Further, the transfer function H1 is an amplitude (gain) and a phase component for converting the combustion sound 391aa (extracted signal) into the estimated in-cylinder pressure 392a (estimated signal).

Further, the signal processing device 10 acquires the combustion vibration 391ba (extracted signal) from the vibration 390b (input physical quantity) by using the mask α2, and also acquires the noise vibration 391bb (noise component). The noise vibration 391bb (noise component) is acquired by removing the combustion vibration 391ba (extracted signal) from the vibration 390b (input physical quantity).

In the learning unit 21 of the signal processing device 10, the coherence between the noise component (noise vibration 391bb) and the teacher signal (in-cylinder pressure 390) becomes small, and the extraction signal (combustion vibration 391ba) and the teacher signal (in-cylinder pressure 390) are combined. To increase the coherence and to minimize the error between the estimated signal (estimated in-cylinder pressure 392b) and the spectrogram obtained by performing a short-time Fourier transform (STFT) on the teacher signal (in-cylinder pressure 390). The neural network 94 learns the weight W2 (not shown) of the network that generates the mask α2 and the transfer function H2. The mask α2 represents the ratio of the combustion vibration included in the vibration 390b (input physical quantity) and the phase component (phase correction amount). Further, the transfer function H2 is an amplitude (gain) and a phase component for converting the combustion vibration 391ba (extracted signal) into the estimated in-cylinder pressure 392b (estimated signal).

Further, the signal processing device 10 multiplies the transmission function H1a between the sound in the engine room and the sound at the ear position of the listener obtained by numerical simulation or vibration experiment with the combustion sound 391aa (extracted signal) to estimate air propagation. Generates sound 393a (estimated signal). Further, the signal processing device 10 multiplies the transmission function H2b between the engine mount vibration obtained by the numerical simulation or the vibration experiment and the sound of the listener's ear position with the combustion vibration 391ba (extracted signal) to estimate the individual propagation sound 393b. Generate (estimated signal). After that, the signal processing device 10 synthesizes the estimated air propagation sound 393a (estimated signal) and the estimated individual propagation sound 393b (estimated signal) to generate the estimated listener ear position sound 393 (synthetic estimated signal).

Such a signal processing device 10 can simulate the combustion sound detected at the ear position of a listener (vehicle occupant (particularly driver)), for example, during a sensory test.

1 ... Engine 2 ... Engine ECU
3 ... Vehicle 3a ... Vehicle interior 3b ... Instrument panel 4 ... Sound pressure sensor 5 ... In-cylinder pressure sensor 6 ... Data collection device 7 ... Monitor 8 ... Headphones (sound release unit)
9 ... Level designation unit 10, 10A ... Signal processing device (estimation device)
11 ... Signal cutting unit 12 ... Spectrogram calculation unit 13 ... Signal storage unit 14 ... Switch 20 ... Learning processing unit 21 ... Learning unit 22 ... Learned parameter storage unit 23 ... First estimation unit 24 ... Threshold calculation unit 25 ... Threshold memory Unit 26a ... Teacher signal storage unit 26b ... Estimated signal storage unit 26c ... Noise component storage unit 26d ... Extraction signal storage unit 30 ... Judgment processing unit 31 ... Second estimation unit (extracted signal estimation unit)
32 ... Threshold determination unit 40 ... Separation unit 50 ... Signal synthesis unit 51 ... Signal adjustment unit 52 ... Signal output unit 81 ... Knocking vibration 82 ... Knocking sound 83 ... Mechanical noise (noise component)
90 ... Sound near the engine (input physical quantity)
91a ... Extraction knocking sound (extraction signal)
91aa ... Level change knocking sound (level change extraction signal)
91b ... Noise (noise component)
91c ... Processing sound 92 ... Estimated knocking in-cylinder pressure (estimated signal)
93 ... Observation knocking in-cylinder pressure (teacher signal)
94 ... Neural network 94A ... Mask generation network 95 ... U-Net
96 ... Downward path (Encoder)
97 ... Hierarchy 98 ... Upward path (Decoder)
99 ... Output 100 ... Signal processing system 190 ... Listener ear position running sound (input physical quantity)
191a ... Extraction instrument panel vibration sound (extraction signal)
191b ... Noise (noise component)
192 ... Estimated instrument panel vibration acceleration (estimated signal)
193 ... Observation instrument panel vibration acceleration (teacher signal)
203 ... Refrigerator 290 ... Refrigerator sound (input physical quantity)
291a ... Extraction compressor sound (extraction signal)
291b ... Noise (noise component)
292 ... Estimated compressor vibration acceleration (estimated signal)
293 ... Observation compressor vibration acceleration (teacher signal)
390 ... In-cylinder pressure (teacher signal)
390a ... Radiated sound (input physical quantity)
390b ... Vibration (input physical quantity)
391aa ... Combustion sound (extraction signal)
391ab ... Noise (noise component)
391ba ... Combustion vibration (extraction signal)
391bb ... Noise vibration (noise component)
392a, 392b ... Estimated in-cylinder pressure (estimated signal)
393 ... Estimated listener ear position sound (synthetic estimated signal)
393a ... Estimated air propagation sound (estimated signal)
393b ... Estimated individual propagation sound (estimated signal)
α, α1, α2 ... Masks F11, F21, F31, F37 ... Spectrograms F12, F22, F32, F36, F93 ... Signal waveforms F13, F23 ... Coherence F33, F35 ... Spectrum F34 ... Frequency response characteristics H, H1, H1a, H2 , H2b, He ... Transmission function Hc ... Change transmission function M1 ... Learning mode connection M2 ... Threshold calculation mode connection M3 ... Judgment mode connection M4 ... Separation mode connection M5 ... Sensory test mode connection W ... Network weight T ... Threshold

Claims (14)

By the neural network, with learning the weights of the network for generating a mask for removing an input physical quantity or found before Symbol noise component that contains the noise component, the extraction signal the noise component is removed from the input physical quantity a learning unit that learns a transfer function for conversion to the estimated signal of the same dimensions as teacher signals,
A separation unit that separates the input physical quantity into the noise component and the extraction signal is provided .
The learning unit learns the weight of the network that generates the mask by adding the phase component related to the input physical quantity, and learns the transfer function by adding the phase component related to the input physical quantity. <br/> A signal processing device characterized in that. In the signal processing device according to claim 1,
The learning unit, the network weights to generate the mask, and, when learning the transfer function, in addition to the phase component, in consideration of the amplitude associated with the input physical quantity, the network generating the mask as well as learning a weight, the addition to the phase component, in consideration of amplitude related to the input physical quantity, a signal processing apparatus characterized by learning the transfer function. In the signal processing device according to claim 2,
In the learning unit, the weight of the network that generates the mask and the weight of the network that generates the mask are increased so that the relationship between the noise component and the teacher signal becomes smaller and the relationship between the extracted signal and the teacher signal becomes larger. A signal processing device characterized by learning a transfer function. In the signal processing device according to claim 3,
The learning unit is characterized in that when learning the weight of the network that generates the mask and the transfer function, the learning unit further learns so that the error of the estimated signal with respect to the teacher signal becomes small. Processing equipment. The signal processing device according to any one of claims 1 to 4.
Further comprising a signal combining unit for generating a processed sound by combining the signals,
The signal processing unit is a signal processing device that changes the magnitude of the extracted signal and synthesizes it with the noise component separated from the input physical quantity to generate a processed sound. The signal processing device according to any one of claims 1 to 4.
Furthermore, it is equipped with a signal synthesizer that synthesizes signals to generate processed sounds.
The signal processing unit has a function of multiplying the teacher signal by the reciprocal of the transfer function and synthesizing it with the noise component separated from the input physical quantity to generate a processed sound. The signal processing device according to any one of claims 1 to 4.
Furthermore, it is equipped with a signal synthesizer that synthesizes signals to generate processed sounds.
The signal synthesis unit has a function of multiplying the changed teacher signal whose magnitude is changed by the inverse of the transfer function and synthesizing it with the noise component separated from the input physical quantity to generate a processed sound. A signal processing device characterized by. The signal processing device according to any one of claims 1 to 4.
Furthermore, it is equipped with a signal synthesizer that synthesizes signals to generate processed sounds.
The signal synthesizer has a function of multiplying an arbitrary signal having the same dimension as the teacher signal by the inverse of the transfer function and synthesizing it with the noise component separated from the input physical quantity to generate a processed sound. A characteristic signal processing device. The signal processing device according to any one of claims 1 to 4.
Furthermore, it is equipped with a signal synthesizer that synthesizes signals to generate processed sounds.
The signal synthesizer has a function of multiplying the teacher signal by the inverse of the change transfer function in which the value of the transfer function is changed and synthesizing it with the noise component separated from the input physical quantity to generate a processed sound. A characteristic signal processing device. The signal processing device according to any one of claims 1 to 4.
Furthermore, it is equipped with a signal synthesizer that synthesizes signals to generate processed sounds.
The signal synthesizer multiplies the changed teacher signal whose magnitude is changed by the inverse number of the changed transfer function whose value of the transfer function is changed, and synthesizes and processes it with the noise component separated from the input physical quantity. A signal processing device characterized by having a function of generating sound. The signal processing device according to any one of claims 1 to 4.
Furthermore, it is equipped with a signal synthesizer that synthesizes signals to generate processed sounds.
The signal synthesizer has a function of multiplying the extracted signal by a transfer function from the measurement target to the listener's ear position and synthesizing the noise component of the listener's ear position to generate a processed sound. Signal processing device. The signal processing device according to any one of claims 1 to 4.
Furthermore, it is equipped with a signal synthesizer that synthesizes signals to generate processed sounds.
In addition to multiplying the teacher signal by the inverse of the transfer function, the signal synthesizer further multiplies the transfer function from the measurement target to the listener's ear position to synthesize the noise component of the listener's ear position. A signal processing device characterized by having a function of generating processed sound. The signal processing device according to any one of claims 1 to 4.
Furthermore, it is equipped with a signal synthesizer that synthesizes signals to generate processed sounds.
The signal processing unit is a signal processing device including a signal adjusting unit that receives a designation of the magnitude of the extracted signal by an operator. By the neural network, with learning the weights of the network for generating a mask for removing an input physical quantity or found before Symbol noise component that contains the noise component, the extraction signal the noise component is removed from the input physical quantity a learning step for learning a transfer function for conversion to the estimated signal of the same dimensions as teacher signals,
Look including a separation step of separating the input physical quantity and the extraction signal and the noise component,
In the learning step, the phase component related to the input physical quantity is added to learn the weight of the network that generates the mask, and the phase component related to the input physical quantity is added to learn the transfer function. <br/> A signal processing method characterized by the fact that.

Images