CN117908168A - Rainfall state detection device and electronic equipment based on sound - Google Patents
Rainfall state detection device and electronic equipment based on sound Download PDFInfo
- Publication number
- CN117908168A CN117908168A CN202410308526.XA CN202410308526A CN117908168A CN 117908168 A CN117908168 A CN 117908168A CN 202410308526 A CN202410308526 A CN 202410308526A CN 117908168 A CN117908168 A CN 117908168A
- Authority
- CN
- China
- Prior art keywords
- rainfall
- rainfall state
- detection device
- audio
- sound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 78
- 230000005236 sound signal Effects 0.000 claims abstract description 99
- 238000003062 neural network model Methods 0.000 claims abstract description 14
- 238000012549 training Methods 0.000 claims description 39
- 238000013528 artificial neural network Methods 0.000 claims description 25
- 238000012545 processing Methods 0.000 claims description 25
- 238000012360 testing method Methods 0.000 claims description 19
- 238000013139 quantization Methods 0.000 claims description 8
- 230000003068 static effect Effects 0.000 claims description 7
- 238000001228 spectrum Methods 0.000 claims description 6
- 230000000087 stabilizing effect Effects 0.000 claims description 4
- 230000001953 sensory effect Effects 0.000 claims description 2
- 230000006641 stabilisation Effects 0.000 claims description 2
- 238000011105 stabilization Methods 0.000 claims description 2
- 238000000034 method Methods 0.000 description 15
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 239000011159 matrix material Substances 0.000 description 7
- 238000013473 artificial intelligence Methods 0.000 description 6
- 239000000284 extract Substances 0.000 description 5
- 239000013598 vector Substances 0.000 description 5
- 230000007613 environmental effect Effects 0.000 description 4
- 238000001914 filtration Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 230000035945 sensitivity Effects 0.000 description 4
- 230000003595 spectral effect Effects 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 239000005357 flat glass Substances 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 239000003990 capacitor Substances 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 230000003111 delayed effect Effects 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Landscapes
- Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
Abstract
The application relates to a rainfall state detection device based on sound and an electronic device, wherein the rainfall state detection device based on sound comprises: the sound signal acquisition module is used for acquiring an audio signal of the environment; and the sensing and storing integrated processor is used for extracting the audio characteristics of the audio signals of the environment and calculating rainfall states in an inference mode according to the extracted audio characteristics and a neural network model, wherein the neural network model is a rainfall state identification model based on rain sound. The rainfall state detection device provided by the application can automatically detect the rainfall state in real time, and can improve the accuracy of the rainfall state detection result, has low cost and low power consumption, and has wider application scenes.
Description
Technical Field
The application relates to the field of artificial intelligence, in particular to a rainfall state detection device based on sound and electronic equipment.
Background
Along with the continuous popularization of automobiles, the automobiles are increasingly used, and people need to accurately and timely identify the rainy weather when meeting the rainy weather during driving so as to accurately and timely open the wiper.
Prior art automobiles typically identify whether there is rain by a rain sensor on the windshield. However, the rain sensor detects the rainy state with inaccurate results and low detection sensitivity. Because the existing rain sensor generally uses the refraction and reflection principles of light to detect whether there are rain drops and the size of the rain drops. The sensing unit comprises a light emitter, a light receiver and a lens plate, wherein the lens plate is used for coupling a light beam emitted by the light emitter into window glass, coupling the light beam out of the window glass and guiding the light beam onto the light receiver, the surface of the lens plate facing the light emitter or the light receiver and the surface facing the window glass are provided with Fresnel lens structures, the light emitted by the light emitter in this way has a wide radiation range, only a small part of the light can be utilized for raindrop detection, the light entering the light receiver is only a small part, and the detection result is inaccurate and the detection sensitivity is low. Particularly, in some 'sparse raining' scenes, the rainfall on the windshield of the existing rainfall sensor has influenced the sight of a driver, the rainfall sensor still cannot detect the raining state, and the windscreen wiper cannot be started. That is, in a vehicle-mounted application scenario, the actual rainfall has already affected the driver's line of sight, but the rainfall sensor has not yet detected the rainfall. In the prior art, rainfall information is obtained according to weather forecast, but whether rainfall and rainfall intensity have spatial non-uniformity or not, and accurate rainfall intensity at a certain position cannot be accurately obtained through weather forecast. The existing accurate measurement of rainfall intensity is completed through a professional measuring tool (such as a rain gauge) which can measure the accurate rainfall intensity, but the measuring tool is expensive, the measuring tool is installed only in a hydrological station and an meteorological station, the space distribution of the measuring station is sparse and uneven, and the measuring data is not shared, so that the measuring tool is not suitable for popular use of mass users. In addition, the rainfall intensity in the range of tens to hundreds of kilometers can be obtained by the rain measuring radar, but the rain measuring radar is high in price, difficult to popularize and high in power consumption.
Therefore, the detection device for detecting the raining state in the prior art has the defects of inaccurate detection, higher cost and higher measurement power consumption.
Disclosure of Invention
In view of the above, the present application provides a rainfall state detection device based on sound and an electronic device, so as to at least solve some or all of the above technical problems.
The rainfall state detection device provided by the application comprises: the sound signal acquisition module is used for acquiring an audio signal of the environment; the sensing and storing integrated processor is used for extracting the audio characteristics of the audio signals of the environment and calculating rainfall states in an inference mode according to the extracted audio characteristics and a neural network model, wherein the neural network model is a rainfall state identification model based on rain sound.
In one embodiment, the sense and compute unified processor includes: an analog signal processing module based on an analog circuit for extracting audio features of an analog audio signal of an environment; and the neural network processor is used for inferentially calculating the rainfall state according to the extracted audio characteristics and the rainfall state identification model based on the rain sound.
In one embodiment, the analog signal processing module comprises: a low noise amplifier for amplifying an input audio signal according to a fixed gain; the sound intensity stabilizing circuit is used for compensating sound intensity changes caused by different sound sizes so as to stabilize the intensity of the amplified audio signal; and the rectification quantization circuit is used for rectifying the stabilized audio signal and quantizing the rectified signal into digital signal pulses so as to form the audio characteristics of the analog audio signal.
In one embodiment, the sound intensity stabilization circuit includes: and the controller is used for controlling the gain of the gain-adjustable amplifier.
In an embodiment, the neural network processor is further configured to: acquiring an audio signal training sample and a rainfall state corresponding to the audio signal training sample; extracting audio characteristics of the audio signal training samples; inputting the extracted audio characteristics of the audio signal training sample and the rainfall state corresponding to the audio signal training sample into a neural network model for training to obtain the rainfall state identification model based on the rain sound.
In an embodiment, the neural network processor is further configured to: obtaining a test result according to the audio signal test sample and the rainfall state identification model obtained by training; judging whether the accuracy and false alarm rate of the rainfall state identification model obtained through training reach a preset accuracy threshold and false alarm rate threshold according to the test result; and if the accuracy of the rainfall state identification model obtained through training is higher than the preset accuracy threshold and/or the false alarm rate is lower than the false alarm rate threshold, determining the rainfall state identification model obtained through training as a final rainfall state identification model.
In an embodiment, the neural network processor is further configured to: obtaining a static rainfall identification model of the rainfall state detection device when the rainfall state detection device is static according to the audio signal test sample, and obtaining a dynamic rainfall identification model of the rainfall state detection device when the rainfall state detection device is dynamic according to the audio signal test sample.
In an embodiment, the audio features comprise a log energy spectrum of mel-like scales corresponding to frequencies of the audio signal.
In an embodiment, the audio features are audio features of 3khz to 10khz sounds.
The application also provides electronic equipment, which comprises any rainfall state detection device.
The rainfall state detection device provided by the application acquires the audio signal of the environment through the sound signal acquisition module, and can calculate the rainfall state by inference through the sensing and storing integrated processor by combining the rainfall state identification model based on the rain sound and the audio characteristics of the extracted audio signal of the environment. The application can identify the rainfall state based on the acquired sound in combination with the artificial intelligence, and the accuracy of the result of the rainfall state detected by the application is higher than that of the rainfall state detected by the method based on the optical mode, the weather forecast and the like. The sensing and storing integrated processor is low in power consumption and low in cost required by a mode of reasoning and calculating the rainfall state, so that the rainfall state detection device is low in measurement power consumption and low in measurement cost, can automatically detect the rainfall state at any time and any place, is convenient to use, and can be widely applied to various electronic equipment in various scenes conveniently.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural diagram of a rainfall state detection device according to an embodiment of the present application;
Fig. 2 is a schematic circuit diagram of an analog signal processing module according to an embodiment of the application;
FIG. 3 is a schematic view showing an installation position of a rainfall state detection device on an automobile according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a neural network processor according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a process of implementing a rainfall state detection device according to an embodiment of the present application;
FIG. 6 is a flow chart illustrating a process for obtaining a rainfall state identification model according to an embodiment of the present application;
FIG. 7 is a diagram showing an analog acquisition of an acoustic energy spectrum in comparison to a digital acquisition of an audio signal according to the prior art in accordance with an embodiment of the present application;
FIG. 8 is a schematic diagram of a sense and compute unified processor extracting audio features from an acquired audio signal of an environment in accordance with an embodiment of the present application.
Detailed Description
For a better understanding of the objects, technical solutions and advantages of the present invention, reference should be made to the various exemplary embodiments described hereinafter with reference to the accompanying drawings, which form a part hereof, and in which are described various exemplary embodiments which may be employed in practicing the present invention. The same reference numbers in different drawings identify the same or similar elements unless expressly stated otherwise. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present disclosure. It is to be understood that they are merely examples of processes, methods, apparatuses, etc. that are consistent with certain aspects of the present disclosure as detailed in the appended claims, other embodiments may be utilized, or structural and functional modifications may be made to the embodiments set forth herein without departing from the scope and spirit of the present disclosure.
In the description of the present invention, it should be understood that the terms "center," "longitudinal," "transverse," and the like are used in an orientation or positional relationship based on that shown in the drawings, and are merely for convenience in describing the present invention and to simplify the description, rather than to indicate or imply that the elements referred to must have a particular orientation, be constructed and operate in a particular orientation. The terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. The term "plurality" means two or more. The terms "connected," "coupled" and "connected" are to be construed broadly and may be, for example, fixedly connected, detachably connected, integrally connected, mechanically connected, electrically connected, communicatively connected, directly connected, indirectly connected via intermediaries, or may be in communication with each other between two elements or in an interaction relationship between the two elements. The term "and/or" includes any and all combinations of one or more of the associated listed items. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In order to illustrate the technical solutions of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the following description, for the purpose of providing a thorough understanding of the present invention, detailed structures and steps are presented in order to illustrate the technical solution presented by the present invention. Preferred embodiments of the present invention are described in detail below, however, the present invention may have other embodiments in addition to these detailed descriptions.
As shown in fig. 1, the present invention provides a rainfall state detection device based on sound, which can be used for detecting rainfall state, such as whether there is rain or the intensity of the rain. The rainfall state detection device provided by the invention comprises: a sound signal acquisition module 20 for acquiring an audio signal of an environment; and a sense and save integrated processor 10 for extracting audio features of the audio signal of the environment and inferentially calculating a rainfall state according to the extracted audio features and a neural network model, wherein the neural network model is a rainfall state identification model based on rain sound. Namely: the rainfall state detection device provided by the invention is used for reasoning and identifying whether the acquired sound has rainy sound or not by acquiring the audio signal of the environment, extracting the audio characteristic of the audio signal and the rainfall state identification model so as to identify the rainfall state. The sound signal acquiring module 20 may be an acoustic sensor such as a microphone to acquire an audio signal of the environment, or the sound signal acquiring module 20 may be a module that directly acquires sound acquired by the acoustic sensor such as the microphone.
The integrated sensing and storing processor 10 integrates sensing, storing and calculating, and can realize the reasoning and calculating of the raining state of the surrounding environment based on artificial intelligence and the sound of the environment by combining a neural network. The integrated processor 10 may be implemented as a chip. As moore's law gradually approaches a limit, the problems of "memory wall" and "power consumption wall" of Artificial Intelligence (AI) chips based on von neumann architecture are increasingly prominent, and the increasing speed of the computing power of the chips becomes slower and slower, and particularly, the memory and the computation are separated so that data needs to be carried between the memory and the computation processing unit for a plurality of times, so that the problems of memory and power consumption are more prominent. And a computer-In-Memory (CIM) architecture can directly complete the computation In the Memory, eliminating frequent accesses between computation and Memory. A unified architecture is understood to be a memory with embedded computing power, with new computing architecture for two-dimensional and three-dimensional matrix multiplication/addition operations, rather than optimization over conventional logic units or processes. Thus, unnecessary delay and power consumption of data movement can be essentially eliminated, the calculation efficiency of Artificial Intelligence (AI) is improved by hundreds or thousands times, the cost is reduced, and a storage wall and a power consumption wall can be broken. Therefore, the present invention can reduce the delay and power consumption of data movement by the integrated processor 10, which can provide a larger power to better energy efficiency ratio, which requires low power consumption and high efficiency in recognizing rainfall conditions. In addition, the rainfall state is calculated by reasoning based on the rainfall state identification model of the rain sound and the audio characteristics of the audio signals of the environment, so that the accuracy of the result of identifying the rainfall state is high, inaccuracy caused by the distribution of space regions and the like weather forecast is avoided, and the cost is high and the detection is inaccurate like radar and an optical rainfall sensor. In addition, the sensing and calculating integrated processor 10 has smaller area and volume, can be more conveniently compatible with various electronic devices, has low cost and low power consumption, and can be widely applied to various electronic devices.
The rainfall state detection device provided by the invention can automatically detect and identify rainfall states at any time and any place, is convenient and intelligent to use, and can be widely applied to various scenes and various electronic equipment. For example, when the rainfall state detection device provided by the invention is arranged on the windshield of an automobile, the device can be combined with the acquired characteristics of the audio signals of the environment around the windshield to identify whether the environment of the windshield is rainy or not, so that whether the windshield needs to be automatically opened or not can be controlled, a user can accurately adjust the windshield according to the rainy state when opening the automobile, the detection result is accurate, and the user experience of the user of the automobile can be improved.
In one embodiment of the present invention, the sense and calculate all-in-one processor 10 includes: an analog signal processing module 11 based on analog circuitry for extracting audio features of an analog audio signal of the environment; and a neural network processor 12 for inferentially calculating a rainfall state from the extracted audio features and a rain sound based rainfall state recognition model. Wherein the analog signal processing module 11 extracts audio features of the analog audio signal in the environment based on the analog circuit. For example, front-end preprocessing is performed on the audio signal in the environment based on the analog signal processing module 11 to extract the audio features of the audio signal in the environment, thereby making the environmental sound become audio features that can be processed by the neural network.
Referring to the embodiment shown in fig. 3, the rainfall state detection device provided in this embodiment is installed in a black sub-frame beside the rearview mirror in the figure. The rainfall state detection device provided in this embodiment includes the microphone and it is directional microphone for the microphone sets up the position towards windshield on the car, and the microphone only acquires the raindrop striking sound that whole windshield passed out, shields other noise beyond the windshield simultaneously, and then improves the signal to noise ratio of raindrop. The audio signals of the environment obtained by the microphone are transmitted to the sensing and calculating integrated processor, the analog signal processing module in the sensing and calculating integrated processor firstly extracts the audio characteristics of the analog audio signals of the environment, and the neural network in the neural network processor can deduce the rainfall state by combining the audio characteristics. Parameters within the neural network are related to what data the neural network is trained with.
For example, different scenes may all be trained together, resulting in a neural network model, and rainfall conditions may be identified by a network model and a set of parameters.
As shown in table 1 below, audio data is acquired for a vehicle in different conditions and with the rain condition detection device placed in two different positions of the vehicle. Namely, the rainfall state detection devices are respectively arranged at the position 1 and the position 2, and the audio data of the weather, which is from no rain to little rain, medium and small rain, little rain and no rain, are obtained when the automobile is detected to be in a running state and a parking state. For further example, in the case that the automobile is traveling at a speed of 1-60 km/h and the actual weather is rain-free to rain-free, audio signals of track 1 and track 2 at a and B of the automobile windshield are acquired for 59 minutes and 38 seconds, respectively, and these audio data can be used to train a neural network model for recognizing rainfall conditions. For example, gradient descent methods are employed to train convolutional neural network models for identifying rainfall conditions. The invention is not limited to this training method and other training methods may be used.
In a preferred embodiment, the analog signal processing module 11 based on analog circuitry comprises: a low noise amplifier for amplifying an input audio signal (i.e., acquired sound in the environment) with a fixed gain; a sound intensity stabilizing circuit for compensating for a sound intensity variation caused by different sound sizes to stabilize the intensity of the amplified audio signal; and the rectification quantization circuit is used for rectifying the stabilized audio signal and quantizing the rectified signal into digital signal pulses so as to form the audio characteristics of the analog audio signal. The low noise amplifier is mainly used for amplifying signals, for example, when a sound signal comes in from the beginning, the signal may be weak, so that the signal needs to be amplified, if the sound signal is directly amplified, the noise of the sound signal can be synchronously amplified, and the noise can be suppressed while the sound signal in the environment is amplified through the low noise amplifier, so that the detection accuracy of the detection device is improved, and the user experience of corresponding electronic equipment is improved. The analog signal processing module 11 can extract the audio characteristics in the environment, and the analog signal processing module 11 is realized based on an analog circuit, so that the processing speed is high, the power consumption is low, and the audio characteristics are stable and reliable.
As shown in detail in fig. 2, in the preferred embodiment, feature extraction of audio signals in the environment is implemented based on hardware of analog circuits to achieve faster speed and lower power consumption. The analog signal processing module 11 (ASP, analog signal processing) mainly includes: a low noise amplifier (LNA, low Noise Amplifier) that amplifies an input signal by a fixed gain, a forward input of the LNA being connected to the bias voltage bias through a resistor, a reverse input of the LNA being connected to the bias voltage through a resistor, and a capacitor being connected across the forward input and the reverse output of the LNA, and a capacitor being connected across the reverse input and the forward output of the LNA; a sound intensity stabilizing circuit, which is formed by a gain adjustable amplifier (VGA, variable GAIN AMPLIFIER) and an automatic gain controller (AGC, automatic Gain Control) connected with the output end of the low noise amplifier, namely, the automatic gain controller is used for controlling the VGA to amplify signals output by the LNA by different gains according to different sound conditions so as to compensate the change of sound signal intensity caused by different sound sizes, so that the output signal intensity is relatively constant; and the rectification quantization circuit comprises a half-wave rectifier (HWR, half Wave Rectifier) and a quantization circuit which are connected in sequence. The half-wave rectifier is used for half-wave rectifying the signal output by the VGA, and the quantization circuit is used for quantizing the rectified signal into digital signal pulses and outputting the digital signal pulses, so that the audio characteristics of the audio signal in the environment are obtained. The quantization circuit in this figure is implemented by IAF (INTEGRATE AND FIRE, also called integral emission).
The audio signal is acquired by adopting a digital circuit in the prior art, and the digital circuit needs more devices, so that the power consumption and the cost are higher in the prior art. The analog signal processing module provided by the invention processes the audio signal based on the analog circuit, and has fewer required devices, less corresponding power consumption and lower cost. For example: in the conventional scheme of the digital circuit in the prior art, a digital signal processing module DSP is combined with an analog-to-digital converter ADC, but in the embodiment shown in fig. 2 of the present invention, an analog-to-digital converter is not required, and the analog signal processing module processes an acquired analog audio signal in a series of analog domains, and then can convert the analog signal into a digital signal at the position where the analog signal output by the half-wave rectifier is subjected to integral emission (i.e., quantization circuit). The cost of the whole rainfall state detection device provided by the invention can only need the cost of the ADC in the digital circuit in the prior art, and therefore, the cost and the power consumption of the rainfall state detection device for obtaining the rainfall state based on the simulation mode provided by the invention are lower. The analog signal processing module 11 can rapidly extract the audio characteristics of the audio signals in the environment, has low power consumption, and can adapt to rapidly processing the environmental sounds of different scenes, so that the signals of the extracted audio characteristics are stable and reliable, and the accuracy of the detection result of the rainfall state detection device is further improved.
Further preferably, a band-pass filter bank (BPF, bandpass Filter) and a buffer (buffer) are electrically connected between the VGA and the half-wave rectifier in sequence, wherein the band-pass filter bank may include multiple groups of filtering channels to more accurately extract audio features in the environment. Further preferably, an attenuator (ATT, attenuator) is bridged between the output end of the low noise amplifier and the VGA, so as to attenuate the signal amplified by the LNA, so that the signal amplitude range can meet the input range requirement of the BPF, and further, irrelevant signals can be better removed and the accuracy of the final monitoring result can be improved.
The neural network processor 12 (also called a neural network accelerator) in one embodiment shown in fig. 4 includes a preprocessing module, a memory computation matrix, a shared memory, and a vector processor, wherein the preprocessing module is connected to the memory computation matrix, the memory computation matrix is connected to the vector processor, and the shared memory is connected to the preprocessing module, the memory computation matrix, and the vector processor. The in-memory computation matrix may be a matrix formed by a plurality of CIMs (computing in memory, in-memory computations), with low area and power consumption of the neural network processor 12. The memory wall problem can be solved by memory calculation. In the prior art, a von neumann architecture computer system divides a memory and a processor into two parts, and the cost of the processor for frequently accessing the memory forms a memory wall, and high-frequency data handling is often a primary cause of power consumption occupied by a chip, especially the chip in the AI field, so as to influence the computing power, efficiency, power consumption and the like of the chip. In-memory computing in the present application combines computing and storage into one, thereby reducing the frequency of processor access to memory (since computing has already been done in memory for the most part), so the present application provides a unified processor 10 with unified technology (integrating sensing, storage and computing) that can have ultra-high computational power, efficiency and energy efficiency ratio. Therefore, the rainfall state detection device with the sensing and storing integrated processor 10 provided by the application can accurately detect rainfall states with high efficiency and low power consumption.
Referring to fig. 5 to 6, the rainfall condition recognition model in the neural network processor 12 may be a model already set therein, or may be a new model updated by retraining in the background according to circumstances. The embodiment shown in fig. 4 is a general implementation process of implementing rainfall for a rainfall state detection device on a windshield of an automobile, and the implementation process of implementing rainfall state detection mainly includes: s100, acquiring an audio signal acquired by a microphone, namely acquiring an audio signal of an environment where a windshield is positioned; s200, extracting the audio features in the audio signals, for example, acquiring the acoustic energy spectral density of the environmental sound; s300, inputting the acquired audio characteristic data of the environment into a rainfall state model based on rain and specially used for the windshield of the automobile, and enabling the sensing and calculating integrated processor 10 to calculate the rainfall state of the environment where the windshield is currently located in an inference mode.
In a preferred embodiment provided by the present application, the neural network processor 12 is further configured to: acquiring an audio signal training sample and a rainfall state corresponding to the audio signal training sample; extracting audio characteristics of the audio signal training samples; inputting the extracted audio characteristics of the audio signal training sample and the rainfall state corresponding to the audio signal training sample into a neural network model for training to obtain the rainfall state identification model based on the rain sound. For example, fig. 5 is a schematic flow chart of an implementation process of obtaining a rainfall state recognition model according to an embodiment of the present application. It mainly comprises: s210, acquiring an audio signal training sample on an automobile windshield collected by a microphone and a rainfall state corresponding to the audio signal training sample; s220, extracting audio features according to the audio signal training samples, namely rain sound features corresponding to the training samples; s230, inputting the data of the audio characteristics and rainfall intensity into a neural network model for training to obtain a rainfall state identification model based on rain sound and specially used for the automobile windshield. The rainfall state recognition model based on the rain sound and specially used for the automobile windshield, which is obtained through training, is combined with the sensing and storing integrated processor 10, so that the rainfall state of the front windshield of the automobile can be simply and quickly obtained in real time. In addition, the rainfall state identification model can be continuously updated according to the external changes such as the use environment of the user, so that the accuracy of the rainfall state detection result and the detection efficiency are improved.
In a preferred embodiment provided by the present invention, the neural network processor 12 is further configured to: obtaining a test result according to the audio signal test sample and the rainfall state identification model obtained by training; judging whether the accuracy and false alarm rate of the rainfall state identification model obtained through training reach a preset accuracy threshold and false alarm rate threshold according to the test result; and if the accuracy of the rainfall state identification model obtained through training is higher than the preset accuracy threshold and/or the false alarm rate is lower than the false alarm rate threshold, determining the rainfall state identification model obtained through training as a final rainfall state identification model. For example, acquiring an audio signal test sample on an automobile windshield collected by a microphone apparatus; obtaining a test result according to the audio signal test sample and the rainfall state identification model based on the rain sound and specially used for the automobile windshield; judging whether the accuracy and false alarm rate of the rainfall state identification model special for the automobile windshield based on the rain sound reach a preset accuracy threshold and false alarm rate threshold according to the test result; and if the accuracy of the rainfall state identification model based on the rain sound and special for the automobile windshield is higher than the preset accuracy threshold and/or the false alarm rate is lower than the false alarm rate threshold, determining the rainfall state identification model based on the rain sound and special for the automobile windshield as an available rainfall state identification model based on the rain sound and special for the automobile windshield, namely determining the rainfall state identification model as a final rainfall state identification model for subsequent detection of the rainfall state in the automobile. By the method, the accuracy of the rainfall state detection result can be improved, for example, after the plurality of audio signals of the environment are collected for a plurality of times in a certain period of time, the rainfall state corresponding to the plurality of audio signals is synthesized to obtain a final rainfall state identification model, and the accuracy of the rainfall state detection result can be improved. In one embodiment, the neural network is a DS-CNN separable convolutional neural network.
Because there are many situations in the use environment of the user, there are a dynamic scene where the automobile is running and a static scene where the automobile is stationary, and preferably, the neural network processor 12 is further configured to: according to the audio signal test sample, a static rainfall identification model when the electronic equipment where the rainfall state detection device is positioned is at rest is obtained, and according to the audio signal test sample, a dynamic rainfall identification model when the electronic equipment where the rainfall state detection device is positioned is dynamic is obtained. The static recognition model is used for recognizing the raining sound state under the static state of the vehicle, and the dynamic recognition model is used for recognizing the raining sound state under the driving state of the vehicle. Because various environmental noises with different sound pressure levels, including wind noise, road tire noise, engine noise and the like, can be obtained in a limited sound dynamic range by the mode, the characteristic distinguishing degree can be improved, and the accuracy of rainfall state detection results can be improved, for example, whether the automobile rains or not and the intensity of rains can be detected more accurately in more use environments. The problem that the detection result of the traditional rainfall sensor is not accurate enough and the detection sensitivity is low is solved.
In a preferred embodiment, the audio characteristics of the audio signal of the environment extracted by the sensory processor 10 include acoustic energy spectral densities corresponding to the audio frequencies. For example, a log energy spectrum of a mel-like scale of the audio signal is extracted. That is, the present embodiment uses the log energy spectrum of the mel-like scale of the analog signal, which is different from the conventional method in which the digital audio signal is subjected to a series of processing, and in this embodiment, the analog signal corresponding to the audio signal is directly subjected to the frequency energy integration in the analog domain to obtain a cepstrum coefficient of the mel-like frequency. Therefore, various redundant links processed by the conventional signal system can be compressed, so that the hardware power consumption is lower and the area is smaller.
Referring to fig. 7, a diagram of a comparison of an acoustic energy spectrum acquired in an analog manner and an audio signal acquired in a digital manner in the prior art is shown in an embodiment of the present application. In this embodiment, the audio features include acoustic energy spectral density, which is a set of two-dimensional vectors [ Nx20] composed of 8Bit data, where N represents a time frame, and the processing shown in fig. 8 may be performed on the two-dimensional vectors by the integrated sensor processor, and fig. 8 is a schematic diagram of the integrated sensor processor extracting the audio features from the acquired audio signal of the environment according to an embodiment of the present application. I.e. pre-emphasis, framing, windowing, FFT (i.e. fast fourier transform), mel filtering (i.e. Mel filtering), logarithmic operation, and DCT transformation (i.e. discrete cosine transform) are performed sequentially. In fig. 7, the energy of the sound of each of the frames n=20, i.e. 20, at different frequencies, i.e. the acoustic energy spectral density, is shown, the upper graph in fig. 7 is an audio signal digitally obtained by Mel filtering in the prior art, and the lower graph in fig. 7 is the audio characteristics obtained by the analog-based analog signal processing module provided by the present application. The signals output by the analog signal processing module are processed by the neural network processor to obtain a rainy state, and the output result of the rainy state is respectively represented by 0 or 1. The signal of the N frames correspondingly outputs N0 and/or 1 results. The mode can reduce power consumption and improve product integration level.
In the prior art, a rainfall sensor is used, only a region with the position corresponding to the region being about 80 square millimeters can be detected, if the rainfall density is insufficient, raindrops cannot be sensed basically in the region where the rainfall sensor is located, in some rain sparse scenes, the windscreen wiper can be delayed to start, and the data tested by experiments are delayed for 1.5-3 minutes. Therefore, in some application scenarios, the sensitivity and coverage of the rainfall sensor are insufficient, so that the terminal (such as an automobile and a robot) where the rainfall sensor is located cannot work effectively, and the normal use and user experience of the terminal are seriously affected. The rainfall state detection device provided by the invention is used for judging whether the rainfall exists or not, so that the situation can be avoided. The rainfall state detection device provided by the invention combines the rainfall state identification model identification result obtained by the training data in the table 1, wherein TPR represents the actual rainfall and the model also detects the rainfall proportion, TNR represents the actual no rainfall and the model also does not detect the rainfall proportion, FPR represents the actual no rainfall and the model detects the rainfall proportion, FNR represents the actual rainfall and the model does not detect the rainfall proportion, and ACC is the comprehensive accuracy. The experimental result shows that the accuracy of the detection result of the detection device provided by the invention is very high and is close to 100%.
Therefore, the present embodiment can reduce the power consumption and area of the rainfall state detection device. In a further preferred embodiment, the audio features extracted by the integrated processor 10 are audio features of 3khz to 10khz of sound, and the audio in the range is divided into several segments, for example, 20 segments, so that the accuracy and precision of rainfall state detection can be further improved. The integrated sensing and storing processor 10 can be realized by a chip, so that the volume and the power consumption of the rainfall state detection device are further reduced.
The rainfall state detection device provided by the invention can detect whether the vehicle rains or not, and also can detect the intensity of the rains, for example, the rainfall state identification model comprises a model of the intensity of the rains, and the intensity of the rains can be identified correspondingly, for example, the rains are small, medium, heavy, and the like, and the corresponding vehicle can trigger the wiper to remove the rainwater on the windshield at different speeds.
In summary, the rainfall state detection device provided by the invention can accurately detect rainfall states, has low detection cost, low power consumption and high efficiency, and further can be suitable for more application scenes and improve user experience. In addition, the invention also provides electronic equipment provided with any rainfall state detection device. Such as automobiles, robots for outdoor work, etc.
The foregoing embodiments of the present application are not limited to the above embodiments, but are intended to be included within the scope of the present application as defined by the appended claims and their equivalents.
In addition, the present application may be identified by the same or different reference numerals for structural elements having the same or similar characteristics. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more features. In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
In the present application, the word "e.g." is used to mean "serving as an example, instance, or illustration". Any embodiment described as "for example" in this disclosure is not necessarily to be construed as preferred or advantageous over other embodiments. The previous description is provided to enable any person skilled in the art to make or use the present application. In the above description, various details are set forth for purposes of explanation.
It will be apparent to one of ordinary skill in the art that the present application may be practiced without these specific details. In other instances, well-known structures and processes have not been shown in detail to avoid unnecessarily obscuring the description of the application. Thus, the present application is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
Claims (10)
1. A sound-based rainfall condition detection device, comprising:
the sound signal acquisition module is used for acquiring an audio signal of the environment;
The sensing and storing integrated processor is used for extracting the audio characteristics of the audio signals of the environment and calculating rainfall states in an inference mode according to the extracted audio characteristics and a neural network model, wherein the neural network model is a rainfall state identification model based on rain sound.
2. The rainfall state detection device of claim 1 wherein the sensory memory integrated processor comprises:
An analog signal processing module based on an analog circuit for extracting audio features of an analog audio signal of an environment;
and the neural network processor is used for inferentially calculating the rainfall state according to the extracted audio characteristics and the rainfall state identification model based on the rain sound.
3. The rainfall state detection device of claim 2 wherein the analog signal processing module comprises:
A low noise amplifier for amplifying an input audio signal according to a fixed gain;
The sound intensity stabilizing circuit is used for compensating sound intensity changes caused by different sound sizes so as to stabilize the intensity of the amplified audio signal;
And the rectification quantization circuit is used for rectifying the stabilized audio signal and quantizing the rectified signal into digital signal pulses so as to form the audio characteristics of the analog audio signal.
4. A rainfall state detection device according to claim 3 wherein the sound intensity stabilization circuit comprises: the gain adjustable amplifier is connected with the output end of the low noise amplifier, and the automatic gain controller is used for controlling the gain of the gain adjustable amplifier.
5. The rainfall state detection device of claim 2 wherein the neural network processor is further configured to:
Acquiring an audio signal training sample and a rainfall state corresponding to the audio signal training sample;
extracting audio characteristics of the audio signal training samples;
Inputting the extracted audio characteristics of the audio signal training sample and the rainfall state corresponding to the audio signal training sample into a neural network model for training to obtain the rainfall state identification model based on the rain sound.
6. The rainfall state detection device of claim 5 wherein the neural network processor is further configured to:
Obtaining a test result according to the audio signal test sample and the rainfall state identification model obtained by training;
judging whether the accuracy and false alarm rate of the rainfall state identification model obtained through training reach a preset accuracy threshold and false alarm rate threshold according to the test result;
And if the accuracy of the rainfall state identification model obtained through training is higher than the preset accuracy threshold and/or the false alarm rate is lower than the false alarm rate threshold, determining the rainfall state identification model obtained through training as a final rainfall state identification model.
7. The rainfall state detection device of claim 6 wherein the neural network processor is further configured to: obtaining a static rainfall identification model when the rainfall state detection device is at rest according to the audio signal test sample, and obtaining a dynamic rainfall identification model when the rainfall state detection device is at dynamic according to the audio signal test sample.
8. The rainfall state detection device according to any one of claims 1 to 7 wherein the audio features comprise a log energy spectrum of mel-like scale corresponding to the frequency of the audio signal.
9. The rainfall state detection device according to any one of claims 1 to 7 wherein the audio features are audio features of sound of 3khz to 10 khz.
10. An electronic device comprising the rainfall state detection device according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410308526.XA CN117908168A (en) | 2024-03-18 | 2024-03-18 | Rainfall state detection device and electronic equipment based on sound |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410308526.XA CN117908168A (en) | 2024-03-18 | 2024-03-18 | Rainfall state detection device and electronic equipment based on sound |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117908168A true CN117908168A (en) | 2024-04-19 |
Family
ID=90689216
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410308526.XA Pending CN117908168A (en) | 2024-03-18 | 2024-03-18 | Rainfall state detection device and electronic equipment based on sound |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117908168A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20020044025A (en) * | 2000-12-05 | 2002-06-14 | 이계안 | Device for detecting rain quantity of an automobile |
| CN111276157A (en) * | 2020-01-21 | 2020-06-12 | 清华大学 | Rainfall intensity recognition and model training method and device based on rain sounds |
| CN115860071A (en) * | 2022-12-08 | 2023-03-28 | 成都市深思创芯科技有限公司 | Neural network oscillator system based on integration of sensing, storage and calculation |
| WO2023247844A1 (en) * | 2022-06-21 | 2023-12-28 | Stellantis Auto Sas | Method and device for detecting rain |
| CN117497004A (en) * | 2024-01-03 | 2024-02-02 | 深圳市九天睿芯科技有限公司 | Noise level monitoring device and method based on neural network and electronic equipment |
-
2024
- 2024-03-18 CN CN202410308526.XA patent/CN117908168A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20020044025A (en) * | 2000-12-05 | 2002-06-14 | 이계안 | Device for detecting rain quantity of an automobile |
| CN111276157A (en) * | 2020-01-21 | 2020-06-12 | 清华大学 | Rainfall intensity recognition and model training method and device based on rain sounds |
| WO2023247844A1 (en) * | 2022-06-21 | 2023-12-28 | Stellantis Auto Sas | Method and device for detecting rain |
| CN115860071A (en) * | 2022-12-08 | 2023-03-28 | 成都市深思创芯科技有限公司 | Neural network oscillator system based on integration of sensing, storage and calculation |
| CN117497004A (en) * | 2024-01-03 | 2024-02-02 | 深圳市九天睿芯科技有限公司 | Noise level monitoring device and method based on neural network and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111507233B (en) | Multi-mode information fusion intelligent vehicle pavement type identification method | |
| CN109949823B (en) | DWPT-MFCC and GMM-based in-vehicle abnormal sound identification method | |
| CN110991289B (en) | Abnormal event monitoring method and device, electronic equipment and storage medium | |
| CN1805008B (en) | Voice detection device, automatic image pickup device and voice detection method | |
| CN108630221B (en) | Audio signal quality enhancement based on quantized SNR analysis and adaptive wiener filtering | |
| CN115052761B (en) | Method and device for detecting tire abnormality | |
| KR101921002B1 (en) | Aircraft noise analyzation system, and method thereof | |
| CN106872018B (en) | Acoustic matrix array apparatus for vehicle testing tire noise power grade | |
| CN108646248A (en) | A kind of passive acoustics for low-speed motion sound source tests the speed distance measuring method | |
| CN112149498A (en) | Online intelligent recognition system and method for abnormal sound of automobile complex part | |
| US10319393B2 (en) | Speech recognition system and gain setting system | |
| CN117908168A (en) | Rainfall state detection device and electronic equipment based on sound | |
| CN206781702U (en) | A kind of speech recognition automotive theft proof system based on quantum nerve network | |
| CN116910690A (en) | Target classification system based on data fusion | |
| CN112230208B (en) | Automobile running speed detection method based on smart phone audio perception | |
| CN116687392A (en) | Clutter elimination method for millimeter wave radar falling detection based on time-frequency information matrix | |
| CN114743562B (en) | Method and system for recognizing airplane voiceprint, electronic equipment and storage medium | |
| CN115980690A (en) | Clear sky echo recognition model construction method, clear sky echo filtering method and clear sky echo filtering device | |
| JP5480009B2 (en) | Noise measurement device | |
| CN114783462A (en) | Mine hoist fault source positioning analysis method based on CS-MUSIC | |
| CN115575508A (en) | Rail transit rail corrugation identification method based on train vibration and sound composite characteristics | |
| Astapov et al. | A method of real-time mobile vehicle identification by means of acoustic noise analysis implemented on an embedded device | |
| Harlow et al. | Acoustic accident detection system | |
| CN116699521B (en) | Urban noise positioning system and method based on environmental protection | |
| CN117150265B (en) | Robust radio frequency signal open set identification method under low signal-to-noise ratio condition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |