WO2021134409A1

WO2021134409A1 - Underwater detection method, device, and storage medium

Info

Publication number: WO2021134409A1
Application number: PCT/CN2019/130348
Authority: WO
Inventors: 边云锋; 吴晟; 赵文泉; 莫品西; 薛政
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-12-31
Filing date: 2019-12-31
Publication date: 2021-07-08

Abstract

An underwater detection method, a device, and a storage medium. The method is applied to an electronic device which is provided with a microphone (61). The method comprises: extracting, according to an ambient sound signal of the environment where the electronic device is located collected by the microphone (61), a frequency-domain feature of the ambient sound signal (101); and determining whether the environment where the electronic device is located is underwater environment according to the frequency-domain feature (102). According to the method, whether the environment is the underwater environment by means of the frequency-domain feature of the ambient sound signal collected by the microphone (61), and the detection result is relatively accurate.

Description

Water ingress detection method, equipment and storage medium

Technical field

The embodiments of the present application relate to the technical field of electronic devices, and in particular, to a water ingress detection method, device, and storage medium.

Background technique

With the widespread application of electronic devices, users have higher and higher performance requirements for electronic devices, and an underwater scene is a typical scene. Underwater applications (such as underwater detection or photography, underwater equipment protection, etc.) will become more and more popular. Therefore, it is necessary to test the state of the equipment before and after entering the water to confirm whether the equipment is in the water, so as to develop different conditions for different situations. Application or mode, such as underwater camera mode, etc.

In related technologies, the following methods are generally used to detect: one is to add additional devices to detect underwater scenes, such as adding pressure sensors, adding optical devices and optical water entry detection, adding thermistors and other thermodynamic water entry Detection etc. This method will increase the cost of electronic equipment. The other is to perform scene detection based on existing devices using the physical laws of underwater scenes, such as detecting whether communication signals can be transmitted normally, but some communication signals (such as Bluetooth) can still be transmitted normally in shallow water, and the detection results will be biased.

Summary of the invention

The present application provides a method, equipment and storage medium for water ingress detection, which improves the accuracy of water ingress detection.

In a first aspect, the present application provides a water entry detection method, which is applied to an electronic device, the electronic device is provided with a microphone, and the method includes:

Collecting, according to the microphone, the environmental sound signal of the environment in which the electronic device is located, and extracting the frequency domain characteristics of the environmental sound signal;

According to the frequency domain characteristics, it is determined whether the environment in which the electronic device is located is an underwater environment.

In the second aspect, this application provides a control device, including: a microphone and a processor;

Wherein, the microphone is used to collect environmental sound signals;

The processor is configured as:

In a third aspect, the present application provides a storage medium, including: a readable storage medium and a computer program, the computer program being used to implement the water ingress detection method provided by any one of the above-mentioned embodiments of the first aspect.

In a fourth aspect, the present application provides a program product. The program product includes a computer program (that is, an execution instruction), and the computer program is stored in a readable storage medium. The processor can read the computer program from a readable storage medium, and the processor executes the computer program to execute the water ingress detection method provided by any one of the embodiments of the first aspect described above.

This application provides a water entry detection method, equipment and storage medium, which extracts the frequency domain characteristics of the environmental sound signal according to the environmental sound signal of the environment in which the electronic device is located by the microphone; due to the environment that the microphone collects in the water environment and the underwater environment The frequency domain characteristics of the sound signal have obvious differences. Therefore, according to the frequency domain characteristics extracted from the environmental sound signal, it is determined whether the environment in which the electronic device is located is an underwater environment, and the accuracy rate is high.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

Figure 1 is a scene diagram provided by an embodiment of the application;

FIG. 2 is a flowchart of a water entry detection method provided by an embodiment of the application;

FIG. 3 is a schematic block diagram of a water entry detection method provided by an embodiment of this application;

FIG. 4 is a flowchart of a water entry detection method provided by another embodiment of this application;

FIG. 5 is a flowchart of a water entry detection method provided by another embodiment of this application;

FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the application;

FIG. 7 is a schematic structural diagram of an electronic device provided by another embodiment of the application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

First, the application scenarios involved in this application are introduced:

The water entry detection method provided in the embodiments of the present application is applied to electronic equipment. It can detect whether the electronic equipment is in an underwater environment, and then control the electronic equipment to perform corresponding operations or adjust the electronic equipment according to the underwater environment in which the electronic equipment is located. Status or corresponding parameters, such as imaging parameters, etc.

The electronic device may be any electronic device capable of working underwater, for example, including wearable devices, camera equipment, underwater measuring equipment, etc., and wearable devices, for example, include smart bracelets, smart watches, smart glasses, and the like. The electronic device is waterproof.

The water entry detection method of the embodiment of the present application uses the obvious difference in the frequency domain characteristics of the environmental sound signal collected by the microphone in the water and in the air (that is, the frequency domain characteristics of the environmental noise) to determine whether the environment in which the device is located is an underwater environment, Compared with the current underwater scene detection scheme, the microphones contained in most electronic devices can be reused, thereby reducing equipment design costs and hardware costs, easy implementation, and high accuracy of detection results.

The method in the embodiment of the present application may be executed by an electronic device, and the electronic device includes a microphone. For example, it can be implemented by an electronic device such as a processor of the electronic device executing corresponding software code. In some embodiments, the methods of the embodiments of the present application may be executed by a device communicatively connected with the electronic device. For example, the method may be a control terminal that communicates with an electronic device. As shown in FIG. 1, the server is in communication connection with the electronic device, and the method of this application may be implemented by the server. In some embodiments, the method may be executed jointly by an electronic device and a device communicatively connected with the electronic device.

In an optional embodiment, the electronic device may include an imaging component, where the imaging component is used to obtain image or video data.

The technical solutions of the present disclosure will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

Fig. 2 is a flowchart of a water entry detection method provided by an embodiment of the application. As shown in Figures 2 and 3, the method provided in this embodiment is applied to an electronic device that is provided with a microphone, and the method includes:

Step 101: According to the microphone collecting the environmental sound signal of the environment in which the electronic device is located, extract the frequency domain characteristics of the environmental sound signal.

Specifically, the microphone collects the environmental sound signal of the environment in which the electronic device is located, and extracts the frequency domain characteristics of the environmental sound signal.

In an optional embodiment, the environmental sound signal collected by the microphone may be sampled and processed to obtain a time-discrete digital sound signal, and then the digital sound signal may be subjected to frequency domain conversion processing, and frequency domain features may be extracted. For example, the frequency domain feature is extracted according to the amplitude of the signal after frequency domain conversion processing.

Step 102: Determine whether the environment in which the electronic device is located is an underwater environment according to the frequency domain characteristics.

Specifically, since the frequency domain characteristics of the environmental sound signals collected by the aquatic environment and the underwater environment are significantly different, it can be determined whether the environment in which the electronic device is located is an underwater environment according to the frequency domain characteristics of the environmental sound signal.

In an optional embodiment, the frequency domain characteristics of the environmental sound signal may be compared with the frequency domain characteristics of the water environment and the frequency domain characteristics of the underwater environment, if the difference between the frequency domain characteristics of the underwater environment and the frequency domain characteristics of the underwater environment is less than that of The difference in the frequency domain characteristics of the aquatic environment can determine that the environment in which the electronic device is located is an underwater environment.

In an optional embodiment, the frequency domain characteristics of the environmental sound signal can be input into a pre-trained machine learning model, and the output result of the model is the label of the environment type in which the electronic device is located, or The machine learning model is established, for example, through a neural network model, a support vector machine classification algorithm model, and a clustering algorithm. Among them, the neural network model is, for example, CNN, VGG, GoogleNet, etc.

In an optional embodiment, this step can be implemented by a judgment function, for example, respectively judging the frequency domain feature and the frequency domain feature of the underwater environment obtained a priori, and the difference with the frequency domain feature of the water environment.

In an embodiment, the electronic device may further include an imaging component, and the method further includes:

When it is determined that the imaging component is in an underwater environment, the imaging parameters of the imaging component are adjusted.

Among them, the imaging parameters include, for example, exposure time, exposure gain, and so on. When taking pictures or videos in an underwater environment, the imaging parameters can be adjusted to adapt to the scene of the underwater environment.

The method of this embodiment extracts the frequency domain characteristics of the environmental sound signal according to the environmental sound signal of the environment in which the electronic device is collected by the microphone; because the frequency domain characteristics of the environmental sound signal collected by the microphone in the water environment and the underwater environment are obvious Therefore, according to the frequency domain characteristics extracted from the environmental sound signal, it is determined whether the environment in which the electronic device is located is an underwater environment, and the accuracy rate is high.

On the basis of the foregoing embodiment, further, as shown in FIG. 4, step 101 can be implemented in the following manner:

Step 1011: Perform sampling processing on the environmental sound signal to obtain a group of digital sound signals of the current working cycle, where the group of digital sound signals includes at least one first digital sound signal subframe;

Step 1012: Perform frequency domain transformation on each first digital sound signal subframe to obtain a frequency domain amplitude spectrum of each first digital sound signal subframe;

Step 1013: Extract the frequency domain characteristic of the current working cycle according to the frequency domain amplitude spectrum of each second digital sound signal subframe, where the second digital sound signal subframe is the time domain characteristic amplitude in the first digital sound signal subframe The digital sound signal sub-frame whose value is less than the environmental noise judgment threshold.

Specifically, the environmental sound signal is sampled, that is, the environmental sound signal is converted into a time-discrete digital signal to obtain a group of digital sound signals, where the group of digital sound signals includes at least one first digital sound signal subframe.

Suppose the time discrete digital signal is x(t), where t is the discrete time index, and the converted sampling rate is fs; each interval L (obtained according to the sampling rate fs) samples x(t) to form K frames with a length of N , The vector x(n) with a frame interval of M, where K>=1, M>=1, x(n)=[x ₁ (n), x ₂ (n),...,x _K (n)], Where x _k (n) is the first digital sound signal subframe with a frame length of N, where each first digital sound signal subframe includes N elements. Among them, n represents the working period of frequency domain feature extraction.

Let x(n,k,i)=x(n×L-N+i-(k-1)×M), then x _k (n) can be abbreviated as:

x _k (n)=[x(n,k,i),x(n,k,i+1),……,x(n,k,i+N-1)]

k is the sequence number of the subframe, where k=1, 2,...K; i is the sequence number of the element in the subframe.

Further, perform frequency domain transformation on each first digital sound signal subframe, such as Fourier transform, wavelet transform, etc., to obtain the frequency domain amplitude spectrum of each first digital sound signal subframe, and filter out the first digital sound signal The second digital sound signal sub-frame in the sub-frame whose time-domain characteristic amplitude is less than the environmental noise judgment threshold.

Among them, the Fourier transform uses, for example, a short-time windowed Fourier transform.

Wherein, the temporal characteristic amplitude can be obtained, for example, by averaging the amplitude of each element of each first digital sound signal subframe, or can also be based on the median value of the amplitude of each element of the first digital sound signal subframe, Or the mode of the amplitude of each element of the first digital sound signal subframe, which is not limited in this application.

In an embodiment, the absolute values of the elements x(n,k,i) in _{the K subframes x k} (n) are respectively summed and then averaged to obtain the temporal characteristic amplitude of _{each subframe x k (n),} For example, the following formula can be used to calculate the time domain characteristic amplitude:

Wherein, the environmental noise judgment threshold may be obtained according to the temporal characteristic amplitudes of the K first digital sound signal subframes.

Further, according to the frequency domain amplitude spectrum of each second digital sound signal subframe, the frequency domain characteristic of the current working period is extracted.

Perform frequency domain transformation on K subframes x _k (n) to obtain K frequency domain spectrum vectors X ₁ (n),..., X _K (n) with length N; take the absolute value operation to obtain each subframe x _k ( The frequency domain amplitude spectrum of _{n) |X 1} (n)|,...,|X _K (n)|.

In an embodiment, if a group of digital sound signals includes a plurality of first digital sound signal subframes, the average value of the frequency domain amplitude spectrum of the second digital sound signal subframe is used as the frequency domain characteristic of the current working cycle;

If a group of digital sound signals includes a first digital sound signal subframe, the frequency domain amplitude spectrum of the second digital sound signal subframe is used as the frequency domain characteristic of the current period.

If K is greater than 1, calculate the average value of the frequency domain amplitude spectrum of the P second digital sound signal subframes in the K subframes whose amplitudes in the time domain are less than the environmental noise judgment threshold Nthr _n _{as the frequency domain characteristic N n} ; for example The frequency domain characteristics can be calculated using the following formula:

Wherein Num(SegEng(k) _n <Nthr _n ) represents the number of second digital sound signal sub-frames whose time domain feature amplitude is less than the environmental noise judgment threshold, and frequency domain feature N _n is a vector of length N.

If K is equal to 1, and the _{time domain characteristic amplitude of x k} (n) is less than the environmental noise judgment threshold, then _{frequency domain transformation is performed on the subframe x k} (n) to obtain a frequency domain spectrum vector X( n); Calculate the absolute value of X(n) as _{the frequency domain feature of the subframe x k} (n), for example, the following formula can be used to calculate the frequency domain feature:

N _n = |X(n)|

In an embodiment, if a group of digital sound signals includes a first digital sound signal sub-frame, and the amplitude of the first digital sound signal sub-frame is greater than or equal to the environmental noise judgment threshold, then the historical work before the current work cycle The frequency domain feature extracted by the period determines the frequency domain feature of the current working period.

If K is equal to 1, and the _{time domain feature amplitude of x k} (n) is greater than or equal to the environmental noise judgment threshold, the frequency domain feature N _{n will} not be updated, and the current working cycle can be obtained from the frequency domain features extracted from the historical working cycle The frequency domain feature of, for example, the frequency domain feature extracted from the previous historical working cycle of the current working cycle is used as the frequency domain feature of the current working cycle, that is, N _n =N _n-1 is set.

Optionally, the frequency domain features can be smoothed.

In the above, β and γ are smoothing coefficients, and the value range is: 0≤β, γ≤1.

Further, compare the extracted frequency domain features N _n with the frequency domain features of the prior environment (the frequency domain features of the water environment N _normal , the frequency domain features of the underwater environment N _Water ) to determine the environment where the equipment is located Whether it is an underwater environment.

In the above-mentioned specific embodiments, the frequency domain amplitude spectrum is obtained by performing frequency domain transformation on the subframes of the environmental sound signal, and the frequency domain characteristics are extracted from the subframes whose time domain characteristic amplitude is less than the environmental noise judgment threshold, that is, the environment is extracted. The frequency domain characteristics of the noise, based on the frequency domain characteristics to determine whether it is in an underwater environment, the accuracy is high.

In an embodiment, if a group of digital sound signals includes a plurality of first digital sound signal sub-frames, the time domain characteristic amplitude of each sub-frame of the plurality of first digital sound signal sub-frames is determined, based on the plurality of first digital sound signal sub-frames. The time domain characteristic amplitude value of a digital sound signal subframe obtains the environmental noise judgment threshold value of the current working cycle;

If a group of digital sound signals includes a first digital sound signal sub-frame, determine the time domain characteristic amplitude in a first digital sound signal sub-frame and the first digital signal sub-frame of the historical working period before the current working period. The time-domain characteristic amplitude, based on the time-domain characteristic amplitude of a first digital sound signal sub-frame and the time-domain characteristic amplitude of the first digital signal sub-frame of the historical working period before the current working cycle, to obtain the current working cycle Environmental noise judgment threshold.

In an optional embodiment, for a plurality of first digital sound signal subframes, the minimum time domain characteristic amplitude of the plurality of first digital sound signal subframes may be determined, and the minimum time domain characteristic amplitude can be obtained according to the minimum time domain characteristic amplitude. The environmental noise judgment threshold of the current working cycle.

In an optional embodiment, the environmental noise judgment threshold of the current working cycle may be determined in the following manner:

Nthr _n = μ1*min(SegEng(1) _n ,SegEng(2) _n ,...,SegEng(K) _n )

Among them, μ1 is the adjustment parameter, μ1≥1, min() is the minimum value operation;

In an optional embodiment, for a first digital sound signal sub-frame, the time domain characteristic amplitude in the first digital sound signal sub-frame and the first historical working period before the current working period can be determined. The minimum time-domain characteristic amplitude among the time-domain characteristic amplitudes of a digital signal subframe, and the environmental noise judgment threshold value of the current working cycle is obtained according to the minimum time-domain characteristic amplitude.

Nthr _n = μ2*min(SegEng _n , SegEng _n-1 ,...SegEng _nK ), where μ is the adjustment parameter, μ2≥1, min() is the minimum operation.

Among them, SegEng _n-1 ,...SegEng _nK are the time domain characteristic amplitudes of the historical work cycle.

Among them, μ1 and μ2 can be the same or different.

In an embodiment, for the environmental noise judgment threshold, smoothing processing may also be performed:

Nthr _n =α×Nthr _n +(1-α)×Nthr _n-1

Among them, α is the adjustment parameter, 0≤α≤1; Nthr _n-1 is the environmental noise judgment threshold value of the previous working cycle of the current working cycle.

In a possible implementation manner, as shown in FIG. 5, step 102 can be implemented in the following ways:

One way to achieve:

Step 1021a: Obtain an environmental frequency domain characteristic threshold, where the environmental frequency domain characteristic threshold is determined according to at least one of the frequency domain characteristic threshold of the underwater environment and the frequency domain characteristic threshold of the aquatic environment;

Step 1022a: Determine whether the environment in which the electronic device is located is an underwater environment according to the environmental frequency domain characteristic threshold and the frequency domain characteristic.

Specifically, the electronic equipment can be placed in the aquatic environment and the underwater environment respectively, and by continuously collecting the environmental sound signal for a period of time T, the frequency domain feature vector of the environmental sound signal in different environments can be obtained

Among them, fs is the sampling rate, L is the sampling interval, and n is the working period. The frequency domain characteristic thresholds of the corresponding a priori environmental sound signals in the aquatic environment and the underwater environment are denoted as N _Normal and N _water , where N _Normal and N _water are vectors of length N respectively.

According to the difference in the frequency domain characteristics of the environmental sound signals of different environments, that is, at least one of the frequency domain characteristic threshold of the aquatic environment and the frequency domain characteristic threshold of the underwater environment is used to determine whether the environment in which the electronic device is located is an underwater environment .

In an optional embodiment, the difference between the frequency domain characteristics of the environmental sound signal acquired in the current working cycle and the frequency domain characteristic threshold of the underwater environment is calculated, and if the difference is less than the preset threshold, the electronic device is determined The environment is underwater. Among them, the difference can be obtained by calculation methods such as Euclidean distance. If the difference is greater than or equal to the preset threshold, it is determined that the environment where the electronic device is located is not an underwater environment.

In an optional embodiment, the difference between the frequency domain characteristics of the environmental sound signal acquired in the current working cycle and the frequency domain characteristic threshold of the marine environment is calculated, and if the difference is greater than the preset threshold, the location of the electronic device is determined. The environment is underwater. Among them, the difference can be obtained by calculation methods such as Euclidean distance. If the difference is less than or equal to the preset threshold, it is determined that the environment where the electronic device is located is not an underwater environment.

In an optional embodiment, the difference between the frequency domain characteristic of the environmental sound signal acquired in the current working cycle and the frequency domain characteristic threshold of the aquatic environment and the frequency domain characteristic threshold of the underwater environment is calculated separately, if a certain difference is If the absolute value of is less than the preset threshold, the environment corresponding to the difference can be regarded as the environment where the electronic device is currently located. For example, the absolute value of the difference between the current frequency domain feature and the frequency domain feature threshold of the underwater environment is less than the preset Threshold, it is determined that the environment in which the electronic device is currently located is an underwater environment.

If the absolute value of the two differences is less than the preset threshold, the absolute value of the two differences can be further compared, and the environment corresponding to the absolute value of the smaller difference is determined as the current environment of the electronic device .

In an optional embodiment, the environmental frequency domain characteristic threshold includes the frequency domain characteristic threshold of the underwater environment and the frequency domain characteristic threshold of the aquatic environment, and this step 102 may be implemented in the following manner:

Determining a first frequency domain characteristic difference value, where the first frequency domain characteristic difference value is determined according to a difference between a frequency domain characteristic and a frequency domain characteristic threshold of the underwater environment;

Determining a second frequency domain characteristic difference value, where the first frequency domain characteristic difference value is determined according to a difference between a frequency domain characteristic and a frequency domain characteristic threshold of the marine environment;

When the first frequency domain characteristic difference value is less than or equal to the second frequency domain characteristic difference value, it is determined that the environment where the electronic device is located is an underwater environment.

Specifically, calculate the distance dis_normal between the frequency domain feature N _n and the frequency domain feature threshold N _Normal of the aquatic environment, and the distance dis_water from the frequency domain feature threshold N _water of the underwater environment. If dis_water is less than dis_normal, it is considered that it is currently Underwater environment, otherwise it is in water environment, update the current electronic equipment status according to the determination result. N _n , N _Normal and N _water are frequency domain feature vectors of length N respectively, and the distance can be calculated as follows:

Among them, N _n =(N _n (1),N _n (2),...N _n (N)); N _Normal =(N _Normal (1),N _Normal (2),...N _Normal (N); N _water = (N _water (1), N _water (2),...N _water (N)).

In this embodiment, distance is used to classify the environment, and other methods can also be used to classify the environment, including cluster analysis, adaptive learning and other methods for classification. For example, clustering is performed on the acquired frequency domain features, the pre-acquired frequency domain features of the underwater environment, and the frequency domain features of the water environment to determine whether the environment in which the electronic device is located is an underwater environment.

In the above manner, the implementation of the solution is easy, the detection accuracy is high, the detection speed is fast, and the robustness is good.

Another way to achieve:

Step 1021b: Input the frequency domain features into the preset environment discrimination model to obtain the environment type label output by the preset environment discrimination model, where the preset environment discrimination model is based on the frequency domain characteristics of the underwater environment and the water environment Optimized frequency domain characteristics;

Step 1022b: Determine whether the environment in which the electronic device is located is an underwater environment according to the environment type label.

Specifically, the preset environment discrimination model can be optimized through the frequency domain characteristics of the aquatic environment and the frequency domain characteristics of the underwater environment, that is, collecting a large number of environmental sound signals of the aquatic environment and the aquatic environment in different scenarios, and extracting the corresponding environment The frequency domain characteristics of, input to the preset environment discrimination model to optimize the parameters of the environment discrimination model.

Among them, the environment discrimination model can be established based on neural networks, support vector machine classification algorithm models, clustering algorithms, and so on.

Input the currently extracted frequency domain features of the electronic device into the optimized environment discrimination model to obtain the output environment type label, and then determine whether the environment in which the electronic device is located is an underwater environment according to the environment type label. For example, if the output environment type label is an underwater environment label, it is determined that the environment in which the electronic device is located is an underwater environment.

If the output environment type label is a water environment label, it is determined that the environment in which the electronic device is located is a water environment.

In other embodiments, the environment discrimination model may also output the probability of the environment type, for example, the final environment is determined according to the magnitude of the probability.

In an embodiment, the preset environment discrimination model is a neural network model, and the neural network is trained on the frequency domain characteristics of the underwater environment and the frequency domain characteristics of the water environment.

In an embodiment, the preset environment discrimination model is a support vector machine classification algorithm model.

Specifically, the electronic equipment is placed in different environments multiple times in advance to obtain multiple sets of priori frequency domain features of the environmental sound signal. The frequency domain feature set of the marine environment is A={N _Normal (1), N _Normal ( 2)...N _Normal (l1)}, the label of the set A is y _i =+1, i∈[1,2,..,l1]; the frequency domain feature set of the underwater environment is B={N _water (1 ), N _water (2)...N _water (l2)}, the label of set B is y _j = -1, i∈[1,2,..,l2]. l1,l2 are generally set to be greater than 10, and all data are randomly mixed, T={(x ₁ ,y ₁ ),(x ₂ ,y ₂ ),...,(x _N ,y _N )}, N=l1+ l2, x _i refers to N _Normal (i) or N _water (i).

First, select an appropriate kernel function K (for example, the kernel function is a Gaussian kernel function) and appropriate parameters C, construct and solve the optimization problem:

Find the optimal solution

Secondly, select a positive component of ^{α *} _{0＜α j} ＜C, and calculate

Again, construct the decision function

sign represents a sign function.

Finally, take the frequency domain characteristics of the environmental sound signal extracted in the current working period as x, and input the above decision function f(x). When the output is +1, it is judged as an aquatic environment, and when the output is -1, it is judged as underwater surroundings.

The microphone-based water entry detection method in the embodiment of the present application uses the obvious difference between the frequency domain characteristics of the environmental sound signal of the microphone in the water and the air (that is, the frequency domain characteristics of the environmental noise) for comparison and discrimination, and determines the environment in which the electronic device is located. , With this solution, the microphones contained in most electronic devices can be reused, and the function of rapid water entry detection is achieved without increasing other costs. It has easy implementation, low equipment requirements, high detection accuracy, and high detection speed. advantage. Moreover, different implementations of the solution can be deployed and implemented according to different electronic devices and different software and hardware platforms.

FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the application. The electronic device provided in this embodiment is used to implement the water ingress detection method provided in any one of the embodiments shown in FIG. 2 to FIG. 5. As shown in FIG. 6, the electronic device provided in this embodiment may include: a microphone 61 and a processor 62. Among them, the microphone 61 is used to collect environmental sound signals; the processor 62 is configured as:

In a possible implementation manner, the processor 62 is configured to:

Performing sampling processing on the environmental sound signal to obtain a group of digital sound signals of the current working cycle, wherein the group of digital sound signals includes at least one first digital sound signal subframe;

Performing frequency domain transformation on each of the first digital sound signal subframes to obtain a frequency domain amplitude spectrum of each of the first digital sound signal subframes;

According to the frequency domain amplitude spectrum of each of the second digital sound signal subframes, the frequency domain characteristics of the current working period are extracted, wherein the second digital sound signal subframe is the first digital sound signal subframe Digital sound signal sub-frames whose mid-time domain feature amplitude is less than the environmental noise judgment threshold.

In a possible implementation manner, the processor 62 is configured to:

If the set of digital sound signals includes a plurality of first digital sound signal subframes, the average value of the frequency domain amplitude spectrum of the second digital sound signal subframe is used as the frequency domain characteristic of the current working period;

If the set of digital sound signals includes a first digital sound signal subframe, the frequency domain amplitude spectrum of the second digital sound signal subframe is used as the frequency domain characteristic of the current period.

In a possible implementation manner, the processor 62 is configured to:

If the set of digital sound signals includes a first digital sound signal sub-frame, and the amplitude of the first digital sound signal sub-frame is greater than or equal to the environmental noise judgment threshold, then according to the current working period before The frequency domain feature extracted from the historical work cycle determines the frequency domain feature of the current work cycle.

In a possible implementation manner, the processor 62 is configured to:

If the set of digital sound signals includes a plurality of first digital sound signal subframes, the time domain characteristic amplitude of each subframe of the plurality of first digital sound signal subframes is determined, based on the plurality of first digital sound signal subframes. The time domain characteristic amplitude value of the sound signal subframe obtains the environmental noise judgment threshold value of the current working period;

If the set of digital sound signals includes a first digital sound signal sub-frame, determine the time domain characteristic amplitude in the one first digital sound signal sub-frame and the first historical working period before the current working period The time domain characteristic amplitude of a digital signal subframe is based on the time domain characteristic amplitude of the one first digital sound signal subframe and the time of the first digital signal subframe of the historical working period before the current working period The domain characteristic amplitude is used to obtain the environmental noise judgment threshold of the current working period.

In a possible implementation manner, the processor 62 is configured to:

Acquiring an environment frequency domain feature threshold, where the environment frequency domain feature threshold is determined according to at least one of the frequency domain feature threshold of the underwater environment and the frequency domain feature threshold of the aquatic environment;

Determine whether the environment in which the electronic device is located is an underwater environment according to the environmental frequency domain characteristic threshold and the frequency domain characteristic.

In a possible implementation, the environmental frequency domain characteristic threshold includes the frequency domain characteristic threshold of the underwater environment and the frequency domain characteristic threshold of the marine environment, and the processor 62 is configured to:

Determining a first frequency domain characteristic difference value, wherein the first frequency domain characteristic difference value is determined according to a difference value between the frequency domain characteristic and a frequency domain characteristic threshold value of the underwater environment;

Determining a second frequency domain characteristic difference value, wherein the first frequency domain characteristic difference value is determined according to a difference value between the frequency domain characteristic and the frequency domain characteristic threshold of the marine environment;

When the first frequency domain characteristic difference value is less than or equal to the second frequency domain characteristic difference value, it is determined that the environment where the electronic device is located is the underwater environment.

In a possible implementation manner, the processor 62 is configured to:

Input the frequency domain features into a preset environment discrimination model to obtain the environment type label output by the preset environment discrimination model, wherein the preset environment discrimination model is based on the frequency domain characteristics of the underwater environment and the water Optimized the frequency domain characteristics of the environment;

Determine whether the environment in which the electronic device is located is an underwater environment according to the environment type label.

In a possible implementation manner, the preset environment discrimination model is a neural network model.

In a possible implementation manner, the preset environment discrimination model is a support vector machine classification algorithm model.

In a possible implementation manner, as shown in FIG. 7, the electronic device may further include an imaging component 63 for shooting images or videos, and optionally, a memory for storing instructions executable by the processor. And/or image or video data captured by the imaging component. The processor 62 is configured as:

The electronic device provided in this embodiment is used to implement the water ingress detection method provided in any of the embodiments in FIG. 2 to FIG. 5. The technical principles and technical effects are similar, and will not be repeated here.

The embodiments of the present application also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the corresponding method in the foregoing method embodiment is implemented. For the specific implementation process, please refer to the foregoing method implementation. For example, the implementation principles and technical effects are similar, so I won’t repeat them here.

The embodiments of the present application also provide a program product. The program product includes a computer program (that is, an execution instruction), and the computer program is stored in a readable storage medium. The processor can read the computer program from a readable storage medium, and the processor executes the computer program to execute the water entry detection method provided by any one of the foregoing method embodiments.

A person of ordinary skill in the art can understand that all or part of the steps in the foregoing method embodiments can be implemented by a program instructing relevant hardware. The aforementioned program can be stored in a computer readable storage medium. When the program is executed, it executes the steps including the foregoing method embodiments; and the foregoing storage medium includes: ROM, RAM, magnetic disk, or optical disk and other media that can store program codes.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present application, not to limit them; although the embodiments of the present application are described in detail with reference to the foregoing embodiments, those of ordinary skill in the art It should be understood that: it can still modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the embodiments of this application. The scope of the technical solution.

Claims

A water ingress detection method, characterized in that it is applied to an electronic device, the electronic device is provided with a microphone, and the method includes:

Extracting the frequency domain characteristics of the environmental sound signal according to the environmental sound signal of the environment in which the electronic device is located collected by the microphone;

According to the frequency domain characteristics, it is determined whether the environment in which the electronic device is located is an underwater environment.
The method according to claim 1, wherein the collecting, according to the microphone, the environmental sound signal of the environment in which the electronic device is located, and extracting the frequency domain characteristics of the environmental sound signal, comprises:

Performing sampling processing on the environmental sound signal to obtain a group of digital sound signals of the current working cycle, wherein the group of digital sound signals includes at least one first digital sound signal subframe;

Performing frequency domain transformation on each of the first digital sound signal subframes to obtain a frequency domain amplitude spectrum of each of the first digital sound signal subframes;

According to the frequency domain amplitude spectrum of each second digital sound signal subframe, the frequency domain characteristics of the current working cycle are extracted, wherein the second digital sound signal subframe is the time in the first digital sound signal subframe. The sub-frame of the digital sound signal whose domain feature amplitude is less than the environmental noise judgment threshold.
The method according to claim 2, wherein the extracting the frequency domain characteristic of the current working period according to the frequency domain amplitude spectrum of the second digital sound signal subframe comprises:

If the set of digital sound signals includes a plurality of first digital sound signal subframes, the average value of the frequency domain amplitude spectrum of the second digital sound signal subframe is used as the frequency domain characteristic of the current working period;

If the set of digital sound signals includes a first digital sound signal subframe, the frequency domain amplitude spectrum of the second digital sound signal subframe is used as the frequency domain characteristic of the current period.
The method according to claim 3, further comprising:

If the set of digital sound signals includes a first digital sound signal sub-frame, and the amplitude of the first digital sound signal sub-frame is greater than or equal to the environmental noise judgment threshold, then according to the current working period before The frequency domain feature extracted from the historical duty cycle determines the frequency domain feature of the current duty cycle.
The method according to any one of claims 2-4, further comprising:

If the set of digital sound signals includes a plurality of first digital sound signal subframes, the time domain characteristic amplitude of each subframe of the plurality of first digital sound signal subframes is determined, based on the plurality of first digital sound signal subframes. The time domain characteristic amplitude value of the sound signal subframe obtains the environmental noise judgment threshold value of the current working period;

If the set of digital sound signals includes a first digital sound signal sub-frame, determine the time domain characteristic amplitude in the one first digital sound signal sub-frame and the first historical working period before the current working period The time domain characteristic amplitude of a digital signal subframe is based on the time domain characteristic amplitude of the one first digital sound signal subframe and the time of the first digital signal subframe of the historical working period before the current working period The domain characteristic amplitude is used to obtain the environmental noise judgment threshold of the current working period.
The method according to any one of claims 1 to 5, wherein the determining whether the environment in which the electronic device is located is an underwater environment according to the frequency domain characteristics comprises:

Acquiring an environment frequency domain feature threshold, where the environment frequency domain feature threshold is determined according to at least one of the frequency domain feature threshold of the underwater environment and the frequency domain feature threshold of the aquatic environment;

Determine whether the environment in which the electronic device is located is an underwater environment according to the environmental frequency domain characteristic threshold and the frequency domain characteristic.
The method according to claim 6, wherein the environmental frequency domain characteristic threshold includes a frequency domain characteristic threshold of an underwater environment and a frequency domain characteristic threshold of an aquatic environment, wherein the threshold is based on the frequency domain characteristic of the environment And the frequency domain characteristics to determine whether the environment in which the electronic device is located is an underwater environment, including:

Determining a first frequency domain characteristic difference value, wherein the first frequency domain characteristic difference value is determined according to a difference value between the frequency domain characteristic and a frequency domain characteristic threshold value of the underwater environment;

Determining a second frequency domain characteristic difference value, wherein the first frequency domain characteristic difference value is determined according to a difference value between the frequency domain characteristic and the frequency domain characteristic threshold of the marine environment;

When the first frequency domain characteristic difference value is less than or equal to the second frequency domain characteristic difference value, it is determined that the environment where the electronic device is located is the underwater environment.
The method according to any one of claims 1 to 7, wherein the determining whether the environment in which the electronic device is located is an underwater environment according to the frequency domain characteristics comprises:

Input the frequency domain features into a preset environment discrimination model to obtain the environment type label output by the preset environment discrimination model, wherein the preset environment discrimination model is based on the frequency domain characteristics of the underwater environment and the water Optimized the frequency domain characteristics of the environment;

Determine whether the environment in which the electronic device is located is an underwater environment according to the environment type tag.
The method according to claim 8, wherein the preset environment discrimination model is a neural network model.
The method according to claim 8, wherein the preset environment discrimination model is a support vector machine classification algorithm model.
The method according to any one of claims 1-10, wherein the electronic device comprises an imaging component, and the method further comprises:

When it is determined that the imaging component is in an underwater environment, the imaging parameters of the imaging component are adjusted.
An electronic device, characterized by comprising: a microphone and a processor;

The microphone is used to collect environmental sound signals;

The processor is configured as:

Collecting, according to the microphone, the environmental sound signal of the environment in which the electronic device is located, and extracting the frequency domain characteristics of the environmental sound signal;

According to the frequency domain characteristics, it is determined whether the environment in which the electronic device is located is an underwater environment.
The electronic device according to claim 12, wherein the processor is configured to:

Performing sampling processing on the environmental sound signal to obtain a group of digital sound signals of the current working cycle, wherein the group of digital sound signals includes at least one first digital sound signal subframe;

Performing frequency domain transformation on each of the first digital sound signal subframes to obtain a frequency domain amplitude spectrum of each of the first digital sound signal subframes;

According to the frequency domain amplitude spectrum of each second digital sound signal subframe, the frequency domain characteristics of the current working cycle are extracted, wherein the second digital sound signal subframe is the time in the first digital sound signal subframe. The sub-frame of the digital sound signal whose domain feature amplitude is less than the environmental noise judgment threshold.
The electronic device according to claim 13, wherein the processor is configured to:

If the set of digital sound signals includes a plurality of first digital sound signal subframes, the average value of the frequency domain amplitude spectrum of the second digital sound signal subframe is used as the frequency domain characteristic of the current working period;

If the set of digital sound signals includes a first digital sound signal subframe, the frequency domain amplitude spectrum of the second digital sound signal subframe is used as the frequency domain characteristic of the current period.
The electronic device according to claim 14, wherein the processor is configured to:

If the set of digital sound signals includes a first digital sound signal sub-frame, and the amplitude of the first digital sound signal sub-frame is greater than or equal to the environmental noise judgment threshold, then according to the current working period before The frequency domain feature extracted from the historical work cycle determines the frequency domain feature of the current work cycle.
The electronic device according to any one of claims 13-15, wherein the processor is configured to:

If the set of digital sound signals includes a plurality of first digital sound signal subframes, the time domain characteristic amplitude of each subframe of the plurality of first digital sound signal subframes is determined, based on the plurality of first digital sound signal subframes. The time domain characteristic amplitude value of the sound signal subframe obtains the environmental noise judgment threshold value of the current working period;

If the set of digital sound signals includes a first digital sound signal sub-frame, determine the time domain characteristic amplitude in the one first digital sound signal sub-frame and the first historical working period before the current working period The time domain characteristic amplitude of a digital signal subframe is based on the time domain characteristic amplitude of the one first digital sound signal subframe and the time of the first digital signal subframe of the historical working period before the current working period The domain characteristic amplitude is used to obtain the environmental noise judgment threshold of the current working period.
The electronic device according to any one of claims 12-16, wherein the processor is configured to:

Acquiring an environment frequency domain feature threshold, where the environment frequency domain feature threshold is determined according to at least one of the frequency domain feature threshold of the underwater environment and the frequency domain feature threshold of the aquatic environment;

Determine whether the environment in which the electronic device is located is an underwater environment according to the environmental frequency domain characteristic threshold and the frequency domain characteristic.
The electronic device according to claim 17, wherein the environmental frequency domain characteristic threshold includes a frequency domain characteristic threshold of an underwater environment and a frequency domain characteristic threshold of an aquatic environment, and the processor is configured to:

Determining a first frequency domain characteristic difference value, wherein the first frequency domain characteristic difference value is determined according to a difference value between the frequency domain characteristic and a frequency domain characteristic threshold value of the underwater environment;

Determining a second frequency domain characteristic difference value, wherein the first frequency domain characteristic difference value is determined according to a difference value between the frequency domain characteristic and the frequency domain characteristic threshold of the marine environment;

When the first frequency domain characteristic difference value is less than or equal to the second frequency domain characteristic difference value, it is determined that the environment where the electronic device is located is the underwater environment.
The electronic device according to any one of claims 12-18, wherein the processor is configured to:

Input the frequency domain features into a preset environment discrimination model to obtain the environment type label output by the preset environment discrimination model, wherein the preset environment discrimination model is based on the frequency domain characteristics of the underwater environment and the water Optimized the frequency domain characteristics of the environment;

Determine whether the environment in which the electronic device is located is an underwater environment according to the environment type tag.
The electronic device according to claim 19, wherein the preset environment discrimination model is a neural network model.
The electronic device according to claim 19, wherein the preset environment discrimination model is a support vector machine classification algorithm model.
The electronic device according to any one of claims 12-21, wherein the electronic device comprises an imaging component, and the processor is configured to:

When it is determined that the imaging component is in an underwater environment, the imaging parameters of the imaging component are adjusted.
A storage medium, comprising: a readable storage medium and a computer program, the computer program being used to implement the water entry detection method according to any one of claims 1-11.