CN114093391A

CN114093391A - Abnormal signal filtering method and device

Info

Publication number: CN114093391A
Application number: CN202010744779.3A
Authority: CN
Inventors: 任兵飞; 毛哲文; 肖艳光
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2020-07-29
Filing date: 2020-07-29
Publication date: 2022-02-25

Abstract

The application provides a method and a device for filtering abnormal signals. The method comprises the following steps: the electronic equipment detects the acquired sound signals according to a detection model trained in advance. If the electronic device detects that an abnormal high-frequency signal exists in the sound signal, a warning can be given to a user, or the abnormal high-frequency signal in the sound signal can be filtered. The benefit of this approach is that the privacy of the user can be prevented from being revealed without the user's knowledge. By the method, potential hidden communication threats can be identified and intercepted, and the risk of privacy information leakage is reduced.

Description

Abnormal signal filtering method and device

Technical Field

The present application relates to the field of communications technologies, and in particular, to a method and an apparatus for filtering an abnormal signal.

Background

With the rapid development of networks and informatization, information release through the internet, network trade through the internet, and the like are widely used by people, meanwhile, various confidential information including national security information, private information (such as personal account numbers), and the like need to be transmitted through the network, and in order to protect the security of various secret information, the hidden communication technology can effectively prevent and treat the secret information from being intercepted and acquired by a third party.

Hidden communication technologies based on acoustic waves have been widely used. The main work flow is as follows: the synthesis equipment modulates information to be transmitted into a carrier signal with a specific frequency and synthesizes the carrier signal into a normal sound signal; the sending end sends out normal sound signals, and the receiving end receives and demodulates information to be sent in the normal sound signals, so that hidden communication based on sound waves is completed.

Hidden communication based on sound waves also poses a risk of personal information leakage for the user. The attacker modulates the special information into a high-frequency carrier signal through the synthesis device, and synthesizes the high-frequency carrier signal into a normal sound signal, wherein the high-frequency carrier signal cannot be perceived by a user. The playback device (e.g., stereo) plays back the synthesized sound signal. When the user approaches the playback device, the synthesized sound signal may be received by the user's electronic device (e.g., a cell phone). If an application developed by an attacker is installed in an electronic device (e.g., a mobile phone) of a user and the application can detect a high-frequency carrier signal modulated by the attacker, the application can collect user or device information, such as an identifier of the electronic device (e.g., the mobile phone) or a current location of the electronic device (e.g., the mobile phone), and send the user or device information to the attacker. At this time, the attacker obtains the information of the user without the user knowing it.

Disclosure of Invention

The application provides a method for filtering abnormal signals, wherein electronic equipment detects collected sound signals, an abnormal signal detection module finds abnormal high-frequency signals, namely when the high-frequency energy value of the sound signals is higher than a threshold value, a user is informed of the abnormal conditions, the user can select to filter the abnormal high-frequency signals to intercept potential hidden communication threats, and meanwhile, the sound signals with the filtered abnormal high-frequency signals are returned to a first application. Therefore, hidden communication can be identified and intercepted, and the safety of user privacy is improved.

In a first aspect, the present application provides a method for detecting and filtering an abnormal high frequency signal. The method comprises the following steps: the electronic equipment receives a sound signal acquisition request; in response to a sound signal acquisition request, the electronic device acquires a sound signal; the electronic device determines a high frequency energy value of the sound signal according to the sound signal. If the high-frequency energy value of the sound signal is larger than the environmental sound energy threshold value, the electronic equipment sends prompt information to the user, and the prompt information is used for prompting the user that an abnormal high-frequency signal exists in the sound signal.

The electronic equipment detects the acquired sound signals according to a detection model trained in advance. If the electronic device detects that an abnormal high-frequency signal exists in the sound signal, a warning can be given to a user, or the abnormal high-frequency signal in the sound signal can be filtered. The benefit of this approach is that the privacy of the user can be prevented from being revealed without the user's knowledge. By the method, potential hidden communication threats can be identified and intercepted, and the risk of privacy information leakage is reduced.

With reference to the first aspect, in a possible implementation manner of the first aspect, after the electronic device sends the prompt information to the user, the electronic device further receives a filtering operation of the user; in response to the filtering operation, the electronic device filters an abnormally high frequency signal present in the sound signal. Therefore, a user can sense that abnormal high-frequency signals exist in the sound signals acquired by the electronic equipment, and the abnormal high-frequency signals existing in the area can be prevented when the user passes through the area next time; or the user reports the condition that the abnormal high-frequency signal exists in the area to the server, and the server summarizes the information reported by the user and takes measures to prevent the privacy information of the user from being leaked due to the abnormal high-frequency signal. In some embodiments, when the electronic device determines that the high frequency energy value of the sound signal is greater than the environmental sound energy threshold value, the electronic device does not need to send a prompt message to the user, and the electronic device directly removes the abnormal high frequency signal from the sound signal. Therefore, when the user makes a call with the friend through the electronic equipment or the electronic equipment displays the navigation route, the electronic equipment does not send prompt information to the user, and the current operation of the user is not influenced.

With reference to the first aspect, in a possible implementation manner of the first aspect, the electronic device further sends the sound signal with the abnormal high-frequency signal filtered out to the receiving device. When a user makes a call with a friend or plays a game with a friend group through the electronic equipment, the electronic equipment sends the sound signal with the abnormal high-frequency signal filtered out to the receiving equipment through the communication module, and the receiving equipment is equipment used by the friend of the user, so that the current operation of the user is not influenced.

With reference to the first aspect, in a possible implementation manner of the first aspect, the electronic device obtains first environment information, where the first environment information includes location information and/or an environmental sound; the electronic device determines an ambient sound energy threshold based on the first environmental information. Here, the first environment information is used to determine a current environment (e.g., a mall), the electronic device determines an environment sound energy threshold corresponding to the current environment according to the current environment, the environment sound energy threshold is used to compare with a high-frequency energy value of the sound signal, and if the high-frequency energy value of the sound signal is greater than the environment sound energy threshold corresponding to the current environment, the electronic device determines that an abnormal high-frequency signal exists in the sound signal.

With reference to the first aspect, in a possible implementation manner of the first aspect, the determining, by the electronic device, a high-frequency energy value of the sound signal according to the sound signal specifically includes the following steps: firstly, the electronic equipment acquires environmental sound before acquiring a sound signal; secondly, the electronic equipment determines an environmental noise energy value according to the environmental sound; and finally, the electronic equipment determines the high-frequency energy value of the sound signal according to the actual high-frequency energy value of the sound signal and the environmental noise energy value. Here, the electronic device determines the high-frequency energy value of the sound signal based on the actual high-frequency energy value of the sound signal and the ambient noise energy value, and removes the ambient noise energy value from the actual high-frequency energy value of the sound signal in order to eliminate the influence of the ambient noise. Thus, the accuracy of confirming the high frequency energy value of the sound signal can be improved.

With reference to the first aspect, in a possible implementation manner of the first aspect, the determining, by the electronic device, the high-frequency energy value of the sound signal according to the actual high-frequency energy value of the sound signal and the environmental noise energy value specifically includes the following steps: the electronic equipment acquires a high-frequency signal in the sound signal; the electronic equipment divides the high-frequency signal into high-frequency signal segments of n time windows, wherein n is a positive integer; the electronic equipment calculates the actual high-frequency energy value of the high-frequency signal segment in the n time windows; the electronic equipment calculates the full decibel high-frequency energy values of the high-frequency signal segments in the n time windows according to the actual high-frequency energy values of the high-frequency signal segments in the n time windows; and the electronic equipment determines the high-frequency energy value of the sound signal according to the full-decibel high-frequency energy value of the high-frequency signal segment in the n time windows and the environmental noise energy value. Here, since the frequency band range of the abnormal high-frequency signal is in the high-frequency band, the high-frequency energy value of the sound signal is calculated according to the high-frequency band signal of the sound signal only by acquiring the high-frequency band signal of the sound signal, so that the calculation of the electronic device can be reduced, and the consumption can be saved.

With reference to the first aspect, in a possible implementation manner of the first aspect, the electronic device may determine, according to actual high-frequency energy values of the high-frequency signal segments in the n time windows, full-decibel high-frequency energy values of the high-frequency signal segments in the n time windows through the following formula:

in which Pb is_iIs the full decibel high frequency energy value of the high frequency signal segment in the ith time window, Sb_iAnd the j is the actual high-frequency energy value of the high-frequency signal segment in the ith time window, the Sj is the number of bits required by the electronic equipment for storing the high-frequency signal segment in the jth time window, the j is a positive integer which is greater than or equal to 1 and less than or equal to n, and the i is a positive integer which is less than or equal to n.

With reference to the first aspect, in a possible implementation manner of the first aspect, the electronic device may determine the high-frequency energy value of the sound signal according to the full-decibel high-frequency energy values and the ambient noise energy values of the high-frequency signal segments in the n time windows by using the following formula:

in which Pb is_iThe method comprises the steps that a full decibel high-frequency energy value of a high-frequency signal segment in an ith time window is defined, Pci is a difference value between the full decibel high-frequency energy value of the high-frequency signal segment in the ith time window and an environmental noise energy value, Pm represents the environmental noise energy value, and i is a positive integer less than or equal to n; p0 represents the high frequency energy value of the sound signal.

With reference to the first aspect, in a possible implementation manner of the first aspect, a frequency band of the abnormal high-frequency signal includes 18KHz to 20 KHz.

In a second aspect, the present application provides an electronic device, including a sound collection module, an abnormal signal detection framework; the voice acquisition module is used for acquiring a voice signal; the abnormal signal detection framework is used for determining the high-frequency energy value of the sound signal; and the abnormal signal detection framework is also used for judging whether the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold value or not, and sending prompt information to a user, wherein the prompt information is used for prompting that the sound signal has an abnormal high-frequency signal.

The electronic device detects the acquired sound signal according to the abnormal signal detection framework. If the abnormal signal detection framework detects that the abnormal high-frequency signal exists in the sound signal, a warning can be sent to a user, or the abnormal high-frequency signal in the sound signal can be filtered. The benefit of this approach is that the privacy of the user can be prevented from being revealed without the user's knowledge. In this way, potential hidden communication threats can be identified and intercepted, and the risk of privacy information disclosure is reduced.

With reference to the second aspect, in a possible implementation manner of the second aspect, the abnormal signal detection framework is further configured to receive a filtering operation of a user; and responding to the filtering operation of the user, and filtering abnormal high-frequency signals existing in the sound signals. Therefore, a user can sense that abnormal high-frequency signals exist in the sound signals acquired by the electronic equipment, and the abnormal high-frequency signals existing in the area can be prevented when the user passes through the area next time; or the user reports the condition that the abnormal high-frequency signal exists in the area to the server, and the server summarizes the information reported by the user and takes measures to prevent the privacy information of the user from being leaked due to the abnormal high-frequency signal. In some embodiments, when the electronic device determines that the high frequency energy value of the sound signal is greater than the environmental sound energy threshold value, the electronic device does not need to send a prompt message to the user, and the electronic device directly removes the abnormal high frequency signal from the sound signal. Therefore, when the user makes a call with the friend through the electronic equipment or the electronic equipment displays the navigation route, the electronic equipment does not send prompt information to the user, and the current operation of the user is not influenced.

With reference to the second aspect, in a possible implementation manner of the second aspect, the sound collection module is further configured to send the sound signal with the abnormal high-frequency signal filtered out to the receiving device. When a user makes a call with a friend through the electronic equipment or plays a game with a friend team, the abnormal signal detection frame calls the sound collection module, the sound collection module sends the sound signal with the abnormal high-frequency signal filtered out to the receiving equipment through the communication module, and the receiving equipment is equipment used by the friend of the user, so that the current operation of the user cannot be influenced. In some embodiments, the abnormal signal detection framework directly sends the sound signal with the abnormal high-frequency signal filtered out to the receiving device through the communication module without calling the sound collection module.

With reference to the second aspect, in a possible implementation manner of the second aspect, the abnormal signal detecting framework is further configured to obtain first environment information, where the first environment information includes location information and/or an environmental sound; an ambient sound energy threshold is determined based on the first environmental information. Here, the first environment information is used to determine a current environment (e.g., a mall), the abnormal signal detection framework determines an environment sound energy threshold corresponding to the current environment according to the current environment, the environment sound energy threshold is used to compare with a high-frequency energy value of the sound signal, and if the high-frequency energy value of the sound signal is greater than the environment sound energy threshold corresponding to the current environment, the abnormal signal detection framework determines that an abnormal high-frequency signal exists in the sound signal.

With reference to the second aspect, in a possible implementation manner of the second aspect, the sound collection module is further configured to obtain an ambient sound before obtaining the sound signal; the abnormal signal detection framework is also used for determining an environmental noise energy value according to the environmental sound; and determining the high-frequency energy value of the sound signal according to the actual high-frequency energy value of the sound signal and the environmental noise energy value. Here, the electronic device determines the high-frequency energy value of the sound signal based on the actual high-frequency energy value of the sound signal and the ambient noise energy value in order to eliminate the influence of the ambient noise, and removes the ambient noise energy value from the actual high-frequency energy value of the sound signal, thereby improving the accuracy of confirming the high-frequency energy value of the sound signal.

With reference to the second aspect, in a possible implementation manner of the second aspect, the abnormal signal detection framework is further configured to acquire a high-frequency signal in the sound signal; dividing the high-frequency signal into high-frequency signal segments of n time windows, wherein n is a positive integer; calculating the actual high-frequency energy value of the high-frequency signal segment in the n time windows; according to the actual high-frequency energy values of the high-frequency signal segments in the n time windows; calculating full decibel high-frequency energy values of high-frequency signal segments in n time windows; and determining the high-frequency energy value of the sound signal according to the full-decibel high-frequency energy value of the high-frequency signal segment in the n time windows and the environmental noise energy value. Here, since the frequency band range of the abnormal high-frequency signal is in the high-frequency band, the abnormal signal detection framework only acquires the high-frequency band signal of the sound signal and calculates the high-frequency energy value of the sound signal according to the high-frequency band signal of the sound signal, so that the calculation can be reduced and the consumption can be saved.

With reference to the second aspect, in a possible implementation manner of the second aspect, the abnormal signal detection framework is further configured to determine, according to actual high-frequency energy values of the high-frequency signal segments in the n time windows, full-decibel high-frequency energy values of the high-frequency signal segments in the n time windows through the following formula:

With reference to the second aspect, in a possible implementation manner of the second aspect, the abnormal signal detecting framework is specifically configured to determine the high-frequency energy value of the sound signal according to the full-decibel high-frequency energy value and the ambient noise energy value of the high-frequency signal segment in the n time windows by using the following formula:

Pc_i＝Pb_i–Pm；

With reference to the second aspect, in a possible implementation manner of the second aspect, the frequency range of the high-frequency signal includes 18KHz to 20 KHz.

In a third aspect, the present application provides an apparatus for filtering an abnormal signal, including one or more processors, one or more memories, and a sound collector; the one or more memories and the sound collector are coupled to the one or more processors, and the one or more memories are configured to store computer program code, where the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the apparatus to perform a method for filtering an exception signal in any one of the possible implementation manners of the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer storage medium, which includes computer instructions, and when the computer instructions are executed on an electronic device, the computer storage medium causes a communication apparatus to perform a method for filtering an exception signal in any one of the possible implementation manners of the first aspect.

In a fifth aspect, the present application provides a computer program product, which when run on a computer, causes the computer to execute a method for filtering an anomaly signal in any one of the possible implementation manners of the first aspect.

The application provides a method and a device for filtering abnormal signals. The electronic equipment detects the acquired sound signals according to a detection model trained in advance. If the electronic device detects that an abnormal high-frequency signal exists in the sound signal, a warning can be given to a user, or the abnormal high-frequency signal in the sound signal can be filtered. The benefit of this approach is that the privacy of the user can be prevented from being revealed without the user's knowledge. By the method, potential hidden communication threats can be identified and intercepted, and the risk of privacy information leakage is reduced.

Drawings

Fig. 1 is a flowchart of a method for obtaining user privacy information through a hidden communication technology according to an embodiment of the present application;

fig. 2 is a schematic architecture diagram of an electronic device 100 according to an embodiment of the present disclosure;

fig. 3 is a schematic block diagram of a system 10 according to an embodiment of the present disclosure;

fig. 4 is a functional diagram of each module provided in the embodiment of the present application;

fig. 5 is a schematic flowchart illustrating a method for filtering an abnormal signal according to an embodiment of the present application;

fig. 6 is a schematic flowchart of another abnormal signal filtering method according to an embodiment of the present application;

7A-7F are a set of UI diagrams provided by embodiments of the application;

FIGS. 8A-8D are schematic diagrams illustrating a method for detecting and filtering abnormal high frequency signals according to an embodiment of the present application;

fig. 9 is a schematic diagram of an apparatus according to an embodiment of the present disclosure.

Detailed Description

The technical solution in the embodiments of the present application will be described in detail and removed with reference to the accompanying drawings. In the description of the embodiments herein, "/" means "or" unless otherwise specified, for example, a/B may mean a or B; "and/or" in the text is only an association relationship describing an associated object, and means that three relationships may exist, for example, a and/or B may mean: three cases of a alone, a and B both, and B alone exist, and in addition, "a plurality" means two or more than two in the description of the embodiments of the present application.

In the following, the terms "first", "second" are used for descriptive purposes only and are not to be understood as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of embodiments of the application, unless stated otherwise, "plurality" means two or more.

At present, the hidden communication technology based on the sound wave is mainly realized by a high-frequency signal and an application program capable of identifying the high-frequency signal. That is, after the application recognizes a specific high frequency signal, the application developer may perform an operation set in advance, for example, acquiring the privacy information of the user. The frequency of the high frequency signal is generally greater than 18KHz, and most users cannot perceive the presence of the high frequency signal in the surrounding environment because most users can only hear sounds with frequencies below 16KHz due to the limitation of the human ear hearing system (HAS). Meanwhile, the hardware configuration of most existing electronic equipment can support the receiving and demodulation of signals above 18KHz, but the control measures for high-frequency signals are lacked. In other words, it is difficult for most users to know whether a high frequency signal is currently present, and whether a hidden communication is currently present. If the application program acquires the privacy information of the user (such as the electronic equipment identification and the position of the electronic equipment) after recognizing the specific high-frequency signal and sends the privacy information to the specific server or the electronic equipment, the privacy information of the user, such as the user identity information, the position information or the operation habit, can be acquired under the condition that the user feels no sense.

Fig. 1 shows a flowchart of a method for obtaining user privacy information through a hidden communication technology according to an embodiment of the present application. As shown in fig. 1, the audio installed at the door of the store is playing a promotional advertisement, which contains an abnormal high frequency signal synthesized by a third party. As the user passes through the store while running, the electronic device carried by the user may capture the sound of the promotional advertisement. When an application program capable of identifying the high-frequency signal contained in the promotion advertisement exists in the consumer electronic device, after the application program identifies the abnormal high-frequency signal, the application program may acquire the identification of the consumer electronic device and the position of the electronic device, and send the identification and the position of the electronic device to a third party. The application may be a normally functioning application. For example, the electronic equipment receives the operation of a user, and turns on a voice assistant which collects a sound signal; for example, a user plays a game with a friend team through a certain game application program, the user opens a microphone control of a game interface for facilitating communication, and a sound acquisition module of user electronic equipment acquires a sound signal; for example, the user performs a video call with a friend in a shopping mall through a social application program, and a sound collection module of the user electronic device collects sound. If the position of the user has the information of hidden communication (for example, high-frequency signals exist in music played in a shopping mall), the user electronic equipment collects the sound signals and sends the sound signals to the application program, the application program (for example, a voice assistant, a certain game application program and a social application program) in the user electronic equipment can identify the high-frequency signals in the sound signals, and the high-frequency signals trigger the application program to acquire the identification of the user electronic equipment and the position of the electronic equipment and send the identification and the position to a third party.

It should be noted that the above application scenarios are only used for explaining the present application, and more or fewer application scenarios may also be included, which should not be construed as limiting.

Therefore, when the electronic device with the voice input function acquires the external sound signal, the first application (for example, the navigation application) installed in the electronic device risks revealing personal privacy due to the fact that the user privacy information is acquired through hidden communication.

In order to identify high frequency signals, there are two ways to detect sound signals.

The first method is as follows: abnormal sound detection and abnormal sound classification techniques, which are mainly used to detect abnormal sounds in environmental sounds, such as screaming sounds, gunshot sounds, breaking sounds, and the like, which represent abnormal events. The abnormal sound detection technology extracts features such as zero-crossing rate, amplitude difference, power spectral density, spectral entropy and Mel-frequency cepstral coefficients (MFCC) from a sample sound set, models are respectively carried out on various abnormal sound signals, and a classifier is trained for each abnormal sound signal. And extracting various characteristics such as zero crossing rate, amplitude difference, power spectral density, spectral entropy, Mel frequency cepstrum coefficient and the like from the sound to be detected for anomaly detection, so that the sound to be detected is classified into a specific sound category, and if the signal to be detected is not classified into the specific sound category, the sound to be detected is judged to be an unknown sound category. The first mode is mainly used for detecting abnormal sounds which can be sensed by human beings in the environmental sounds, and is large in calculation amount and poor in real-time performance, so that the first mode is not beneficial to being realized in electronic equipment.

The second method comprises the following steps: the method comprises the steps of carrying out spectrum analysis on an ultrasonic signal acquired by electronic equipment to obtain the center frequency of the ultrasonic signal, judging whether the center frequency is in a set attack frequency range, if so, carrying out band-pass filtering on the ultrasonic signal, filtering out frequency components of the upper limit and the lower limit of the voice frequency to obtain a baseband signal, and detecting whether the baseband signal contains voice. The second mode can only detect the attack mode of the ultrasonic signal carrying the voice signal, and can not identify the hidden attack behavior generated by transmitting the non-voice signal by using the ultrasonic signal.

The application provides an abnormal high-frequency signal detection and filtering method. The electronic equipment detects the acquired sound signals according to a detection model trained in advance. If the electronic device detects that an abnormal high-frequency signal exists in the sound signal, a warning can be given to a user, or the abnormal high-frequency signal in the sound signal can be filtered. The benefit of this approach is that the privacy of the user can be prevented from being revealed without the user's knowledge. By the method, potential hidden communication threats can be identified and intercepted, the risk of privacy information leakage is reduced, and user experience is guaranteed.

For example, when a user passes a certain store, a sound device installed in the store is playing out a promotional advertisement or music. The promotion advertisement or music is synthesized with an abnormal high-frequency signal, and the electronic equipment (such as a mobile phone) of a user can detect the abnormal high-frequency signal and send an alarm prompt to the user, so that a third party is prevented from acquiring the privacy of the user, and the advertisement is pushed to the user.

For another example, when a user watches a television program at home, an abnormal high-frequency signal is synthesized in an audio frequency played in the television program (special information carried by the abnormal high-frequency signal can be used for marking currently played video content, such as computer promotion, clothes advertisement and the like), and a user electronic device (such as a mobile phone) can detect the abnormal high-frequency signal and send an alarm prompt to the user, so that a third party is prevented from mastering a user's habits of watching the program, and a recommended program is pushed for the user.

For another example, a user accesses an anonymous network through an electronic device (e.g., a desktop computer), and a third-party server distributes audio and video contents synthesized with an abnormally high frequency signal in the anonymous network, so that the audio and video contents synthesized with the abnormally high frequency signal can attract the anonymous user to watch. The user's electronic device (e.g., a cell phone) may detect the abnormally high frequency signal and send an alert prompt to the user to prevent a third party from obtaining real information for accessing the anonymous network user.

It should be noted that hidden communication based on sound waves has been widely applied to security protection of secret information, for example, information that needs to be transmitted in a confidential manner at a transmitting end is modulated into a high-frequency carrier, the high-frequency carrier is then hidden in common information, and the secret information is released through the common information, so that an attacker is difficult to perceive the existing secret information from the common information, thereby enabling the secret information to have greater concealment and security. The receiving end receives the common information and demodulates the secret information in the common information, at this time, the receiving end displays corresponding prompt information, the prompt information may be prompt information containing an identifier of the sending end device, for example, the prompt information may be "receiving the secret information sent from the huaboei P30", a user of the receiving end determines that the information is useful information rather than junk information according to the prompt information, and the user may choose not to filter the signal, so as to receive the secret information sent by the sending end. The method can ensure that the useful information is not mistaken for filtering the junk information.

The above examples are provided only for explaining the present application and should not be construed as limiting.

Fig. 2 shows a schematic structural diagram of the electronic device 100.

The following describes an embodiment specifically by taking the electronic device 100 as an example. The device types of the electronic device 100 may include a mobile phone, a television, a tablet computer, a speaker, a watch, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a Personal Digital Assistant (PDA), an Augmented Reality (AR)/Virtual Reality (VR) device, etc. The embodiment of the present application does not particularly limit the type of the electronic device 100.

It should be understood that the electronic device 100 shown in fig. 2 is merely an example, and that the electronic device 100 may have more or fewer components than shown in fig. 2, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The electronic device 100 may include: the mobile terminal includes a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, a Subscriber Identity Module (SIM) card interface 195, and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It is to be understood that the illustrated structure of the embodiment of the present invention does not specifically limit the electronic device 100. In other embodiments of the present application, electronic device 100 may include more or fewer components than shown, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units, such as: the processor 110 may include an Application Processor (AP), a modem processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.

The controller may be, among other things, a neural center and a command center of the electronic device 100. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 110, thereby increasing the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (I2C) interface, an integrated circuit built-in audio (I2S) interface, a Pulse Code Modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a general-purpose input/output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and/or a Universal Serial Bus (USB) interface, etc.

The I2C interface is a bi-directional synchronous serial bus that includes a serial data line (SDA) and a Serial Clock Line (SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces, respectively. For example: the processor 110 may be coupled to the touch sensor 180K via an I2C interface, such that the processor 110 and the touch sensor 180K communicate via an I2C bus interface to implement the touch functionality of the electronic device 100.

The I2S interface may be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 via an I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 may communicate audio signals to the wireless communication module 160 via the I2S interface, enabling answering of calls via a bluetooth headset.

The PCM interface may also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 may be coupled by a PCM bus interface. In some embodiments, the audio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to implement a function of answering a call through a bluetooth headset. Both the I2S interface and the PCM interface may be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communications. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 with the wireless communication module 160. For example: the processor 110 communicates with a bluetooth module in the wireless communication module 160 through a UART interface to implement a bluetooth function. In some embodiments, the audio module 170 may transmit the audio signal to the wireless communication module 160 through a UART interface, so as to realize the function of playing music through a bluetooth headset.

MIPI interfaces may be used to connect processor 110 with peripheral devices such as display screen 194, camera 193, and the like. The MIPI interface includes a Camera Serial Interface (CSI), a Display Serial Interface (DSI), and the like. In some embodiments, processor 110 and camera 193 communicate through a CSI interface to implement the capture functionality of electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal and may also be configured as a data signal. In some embodiments, a GPIO interface may be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface may also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge the electronic device 100, and may also be used to transmit data between the electronic device 100 and a peripheral device. And the earphone can also be used for connecting an earphone and playing audio through the earphone. The interface may also be used to connect other electronic devices, such as AR devices and the like.

It should be understood that the connection relationship between the modules according to the embodiment of the present invention is only illustrative, and is not limited to the structure of the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners or a combination of multiple interface connection manners in the above embodiments.

The charging management module 140 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the electronic device 100. The charging management module 140 may also supply power to the electronic device through the power management module 141 while charging the battery 142.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140 and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 may also be used to monitor parameters such as battery capacity, battery cycle count, battery state of health (leakage, impedance), etc. In some other embodiments, the power management module 141 may also be disposed in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be disposed in the same device.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution including 2G/3G/4G/5G wireless communication applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module 150 may receive the electromagnetic wave from the antenna 1, filter, amplify, etc. the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modem processor, and convert the signal into electromagnetic wave through the antenna 1 to radiate the electromagnetic wave. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then passes the demodulated low frequency baseband signal to a baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional modules, independent of the processor 110.

The wireless communication module 160 may provide a solution for wireless communication applied to the electronic device 100, including Wireless Local Area Networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), bluetooth (bluetooth, BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), and the like. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification on the signal, and convert the signal into electromagnetic waves through the antenna 2 to radiate the electromagnetic waves.

In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150 and antenna 2 is coupled to wireless communication module 160 so that electronic device 100 can communicate with networks and other devices through wireless communication techniques. The wireless communication technology may include global system for mobile communications (GSM), General Packet Radio Service (GPRS), code division multiple access (code division multiple access, CDMA), Wideband Code Division Multiple Access (WCDMA), time-division code division multiple access (time-division code division multiple access, TD-SCDMA), Long Term Evolution (LTE), LTE, BT, GNSS, WLAN, NFC, FM, and/or IR technologies, etc. The GNSS may include a Global Positioning System (GPS), a global navigation satellite system (GLONASS), a beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and/or a Satellite Based Augmentation System (SBAS).

The electronic device 100 implements display functions via the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 194 is used to display images, video, and the like. The display screen 194 includes a display panel. The display panel may adopt a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), a flexible light-emitting diode (FLED), a miniature, a Micro-oeld, a quantum dot light-emitting diode (QLED), and the like. In some embodiments, the electronic device 100 may include 1 or N display screens 194, with N being a positive integer greater than 1.

The electronic device 100 may implement a shooting function through the ISP, the camera 193, the video codec, the GPU, the display 194, the application processor, and the like.

The ISP is used to process the data fed back by the camera 193. For example, when a photo is taken, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converting into an image visible to naked eyes. The ISP can also carry out algorithm optimization on the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in camera 193.

The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In some embodiments, the electronic device 100 may include 1 or N cameras 193, N being a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process digital image signals and other digital signals. For example, when the electronic device 100 selects a frequency bin, the digital signal processor is used to perform fourier transform or the like on the frequency bin energy.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 may play or record video in a variety of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

The NPU is a neural-network (NN) computing processor that processes input information quickly by using a biological neural network structure, for example, by using a transfer mode between neurons of a human brain, and can also learn by itself continuously. Applications such as intelligent recognition of the electronic device 100 can be realized through the NPU, for example: image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to extend the memory capability of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music, video, etc. are saved in an external memory card.

The internal memory 121 may be used to store computer-executable program code, which includes instructions. The processor 110 executes various functional applications of the electronic device 100 and data processing by executing instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required by at least one function, and the like. The storage data area may store data (such as audio data, phone book, etc.) created during use of the electronic device 100, and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), and the like.

The electronic device 100 may implement audio functions via the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor. Such as music playing, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also called a "horn", is used to convert the audio electrical signal into an acoustic signal. The electronic apparatus 100 can listen to music through the speaker 170A or listen to a handsfree call.

The receiver 170B, also called "earpiece", is used to convert the electrical audio signal into an acoustic signal. When the electronic apparatus 100 receives a call or voice information, it can receive voice by placing the receiver 170B close to the ear of the person.

The microphone 170C, also referred to as a "microphone," is used to convert sound signals into electrical signals. When making a call or transmitting voice information, the user can input a voice signal to the microphone 170C by speaking the user's mouth near the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In other embodiments, the electronic device 100 may be provided with two microphones 170C to achieve a noise reduction function in addition to collecting sound signals. In other embodiments, the electronic device 100 may further include three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, perform directional recording, and so on.

The headphone interface 170D is used to connect a wired headphone. The headset interface 170D may be the USB interface 130, or may be a 3.5mm open mobile electronic device platform (OMTP) standard interface, a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

The pressure sensor 180A is used for sensing a pressure signal, and converting the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194. The pressure sensor 180A can be of a wide variety, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor may be a sensor comprising at least two parallel plates having an electrically conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the strength of the pressure from the change in capacitance. When a touch operation is applied to the display screen 194, the electronic apparatus 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic apparatus 100 may also calculate the touched position from the detection signal of the pressure sensor 180A. In some embodiments, the touch operations that are applied to the same touch position but different touch operation intensities may correspond to different operation instructions. For example: and when the touch operation with the touch operation intensity smaller than the first pressure threshold value acts on the short message application icon, executing an instruction for viewing the short message. And when the touch operation with the touch operation intensity larger than or equal to the first pressure threshold value acts on the short message application icon, executing an instruction of newly building the short message.

The gyro sensor 180B may be used to determine the motion attitude of the electronic device 100. In some embodiments, the angular velocity of electronic device 100 about three axes (i.e., the x, y, and z axes) may be determined by gyroscope sensor 180B. The gyro sensor 180B may be used for photographing anti-shake. For example, when the shutter is pressed, the gyro sensor 180B detects a shake angle of the electronic device 100, calculates a distance to be compensated for by the lens module according to the shake angle, and allows the lens to counteract the shake of the electronic device 100 through a reverse movement, thereby achieving anti-shake. The gyroscope sensor 180B may also be used for navigation, somatosensory gaming scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, electronic device 100 calculates altitude, aiding in positioning and navigation, from barometric pressure values measured by barometric pressure sensor 180C.

The magnetic sensor 180D includes a hall sensor. The electronic device 100 may detect the opening and closing of the flip holster using the magnetic sensor 180D. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 may detect the opening and closing of the flip according to the magnetic sensor 180D. And then according to the opening and closing state of the leather sheath or the opening and closing state of the flip cover, the automatic unlocking of the flip cover is set.

The acceleration sensor 180E may detect the magnitude of acceleration of the electronic device 100 in various directions (typically three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. The method can also be used for recognizing the posture of the electronic equipment, and is applied to horizontal and vertical screen switching, pedometers and other applications.

A distance sensor 180F for measuring a distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, taking a picture of a scene, electronic device 100 may utilize range sensor 180F to range for fast focus.

The proximity light sensor 180G may include, for example, a Light Emitting Diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light to the outside through the light emitting diode. The electronic device 100 detects infrared reflected light from nearby objects using a photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 may determine that there are no objects near the electronic device 100. The electronic device 100 can utilize the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear for talking, so as to automatically turn off the screen to achieve the purpose of saving power. The proximity light sensor 180G may also be used in a holster mode, a pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense the ambient light level. Electronic device 100 may adaptively adjust the brightness of display screen 194 based on the perceived ambient light level. The ambient light sensor 180L may also be used to automatically adjust the white balance when taking a picture. The ambient light sensor 180L may also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

The fingerprint sensor 180H is used to collect a fingerprint. The electronic device 100 can utilize the collected fingerprint characteristics to unlock the fingerprint, access the application lock, photograph the fingerprint, answer an incoming call with the fingerprint, and so on.

The temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 implements a temperature processing strategy using the temperature detected by temperature sensor 180J. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 performs a reduction in performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, the electronic device 100 heats the battery 142 when the temperature is below another threshold to avoid the low temperature causing the electronic device 100 to shut down abnormally. In other embodiments, when the temperature is lower than a further threshold, the electronic device 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

The touch sensor 180K is also referred to as a "touch panel". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is used to detect a touch operation applied thereto or nearby. The touch sensor can communicate the detected touch operation to the application processor to determine the touch event type. Visual output associated with the touch operation may be provided through the display screen 194. In other embodiments, the touch sensor 180K may be disposed on a surface of the electronic device 100, different from the position of the display screen 194.

The bone conduction sensor 180M may acquire a vibration signal. In some embodiments, the bone conduction sensor 180M may acquire a vibration signal of the human vocal part vibrating the bone mass. The bone conduction sensor 180M may also contact the human pulse to receive the blood pressure pulsation signal. In some embodiments, the bone conduction sensor 180M may also be disposed in a headset, integrated into a bone conduction headset. The audio module 170 may analyze a voice signal based on the vibration signal of the bone mass vibrated by the sound part acquired by the bone conduction sensor 180M, so as to implement a voice function. The application processor can analyze heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

The keys 190 include a power-on key, a volume key, and the like. The keys 190 may be mechanical keys. Or may be touch keys. The electronic apparatus 100 may receive a key input, and generate a key signal input related to user setting and function control of the electronic apparatus 100.

The motor 191 may generate a vibration cue. The motor 191 may be used for incoming call vibration cues, as well as for touch vibration feedback. For example, touch operations applied to different applications (e.g., photographing, audio playing, etc.) may correspond to different vibration feedback effects. The motor 191 may also respond to different vibration feedback effects for touch operations applied to different areas of the display screen 194. Different application scenes (such as time reminding, receiving information, alarm clock, game and the like) can also correspond to different vibration feedback effects. The touch vibration feedback effect may also support customization.

Indicator 192 may be an indicator light that may be used to indicate a state of charge, a change in charge, or a message, missed call, notification, etc.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be brought into and out of contact with the electronic apparatus 100 by being inserted into the SIM card interface 195 or being pulled out of the SIM card interface 195. The electronic device 100 may support 1 or N SIM card interfaces, N being a positive integer greater than 1. The SIM card interface 195 may support a Nano SIM card, a Micro SIM card, a SIM card, etc. The same SIM card interface 195 can be inserted with multiple cards at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 may also be compatible with different types of SIM cards. The SIM card interface 195 may also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as communication and data communication. In some embodiments, the electronic device 100 employs esims, namely: an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

In order to facilitate understanding of the embodiments of the present application, a system architecture of the embodiments of the present application is described below.

Fig. 3 schematically illustrates an architecture diagram of a system 10 provided in an embodiment of the present application. As shown in fig. 3, the system 10 includes an electronic device 100 and a server 200.

The electronic device 100 includes a sound collection module 101, an abnormal signal detection framework 102, and an application 103. The sound collection module 101 may be configured to collect sound and transmit a sound signal to the application 103. The abnormal signal detection framework 102 may be configured to detect the sound signal received by the application 103 and determine whether there is an abnormal high frequency signal in the sound signal. If the abnormal high-frequency signal exists in the sound signal, the abnormal signal detection framework 102 may send an alarm prompt to the user and filter the abnormal high-frequency signal in the sound signal. The application 103 may be configured to send an acquisition request to the sound collection module 101, where the acquisition request is used to request the sound collection module 101 to collect a sound signal, and the application 103 is further configured to receive the sound signal sent by the sound collection module 101.

The server 200 includes a detection module 104. The detection module 104 may be configured to obtain each environmental sound energy threshold value by training according to the sample sound data set using the detection model.

The detection model can comprise each environment identifier and an environment sound energy threshold value corresponding to each environment identifier; or, the detection model comprehensively considers the environmental sound energy threshold corresponding to each environmental identifier to obtain an average environmental sound energy threshold.

In some embodiments, the server 200 trains the detection model and then sends the detection model to the electronic device 100. The electronic device 100 receives the detection model transmitted by the server 200.

Alternatively, the server 200 periodically updates the test model according to the sample sound data set, and the server 200 periodically transmits the test model to the electronic device 100. The electronic device 100 receives the detection model transmitted by the server 200 and periodically updates the detection model.

In other embodiments, the manufacturer of the electronic device 100 has the detection model pre-installed in the electronic device 100. That is, when the user starts up and starts to use the electronic apparatus 100 after purchasing the electronic apparatus 100, the detection model already exists in the electronic apparatus 100, and the electronic apparatus 100 does not need to acquire the detection model from the server 200.

In other embodiments, the application developer of the first application presets a detection model in the first application, and when the electronic device 100 receives a user operation to download the first application from an application store, the detection model will be downloaded to the electronic device 100 simultaneously with the first application. Thus, the electronic device 100 does not need to acquire the detection model from the server 200.

In other embodiments, the electronic device 100 may train the detection model by collecting voice samples or by obtaining voice samples from a network.

In the above embodiment in which the electronic device 100 does not need to obtain the detection model from the server 200, the detection model in the electronic device 100 is trained, that is, the detection model includes each environment identifier and an environment sound energy threshold corresponding to each environment identifier; or, the detection model comprehensively considers the environmental sound energy threshold corresponding to each environmental identifier, and only includes the average environmental sound energy threshold. It can be understood that the embodiments of the present application

The above-described embodiments are merely illustrative of the present application and should not be construed as limiting.

The detection model is used for detecting whether the sound signal has abnormal high-frequency signals. After acquiring the detection model, the electronic apparatus 100 may determine whether an abnormal high frequency signal exists in the acquired sound signal according to the detection model.

In some embodiments, the detection model comprehensively considers the environmental sound energy threshold corresponding to each environmental identifier to obtain an average environmental sound energy threshold. When the electronic device 100 needs to detect the acquired sound signal, the electronic device 100 first determines a high-frequency energy value of the sound signal, the electronic device 100 inputs the high-frequency energy value of the sound signal into the detection model, and the detection model determines that the detection result is output by the detection model if the high-frequency energy value of the sound signal is greater than the average environmental sound energy threshold, and the detection result is that an abnormal high-frequency signal exists in the sound signal.

In other embodiments, each environment identifier and an environment sound energy threshold corresponding to each environment identifier may be included in the detection model, when the electronic device 100 needs to detect the acquired sound signal, the electronic device 100 determines a high-frequency energy value of the sound signal, the electronic device 100 determines a current environment identifier (for example, a shopping mall) according to the sound signal, the electronic device 100 inputs the high-frequency energy value of the sound signal into the detection model, and determines an environment sound energy threshold corresponding to the current environment identifier from the detection model according to the current environment identifier. And the detection model judges that if the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold value corresponding to the current environmental identifier, the detection model outputs a detection result, and the detection result is that an abnormal high-frequency signal exists in the sound signal.

The abnormal signal detection framework 102 includes an abnormal signal detection module 1021, a model management module 1022, an alarm module 1023, an alarm application 1024, and a filter module 1025.

The abnormal signal detection module 1021 may be configured to detect whether an abnormal high-frequency signal exists in the sound signal collected by the sound collection module 101. The abnormal signal detection module 1021 receives the detection model sent by the model management module 1022, the abnormal signal detection module 1021 may calculate a high-frequency energy value of the sound signal, and then input the high-frequency energy value into the detection model, and if the detection model determines that the high-frequency energy value of the sound signal is greater than an environmental sound energy threshold (the environmental sound energy threshold may be an average environmental sound energy threshold, or an environmental sound energy threshold corresponding to a current environmental identifier), the detection model determines that an abnormal high-frequency signal exists in the sound signal.

The model management module 1022 is responsible for downloading and updating the detection model, and sends the detection model to the abnormal signal detection module 1021.

The alert module 1023 may be configured to send an alert instruction to the alert application 1024, where the alert instruction is used to instruct the alert application 1024 to present alert information to a user.

The alert application 1024 may be used to present alert prompts to a user, provide user interface operations, and obtain user actions (e.g., select ignore alerts or select confirm alerts).

And the filtering module 1025 can be used for filtering out abnormal high-frequency signals in the sound signals after receiving the user confirmation operation.

In a possible implementation manner, after the electronic device 100 receives that the user cancels the filtering operation, that is, the user selects not to filter the abnormal high-frequency signal in the sound signal, the alarm application 1024 sends a first prompt message to the alarm module 1023, where the first prompt message is used to prompt the alarm module 1023 to call the sound collection module 101, and the sound collection module 101 returns the sound signal with the abnormal high-frequency signal unfiltered to the application 103.

In another possible implementation manner, after the electronic device 100 receives the confirmation of the filtering operation by the user, that is, the user selects to filter out the abnormal high-frequency signal in the sound signal, the alarm application 1024 sends a second prompt message to the alarm module 1023, where the second prompt message is used to prompt the alarm module 1023 to send a filtering instruction to the filtering module 1025, the filtering module 1025 filters out the abnormal high-frequency signal in the sound signal, and meanwhile, the filtering module 1025 calls the sound collection module 101, and the sound collection module 101 returns the sound signal with the abnormal high-frequency signal filtered out to the application 103.

The above-described embodiments are merely illustrative of the present application and should not be construed as limiting. In some embodiments, the detection model is located in server 200. That is, after the detection module 104 has trained the detection model, the server 200 does not need to transmit the detection model to the electronic device 100. At this time, after acquiring the sound signal, the electronic device may send the sound signal to the server. After receiving the sound signal, the server may determine whether the sound signal has an abnormal high-frequency signal according to the detection model, and return the determination result to the electronic device.

For example, after acquiring the sound signal, the electronic device 100 sends the acquired sound signal to the server 200. The abnormal signal detection framework 102 in the server 200 confirms the high-frequency energy value of the sound signal, and the server 200 confirms the current environment identifier according to the sound signal, and matches the environment sound energy threshold corresponding to the current environment identifier from the detection model. If the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold value, the detection model outputs a detection result, the detection result is that an abnormal high-frequency signal exists in the sound signal, the server 200 sends a warning instruction to the electronic device 100, the electronic device 100 receives and responds to the warning instruction, and the electronic device 100 sends a warning prompt to a user. The electronic device 100 generates a filtering instruction according to the user confirmation operation, the electronic device 100 sends the filtering instruction to the server 200, and the abnormal signal detection framework 102 in the server 200 filters out abnormal high-frequency signals in the sound signals. The server 200 returns the sound signal with the abnormal high frequency signal filtered out to the electronic device 100, and the electronic device 100 receives the sound signal with the abnormal high frequency signal filtered out and sends the sound signal to the application.

It is understood that the detection model, anomaly signal detection framework 102 shown in fig. 3 may be located in the server 200. At this time, the abnormal signal detection module 1021, the model management module 1022 and the filtering module 1025 in the abnormal signal detection framework 102 are located in the server 200; the alarm application 1024 and the alarm module 1023 in the abnormal signal detection framework 102 are located in the electronic device 100. There is no limitation on whether the respective modules are located in the server 200 or the electronic device 100. Another possible implementation is that the anomaly signal detection framework 102 may be located in the electronic device 100 and the detection model in the server 200. After the electronic device 100 collects the sound signal, the electronic device 100 confirms the high-frequency energy value of the sound signal, the electronic device 100 confirms the current environment identifier according to the sound signal, and the electronic device 100 sends the current environment identifier to the server 200. The server 200 matches the environmental sound energy threshold corresponding to the current environmental identifier from the detection model according to the current environmental identifier. The server 200 transmits the ambient sound energy threshold to the electronic device 100. If the electronic device 100 determines that the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold value, an abnormal high-frequency signal exists in the sound signal, the server 200 sends a warning prompt to the electronic device 100, and the electronic device 100 displays warning information to a user. The electronic device 100 generates a filtering instruction according to the user confirmation operation, the electronic device 100 sends the filtering instruction to the server 200, the abnormal signal detection framework 102 in the server 200 filters out abnormal high-frequency signals in the sound signals, the server 200 returns the sound signals with the abnormal high-frequency signals filtered out to the electronic device 100, and the electronic device 100 receives the sound signals with the abnormal high-frequency signals removed and sends the sound signals to the application 103.

In other embodiments, the detection model is located in the electronic device 100. That is, after the detection module 104 has trained the detection model, the server 200 sends the detection model to the electronic device 100 or the detection model already exists in the electronic device 100 in advance, and the electronic device 100 does not need to acquire the detection model from the server 200.

One possible implementation is that the detection model, anomaly signal detection framework 102, may be located in the electronic device 100. After the electronic device 100 acquires the sound signal, the abnormal signal detection framework 102 in the electronic device 100 confirms the high-frequency energy value of the sound signal, and the electronic device 100 confirms the current environment identifier according to the sound signal, and matches the environment sound energy threshold corresponding to the current environment identifier from the detection model. If the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold value, the detection model outputs a detection result, and the detection result is that an abnormal high-frequency signal exists in the sound signal, and the electronic device 100 sends a warning prompt to a user. The electronic device 100 generates a filtering instruction according to the user confirmation operation, and the abnormal signal detection framework 102 in the electronic device 100 filters the abnormal high-frequency signal in the sound signal and sends the sound signal with the abnormal high-frequency signal filtered to the application.

Another possible implementation is that the detection model is located in the electronic device 100 and the anomaly signal detection framework 102 may be located in the server 200.

At this time, the abnormal signal detection module 1021, the model management module 1022 and the filtering module 1025 in the abnormal signal detection framework 102 are located in the server 200; the alarm application 1024 and the alarm module 1023 in the abnormal signal detection framework 102 are located in the electronic device 100. There is no limitation on whether the respective modules are located in the server 200 or the electronic device 100.

After the electronic device 100 collects the sound signal, the electronic device 100 sends the sound signal to the server 200, the abnormal signal detection framework 102 in the server 200 confirms the high-frequency energy value of the sound signal, and the electronic device 100 confirms the current environment identifier according to the sound signal and matches the environment sound energy threshold corresponding to the current environment identifier. The electronic device 100 sends the environmental sound energy threshold to the server 200, and the server 200 determines that the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold, and determines that an abnormal high-frequency signal exists in the sound signal. The server 200 transmits a warning prompt to the electronic apparatus 100, and the electronic apparatus 100 displays warning information to the user. The electronic device 100 generates a filtering instruction according to the user confirmation operation, the electronic device 100 sends the filtering instruction to the server 200, the abnormal signal detection framework 102 in the server 200 filters out abnormal high-frequency signals in the sound signals, the server 200 returns the sound signals with the abnormal high-frequency signals filtered out to the electronic device 100, and the electronic device 100 receives the sound signals with the abnormal high-frequency signals removed and sends the sound signals to the application.

As shown in fig. 4, the functions of the respective modules included in the sound collection module 101, the detection module 104, and the abnormal signal detection frame 102 are described.

The sound collection module 101 may be composed of a microphone for collecting a sound signal, and a processor for converting the collected sound signal into an analog signal.

For example, the electronic device 100 does not need to collect a sound signal through the sound collection module 101, the sound signal acquired by the electronic device 100 may be acquired through a smart wearable device (e.g., a bluetooth watch, a bluetooth headset, etc.), the smart wearable device (e.g., the bluetooth watch, the bluetooth headset, etc.) establishes a communication connection with the electronic device 100, the smart wearable device (e.g., the bluetooth watch, the bluetooth headset, etc.) acquires the sound signal and transmits the sound signal to the electronic device 100, and the electronic device 100 acquires the sound signal.

The model management module 1022 receives the detection model and sends it to the abnormal signal detection module 1021. In some embodiments, the detection model may be transmitted by the detection module 104 shown in FIG. 3. In other embodiments, the detection model may also be trained by the electronic device itself.

The sound collection module 101 collects sound, and the sound collection module 101 sends a sound signal to the abnormal signal detection module 1021.

The abnormal signal detection module 1021 detects whether an abnormal high-frequency signal exists in the sound signal, and when the abnormal high-frequency signal exists in the sound signal, the abnormal signal detection module 1021 sends an abnormal instruction which is used for prompting the alarm module 1013 that the abnormal high-frequency signal exists in the sound signal.

Meanwhile, the abnormal signal detecting module 1021 transmits the sound signal to the filtering module 1025.

Specifically, the abnormal signal detecting module 1021 detecting whether there is an abnormal high-frequency signal in the sound signal includes that the abnormal signal detecting module 1021 performs spectrum analysis on the sound signal to obtain a high-frequency energy value of the sound signal, and inputs the high-frequency energy value of the sound signal into the detection model, and the detection model determines that if the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold, the detection model outputs a detection result, and the detection result is that there is an abnormal high-frequency signal in the sound signal. The abnormal signal detecting module 1021 determines that an abnormal high frequency signal exists in the sound signal.

The abnormal high-frequency signal refers to a high-frequency signal in a specified frequency band (for example, 18KHz to 20KHz) in the sound signal when the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold value.

The alarm module 1023 receives and responds to the abnormal instruction sent by the abnormal signal detection module 1021, and in response to the abnormal instruction, the alarm module 1023 is used for sending an alarm instruction to the alarm application 1024, and the alarm instruction is used for instructing the alarm application 1024 to send alarm information to a user. The alarm information can be presented in a variety of ways, and the specific form will be described in detail below.

Electronic device 100 receives the user operation and generates a filtering instruction according to the user operation, and electronic device 100 sends the filtering instruction to filtering module 1025.

The filtering module 1025 receives and responds to the filtering instruction to filter out abnormal high frequency signals in the sound signal.

The filtering module 1025 calls the sound collection module 101, and the sound collection module 101 sends the sound signal with the abnormal high-frequency signal filtered out to the application 1013.

In some embodiments, the filtering module 1025 does not need to invoke the sound collection module 101, and the filtering module 1025 sends the sound signal filtered out the anomalous high frequency signal directly to the application 1013.

Referring to fig. 5, fig. 5 is a flowchart illustrating a method for filtering an abnormal signal according to an embodiment of the present application, where the method may be applied to the system 10 illustrated in fig. 3, where the system 10 may include an electronic device 100 and a server 200. The electronic device 100 may include a sound collection module 101, an anomaly signal detection framework 102, and an application 103. For a detailed description of the system 10, reference may be made to the foregoing embodiment shown in fig. 3, which is not described herein again. Wherein, the method can comprise the following steps:

s501, the server 200 trains to obtain the sound energy threshold value in each different environment by using the sound signal data set in each different environment, obtains the average environment sound energy threshold value, and updates the detection model.

Specifically, 1, the server 200 calculates the high-frequency energy value P of the environmental noise according to the environmental noise signal_z。

The server 200 acquires the ambient noise signal and reads the ambient noise signals of n time windows according to the ambient noise signal. The server 200 calculates a high frequency energy value at a full decibel scale (DBFS) of the ambient noise signal of each time window according to the actual high frequency energy value of the ambient noise signal of each time window, and stores the high frequency energy value in a queue 1, where the DBFS is a decibel value in the digital sound system based on the full decibel scale, and a calculation formula of the high frequency energy value at the full decibel scale of each time window may be:

pi represents the full decibel high frequency energy value of the high frequency ambient noise signal segment in the ith time window, S, as shown in equation (1)_iRepresentation of high frequency ambient noise signal segments in the ith time windowThe actual high frequency energy value of the environment noise signal can be obtained by short-time Fourier transform and the like. Sj represents the maximum value that can be expressed by the number of bits required to store the high frequency ambient noise signal segment in the jth time window, i.e. S_j＝2^j-1, j is a positive integer greater than or equal to 1 and less than or equal to n, and i is a positive integer less than or equal to n.

For example, set S_i7841 and each sound signal value is stored using a 16-bit signed number, then S_jThe maximum value that can be expressed is 2¹⁵-1, obtaining the value of the full decibel high frequency energy of the ambient noise signal in the current time window according to the formula (1) to be-12.4.

The server 200 can calculate the full-decibel high-frequency energy values of the high-frequency environmental noise signal segments of the n time windows respectively through the formula (1).

It is to be understood that equation (1) is merely for purposes of explaining the present application and should not be construed as limiting.

The server 200 obtains the high frequency energy value of the first group of ambient noise signals according to the average high frequency energy of the full decibel high frequency energy values of the high frequency ambient noise signal segments of the n time windows, and stores the average energy in the queue 2, and the calculation formula of the average high frequency energy may be:

as shown in equation (2), RMS represents the average high frequency energy, P, of the ambient noise signal_iAnd the full-decibel high-frequency energy value of the high-frequency environment noise signal segment of the ith time window is represented, n is a positive integer, and i is a positive integer less than or equal to n.

It is to be understood that equation (2) is merely for purposes of explaining the present application and should not be construed as limiting.

The server 200 continues to acquire new noise signals from the environment, reads the sound signals of n time windows according to the new noise signals, discards the existing sound signals in the n time windows, calculates the average high-frequency energy of the new noise signals according to formula (1) and formula (2), and adds the average high-frequency energy to the queue 2.

By analogy, the average high-frequency energy of m groups of ambient noise signals is obtained in the above manner and stored in the queue 2.

The server 200 calculates the arithmetic mean of the average high-frequency energies of the m groups of ambient noise signals in the queue 2 as the current ambient noise high-frequency energy value P_z。

2. The server 200 calculates an environmental sound energy threshold Pt according to the environmental sound signal sample data set.

The server 200 first extracts a first group of sound signals from the current environmental sound signal sample data set, reads the sound signals of n time windows according to the first group of sound signals, and calculates the full-decibel high-frequency energy values of the n time windows according to the above disclosure (1).

In order to eliminate the high-frequency signal interference caused by the frequency shift phenomenon of the ambient noise signal, the present application subtracts the current high-frequency energy value Pz of the ambient noise signal from the full-decibel high-frequency energy value of the first group of sound signals calculated according to the formula (1), and the calculation formula may be:

as shown in equation (3), Pwi represents the difference between the full db high frequency energy value of the high frequency sound signal segment in the ith time window and the current ambient noise signal high frequency energy value Pz, where Pz is the current ambient noise high frequency energy value.

Next, the server 200 may calculate an average high frequency energy value of the first group of sound signals according to formula (2).

Then, the server 200 continues to extract m groups of sound signals from the current environmental sound signal data set, and calculates the average high-frequency energy value of the m groups of sound signals by the method described in step two.

Finally, the server 200 calculates an arithmetic average of the average high-frequency energies of the m groups of sound signals to obtain an environmental sound energy threshold Pt, and the calculation formula may be:

pt ═ m (Pw1+ Pw2 … Pwm)/m equation (4)

As shown in equation (4), Pwm is the average high-frequency energy value of the mth group of sound signals, and m represents a total of m groups of sound signals in different environments.

It is to be understood that equation (4) is merely illustrative of the present application and should not be construed as limiting.

It is noted that the server 200 comprehensively considers the high frequency energy values of the sound signals in the respective environments, and the environment sound energy threshold Pt represents the average environment sound energy threshold.

In some embodiments, the server 200 may obtain each environment identifier and an environment sound energy threshold corresponding to each environment identifier respectively. At this time, each environment identifier and the environment sound energy threshold corresponding to each environment identifier can be obtained only by using the formula (1), the formula (2) and the formula (3), which is specifically referred to the above embodiment and is not repeated herein.

In some embodiments, the detection model comprehensively considers the environmental sound energy threshold corresponding to each environmental identifier to obtain an average environmental sound energy threshold, and the server 200 updates the detection model, where the average environmental sound energy threshold is included in the detection model.

In other embodiments, the server 200 updates the detection model, which includes the environment identifiers and the environment sound energy thresholds corresponding to the environment identifiers.

In some embodiments, after the server 200 updates the detection model, the server 200 transmits the detection model to the electronic device 100, and the electronic device 100 receives the detection model.

Here, the electronic device 100 may refer to the embodiment illustrated in fig. 3 for determining whether the abnormal high-frequency signal exists in the sound signal according to the detection model, and details of this application are not repeated herein.

In other embodiments, after the server 200 updates the detection model, the server 200 does not need to transmit the detection model to the electronic device 100.

The detection model may include various environment identifiers and environment sound energy thresholds corresponding to the various environment identifiers. The electronic device 100 confirms the high-frequency energy value of the sound signal, and the electronic device 100 confirms the current environment identifier (e.g., a shopping mall) according to the sound signal, or the electronic device 100 acquires the first environment information (e.g., the location information and/or the environmental sound) and confirms the current environment identifier (e.g., the shopping mall) according to the first environment information. The electronic device 100 sends the current environment identifier to the server 200, the detection model determines the environment sound energy threshold corresponding to the current environment identifier according to the current environment identifier, and the server 200 sends the environment sound energy threshold corresponding to the current environment identifier to the electronic device 100. The electronic device 100 determines whether the high-frequency energy value of the sound signal is greater than an environmental sound energy threshold corresponding to the current environment. If the high-frequency energy value of the sound signal is larger than the environmental sound energy threshold value corresponding to the current environment, an abnormal high-frequency signal exists in the sound signal.

Or the detection model comprehensively considers the environment sound energy threshold corresponding to each environment identifier to obtain an average environment sound energy threshold. When the electronic device 100 needs to detect the acquired sound signal, the electronic device 100 sends the sound signal to the server 200, the server 200 determines the high-frequency energy value of the sound signal, inputs the high-frequency energy value of the sound signal into the detection model, and the detection model outputs a detection result, where the detection result may be that an abnormal high-frequency signal exists in the sound signal or that an abnormal high-frequency signal does not exist in the sound signal. If the detection result is that an abnormal high frequency signal is present in the audio signal, the server 200 transmits the audio signal to the electronic device 100 when the abnormal high frequency signal is present in the audio signal.

S502, the application 103 sends a request for acquiring a sound signal to the sound collection module 101.

In some embodiments, the application 103 may send a get sound signal request to the sound collection module 101 by the application 103 in response to a user operation. Illustratively, the application 103 may be a voice assistant and the user action may be a voice instruction. For example, the voice command may be a wake word "small E", the voice assistant receives and responds to the wake word "small E", and the wake word "small E" triggers the voice assistant to turn on, and in response to the voice assistant turning on, the voice assistant sends a request for acquiring a sound signal to the sound collection module 101. The application 103 may also be a certain game application. The user interface displayed by the application 103 comprises a microphone control, the microphone control receives and responds to clicking operation of a user, a certain game application program sends a request for acquiring a sound signal to the sound acquisition module 101, the sound acquisition module 101 acquires the sound signal and sends the sound signal to receiving equipment through the communication module, and therefore the user can conveniently communicate game experience with game friends of the receiving equipment. The application 103 may also be a social application, where a user interface displayed by the application 103 includes a video call or audio call control, the video call or audio call control receives and responds to a click operation of a user, the social application sends a request for obtaining a sound signal to the sound collection module 101, and the sound collection module 101 collects the sound signal and sends the sound signal to a receiving device through the communication module. In this way, a user may engage in a voice call or a video call with a friend through a social application.

In other embodiments, the application 103 may also send the request to acquire the sound signal to the sound collection module 101 without user consent. The embodiment of the present application does not limit the trigger condition for sending the request for acquiring the sound signal.

S503, the sound collection module 101 responds to the request for obtaining the sound signal, and the sound collection module 101 collects sound.

In some embodiments, the sound collection module starts collecting sound after receiving the request for obtaining the sound signal. For example, in a video call application scenario, application 103 is assumed to be a video call application. When a user sends a request for establishing a video connection to a friend through the application 103 and the friend receives the request for establishing the video connection, at this time, the application 103 sends a request for acquiring a sound signal to the sound collection module 101, where the request is used for acquiring an external sound signal. When the sound collection module receives the sound signal acquisition request, the sound collection module starts to collect the sound signal.

In other embodiments, the application 103 need not send a get sound signal request to the sound collection module 101. The electronic device 100 may acquire the sound signal through an intelligent wearable device (e.g., a bluetooth watch, a bluetooth headset, etc.), the intelligent wearable device (e.g., the bluetooth watch, the bluetooth headset, etc.) establishes a communication connection with the electronic device 100, the intelligent wearable device (e.g., the bluetooth watch, the bluetooth headset, etc.) acquires the sound signal and transmits the sound signal to the electronic device 100, and the electronic device 100 acquires the sound signal.

S504, the sound collection module 101 sends the collected sound signal to the abnormal signal detection frame 102.

After acquiring the sound signal, the sound collection module 101 sends the sound signal to the abnormal signal detection framework 102. For example, in the above-mentioned video call application scenario, before the sound collection module 101 sends the collected sound signal to the application 103, the abnormal signal detection framework 102 detects the sound signal, and determines whether there is an abnormal high-frequency signal in the sound signal, thereby avoiding the leakage of user privacy.

S505, the abnormal signal detecting framework 102 sends an acquisition request to the server 200.

The acquisition request is used to request acquisition of a detection model from the server 200, the detection model including an average ambient sound energy threshold. Or the detection model comprises each environment identifier and an environment sound energy threshold value corresponding to each environment identifier.

The electronic device may perform this step before step S502 or after step S504. The execution sequence of the steps is not limited in the embodiment of the present application.

It will be appreciated that this step is an optional step. In some embodiments, the detection model is generated by the electronic device. At this time, after the abnormal signal detection framework acquires the sound signal, whether the sound signal has an abnormal high-frequency signal may be determined according to a detection model acquired in advance or a detection model generated by the electronic apparatus 100.

S506, the server 200 sends a detection model to the abnormal signal detection framework 102, wherein the detection model comprises an average environmental sound energy threshold value.

The anomaly signal detection framework 102 receives the detection model.

In some embodiments, the detection model includes each environment identifier and an environment sound energy threshold corresponding to each environment identifier, and the electronic device 100 may determine a current environment identifier (for example, a mall) according to the collected sound signal, and determine the environment sound energy threshold from the detection model according to the current environment identifier.

In some embodiments, the electronic device 100 may also obtain first environment information, which may be location information and/or ambient sound.

The electronic apparatus 100 transmits the first environment information to the server 200. Based on the current location positioning technology, the server 200 may accurately recognize the current environment identifier of the user according to the current location information of the user, where the current environment identifier may be a shopping mall, a coffee shop, or a library.

It should be noted that the first environment information includes the location information and/or the environment sound only for explaining the present application, and should not be limited, and in a specific implementation, the first environment information may include other contents.

In other embodiments, the electronic device 100 may determine a detection model or a corresponding ambient sound energy threshold that needs to be obtained according to the first environmental information, and then request the server 200 to obtain the determined detection model or ambient sound energy threshold through a obtaining request.

S507, the abnormal signal detection framework 102 performs spectrum analysis on the sound signal to obtain a high-frequency energy value P0 of the sound signal.

Firstly, the abnormal signal detection frame 102 calculates an environmental noise energy value Pm according to the environmental noise.

The sound collection module 101 collects a sound signal as ambient noise for a period of time before the application 103 responds to the get sound request.

For example, the sound collection module 101 takes the collected sound signal as the environmental noise two seconds before the application 103 responds to the sound acquisition request, and the abnormal signal detection framework 102 calculates the environmental noise high-frequency energy value Pm according to the environmental noise.

For example, in a video call application scenario, when a user sends a request for establishing a video connection to a friend through the application 103, before the friend receives the request for establishing the video connection, the application 103 does not start to collect a sound signal, at this time, the sound collection module 101 collects 2 seconds of ambient sound, and calculates an ambient noise energy value according to the 2 seconds of ambient sound. After the friend receives the request to establish the video connection, the application 103 sends the collected sound signal to the friend.

Specifically, the method comprises the following steps:

1. the abnormal signal detecting framework 102 obtains a high-frequency range (e.g. 18KHz to 20KHz) signal of the environmental sound, reads the high-frequency environmental sound signals of n time windows, respectively calculates full-decibel high-frequency energy values of the high-frequency environmental sound signals of the n time windows and stores the full-decibel high-frequency energy values in the queue 1, and the calculation formula may be:

pa is as shown in formula (5)_iIs the full decibel high frequency energy value, Sa, of the high frequency ambient sound signal segment in the ith time window_iThe actual high-frequency energy value of the high-frequency environmental sound signal segment in the ith time window is represented, and the high-frequency actual (for example, 18KHz-20KHz) energy value of the environmental sound signal can be obtained by means of short-time Fourier transform and the like. Sj represents the maximum value that can be expressed by the number of bits required to store the high frequency ambient noise signal segment in the jth time window, i.e. S_j＝2^j-1, j is a positive integer greater than or equal to 1 and less than or equal to n, and i is a positive integer less than or equal to n.

The abnormal signal detection framework 102 may calculate full-db high-frequency energy values of the high-frequency environmental sound signal segments of the n time windows respectively according to formula (5).

2. The abnormal signal detection frame 102 obtains the high frequency energy value of the first group of ambient sound signals according to the average high frequency energy of the full decibel high frequency energy values of the high frequency ambient sound signal segments of the n time windows, and stores the average energy in the queue 2, wherein the calculation formula of the average high frequency energy may be:

as shown in equation (6), Pu represents the average high frequency energy of the ambient sound signal, P_aiAnd the full decibel high-frequency energy value of the high-frequency environment sound signal segment of the ith time window is represented, and i is a positive integer less than or equal to n.

It is to be understood that equation (6) is only for explaining the present application and should not be construed as limiting.

The abnormal signal detection framework 102 continues to acquire new environmental noise from the environment, reads the sound signals of n time windows according to the new environmental noise, discards the existing sound signals in the n time windows, calculates the average high-frequency energy of the new environmental sound according to the formula (5) and the formula (6), and adds the average high-frequency energy into the queue 2.

By analogy, the average high frequency energy of the w sets of ambient sounds is obtained in the manner described above and is present in queue 2.

The abnormal signal detecting frame 102 calculates an arithmetic average of the average high-frequency energies of the w sets of ambient sounds in the queue 2 as an ambient noise energy value Pm.

3. The abnormal signal detection frame 102 calculates the high frequency energy value P0 on the full db scale of the sound signal according to the actual high frequency energy value of the sound signal.

After the application 103 responds to the sound acquiring request, the sound acquiring module 101 acquires sound, the sound acquiring module 101 sends a sound signal to the abnormal signal detecting framework 102, and the abnormal signal detecting framework 102 calculates a high-frequency energy value P0 under the full-decibel scale of the sound signal according to the sound signal, specifically:

firstly, reading sound signals of n time windows, calculating a high-frequency energy value of the sound signal of each time window under the full decibel scale according to an actual high-frequency energy value of the sound signal of each time window, and storing the high-frequency energy value in a queue 1, wherein the calculation formula is as follows:

pb, as shown in the formula (7)_iIs the full decibel high frequency energy value of the high frequency signal segment in the ith time window, Sb_iThe actual high-frequency energy value of the high-frequency signal segment in the ith time window and the high-frequency energy value can be obtained through short-time Fourier transform and other modes. Sj is the number of bits S needed by the electronic equipment to store the high-frequency signal segment in the ith time window_j＝2^j-1, to S_jFor explanation, reference may be made to the foregoing embodiments, which are not described in detail herein.

Secondly, in order to eliminate the high frequency signal interference caused by the frequency shift phenomenon of the environmental noise signal, the environmental noise energy value Pm is subtracted from the full decibel high frequency energy value of the sound signal calculated according to the formula (7), and the calculation formula can be:

Pc_i＝Pb_ipm formula (8)

Pb, as shown in the formula (8)_iThe method comprises the steps that a full decibel high-frequency energy value of a high-frequency signal section in the ith time window is obtained, Pci is a difference value between the full decibel high-frequency energy value of the high-frequency signal section in the ith time window and an environmental noise energy value, and Pm is the environmental noise energy value.

Finally, the average energy value of the difference between the full db high frequency energy value of the sound signal of the n time windows and the high frequency energy value Pm of the ambient noise signal is used as the high frequency energy value P0 of the sound signal. The calculation formula may be:

as shown in formula (9), Pci is a difference value between a full decibel high frequency energy value of a high frequency signal segment in the ith time window and the environmental noise energy value, and i is a positive integer less than or equal to n; p0 represents the high frequency energy value of the sound signal.

S508, the abnormal signal detecting frame 102 determines whether the high frequency energy value P0 of the sound signal is greater than the average ambient sound energy threshold Pt.

The abnormal signal detecting frame 102 compares the high frequency energy value P0 of the sound signal with the average environmental sound energy threshold, and if the high frequency energy value P0 of the sound signal is smaller than the average environmental sound energy threshold, the electronic device 100 continues to acquire a new sound signal from the current environment and calculates the high frequency energy value P0 of the new sound signal. Until the high frequency energy value P0 of the new sound signal is greater than the average ambient sound energy threshold.

If the abnormal signal detection framework 102 does not detect that the high frequency energy value P0 of the new sound signal is greater than the average environmental sound energy threshold, it indicates that there is no abnormal high frequency signal in the current environment of the user.

And S509, when the high-frequency energy value P0 of the sound signal is larger than the average environmental sound energy threshold value, the abnormal signal detection framework 102 sends an alarm prompt to the user.

The abnormal signal detection framework 102 determines that the high frequency energy value P0 of the sound signal is greater than the average environmental sound energy threshold, and then there is an abnormal high frequency signal in the current environment. The anomaly signal detection framework 102 outputs an alert prompt to the user.

The alarm prompt may be a user interface display prompt box of the electronic device 100 for prompting the user that an abnormal high-frequency signal exists in the current environment.

The alarm prompt may also be a prompt bar displayed on the edge of the display screen of the electronic device 100, where the prompt bar may be static or flashing and is used to prompt the user that an abnormal high-frequency signal exists in the current environment, and the prompt bar may receive a user click operation, and in response to the user click operation, the user interface of the electronic device 100 displays the prompt bar. The effect of displaying the cue bar is that the current operation of the user, such as the user being in a video call, may not be affected, and the cue bar does not affect the quality of the call (e.g., a stuck condition) of the user.

The alert prompt may also be a voice prompt, such as the electronic device 100 playing an alert sound through a speaker, etc.

The alert prompt may also be a vibration of the electronic device 100.

The alert prompt may also be a status bar display prompt message for the electronic device 100.

The alert prompt may also be a flashlight light flashing of the electronic device 100.

It is to be understood that the alert prompt may be a combination of two or more alert prompt notification manners, and the present application is not limited herein.

In some possible embodiments, when the abnormal signal detection frame 102 detects the abnormal high-frequency signal, the electronic device 100 outputs the warning prompt to the user for displaying only once, and when the abnormal signal detection frame 102 detects the abnormal high-frequency signal again, the abnormal signal detection frame 102 directly filters out the abnormal high-frequency signal in the sound signal, and the electronic device 100 does not output the warning prompt to the user any more, so that the current operation of the user is not affected.

S510, the abnormal signal detection framework 102 receives a filtering instruction.

S511, in response to the filtering instruction, the abnormal signal detecting frame 102 filters out the abnormal high frequency signal in the sound signal.

After the user selects and executes filtering of abnormal high-frequency signals in the sound signals, the electronic device 100 generates a filtering instruction according to the operation of the user, the electronic device 100 sends the filtering instruction to the abnormal signal detection framework 102, the abnormal signal detection framework 102 receives and responds to the filtering instruction, and the abnormal signal detection framework 102 filters the abnormal high-frequency signals in the sound signals.

Specifically, after the user selects to perform the filtering operation, the filtering module 1025 in the abnormal signal detecting frame 102 may process the sound signal using a low-pass filter or a band-stop filter, i.e., filter out the abnormal high-frequency signal in the sound signal.

Here, a band-stop filter is taken as an example, and the form of the filter is:

y_IRR(m)＝b(0)x(m)+b(1)x(m-1)+…+b(p)x(m-p)-a(1)y_IRR(m-1)-a(2)y_IRR(m-2)-…--a(q)y_IRR(m-q) formula (10)

As shown in equation (10), x (m) represents a sound signal collected at time m, m represents a certain time of the sound signal, p and q represent a certain time in the middle of time 0 to time m, a (q) is a correlation coefficient of a filter at time q, and b (p) is a correlation coefficient of a filter at time p.

S512, the abnormal signal detection framework 102 calls the sound collection module 101.

S513, the sound collection module 101 sends the sound signal with the abnormal high frequency signal filtered out to the application 103.

The abnormal signal detection framework 102 calls the sound collection module 101, and the sound collection module 101 sends the sound signal with the abnormal high-frequency signal filtered out to the application 103, so that the operation of a user can not be influenced.

In the above video call application scenario, the sound collection module 101 sends the sound signal with the abnormal high-frequency signal filtered out to the communication module in the electronic device 100, and the communication module is configured to send the sound signal with the abnormal high-frequency signal filtered out to the receiving device.

In some embodiments, the abnormal signal detection framework 102 does not need to call the sound collection module 101, and the abnormal signal detection framework 102 directly sends the sound signal with the abnormal high-frequency signal filtered out to the application 103.

For example, in the above video call application scenario, the abnormal signal detection framework 102 does not need to call the sound collection module 101, and the abnormal signal detection framework 102 directly sends the sound signal with the abnormal high-frequency signal filtered out to the communication module in the electronic device 100, where the communication module is configured to send the sound signal with the abnormal high-frequency signal filtered out to the receiving device.

As shown in fig. 6, fig. 6 is a schematic flowchart of another abnormal signal filtering method according to an embodiment of the present application.

S601, the electronic device 100 acquires a detection model.

In some embodiments, the electronic device 100 may obtain the detection model from the server 200.

In other embodiments, the detection model may also be preset in the electronic device 100. Please be detailed herein.

The electronic device 100 may obtain the detection model before the abnormal high-frequency signal detection function is turned on, or may obtain the detection model after the abnormal high-frequency signal detection function is turned on, which is not limited herein.

In some embodiments, the alert application 1024 is present in the electronic device 100 in the form of a system application.

The abnormal high frequency signal detection function in the electronic device 100 may be turned on by the electronic device 100 receiving a user operation, or the electronic device 100 may be turned on automatically, which is not limited herein.

As shown in fig. 7A to 7C, fig. 7A to 7C are UI diagrams of the warning application 1024 receiving a user operation to turn on the abnormal high-frequency signal detection function.

Fig. 7A is a UI diagram of a setting user interface including the alert application 1024 in the electronic apparatus 100.

As shown in FIG. 7A, FIG. 7A includes a settings user interface 700 of the electronic device 100.

The settings user interface 700 includes one or more function switch controls, such as an airplane mode control 7001, a wireless local area network control 7002, a bluetooth control 7003, an alert application control 7004, and a notification control 7005.

The flight mode control 7001 displays off, the wireless local area network control 7002 displays on, the bluetooth control 7003 displays on, the alarm application control 7004 displays off, and the notification control 7005 displays on.

It should be noted that the setting user interface 700 may include more or fewer setting controls, and the application is not limited herein.

The alert application control 7004 may receive a user operation (e.g., a single click), and in response to the user operation (e.g., a single click), the electronic device 100 displays an alert application user interface 710 as shown in fig. 7B.

Fig. 7B is a UI diagram of an alert application setting user interface in the electronic apparatus 100.

The alarm application setting user interface can receive user operation to start or close the abnormal high-frequency signal detection function, and can also receive time for setting the abnormal high-frequency signal detection function by the user operation.

The alert application user interface 710 includes an alert application control 7004, the alert application control 7004 displays "close", and when the alert application control 7004 displays "close", the abnormal high-frequency signal detection function is closed; a detection time control 7006, the detection time control 7006 being displayed as "07: 00-22: 00 ", i.e. the electronic device 100 is in a state of" 07: 00-22: 00', consumption can be saved by detecting whether the sound signals include abnormal high-frequency signals, resting the user at home in the rest time period, turning off the function of detecting the abnormal high-frequency signals and not detecting the electronic equipment 100.

Fig. 7C is a UI diagram of the function of detecting the abnormal high-frequency signal when the alarm application receives a user operation and turns on in the electronic device 100.

When the alert application control 7004 displays "off", the alert application control 7004 may receive a user operation (e.g., a single click) to turn on the abnormal high-frequency signal detection function.

When the alert application control 7004 displays "on," the alert application control 7004 may receive a user operation (e.g., a single click) to turn off the abnormal high-frequency signal detection function.

Fig. 7D to 7F are UI diagrams of the electronic apparatus 100 receiving a user operation to set the abnormal high-frequency signal detection function on time.

As shown in fig. 7D, a detection time control 7006 may receive a user operation (e.g., clicking) to set the on time of the abnormal high-frequency signal detection function.

In response to the user clicking on the detection time control 7006, the electronic device 100 displays a set detection time user interface 720 as shown in fig. 7E, the user interface 720 including a time selection control 7007, a save control 7008.

The time selection control 7007 may receive a single finger of the user to slide up or down to select a time to be set by the user, for example, if the user sets the detection time to "06: 00-21: 00", the abnormal high-frequency signal detection function is turned on during the time period of "06: 00-21: 00", that is, the electronic device 100 performs whether the sound signal includes the abnormal high-frequency signal detection during the time period of "06: 30-21: 30".

As shown in FIG. 7F, the time selection control 7007 in the user interface 720 is displayed as "06: 30-21: 30". The save control 7008 may receive a user operation (e.g., a single click), and in response to the user operation (e.g., a single click), the electronic apparatus 100 sets the turn-on period of the abnormal high-frequency signal detection function to "06: 30-21: 30".

The electronic apparatus 100 may also automatically turn on the abnormal signal detection function.

After the abnormal signal detection function in the electronic device 100 is turned on, the electronic device 100 detects the collected sound signal and detects whether an abnormal high-frequency signal exists in the sound signal.

In some embodiments, the electronic device 100 starts the abnormal signal detection function when the electronic device 100 is in the power-on state.

In some embodiments, when the electronic device 100 turns off the screen, the electronic device 100 turns off the abnormal signal detecting function.

In some embodiments, the electronic device 100 is in a fixed location (e.g., a library) for an extended period of time. When the electronic device 100 carried by the user just starts to reach the location, and after the electronic device 100 starts the abnormal signal detection function for a period of time, the electronic device 100 does not detect an abnormal high-frequency signal in the collected sound signal, because the electronic device 100 is located at the location (e.g., a library) for a long time and the surrounding environment of the location is single, the abnormal signal detection function can be turned off after being turned on for a period of time, thereby saving consumption.

In some embodiments, the electronic device 100 may selectively enable the abnormal signal detection function based on historical location information of the electronic device 100.

For example, in the time period "8: 00-22: 00 ", the electronic device 100 turns on the abnormal signal detection function because in the time period" 22: 00-8: 00 ", the user is at home, and the electronic device 100 detects that, at time period" 22: 00-8: 00 ", the historical location information (e.g., the location of the user's home) of the electronic device 100 is substantially the same, and thus the abnormal signal detection function may be turned off, reducing consumption.

Another example is the time period "9: 00-17: 00 ", electronic device 100 detects that, during time period" 9: 00-17: 00 ", the historical location information (e.g., the location of the company) of the electronic device 100 is substantially the same, and the environment of the company is relatively single, then the electronic device 100 may turn off the abnormal signal detection function.

Another example is the time period "7: 00-9: between 00 "and period" 17: 00-19: 00 ", the electronic device 100 starts the abnormal signal detection function. The electronic apparatus 100 determines, from the historical location information, that at time period "7: 00-9: 00 "the location information of the electronic device 100 is all from the first location to the second location, and in the time period" 17: 00-19: 00' when the position information of the electronic device 100 arrives at the first location from the second location, it can be determined that the user is on the way between work and work between the two time periods. Since the surrounding environment is complicated during work and work, the risk of acquiring the user privacy due to the hidden communication technology may exist, and the electronic device 100 starts the abnormal signal detection function to prevent the user privacy information from being leaked.

In some embodiments, the electronic device 100 may selectively turn on the abnormal signal detection function according to the behavior habits of the user collected by the electronic device 100.

For example, the electronic device 100 may detect that a website that the user frequently logs in may be a high-risk website, which may acquire user terminal information and user personal information, according to the user's internet surfing habits. When the user accesses such a website through the electronic device 100, the electronic device 100 starts an abnormal signal detection function, so as to prevent such a website from acquiring the privacy information of the user.

In some embodiments, the server may transmit an abnormal signal detection function on notification to the electronic apparatus 100, and in response to this on notification, the electronic apparatus 100 turns on the abnormal signal detection function.

The server may collect information reported by the plurality of electronic devices, and when one or more electronic devices of the plurality of electronic devices detect special information in a specific area (e.g., a mall or a supermarket), the one or more electronic devices report a condition that a special signal exists in the specific area to the server, and the server summarizes the information reported by the one or more electronic devices.

Before the user reaches the specific area, the server may send an abnormal signal detection function start notification to the electronic device 100 carried by the user, where the notification may be used to notify the electronic device 100 to start the abnormal signal detection function when the electronic device 100 is at a certain distance (e.g., 100 meters) from the specific area.

Or when the user reaches the specific area, the server sends an abnormal signal detection function starting notification to the electronic device 100 carried by the user, and the electronic device 100 carried by the user starts the abnormal signal detection function in response to the notification.

In some other embodiments, the server may send an advertisement of an advertisement service provider in a specific area (e.g., a shopping mall or a supermarket), and when the server 200 detects that special information exists in the advertisement, the server 200 sends an abnormal signal detection function start notification to the electronic device of the user, where the notification may be used to notify that the electronic device is at a certain distance (e.g., 100 meters) from the specific area or that the electronic device starts an abnormal signal detection function when the user arrives at the specific area.

In this way, a third party can be prevented from obtaining the privacy information of the user without knowing and authorizing the user.

S602, the electronic device 100 acquires an audio signal.

The electronic device 100 may receive a user operation to acquire a sound signal through the sound collection module.

The user operation may be a voice instruction. For example, the voice command may be a wake word "small E", the electronic device 100 receives and responds to the wake word "small E", which triggers the voice assistant to turn on, and in response to the voice assistant turning on, the electronic device 100 starts to collect the voice signal.

The user operation may also be an operation in which the user triggers the social application to communicate with the friend, and in response to the user operation, the electronic device 100 acquires a sound signal through the sound acquisition module and sends the sound signal to the electronic device on the friend side.

In some embodiments, the electronic device 100 may further acquire the sound signal by using a smart wearable device (e.g., a bluetooth watch, a bluetooth headset, etc.), the smart wearable device (e.g., the bluetooth watch, the bluetooth headset, etc.) establishes a communication connection with the electronic device 100, the smart wearable device (e.g., the bluetooth watch, the bluetooth headset, etc.) acquires the sound signal and transmits the sound signal to the electronic device 100, and the electronic device 100 acquires the sound signal.

S603, the electronic device 100 determines whether an abnormal high-frequency signal exists in the sound signal according to the detection model.

After the electronic device 100 confirms the high-frequency energy value P0 of the audio signal from the acquired audio signal, the electronic device 100 inputs the high-frequency energy value of the audio signal into the detection model, and if the electronic device 100 determines that the high-frequency energy value P0 of the audio signal is greater than the average ambient audio energy threshold, the audio signal contains an abnormal high-frequency signal.

Or after the electronic device 100 confirms the high-frequency energy value P0 of the sound signal according to the acquired sound signal, the electronic device 100 inputs the high-frequency energy value of the sound signal into the detection model, the electronic device 100 confirms the environmental sound energy threshold corresponding to the current environmental identifier from the detection model according to the sound signal, and if the electronic device 100 judges that the high-frequency energy value P0 of the sound signal is greater than the environmental sound energy threshold corresponding to the current environmental identifier, the sound signal has an abnormal high-frequency signal.

Here, the electronic device 100 may refer to the embodiment described in S507 in fig. 5 to determine the high-frequency energy value P0 of the sound signal according to the acquired sound signal, and details of this application are not repeated again.

The electronic device 100 determines whether the abnormal high-frequency signal exists in the sound signal according to the detection model, which is not described again in this application, with reference to the embodiment described in S501 in fig. 5.

If the electronic device 100 determines that an abnormal high-frequency signal exists in the sound signal, the electronic device 100 may send an alarm prompt to the user.

Here, the alarm prompt may refer to the embodiment described in S509 in the embodiment of fig. 5, and details thereof are not repeated here.

For example, when a user is in a shopping mall, music is being played in the shopping mall, the user makes a video call with a friend, the electronic device 100 collects a sound signal through the sound collection module, and if hidden communication information (e.g., an abnormal high-frequency signal) exists in the music played in the shopping mall, the hidden communication information (e.g., the abnormal high-frequency signal) may steal user privacy information existing in the electronic device 100. Hidden communication information (e.g., abnormal high frequency signals) in the sound signal may be detected and filtered out in the electronic device 100.

Fig. 8A to 8C are UI diagrams of the electronic device 100 displaying the alert prompt.

Fig. 8A includes a UI diagram of a user interface 801 for a user to video-call with a buddy.

The electronic device 100 collects the user image and sound signal and sends the user image and sound signal to the user friend end device, and meanwhile, the electronic device 100 receives and displays the user friend image and sound signal collected by the user friend end device.

The user interface 801 includes a first screen 8001, a second screen 8002, a shooting direction before and after switching control 8003, a call ending control 8004, and a mute control 8005.

The first screen 8001 is a user image captured by the electronic device 100.

The second screen 8002 is an image of a friend of the user received and displayed by the electronic device 100.

The flip shooting direction control 8003 can receive a user operation (e.g., clicking), and if the current shooting direction of the electronic device 100 is the front direction (e.g., the self-timer shooting direction), the electronic device 100 converts the shooting direction to the rear direction; if the current shooting direction of the electronic device 100 is the rear direction, the electronic device 100 converts the shooting direction to the front direction (e.g., the self-timer shooting direction).

The end call control 8004 can receive a user operation (e.g., a single click), and the electronic device 100 ends the current video call.

The mute control 8005 may receive a user operation (e.g., a single click), and the electronic device 100 may no longer send the sound signal collected by the electronic device 100 to the user's friends.

In one possible implementation, when the electronic device 100 detects that hidden communication information (e.g., an abnormal high frequency signal) exists in the current environment, as shown in fig. 8B, the user interface 801 of the electronic device 100 displays a prompt box 8006, where the prompt box 8006 is used for prompting the user that hidden communication information (e.g., an abnormal high frequency signal) exists in the current environment, and the prompt box 8006 includes "detect an abnormal high frequency signal, do it filter? "yes" control 8007, and "no" control 8008.

The yes control 8007 can receive a user operation (e.g., clicking), and in response to the user operation (e.g., clicking), the abnormal signal detection framework 102 will filter out hidden communication information (e.g., abnormal high frequency signals) in the sound signal.

The no control 8008 may receive a user operation (e.g., a single click), and in response to the user operation (e.g., the single click), the abnormal signal detecting frame 102 does not filter out hidden communication information (e.g., an abnormal high frequency signal) in the sound signal.

In another possible implementation, as shown in fig. 8C, the user interface 801 displays a cue bar 8009, and the cue bar 8009 is used for prompting the user that hidden communication information (e.g., abnormal high frequency signals) exists in the current environment. This may not affect the current operation of the user.

The prompt bar 8009 may receive a user operation (e.g., single click), and in response to the user single click operation, the user interface 801 displays a prompt box 8006 as shown in fig. 8B.

S604, the electronic device 100 filters the abnormal high-frequency signal in the sound signal according to the user confirmation filtering operation.

In some embodiments, after the electronic device 100 detects that an abnormal high-frequency signal exists in the sound signal, the user interface of the electronic device 100 displays an alarm prompt, the alarm prompt may receive a user operation (e.g., a single click), and in response to the user operation, the electronic device 100 filters out the abnormal high-frequency signal in the sound signal.

In some embodiments, when the electronic device 100 detects the abnormal high frequency signal, the electronic device 100 directly filters the abnormal high frequency signal from the sound signal, and the electronic device 100 does not output the warning prompt to the user.

For example, fig. 8D is a UI diagram of the electronic device 100 displaying the prompt message "abnormal high frequency signal has been filtered out".

In response to the user clicking the "yes" control 8007 shown in fig. 8B, as shown in fig. 8D, the user interface 801 of the electronic apparatus 100 displays a prompt box 8010 for prompting the user that the electronic apparatus 100 has filtered out the abnormal high frequency signal in the sound signal, and the prompt box 8010 includes a prompt message of "the abnormal high frequency signal has been filtered out".

In other possible embodiments, the user is driving, the user interface of the electronic device 100 is displaying a navigation route, the bluetooth headset has established a communication connection with the electronic device 100, when the electronic device 100 detects that an abnormal high-frequency signal exists nearby, the user interface of the electronic device 100 does not display an alarm prompt, so that the navigation route of the user is not affected, but a voice prompt message is played through the bluetooth headset to prompt that an abnormal high-frequency signal exists nearby, and the user can inform the electronic device 100 in a voice input manner to filter the abnormal high-frequency signal.

In other possible embodiments, the bluetooth watch has established a communication connection with the electronic device 100, and when the electronic device 100 detects that an abnormal high frequency signal exists nearby, the electronic device 100 does not display the warning message, but prompts the abnormal high frequency signal through the bluetooth watch in a manner of vibrating the watch, prompting with voice, or the like, and the user informs the electronic device 100 to filter out the abnormal high frequency in a manner of voice input. In other possible embodiments, after the user filters out the abnormal high frequency signal detected by the electronic device 100, the user may report the abnormal high frequency signal to the server 200, and the server receives the abnormal high frequency signal information uploaded by the electronic device 100. The server 200 counts the abnormal high-frequency signal information uploaded by each user in each area, counts the probability of the abnormal high-frequency signal information in each area, and informs the map manufacturer of the area with high probability of the abnormal high-frequency signal information, when the user plans a route by using the map, the navigation route displayed by the map can prevent the user from passing through the area with high probability of the abnormal high-frequency signal information, or the map prompts the user to 'have the abnormal high-frequency signal nearby' by voice when the user is close to the area with high probability of the abnormal high-frequency signal information, so that personal information of the user is prevented from being leaked.

It should be noted that the above embodiments can be combined with the embodiments shown in fig. 3 to 5.

Referring to fig. 9, fig. 9 is a schematic view of an apparatus according to an embodiment of the present disclosure. The apparatus 910 includes a sound collector 9001 and a processor 9002. Wherein:

the sound collector 9001 can be used to collect and transmit sound signals to the electronic device 100.

The processor 9002 may be configured to process a sound signal, and specifically include:

1. the processor 9002 determines an ambient sound energy threshold from the sound signal.

2. The processor 9002 determines a high frequency energy value of the sound signal from the sound signal.

3. The processor 9002 determines the high frequency energy value of the sound signal and the threshold of the environmental sound energy, and determines that an abnormal high frequency signal exists in the sound signal if the high frequency energy value of the sound signal is greater than the threshold of the current environmental sound energy.

4. The processor 9002 sends an alert message to the user.

5. The processor 9002 filters out abnormal high frequency signals from the sound signals according to the filtering operation of the user.

6. The processor 9002 invokes the sound collector 9001, and the sound collector 9001 transmits the sound signal with the abnormal high-frequency signal filtered out to the application of the electronic device 100.

The sound collector 9001 may perform each step performed by the sound collecting module, which may specifically refer to the foregoing embodiments and is not described herein again.

The processor 9002 may perform each step performed by the foregoing abnormal signal detecting framework, which may specifically refer to the foregoing embodiments and will not be described herein again.

The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for filtering an anomaly signal, comprising:

the electronic equipment receives a sound signal acquisition request;

in response to the sound signal acquisition request, the electronic equipment acquires a sound signal;

the electronic equipment determines a high-frequency energy value of the sound signal according to the sound signal;

and if the high-frequency energy value of the sound signal is greater than the environmental sound energy threshold value, the electronic equipment sends prompt information to a user, and the prompt information is used for prompting that the sound signal has an abnormal high-frequency signal.

2. The method of claim 1, wherein after the electronic device issues the prompt to the user, the method further comprises:

the electronic equipment receives filtering operation of a user;

in response to the filtering operation, the electronic device filters the abnormal high frequency signal.

3. The method of claim 2, further comprising:

and the electronic equipment sends the sound signal subjected to the filtering of the abnormal high-frequency signal to receiving equipment.

4. The method according to any one of claims 1 to 3, further comprising:

the electronic equipment acquires first environment information, wherein the first environment information comprises position information and/or environment sound;

the electronic device determines the ambient sound energy threshold according to the first ambient information.

5. The method according to any one of claims 1 to 3, wherein the electronic device determines the high-frequency energy value of the sound signal according to the sound signal, specifically comprising:

the electronic equipment acquires environmental sound before acquiring the sound signal;

the electronic equipment determines an environmental noise energy value according to the environmental sound;

and the electronic equipment determines the high-frequency energy value of the sound signal according to the actual high-frequency energy value of the sound signal and the environment noise energy value.

6. The method according to claim 5, wherein the determining, by the electronic device, the high-frequency energy value of the sound signal according to the actual high-frequency energy value of the sound signal and the environmental noise energy value specifically comprises:

the electronic equipment acquires a high-frequency signal in the sound signal;

the electronic equipment divides the high-frequency signal into high-frequency signal segments of n time windows, wherein n is a positive integer;

the electronic equipment calculates the actual high-frequency energy value of the high-frequency signal segment in the n time windows;

the electronic equipment calculates the full decibel high-frequency energy value of the high-frequency signal segment in the n time windows according to the actual high-frequency energy value of the high-frequency signal segment in the n time windows;

and the electronic equipment determines the high-frequency energy value of the sound signal according to the full-decibel high-frequency energy value of the high-frequency signal segment in the n time windows and the environmental noise energy value.

7. The method according to claim 6, wherein the electronic device calculates full-decibel high-frequency energy values of the high-frequency signal segments in the n time windows according to the actual high-frequency energy values of the high-frequency signal segments in the n time windows, specifically comprising:

the electronic equipment determines the full-decibel high-frequency energy value of the high-frequency signal segment in the n time windows through the following formula:

in which Pb is_iIs the full decibel high frequency energy value of the high frequency signal segment in the ith time window, Sb_iAnd for the actual high-frequency energy value of the high-frequency signal segment in the ith time window, Sj is the number of bits required by the electronic equipment for storing the high-frequency signal segment in the jth time window, j is a positive integer which is greater than or equal to 1 and less than or equal to n, and i is a positive integer which is less than or equal to n.

8. The method according to claim 6, wherein the determining, by the electronic device, the high-frequency energy value of the sound signal according to the full-decibel high-frequency energy values of the high-frequency signal segments in the n time windows and the environmental noise energy value specifically comprises:

the electronic equipment determines the high-frequency energy value of the sound signal through the following formula:

Pc_i＝Pb_i–Pm；

in which Pb is_iThe method comprises the steps that a full decibel high-frequency energy value of a high-frequency signal segment in an ith time window is obtained, Pci is a difference value between the full decibel high-frequency energy value of the high-frequency signal segment in the ith time window and the environmental noise energy value, Pm represents the environmental noise energy value, and i is a positive integer less than or equal to n; p0 represents the high frequency energy value of the sound signal.

9. The method according to any one of claims 1 to 8, wherein the frequency band of the abnormal high frequency signal comprises 18KHz-20 KHz.

10. The filtering device of an abnormal signal is characterized by comprising one or more processors, one or more memories and a sound collector; the one or more memories, the sound collector, and the one or more processors are coupled to store computer program code, the computer program code including computer instructions that are invoked by the one or more processors to cause the apparatus to perform the method of any of claims 1-9.

11. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the method of any of claims 1 to 9.