CN112104949A

CN112104949A - Method and device for detecting pickup assembly and electronic equipment

Info

Publication number: CN112104949A
Application number: CN202010912980.8A
Authority: CN
Inventors: 齐建永
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Beijing ByteDance Network Technology Co Ltd
Priority date: 2020-09-02
Filing date: 2020-09-02
Publication date: 2020-12-18
Anticipated expiration: 2040-09-02
Also published as: CN112104949B

Abstract

The embodiment of the disclosure discloses a method and a device for detecting a pickup assembly and electronic equipment. One embodiment of the method is applied to a terminal device, the terminal device comprises a playing component and a pickup component, and the method comprises the following steps: playing, by the playing component, the detection sound indicated by the target sound signal; picking up the detection sound and generating a first sound signal by the sound pickup assembly; in response to determining that a preset detection condition is satisfied, determining whether the pickup assembly is malfunctioning based on the target sound signal and the first sound signal. Thus, a means for detecting a pickup assembly is provided.

Description

Method and device for detecting pickup assembly and electronic equipment

Technical Field

The present disclosure relates to the field of computer hardware, and in particular, to a method and an apparatus for detecting a pickup assembly, and an electronic device.

Background

With the development of the internet, users use more and more functions of terminal equipment, so that work and life are more convenient.

For example, with the large-scale development of global artificial intelligence technology in recent years, an important application field, namely intelligent voice technology, is adopted by more devices, and almost various devices have the capability of voice interaction, so that the convenience of using the devices by people is greatly improved. However, the important components of these sound devices, such as microphone arrays, may be damaged with a certain probability during the use process, including the microphone components and corresponding hardware links, which may cause the user to have limited functions or reduced performance during the use of the device.

Disclosure of Invention

This disclosure is provided to introduce concepts in a simplified form that are further described below in the detailed description. This disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

In a first aspect, an embodiment of the present disclosure provides a method for detecting a pickup assembly, which is applied to a terminal device, where the terminal device includes a play assembly and a pickup assembly, and the method includes: playing, by the playing component, the detection sound indicated by the target sound signal; picking up the sound for detection and generating a first sound signal by the sound pickup assembly; in response to determining that a preset detection condition is satisfied, determining whether the pickup assembly is malfunctioning based on the target sound signal and the first sound signal.

In a second aspect, an embodiment of the present disclosure provides an apparatus for detecting a pickup assembly, which is applied to a terminal device, where the terminal device includes a playing assembly and a pickup assembly, and the apparatus includes: a playing unit for playing the detection sound indicated by the target sound signal through the playing component; a sound pickup unit configured to pick up the detection sound and generate a first sound signal by the sound pickup assembly; a determination unit configured to determine whether the sound pickup assembly is malfunctioning based on the target sound signal and the first sound signal in response to a determination that a preset detection condition is satisfied.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: one or more processors; storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to carry out a method according to the first aspect or according to the second aspect.

In a fourth aspect, the disclosed embodiments provide a computer readable medium, on which a computer program is stored, which when executed by a processor, implements a method according to the first aspect or according to the second aspect.

According to the method for detecting the pickup assembly provided by some embodiments of the present disclosure, a detection sound played by the playing assembly is picked up by the pickup assembly, and a first sound signal is generated; then, in response to the preset detection condition being satisfied, it is determined whether the sound pickup assembly is malfunctioning based on the target sound signal and the first sound signal, it is possible for the terminal device itself to pick up sound while playing the sound, and it is determined whether the sound pickup device is malfunctioning. Therefore, a new mode for detecting the pickup assembly is provided, and whether the terminal where the pickup equipment is located breaks down is determined, so that the equipment required for detection can be reduced, and the detection cost is reduced.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale.

FIG. 1 is a flow diagram of one embodiment of a method for detecting a pickup assembly according to the present disclosure;

fig. 2 is an exemplary architecture diagram of an electronic device with smart voice functionality according to the present disclosure.

FIG. 3 is a flow diagram of another embodiment of a method for detecting a pickup assembly according to the present disclosure;

FIG. 4 is a schematic block diagram of one embodiment of an apparatus for detecting a pickup assembly according to the present disclosure;

FIG. 5 is a system architecture diagram in which one embodiment of the present disclosure may be applied;

fig. 6 is a schematic diagram of a basic structure of an electronic device provided according to an embodiment of the present disclosure.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence relationship of the functions performed by the devices, modules or units.

It is noted that references to "a", "an", and "the" modifications in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will recognize that "one or more" may be used unless the context clearly dictates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

Referring to fig. 1, a flow diagram of one embodiment of a method for detecting a pickup assembly according to the present disclosure is shown. The method for detecting the pickup assembly is applied to the terminal equipment, and the terminal equipment can comprise a playing assembly and the pickup assembly. The method for detecting a pickup assembly as shown in fig. 1 includes the steps of:

step 101, playing the detection sound indicated by the target sound signal through the playing component.

In this embodiment, the execution subject (e.g., terminal device) of the method for detecting a sound pickup assembly may play the detection sound indicated by the target sound signal through the play assembly.

Here, the terminal device may include a play component and a sound pickup component.

Here, the playback component may include sound playback front end hardware and a speaker. The sound playing front-end hardware is mainly used for converting digital sound signals into analog signals for driving a loudspeaker. The sound playing front end hardware may include digital-to-analog converters, power amplifiers, etc.

Here, the pickup assembly may include a microphone and sound pickup front end hardware. Here, the sound pickup front end hardware converts an analog signal picked up by a microphone (including a single microphone and a microphone array) into a digital signal. The sound pickup front end hardware may include a programmable amplifier, analog to digital conversion means.

Here, the target sound signal may be a digital sound signal in a time domain. It can be understood that the sound playing front-end hardware may convert the digital sound signal (target sound signal) in the time domain into an analog signal, and the analog signal may be used to drive a speaker to generate sound, that is, the speaker may play the sound for detection. The detection sound is an acoustic signal.

The detection sound may be a general term for various sounds. In other words, the detection sound may not include the voice of the person speaking.

Here, the "target" in the target audio signal is added for convenience of description, and does not limit the audio signal. The content of the target sound signal may be set according to an actual application scenario, and is not limited herein.

Here, the determination method of the target sound signal may be set according to an actual application scenario, and is not limited herein. As an example, the target sound signal may be set in advance, or may be acquired in real time.

Step 102, picking up the sound for detection through the sound pickup assembly and generating a first sound signal.

In this embodiment, the execution body may pick up the detection sound played by the playing component and generate the first sound signal through the sound pickup component.

Here, the microphone may acquire the detection sound, convert an acoustic signal into an electric signal, and generate an analog signal corresponding to the detection sound. Then, the sound pickup front-end hardware corresponding to the microphone may convert the analog signal converted by the microphone into a digital signal (i.e., the first sound signal).

Here, the microphone in the sound pickup assembly may be a single microphone or a microphone array.

In some application scenarios, if a microphone array is included in the sound pickup assembly, each microphone in the microphone array may be connected to an analog-to-digital conversion circuit (i.e., sound pickup front-end hardware) corresponding to the microphone, and a sound signal acquired by the microphone may be output separately. Therefore, each microphone in the microphone array performs the method of the present embodiment, i.e. the detection of the microphone and the corresponding sound pickup front end hardware can be completed.

In some application scenarios, if the sound pickup assembly includes a microphone array and a user desires to detect the performance of the whole microphone array, the whole microphone array may also be regarded as the sound pickup assembly, and the method shown in this embodiment is performed, so that the detection of the whole performance of the microphone array can be completed.

And 103, in response to the preset detection condition being met, determining whether the pickup assembly is in failure or not based on the target sound signal and the first sound signal.

In this embodiment, the execution body may determine whether the sound pickup assembly malfunctions based on the target sound signal and the first sound signal in response to the detection instruction.

Here, the preset detection condition may indicate a condition for detecting the sound pickup assembly. The specific content of the preset detection condition may be set according to an actual application scenario, and is not limited herein.

As an example, the preset detection condition may include at least one of, but is not limited to: presetting detection environment conditions and terminal calculation allowance conditions.

As an example, the terminal calculation margin condition may indicate a calculation capability of the terminal when the pickup assembly detection is performed. For example, the terminal calculation margin condition may include that the remaining calculation power of the terminal is greater than a preset remaining calculation power threshold. In other words, the detection of the sound pickup assembly is performed only when the remaining computing power of the terminal is sufficient.

In this embodiment, the determining whether the sound pickup assembly is failed or not based on the target sound signal and the first sound signal may be implemented in various ways, which is not limited herein.

As an example, a similarity between the target sound signal and the first sound signal may be calculated. If the similarity is greater than a preset similarity threshold, it may be determined that no failure has occurred. If the similarity is not greater than the preset similarity threshold, it can be determined that a fault has occurred.

It should be noted that, in practice, the performance of the components of the pickup assembly other than the microphone is generally stable, and therefore, if it is determined that the pickup assembly is out of order, it can be determined that the microphone in the pickup assembly is out of order.

It should be noted that, in the method for detecting a sound pickup assembly provided in this embodiment, a sound pickup assembly picks up a detection sound played by a playing assembly, and generates a first sound signal; then, in response to the preset detection condition being satisfied, it is determined whether the sound pickup assembly is malfunctioning based on the target sound signal and the first sound signal, it is possible for the terminal device itself to pick up sound while playing the sound, and it is determined whether the sound pickup device is malfunctioning. Therefore, a new mode for detecting the pickup assembly is provided, and whether the terminal where the pickup equipment is located breaks down is determined, so that the equipment required for detection can be reduced, and the detection cost is reduced.

In some embodiments, the method further comprises: in response to determining that the pickup assembly is malfunctioning, performing at least one of: presenting the fault prompt information, playing the fault prompt sound and sending the fault prompt information to the server.

Here, the fault notification information may be in the form of text or image, so that the fault notification information may be presented by the terminal device.

Here, the terminal device may play a voice-form failure indication sound.

Here, the terminal device may transmit the failure indication information to the server. And storing fault prompt information corresponding to the sound pickup assembly of the terminal equipment by the server.

Therefore, fault information about the sound pickup equipment can be quickly prompted, and the intelligent voice function of the terminal equipment is prevented from being influenced.

In some application scenarios, if the tone pickup device terminal includes multiple microphones, prompt information indicating a single microphone fault may be generated, facilitating the location of the faulty microphone.

In some embodiments, the target sound signal may include at least one of, but is not limited to: presetting a sound signal and acquiring the sound signal in real time.

Here, the preset audio signal may be an audio signal preset in the terminal. Optionally, the preset sound signal may be specifically set for the detection and sound pickup assembly, or may have other functions besides the detection and sound pickup assembly.

As an example, the preset sound signal may include at least one of, but is not limited to: a power-on prompt sound signal and a wake-up prompt sound signal. It will be appreciated that the power-on prompt tone and the wake-up prompt tone may have other functions in addition to detecting the pickup assembly.

As an example, the preset sound signal may include a detection sound signal provided specifically for detection by the sound pickup assembly.

Here, the real-time acquired audio signal may refer to an audio signal that the execution main body acquires from a local or other electronic device in real time and is converted in real time by the audio playback front-end hardware. As an example, the sound signal acquired in real time as described above may indicate a song played in real time.

It should be noted that, the target sound signal includes a preset sound signal, which can improve the accuracy of detection. In other words, the preset sound signal is used as the target sound signal, so that when the pickup assembly is detected, the interference caused by the fact that the target sound signal is used as an uncontrollable factor can be eliminated, the detection mode or the detection parameters can be set aiming at the preset sound signal in advance, and the detection accuracy is improved.

In some embodiments, the preset speech signal comprises at least one of: a power-on prompt voice signal and a wake-up prompt voice signal; and the method further comprises: and in response to determining to power on or to be awakened, determining the corresponding preset voice signal as the target sound signal. In some embodiments, the determining, in response to determining to power on or to be woken up, a corresponding preset voice signal as the target sound signal may include: and in response to determining to power on, determining a preset power on prompt sound signal as a target sound signal.

Here, the terminal determines to power on, which may be that the terminal detects that a user performs a predefined power-on operation. The specific content of the predefined boot operation can be set according to the actual application scenario. As an example, the predefined power-on operation may be the triggering of a power-on button.

Here, the specific content of the power-on prompt sound signal may be set according to an actual application scenario, and is not limited herein.

As an example, the power-on prompt tone signal may indicate a tone "power on, waiting to serve you".

It should be noted that, the starting-up prompting sound signal is used as the target sound signal, and the pickup assembly can be detected each time the computer is started up, so that on one hand, the sound detection times can be increased, and whether the pickup assembly fails or not can be determined in time; on the other hand, the sound which needs to be played on the terminal equipment can be used as the sound for detection, so that the condition that extra sound needs to be made during detection is avoided, and the interference to the user operation caused by detection is reduced.

In some embodiments, in response to determining to power on or to be woken up, determining a corresponding preset voice signal as the target sound signal may include: in response to determining to be woken up, a preset wake-up cue sound signal is determined as the target sound signal.

In some application scenarios, the terminal is awakened, either by detecting an awakening word or by detecting a touch to the terminal.

Here, the specific content of the wake-up prompt sound signal may be set according to an actual application scenario, and is not limited herein.

As an example, the wake-up cue sound signal described above may indicate a sound "wake up, wait for you to service".

It should be noted that, the wake-up prompt sound signal is used as the target sound signal, and the pickup assembly can be detected each time the pickup assembly is awakened, so that on one hand, the sound detection times can be increased, and whether the pickup assembly fails or not can be determined in time; on the other hand, the sound which needs to be played on the terminal equipment can be used as the sound for detection, so that the condition that extra sound needs to be made during detection is avoided, and the interference to the user operation caused by detection is reduced.

In some embodiments, the method may further include: in response to determining that the user turns on the pickup assembly detection function, acquiring a preset sound signal for detection, and determining the sound signal for detection as a target sound signal.

In some application scenarios, the terminal device may set a pickup assembly detection function. When a user (e.g., an equipment user or an equipment service person) desires to detect the pickup assembly in consideration of a possibility of malfunction of the pickup assembly, the user triggers a pickup assembly detection function (e.g., triggers a control provided in an application controlling the terminal equipment). Then, the terminal device can turn on the automatic detection pickup assembly and feed back the detection result to the user.

Here, specific contents of the sound signal for detection may be set according to an actual application scenario, and are not limited herein.

As an example, the detection sound signal may indicate that the sound is "detecting, please wait".

When a user desires to detect the sound pickup unit, the sound pickup unit is detected using the detection sound signal as the target sound signal, so that the detection pertinence is improved and the amount of calculation required for the detection is reduced. In other words, when the user detects that the pickup assembly may have a problem, the number of detections performed when the pickup assembly has no problem can be reduced, thereby reducing the amount of calculations required for the detection.

In some embodiments, the method may further include: and in response to the detection of the sound signal to be played acquired in real time, determining the sound to be played acquired in real time as a target sound signal.

As an example, a user may select to play a song. When a song is played, the terminal device may need to acquire a sound signal of the song to be played in real time. Then, the terminal device may determine the sound signal of the song to be played as the target sound signal.

It should be noted that, by using the sound signal to be played acquired in real time as the target sound signal, the detection times can be increased, and the failure of the sound pickup apparatus can be found in time.

In some application scenarios, the terminal device may be an electronic device with smart sound functionality. An electronic device with an intelligent voice function, please refer to fig. 2, may generally include a network-side data transceiving module; the text-to-sound conversion module is used for converting the text to be broadcasted by the system into sound content; the sound effect post-processing module is used for optimizing the actual effect of main sound, and comprises processing such as a common equalizer and the like; the sound playing front-end hardware mainly converts the digital sound signal into an analog signal for driving the loudspeaker 201, and comprises a digital-to-analog converter, a power amplifier and the like; the sound pickup front-end hardware is used for converting an analog signal picked up by the microphone array 202 into a digital signal and comprises a programmable amplifier and an analog-to-digital conversion device; the sound signal front-end processing module is mainly used for enhancing the sound of the sound signal picked in front, and comprises echo elimination in the sound broadcasting process and ambient noise suppression, so that a high signal-to-noise ratio signal is provided for rear-end voice recognition, and the voice recognition rate is improved; the speech recognition and semantic understanding module is mainly used for converting the sound signals processed by the front end into texts and for translating and understanding the text expression semantics.

In some embodiments, a microphone array performance detection module and an environment detection module can be added to an electronic device with a smart voice function. The microphone array performance detection module is positioned in the module for processing the sound front-end signal. The microphone array performance detection module 207 includes two part inputs: a part of the signals from the hardware part comprises the input of the signals of each microphone; the other part is a software part, and the target sound signal is a post-processed signal. The environment detection module basically receives original sound signals (without front-end sound enhancement processing) of a microphone array, and comprises prompt tones and possible environment prompt tones emitted by a loudspeaker, wherein the environment detection module is mainly used for identifying whether the current sound playing prompt tones are recorded in a relatively quiet environment. If the judgment belongs to a relatively quiet environment, the sound recorded by the current microphone array can be used as the input of the performance detection of the microphone array module; otherwise, the current recording signal is abandoned and is not used as the basis for the performance detection of the subsequent microphone array.

In some embodiments, the step 103 may include: generating frequency response information to be detected of the pickup assembly according to the first frequency spectrum information and the target frequency spectrum information; and determining whether the pickup assembly breaks down or not according to the frequency response information to be detected.

Here, the first spectrum information is a frequency domain signal corresponding to the first sound signal. Namely, the first spectrum information is obtained by performing time-frequency transformation on the first sound signal. The target frequency spectrum information is a frequency domain signal corresponding to the target sound signal. Namely, the target frequency spectrum information can be obtained by performing time-frequency conversion on the target sound signal. Here, the method of time-frequency conversion for the first sound signal should be the same as the method of video conversion for the target sound signal, and the specific time-frequency conversion method is not described herein again.

Here, the difference value of the target spectrum information subtracted from the first spectrum information may be used as the frequency response information to be detected of the sound pickup apparatus.

Here, the frequency response information may refer to frequency response information. The frequency response information may indicate the responsiveness of the hardware device for various frequencies.

Here, the form of the frequency response information may be various. As an example, the frequency response information may include a frequency response curve.

Here, the frequency response information to be detected may be frequency response information of the video device. The information to be detected to indicate the frequency response information will be used for detection later, and does not constitute a limitation on the specific content of the frequency response information.

Here, whether the frequency response information meets the expectation or not may be determined according to a preset determination condition to determine whether the sound pickup assembly malfunctions or not.

As an example, it may be determined that the sound pickup assembly has a failure that a plurality of negative values, i.e., the first spectrum information does not respond to a plurality of frequencies so that the difference between the first spectrum information and the rated spectrum information is a negative value at the plurality of frequencies.

It should be noted that, by determining the to-be-detected frequency response information for the first frequency spectrum information and the target frequency spectrum information, the accuracy of detection can be improved. Specifically, adopt the judgement mode of waiting to detect frequency response information, can judge whether the response of pickup assembly in each frequency department goes wrong, from this, can improve the granularity of detection, can discover fast that pickup assembly roughly can pickup but the inaccurate condition of pickup.

In some embodiments, the determining whether the pickup assembly fails according to the to-be-detected frequency response information may include: determining whether the frequency response information to be detected is matched with the rated frequency response information; in response to determining a match, determining that the pickup assembly is not malfunctioning; in response to determining the mismatch, a failure of the pickup assembly is determined.

Here, the rated frequency response information is preset for the sound pickup assembly. Here, after the whole body structure of the terminal device is designed, the trend of the frequency response information (such as a frequency response curve) of each sound pickup assembly is basically fixed. Thus, the rated frequency response information can be preset for the sound pickup assembly.

Here, the specific determination manner for determining whether the to-be-detected frequency response information matches the rated frequency response information may be set according to an actual application scenario, and is not limited herein.

As an example, the similarity between the to-be-detected frequency response information and the rated frequency response information may be calculated as the matching degree. If the matching degree is greater than a preset matching threshold value, determining that the matching is performed; and if the matching degree is not greater than the preset matching threshold, determining that the matching degree is not matched.

As an example, a floating range threshold (e.g., ± 5) may be set, and then the floating range threshold may be superimposed with the nominal frequency response information to determine the nominal frequency response range. And then determining whether the frequency response information to be detected is within the rated frequency response range. If so, determining a match; if not, a mismatch is determined.

In some application scenarios, the device may have a certain nominal frequency response information for each microphone device before the device leaves the factory. Considering the slight difference between the sensitivity of the single pickup assembly and the cavity of the body structure, the frequency response of the individual pickup assemblies of the device may have a certain difference, and therefore, a floating range threshold value is set. Therefore, the frequency response information to be detected calculated in the using process of the terminal equipment can reduce the false detection rate while ensuring the detection rate through setting the floating range threshold.

Alternatively, different floating range thresholds may be set for microphones installed at different locations. Thereby, the detection accuracy for a single microphone can be improved.

It should be noted that by comparing the rated frequency response information with the to-be-detected frequency response information, the detection accuracy can be improved. Specifically, through rated frequency response information, the complete function (response at each frequency) of the pickup assembly can be detected, and for the detection of whether the pickup assembly works or the detection of frequency response at partial frequency, the pickup assembly can be comprehensively detected, so that the detection efficiency is improved.

In some embodiments, the predetermined detection condition includes a predetermined detection environmental condition. Before the step 103, the method may further include: and determining whether the sound playing environment meets a preset detection environment condition or not according to the first sound signal.

Here, specific contents of the preset detection environmental conditions may be set according to an actual application scenario, and are not limited herein.

In some embodiments, the preset detection environment condition may be associated with a specific determination manner.

It should be noted that, the preset detection environment condition is used as the premise of detection, so that it can be ensured that the environmental interference is small during detection, and the accuracy of detection is improved.

In some embodiments, determining the manner may include: determining a signal difference value of the first sound signal and the target sound signal, and taking the signal difference value as a background signal; if the background signal is larger than a preset background noise threshold value, determining that a preset test environment condition is met; and if the background signal is not greater than the preset background noise threshold value, determining that the preset test environment condition is met. It can be seen that the predetermined test condition associated with this determination is that the background signal is not greater than the predetermined background noise threshold.

In some embodiments, determining the manner may include: and performing voice recognition on the first sound signal, and determining whether the sound playing environment meets the preset detection environment condition or not according to the recognition result.

In some application scenarios, various speech recognition methods may be employed to perform speech recognition on the first sound signal. And, the specific parameters of the expected speech recognition result can be set according to the actual application scene.

As an example, the speech recognition result may be the number of sound sources indicated in the first sound signal. If the number of the sound sources is more than 1, determining that the sound playing environment does not meet the preset detection environment condition; and if the number of the sound sources is not more than 1, determining that the sound playing environment meets the preset detection environment condition.

It should be noted that, whether the sound playing environment meets the preset detection environment condition is determined by adopting a voice recognition mode, so that the requirement on the sound playing environment is reduced while the accuracy for determining whether the sound playing environment is proper can be improved, the detection times of detecting the pickup assembly is further improved, and faults are found in time. Specifically, the environment detection is performed by voice recognition, so that whether sound affecting the performance of the pickup assembly is detected or not in the environment can be determined in a targeted manner, and background noise which may not affect the performance detection of the pickup assembly is not concerned. Therefore, on the premise of ensuring the sound playing environment to be suitable, the requirements for the sound playing environment are reduced, the detection times are increased, and faults are found in time.

In some application scenarios, the target sound signal is a preset sound signal. The voice recognition of the first sound signal is performed, and whether the sound playing environment meets preset detection environment conditions or not is determined according to a recognition result, including: carrying out voice recognition on the first sound signal to obtain first semantic information; acquiring preset semantic information corresponding to a preset sound signal; and determining whether the sound playing environment meets a preset detection environment condition or not according to the semantic matching degree of the first semantic information and the preset semantic information.

Here, if the target sound signal is a preset sound signal, speech recognition may be performed on the first sound signal, and the speech recognition result includes a semantic meaning (first semantic meaning information) indicated by the first sound signal, that is, the speech recognition result may be the first semantic meaning information. Then, comparing the semantic meaning indicated by the first sound signal with the semantic meaning (preset semantic meaning information) indicated by the target sound signal, and if the comparison result shows that the accuracy (namely, semantic matching degree) is greater than an accuracy threshold, determining that the sound playing environment meets the preset detection environment condition; and if the comparison result shows that the accuracy is not greater than the accuracy threshold, determining that the sound playing environment does not meet the preset detection environment condition.

It should be noted that, the target sound signal includes a preset sound signal, and whether the sound playing environment meets the preset detection environment condition is determined by semantic comparison, so that the detection accuracy can be improved.

Referring to fig. 3, a flow diagram of one embodiment of a method for detecting a pickup assembly according to the present disclosure is shown. The method for detecting the pickup assembly is applied to the terminal equipment. The method for detecting a pickup assembly as shown in fig. 1 includes the steps of:

step 301, playing the detection sound indicated by the preset sound signal through the playing component.

Here, the preset sound signal includes at least one of: a power-on prompt sound signal and a wake-up prompt sound signal.

Step 302, picking up the sound for detection through a sound pickup assembly and generating a first sound signal.

And 303, performing voice recognition on the first sound signal, and determining whether the sound playing environment meets a preset detection environment condition according to a recognition result.

And 304, in response to the fact that the preset detection environment condition is determined to be met, generating frequency response information to be detected of the pickup assembly according to the first frequency spectrum information and the target frequency spectrum information.

Here, the first spectrum information is obtained by performing time-frequency transformation on the first sound signal, and the target spectrum information is a frequency domain signal corresponding to the target sound signal.

And 305, determining whether the to-be-detected frequency response information is matched with the rated frequency response information.

Here, the rated frequency response information is preset for the sound pickup assembly.

In response to determining a match, step 306 determines that the pickup assembly is not malfunctioning.

Step 307, in response to determining a mismatch, determining that the pickup assembly is malfunctioning.

It should be noted that, for detailed implementation of each step in the embodiment shown in fig. 2, reference may be made to descriptions of other parts in this application, and details are not described here again.

It should be noted that, in the embodiment corresponding to fig. 3, first, a preset sound signal is used as a target sound signal, so that when a pickup assembly is detected, interference caused by the target sound signal itself as an uncontrollable factor can be eliminated, and a detection mode or a detection parameter can be set for the preset sound signal in advance; moreover, whether the sound playing environment meets the preset detection environment condition is determined by adopting a voice recognition mode, so that the accuracy for determining whether the sound playing environment is proper can be improved, the requirement on the sound playing environment is reduced, the detection times of the pickup assembly are improved, and faults are found in time; in addition, the complete function (response at each frequency) of the pickup assembly can be detected through the rated frequency response information, and the pickup assembly can be comprehensively detected and the detection efficiency is improved compared with the detection of the approximate work of the pickup assembly or the detection of the frequency response at partial frequency.

With further reference to fig. 4, as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an apparatus for detecting a pickup assembly, which corresponds to the method embodiment shown in fig. 1, and which is particularly applicable to various electronic devices.

As shown in fig. 4, the apparatus for detecting a pickup assembly of the present embodiment includes: a playback unit 401, a sound pickup unit 402, and a determination unit 403. The playing unit is used for playing the detection sound indicated by the target sound signal through the playing component; a sound pickup unit configured to pick up the detection sound and generate a first sound signal by the sound pickup assembly; a determination unit configured to determine whether the sound pickup assembly is malfunctioning based on the target sound signal and the first sound signal in response to a determination that a preset detection condition is satisfied.

In this embodiment, specific processes of the playing unit 401, the sound pickup unit 402, and the determining unit 403 of the apparatus for detecting a sound pickup assembly and technical effects brought by the specific processes can refer to the related descriptions of step 101, step 102, and step 103 in the corresponding embodiment of fig. 1, which are not described herein again.

In some embodiments, the target sound signal comprises at least one of: presetting a sound signal and acquiring the sound signal in real time.

In some embodiments, the apparatus is further configured to: and in response to the determination of the power-on, determining a preset power-on prompting sound signal as the target sound signal.

In some embodiments, the apparatus is further configured to: in response to determining to be woken up, determining a preset wake-up prompt sound signal as the target sound signal.

In some embodiments, the apparatus is further configured to: in response to determining that the user turns on the pickup assembly detection function, acquiring a preset sound signal for detection, and determining the sound signal for detection as a target sound signal.

In some embodiments, the apparatus is further configured to: and determining the sound signal to be played acquired in real time as a target sound signal.

In some embodiments, the determining whether the pickup assembly is malfunctioning based on the target sound signal and the first sound signal in response to determining that a preset detection condition is satisfied includes: generating frequency response information to be detected of the pickup assembly according to first frequency spectrum information and target frequency spectrum information, wherein the first frequency spectrum information is obtained by performing time-frequency transformation on the first sound signal, and the target frequency spectrum information is a frequency domain signal corresponding to the target sound signal; and determining whether the pickup assembly breaks down or not according to the frequency response information to be detected.

In some embodiments, the determining whether the pickup assembly fails according to the to-be-detected frequency response information includes: determining whether the frequency response information to be detected is matched with rated frequency response information, wherein the rated frequency response information is preset aiming at the pickup assembly; in response to determining a match, determining that the pickup assembly is not malfunctioning; in response to determining the mismatch, a failure of the pickup assembly is determined.

In some embodiments, the preset detection condition includes a preset detection environment condition, wherein the preset detection environment condition indicates environment information for detecting the pickup assembly; and prior to said determining whether the pickup assembly is malfunctioning based on the target sound signal and the first sound signal in response to determining that a preset detection condition is satisfied, the apparatus is further configured to: and determining whether the sound playing environment of the sound for detection meets a preset detection environment condition or not according to the first sound signal.

In some embodiments, the determining whether the sound playing environment of the sound for detection satisfies a preset detection environment condition according to the first sound signal includes: and performing voice recognition on the first sound signal, and determining whether the sound playing environment meets a preset detection environment condition or not according to a recognition result.

In some embodiments, the target sound signal comprises a preset sound signal; and the voice recognition is carried out on the first sound signal, and whether the sound playing environment meets the preset detection environment condition or not is determined according to the recognition result, wherein the method comprises the following steps: carrying out voice recognition on the first sound signal to obtain first semantic information; acquiring preset semantic information corresponding to a preset sound signal; and determining whether the sound playing environment meets a preset detection environment condition or not according to the semantic matching degree of the first semantic information and the preset semantic information.

In some embodiments, the apparatus is further configured to: in response to determining that the pickup assembly is malfunctioning, performing at least one of: presenting the fault prompt information, playing the fault prompt information and sending the fault information to the server.

Referring to fig. 5, fig. 5 illustrates an exemplary system architecture in which a method for detecting pickup assemblies of one embodiment of the present disclosure may be applied.

As shown in fig. 5, the system architecture may include

terminal devices

501, 502, 503, a network 504, and a server 505. The network 504 serves to provide a medium for communication links between the

terminal devices

501, 502, 503 and the server 505. Network 504 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The

terminal devices

501, 502, 503 may interact with a server 505 over a network 504 to receive or send messages or the like. The

terminal devices

501, 502, 503 may have various client applications installed thereon, such as a web browser application, a search-type application, and a news-information-type application. The client application in the

terminal device

501, 502, 503 may receive the instruction of the user, and complete the corresponding function according to the instruction of the user, for example, add the corresponding information in the information according to the instruction of the user.

The

terminal devices

501, 502, 503 may be hardware or software. When the

terminal devices

501, 502, 503 are hardware, they may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, mpeg compression standard Audio Layer 3), MP4 players (Moving Picture Experts Group Audio Layer IV, mpeg compression standard Audio Layer 4), laptop portable computers, desktop computers, and the like. When the

terminal devices

501, 502, and 503 are software, they can be installed in the electronic devices listed above. It may be implemented as multiple pieces of software or software modules (e.g., software or software modules used to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.

The server 505 may be a server providing various services, for example, receiving an information acquisition request sent by the

terminal device

501, 502, 503, and acquiring the presentation information corresponding to the information acquisition request in various ways according to the information acquisition request. And the relevant data of the presentation information is sent to the

terminal equipment

501, 502, 503.

It should be noted that the method for detecting a sound pickup assembly provided by the embodiment of the present disclosure may be performed by a terminal device, and accordingly, the apparatus for detecting a sound pickup assembly may be provided in the

terminal device

501, 502, 503. In addition, the method for detecting the sound pickup assembly provided by the embodiment of the present disclosure may also be performed by the server 505, and accordingly, the apparatus for detecting the sound pickup assembly may be disposed in the server 505.

It should be understood that the number of terminal devices, networks, and servers in fig. 5 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Referring now to fig. 6, shown is a schematic diagram of an electronic device (e.g., a terminal device or a server of fig. 5) suitable for use in implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 6, the electronic device may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 illustrates an electronic device having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may interconnect with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: playing, by the playing component, the detection sound indicated by the target sound signal; picking up the sound for detection and generating a first sound signal by the sound pickup assembly; in response to determining that a preset detection condition is satisfied, determining whether the pickup assembly is malfunctioning based on the target sound signal and the first sound signal.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: receiving a live stream acquisition request; and determining whether to return the live stream or not according to the live stream acquisition request.

Computer program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not constitute a limitation of the unit itself in some cases, for example, the determination unit may also be described as a "unit that determines whether or not the sound pickup assembly is malfunctioning".

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), systems on a chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and (but not limited to) the features disclosed in this disclosure having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method for detecting a pickup assembly, applied to a terminal device, wherein the terminal device comprises a play assembly and a pickup assembly, the method comprising:

playing, by the playing component, the detection sound indicated by the target sound signal;

picking up the sound for detection and generating a first sound signal by the sound pickup assembly;

in response to determining that a preset detection condition is satisfied, determining whether the pickup assembly is malfunctioning based on the target sound signal and the first sound signal.

2. The method of claim 1, wherein the target sound signal comprises at least one of: presetting a sound signal and acquiring the sound signal in real time.

3. The method of claim 2, wherein the preset speech signal comprises at least one of: a power-on prompt voice signal and a wake-up prompt voice signal; and

the method further comprises the following steps:

and in response to determining to power on or to be awakened, determining the corresponding preset voice signal as the target sound signal.

4. The method of claim 2, further comprising:

in response to determining that the user turns on the pickup assembly detection function, acquiring a preset sound signal for detection, and determining the sound signal for detection as a target sound signal.

5. The method of claim 2, further comprising:

and determining the sound signal to be played acquired in real time as a target sound signal.

6. The method of claim 1, wherein determining whether the pickup assembly is malfunctioning based on the target sound signal and the first sound signal in response to determining that a preset detection condition is satisfied comprises:

generating frequency response information to be detected of the pickup assembly according to first frequency spectrum information and target frequency spectrum information, wherein the first frequency spectrum information is a frequency domain signal corresponding to the first sound signal, and the target frequency spectrum information is a frequency domain signal corresponding to the target sound signal;

and determining whether the pickup assembly breaks down or not according to the frequency response information to be detected.

7. The method of claim 6, wherein the determining whether the pickup assembly is malfunctioning according to the to-be-detected frequency response information comprises:

determining whether the frequency response information to be detected is matched with rated frequency response information, wherein the rated frequency response information is preset aiming at the pickup assembly;

in response to determining a match, determining that the pickup assembly is not malfunctioning;

in response to determining the mismatch, a failure of the pickup assembly is determined.

8. The method of claim 1, wherein the preset detection condition comprises a preset detection environmental condition, wherein the preset detection environmental condition indicates environmental information for detecting a pickup assembly; and

before the determining whether the pickup assembly is malfunctioning based on the target sound signal and the first sound signal in response to determining that a preset detection condition is satisfied, the method further includes:

and determining whether the sound playing environment of the sound for detection meets a preset detection environment condition or not according to the first sound signal.

9. The method according to claim 8, wherein determining whether the sound playing environment of the sound for detection satisfies a preset detection environment condition according to the first sound signal comprises:

and performing voice recognition on the first sound signal, and determining whether the sound playing environment meets a preset detection environment condition or not according to a recognition result.

10. The method of claim 9, wherein the target sound signal comprises a preset sound signal; and

the voice recognition of the first sound signal is performed, and whether the sound playing environment meets preset detection environment conditions or not is determined according to a recognition result, including:

carrying out voice recognition on the first sound signal to obtain first semantic information;

acquiring preset semantic information corresponding to a preset sound signal;

and determining whether the sound playing environment meets a preset detection environment condition or not according to the semantic matching degree of the first semantic information and the preset semantic information.

11. The method of claim 1, further comprising:

in response to determining that the pickup assembly is malfunctioning, performing at least one of: presenting the fault prompt information, playing the fault prompt information and sending the fault information to the server.

12. An apparatus for detecting a pickup assembly, applied to a terminal device, the terminal device including a play assembly and a pickup assembly, the apparatus comprising:

a playing unit for playing the detection sound indicated by the target sound signal through the playing component;

a sound pickup unit configured to pick up the detection sound and generate a first sound signal by the sound pickup assembly;

a determination unit configured to determine whether the sound pickup assembly is malfunctioning based on the target sound signal and the first sound signal in response to a determination that a preset detection condition is satisfied.

13. An electronic device, comprising:

one or more processors;

the playing component is used for playing the sound signal;

the sound pickup assembly is used for picking up sound signals;

a storage device for storing one or more programs,

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-11.

14. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 11.