WO2018210192A1 - Unmanned aerial vehicle monitoring method and audio/video linkage apparatus - Google Patents

Abstract

Embodiments of the present application provide an unmanned aerial vehicle monitoring method and an audio/video linkage apparatus. The unmanned aerial vehicle monitoring method comprises: acquiring a sound signal by means of an audio acquisition array; performing signal processing on the sound signal and obtaining, in the sound signal, at least one sound component with a signal characteristic satisfying a preset condition; determining the location of a target object corresponding to the sound component with the signal characteristic satisfying the preset condition; controlling a camera towards the location of target object; and determining whether the target object is an unmanned aerial vehicle according to an image photographed by the camera. According to the present solution, monitoring on an unmanned aerial vehicle can be achieved, and thus social and personal safety can be ensured.

Description

UAV monitoring method and audio and video linkage device

This application claims the priority of the Chinese Patent Application entitled "A UAV Monitoring Method and Audio and Video Linkage Device" submitted by the Chinese Patent Office on May 17, 2017, and the application number is 201710349350.2, the entire contents of which are incorporated by reference. In this application.

Technical field

The present application relates to the field of security monitoring technologies, and in particular, to a drone monitoring method and an audio and video linkage device.

Background technique

The drone is a short name for a class of powered, unmanned, reusable aircraft. It is small in size and low in cost, and can be equipped with telemetry equipment such as sensors and cameras. Since drones can take pictures of the ground from the air, drones are often used to monitor the ground.

With the development of drones and manufacturing costs, the drone industry is developing rapidly and the threshold is getting lower and lower. Therefore, the use of drones by individuals, groups, organizations, etc. is becoming more and more common. Small and flying at low altitudes, most of the radar monitoring systems do not have the ability to monitor drones, which leads to the prevalence of “black flying” of drones, affecting social and personal safety.

Summary of the invention

The purpose of the embodiment of the present application is to provide a UAV monitoring method and an audio and video linkage device to implement monitoring of the UAV, thereby ensuring social and personal security. The specific technical solutions are as follows:

In a first aspect, an embodiment of the present application provides a UAV monitoring method, where the method includes:

Acquiring sound signals through an audio collection array;

Performing signal processing on the sound signal to obtain a sound component in which at least one of the sound signals satisfies a preset condition;

Determining a location of the target object corresponding to the sound component that satisfies the preset condition;

Controlling the position of the camera at the target object;

Determining whether the target object is a drone according to an image captured by the camera.

Optionally, the signal feature includes: a spectrum feature;

Performing signal processing on the sound signal to obtain a sound component in which at least one of the sound signals meets a preset condition, including:

Extracting a sound signal collected by the audio collection array to obtain at least one sound component of the sound signal;

Acquiring, for each of the at least one sound component, the energy collected by each of the audio collection devices in the audio collection array; for the sound component whose energy is greater than a preset threshold, for each of the audio collection arrays The frequency domain signals of the sound component collected by the audio collection device are fused to obtain a spectrum feature;

A sound component in which the spectral features of all sound components satisfy the preset spectral features is obtained.

Optionally, the acquiring the energy collected by each audio collection device in the audio collection array for each of the at least one sound component comprises:

For each sound component, the sound component collected by each audio collection device in the audio collection array is subjected to frame processing in a preset period of the current time, and the sound component corresponding to each audio collection device is correspondingly obtained. Frame signal

Calculating a sum of energy of a frame signal corresponding to the sound component collected by any audio collection device, and determining that the sum of the energy is the energy of the sound component collected by the audio collection device;

The frequency domain signal of the sound component collected by each audio collection device in the audio collection array is fused to obtain a frequency characteristic, and the method includes:

Extracting a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold;

Performing frequency domain transformation on a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than the preset threshold according to a preset time-frequency transform method, to obtain an audio collection in which each energy is greater than the preset threshold. a frequency domain signal of a frame signal corresponding to the sound component collected by the device;

The frequency domain signals of the frame signals corresponding to the sound components collected by the audio collection devices whose energy is greater than the preset threshold are fused to obtain the spectral features of the sound components.

Optionally, before the extracting the frame signal corresponding to the sound component collected by the audio collection device that is greater than the preset threshold, the method further includes:

Determining whether the number of audio collection devices whose energy is greater than a preset threshold is greater than a first preset number;

If yes, performing the extracting the frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold.

Optionally, after the determining whether the number of the audio collection devices that is greater than the preset threshold is greater than the first preset number, the method further includes:

If not, it is determined that the sound component is noise.

Optionally, the determining, by the sound component that the signal feature meets the preset condition, the location of the target object, includes:

For any sound component, when the signal characteristic of the sound component satisfies a preset condition, determining that the object corresponding to the sound component is the target object;

Determining a location of the target object according to a position of each of the audio collection devices in the audio collection array relative to the camera and a position of the target object relative to each of the audio collection devices.

Optionally, after the determining that the object corresponding to the sound component is the target object, when the signal feature of the sound component meets the preset condition, the method further includes:

Selecting, from an audio collection device whose energy is greater than the preset threshold, a second preset number of audio collection devices;

And combining the second preset number of audio collection devices to form a pair of audio collection device pairs;

Determining, according to a preset time delay estimation method, a position of the target object with respect to each pair of audio collection device pairs;

Determining a location of the target object according to a position of each of the audio collection devices in the audio collection array relative to the camera and a position of the target object relative to each of the audio collection devices, including:

The location of the target object is determined according to the position of each audio collection device relative to the camera and the position of the target object relative to each pair of audio collection devices.

Optionally, determining the location of the target object according to a location of each audio collection device relative to the camera and a location of the target object relative to each pair of audio collection device, including:

Determining a position parameter of the target object relative to each pair of audio collection device pairs according to a position of the target object relative to each pair of audio collection device pairs, wherein the position parameter includes an angle value or a displacement value;

Calculating a difference value between each two position parameters, and selecting a position parameter corresponding to the difference value of the preset difference value;

Locating a position parameter corresponding to the difference value smaller than the preset difference, and determining a position of the target object relative to any audio collection device;

And determining a location of the target object according to a position of the audio collection device relative to the camera and a position of the target object relative to the audio collection device.

Optionally, after the determining that the signal feature meets the location of the target object corresponding to the sound component of the preset condition, the method further includes:

Generate an alarm command;

The method further includes: after determining, according to the image captured by the camera, whether the target object is a drone, the method further includes:

Eliminating the alarm command when the target object is not a drone;

An alarm enhancement command is generated when the target object is a drone.

Optionally, after the determining whether the target object is a drone according to the image captured by the camera, the method further includes:

When the target object is a drone, the camera is controlled to perform tracking shooting on the drone.

In a second aspect, an embodiment of the present application provides an audio and video linkage device, where the device includes:

An audio collection array for collecting sound signals;

a camera for shooting a target object;

a processor, configured to perform signal processing on the sound signal collected by the audio collection array, to obtain a sound component in which at least one of the sound signals meets a preset condition; and determine a sound component corresponding to the preset condition Positioning the target object; controlling the position of the camera to be aligned with the target object; determining whether the target object is a drone according to an image captured by the camera;

The housing assembly includes: a rotating member and a base, wherein the rotating member is configured to provide rotational kinetic energy to the camera, and the base is configured to fix a bottom of the camera, the audio collection array, and the processor.

Optionally, the feature signal includes: a spectrum feature;

The processor is specifically configured to:

Extracting a sound signal collected by the audio collection array to obtain at least one sound component of the sound signal;

Acquiring, for each of the at least one sound component, the energy collected by each of the audio collection devices in the audio collection array; for the sound component whose energy is greater than a preset threshold, for each of the audio collection arrays The frequency domain signals of the sound component collected by the audio collection device are fused to obtain a spectrum feature;

A sound component in which the spectral features of all sound components satisfy the preset spectral features is obtained.

Optionally, the processor is specifically configured to:

For each sound component, the sound component collected by each audio collection device in the audio collection array is subjected to frame processing in a preset period of the current time, and the sound component corresponding to each audio collection device is correspondingly obtained. Frame signal

Calculating a sum of energy of a frame signal corresponding to the sound component collected by any audio collection device, and determining that the sum of the energy is the energy of the sound component collected by the audio collection device;

Extracting a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold;

Performing frequency domain transformation on a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than the preset threshold according to a preset time-frequency transform method, to obtain an audio collection in which each energy is greater than the preset threshold. a frequency domain signal of a frame signal corresponding to the sound component collected by the device;

The frequency domain signals of the frame signals corresponding to the sound components collected by the audio collection devices whose energy is greater than the preset threshold are fused to obtain the spectral features of the sound components.

Optionally, the processor is specifically configured to:

Determining whether the number of audio collection devices whose energy is greater than a preset threshold is greater than a first preset number;

If yes, performing the extracting the frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold.

Optionally, the processor is specifically configured to:

If the number of audio collection devices whose energy is greater than a preset threshold is less than or equal to the first preset number, then the sound component is determined to be noise.

Optionally, the processor is specifically configured to:

For any sound component, when the signal characteristic of the sound component satisfies a preset condition, determining that the object corresponding to the sound component is the target object;

Determining a location of the target object according to a position of each of the audio collection devices in the audio collection array relative to the camera and a position of the target object relative to each of the audio collection devices.

Optionally, the processor is specifically configured to:

Selecting, from an audio collection device whose energy is greater than the preset threshold, a second preset number of audio collection devices;

And combining the second preset number of audio collection devices to form a pair of audio collection device pairs;

Determining, according to a preset time delay estimation method, a position of the target object with respect to each pair of audio collection device pairs;

The location of the target object is determined according to the position of each audio collection device relative to the camera and the position of the target object relative to each pair of audio collection devices.

Optionally, the processor is specifically configured to:

Determining a position parameter of the target object relative to each pair of audio collection device pairs according to a position of the target object relative to each pair of audio collection device pairs, wherein the position parameter includes an angle value or a displacement value;

Calculating a difference value between each two position parameters, and selecting a position parameter corresponding to the difference value of the preset difference value;

Locating a position parameter corresponding to the difference value smaller than the preset difference, and determining a position of the target object relative to any audio collection device;

And determining a location of the target object according to a position of the audio collection device relative to the camera and a position of the target object relative to the audio collection device.

Optionally, the device further includes:

The alarm component is used to perform an alarm operation when an alarm command is received.

Optionally, the processor is specifically configured to:

After determining that the signal feature meets the position of the target object corresponding to the sound component of the preset condition, generating an alarm instruction, and transmitting the alarm instruction to the alarm component to drive the alarm component to perform an alarm operation;

Determining the alarm command when determining that the target object is not a drone according to an image captured by the camera;

And determining, according to the image captured by the camera, that the target object is a drone, generating an alarm enhancement command, and transmitting the alarm enhancement command to the alarm component to drive the alarm component to perform an alarm enhancement operation.

Optionally, the processor is specifically configured to:

When the target object is a drone, the camera is controlled to perform tracking shooting on the drone.

In a third aspect, an embodiment of the present application provides a storage medium for storing executable code, where the executable code is used to execute at a runtime: the method steps provided by the first aspect of the embodiment of the present application.

In a fourth aspect, an embodiment of the present application provides an application program for performing, at runtime, the method steps provided by the first aspect of the embodiment of the present application.

The UAV monitoring method and the audio and video linkage device provided by the embodiment of the present invention process the sound signal collected by the audio collection array to determine the location of the target object corresponding to the sound component whose signal characteristic meets the preset condition, and According to the position of the target object, the camera is controlled to be aimed at the target object, and the target object is judged to be a drone according to the captured image, thereby realizing the monitoring of the drone and ensuring the accuracy of the monitoring.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application and the technical solutions of the prior art, the following description of the embodiments and the drawings used in the prior art will be briefly introduced. Obviously, the drawings in the following description are only Some embodiments of the application may also be used to obtain other figures from those of ordinary skill in the art without departing from the scope of the invention.

1 is a schematic flow chart of a method for monitoring a drone according to an embodiment of the present application;

2 is a schematic flow chart of a method for monitoring a drone according to another embodiment of the present application;

3 is a schematic flow chart of a method for monitoring a drone according to still another embodiment of the present application;

4 is a schematic structural diagram of an audio-video linkage device according to an embodiment of the present application;

FIG. 5 is a schematic structural diagram of an audio-video linkage device according to another embodiment of the present application.

detailed description

In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings. It is apparent that the described embodiments are only a part of the embodiments of the present application, and not all of them. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

In order to realize the monitoring of the drone, thereby ensuring social and personal safety, the embodiment of the present application provides a drone monitoring method and an audio and video linkage device.

The following describes an unmanned aerial vehicle monitoring method provided by the embodiment of the present application.

The execution body of the UAV monitoring method provided by the embodiment of the present application may be a controller equipped with a core processing chip, wherein the core processing chip may be a DSP (Digital Signal Processor), an ARM. Any of core processing chips such as (Advanced Reduced Instruction Set Computer Machines) or FPGA (Field-Programmable Gate Array). A manner of implementing a UAV monitoring method provided by an embodiment of the present application may be at least one of software, hardware circuits, and logic circuits disposed in an execution body.

As shown in FIG. 1 , a method for monitoring a drone provided by an embodiment of the present application may include the following steps:

S101. Acquire a sound signal through an audio collection array.

The audio collection array may be an array of multiple audio collection devices having audio collection functions, such as an array of multiple microphones or multiple sound sensors. In order to ensure that the drone can be monitored at a long distance from the edge of the area to be monitored, the array can be installed on the periphery of the area to be monitored; of course, the array can also be installed in the area to be monitored. An important location within the area so that only that important location is monitored. Generally, the number of audio collection devices in the audio collection array can be selected according to the size of the area to be monitored. Generally speaking, the larger the area to be monitored, the audio collection is required to obtain the same detection result. The more devices there are, the wider the distribution of audio capture devices. For example, in an ordinary electret microphone, an electret microphone can be installed every 10 cm to 20 cm, and the distance between each two electret microphones may not be exactly the same, which is not limited herein.

S102: Perform signal processing on the sound signal to obtain a sound component in which at least one of the sound signals satisfies a preset condition.

In the area that needs to be monitored, there may be multiple sound sources, that is, there may be multiple objects in the vicinity, then the sound signals collected by the audio collection array may contain sounds from multiple objects. As well as other noises (such as wind noise), it is necessary to separate the sound components generated by different objects in the sound signal when performing recognition. The sound signal collected by the audio collection array may be a time domain signal or a frequency domain signal, and the signal characteristics may be any one or more of characteristics such as audio, pitch, volume, and frequency spectrum, and the signal characteristics may be different. The sound components are separated. Moreover, in order to locate the drone more accurately, a preset condition is defined in the embodiment of the present application, and the preset condition is a condition that the signal feature needs to reach. Since most of the drones are driven by propellers, the sound signals generated by the propellers have distinct features, and all of the properties of the sound signal characteristics of the propellers that can represent the drone can be set to the preset conditions. For example, there are a plurality of spectral lines having harmonic relations in the noise, and the spectral lines are arranged at equal intervals on the frequency axis. Based on the feature, signals having spectral lines arranged at equal intervals on the frequency axis can be carried. Any attribute information is set as a preset condition. By analyzing the signal characteristics of all sound components, and obtaining the sound components whose signal characteristics meet the preset conditions, the noise signal similar to noise can be filtered out, the accuracy of positioning the drone by sound is improved, and the identification drone is improved. s efficiency.

Optionally, the step of performing signal processing on the sound signal to obtain a sound component that satisfies a predetermined condition in at least one of the sound signals may include:

In the first step, the sound signal collected by the audio collection array is extracted to obtain at least one sound component in the sound signal.

Since the sound signal collected by the audio collection array may be a sound emitted by a plurality of objects, at least one sound component may be obtained by extracting the sound signal, for example, dividing the sound signal according to the spectrum of the sound, in addition to the manner, Other ways of extracting the sound signal to obtain at least one sound component are also within the protection scope of the embodiment, and are not described herein again.

In the second step, for each of the at least one sound component, the energy collected by each audio collection device in the audio collection array is acquired; for the sound component whose energy is greater than the preset threshold, each audio collection device in the audio collection array The frequency domain signals of the collected sound components are fused to obtain spectral features.

For an object that is detected, the closer the energy collection device of the object is, the greater the energy of the sound component of the object, and the audio collecting device that collects the energy component of the object is greater than a preset threshold. The greater the number, the closer the object is to the area to be monitored or the area to be monitored. Therefore, in this embodiment, the energy of the sound component collected by each audio collection device is extracted, and an audio collection device whose energy is greater than a preset threshold is determined, wherein the preset threshold and the area to be monitored are not included. The volume of the man-machine is related to other factors, and can be initialized according to the environment when the audio acquisition array is installed for the first time. Specifically, in order to further improve the limitation of the monitoring range, to improve the accuracy of the monitoring, the number of audio collection devices whose energy is greater than a preset threshold may be limited, for example, the number of audio collection devices whose energy is greater than a preset threshold. The frequency domain is converted to half of the total number of all audio collection devices in the audio collection array. Otherwise, the sound component is considered as noise and is not processed. The spectral characteristic is a characteristic of the sound emitted by the propeller of the drone when it is rotating, that is, a plurality of spectral lines having harmonic relations in the noise, and the spectral lines are arranged at equal intervals on the frequency axis, and therefore, the sound component is Frequency domain transform is performed to obtain a frequency domain signal, and then the spectral characteristics of the sound component are obtained by fusion.

Optionally, the step of acquiring the energy collected by each audio collection device in the audio collection array for each of the at least one sound component may include:

For any sound component, the sound component collected by each audio collection device in the audio collection array is subjected to frame processing in a preset period of the current time to obtain a frame corresponding to the sound component collected by each audio collection device. signal;

Calculating the sum of the energy of the frame signal corresponding to the sound component collected by any of the audio collection devices, and determining the sum of the energy of the sound component collected by the audio collection device.

Since the sound component collected by each audio collection device in the audio collection array is audio for a period of time, in order to ensure real-time processing of the signal, the sound components collected by each audio collection device may be divided in a preset period of the current time. Frame processing, after performing framing processing, it is necessary to perform energy calculation on the frame signal, and for each audio acquisition device, calculate the sum of energy calculated for all frame signals.

Optionally, the step of merging the frequency domain signals of the sound component collected by each of the audio collection devices in the audio collection array for the sound component whose energy is greater than the preset threshold to obtain the spectral features may include:

Extracting a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold;

Performing frequency domain transformation on the frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than the preset threshold according to the preset time-frequency transform method, and obtaining the audio acquisition device that is collected by the audio collection device whose energy is greater than a preset threshold. a frequency domain signal of a frame signal corresponding to the sound component;

The frequency domain signals of the frame signals corresponding to the sound components collected by the audio collection devices whose energy is greater than the preset threshold are fused to obtain the spectral features of the sound components.

After performing frame processing on the sound component collected by each audio collection device, the frame signal obtained by the frame processing may be subjected to frequency domain transformation according to a preset time-frequency transform method, wherein the preset time-frequency transform method may be The leaf transform, of course, other methods for transforming the time domain signal into the frequency domain signal are also within the scope of protection of the embodiments of the present application, and are not described herein again. After the frequency domain transform of the frame signal, the multi-order harmonic signal processing result of the obtained frequency domain signal can be fused to obtain a spectrum feature. The fusion of the harmonic signals may be a fusion of the cyclostationary features of the multi-order harmonics with the multiplication relationship. Of course, other ways of fusing the harmonic signals are also within the protection scope of the embodiment of the present application. Let me repeat.

Optionally, before the step of extracting the frame signal corresponding to the sound component collected by the audio collection device with each energy greater than a preset threshold, the method may further include the following steps:

Determining whether the number of audio collection devices whose energy is greater than a preset threshold is greater than a first preset number;

If it is greater, performing a step of extracting a frame signal corresponding to the sound component collected by each audio collection device whose energy is greater than a preset threshold;

If not greater, it is determined that the sound component is noise.

In order to further improve the limitation of the monitoring range, to improve the accuracy of the monitoring, the number of the audio collecting devices whose energy is greater than the preset threshold may be limited, for example, a first preset number is set, and the energy is greater than the preset threshold. The step of extracting the frame signal is performed only when the number of the audio collection devices is greater than the first preset number, otherwise the sound component is considered to be noise and is not processed. The first preset number may be set to be half of the total number of audio collection devices, or may be set to one quarter of the total number of audio collection devices. The smaller the first preset number is set, the larger the monitoring range is, but the accuracy is May be reduced. Normally, the first preset number is set to half of the total number of audio collection devices.

In the third step, the sound components of all the sound components whose spectral features satisfy the preset spectral features are obtained.

Based on the sound signal of the drone, there are multiple spectral lines with harmonic relationship in the noise. These spectral lines are arranged at equal intervals on the frequency axis. It is possible to establish multi-frequency cyclic stationary detection and analyze the multi-frequency cyclic stationary detection. Whether the spectrum is the propeller sound of the unmanned aerial vehicle, wherein the multi-frequency cyclic stationary detection utilizes the characteristics that the harmonics of the signals are coherently superimposed, and the noise is non-coherently superimposed, which has the ability to extract spectral features and reduce random noise. The advantage of the impact. Certainly, other ways of analyzing the spectrum features are also included in the protection scope of the embodiments of the present application, and are not described herein again.

S103. Determine a location of the target object corresponding to the sound component whose signal characteristic meets the preset condition.

The target object corresponding to the sound component satisfying the preset condition can be determined by analyzing the signal feature, and the position of the target object compared to the audio collecting device can be determined by the attribute information such as volume and audio of the sound component of the target object.

Optionally, the step of determining that the signal feature meets the location of the target object corresponding to the sound component of the preset condition may include:

For any sound component, when the signal characteristic of the sound component satisfies a preset condition, determining that the object corresponding to the sound component is the target object;

Then, the location of the target object is determined according to the position of each of the audio collection devices in the audio collection array relative to the camera and the position of the target object relative to each of the audio collection devices.

When the position of each audio collection device in the audio collection array relative to the camera is arranged in the audio collection array, it can be determined according to the position information provided by the audio collection device, that is, when determining the arrangement of the audio collection array. The position of each audio capture device in the audio capture array relative to the camera has been determined. The position mentioned here can be either an angle or a distance.

Optionally, after the step of determining that the object corresponding to the sound component is the target object when the signal feature of the sound component meets the preset condition, the method further includes the following steps:

The first step is to select an audio collection device that meets a second preset number from the audio collection devices whose energy is greater than a preset threshold;

In the second step, the second preset number of audio collection devices are combined to form a pair of audio collection device pairs;

In the third step, the position of the target object relative to each pair of audio collection devices is determined according to a preset time delay estimation method.

After determining the target object, in order to improve the computational efficiency of calculating the position of the target object, the audio collection device with the energy greater than the preset threshold may select a plurality of audio collection devices in descending order of energy values, in this embodiment. The second preset number is determined by the energy value from large to small, and can be set as five audio collection devices, and the selected audio collection devices are combined to form an audio collection device pair. The preset time delay estimation method can calculate a corresponding azimuth of each pair of audio collection devices, and can convert the azimuth angle into an angle at which the camera rotates. The preset time delay estimation method may be a classical time delay estimation method, and the estimated sound signal reaches a time difference between different audio collection devices, and the position of the sound source may be determined through a geometric relationship. The classical time delay estimation method includes The method of the cross-correlation function, the method using the speech feature, and the method based on the channel transfer function are all applicable to the embodiments of the present application.

Optionally, the step of determining the location of the target object according to the position of each of the audio collection devices in the audio collection array relative to the camera and the position of the target object relative to each of the audio collection devices may include:

The location of the target object is determined according to the position of each audio collection device relative to the camera and the position of the target object relative to each pair of audio collection devices.

Due to the influence of noise, the location of the target object determined above often has a certain deviation. If the target position is to be accurately calculated, the estimated orientation of each pair of audio collection devices needs to be processed. Optionally, the step of determining the location of the target object according to the position of each audio collection device relative to the camera and the position of the target object relative to each pair of the audio collection device may include:

In the first step, determining a position parameter of the target object relative to each pair of the audio collection device according to a position of the target object relative to each pair of the audio collection device, wherein the position parameter includes an angle value or a displacement value;

In the second step, calculating a difference value between each two position parameters, and selecting a position parameter corresponding to the difference value of the preset difference value;

The third step is to average the position parameters corresponding to the difference value of the preset difference, and determine the position of the target object relative to any of the audio collection devices;

In the fourth step, the location of the target object is determined according to the position of the audio collection device relative to the camera and the position of the target object relative to the audio collection device.

After determining the position of the target object relative to each pair of audio collection devices, there is a certain difference between the angle value or the displacement value of the target object relative to each pair of audio collection device pairs, and the selection difference is within the preset difference value. The pair of audio collection devices, for example, calculates an angle of the target object relative to each pair of audio collection devices, calculates an audio collection device pair whose angle difference value is within 10 degrees, and determines the target object by using an average value. Accurate position can further improve the accuracy of determining the target position.

S104: Control the position of the camera at the target object.

After determining the location of the target object, in order to further determine whether the target object is a drone for the determined target object, the control camera is aimed at the target object, and the camera can be driven to rotate by driving the rotating device. Here, the alignment operation of the camera is not specifically limited, and may be aligned by the camera, or may be aligned by the camera, or may be translated by the camera.

S105. Determine, according to the image captured by the camera, whether the target object is a drone.

The camera can be always in the shooting state, and only the driving camera is aimed at the target object; or after the camera is aimed at the target object, a shooting instruction is received to start shooting the target object. By identifying the image of the captured target object, it is determined whether the target object is an unmanned aerial vehicle. Specifically, the image recognition technology may be an automatic recognition technology such as a neural network or a pixel comparison in the field of image processing, or may be artificially Compare, there is no specific limit here.

Applying the embodiment, the sound signal collected by the audio collection array is processed to determine the position of the target object corresponding to the sound component whose signal characteristic meets the preset condition, and the camera is controlled to be aligned with the target object according to the position of the target object. Shooting, according to the captured image to determine whether the target object is a drone, thus achieving monitoring of the drone, and using the double confirmation of audio and video to prevent false positives, to ensure the accuracy of monitoring.

Based on the embodiment shown in FIG. 1, as shown in FIG. 2, the embodiment of the present application provides another UAV monitoring method. After S104, the following steps may be further included:

S201, generating an alarm instruction.

Since the drone enters the area to be monitored, it may bring safety hazards to the users in the area. Therefore, when it is judged by the audio that the target object approaches or enters the area to be monitored, an alarm command is generated to remind the user that there is a suspected target. Need to be alert.

In the judgment of S105, if the target object is a drone, S202 is performed, otherwise, S203 is performed.

S202, generating an alarm enhancement instruction.

If it is further confirmed by video that the drone is approaching or entering the area to be monitored, an alarm enhancement command is generated to enhance the effect of the alarm, for example, increasing the sound of the alarm to remind the user that there is a drone in the area.

S203, eliminating the alarm instruction.

If there is no drone in the area that needs to be monitored by video, the alarm can be eliminated.

Applying the embodiment, the sound signal collected by the audio collection array is processed to determine the position of the target object corresponding to the sound component whose signal characteristic meets the preset condition, and the camera is controlled to be aligned with the target object according to the position of the target object. Shooting, according to the captured image to determine whether the target object is a drone, thus achieving monitoring of the drone, and using the double confirmation of audio and video to prevent false positives, to ensure the accuracy of monitoring. And by alerting the presence of the target object, the alarm is enhanced when there is a drone, and the user is promptly reminded to further improve the safety.

Based on the embodiment shown in FIG. 1, as shown in FIG. 3, the embodiment of the present application provides another UAV monitoring method. After S105, the following steps may be further included:

S301: When the target object is a drone, the camera is controlled to perform tracking shooting on the drone.

When the drone is detected to be close to or enters the area to be monitored, since the drone has been in flight, in order to better track the drone and monitor the flight behavior of the drone in real time, the camera can be controlled to the drone. Tracking and shooting is performed. Specifically, according to the motion track of the drone in the captured video, the camera can be driven to move according to the motion track to achieve tracking shooting.

Applying the embodiment, the sound signal collected by the audio collection array is processed to determine the position of the target object corresponding to the sound component whose signal characteristic meets the preset condition, and the camera is controlled to be aligned with the target object according to the position of the target object. Shooting, according to the captured image to determine whether the target object is a drone, thus achieving monitoring of the drone, and using the double confirmation of audio and video to prevent false positives, to ensure the accuracy of monitoring. And by controlling the camera to track and shoot the drone, real-time monitoring of the flight status of the drone, to ensure real-time monitoring and security.

Corresponding to the above embodiment, the embodiment of the present application provides an audio and video linkage device. As shown in FIG. 4, the audio and video linkage device may include:

An audio collection array 410, configured to collect a sound signal;

a camera 420 for photographing a target object;

The processor 430 is configured to perform signal processing on the sound signal collected by the audio collection array 410, obtain a sound component in which at least one of the sound signals meets a preset condition, and determine a sound that the signal feature meets a preset condition. Positioning the target object corresponding to the component; controlling the camera 420 to align the location of the target object; determining, according to the image captured by the camera 420, whether the target object is a drone;

The housing assembly 440 includes a rotating member and a base, wherein the rotating member is configured to provide rotational kinetic energy to the camera, and the base is configured to fix a bottom of the camera, the audio collection array, and the processor.

Applying the embodiment, the sound signal collected by the audio collection array is processed to determine the position of the target object corresponding to the sound component whose signal characteristic meets the preset condition, and the camera is controlled to be aligned with the target object according to the position of the target object. Shooting, according to the captured image to determine whether the target object is a drone, thus achieving monitoring of the drone, and using the double confirmation of audio and video to prevent false positives, to ensure the accuracy of monitoring.

Optionally, the feature signal may include: a spectrum feature;

The processor 430 is specifically configured to:

Extracting a sound signal collected by the audio collection array 410 to obtain at least one sound component of the sound signal;

Acquiring, for each of the at least one sound component, the energy collected by each of the audio collection devices in the audio collection array; for the sound component whose energy is greater than a preset threshold, for each of the audio collection arrays The frequency domain signals of the sound component collected by the audio collection device are fused to obtain a spectrum feature;

A sound component in which the spectral features of all sound components satisfy the preset spectral features is obtained.

Optionally, the processor 430 is specifically configured to:

For each sound component, the sound component collected by each audio collection device in the audio collection array 410 is subjected to frame processing in a preset period of the current time to obtain the sound component collected by each audio collection device. Corresponding frame signal;

Calculating a sum of energy of a frame signal corresponding to the sound component collected by any audio collection device, and determining that the sum of the energy is the energy of the sound component collected by the audio collection device;

Extracting a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold;

Performing frequency domain transformation on a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than the preset threshold according to a preset time-frequency transform method, to obtain an audio collection in which each energy is greater than the preset threshold. a frequency domain signal of a frame signal corresponding to the sound component collected by the device;

The frequency domain signals of the frame signals corresponding to the sound components collected by the audio collection devices whose energy is greater than the preset threshold are fused to obtain the spectral features of the sound components.

Optionally, the processor 430 is specifically configured to:

Determining whether the number of audio collection devices whose energy is greater than a preset threshold is greater than a first preset number;

If yes, performing the extracting the frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold.

Optionally, the processor 430 is specifically configured to:

If the number of audio collection devices whose energy is greater than a preset threshold is less than or equal to the first preset number, it is determined that the sound component is noise.

Optionally, the processor 430 is specifically configured to:

For any sound component, when the signal characteristic of the sound component satisfies a preset condition, determining that the object corresponding to the sound component is the target object;

The location of the target object is determined according to the position of each of the audio collection devices in the audio collection array 410 relative to the camera 420 and the position of the target object relative to each of the audio collection devices.

Optionally, the processor 430 is specifically configured to:

Selecting, from an audio collection device whose energy is greater than the preset threshold, a second preset number of audio collection devices;

And combining the second preset number of audio collection devices to form a pair of audio collection device pairs;

Determining, according to a preset time delay estimation method, a position of the target object with respect to each pair of audio collection device pairs;

The location of the target object is determined according to the position of each audio collection device relative to the camera 420 and the position of the target object relative to each pair of audio collection devices.

Optionally, the processor 430 is specifically configured to:

Determining a position parameter of the target object relative to each pair of audio collection device pairs according to a position of the target object relative to each pair of audio collection device pairs, wherein the position parameter includes an angle value or a displacement value;

Calculating a difference value between each two position parameters, and selecting a position parameter corresponding to the difference value of the preset difference value;

Locating a position parameter corresponding to the difference value smaller than the preset difference, and determining a position of the target object relative to any audio collection device;

The location of the target object is determined according to the position of the audio capture device relative to the camera 420 and the position of the target object relative to the audio capture device.

The audio-video linkage device of the embodiment of the present application is a device applying the above-described UAV monitoring method, and all the embodiments of the UAV monitoring method are applicable to the device, and all of the same or similar beneficial effects can be achieved.

Further, based on the audio capture array 410, the camera 420, the processor 430, and the outer casing assembly 440, as shown in FIG. 5, the audio and video linkage device provided by the embodiment of the present application may further include:

The alarm component 510 is configured to perform an alarm operation when receiving an alarm instruction.

The alarm component 510 can be a device for alarms such as a buzzer, a light emitting diode, a display, etc., and the alarm component 510 has a function of enhancing an alarm, for example, the buzzer of the buzzer is getting louder and the light of the light emitting diode is coming. The brighter, the display content of the display changes, and so on. The alarm operation may be a beeping sound, lighting a light emitting diode, displaying a suspected target, and the like.

Applying the embodiment, the sound signal collected by the audio collection array is processed to determine the position of the target object corresponding to the sound component whose signal characteristic meets the preset condition, and the camera is controlled to be aligned with the target object according to the position of the target object. Shooting, according to the captured image to determine whether the target object is a drone, thus achieving monitoring of the drone, and using the double confirmation of audio and video to prevent false positives, to ensure the accuracy of monitoring. And by alerting the presence of the target object, the alarm is enhanced when there is a drone, and the user is promptly reminded to further improve the safety.

Optionally, the processor 430 is specifically configured to:

After determining that the signal feature meets the position of the target object corresponding to the preset sound component, generating an alarm instruction, and transmitting the alarm command to the alarm component 510 to drive the alarm component 510 to perform an alarm operation;

Determining the alarm instruction when determining that the target object is not a drone according to an image captured by the camera 420;

And determining, according to the image captured by the camera 420, that the target object is a drone, generating an alarm enhancement command, and transmitting the alarm enhancement command to the alarm component 510 to drive the alarm component 510 to perform an alarm. Enhance operations.

Optionally, the processor 430 is specifically configured to:

When the target object is a drone, the camera 420 is controlled to perform tracking shooting on the drone.

In addition, corresponding to the UAV monitoring method provided by the foregoing embodiment, the embodiment of the present application provides a storage medium for storing executable code, which is used to execute at runtime: the embodiment of the present application The UAV monitoring method provided; specifically, the UAV monitoring method includes:

Acquiring sound signals through an audio collection array;

Performing signal processing on the sound signal to obtain a sound component in which at least one of the sound signals satisfies a preset condition;

Determining a location of the target object corresponding to the sound component that satisfies the preset condition;

Controlling the position of the camera at the target object;

Determining whether the target object is a drone according to an image captured by the camera.

Optionally, the signal feature includes: a spectrum feature;

Performing signal processing on the sound signal to obtain a sound component in which at least one of the sound signals meets a preset condition, including:

Extracting a sound signal collected by the audio collection array to obtain at least one sound component of the sound signal;

Acquiring, for each of the at least one sound component, the energy collected by each of the audio collection devices in the audio collection array; for the sound component whose energy is greater than a preset threshold, for each of the audio collection arrays The frequency domain signals of the sound component collected by the audio collection device are fused to obtain a spectrum feature;

A sound component in which the spectral features of all sound components satisfy the preset spectral features is obtained.

Optionally, the acquiring the energy collected by each audio collection device in the audio collection array for each of the at least one sound component comprises:

For each sound component, the sound component collected by each audio collection device in the audio collection array is subjected to frame processing in a preset period of the current time, and the sound component corresponding to each audio collection device is correspondingly obtained. Frame signal

Calculating a sum of energy of a frame signal corresponding to the sound component collected by any audio collection device, and determining that the sum of the energy is the energy of the sound component collected by the audio collection device;

And the frequency domain signal of the sound component collected by each audio collection device in the audio collection array is fused to obtain a spectrum feature, where the sound component is greater than a preset threshold, and the spectrum features are obtained, including:

Extracting a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold;

Performing frequency domain transformation on a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than the preset threshold according to a preset time-frequency transform method, to obtain an audio collection in which each energy is greater than the preset threshold. a frequency domain signal of a frame signal corresponding to the sound component collected by the device;

The frequency domain signals of the frame signals corresponding to the sound components collected by the audio collection devices whose energy is greater than the preset threshold are fused to obtain the spectral features of the sound components.

Optionally, before the extracting the frame signal corresponding to the sound component collected by the audio collection device that is greater than the preset threshold, the method further includes:

Determining whether the number of audio collection devices whose energy is greater than a preset threshold is greater than a first preset number;

If yes, performing the extracting the frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold.

Optionally, after the determining whether the number of the audio collection devices that is greater than the preset threshold is greater than the first preset number, the method further includes:

If not, it is determined that the sound component is noise.

Optionally, the determining, by the sound component that the signal feature meets the preset condition, the location of the target object, includes:

For any sound component, when the signal characteristic of the sound component satisfies a preset condition, determining that the object corresponding to the sound component is the target object;

Determining a location of the target object according to a position of each of the audio collection devices in the audio collection array relative to the camera and a position of the target object relative to each of the audio collection devices.

Optionally, after the determining that the object corresponding to the sound component is the target object, when the signal feature of the sound component meets the preset condition, the method further includes:

Selecting, from an audio collection device whose energy is greater than the preset threshold, a second preset number of audio collection devices;

And combining the second preset number of audio collection devices to form a pair of audio collection device pairs;

Determining, according to a preset time delay estimation method, a position of the target object with respect to each pair of audio collection device pairs;

Determining a location of the target object according to a position of each of the audio collection devices in the audio collection array relative to the camera and a position of the target object relative to each of the audio collection devices, including:

The location of the target object is determined according to the position of each audio collection device relative to the camera and the position of the target object relative to each pair of audio collection devices.

Optionally, determining the location of the target object according to a location of each audio collection device relative to the camera and a location of the target object relative to each pair of audio collection device, including:

Determining a position parameter of the target object relative to each pair of audio collection device pairs according to a position of the target object relative to each pair of audio collection device pairs, wherein the position parameter includes an angle value or a displacement value;

Calculating a difference value between each two position parameters, and selecting a position parameter corresponding to the difference value of the preset difference value;

Locating a position parameter corresponding to the difference value smaller than the preset difference, and determining a position of the target object relative to any audio collection device;

And determining a location of the target object according to a position of the audio collection device relative to the camera and a position of the target object relative to the audio collection device.

Optionally, after the determining that the signal feature meets the location of the target object corresponding to the sound component of the preset condition, the method further includes:

Generate an alarm command;

The method further includes: after determining, according to the image captured by the camera, whether the target object is a drone, the method further includes:

Eliminating the alarm command when the target object is not a drone;

An alarm enhancement command is generated when the target object is a drone.

Optionally, after the determining whether the target object is a drone according to the image captured by the camera, the method further includes:

When the target object is a drone, the camera is controlled to perform tracking shooting on the drone.

In this embodiment, the storage medium stores an application program for performing the UAV monitoring method provided by the embodiment of the present application during operation, so that the sound signal collected by the audio collection array can be processed to determine that the signal characteristic satisfies the pre-processing. Setting the position of the target object corresponding to the conditional sound component, and controlling the camera to aim at the target object according to the position of the target object, and determining whether the target object is a drone according to the captured image, thereby realizing the drone Monitoring and ensuring the accuracy of the monitoring.

In addition, corresponding to the UAV monitoring method provided by the foregoing embodiment, the embodiment of the present application provides an application program for performing the following steps of the UAV monitoring method provided by the embodiment of the present application.

In this embodiment, the application performs the UAV monitoring method provided by the embodiment of the present application during operation, so that the sound component collected by the audio collection array can be processed to determine the sound component whose signal characteristic meets the preset condition. Corresponding position of the target object, according to the position of the target object, controlling the camera to aim at the target object, and determining whether the target object is a drone according to the captured image, thereby realizing monitoring of the drone and ensuring The accuracy of the monitoring.

For the storage medium and the application embodiment, since the method content involved is basically similar to the foregoing method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment.

It should be noted that, in this context, relational terms such as first and second are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply such entities or operations. There is any such actual relationship or order between them. Furthermore, the term "comprises" or "comprises" or "comprises" or any other variations thereof is intended to encompass a non-exclusive inclusion, such that a process, method, article, or device that comprises a plurality of elements includes not only those elements but also Other elements, or elements that are inherent to such a process, method, item, or device. An element that is defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in the process, method, item, or device that comprises the element.

The various embodiments in the present specification are described in a related manner, and the same or similar parts between the respective embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for audio and video linkages, storage media, and application embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and the relevant portions can be referred to the description of the method embodiments.

The above is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., which are made within the spirit and principles of the present application, should be included in the present application. Within the scope of protection.

Claims (23)

1. A method for monitoring a drone, characterized in that the method comprises:

   Acquiring sound signals through an audio collection array;

   Performing signal processing on the sound signal to obtain a sound component in which at least one of the sound signals satisfies a preset condition;

   Determining a location of the target object corresponding to the sound component that satisfies the preset condition;

   Controlling the position of the camera at the target object;

   Determining whether the target object is a drone according to an image captured by the camera.
2. The method of claim 1 wherein said signal characteristics comprise: spectral features;

   Performing signal processing on the sound signal to obtain a sound component in which at least one of the sound signals meets a preset condition, including:

   Extracting a sound signal collected by the audio collection array to obtain at least one sound component of the sound signal;

   Acquiring, for each of the at least one sound component, the energy collected by each of the audio collection devices in the audio collection array; for the sound component whose energy is greater than a preset threshold, for each of the audio collection arrays The frequency domain signals of the sound component collected by the audio collection device are fused to obtain a spectrum feature;

   A sound component in which the spectral features of all sound components satisfy the preset spectral features is obtained.
3. The method according to claim 2, wherein the acquiring the energy collected by each of the audio collection devices in the audio collection array for each of the at least one sound component comprises:

   For each sound component, the sound component collected by each audio collection device in the audio collection array is subjected to frame processing in a preset period of the current time, and the sound component corresponding to each audio collection device is correspondingly obtained. Frame signal

   Calculating a sum of energy of a frame signal corresponding to the sound component collected by any audio collection device, and determining that the sum of the energy is the energy of the sound component collected by the audio collection device;

   The frequency domain signal of the sound component collected by each audio collection device in the audio collection array is fused to obtain a frequency characteristic, and the method includes:

   Extracting a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold;

   Performing frequency domain transformation on a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than the preset threshold according to a preset time-frequency transform method, to obtain an audio collection in which each energy is greater than the preset threshold. a frequency domain signal of a frame signal corresponding to the sound component collected by the device;

   The frequency domain signals of the frame signals corresponding to the sound components collected by the audio collection devices whose energy is greater than the preset threshold are fused to obtain the spectral features of the sound components.
4. The method according to claim 3, wherein the method further comprises: before extracting a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold, the method further comprises:

   Determining whether the number of audio collection devices whose energy is greater than a preset threshold is greater than a first preset number;

   If yes, performing the extracting the frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold.
5. The method according to claim 4, wherein after the determining whether the number of the audio collection devices is greater than the preset threshold, the method further comprises:

   If not, it is determined that the sound component is noise.
6. The method according to claim 1, wherein the determining the location of the target object corresponding to the sound component that the signal feature meets the preset condition comprises:

   For any sound component, when the signal characteristic of the sound component satisfies a preset condition, determining that the object corresponding to the sound component is the target object;

   Determining a location of the target object according to a position of each of the audio collection devices in the audio collection array relative to the camera and a position of the target object relative to each of the audio collection devices.
7. The method according to claim 6, wherein after the signal characteristic of the sound component satisfies a preset condition, after determining that the object corresponding to the sound component is the target object, the method further includes:

   Selecting, from an audio collection device whose energy is greater than the preset threshold, a second preset number of audio collection devices;

   And combining the second preset number of audio collection devices to form a pair of audio collection device pairs;

   Determining, according to a preset time delay estimation method, a position of the target object with respect to each pair of audio collection device pairs;

   Determining a location of the target object according to a position of each of the audio collection devices in the audio collection array relative to the camera and a position of the target object relative to each of the audio collection devices, including:

   The location of the target object is determined according to the position of each audio collection device relative to the camera and the position of the target object relative to each pair of audio collection devices.
8. The method according to claim 7, wherein said determining said target object according to a position of each audio collecting device relative to said camera and a position of said target object relative to each pair of audio collecting device pairs Location, including:

   Determining a position parameter of the target object relative to each pair of audio collection device pairs according to a position of the target object relative to each pair of audio collection device pairs, wherein the position parameter includes an angle value or a displacement value;

   Calculating a difference value between each two position parameters, and selecting a position parameter corresponding to the difference value of the preset difference value;

   Locating a position parameter corresponding to the difference value smaller than the preset difference, and determining a position of the target object relative to any audio collection device;

   And determining a location of the target object according to a position of the audio collection device relative to the camera and a position of the target object relative to the audio collection device.
9. The method according to any one of claims 1 to 8, wherein after the determining that the signal feature satisfies the location of the target object corresponding to the sound component of the preset condition, the method further includes:

   Generate an alarm command;

   The method further includes: after determining, according to the image captured by the camera, whether the target object is a drone, the method further includes:

   Eliminating the alarm command when the target object is not a drone;

   An alarm enhancement command is generated when the target object is a drone.
10. The method according to claim 9, wherein the method further comprises: after determining whether the target object is a drone according to an image captured by the camera, the method further comprising:

    When the target object is a drone, the camera is controlled to perform tracking shooting on the drone.
11. An audio and video linkage device, the device comprising:

    An audio collection array for collecting sound signals;

    a camera for shooting a target object;

    a processor, configured to perform signal processing on the sound signal collected by the audio collection array, to obtain a sound component in which at least one of the sound signals meets a preset condition; and determine a sound component corresponding to the preset condition Positioning the target object; controlling the position of the camera to be aligned with the target object; determining whether the target object is a drone according to an image captured by the camera;

    The housing assembly includes: a rotating member and a base, wherein the rotating member is configured to provide rotational kinetic energy to the camera, and the base is configured to fix a bottom of the camera, the audio collection array, and the processor.
12. The apparatus according to claim 11, wherein said characteristic signal comprises: a spectral characteristic;

    The processor is specifically configured to:

    Extracting a sound signal collected by the audio collection array to obtain at least one sound component of the sound signal;

    Acquiring, for each of the at least one sound component, the energy collected by each of the audio collection devices in the audio collection array; for the sound component whose energy is greater than a preset threshold, for each of the audio collection arrays The frequency domain signals of the sound component collected by the audio collection device are fused to obtain a spectrum feature;

    A sound component in which the spectral features of all sound components satisfy the preset spectral features is obtained.
13. The device according to claim 12, wherein the processor is further configured to:

    For each sound component, the sound component collected by each audio collection device in the audio collection array is subjected to frame processing in a preset period of the current time, and the sound component corresponding to each audio collection device is correspondingly obtained. Frame signal

    Calculating a sum of energy of a frame signal corresponding to the sound component collected by any audio collection device, and determining that the sum of the energy is the energy of the sound component collected by the audio collection device;

    Extracting a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold;

    Performing frequency domain transformation on a frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than the preset threshold according to a preset time-frequency transform method, to obtain an audio collection in which each energy is greater than the preset threshold. a frequency domain signal of a frame signal corresponding to the sound component collected by the device;

    The frequency domain signals of the frame signals corresponding to the sound components collected by the audio collection devices whose energy is greater than the preset threshold are fused to obtain the spectral features of the sound components.
14. The device according to claim 13, wherein the processor is further configured to:

    Determining whether the number of audio collection devices whose energy is greater than a preset threshold is greater than a first preset number;

    If yes, performing the extracting the frame signal corresponding to the sound component collected by the audio collection device whose energy is greater than a preset threshold.
15. The device according to claim 14, wherein the processor is further configured to:

    If the number of audio collection devices whose energy is greater than a preset threshold is less than or equal to the first preset number, it is determined that the sound component is noise.
16. The device according to claim 11, wherein the processor is further configured to:

    For any sound component, when the signal characteristic of the sound component satisfies a preset condition, determining that the object corresponding to the sound component is the target object;

    Determining a location of the target object according to a position of each of the audio collection devices in the audio collection array relative to the camera and a position of the target object relative to each of the audio collection devices.
17. The device according to claim 16, wherein the processor is further configured to:

    Selecting, from an audio collection device whose energy is greater than the preset threshold, a second preset number of audio collection devices;

    And combining the second preset number of audio collection devices to form a pair of audio collection device pairs;

    Determining, according to a preset time delay estimation method, a position of the target object with respect to each pair of audio collection device pairs;

    The location of the target object is determined according to the position of each audio collection device relative to the camera and the position of the target object relative to each pair of audio collection devices.
18. The device according to claim 17, wherein the processor is further configured to:

    Determining a position parameter of the target object relative to each pair of audio collection device pairs according to a position of the target object relative to each pair of audio collection device pairs, wherein the position parameter includes an angle value or a displacement value;

    Calculating a difference value between each two position parameters, and selecting a position parameter corresponding to the difference value of the preset difference value;

    Locating a position parameter corresponding to the difference value smaller than the preset difference, and determining a position of the target object relative to any audio collection device;

    And determining a location of the target object according to a position of the audio collection device relative to the camera and a position of the target object relative to the audio collection device.
19. The device according to claim 11, wherein the device further comprises:

    The alarm component is used to perform an alarm operation when an alarm command is received.
20. The device according to claim 19, wherein the processor is further configured to:

    After determining that the signal feature meets the position of the target object corresponding to the sound component of the preset condition, generating an alarm instruction, and transmitting the alarm instruction to the alarm component to drive the alarm component to perform an alarm operation;

    Determining the alarm command when determining that the target object is not a drone according to an image captured by the camera;

    And determining, according to the image captured by the camera, that the target object is a drone, generating an alarm enhancement command, and transmitting the alarm enhancement command to the alarm component to drive the alarm component to perform an alarm enhancement operation.
21. The device according to claim 20, wherein the processor is further configured to:

    When the target object is a drone, the camera is controlled to perform tracking shooting on the drone.
22. A storage medium for storing executable code for performing at runtime: the method steps of any of claims 1-10.
23. An application, characterized in that it is executed at runtime: the method steps of any of claims 1-10.

Images