CN110347187B - Target detection tracking system and method based on sound and image information - Google Patents
Target detection tracking system and method based on sound and image information Download PDFInfo
- Publication number
- CN110347187B CN110347187B CN201910735059.8A CN201910735059A CN110347187B CN 110347187 B CN110347187 B CN 110347187B CN 201910735059 A CN201910735059 A CN 201910735059A CN 110347187 B CN110347187 B CN 110347187B
- Authority
- CN
- China
- Prior art keywords
- target
- audio
- tracking
- module
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01M—CATCHING, TRAPPING OR SCARING OF ANIMALS; APPARATUS FOR THE DESTRUCTION OF NOXIOUS ANIMALS OR NOXIOUS PLANTS
- A01M27/00—Apparatus having projectiles or killing implements projected to kill the animal, e.g. pierce or shoot, and triggered thereby
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D3/00—Control of position or direction
- G05D3/12—Control of position or direction using feedback
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01M—CATCHING, TRAPPING OR SCARING OF ANIMALS; APPARATUS FOR THE DESTRUCTION OF NOXIOUS ANIMALS OR NOXIOUS PLANTS
- A01M2200/00—Kind of animal
- A01M2200/01—Insects
- A01M2200/012—Flying insects
Abstract
The invention relates to a target detection tracking system and method based on sound and image information, which belong to the technical field of target detection and solve the problem of detection and tracking of harmful flying organisms capable of emitting special audio frequencies, wherein the system comprises an audio sensing unit, a target detection tracking unit and a target tracking unit, wherein the audio sensing unit is used for collecting indoor audio signals in all directions; the target detection unit is used for adjusting the detection direction and acquiring image information containing a target; the comprehensive control unit is used for capturing target audio according to the indoor audio signal, performing audio coarse tracking on the target, controlling the detection direction of the target detection unit to point to a target area, and acquiring image information containing the target; and capturing a target image according to the image information, performing video fine tracking on the target, controlling the detection direction of the target detection unit to be aligned to the target, and outputting a fine tracking result. The invention combines the audio and image technologies, and realizes the accurate detection and tracking of the harmful flying organisms capable of emitting special audio in an active mode, thereby eliminating the harmful flying organisms.
Description
Technical Field
The invention relates to the technical field of target detection, in particular to a target detection tracking system and a target detection tracking method based on sound and image information.
Background
At present, for flying harmful organisms capable of emitting special audio, such as mosquitoes, mainly depend on passive mosquito killing products such as mosquito coils, mosquito killing lamps and fan type mosquito killers, the mosquito killing efficiency is low, more mosquitoes are attracted by people in a room instead of the passive mosquito killing products, and people still can receive the disturbance of the mosquitoes, so that the due effect cannot be achieved.
In addition, an active laser tracking mosquito killing device scheme is provided, but the laser is far away from the characteristic emission distance, so that an indoor effective safe distance cannot be reserved, a laser device which can burn and kill mosquito power can harm human beings or animals, and the hidden danger of igniting inflammable matters is also caused, so that the safety is poor, and the laser device cannot be put into the market so far.
Disclosure of Invention
In view of the above analysis, the present invention aims to provide a target detection and tracking system and method based on sound and image information, which solves the problem of detecting and tracking harmful flying organisms capable of emitting special audio frequency, and then eliminates the harmful flying organisms.
The purpose of the invention is mainly realized by the following technical scheme:
the invention discloses a target detection tracking system based on sound and image information, which comprises an audio sensing unit (1), a target detection unit (2) and a comprehensive control unit (3);
the audio sensing unit (1) is used for collecting indoor audio signals in an omnidirectional manner and sending the indoor audio signals to the comprehensive control unit (3);
the target detection unit (2) is used for adjusting a detection direction according to the control instruction sent by the comprehensive control unit (3), acquiring image information containing a target and sending the image information to the comprehensive control unit (3);
the comprehensive control unit (3) is used for capturing target audio according to the indoor audio signal, performing audio coarse tracking on a target, and sending a control instruction to control the detection direction of the target detection unit (2) to point to a target area; the system is used for capturing a target image according to the image information, performing video fine tracking on the target, sending a control instruction to control the detection direction of the target detection unit (2) to be aligned with the target, and outputting a fine tracking result.
Further, the target detection unit (2) comprises a high-definition night vision camera module (201), a holder servo device (202) and a base (203);
the high-definition night vision camera module (201) adopts an amorphous silicon microbolometer type sensor and is used for carrying out high-definition night vision imaging on a target and sending an imaging result to the comprehensive control unit (3);
the holder servo device (202) is used for bearing the high-definition night vision camera module (201), receiving the instruction sent by the comprehensive control unit (3) and adjusting the shooting visual angle of the high-definition night vision camera module (201);
the base (203) is used for installing and fixing the holder servo device (202) and the audio sensing unit (1).
Further, the audio perception unit (1) comprises a microphone array, wherein a plurality of microphones are annularly arranged at the edge of the base (203), and a microphone is also arranged at the central position of the base (203); in the microphones which are arranged in an annular manner, each microphone corresponds to a sound receiving area with a certain angle, and a plurality of sound receiving areas are spliced to form a 360-degree omnidirectional sound acquisition area for acquiring indoor audio signals; the microphone at the center collects noise from the stage servo (202).
Further, the integrated control unit (3) comprises an audio signal processing module (301), a pan-tilt control module (302), a target feature extraction module (303) and a target information output module (304);
the audio signal processing module (301) is configured to perform audio processing on the omnidirectional indoor audio signal acquired by the audio sensing unit (1), capture a target audio, and perform audio coarse tracking on the target; obtaining coarse tracking information including a target coarse azimuth angle, a coarse pitch angle and an approximate flight track, and outputting the coarse tracking information to the holder control module (302);
the target feature extraction module (303) is connected with the high-definition night vision camera module (201) and is used for receiving image data shot by the high-definition night vision camera module (201), performing video image processing, capturing a target image, performing video fine tracking on a target and obtaining fine tracking information including a target fine azimuth angle, a fine pitch angle and a precise flight track;
the holder control module (302) is connected with the audio signal processing module (301) and the target feature extraction module (303) and is used for generating a coarse tracking control instruction according to coarse tracking information output by the audio signal processing module (301) and outputting the coarse tracking control instruction to the holder servo device (202) to control the shooting visual angle of the high-definition night vision camera module (201) to be aligned to a target flight area; generating a fine tracking control instruction according to the fine tracking information output by the target feature extraction module (303), and outputting the fine tracking control instruction to a holder servo device (202) to control the shooting center of mass of the high-definition night vision camera module (201) to be aligned with the target position;
and the target information output module (304) is connected with the target feature extraction module (303) and is used for outputting a fine tracking result.
Further, the device also comprises a transmitting unit (4) which is used for aiming at the target in flight according to the video fine tracking result and transmitting water bombs/salt particles to hit the target when the transmitting condition is met;
the emission conditions are satisfied in such a way that the target is located at the centroid position of the image and does not include the forbidden object in the centroid position area.
The invention also discloses a detection tracking method according to the target detection tracking system, which is characterized by comprising the following steps of,
s1, powering up a system, processing indoor all-dimensional real-time audio signals collected by an audio sensing unit (1) by a comprehensive control unit (3), and searching whether suspicious target audio signals exist or not; the target detection unit (2) is in a standby state;
s2, after capturing target audio according to the real-time audio signal, the comprehensive control unit (3) performs audio coarse tracking on the target, controls the target detection unit (2) to work, adjusts the detection direction, collects image information containing the target and sends the image information to the comprehensive control unit (3);
and S3, the comprehensive control unit (3) captures a target image according to the received image information collected by the target detection unit (2), performs video fine tracking, adjusts the shooting angle of the target detection unit (2), enables the target position to be located at the centroid position of the image, and outputs a fine tracking result.
Further, an audio signal processing module (301) of the integrated control unit (3) searches whether a target exists in a room or not based on a speech recognition technology of a deep neural network according to comparison audio data stored in an audio library.
Further, the step S2 includes,
1) After capturing target audio, the audio signal processing module (301) performs audio coarse tracking, determines a coarse azimuth angle, a coarse pitch angle and a rough flight trajectory of the target, and outputs a coarse tracking result;
2) The cloud deck control module (302) receives the coarse tracking result information, generates a coarse tracking control instruction, and outputs the coarse tracking control instruction to a cloud deck servo device (202) of the target detection unit (2) to enable the shooting visual angle of the high-definition night vision camera shooting module (201) to be aligned with the target flight area;
3) The high-definition night vision camera module (201) is started to shoot a video image containing a target.
Further, the step S3 includes,
1) The target feature extraction module (303) receives a video image shot by the high-definition night vision camera module (201), performs target image feature extraction according to a target image sample in an image sample library and based on an image recognition technology of a deep neural network, and captures a target image; calculating a fine tracking adjustment azimuth angle and a pitch angle according to the relative position characteristics in the target image characteristics, performing video fine tracking, and outputting fine tracking information;
2) The cloud deck control module (302) receives the fine tracking information, generates a fine tracking control instruction, and outputs the fine tracking control instruction to a cloud deck servo device (202) of the target detection unit (2) to adjust the shooting visual angle of the high-definition night vision camera module (201); when the position of the target is adjusted to the image centroid position, azimuth angle and pitch angle information of the target is obtained, the target is continuously positioned to obtain accurate track information of the target, and an accurate tracking result comprising an accurate azimuth angle, an accurate pitch angle and an accurate track of the target is output.
3) And the target information output module (304) outputs the fine tracking result.
Further, the method also comprises the following steps:
aiming a target in flight according to the fine tracking result, and launching water bombs/salt particles to hit the target when launching conditions are met;
the emission condition is met, the target is located at the centroid position of the image, and no emission-forbidden object is included in the centroid position area.
The invention has the following beneficial effects:
the invention provides a target detection tracking system and a target detection tracking method based on sound and image information, which are used for roughly tracking harmful flying organisms emitting special audio frequencies in an active mode through an audio frequency identification technology; through the night vision image processing technology, the living beings are precisely tracked, the accurate angle and track information of the living beings is provided, the problem of detection and tracking which can be sent is solved, and then the living beings are eliminated.
Moreover, by adopting a relatively harmless attack mode such as water bombs/salt particles and the like and having an ejectable area/an ejection forbidden area, the safety of personnel, pets, valuables, fragile articles and food is ensured, thereby solving the safety requirement under the environment with people.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, wherein like reference numerals are used to designate like parts throughout.
FIG. 1 is a schematic diagram of a target detection and tracking system according to an embodiment of the present invention;
FIG. 2 is a flowchart of a target detection and tracking method according to an embodiment of the present invention.
Detailed Description
The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form a part hereof, and which together with the embodiments of the invention serve to explain the principles of the invention.
The embodiment discloses a target detection and tracking system based on sound and image information, which is used for detecting, identifying and tracking mosquitoes flying in night environment.
As shown in fig. 1, the object detection and recognition system of the present embodiment includes an audio sensing unit (1), an object detection unit (2) and a comprehensive control unit (3);
the audio sensing unit (1) is used for collecting indoor audio signals in an omnidirectional manner and sending the indoor audio signals to the comprehensive control unit (3);
the target detection unit (2) is used for adjusting a detection direction according to the control instruction sent by the comprehensive control unit (3), acquiring image information containing a target and sending the image information to the comprehensive control unit (3);
the comprehensive control unit (3) is used for capturing target audio according to the indoor audio signal, performing audio coarse tracking on a target, and sending a control instruction to control the detection direction of the target detection unit (2) to point to a target area; the system is used for capturing a target image according to the image information, performing video fine tracking on the target, sending a control instruction to control the detection direction of the target detection unit (2) to be aligned with the target, and outputting a fine tracking result.
Specifically, the audio sensing unit (1) comprises a microphone array, wherein the microphone array adopts a ring configuration of 8+1 microphones, the 8 microphones are annularly arranged at the edge of a base (203) of the target detection unit (2), and 1 microphone is additionally arranged at the central position of the base (203); among 8 microphones arranged in an annular mode, each microphone corresponds to a sound receiving area with a certain angle, the sound receiving areas are spliced to form a 360-degree omnidirectional sound collecting area used for collecting indoor audio signals, suppressing noise and eliminating reverberation, and a microphone collecting platform servo device (202) at the central position is used for noise elimination processing in audio resolving.
Specifically, the target detection unit (2) comprises a high-definition night vision camera module (201), a holder servo device (202) and a base (203);
the high-definition night vision camera module (201) adopts an amorphous silicon microbolometer type sensor, has the functions of megapixel high definition, fog penetration, whole-course automatic focusing and IR day and night, is used for carrying out high-definition night vision imaging on a target and sending an imaging result to the comprehensive control unit (3);
the holder servo device (202) is used for bearing the high-definition night vision camera module (201), receiving the instruction sent by the comprehensive control unit (3) and adjusting the shooting visual angle of the high-definition night vision camera module (201).
Specifically, the pan-tilt servo device (202) comprises an azimuth servo motor, a pitch servo motor, a matched transmission mechanism of the pitch servo motor, a sensor and the like. The closed-loop PID control mode is adopted, the response speed is high, the control precision is high, the azimuth servo motor and the matched transmission mechanism and the sensor thereof are used for realizing the azimuth rotation control of the holder, and the pitching servo motor and the matched transmission mechanism and the sensor thereof are used for realizing the pitching rotation control of the holder.
The base (203) is used for installing and fixing the holder servo device (202) and the audio sensing unit (1).
Specifically, the integrated control unit (3) comprises an audio signal processing module (301), a pan-tilt control module (302), a target feature extraction module (303) and a target information output module (304);
the audio signal processing module (301) is configured to perform audio processing on omnidirectional indoor audio signals collected by the audio sensing unit (1), realize sound enhancement and find a sound source target azimuth by using a beam forming technology and a filtering algorithm, compare acoustic features in a database by using a machine learning technology, capture target audio, and perform audio rough tracking on a target;
the method comprises the following steps of performing audio coarse tracking on a target, extracting collected sound source characteristics, particularly sound frequency characteristics according to target audio samples stored in an audio library and combining a voice recognition technology based on a deep neural network, comparing the sound source characteristics with stored mosquito flight characteristics to confirm that a sound source accords with mosquito acoustic characteristics, capturing an audio signal generated in the target flight process, and determining a coarse azimuth angle, a coarse pitch angle and an approximate flight track of the nearest target; outputting the rough tracking result to the cloud deck servo device (202) to control the cloud deck to point the shooting visual angle of the high-definition night vision camera module (201) to a target flight area according to the rough tracking result;
specifically, the audio signal processing module (301) comprises an audio acquisition card array and an audio processing card module;
the audio acquisition card array is correspondingly connected with the microphone array; carrying out digital acquisition on sound signals received by each microphone covering a specific direction to obtain multi-channel digital audio data;
the audio processing card module is connected with the audio acquisition card array; receiving multi-channel digital audio data, judging whether the multi-channel digital audio data has audio data the same as that of a target audio sample or not according to the target audio sample stored in an audio library and based on a voice recognition technology of a deep neural network, determining azimuth angle and pitch angle information of rough tracking according to the azimuth angle and pitch angle of a microphone corresponding to the digital audio data, and obtaining approximate track information of a target by continuously measuring the angle information of the target.
And when a plurality of suspected targets exist, performing audio coarse tracking by taking the digital audio data with the maximum amplitude as a target audio signal, and outputting a coarse tracking result.
The target feature extraction module (303) is connected with the target detection unit (2) and is used for extracting target image features according to video image data shot by the high-definition night vision camera module (201) of the target detection unit (2) and target image samples in the image sample library and based on the image recognition technology of the deep neural network, comparing the extracted target image features with the target image features in the database, capturing a target image and simultaneously recognizing targets such as people, animals, valuables and the like; calculating a fine tracking adjustment azimuth angle and a pitch angle according to relative position characteristics in the target image characteristics to form fine tracking information, and outputting the fine tracking information to the holder control module (302) for adjusting the shooting visual angle of the high-definition night vision camera module (201); when the position of the target is adjusted to the image centroid position, the accurate azimuth angle and the accurate pitch angle information of the target are obtained, the target is continuously positioned, the accurate track information of the target is obtained, and the accurate tracking result including the accurate azimuth angle, the accurate pitch angle and the accurate track of the target is output to a target information output module (304).
The holder control module (302) is connected with the audio signal processing module (301) and the target feature extraction module (303) and is used for generating a coarse tracking control command comprising a coarse azimuth angle and a coarse pitch angle according to coarse tracking result information output by the audio signal processing module (301) and outputting the coarse tracking control command to the holder servo device (202) of the target detection unit (2) to control the shooting visual angle of the high-definition night vision camera shooting module (201) to be aligned with the target flight area; and generating a fine tracking control instruction including a fine azimuth angle and a fine pitch angle according to the fine tracking information output by the target feature extraction module (303), and outputting the fine tracking control instruction to a holder servo device (202) of the target detection unit (2) to control the shooting visual angle of the high-definition night vision camera module (201) so that the shooting center of mass is aligned to the target position.
And the target information output module (304) is connected with the target feature extraction module (303) and is used for outputting a fine tracking result.
Preferably, the present embodiment further comprises a launching unit (4) for aiming the target in flight according to the fine tracking result, and launching water bombs/salt particles to hit and kill the target when the launching condition is met.
Specifically, the launching unit (4) comprises a launching control module (401) and a water bomb/salt particle launching module (402);
the launching control module (401) receives the fine positioning result, judges whether launching conditions are met or not, and controls the water bomb/salt particle launching module (402) to launch water bombs/salt particles if the launching conditions are met; otherwise, the water bomb/salt particle launching module (402) is prohibited from launching.
The emission-forbidden objects which are forbidden to be emitted and comprise people or pets are not included in the area of the centroid position.
The water bullet/salt particle launching module (402) consists of a gun barrel, a bullet chamber and an electric launching device.
The water bomb comprises: the spherical water bullet for the children toy gun has the diameter of 6-7 mm, the components of the spherical water bullet are water-absorbing resin, the spherical water bullet is a novel functional polymer material, can absorb water which is thousands of times of the self weight, has excellent water-retaining property, is non-toxic and non-irritant to a human body, is broken when being impacted, is evaporated and disappears finally, and is safe and reliable.
The salt shot (shot): the first product of SALT-POWER-GUN, available from KO-TING, usa, was used to provide ammunition that was normally dry, fine-SALT, which when fired shoots the target as a shot, killing targets such as mosquitoes and flies, in a closer effective distance than water bombs, but safer.
Specifically, the present embodiment further includes a power supply unit (5) for supplying power to the entire device;
and the wireless communication circuit (6) is used for connecting a network through WiFi to realize functions of remote access control, firmware upgrading and the like of a user, such as a target audio library, an image library, a newly-added working mode and the like.
The embodiment also discloses a target detection and tracking method based on sound and image information, as shown in fig. 2, the specific steps are as follows:
s1, powering up a system, processing indoor all-dimensional real-time audio signals collected by the audio sensing unit (1) by the comprehensive control unit (3), and searching whether suspicious targets exist or not; the target detection unit (2) is in a standby state;
specifically, the comprehensive control unit (3) searches whether a target exists in a room or not according to comparison audio data stored in an audio library and based on a speech recognition technology of a deep neural network.
S2, after capturing target audio according to the real-time audio signal, the comprehensive control unit (3) performs audio coarse tracking on the target, controls the target detection unit (2) to work, adjusts the detection direction, collects image information containing the target and sends the image information to the comprehensive control unit (3);
specifically, the method comprises the following steps of,
1) After the target audio is captured, the audio signal processing module (301) carries out audio coarse tracking, determines a coarse azimuth angle, a coarse pitch angle and an approximate flight track of the target, and outputs a coarse tracking result;
specifically, audio coarse tracking is performed, wherein audio signals generated in the target flight process are captured according to target audio samples stored in an audio library and by combining a voice recognition technology based on a deep neural network, an azimuth angle, a pitch angle and a rough flight track of a nearest target are determined, and a coarse tracking result is output;
2) The cloud deck control module (302) receives the coarse tracking result information, generates a coarse tracking control command comprising a coarse azimuth angle and a coarse pitch angle, and outputs the coarse tracking control command to a cloud deck servo device (202) of the target detection unit (2) to enable the shooting visual angle of the high-definition night vision camera module (201) to be aligned to a target flight area;
3) The high-definition night vision camera module (201) is started to shoot a video image containing a target.
And S3, the comprehensive control unit (3) captures a target image according to the received image information acquired by the target detection unit (2), performs video fine tracking, adjusts the shooting angle of the target detection unit (2), enables the target position to be located at the centroid position of the image, and outputs a fine tracking result.
Specifically, the method comprises the following steps of,
1) The target feature extraction module (303) receives a video image shot by the high-definition night vision camera module (201), performs target image feature extraction according to a target image sample in an image sample library and based on an image recognition technology of a deep neural network, and captures a target image; calculating a fine tracking adjustment azimuth angle and a pitch angle according to the relative position characteristics in the target image characteristics, performing video fine tracking, and outputting fine tracking information;
2) The cloud deck control module (302) receives the fine tracking information, generates a fine tracking control instruction, and outputs the fine tracking control instruction to a cloud deck servo device (202) of the target detection unit (2) to adjust the shooting visual angle of the high-definition night vision camera module (201); when the position of the target is adjusted to the image centroid position, azimuth angle and pitch angle information of the target is obtained, the target is continuously positioned to obtain accurate track information of the target, and an accurate tracking result comprising an accurate azimuth angle, an accurate pitch angle and an accurate track of the target is output.
3) And the target information output module (304) outputs a fine tracking result.
Preferably, the present embodiment may further include the following steps:
and aiming the target in flight according to the fine tracking result, and shooting water bombs/salt particles to hit and kill the target when the shooting condition is met.
The emission-forbidden objects which are forbidden to be emitted and comprise people or pets are not included in the area of the centroid position.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention.
Claims (2)
1. A target detection tracking system based on sound and image information is characterized by comprising an audio sensing unit (1), a target detection unit (2) and a comprehensive control unit (3);
the audio sensing unit (1) is used for collecting indoor audio signals in an omnidirectional manner and sending the indoor audio signals to the comprehensive control unit (3);
the target detection unit (2) is used for adjusting a detection direction according to the control instruction sent by the comprehensive control unit (3), acquiring image information containing a target and sending the image information to the comprehensive control unit (3);
the comprehensive control unit (3) is used for capturing target audio according to the indoor audio signal, performing audio coarse tracking on a target, and sending a control instruction to control the detection direction of the target detection unit (2) to point to a target area; the system is used for capturing a target image according to the image information, performing video fine tracking on the target, sending a control instruction to control the detection direction of the target detection unit (2) to be aligned with the target, and outputting a fine tracking result;
the target detection unit (2) comprises a high-definition night vision camera module (201), a holder servo device (202) and a base (203);
the high-definition night vision camera module (201) adopts an amorphous silicon microbolometer type sensor and is used for carrying out high-definition night vision imaging on a target and sending an imaging result to the comprehensive control unit (3);
the holder servo device (202) is used for bearing the high-definition night vision camera module (201), receiving the instruction sent by the comprehensive control unit (3) and adjusting the shooting visual angle of the high-definition night vision camera module (201);
the base (203) is used for installing and fixing the pan-tilt servo device (202) and the audio sensing unit (1);
the audio sensing unit (1) comprises a microphone array, the microphone array adopts a ring configuration of 8+1 microphones, the 8 microphones are annularly arranged at the edge of a base (203) of the target detection unit (2), and 1 microphone is additionally arranged at the central position of the base (203); among 8 microphones which are annularly arranged, each microphone corresponds to a sound receiving area with a certain angle, a plurality of sound receiving areas are spliced to form a 360-degree omnidirectional sound collecting area which is used for collecting indoor audio signals, suppressing noise and eliminating reverberation, and the microphone at the central position is used for collecting the noise of a holder servo device (202) and is used for audio resolving and noise elimination processing;
the comprehensive control unit (3) comprises an audio signal processing module (301), a holder control module (302), a target feature extraction module (303) and a target information output module (304);
the audio signal processing module (301) comprises an audio acquisition card array and an audio processing card module;
the audio acquisition card array is correspondingly connected with the microphone array; carrying out digital acquisition on sound signals received by each microphone covering a specific direction to obtain multi-channel digital audio data;
the audio processing card module is connected with the audio acquisition card array; receiving multi-channel digital audio data, judging whether the multi-channel digital audio data has audio data which is the same as that of a target audio sample according to the target audio sample stored in an audio library and based on a voice recognition technology of a deep neural network, determining azimuth angle and pitch angle information of rough tracking according to the azimuth angle and pitch angle of a microphone corresponding to the digital audio data, and obtaining approximate track information of a target by continuously measuring the angle information of the target and outputting the approximate track information of the target to a holder control module (302) of a comprehensive control unit (3);
the method comprises the following steps of roughly tracking the audio frequency of a target, extracting collected sound source characteristics, particularly sound frequency characteristics according to target audio samples stored in an audio library and combining a voice recognition technology based on a deep neural network, comparing the collected sound source characteristics with stored mosquito flight characteristics to confirm that a sound source accords with mosquito acoustic characteristics, and capturing audio signals generated in the target flight process; when a plurality of suspected targets exist, digital audio data with the maximum amplitude are used as target audio signals, audio coarse tracking is carried out to determine a coarse azimuth angle, a coarse pitch angle and an approximate flight track of the target as coarse tracking results, and the coarse tracking results are output to a holder control module (302) to generate a coarse tracking control instruction; outputting a coarse tracking control instruction to a holder servo device (202) to control a holder to control the shooting visual angle of the high-definition night vision camera module (201) to point to a target flight area according to a coarse tracking result;
the target feature extraction module (303) is connected with the high-definition night vision camera module (201) and is used for receiving image data shot by the high-definition night vision camera module (201), performing video image processing, capturing a target image, performing video fine tracking on a target and obtaining fine tracking information including a target fine azimuth angle, a fine pitch angle and a precise flight track;
the holder control module (302) is connected with the audio signal processing module (301) and the target feature extraction module (303) and is used for generating a coarse tracking control instruction according to coarse tracking information output by the audio signal processing module (301) and outputting the coarse tracking control instruction to the holder servo device (202) to control the shooting visual angle of the high-definition night vision camera module (201) to be aligned to a target flight area; the high-definition night vision camera module (201) is started to shoot a video image containing a target; generating a fine tracking control instruction according to the fine tracking information output by the target feature extraction module (303), and outputting the fine tracking control instruction to a holder servo device (202) to control the shooting mass center of the high-definition night vision camera module (201) to be aligned with the target position;
the target information output module (304) is connected with the target feature extraction module (303) and used for outputting a fine tracking result;
the device also comprises a launching unit (4) which is used for aiming at a target in flight according to the video fine tracking result and launching water bombs/salt particles to hit the target when launching conditions are met;
the emission conditions are satisfied in such a way that the target is located at the centroid position of the image and does not include the forbidden object in the centroid position area.
2. The detection tracking method of the object detection tracking system based on sound and image information according to claim 1, characterized by comprising,
s1, powering up a system, processing indoor all-around real-time audio signals acquired by an audio sensing unit (1) by a comprehensive control unit (3), and searching whether suspicious target audio signals exist or not; the target detection unit (2) is in a standby state;
s2, when the comprehensive control unit (3) captures a target audio according to the real-time audio signal, performing audio coarse tracking on the target, controlling the target detection unit (2) to work, adjusting a detection direction, collecting image information containing the target, and sending the image information to the comprehensive control unit (3);
s3, the comprehensive control unit (3) captures a target image according to the received image information collected by the target detection unit (2), performs video fine tracking, adjusts the shooting angle of the target detection unit (2), enables the target position to be located at the centroid position of the image, and outputs a fine tracking result;
an audio signal processing module (301) of the comprehensive control unit (3) searches whether a target exists in a room or not according to comparison audio data stored in an audio library and based on a voice recognition technology of a deep neural network;
said step S2 comprises the steps of,
1) After capturing target audio, the audio signal processing module (301) performs audio coarse tracking, determines a coarse azimuth angle, a coarse pitch angle and an approximate flight track of a target, and outputs a coarse tracking result;
2) The pan-tilt control module (302) receives the coarse tracking result information, generates a coarse tracking control command, and outputs the coarse tracking control command to a pan-tilt servo device (202) of the target detection unit (2) to enable the shooting visual angle of the high-definition night vision camera shooting module (201) to be aligned to the target flight area;
3) The high-definition night vision camera module (201) is started to shoot a video image containing a target;
said step S3 comprises the steps of,
1) The target feature extraction module (303) receives a video image shot by the high-definition night vision camera module (201), extracts the features of the target image according to a target image sample in an image sample library and based on the image recognition technology of a deep neural network, and captures the target image; calculating a fine tracking adjustment azimuth angle and a pitch angle according to relative position characteristics in the target image characteristics, performing video fine tracking, and outputting fine tracking information;
2) The pan-tilt control module (302) receives the fine tracking information, generates a fine tracking control instruction, and outputs the fine tracking control instruction to a pan-tilt servo device (202) of the target detection unit (2) to adjust the shooting visual angle of the high-definition night vision camera module (201); when the position of the target is adjusted to the image centroid position, azimuth angle and pitch angle information of the target is obtained, accurate track information of the target is obtained by continuously positioning the target, and an accurate tracking result comprising a target accurate azimuth angle, an accurate pitch angle and an accurate track is output;
3) The target information output module (304) outputs the fine tracking result;
further comprising the steps of:
aiming a target in flight according to the fine tracking result, and launching water bombs/salt particles to hit the target when launching conditions are met;
the emission conditions are satisfied in such a way that the target is located at the centroid position of the image and does not include the forbidden object in the centroid position area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910735059.8A CN110347187B (en) | 2019-08-09 | 2019-08-09 | Target detection tracking system and method based on sound and image information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910735059.8A CN110347187B (en) | 2019-08-09 | 2019-08-09 | Target detection tracking system and method based on sound and image information |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110347187A CN110347187A (en) | 2019-10-18 |
| CN110347187B true CN110347187B (en) | 2023-01-06 |
Family
ID=68184533
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910735059.8A Active CN110347187B (en) | 2019-08-09 | 2019-08-09 | Target detection tracking system and method based on sound and image information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110347187B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111387170A (en) * | 2020-03-05 | 2020-07-10 | 中国地质大学（武汉） | Intelligent inspection mouse trapping robot device |
| CN111526292B (en) * | 2020-04-30 | 2021-08-03 | 浙江大华技术股份有限公司 | Audio and video mixing monitoring method, equipment and storage device |
| CN112418052A (en) * | 2020-11-18 | 2021-02-26 | 陕西科技大学 | Mosquito recognition and expelling device and method based on deep learning |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060118805A (en) * | 2005-05-17 | 2006-11-24 | 재능대학산학협력단 | The mosquito extirpation system which uses a multiplex frequency |
| CN101766156A (en) * | 2008-12-28 | 2010-07-07 | 张少伟 | Solar broad-spectrum multicolor stroboscopic concentrating long-shot high-trapping LED lamp |
| CN105104332A (en) * | 2015-08-28 | 2015-12-02 | 宋健 | Fixed type intelligentized mosquito catching and killing system |
| CN106070133A (en) * | 2016-06-28 | 2016-11-09 | 深圳市元征科技股份有限公司 | Flight mosquito killer control method and device |
| CN106417220A (en) * | 2016-09-30 | 2017-02-22 | 叶志刚 | Intelligent net drawing and mosquito catching system |
| US9807996B1 (en) * | 2016-05-28 | 2017-11-07 | Simon Siu-Chi Yu | Bug eater |
| CN108271753A (en) * | 2018-01-05 | 2018-07-13 | 广州翼鲲生物科技有限公司 | A kind of insect pest control method and system |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102726358A (en) * | 2012-05-31 | 2012-10-17 | 湖北天罡投资有限公司 | Laser mosquito killer |
| US10588306B2 (en) * | 2015-04-13 | 2020-03-17 | University Of Florida Research Foundation, Inc. | Wireless smart mosquito and insect trap device, network and method of counting a population of the mosquitoes or insects |
| CN105052867B (en) * | 2015-08-28 | 2017-08-25 | 淄博职业学院 | A kind of intelligent mosquito catching-killing method |
| CN106719524B (en) * | 2016-03-11 | 2023-10-31 | 深圳市镭神智能系统有限公司 | Intelligent laser mosquito eradication device and robot |
| CN107333120B (en) * | 2017-08-11 | 2020-08-04 | 吉林大学 | Integrated sensor based on microphone array and stereoscopic vision |
| US10375947B2 (en) * | 2017-10-18 | 2019-08-13 | Verily Life Sciences Llc | Insect sensing systems and methods |
| CN208159895U (en) * | 2018-01-18 | 2018-11-30 | 贵州省烟草公司黔南州公司 | A kind of tobacco field the moth pests multifunctional monitoring equipment |
| CN109430197B (en) * | 2018-10-23 | 2021-06-15 | 西安理工大学 | Self-aiming injection deinsectization device based on Venturi effect |
-
2019
- 2019-08-09 CN CN201910735059.8A patent/CN110347187B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060118805A (en) * | 2005-05-17 | 2006-11-24 | 재능대학산학협력단 | The mosquito extirpation system which uses a multiplex frequency |
| CN101766156A (en) * | 2008-12-28 | 2010-07-07 | 张少伟 | Solar broad-spectrum multicolor stroboscopic concentrating long-shot high-trapping LED lamp |
| CN105104332A (en) * | 2015-08-28 | 2015-12-02 | 宋健 | Fixed type intelligentized mosquito catching and killing system |
| US9807996B1 (en) * | 2016-05-28 | 2017-11-07 | Simon Siu-Chi Yu | Bug eater |
| CN106070133A (en) * | 2016-06-28 | 2016-11-09 | 深圳市元征科技股份有限公司 | Flight mosquito killer control method and device |
| CN106417220A (en) * | 2016-09-30 | 2017-02-22 | 叶志刚 | Intelligent net drawing and mosquito catching system |
| CN108271753A (en) * | 2018-01-05 | 2018-07-13 | 广州翼鲲生物科技有限公司 | A kind of insect pest control method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110347187A (en) | 2019-10-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110347187B (en) | Target detection tracking system and method based on sound and image information | |
| US10040554B1 (en) | Method and apparatus for drone detection and disablement | |
| US11315396B2 (en) | Threat identification device and system with optional active countermeasures | |
| US11879705B2 (en) | System and method for active shooter defense | |
| US10586109B1 (en) | Indoor gunshot detection with video analytics | |
| US8248473B2 (en) | Robotic sentry with low dispersion acoustic projector | |
| KR101251867B1 (en) | Adaptive apparatus extirpating the nocturnal animal for protecting farm products, and control terminal for adaptive apparatus extirpating the nocturnal animal | |
| CN106569511A (en) | Unmanned aerial vehicle trapping interception system based on electromagnetic interference and trapping interception method thereof | |
| US20180240230A1 (en) | Mobile system for explosive device detection and instant active response | |
| US10229348B2 (en) | Induction detector systems | |
| RU2700107C1 (en) | Anti-drones combat system | |
| US11282353B1 (en) | Gunshot detection within an indoor environment with video analytics | |
| US20220050479A1 (en) | Uav-based protection from active shooters | |
| US20070291123A1 (en) | Remote operated surveillance system | |
| US11021249B2 (en) | Drone-based, attacker neutralization | |
| US8351659B2 (en) | Eye detection system | |
| US10339426B2 (en) | Induction system for crowd monitoring | |
| CN105578410A (en) | Novel unmanned plane | |
| US10185896B2 (en) | Induction system for mold remediation | |
| CN106873626A (en) | A kind of target-seeking system of passive location | |
| RU2578524C2 (en) | System for controlling integrated methods for combating small-sized unmanned aerial vehicles | |
| JP2006029754A (en) | Flying body tracking method and device therefor | |
| RU2737634C2 (en) | Firing method of guided missile with laser half-active homing head and device realizing thereof | |
| KR102505309B1 (en) | Remote shooting control device for drones using radar | |
| KR20190141941A (en) | Detecting and neutralizing apparatus of flight vehicle capable of flying underground and method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |