CN109669460B

CN109669460B - Intelligent control method for small and medium-sized photoelectric turntable for target detection

Info

Publication number: CN109669460B
Application number: CN201811640570.1A
Authority: CN
Inventors: 张静; 桑柳; 胡锐; 魏磊杰; 周秦; 李云松
Original assignee: Xidian University
Current assignee: Xidian University
Priority date: 2018-12-29
Filing date: 2018-12-29
Publication date: 2021-09-03
Anticipated expiration: 2038-12-29
Also published as: CN109669460A

Abstract

The invention provides an intelligent control method of a small and medium-sized photoelectric turntable for target detection, which relates to the intelligent control of the small photoelectric turntable and is divided into three working modes according to the running and application scenes of small photoelectric turntable equipment, wherein the first mode is as follows: the panoramic camera obtains a plurality of suspected targets, and the plurality of suspected targets are sequentially confirmed by the zooming high-definition camera; and a second mode: the panoramic camera obtains a plurality of suspected targets and confirms the targets by using a zooming high-definition camera, wherein one target with extremely high confidence coefficient exists; and a third mode: when the panoramic camera fails, the photoelectric turret drives the zooming high-definition or zooming infrared camera to rotate for a circle to work. The method has the advantages that false detection and missing detection of target detection, photoelectric turret anti-shaking processing and intelligent zooming of the camera are avoided by fusing similar targets, so that the intelligent control method has faster response and more accurate control; the device is used for unmanned equipment, and can also be used in manned equipment or fixed positions.

Description

Intelligent control method for small and medium-sized photoelectric turntable for target detection

Technical Field

The invention belongs to the technical field of equipment control, mainly relates to intelligent control of a small-sized photoelectric turntable, and particularly relates to an intelligent control method of a small-sized photoelectric turntable for target detection. The target detection method is used for target detection in target search occasions requiring 360 degrees or specified azimuth ranges.

Background

With the rapid development of unmanned equipment such as unmanned aerial vehicles and unmanned vehicles, the requirements on cameras carried on the unmanned equipment are higher and higher, and the real-time detection is performed from the requirement on higher resolution to the requirement on a larger detection area and a longer detection distance to 360 degrees at present, so that the method can better adapt to the richer functional requirements of the unmanned equipment, such as target search, target tracking or obstacle judgment and the like.

Early unmanned devices, such as the genistein 1 in large, have carried thereon a small, lightweight, wide-angle motion camera GoPro. However, the third party camera is not convenient for image transmission and parameter control of the camera, some manufacturers develop a camera holder with better compatibility as an intermediary between the camera and the unmanned equipment, the camera is installed on the holder, and the unmanned equipment controls the camera through the holder. In recent years, unmanned equipment is more and more widely applied to severe environments such as geological exploration and water quality detection, photoelectric equipment mounted on the unmanned aerial vehicle needs to be continuously searched in one area, in order to improve the searching efficiency, the photoelectric equipment mounted on the unmanned aerial vehicle starts to appear as a panoramic camera, and meanwhile, higher requirements are provided for time delay of image transmission and camera parameter control. However, the panoramic camera still cannot meet the requirement that the unmanned equipment carries out accurate detection on a certain specific position or target, and by combining the requirement, the unmanned equipment needs the photoelectric equipment which can carry out panoramic detection, is rotatable and flexible in rotation, has short response time and is high in precision.

Therefore, many manufacturers develop special optoelectronic devices to solve the rotation problem, but the optoelectronic devices have a small field angle and a long detection distance, and are difficult to detect a near target and 360 degrees around the device in real time. In order to better solve the problems, some manufacturers integrate a high-definition camera on a rotatable turret, and in order to realize short-distance and long-distance detection at the same time, a panoramic camera and photoelectric equipment are combined together to form a small photoelectric turntable, the small photoelectric turntable is directly installed at a detection part of the unmanned equipment, surrounding environment information such as image data is collected in real time and is transmitted to a user through a link, and the user controls the unmanned equipment through the returned information.

The processing flow of the small photoelectric turntable which is put into use at present is generally as follows: the unmanned equipment transmits the information collected by the high-definition camera back to the user console, and the user controls the rotation of the turret through the information displayed on the interface of the console. However, the data downloading link rate of the unmanned equipment is often low, the data returning speed is slow, and great transmission delay is brought; in addition, the user processes the information through the interface display, the precision is low, and meanwhile, the processing delay is increased.

The invention is searched in a certain range, and no patent or report which is closely related to the subject of the invention is found.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides an intelligent control method for a small and medium-sized photoelectric turntable, which is faster in response and more accurate in control, for target detection.

The invention relates to an intelligent control method of a small-sized photoelectric turntable for target detection, which mainly relates to the intelligent control of the small-sized photoelectric turntable, wherein the small-sized photoelectric turntable is provided with a panoramic camera and a photoelectric turret integrating a zooming high-definition camera, a zooming infrared camera and a laser range finder, the centers of the zooming high-definition camera, the zooming infrared camera and the laser range finder are consistent with the direction of the photoelectric turret, and the small-sized photoelectric turntable is subjected to motion control through an interface.

According to the difference of the operation condition and the application scene of the small-sized photoelectric turntable equipment, the following three working modes are adopted:

(1) the first working mode is as follows: all the small-size photoelectric turntable equipment normally operate, weather visibility is high, a plurality of suspected targets are obtained by using a panoramic camera, the plurality of suspected targets are sequentially confirmed by a high-definition zooming camera, and the method specifically comprises the following steps:

1.1 target information fusion: the panoramic camera in the small photoelectric turntable is always in a working state of target detection, and after target detection is carried out through videos output by the panoramic camera, if a target detection result is a plurality of suspected targets and one target with extremely high confidence coefficient exists in the suspected targets, a working mode II is switched to; if the target detection result is only a plurality of suspected targets, entering a first working mode, and setting the movement control variable of the photoelectric turret to be 1; when the photoelectric turret motion control variable is '1', continuously reading N _ p frames of panoramic video target detection results from the target detection results, fusing similar target information in the N _ p frames to obtain the total number N _ s of the current suspected targets and information of each suspected target, wherein N _ p is the total number of the read panoramic video frames;

1.2 sorting with confidence as a reference: sorting the N _ s suspected targets from large to small by taking the confidence coefficient in the suspected target information as a reference;

1.3, planning the azimuth angle and the pitch angle of the photoelectric turret for the current target to be confirmed for the first time: according to the suspected target sorting result, confirming suspected targets from big to small in sequence; taking the Nth target in the suspected target sorting result as an example, reading the azimuth angle and the pitch angle in the Nth suspected target information, taking the read values as the first planning result of the photoelectric turret and generating a movement instruction of the photoelectric turret, and rotating the photoelectric turret according to the first planned azimuth angle and pitch angle after receiving the movement instruction;

1.4 camera focal length adjustment: the zooming high-definition camera is always in a working state of target detection, after the photoelectric turret rotates to an azimuth angle and a pitch angle which are planned for the first time, an N _ v frame zooming high-definition camera target detection result is read, the width and height information of a target with the maximum reliability in the target detection result is taken, the focal length of the zooming high-definition camera which needs to be adjusted is calculated by combining the focal length and the field angle of the current zooming high-definition camera, and the focal length is adjusted, wherein N _ v is the total number of the read zooming high-definition camera video frames;

1.5 photoelectric turret anti-shaking treatment: taking the firstly planned azimuth angle and the firstly planned pitch angle as centers, respectively generating four offset positions after the centers are offset by an angle of M degrees up, down, left and right, and taking the four offset positions as the re-planned results of the photoelectric turret to generate a photoelectric turret motion instruction, and after receiving the motion instruction, the photoelectric turret rotates according to the re-planned azimuth angle and the re-planned pitch angle to complete the anti-shaking treatment of the photoelectric turret;

1.6 target confirmation based on the target detection probability of the zooming high-definition camera: after the photoelectric turret rotates to an offset position every time, reading an N _ v frame zooming high-definition camera target detection result, taking a central position and a probability value with the maximum target confidence coefficient in target detection results of all offset positions with the center as a reference as the detection probability of a current target, and recording corresponding target information such as an azimuth angle, a pitch angle and the like;

1.7 repeating the above steps 1.3-1.6 for the remaining N _ s-1 targets;

1.8 outputting the target processing result: and outputting the number of the suspected targets, the detection result information of each suspected target and the confidence degree of the target in the panoramic video for other equipment or users to use in the subsequent processing process.

(2) And a second working mode: the small photoelectric rotary table equipment runs normally and is high in weather visibility, a plurality of suspected targets are obtained by using a panoramic camera, one target with high confidence coefficient exists in the suspected targets, namely the target to be confirmed is obtained, a high-definition camera needs to be zoomed to confirm the target to be confirmed, the detection result information of the target to be confirmed is recorded, a second working mode is entered, and the movement control variable of the photoelectric rotary tower is set to be 2; when the movement control variable of the photoelectric turret is '2', the azimuth angle and the pitch angle in the detection result information of the target to be confirmed are taken, and the first mode is similar to the working mode: planning the azimuth angle and the pitch angle of the photoelectric turret for the current target to be confirmed for the first time; adjusting the focal length of the camera; performing anti-shaking treatment on the photoelectric turret; target confirmation and target processing result output based on the target detection probability of the zooming high-definition camera; the method is different from the working mode I in that laser ranging is added between target confirmation and target processing result output based on the target detection probability of the zooming high-definition camera, if the detection probability of the current target exceeds a set laser ranging threshold, a laser ranging machine is started to carry out distance measurement on the current target, and the obtained laser ranging result information is added into the detection result of the current target.

(3) And a third working mode: if the panoramic camera breaks down, unable normal operating, other equipment operation are normal, use the rotatory a week of photoelectricity capstan head drive zoom high definition video camera or zoom infrared camera and replace panoramic camera work. If the visibility in the current weather is low and the imaging effect of the optical camera is not as good as that of the infrared camera, directly using the photoelectric turret to drive the zooming infrared camera to rotate for a circle to replace the panoramic camera to work, and executing the step 3.2; if the current weather visibility is better, the imaging effect of the optical camera is better, then the photoelectric turret is used for driving the zooming high-definition video camera to rotate for a circle to replace the panoramic camera to work, the step 3.1 is executed, and the method specifically comprises the following procedures:

3.1 replace the panoramic camera with a zoom high definition video camera: the zooming high-definition camera is always in a working state of target detection, the photoelectric turret is calculated according to the upward maximum field angle of the zooming high-definition camera in the azimuth direction and needs to rotate for R _ v times for one circle, similar target information in the results of the R _ v times of target detection is fused, and the fused target information is recorded and output for other equipment or users to use in the subsequent processing process;

3.2 use zoom infrared camera instead of panoramic camera: and the zooming infrared camera is always in a working state of target detection, the photoelectric turret is calculated to rotate R _ f times for one circle according to the upward maximum field angle of the zooming infrared camera, similar target information in the results of the R _ f times of target detection is fused, and the fused target information is recorded and output for other equipment or users in the subsequent processing process.

The invention can directly process the acquired ambient environment data in real time on unmanned equipment and control the small photoelectric turntable to work synchronously.

Compared with the prior art, the invention has the technical advantages that:

firstly, in the invention, all control instructions of the small-sized photoelectric turntable are generated in real time by an intelligent control method and comprise a photoelectric turret rotation instruction, a laser range finder starting instruction, a laser range finder closing instruction, a zooming high-definition camera focal length adding instruction, a zooming high-definition camera focal length subtracting instruction, a zooming infrared camera focal length adding instruction and a zooming infrared camera focal length subtracting instruction, parameters required for controlling the movement of the small-sized photoelectric turntable are obtained and a control instruction is generated by fusing and analyzing target detection result information of a panoramic camera and the zooming high-definition camera or target detection result information of the zooming infrared camera, data are not required to be downloaded to a control console and manual intervention is not required, and time delay caused by data transmission and manual operation is reduced;

secondly, in the invention, target information including an azimuth angle and a pitch angle within a range of 360 degrees around the small-sized photoelectric turntable is obtained through target detection of the panoramic camera, similar targets in multi-frame panoramic video target detection results are fused, and the problems that the same target repeatedly appears in multi-frame panoramic videos and errors possibly appear in single-frame panoramic video target detection are solved; after the targets are fused, a plurality of suspected targets are obtained, the azimuth angle and the pitch angle of the photoelectric turret are planned for each suspected target for the first time, each suspected target is confirmed through a zooming high-definition camera, and anti-shaking processing is carried out on the photoelectric turret;

thirdly, according to the running condition of the small photoelectric turntable equipment and the difference of application scenes, three working modes of the small photoelectric turntable are designed and respectively applied to the condition that the small photoelectric turntable equipment runs normally and is high in weather visibility, a plurality of suspected targets are obtained by using a panoramic camera, and the plurality of suspected targets are sequentially confirmed by a zooming high-definition camera; all small photoelectric turntable equipment normally operates and has high weather visibility, a plurality of suspected targets are obtained by using a panoramic camera, one target with high confidence coefficient exists in the suspected targets, namely the target to be confirmed, the condition that the target to be confirmed is confirmed by a zooming high-definition camera and the panoramic camera fails to operate normally, other equipment normally operates, the zooming high-definition camera or the zooming infrared camera needs to be driven by using a photoelectric turret to rotate for one circle to replace the panoramic camera to work, and most application scenes can be covered;

fourthly, in the invention, the panoramic camera is formed by combining a plurality of cameras, the video output of the panoramic camera is formed by respectively outputting videos by each camera and splicing the videos output by each camera into a panoramic video output two formats after correcting the videos, the original videos output by each camera which are not spliced into a panoramic image are used in the invention, the original resolution of each camera in the panoramic camera is kept, the time consumed during graph splicing is reduced, the problems of inaccurate time matching and distortion correction errors during splicing of the output videos of each camera are avoided, and more accurate detection results can be obtained in subsequent target detection.

Drawings

Fig. 1 is a flow chart of the operation mode selection of the present invention.

Fig. 2 is a flow chart of the mode of operation of the present invention.

FIG. 3 is a flow chart of the second mode of operation of the present invention.

Fig. 4 is a flow chart of the operation mode three of the present invention.

Fig. 5 is a schematic diagram of the offset position in the photoelectric turret anti-shaking process.

Detailed Description

The invention is described in detail below with reference to the figures and specific embodiments.

Example 1

The small-sized photoelectric turntable adopted in the invention is expanded in function. The high-definition camera is replaced by the zooming high-definition camera, so that the detection range of the small photoelectric turntable is expanded; the zoom infrared camera is added on the photoelectric turret, so that the photoelectric turret can be used for replacing a zoom high-definition camera under the condition of low visibility, and applicable scenes are increased; a laser range finder is added on the photoelectric turret, and can detect the distance of a specified target. The 0 degree of the photoelectric turret is consistent with the 0 degree of the panoramic camera, the photoelectric turret can rotate by Nx 360 degrees upwards in azimuth, the maximum pitching angle can reach +/-90 degrees, the centers of the zooming high-definition camera, the zooming infrared camera and the laser range finder which are carried on the photoelectric turret are unified, and the pointing angles are the azimuth angle and the pitch angle of the photoelectric turret. The azimuth angle and the pitch angle of the photoelectric turret, the focal length and the field angle of the zooming high-definition camera, and the focal length and the field angle of the zooming infrared camera can be acquired in real time.

The invention relates to an intelligent control method of a small and medium-sized photoelectric turntable for target detection, which mainly relates to the intelligent control of the small photoelectric turntable, wherein the small photoelectric turntable is provided with a panoramic camera and a photoelectric turret integrating a zooming high-definition camera, a zooming infrared camera and a laser range finder, the centers of the zooming high-definition camera, the zooming infrared camera and the laser range finder are consistent with the direction of the photoelectric turret, and a user needs to control the movement of the small photoelectric turntable through a matched small photoelectric turntable interface.

The method comprises the following steps:

according to the difference of the running condition and the application scene of the small-sized photoelectric turntable equipment, whether the equipment normally works or not and the application condition under different weather conditions are comprehensively considered, referring to fig. 1, the intelligent control method for the target detection of the small-sized and medium-sized photoelectric turntable is divided into the following three working modes:

(1) the first working mode is as follows: all small-size photoelectricity revolving stage equipment are in the state of normal work, and the weather condition is good simultaneously, and weather visibility is high, can use the optical camera that is more clear under the good condition of weather condition, and panoramic camera and zoom high definition video camera promptly. The method includes the steps that target detection is carried out on a panoramic camera video to obtain a plurality of suspected targets, the plurality of suspected targets are sequentially confirmed by a zooming high-definition camera, namely, a photoelectric turret sequentially aligns to each suspected target according to the probability from large to small, and the method specifically comprises the following steps of:

1.1 target information fusion: the panoramic camera in the small photoelectric turntable is always in a working state of target detection, and after target detection is carried out through videos output by the panoramic camera, if a target detection result is a plurality of suspected targets and one target with extremely high confidence coefficient exists in the suspected targets, a working mode II is switched to; if the target detection result is only a plurality of suspected targets, the confidence degrees of the suspected targets are not greatly different, and no target with extremely high confidence degree exists, the working mode I is entered, and the movement control variable of the photoelectric turret is set to be '1'; and storing the panoramic video target detection result in a cache, and when the photoelectric turret motion control variable is 1, continuously reading the N _ p frames of panoramic video target detection results from the target detection result cache. The detection results of continuous frame targets are very similar, but in order to avoid false detection or missed detection of target detection, similar target information in N _ p frames needs to be fused, so that the total number N _ s of the current suspected targets and information of each suspected target are obtained, N _ p is the total frame number of the read panoramic video, the selection of N _ p is related to practical application, the target fusion result is inaccurate when N _ p is too small, and the target fusion time is too long when N _ p is too large.

Target information including an azimuth angle and a pitch angle within a range of 360 degrees around the small-sized photoelectric turntable is obtained through target detection of the panoramic camera, similar targets in multi-frame panoramic video target detection results are fused, and the problems that the same target repeatedly appears in multi-frame panoramic videos and errors possibly occur in single-frame panoramic video target detection are solved; the method comprises the steps of obtaining a plurality of suspected targets after target fusion, planning azimuth angles and pitch angles of the photoelectric turret for each suspected target for the first time, confirming each suspected target through a zooming high-definition camera, and performing anti-shaking treatment on the photoelectric turret.

1.2 sorting with confidence as a reference: sorting the N _ s suspected targets from large to small by taking the confidence degrees in the N _ s suspected target information as a reference; and according to the suspected target sorting result, sequentially carrying out azimuth angle and pitch angle planning, camera focal length adjustment, photoelectric turret anti-shaking processing and final target confirmation on the suspected target from big to small.

1.3 for target confirmation, the photoelectric turret azimuth angle and the pitch angle are planned for the current target to be confirmed for the first time: according to the suspected target sorting result in the step 1.2, confirming suspected targets from big to small in sequence; taking the nth target in the suspected target sorting results as an example, if the currently confirmed target is the nth target in the suspected target sorting results, reading the azimuth angle and the pitch angle in the nth suspected target information, taking the read values as the first planning result of the photoelectric turret and generating a movement command of the photoelectric turret, and after receiving the movement command, reading the first planning result and rotating according to the first planned azimuth angle and pitch angle by the photoelectric turret.

1.4 camera focal length adjustment: the zooming high-definition camera is always in a working state of target detection, after the photoelectric turret rotates to an azimuth angle and a pitch angle which are planned for the first time, an N _ v frame zooming high-definition camera target detection result is read, the width and height information of a target with the maximum reliability in the target detection result is obtained, the focal length which needs to be adjusted of the zooming high-definition camera is calculated by combining the focal length and the field angle of the current zooming high-definition camera to adjust the focal length, N _ v is the total video frame number of the read zooming high-definition camera, the selection of N _ v is related to practical application, the processing result is inaccurate when the N _ v is too small, and the processing time is too long when the N _ v is too large.

1.5 photoelectric turret anti-shaking treatment: the small-size photoelectric rotary table is mostly installed on the unmanned aerial vehicle, and the unmanned aerial vehicle is often small in size, light in weight, has more serious shake in the motion process, needs to prevent shaking the photoelectric turret to eliminate the error that the unmanned aerial vehicle shake brought to a certain extent. The specific operation of the photoelectric turret anti-shaking processing is that, referring to fig. 5, taking the firstly planned azimuth angle and pitch angle as the center, namely the position marked by the five-pointed star in fig. 5, four offset positions are generated after the center is offset by an angle of M degrees respectively from top to bottom, left to right, and the four offset positions are used as the result of the photoelectric turret replanning and generating a photoelectric turret motion instruction, and the photoelectric turret rotates according to the replanned azimuth angle and pitch angle after receiving the motion instruction, so that the photoelectric turret anti-shaking processing is completed; namely, adding an offset M DEG on the basis of the firstly planned azimuth angle and the firstly planned pitch angle, and generating a replanned azimuth angle and a replanned pitch angle as the offset position of the current center.

1.6 target confirmation based on the target detection probability of the zooming high-definition camera: after the photoelectric turret rotates to the offset position every time, reading the target detection result of the N _ v frame zooming high-definition camera, taking the central position and the probability value with the maximum target confidence coefficient in the target detection results of all the offset positions with the center as the reference as the detection probability of the current target, and recording the corresponding target information such as the azimuth angle, the pitch angle and the like.

1.7 repeat the above steps 1.3-1.6 for the remaining N _ s-1 targets.

The working mode of the invention is to detect and confirm a plurality of suspected targets, especially to the condition that the confidence degrees of the plurality of suspected targets are close.

(2) And a second working mode: the small photoelectric rotary table equipment runs normally, weather conditions are good, weather visibility is high, a plurality of suspected targets are obtained through the panoramic camera, one target with high confidence coefficient exists in the suspected targets, namely the target to be confirmed is obtained, the target to be confirmed needs to be confirmed through the zooming high-definition camera, detection result information of the target to be confirmed is recorded, a second working mode is entered, and a movement control variable of the photoelectric rotary tower is set to be 2; when the movement control variable of the photoelectric turret is '2', the azimuth angle and the pitch angle in the detection result information of the target to be confirmed are taken, and the first mode is similar to the working mode: planning the azimuth angle and the pitch angle of the photoelectric turret for the current target to be confirmed for the first time; adjusting the focal length of the camera; performing anti-shaking treatment on the photoelectric turret; target confirmation and target processing result output based on the target detection probability of the zooming high-definition camera; the method is different from the working mode I in that laser ranging is added between target confirmation and target processing result output based on the target detection probability of the zooming high-definition camera, if the detection probability of the current target exceeds a set laser ranging threshold, a laser ranging machine is started to carry out distance measurement on the current target, and the obtained laser ranging result information is added into the detection result of the current target.

The second working mode of the invention is mainly to detect and confirm one of a plurality of suspected targets, especially to the condition that one target with extremely high confidence coefficient exists in the plurality of suspected targets.

(3) And a third working mode: if the panoramic camera fails and cannot normally operate, and other equipment normally operates, the photoelectric turret is used for driving the zooming high-definition video camera or the zooming infrared video camera to rotate for one circle to replace the panoramic camera to work, referring to fig. 4, if the visibility of the current weather is low, the imaging effect of the optical camera is not good as that of the infrared camera, and the photoelectric turret is directly used for driving the zooming infrared video camera to rotate for one circle to replace the panoramic camera to work; if current weather visibility is better, and the optical camera imaging effect is better, then use the rotatory a week of photoelectricity capstan head drive zoom high definition video camera to replace panoramic camera work, see fig. 4, specifically include following flow:

3.1 replace the panoramic camera with a zoom high definition video camera: the zooming high-definition camera is always in a working state of target detection, the photoelectric turret is calculated according to the upward maximum field angle of the zooming high-definition camera in the azimuth direction and needs to rotate for R _ v times for one circle, similar target information in the results of the R _ v times of target detection is fused, and the fused target information is recorded and output for other equipment or users to use in the subsequent processing process.

The third working mode of the invention is mainly aimed at the situation that when the panoramic camera is in failure or has low weather visibility and is not suitable for using the optical camera, the photoelectric turret is required to drive the zooming high-definition video camera or the zooming infrared video camera to rotate for one circle to replace the panoramic camera to work for object census.

In the invention, control instructions of the small photoelectric turntable are all generated in real time by the intelligent control method, and comprise a photoelectric turret rotation instruction, a laser range finder starting instruction, a laser range finder closing instruction, a zooming high-definition camera focal length adding instruction, a zooming high-definition camera focal length reducing instruction, a zooming infrared camera focal length adding instruction and a zooming high-definition camera focal length reducing instruction. According to the intelligent control method, parameters required for controlling the movement of the small photoelectric turntable are obtained and control instructions are generated by fusing and analyzing target detection result information of the panoramic camera and the zooming high-definition camera or target detection result information of the zooming infrared camera, data are not required to be downloaded to a control console and manual intervention is not required, and time delay caused by data transmission and manual operation is reduced.

The technical idea of the invention is as follows:

in different application scenes, the use modes of the camera are different, and the intelligent control method of the small photoelectric turntable is designed under three application scenes by integrating the working state of equipment and the weather condition.

A panoramic camera in the small photoelectric turntable is formed by combining a plurality of cameras, and has two video output modes, wherein one mode is to output images of a plurality of paths of cameras after the images are spliced into panoramic images, and the other mode is to output the panoramic images by the plurality of paths of cameras respectively. In the invention, the output result of each camera is independently processed by adopting a mode of respectively outputting multiple cameras, so that errors and reduction of image resolution caused by a correction algorithm in an image splicing process are avoided. As the single camera in the panoramic camera has a larger field angle and a wider field of view relative to the high-definition camera, the result of the panoramic camera after the video is subjected to target detection is used as target general survey, redundant target information is removed through later-stage target fusion, a target needing fixed-point detection is determined through target screening, and the target is detected by the camera on the photoelectric turret. The result information of the target detection comprises the camera number of the panoramic camera, and the position of each camera in the panoramic camera is fixed, so that the azimuth angle and the pitch angle information of the target relative to the photoelectric turret can be determined according to the position of the target in an image and the camera number, and the information is the azimuth angle and the pitch angle required by the rotation of the photoelectric turret; a focal length value can be calculated by combining the width and the height of a target on an image in the target detection result information with the current camera focal length parameter, the camera focal length is adjusted to the focal length value, and the target can be amplified or reduced to an expected size so as to obtain a more accurate target detection result.

Example 2

The intelligent control method of the target detection small and medium-sized photoelectric turntable is the same as the embodiment 1, and the working mode II of the invention is shown in figure 3, and specifically comprises the following procedures:

2.1 planning azimuth angle and pitch angle of the photoelectric turret for the first time: the panoramic camera is always in a working state of target detection, after target detection is carried out on videos output by the panoramic camera, similar target information in a target detection result is fused to obtain a plurality of suspected targets, if one target with extremely high confidence coefficient exists in the plurality of suspected targets, the target is the target to be confirmed, the detection result information of the target to be confirmed is recorded, a working mode II is entered, and the photoelectric turret motion control variable is set to be 2; and when the movement control variable of the photoelectric turret is '2', reading the azimuth angle and the pitch angle in the detection result information of the target to be confirmed to plan the azimuth angle and the pitch angle of the photoelectric turret for the first time, and rotating the photoelectric turret after receiving the first planning information.

2.2 camera focal length adjustment: the zooming high-definition camera is always in a working state of target detection, after the photoelectric turret rotates to an azimuth angle and a pitch angle which are planned for the first time, an N _ v frame zooming high-definition camera target detection result is read, the width and height information of a target with the maximum reliability in the target detection result is obtained, the focal length of the zooming high-definition camera which needs to be adjusted is calculated by combining the focal length and the field angle of the current zooming high-definition camera, and the focal length is adjusted, wherein N _ v is the total number of the read zooming high-definition camera video frames.

2.3 photoelectric turret anti-shaking treatment: and taking the azimuth angle and the pitch angle which are planned for the first time as the center, generating four offset positions after the center is offset by an angle M degrees respectively from the upper side, the lower side, the left side and the right side, and taking the four offset positions as a result of replanning of the photoelectric turret, rotating the photoelectric turret after receiving the planning information as a result of replanning the photoelectric turret and generating a movement instruction of the photoelectric turret, and rotating the photoelectric turret according to the replanning azimuth angle and the pitch angle after receiving the movement instruction to complete the anti-shaking treatment of the photoelectric turret.

2.4 target confirmation based on the target detection probability of the zooming high-definition camera: and after the photoelectric turret rotates to a planned offset position every time, reading the target detection result of the N _ v frame zooming high-definition camera, taking the central position and the probability value with the maximum target confidence coefficient in the target detection results of all offset positions taking the center as the reference as the detection probability of the current target, recording corresponding target detection result information comprising an azimuth angle and a pitch angle as the detection result information of the target to be confirmed, and finishing the confirmation of the current target.

2.5 laser ranging: if the detection probability of the target exceeds the set laser ranging threshold value, generating a laser ranging starting signal, starting a laser ranging machine to measure the distance of the target, rotating the photoelectric turret to the azimuth angle and the pitch angle of the target which is obtained in the step 2.4 and completes target confirmation, reading laser ranging results for multiple times, taking the average value of the ranging results as the distance of the target relative to the small photoelectric turntable, and adding the distance information into the detection result information of the current target; and after the laser ranging is finished, sending an instruction to stop the work of the laser ranging machine.

2.6 outputting the target processing result: and outputting the detection result information of the target to be confirmed and the confidence degree of the target in the panoramic video for other equipment or users to use in the subsequent processing process.

The second working mode of the invention processes the target with extremely high confidence degree preferentially, the extremely high confidence degree represents that the target has extremely high possibility as the final target, and the target is confirmed preferentially, so that the target can be confirmed rapidly in most cases, and the time delay caused by processing other multiple targets is reduced.

Example 3

The intelligent control method of the target detection small and medium-sized photoelectric turntable is the same as that of the embodiment 1-2, and the detailed information of the target detection results of the panoramic camera, the zooming high-definition camera and the zooming infrared camera specifically comprises the following steps: the target detection result information comprises horizontal and vertical coordinates of a target center point on the image, the width and the height of a target identification frame, the confidence coefficient of the target, a target classification number, an azimuth angle and a pitch angle of the target relative to the small photoelectric rotary table and the distance of the target relative to the small photoelectric rotary table.

The azimuth angle and the pitch angle of the target relative to the small photoelectric turntable in the target detection result information are obtained through the proportional calculation of the horizontal and vertical coordinates of the target center point on the image and the image, and are used for subsequently planning the azimuth angle and the pitch angle of the photoelectric turret, and the subsequent planning is the motion control of the photoelectric turret in an intelligent control method; the width and the height of the target identification frame are used for subsequently calculating focal lengths of the zooming high-definition camera and the zooming infrared camera which need to be adjusted; the target confidence and the classification number are used for determining the target type and probability; the distance between the target and the small photoelectric turntable is used when the laser range finder is started, and the distance measurement result is filled in the target detection result.

Example 4

The intelligent control method of the target detection small and medium-sized photoelectric turntable is the same as that of the embodiment 1-3, and similar target information in the target detection result is fused in the steps 1.1, 2.1, 3.1 and 3.2, and the method specifically comprises the following steps:

(A) judging whether target fusion is needed according to the target classification number: taking the target detection result of the 1 st frame as a target reference table, sequentially comparing targets detected in other frames with each target in the 1 st frame, if the classification numbers of the current target and each target in the 1 st frame are not consistent, not needing to perform target fusion, and adding the target information into the target reference table; and if the target classification numbers are consistent in the comparison process, performing the next judgment.

(B) Judging whether target fusion is needed according to the difference value of the azimuth angle and the pitch angle: setting an azimuth angle error threshold T _ a and a pitch angle error threshold T _ p, respectively subtracting the azimuth angle and the pitch angle of two currently compared targets, and if the difference value of the azimuth angle and the pitch angle is respectively smaller than the azimuth angle error threshold T _ a and the pitch angle error threshold T _ p, considering that the two currently compared targets are the same target, and performing next-step target fusion on the two targets; otherwise, the target information is added into the target reference table without target fusion.

(C) Target fusion: and taking the average value of the values with the same parameter meaning in the detection result information which is currently judged as the same target as the final detection result of the fused target.

(D) And (3) outputting a target fusion result: after the fusion processing of all similar targets in step 1.1 or step 2.1 or step 3.1 or step 3.2 is completed, the total number of suspected targets and target detection result information corresponding to each suspected target are output for use by other devices or users in the subsequent processing process.

The target fusion of the invention is the premise and the basis of planning the azimuth angle and the pitch angle of the photoelectric turret, can effectively avoid the problems of false detection, missed detection and error of target detection of a single-frame image, simultaneously avoid the problem of multiple confirmation of the same target when processing a plurality of frames of images, improve the processing precision and reduce the possible redundant operation and time delay.

Example 5

The intelligent control method of the target detection small and medium-sized photoelectric turntable is the same as the camera focal length adjustment in the embodiments 1-4, the step 1.4 and the step 2.2, and specifically comprises the following steps: the method comprises the following steps that after receiving planning information, a photoelectric turret rotates, after the photoelectric turret rotates to a specified azimuth angle and a pitch angle, a target detection result of an N _ v frame zooming high-definition camera is read, identification frame width and height information of a target with the maximum confidence degree in the detection result is obtained, and the focal length of the zooming high-definition camera to be adjusted is calculated by combining the focal length and the field angle of the current zooming high-definition camera to adjust the focal length, wherein the specific calculation method comprises the following steps:

(a) the size width x height of the target on the image is w x h, and the width x height needs to be adjusted to be w '× h', namely, the wide magnification

High magnification

Doubling; to prevent the object from exceeding the image area after enlargement, the image area is taken

And

medium and small values as the magnification of the target, assuming

Smaller, i.e. the target needs to be enlarged

And (4) doubling.

(b) According to the relation between the size of the camera target and the field angle, the real height of the target before the focal length adjustment can be calculated to be height _ before, and the real height of the target after the focal length adjustment, height _ after, is equivalent to the real height of the target to be amplified

And (4) doubling.

(c) The relationship between the target size and the field angle can be approximately considered as follows: the target is magnified by X times, the field angle becomes 1/X of the original size, and the field angle can be read in real time.

(d) According to the corresponding relation between the focal length of the camera and the field angle, the focal length of the camera can be adjusted by adjusting the field angle.

If the focal length of the zooming infrared camera needs to be adjusted, the adjusting method is the same as the above.

Example 6

The intelligent control method for the target detection small and medium-sized photoelectric turntable is the same as that in the embodiment 1-5, namely the working mode III is that if the panoramic camera fails to work normally and other equipment runs normally in the step (3), the photoelectric turret is used for driving the zooming high-definition camera or the zooming infrared camera to rotate for one circle to replace the panoramic camera to work, if the visibility of the current weather is low, the imaging effect of the optical camera is not good as that of the infrared camera, and the photoelectric turret is directly used for driving the zooming infrared camera to rotate for one circle to replace the panoramic camera to work; if the visibility of the current weather is good and the imaging effect of the optical camera is good, the photoelectric turret is used for driving the zooming high-definition video camera to rotate for a circle to replace the panoramic camera to work. Particularly, the calculation of the number R _ v or R _ f of times that the photoelectric turret needs to rotate for one circle in the scanning process is related to 3.1 and 3.2, and the method specifically comprises the following steps:

(i) use of a zoom high definition video camera instead of a panoramic camera: the maximum view angle of the zoom high-definition camera in the azimuth direction is A _ v, and the number R _ v of times that the photoelectric turret needs to rotate for scanning a circle can be calculated through the relationship between the view angle and the rotation number of the photoelectric turret.

And fusing similar results of the R _ v times of target detection, and recording fused target information.

(ii) Use of a zoom infrared camera instead of a panoramic camera: the maximum upward angle of view of the zoom infrared camera is A _ f, and the number R _ f of times that the photoelectric turret needs to rotate when scanning for one circle can be calculated through the relationship between the angle of view and the number of times that the photoelectric turret rotates.

And fusing the results of the R _ f times of target detection similarly, and recording fused target information.

The relationship between the angle of view and the number of rotations of the photoelectric turret:

in the relational expression between the Angle of view and the number of times of rotation of the photoelectric turret, R is the number of times the photoelectric turret needs to rotate, and Angle is the maximum Angle of view in the azimuth direction of the camera.

In the invention, all control instructions of the small photoelectric turntable are generated in real time by an intelligent control method, the intelligent control method obtains parameters required for controlling the movement of the small photoelectric turntable and generates control instructions by fusing and analyzing target detection result information of a panoramic camera and a zooming high-definition camera or target detection result information of a zooming infrared camera, data are not required to be downloaded to a control console and manual intervention is not required, and time delay caused by data transmission and manual operation is reduced; target information including an azimuth angle and a pitch angle within a range of 360 degrees around the small-sized photoelectric turntable is obtained through target detection of the panoramic camera, similar targets in multi-frame panoramic video target detection results are fused, and the problems that the same target repeatedly appears in multi-frame panoramic videos and errors possibly appear in single-frame panoramic video target detection are solved; the method comprises the steps of obtaining a plurality of suspected targets after target fusion, planning azimuth angles and pitch angles of the photoelectric turret for each suspected target for the first time, confirming each suspected target through a zooming high-definition camera, and performing anti-shaking treatment on the photoelectric turret.

A more particular example is given below in which the invention is described in further detail with reference to the figures and the detailed description of the preferred embodiments.

Example 7

The intelligent control method of the small and medium-sized photoelectric turntable for target detection is the same as that in the embodiments 1 to 6, most of the small photoelectric turntables are installed on unmanned equipment, and can also be used on manned equipment or other occasions needing target detection and positioning.

Referring to fig. 1-4, the present invention includes the following steps.

Step 1, selecting a working mode, referring to fig. 1: judging whether the weather condition is good or not according to the current application scene, and if the visibility is high and the imaging effect of the optical camera is better than that of the infrared camera, continuously judging whether the panoramic camera is available or not; and if the visibility is low, the zoom infrared camera is used for continuously judging whether the infrared camera is available.

If the panoramic camera runs normally, the panoramic camera is used to continuously judge whether the zooming high-definition video camera is available; if the panoramic camera is in fault and cannot normally operate at the moment, the zooming high-definition camera is used for replacing the panoramic camera to work, and whether the zooming high-definition camera is available or not is judged.

Under the condition that the panoramic camera operates normally, if the zooming high-definition camera is available, the panoramic video is subjected to target detection to obtain a plurality of suspected targets with relatively close confidence coefficients, meanwhile, the targets with extremely high confidence coefficients do not exist, the working mode I is entered, and the step 2 is executed; if one target with extremely high confidence coefficient exists in the plurality of suspected targets, entering a working mode II, and executing a step 3; and if the zooming high-definition camera is unavailable, the equipment is considered to be in failure, and subsequent processing cannot be carried out.

Under the condition that the panoramic camera is in fault, if the zooming high-definition camera is available, the photoelectric turret is used for driving the zooming high-definition camera to rotate for a circle to replace the panoramic camera to work, the working mode III is entered, and the step 4 is executed; if the zooming high-definition camera is not available, driving the zooming infrared camera to rotate for a circle by using the photoelectric turret to replace the panoramic camera to work, entering a working mode III, and executing a step 4; and if the zooming high-definition camera is unavailable and the zooming infrared camera is unavailable, the equipment is considered to be in failure, and subsequent processing cannot be carried out.

The panoramic camera is formed by combining a plurality of cameras, the video output of the panoramic camera is formed by respectively outputting videos by all the cameras and splicing the videos output by all the cameras into two formats of panoramic video output after correcting the videos.

The target detection result information comprises horizontal and vertical coordinates of a target central point on the image, the width and height of a target identification frame, the confidence coefficient of the target, a target classification number, an azimuth angle and a pitch angle of the target relative to the small photoelectric rotary table and the distance of the target relative to the small photoelectric rotary table.

Step 2, a first working mode: a plurality of suspected targets are obtained through panoramic video target detection, and the zooming high-definition camera needs to sequentially confirm the suspected targets. Referring to fig. 2, the specific steps implemented are as follows:

and 2a, reading and caching the detection results of the N _ p frames of panoramic video targets, wherein the same target can be detected for multiple times in a short time due to the high frame rate of the panoramic video, and therefore the cached detection results of the panoramic video targets are fused to obtain the number N _ s of the suspected targets.

The target fusion mode is as follows: taking the target detection result of the 1 st frame as a target reference table, sequentially comparing targets detected in other frames with each target in the 1 st frame, if the classification numbers of the current target and each target in the 1 st frame are not consistent, not needing to perform target fusion, and adding the target information into the target reference table; if the target classification numbers are consistent in the comparison process, carrying out next judgment; setting an azimuth angle error threshold T _ a and a pitch angle error threshold T _ p, respectively subtracting the azimuth angle and the pitch angle of two currently compared targets, and if the difference value of the azimuth angle and the pitch angle is respectively smaller than the azimuth angle error threshold T _ a and the pitch angle error threshold T _ p, considering that the two currently compared targets are the same target, and performing next-step target fusion on the two targets; otherwise, the target information is added into the target reference table without target fusion; and taking the average value of the values with the same parameter meaning in the detection result information which is currently judged as the same target as the final detection result of the fused target to obtain the total number N _ s of the current suspected targets.

And 2b, taking the confidence degrees of the N _ s suspected targets as a reference, and sorting from large to small.

And 2c, according to the suspected target sorting results, sequentially carrying out azimuth angle and pitch angle planning, camera focal length adjustment, anti-shaking processing and final target confirmation on the suspected targets from large to small, taking the Nth target in the suspected target sorting results as an example, if the currently confirmed target is the Nth target in the suspected target sorting results, reading the azimuth angle and pitch angle in the Nth suspected target information, taking the read value as the first planning result of the photoelectric turret and generating a photoelectric turret motion instruction, and after receiving the motion instruction, reading the first planning result and rotating according to the first planned azimuth angle and pitch angle by the photoelectric turret.

Step 2d, the zooming high-definition camera is always in the working state of target detection, after the photoelectric turret rotates to the azimuth angle and the pitch angle which are planned for the first time, the target detection result of the N _ v frame zooming high-definition camera is read, the width and the height information of the target with the maximum reliability in the target detection result are taken, the focal length which needs to be adjusted of the zooming high-definition camera is calculated by combining the focal length and the field angle of the current zooming high-definition camera for focal length adjustment, N _ v is the rotating instruction which is received by the photoelectric turret of the total video frame number of the read zooming high-definition camera and the values of the azimuth angle and the pitch angle, the photoelectric turret starts to rotate after the rotation to the appointed azimuth angle and the pitch angle, the target detection result of the N _ v frame zooming high-definition camera is read, the width and the height information of the mark frame of the target with the maximum reliability in the detection result are taken, and the focal length and the field angle of the current zooming high-definition camera are combined, and calculating the focal length of the zooming high-definition camera to be adjusted to adjust the focal length.

The focal length calculation method comprises the following steps:

the size width x height of the target on the image is w x h, and the width x height needs to be adjusted to be w '× h', namely, the wide magnification

High magnification

And

the smaller value of the medium is used as the targetTarget magnification, suppose

Smaller, i.e. the target needs to be enlarged

Doubling; according to the relation between the focal length of the camera and the field angle, the real height of the target before the focal length adjustment can be calculated to be height _ before, and the real height of the target after the focal length adjustment, height _ after, is equivalent to the real height amplification of the target

Doubling; the relationship between the target size and the field angle can be approximately considered as follows: the target is amplified by X times, the field angle is changed into 1/X of the original size, and the field angle can be read in real time; according to the corresponding table of the focal length and the field angle of the camera, the focal length of the camera can be adjusted by adjusting the field angle.

And 2e, the small photoelectric rotary table is usually installed on unmanned equipment with small volume, such as an unmanned aerial vehicle, an unmanned vehicle and the like, and the unmanned equipment with small volume usually has the characteristic of insufficient stability, so that the small photoelectric rotary table is difficult to accurately point to a target position. In order to reduce the error caused by the above factors and improve the fault tolerance of the turntable, referring to fig. 5, the positions of the center of the azimuth angle and the center of the pitch angle planned in step 2c, which are shifted by M ° from top to bottom, left to right, are all checked. After the photoelectric turret rotates to a specified offset position every time, an N _ v frame zooming high-definition camera target detection result is read, the probability value with the maximum target confidence coefficient in five positions including the upper position, the lower position, the left position, the right position and the center with the azimuth angle and the pitch angle as the centers is recorded as the detection probability of the current target, and information of the corresponding target is recorded.

And 2f, repeating the steps 2c to 2e for the remaining N _ s-1 targets.

And 2g, recording the number of the suspected targets, the detection result information of each suspected target and the confidence degree of the target detection of the target in the panoramic video.

Step 3, a second working mode: a plurality of suspected targets are obtained through panoramic video target detection, one target with extremely high confidence coefficient exists in the suspected targets, and the zooming high-definition camera needs to confirm the specific target, and the method comprises the following specific steps:

and 3a, reading and caching the detection result of the N _ p frames of panoramic video targets, wherein the same target can be detected for multiple times in a short time due to the high frame rate of the panoramic video, so that the cached detection results of the panoramic video targets are fused to obtain a plurality of suspected targets and a target with high confidence coefficient, the target is used as a target to be confirmed, and the target detection result information is recorded.

And 3b, reading the azimuth angle and the pitch angle in the target detection result information to be confirmed to plan the azimuth angle and the pitch angle of the photoelectric turret for the first time, and rotating the photoelectric turret after receiving the first planning information.

And 3c, the zooming high-definition camera is always in a working state of target detection, after the photoelectric turret rotates to the azimuth angle and the pitch angle which are planned for the first time, the target detection result of the zooming high-definition camera with N _ v frames is read, the width and height information of the target with the maximum reliability in the target detection result is obtained, the focal length which needs to be adjusted by the zooming high-definition camera is calculated by combining the focal length and the field angle of the current zooming high-definition camera, and the N _ v is the total video frame number of the read zooming high-definition camera. The focal length calculation method comprises the following steps:

if the size width x height of the image is w x h, the adjustment is made to w '× h', i.e. the wide zoom-in

High magnification

And

medium and small values as the magnification of the target, assuming

Smaller, i.e. the target needs to be enlarged

Doubling; according to the relation between the size of the camera target and the field angle, the real height of the target before the focal length adjustment can be calculated to be height _ before, and the real height of the target after the focal length adjustment, height _ after, is equivalent to the real height of the target to be amplified

And 3d, the small photoelectric rotary table is usually installed on unmanned equipment with small volume, such as an unmanned aerial vehicle, an unmanned vehicle and the like, and the unmanned equipment with small volume usually has the characteristic of insufficient stability, so that the small photoelectric rotary table is difficult to accurately point to a target position. In order to reduce the error caused by the above factors and improve the fault tolerance of the turntable, referring to fig. 5, the positions of the center of the azimuth angle and the center of the pitch angle planned in step 3b, which are shifted by M ° from top to bottom, left to right, are all checked. After the photoelectric turret rotates to a specified offset position every time, reading a target detection result of the N _ s frame zooming high-definition camera, recording the detection probability of the current target, wherein the probability value with the maximum target confidence coefficient is used as the detection probability of the current target in the five positions of the upper position, the lower position, the left position, the right position and the center which take the azimuth angle and the pitch angle as the centers, and recording the information of the corresponding target.

And 3e, after the photoelectric turret rotates to a planned offset position each time, reading the target detection result of the N _ v frame zooming high-definition camera, taking the central position and the probability value with the maximum target confidence coefficient in the target detection results of all offset positions with the center as the reference as the detection probability of the current target, recording corresponding target detection result information including an azimuth angle and a pitch angle as the detection result information of the target to be confirmed, and finishing the confirmation of the current target.

Step 3f, if the detection probability of the target exceeds a set laser ranging threshold value, generating a laser ranging starting signal, starting a laser ranging machine to measure the distance of the target, rotating the photoelectric turret to the azimuth angle and the pitch angle of the target which is confirmed by the target and obtained in the step 3e, reading laser ranging results for multiple times, taking the average value of the ranging results as the distance of the target relative to the small photoelectric turntable, and adding the distance information into the detection result information of the current target; and after the laser ranging is finished, sending an instruction to stop the work of the laser ranging machine.

And 4, when the panoramic camera fails and cannot normally operate, the photoelectric turret is used for driving the zooming high-definition camera or the zooming infrared camera to rotate for one circle to replace the panoramic camera to work. If the current weather condition is good and the visibility is high, the zooming high-definition camera is used, and the step 4a is carried out; if the current weather condition is poor, the visibility is low or the night is in place, the zoom infrared camera is used, and the process goes to step 4 b.

And 4a, calculating to obtain the required rotation R _ v times of the photoelectric turret in one scanning circle according to the relationship between the angle of view and the rotation times of the photoelectric turret, wherein the maximum upward angle of view of the zooming high-definition camera is A _ v. And (5) fusing the results of the R _ v times of target detection similarly, wherein the target fusion method is the same as that in the step 2a, and recording fused target information.

And 4a, calculating to obtain the required rotation R _ f times of the photoelectric turret in one scanning circle according to the relationship between the angle of view and the rotation times of the photoelectric turret, wherein the maximum upward angle of view of the zoom infrared camera is A _ f. And (5) fusing the results of the R _ f times of target detection similarly, wherein the target fusion method is the same as that in the step 2a, and recording the fused target information.

And 5, after the task of the small photoelectric turntable is completed, restoring the small photoelectric turntable to the initial state.

The invention integrates the intelligent control method of the target detection small and medium-sized photoelectric turntable on the small photoelectric turntable, realizes the accurate control of the small photoelectric turntable by combining the target detection through a plurality of tests, and achieves the purposes of intelligent control, high precision and real-time processing.

In short, the invention provides an intelligent control method for a small and medium-sized photoelectric turntable for target detection, which mainly relates to intelligent control of the small photoelectric turntable, and is divided into the following three working modes according to the running condition and application scene of small photoelectric turntable equipment, wherein the working mode is as follows: the small photoelectric turntable equipment runs normally and is high in weather visibility, a plurality of suspected targets are obtained by using the panoramic camera, and the plurality of suspected targets are sequentially confirmed by the zooming high-definition camera; and a second working mode: the small photoelectric turntable equipment runs normally and is high in weather visibility, a plurality of suspected targets are obtained by using the panoramic camera, and one target with extremely high confidence coefficient exists in the suspected targets, namely the target to be confirmed is obtained, and the target to be confirmed needs to be confirmed by the zooming high-definition camera; and a third working mode: the panoramic camera breaks down, unable normal operating, other equipment operation are normal, need use the photoelectric turret to drive zoom high definition video camera or zoom infrared camera and rotate a week and replace panoramic camera work. The intelligent control method for the target detection small and medium photoelectric turntable is faster in response and more accurate in control. The integration of similar targets avoids false detection and missing detection of target detection, photoelectric turret anti-shaking processing and intelligent zooming of a camera, so that the intelligent control method has faster response and more accurate control; the intelligent control method is used for unmanned equipment, and can also be used for manned equipment or fixed positions.

Claims

1. An intelligent control method for a small and medium-sized photoelectric turntable for target detection is used for intelligent control of the small photoelectric turntable, the small photoelectric turntable is provided with a panoramic camera and a photoelectric turret integrated with a zooming high-definition camera, a zooming infrared camera and a laser range finder, the centers of the zooming high-definition camera, the zooming infrared camera and the laser range finder are consistent with the direction of the photoelectric turret, and the small photoelectric turntable is controlled by an interface to move, and the intelligent control method is characterized in that the intelligent control method integrated with a target detection result is integrated on the small photoelectric turntable to perform data acquisition and real-time processing of control of the small photoelectric turntable, and comprises the following steps:

1.1 target information fusion: the panoramic camera in the small photoelectric turntable is always in a working state of target detection, and after the target detection is carried out through the video output by the panoramic camera, the target detection result is a plurality of suspected targets, and when one target with extremely high confidence coefficient exists in the suspected targets, the working mode II is switched; when the target detection result is only a plurality of suspected targets, the working mode I is entered, and the movement control variable of the photoelectric turret is set to be '1'; when the photoelectric turret motion control variable is '1', continuously reading N _ p frames of panoramic video target detection results from the target detection results, fusing similar target information in the N _ p frames to obtain the total number N _ s of the current suspected targets and information of each suspected target, wherein N _ p is the total number of the read panoramic video frames;

1.6 target confirmation based on the target detection probability of the zooming high-definition camera: after the photoelectric turret rotates to an offset position every time, reading an N _ v frame zooming high-definition camera target detection result, taking a central position and a probability value with the maximum target confidence coefficient in target detection results of all offset positions with the center as a reference as the detection probability of a current target, and recording corresponding azimuth angle and pitch angle target information;

1.7 repeating the above steps 1.3-1.6 for the remaining N _ s-1 targets;

1.8 outputting the target processing result: outputting the number of suspected targets, the detection result information of each suspected target and the confidence degree of the target detection in the panoramic video for other equipment or users in the subsequent processing process;

(2) and a second working mode: the small photoelectric rotary table equipment runs normally and is high in weather visibility, a plurality of suspected targets are obtained by using a panoramic camera, one target with high confidence coefficient exists in the suspected targets, namely the target to be confirmed is obtained, a high-definition camera needs to be zoomed to confirm the target to be confirmed, the detection result information of the target to be confirmed is recorded, a second working mode is entered, and the movement control variable of the photoelectric rotary tower is set to be 2; when the movement control variable of the photoelectric turret is '2', the azimuth angle and the pitch angle in the detection result information of the target to be confirmed are taken, and the azimuth angle and the pitch angle of the photoelectric turret are planned for the current target to be confirmed for the first time; adjusting the focal length of the camera; performing anti-shaking treatment on the photoelectric turret; target confirmation and target processing result output based on the target detection probability of the zooming high-definition camera; increasing laser ranging between target confirmation and target processing result output based on the target detection probability of the zooming high-definition camera, and when the detection probability of the current target exceeds a set laser ranging threshold, starting a laser ranging machine to measure the distance of the current target, and adding the obtained laser ranging result information into the detection result of the current target;

(3) and a third working mode: the panoramic camera breaks down, unable normal operating, other equipment operations except that the panoramic camera is normal, specifically includes following flow:

3.1 Camera selection: the visibility of the current weather is poor, the imaging effect of an optical camera is not as good as that of an infrared camera, a photoelectric turret is directly used for driving a zooming infrared camera to rotate for a circle to replace a panoramic camera to work, and the step 3.3 is executed; when the visibility of the current weather is good, and the imaging effect of the optical camera is good, the photoelectric turret is used for driving the zooming high-definition video camera to rotate for one circle to replace the panoramic camera to work, and the step 3.2 is executed;

3.2 replace the panoramic camera with a zoom high definition video camera: the zooming high-definition camera is always in a working state of target detection, the photoelectric turret is calculated according to the upward maximum field angle of the zooming high-definition camera in the azimuth direction and needs to rotate for R _ v times for one circle, similar target information in the results of the R _ v times of target detection is fused, and the fused target information is recorded and output for other equipment or users to use in the subsequent processing process;

3.3 use zoom infrared camera instead of panoramic camera: and the zooming infrared camera is always in a working state of target detection, the photoelectric turret is calculated to rotate R _ f times for one circle according to the upward maximum field angle of the zooming infrared camera, similar target information in the results of the R _ f times of target detection is fused, and the fused target information is recorded and output for other equipment or users in the subsequent processing process.

2. The intelligent control method of the small and medium size photoelectric turntable for target detection according to claim 1, characterized in that the second working mode specifically comprises the following procedures:

2.1 planning azimuth angle and pitch angle of the photoelectric turret for the first time: the panoramic camera is always in a working state of target detection, after target detection is carried out on videos output by the panoramic camera, similar target information in target detection results is fused to obtain a plurality of suspected targets and a target with extremely high confidence coefficient, the target is a target to be confirmed, the detection result information of the target to be confirmed is recorded, a working mode II is entered, and the movement control variable of the photoelectric turret is set to be 2; when the movement control variable of the photoelectric turret is '2', reading an azimuth angle and a pitch angle in the detection result information of the target to be confirmed to plan the azimuth angle and the pitch angle of the photoelectric turret for the first time, and rotating the photoelectric turret after receiving the first planning information;

2.2 camera focal length adjustment: the zooming high-definition camera is always in a working state of target detection, after the photoelectric turret rotates to an azimuth angle and a pitch angle which are planned for the first time, an N _ v frame zooming high-definition camera target detection result is read, the width and height information of a target with the maximum reliability in the target detection result is taken, the focal length of the zooming high-definition camera which needs to be adjusted is calculated by combining the focal length and the field angle of the current zooming high-definition camera, and the focal length is adjusted, wherein N _ v is the total number of the read zooming high-definition camera video frames;

2.3 photoelectric turret anti-shaking treatment: taking the azimuth angle and the pitch angle which are planned for the first time as the center, generating four offset positions after the center is offset by an angle M degrees respectively from the upper side, the lower side, the left side and the right side, and taking the four offset positions as a result of replanning of the photoelectric turret;

2.4 target confirmation based on the target detection probability of the zooming high-definition camera: after the photoelectric turret rotates to a planned offset position every time, reading a target detection result of the N _ v frame zooming high-definition camera, taking a central position and a probability value with the maximum target confidence coefficient in target detection results of all offset positions taking the center as a reference as the detection probability of a current target, recording corresponding target detection result information including an azimuth angle and a pitch angle as the detection result information of the target to be confirmed, and finishing the confirmation of the current target;

2.5 laser ranging: if the detection probability of the target exceeds a set laser ranging threshold value, generating a laser ranging starting signal, starting a laser ranging machine to measure the distance of the target, rotating the photoelectric turret to the azimuth angle and the pitch angle of the target which is confirmed by the target, reading the laser ranging results for multiple times, taking the average value of the ranging results as the distance of the target relative to the small photoelectric turntable, and adding the distance information into the detection result information of the current target; after the laser ranging is finished, sending an instruction to stop the work of the laser ranging machine;

3. The intelligent control method for the small and medium size photoelectric turntable for target detection according to claim 1, wherein detailed information of target detection results of the panoramic camera, the zooming high definition video camera and the zooming infrared video camera specifically comprises: the target detection result information comprises horizontal and vertical coordinates of a target center point on the image, the width and the height of a target identification frame, the confidence coefficient of the target, a target classification number, an azimuth angle and a pitch angle of the target relative to the small-sized broadcasting and television rotary table and the distance of the target relative to the small-sized photoelectric rotary table.

4. The intelligent control method for the small and medium size photoelectric turntable for target detection according to claim 1, wherein similar target information in the target detection result in step 1.1, step 2.1, step 3.2 and step 3.3 is fused, and the method specifically comprises the following steps:

(A) judging whether target fusion is needed according to the target classification number: taking the target detection result of the 1 st frame as a target reference table, sequentially comparing targets detected in other frames with each target in the 1 st frame, and if the classification numbers of the current target and each target in the 1 st frame are inconsistent, adding the target information into the target reference table without target fusion; in the comparison process, the target classification numbers are consistent, and the next judgment is carried out;

(B) judging whether target fusion is needed according to the difference value of the azimuth angle and the pitch angle: setting an azimuth angle error threshold T _ a and a pitch angle error threshold T _ p, respectively subtracting the azimuth angle and the pitch angle of two currently compared targets, and when the difference value of the azimuth angle and the pitch angle is respectively smaller than the azimuth angle error threshold T _ a and the pitch angle error threshold T _ p, considering the two currently compared targets as the same target, and performing next-step target fusion on the two targets; otherwise, the target information is added into the target reference table without target fusion;

(C) the target fusion method comprises the following steps: taking the average value of the values with the same parameter meaning in the detection result information which is judged as the same target at present as the final detection result of the fused target;

5. The intelligent control method for the small and medium size photoelectric turntable for target detection according to claim 1, wherein the camera focal length adjustment in step 1.4 and step 2.2 specifically comprises: the method comprises the following steps that after receiving planning information, a photoelectric turret rotates, after the photoelectric turret rotates to a specified azimuth angle and a pitch angle, a target detection result of an N _ v frame zooming high-definition camera is read, identification frame width and height information of a target with the maximum confidence degree in the detection result is obtained, and the focal length of the zooming high-definition camera to be adjusted is calculated by combining the focal length and the field angle of the current zooming high-definition camera to adjust the focal length, wherein the specific calculation method comprises the following steps:

High magnification

And

the middle and smaller value is used as the magnification of the target, when the magnification is wide

Magnification less than high magnification

Time of day, target magnification

Doubling;

(b) calculating the real height of the target before the focal length adjustment to be height _ before according to the relation between the focal length of the camera and the field angle, and enlarging the real height of the target after the focal length adjustment to be equivalent to the real height of the target

Doubling;

(c) the relationship between the size of the target and the angle of view is: amplifying the target by X times, changing the field angle to 1/X of the original size, and reading the field angle in real time;

(d) according to the corresponding table of the focal length and the field angle of the camera, the focal length of the camera can be adjusted by adjusting the field angle;

when the focal length of the zooming infrared camera needs to be adjusted, the adjusting method is the same as the above.

6. The intelligent control method for the small and medium size photoelectric turntable for target detection according to claim 1, 2 or 3, wherein when the panoramic camera fails in step (3), the photoelectric turret is required to drive the zoom high definition video camera or the zoom infrared video camera to work instead of the panoramic camera, and the method specifically comprises the following steps: when the visibility of the current weather is high, a zooming high-definition camera is used; when the visibility of the current weather is low or the current weather is at night, the zooming infrared camera is used; the photoelectric turret is used for driving the zooming high-definition camera or the zooming infrared camera to rotate for a circle to replace the panoramic camera to work;

(i) use of a zoom high definition video camera instead of a panoramic camera: the maximum field angle of the zoom high-definition camera in the azimuth direction is A _ v, and the number R _ v of the rotation required by the scanning of the photoelectric turret for one circle is obtained by calculating the relation between the field angle and the number of the rotation of the photoelectric turret;

fusing similar results of the R _ v times of target detection, and recording fused target information;

(ii) use of a zoom infrared camera instead of a panoramic camera: the maximum upward field angle of the zoom infrared camera is A _ f, and the number R _ f of the rotation required by the photoelectric turret scanning for one circle is obtained by calculating the relation between the field angle and the rotation number of the photoelectric turret;

fusing the results of the R _ f times of target detection similarly, and recording fused target information;