CN115994872A

CN115994872A - Smooth adjustment method and device for objects around aircraft

Info

Publication number: CN115994872A
Application number: CN202310169666.9A
Authority: CN
Inventors: 温建伟; 邓迪旻; 李营; 袁潮
Original assignee: Beijing Zhuohe Technology Co Ltd
Current assignee: Beijing Zhuohe Technology Co Ltd
Priority date: 2023-02-14
Filing date: 2023-02-14
Publication date: 2023-04-21

Abstract

The invention discloses a smooth adjustment method and device for objects around an aircraft. Wherein the method comprises the following steps: acquiring an original image set in a camera array; extracting the original image set into boundary frame data by a background difference method; extracting the coordinate parameters of the boundary frame, and converting the coordinate parameters of the boundary frame into global coordinate parameters; and generating a regulating result according to the global coordinate parameters. The invention solves the technical problems that in the prior art, flying objects such as flying tracks of birds are usually obtained through a high-precision camera array by utilizing an intelligent means so as to avoid flying accidents, birds belong to small targets, flying heights are not high, radar clutter is large, false recognition probability is high, an airplane airport airspace is large, the coverage range of traditional optical equipment is small, and flying birds are difficult to recognize from images based on a target detection method.

Description

Smooth adjustment method and device for objects around aircraft

Technical Field

The invention relates to the field of image processing, in particular to a method and a device for smoothly adjusting objects around an airplane.

Background

Along with the continuous development of intelligent science and technology, intelligent equipment is increasingly used in life, work and study of people, and the quality of life of people is improved and the learning and working efficiency of people is increased by using intelligent science and technology means.

At present, in the field of aircraft flight monitoring, an intelligent means is utilized to obtain the flight track of a flying object, such as birds, through a high-precision camera array so as to avoid the occurrence of flight accidents, the birds belong to small targets, the flying height is not high, the number of radar clutter is high, the false recognition probability is high, the aircraft airport airspace is large, the coverage range of traditional optical equipment is small, and the flying birds are difficult to recognize from images based on a target detection method. The invention can monitor a larger range by using the array camera, and realize quick discovery and real-time tracking of the flying bird by using measures such as dynamic target detection and tracking, flying bird track characteristic judgment and the like.

In view of the above problems, no effective solution has been proposed at present.

Disclosure of Invention

The embodiment of the invention provides a smooth adjustment method and device for objects around an airplane, which at least solve the technical problems that in the prior art, flying objects such as birds are obtained through a high-precision camera array by utilizing an intelligent means so as to avoid flying accidents, the birds belong to small targets, the flying heights are not high, radar clutter is large, the false recognition probability is high, the airport airspace of the airplane is large, the coverage range of traditional optical equipment is small, and the bird is difficult to recognize from images based on the target detection method.

According to an aspect of the embodiment of the invention, there is provided a method for smoothly calibrating objects around an aircraft, including: acquiring an original image set in a camera array; extracting the original image set into boundary frame data by a background difference method; extracting the coordinate parameters of the boundary frame, and converting the coordinate parameters of the boundary frame into global coordinate parameters; and generating a regulating result according to the global coordinate parameters.

Optionally, the extracting the original image set as bounding box data by a background difference method includes: generating a background difference operator according to the smooth demand parameters; and extracting the boundary box data in the original image set according to the background difference method operator.

Optionally, the extracting the coordinate parameters of the bounding box and converting the coordinate parameters of the bounding box into global coordinate parameters includes: inputting the boundary frame data into a coordinate extraction model to obtain the boundary frame coordinate parameters; and converting the coordinate parameters of the boundary frame into the global coordinate parameters through a coordinate conversion matrix.

Optionally, the generating the adjustment result according to the global coordinate parameter includes: generating target track data according to the global coordinate parameters; generating the adjustment result according to the target track data, wherein the adjustment result comprises: and (5) confirming a result and a track calibration result.

According to another aspect of the embodiment of the present invention, there is also provided a device for smoothly adjusting objects around an aircraft, including: the acquisition module is used for acquiring an original image set in the camera array; the extraction module is used for extracting the original image set into boundary frame data through a background difference method; the conversion module is used for extracting the coordinate parameters of the boundary frame and converting the coordinate parameters of the boundary frame into global coordinate parameters; and the generating module is used for generating a regulating result according to the global coordinate parameters.

Optionally, the extracting the original image set as bounding box data by a background difference method includes: the generating unit is used for generating a background difference operator according to the smooth demand parameters; and the extraction unit is used for extracting the boundary frame data in the original image set according to the background difference operator.

Optionally, the conversion module includes: the extraction unit is used for inputting the boundary frame data into a coordinate extraction model to obtain the boundary frame coordinate parameters; and the conversion unit is used for converting the coordinate parameters of the boundary frame into the global coordinate parameters through a coordinate conversion matrix.

Optionally, the generating module includes: the generation unit is used for generating target track data according to the global coordinate parameters; and a result unit, configured to generate the adjustment result according to the target track data, where the adjustment result includes: and (5) confirming a result and a track calibration result.

According to another aspect of the embodiment of the present invention, there is further provided a nonvolatile storage medium, where the nonvolatile storage medium includes a stored program, and when the program runs, the device where the nonvolatile storage medium is controlled to execute a method for smoothly adjusting an object around an aircraft.

According to another aspect of the embodiment of the present invention, there is also provided an electronic device including a processor and a memory; the memory stores computer readable instructions, and the processor is configured to execute the computer readable instructions, where the computer readable instructions execute a method for smooth adjustment of objects around an aircraft.

In the embodiment of the invention, an original image set in a camera array is acquired; extracting the original image set into boundary frame data by a background difference method; extracting the coordinate parameters of the boundary frame, and converting the coordinate parameters of the boundary frame into global coordinate parameters; according to the global coordinate parameters, the method for generating the adjustment result solves the technical problems that in the prior art, flying objects such as birds are obtained through a high-precision camera array by utilizing an intelligent means so as to avoid flying accidents, birds belong to small targets, the flying heights are not high, radar clutter is large, the false recognition probability is high, the airplane airport airspace is large, the coverage range of traditional optical equipment is small, and the target detection-based method is difficult to recognize the birds from the images.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the invention and do not constitute a limitation on the invention. In the drawings:

FIG. 1 is a flow chart of a method for smooth alignment of objects around an aircraft in accordance with an embodiment of the invention;

FIG. 2 is a block diagram of a device for smooth alignment of objects around an aircraft in accordance with an embodiment of the present invention;

fig. 3 is a block diagram of a terminal device for performing the method according to the invention according to an embodiment of the invention;

fig. 4 is a memory unit for holding or carrying program code for implementing a method according to the invention, according to an embodiment of the invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

According to an embodiment of the present invention, there is provided a method embodiment of a method for smooth adjustment of objects around an aircraft, it being noted that the steps shown in the flowchart of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical sequence is shown in the flowchart, in some cases the steps shown or described may be performed in a different order than what is shown or described herein.

Example 1

Fig. 1 is a flowchart of a method for smooth adjustment of objects around an aircraft according to an embodiment of the invention, as shown in fig. 1, the method includes the steps of:

step S102, acquiring an original image set in the camera array.

And step S104, extracting the original image set into boundary frame data through a background difference method.

And S106, extracting the coordinate parameters of the boundary frame, and converting the coordinate parameters of the boundary frame into global coordinate parameters.

And S108, generating a tuning result according to the global coordinate parameters.

Specifically, in order to solve the technical problems that in the prior art, flying objects, such as flying tracks of birds, are generally obtained through a high-precision camera array by utilizing an intelligent means so as to avoid flying accidents, the birds belong to small targets, the flying heights are not high, radar clutter is relatively large, the probability of misidentification is large, the airport airspace of an airplane is large, the coverage range of traditional optical equipment is small, and the flying birds are difficult to identify from images based on a target detection method, the method is implemented by adopting the following steps:

firstly, performing dynamic target detection on the image of each lens of the array camera by using a background difference method to obtain a bounding box (bounding box) of a moving target contained in each frame of image. In this step, since there is no feature judgment, all targets causing image change are identified, including other noise data such as flying birds, airplanes, and jittered leaves.

And secondly, converting a boundary frame of the moving object from a local coordinate system where a single lens is positioned to a global coordinate system corresponding to the spliced image, and tracking the object through an algorithm similar to DeepSort to obtain a track of each object.

And thirdly, judging whether the target is birds or not through the motion and track characteristics. For example, the motion trajectory of the object is divided into a plurality of consecutive time slices according to a set frame number, and the average position (x, y coordinates of the center point), average width and height, average aspect ratio, average time point, and variance of the object position in x and y directions of each time slice are calculated, which is to smooth out noise and errors caused by detection as much as possible.

Then, using the above result as the track point parameter of the target, calculating the speed and acceleration of the target through the moving distance and time information of the adjacent track points, then calculating the average aspect ratio, average speed, average acceleration, accumulated distance in x and y directions and maximum range equivalent to the current time point, analyzing the data by using the calculation result, searching the characteristic parameter capable of distinguishing the flying bird, and judging whether the target belongs to the flying bird or not by using a support vector machine and other methods.

Through the embodiment, the technical problems that in the prior art, flying objects such as flying tracks of birds are obtained through a high-precision camera array by utilizing an intelligent means generally so as to avoid flying accidents, birds belong to small targets, flying heights are not high, radar clutter is large, misidentification probability is high, an airport airspace of an airplane is large, the coverage range of traditional optical equipment is small, and a target detection-based method is difficult to identify the flying birds from images are solved.

Example two

Fig. 2 is a block diagram of a device for smoothly calibrating objects around an aircraft according to an embodiment of the present invention, as shown in fig. 2, the device includes:

an acquisition module 20 is configured to acquire an original image set in the camera array.

The extracting module 22 is configured to extract the original image set as bounding box data by a background difference method.

The conversion module 24 is configured to extract the bounding box coordinate parameters and convert the bounding box coordinate parameters into global coordinate parameters.

The generating module 26 is configured to generate a tuning result according to the global coordinate parameter.

Specifically, the method comprises the following steps: acquiring an original image set in a camera array; extracting the original image set into boundary frame data by a background difference method; extracting the coordinate parameters of the boundary frame, and converting the coordinate parameters of the boundary frame into global coordinate parameters; and generating a regulating result according to the global coordinate parameters. Optionally, the extracting the original image set as bounding box data by a background difference method includes: generating a background difference operator according to the smooth demand parameters; and extracting the boundary box data in the original image set according to the background difference method operator. Optionally, the extracting the coordinate parameters of the bounding box and converting the coordinate parameters of the bounding box into global coordinate parameters includes: inputting the boundary frame data into a coordinate extraction model to obtain the boundary frame coordinate parameters; and converting the coordinate parameters of the boundary frame into the global coordinate parameters through a coordinate conversion matrix. Optionally, the generating the adjustment result according to the global coordinate parameter includes: generating target track data according to the global coordinate parameters; generating the adjustment result according to the target track data, wherein the adjustment result comprises: and (5) confirming a result and a track calibration result.

The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.

In the foregoing embodiments of the present invention, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed technology content may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, fig. 3 is a schematic hardware structure of a terminal device according to an embodiment of the present application. As shown in fig. 3, the terminal device may include an input device 30, a processor 31, an output device 32, a memory 33, and at least one communication bus 34. The communication bus 34 is used to enable communication connections between the elements. The memory 33 may comprise a high-speed RAM memory or may further comprise a non-volatile memory NVM, such as at least one magnetic disk memory, in which various programs may be stored for performing various processing functions and implementing the method steps of the present embodiment.

Alternatively, the processor 31 may be implemented as, for example, a central processing unit (Central Processing Unit, abbreviated as CPU), an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and the processor 31 is coupled to the input device 30 and the output device 32 through wired or wireless connections.

Alternatively, the input device 30 may include a variety of input devices, for example, may include at least one of a user-oriented user interface, a device-oriented device interface, a programmable interface of software, a camera, and a sensor. Optionally, the device interface facing the device may be a wired interface for data transmission between devices, or may be a hardware insertion interface (such as a USB interface, a serial port, etc.) for data transmission between devices; alternatively, the user-oriented user interface may be, for example, a user-oriented control key, a voice input device for receiving voice input, and a touch-sensitive device (e.g., a touch screen, a touch pad, etc. having touch-sensitive functionality) for receiving user touch input by a user; optionally, the programmable interface of the software may be, for example, an entry for a user to edit or modify a program, for example, an input pin interface or an input interface of a chip, etc.; optionally, the transceiver may be a radio frequency transceiver chip, a baseband processing chip, a transceiver antenna, etc. with a communication function. An audio input device such as a microphone may receive voice data. The output device 32 may include a display, audio, or the like.

In this embodiment, the processor of the terminal device may include functions for executing each module of the data processing apparatus in each device, and specific functions and technical effects may be referred to the above embodiments and are not described herein again.

Fig. 4 is a schematic hardware structure of a terminal device according to another embodiment of the present application. Fig. 4 is a specific embodiment of the implementation of fig. 3. As shown in fig. 4, the terminal device of the present embodiment includes a processor 41 and a memory 42.

The processor 41 executes the computer program code stored in the memory 42 to implement the methods of the above-described embodiments.

The memory 42 is configured to store various types of data to support operation at the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, such as messages, pictures, video, etc. The memory 42 may include a random access memory (random access memory, simply referred to as RAM) and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

Optionally, a processor 41 is provided in the processing assembly 40. The terminal device may further include: a communication component 43, a power supply component 44, a multimedia component 45, an audio component 46, an input/output interface 47 and/or a sensor component 48. The components and the like specifically included in the terminal device are set according to actual requirements, which are not limited in this embodiment.

The processing component 40 generally controls the overall operation of the terminal device. The processing component 40 may include one or more processors 41 to execute instructions to perform all or part of the steps of the methods described above. Further, the processing component 40 may include one or more modules that facilitate interactions between the processing component 40 and other components. For example, processing component 40 may include a multimedia module to facilitate interaction between multimedia component 45 and processing component 40.

The power supply assembly 44 provides power to the various components of the terminal device. Power supply components 44 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for terminal devices.

The multimedia component 45 comprises a display screen between the terminal device and the user providing an output interface. In some embodiments, the display screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the display screen includes a touch panel, the display screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may sense not only the boundary of a touch or slide action, but also the duration and pressure associated with the touch or slide operation.

The audio component 46 is configured to output and/or input audio signals. For example, the audio component 46 includes a Microphone (MIC) configured to receive external audio signals when the terminal device is in an operational mode, such as a speech recognition mode. The received audio signals may be further stored in the memory 42 or transmitted via the communication component 43. In some embodiments, audio assembly 46 further includes a speaker for outputting audio signals.

The input/output interface 47 provides an interface between the processing assembly 40 and peripheral interface modules, which may be click wheels, buttons, etc. These buttons may include, but are not limited to: volume button, start button and lock button.

The sensor assembly 48 includes one or more sensors for providing status assessment of various aspects for the terminal device. For example, the sensor assembly 48 may detect the open/closed state of the terminal device, the relative positioning of the assembly, the presence or absence of user contact with the terminal device. The sensor assembly 48 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact, including detecting the distance between the user and the terminal device. In some embodiments, the sensor assembly 48 may also include a camera or the like.

The communication component 43 is configured to facilitate communication between the terminal device and other devices in a wired or wireless manner. The terminal device may access a wireless network based on a communication standard, such as WiFi,2G or 3G, or a combination thereof. In one embodiment, the terminal device may include a SIM card slot, where the SIM card slot is used to insert a SIM card, so that the terminal device may log into a GPRS network, and establish communication with a server through the internet.

From the above, it will be appreciated that the communication component 43, the audio component 46, and the input/output interface 47, the sensor component 48 referred to in the embodiment of fig. 4 may be implemented as an input device in the embodiment of fig. 3.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.

Claims

1. A method for smooth alignment of objects around an aircraft, comprising:

acquiring an original image set in a camera array;

extracting the original image set into boundary frame data by a background difference method;

extracting the coordinate parameters of the boundary frame, and converting the coordinate parameters of the boundary frame into global coordinate parameters;

and generating a regulating result according to the global coordinate parameters.

2. The method of claim 1, wherein the extracting the original image set as bounding box data by a background difference method comprises:

generating a background difference operator according to the smooth demand parameters;

and extracting the boundary box data in the original image set according to the background difference method operator.

3. The method of claim 1, wherein the extracting the bounding box coordinate parameters and converting the bounding box coordinate parameters to global coordinate parameters comprises:

inputting the boundary frame data into a coordinate extraction model to obtain the boundary frame coordinate parameters;

and converting the coordinate parameters of the boundary frame into the global coordinate parameters through a coordinate conversion matrix.

4. The method of claim 1, wherein generating a tuning result from the global coordinate parameter comprises:

generating target track data according to the global coordinate parameters;

generating the adjustment result according to the target track data, wherein the adjustment result comprises: and (5) confirming a result and a track calibration result.

5. An aircraft surrounding object smooth adjustment device, comprising:

the acquisition module is used for acquiring an original image set in the camera array;

the extraction module is used for extracting the original image set into boundary frame data through a background difference method;

the conversion module is used for extracting the coordinate parameters of the boundary frame and converting the coordinate parameters of the boundary frame into global coordinate parameters;

and the generating module is used for generating a regulating result according to the global coordinate parameters.

6. The apparatus of claim 5, wherein the extracting the original image set as bounding box data by a background subtraction method comprises:

the generating unit is used for generating a background difference operator according to the smooth demand parameters;

and the extraction unit is used for extracting the boundary frame data in the original image set according to the background difference operator.

7. The apparatus of claim 5, wherein the conversion module comprises:

the extraction unit is used for inputting the boundary frame data into a coordinate extraction model to obtain the boundary frame coordinate parameters;

and the conversion unit is used for converting the coordinate parameters of the boundary frame into the global coordinate parameters through a coordinate conversion matrix.

8. The apparatus of claim 5, wherein the generating module comprises:

the generation unit is used for generating target track data according to the global coordinate parameters;

and a result unit, configured to generate the adjustment result according to the target track data, where the adjustment result includes: and (5) confirming a result and a track calibration result.

9. A non-volatile storage medium, characterized in that the non-volatile storage medium comprises a stored program, wherein the program, when run, controls a device in which the non-volatile storage medium is located to perform the method of any one of claims 1 to 4.

10. An electronic device comprising a processor and a memory; the memory has stored therein computer readable instructions for executing the processor, wherein the computer readable instructions when executed perform the method of any of claims 1 to 4.