CN117061808A - Display control system for portable operation terminal of airborne photoelectric equipment - Google Patents

Display control system for portable operation terminal of airborne photoelectric equipment Download PDF

Info

Publication number
CN117061808A
CN117061808A CN202311062405.3A CN202311062405A CN117061808A CN 117061808 A CN117061808 A CN 117061808A CN 202311062405 A CN202311062405 A CN 202311062405A CN 117061808 A CN117061808 A CN 117061808A
Authority
CN
China
Prior art keywords
video
module
light video
visible light
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311062405.3A
Other languages
Chinese (zh)
Inventor
张宠
宋悦铭
徐芳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changchun Institute of Optics Fine Mechanics and Physics of CAS
Original Assignee
Changchun Institute of Optics Fine Mechanics and Physics of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changchun Institute of Optics Fine Mechanics and Physics of CAS filed Critical Changchun Institute of Optics Fine Mechanics and Physics of CAS
Priority to CN202311062405.3A priority Critical patent/CN117061808A/en
Publication of CN117061808A publication Critical patent/CN117061808A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/04—Programme control other than numerical control, i.e. in sequence controllers or logic controllers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
    • H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
    • H04N21/4316—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47217—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/265—Mixing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
    • H04N5/445—Receiver circuitry for the reception of television signals according to analogue transmission standards for displaying additional information
    • H04N5/45—Picture in picture, e.g. displaying simultaneously another television channel in a region of the screen

Abstract

The invention relates to the technical field of airborne photoelectric equipment, in particular to a display control system of a portable operation terminal for the airborne photoelectric equipment, wherein an information data display unit is used for displaying multiplexed data acquired by the airborne photoelectric equipment; the virtual single-rod operation unit is used for controlling the movement direction, the pitching direction and the movement speed of the airborne photoelectric equipment; the data communication unit is used for communicating with the airborne photoelectric equipment and the flight control system; the instruction control unit is used for sending an instruction to the airborne photoelectric equipment; the video display unit is used for receiving, decoding and displaying the video acquired by the airborne photoelectric equipment; the video playback unit is used for storing and playing back multiplexed data, visible light video and infrared light video acquired by the onboard photoelectric equipment. The invention solves the problem of single display control system of the existing portable control terminal, and has the relevant functions of image processing.

Description

Display control system for portable operation terminal of airborne photoelectric equipment
Technical Field
The invention relates to the technical field of airborne photoelectric equipment, in particular to a display control system of a portable operation terminal for the airborne photoelectric equipment.
Background
The display control system of the airborne photoelectric equipment is an important component of the airborne photoelectric equipment and is used for carrying out tasks such as information acquisition, platform control, data recording and data display. The control and display of the onboard photoelectric device and the state forwarding of the onboard photoelectric device are realized through communication with the onboard photoelectric device and the flight control system.
Most of the existing onboard display control software is developed based on an industrial control host, and due to the limitations of the configuration, the memory and other factors of the portable operation terminal, the existing display control software based on the portable operation terminal has single functions and cannot realize the related functions of excessive image processing.
Disclosure of Invention
The invention provides a display control system of a portable operation terminal for an on-board photoelectric device, which aims to solve the problems that the display control system of the existing portable operation terminal is single and does not have the related functions of multiple image processing, has multiple display control systems and multiple image processing functions including a normal display mode, a picture-in-picture mode, a splicing mode, a fusion mode and the like.
The invention provides a display control system for a portable operating terminal of an onboard photoelectric device, which comprises a video display unit, an information data display unit, a virtual single-rod operation unit, a data communication unit, an instruction control unit and a data playback unit, wherein,
the information data display unit is used for displaying multiplexed data acquired by the airborne photoelectric equipment; the virtual single-rod operation unit is used for controlling the movement direction, the pitching direction and the movement speed of the airborne photoelectric equipment; the data communication unit is used for communicating with the airborne photoelectric equipment and the flight control system; the instruction control unit is used for sending an instruction to the airborne photoelectric equipment; the video display unit is used for receiving, decoding and displaying multiplexed data, visible light video and infrared light video acquired by the airborne photoelectric equipment; the video playback unit is used for storing and playing back multiplexed data, visible light video and infrared light video acquired by the onboard photoelectric equipment.
Preferably, the video display unit comprises a video receiving module, a video parsing module, an image processing module and a video display module, wherein,
the video receiving module is used for receiving visible light video, infrared light video and multiplexing data acquired by the onboard photoelectric equipment; the video analysis module is used for separating the visible light video, the infrared light video and the multiplexing data, decoding the visible light video and the infrared light video and analyzing the multiplexing data; the image processing module is used for splicing, fusing and picture-in-picture processing of the visible light video and the infrared light video acquired by the airborne photoelectric equipment; the video display module is used for displaying a single-channel visible light video, a single-channel infrared light video, a picture-in-picture video, a spliced video or a fusion video.
Preferably, the image processing module comprises a picture-in-picture sub-module, an image splicing sub-module and an image fusion sub-module, wherein the picture-in-picture sub-module is used for outputting a picture-in-picture video in which a visible light video and an infrared light video are overlapped by combining the visible light video, the infrared light video and multiplexing data; the image stitching submodule is used for outputting a visible light stitching image or an infrared stitching image by combining visible light video or infrared video with multiplexing data; the image fusion submodule is used for outputting a fusion image by combining the visible light video, the infrared light video and the output type of image fusion.
Preferably, the information data display unit comprises a state display module and a data display module, wherein the state display module is used for analyzing and displaying the load state in the multiplexed data; the data display module is used for analyzing and displaying other data except the load state in the multiplexed data.
Preferably, the data communication unit comprises an airborne photoelectric device communication module and a flight control communication module, wherein the airborne photoelectric device communication module is used for inputting control instructions and control parameters to the airborne photoelectric device according to received multiplexing data of the airborne photoelectric device; the flight control communication module is used for inputting control instructions and control parameters to the flight control system through the operation of typing the portable control terminal.
Preferably, the video playback unit comprises a data storage module, a data information playback module and a video information playback module, wherein,
the data storage module is used for storing visible light video, infrared light video and multiplexing data acquired by the airborne photoelectric equipment; the data information playback module is used for analyzing and playing back the multiplexed data; the video information playback module is used for decoding and playing back visible light video and infrared light video.
Preferably, the communication module of the airborne photoelectric device comprises an instruction parameter sending sub-module and a data information receiving sub-module, wherein the instruction parameter sending sub-module is used for inputting instruction parameters to the airborne photoelectric device, and the data information receiving sub-module is used for receiving multiplexing data of the airborne photoelectric device;
the flight control communication module comprises an operation terminal control sub-module and a state forwarding sub-module, wherein the operation terminal control sub-module is used for sending an instruction to the flight control system by operating the portable operation terminal, and the state forwarding sub-module is used for forwarding the multiplexing data of the airborne photoelectric equipment to the flight control system according to the received multiplexing data of the airborne photoelectric equipment.
Preferably, the multiplexing data includes load status, load focal length, laser ranging value, azimuth angle, pitch angle, working mode, instruction return and longitude, latitude and altitude of the target collected by the onboard photoelectric device.
Preferably, the real-time splicing processing process of the image splicing sub-module is as follows:
s1, selecting two frames of visible light videos or two frames of infrared light videos, introducing a rapid guiding filter into a splicing algorithm, and carrying out filtering treatment on one frame of visible light videos or infrared light videos:
wherein I is the pixel index of the visible light video or the infrared light video, k is the window size, I is the image, ω is the selected local square window, a k And b k Is a constant in a local square window, mu k Sum sigma k Mean and variance of visible light video or infrared light video in local square window respectively, q i Output pixel value p for visible light video or infrared light video i To guide the pixel values of the image, P k Representing a guide image mean;
s2, performing feature matching on the visible light video or the infrared light video filtered in the step S1 and SURF feature points of the other frame of the visible light video or the infrared light video in the step S1, and eliminating mismatching point pairs by using a RANSAC algorithm:
wherein x 'is the abscissa of the output pixel, y' is the ordinate of the output pixel, x is the abscissa of the input pixel, y is the ordinate of the input pixel,a conversion matrix of pixel points, h 11 、h 12 、h 13 、h 21 、h 22 、h 23 、h 31 、h 32 、h 33 All are parameters of the conversion matrix;
s3, calculating a loss function f according to the formula (4) loss ：
Wherein x is i The' is the abscissa, y of the pixel point of the ith output i ' as the firstOrdinate, x of i output pixel points i The abscissa, y, of the pixel point input as the ith i The ordinate of the pixel point input for the ith;
and S4, performing distortion deformation on the two frames of visible light videos or the two frames of infrared light videos according to the characteristic points obtained in the formula (5), and fusing the distorted two frames of visible light videos or the two frames of infrared light videos to obtain a spliced image.
Preferably, the image fusion process is as follows: registering the visible light video and the infrared light video, drawing a fitting curve according to the characteristic points of the visible light video and the infrared light video, and carrying out interpolation calculation on the fitting curve by combining the output type of image fusion to obtain a fusion image.
Compared with the prior art, the invention has the following beneficial effects:
(1) According to the display control system for the portable control terminal of the airborne photoelectric equipment, provided by the invention, the video collected by the airborne photoelectric equipment is separated into the video information and the data information, and the video information is selectively output in one of a normal display mode, a picture-in-picture mode, a splicing mode and a fusion mode according to the data information, so that the resource utilization rate of the display control system is reduced.
(2) The invention can realize the real-time splicing of the video information and display the video information in real time.
(3) The invention sets the virtual single rod to make the operation interface more concise, and sets the video playback function which is convenient for the off-line searching problem to make the function of the display control system more comprehensive.
Drawings
Fig. 1 is a schematic diagram of a logic structure of a display control system for a portable manipulation terminal of an on-board electro-optical device according to an embodiment of the present invention;
fig. 2 is a schematic diagram of an external interface of a display control system for a portable manipulation terminal of an on-board electro-optical device according to an embodiment of the present invention;
fig. 3 is a schematic control flow diagram of a display control system for a portable manipulation terminal of an on-board electro-optical device according to an embodiment of the present invention.
The reference numerals include: the video display unit 1, the information data display unit 2, the virtual single-lever operation unit 3, the data communication unit 4, the instruction control unit 5, the video playback unit 6, the video receiving module 11, the video parsing module 12, the image processing module 13, the picture-in-picture module 131, the image splicing module 132, the image fusion module 133, the video display module 14, the status display module 21, the data display module 22, the on-board photoelectric device communication module 41, the flight control communication module 42, the data storage module 61, the data information playback module 62, the video information playback module 63, the instruction parameter transmitting sub-module 411, the data information receiving sub-module 412, the manipulation terminal control sub-module 421, the status forwarding sub-module 422, the on-board photoelectric device 7, the portable manipulation terminal 8 for the on-board photoelectric device, and the flight control system 9.
Detailed Description
Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, like modules are denoted by like reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, a detailed description thereof will not be repeated.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be further described in detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not to be construed as limiting the invention.
Fig. 1 illustrates a logic structure of a display control system for a portable manipulation terminal of an on-board electro-optical device according to an embodiment of the present invention.
As shown in fig. 1, the display control system for a portable manipulation terminal of an on-board electro-optical device provided by the present invention includes a video display unit 1, an information data display unit 2, a virtual single lever operation unit 3, a data communication unit 4, an instruction control unit 5, a video playback unit 6, wherein,
the information data display unit 2 is used for displaying multiplexed data acquired by the airborne photoelectric equipment 7; the virtual single-rod operation unit 3 is used for controlling the movement direction, the pitching direction and the movement speed of the airborne photoelectric equipment 7; the data communication unit 4 is used for communicating with the onboard photoelectric device 7 and the flight control system; the instruction control unit 5 is used for sending instructions to the onboard photoelectric equipment 7; the video display unit 1 is used for receiving, decoding and displaying multiplexed data, visible light video and infrared light video acquired by the airborne photoelectric equipment 7; the image processing module 13 is used for splicing, fusing and picture-in-picture processing of the visible light video and the infrared light video acquired by the airborne photoelectric equipment 7; the video playback unit 6 is used for storing and playing back multiplexed data, visible light video and infrared light video acquired by the onboard photoelectric device 7.
The multiplexing data comprises a load state, a load focal length, a laser ranging value, an azimuth angle, a pitch angle, a working mode of the airborne photoelectric equipment, instruction return and longitude, latitude and altitude of a target acquired by the airborne photoelectric equipment.
The command control unit 5 issues commands, such as focusing, zooming, manual tracking commands, to the on-board optoelectronic device 7 through the man-machine interaction buttons of the portable manipulation terminal.
The portable operation terminal also comprises a touch screen, a cable, a keyboard and the like, and the display control system of the portable operation terminal for the airborne photoelectric equipment 7 can also be controlled through a physical single rod.
The video display unit 1 comprises a video receiving module 11, a video analyzing module 12, an image processing module 13 and a video display module 14, wherein the video receiving module 11 is used for receiving visible light video, infrared light video and multiplexing data collected by the onboard photoelectric equipment 7; the video parsing module 12 is used for separating visible light video, infrared light video and multiplexing data, decoding the visible light video and the infrared light video, and parsing the multiplexing data; the image processing module 13 is used for splicing, fusing and picture-in-picture processing of the visible light video and the infrared light video acquired by the airborne photoelectric equipment; video display module 14 is configured to display a single-pass visible light video, a single-pass infrared light video, a picture-in-picture video, a stitched video, or a blended video.
The video display module 14 firstly analyzes the multiplexed data, and plays corresponding video according to the information of the video corresponding to the multiplexed data.
The image processing module 13 includes a pip sub-module 131, an image stitching sub-module 132 and an image fusion sub-module 133, where the pip sub-module 131 is configured to combine a visible light video, an infrared light video and multiplexing data to output a pip video where the visible light video and the infrared light video are superimposed; the image stitching sub-module 132 is configured to output a visible light stitched image or an infrared stitched image by using a visible light video or an infrared video in combination with multiplexing data; the image fusion sub-module 133 is configured to output a fused image in combination with the output types of the visible light video, the infrared light video, and the image fusion.
The real-time splicing processing process of the image splicing sub-module comprises the following steps:
s1, selecting two frames of visible light videos or two frames of infrared light videos, introducing a rapid guiding filter into a splicing algorithm, and carrying out filtering treatment on one frame of visible light videos or infrared light videos:
wherein I is the pixel index of the visible light video or the infrared light video, k is the window size, I is the image, ω is the selected local square window, a k And b k Is a constant in a local square window, mu k Sum sigma k Mean and variance of visible light video or infrared light video in local square window respectively, q i Output pixel value p for visible light video or infrared light video i To guide the pixel values of the image, P k Representing a guide image mean;
s2, performing feature matching on the visible light video or the infrared light video filtered in the step S1 and SURF feature points of the other frame of the visible light video or the infrared light video in the step S1, and eliminating mismatching point pairs by using a RANSAC algorithm:
wherein x 'is the abscissa of the output pixel, y' is the ordinate of the output pixel, x is the abscissa of the input pixel, y is the ordinate of the input pixel,a conversion matrix of pixel points, h 11 、h 12 、h 13 、h 21 、h 22 、h 23 、h 31 、h 32 、h 33 All are parameters of the conversion matrix;
s3, calculating a loss function f according to the formula (4) loss ：
Wherein x is i The' is the abscissa, y of the pixel point of the ith output i The' is the ordinate, x of the pixel point of the ith output i The abscissa, y, of the pixel point input as the ith i The ordinate of the pixel point input for the ith;
and S4, performing distortion deformation on the two frames of visible light videos or the two frames of infrared light videos according to the characteristic points obtained in the formula (5), and fusing the distorted two frames of visible light videos or the two frames of infrared light videos to obtain a spliced image.
The image fusion process comprises the following steps: registering the visible light video and the infrared light video, drawing a fitting curve according to the characteristic points of the visible light video and the infrared light video, and carrying out interpolation calculation on the fitting curve by combining the output type of image fusion to obtain a fusion image.
The video information comprises visible light video information and infrared light video information, and in the image fusion sub-module, the visible light video information is the main fusion video information to display color, and the infrared light video information is the main fusion video information to display gray.
The information data display unit 2 comprises a state display module 21 and a data display module 22, wherein the state display module 21 is used for analyzing and displaying the load state in the multiplexed data; the data display module 22 is used for analyzing and displaying other data except the load state in the multiplexed data.
The load state is displayed by using a red-green indicator lamp, if the load state is normal, the red-green indicator lamp displays green, and if the load state is abnormal, the red-green indicator lamp displays red.
The data communication unit 4 comprises an airborne photoelectric device 7 communication module 41 and a flight control communication module 42, wherein the airborne photoelectric device 7 communication module 41 is used for inputting control instructions and control parameters to the airborne photoelectric device 7 according to received multiplexing data of the airborne photoelectric device 7; the flight control communication module 42 is used for inputting control instructions and control parameters to the flight control system through the key input of the portable operation terminal.
The communication module 41 of the airborne optoelectronic device 7 comprises an instruction parameter sending sub-module 411 and a data information receiving sub-module 412, wherein the instruction parameter sending sub-module 411 is used for inputting instruction parameters to the airborne optoelectronic device 7, and the data information receiving sub-module 412 is used for receiving multiplexing data of the airborne optoelectronic device 7;
the flight control communication module 42 includes a manipulation terminal control sub-module 421 and a state forwarding sub-module 422, where the manipulation terminal control sub-module 421 is configured to send an instruction to the flight control system by manipulating the portable manipulation terminal, and the state forwarding sub-module 422 is configured to forward the multiplexed data of the airborne optoelectronic device 7 to the flight control system according to the received multiplexed data of the airborne optoelectronic device 7.
Because the information such as the equipment state, the angle data and the like is stored in the multiplexing data, the multiplexing data and the video data are mixed together during real-time receiving, and therefore the multiplexing data are required to be separated and analyzed and displayed during video analysis; similarly, the data stored during video playback is mixed, and the data also needs to be separated for analysis and display during playback.
The video playback unit 6 comprises a data storage module 61, a data information playback module 62 and a video information playback module 63, wherein the data storage module 61 is used for storing visible light video, infrared light video and multiplexing data acquired by the onboard photoelectric device 7; the data information playback module 62 is used for analyzing and playing back the multiplexed data; the video information playback module 63 is used for decoding and playing back visible light video and infrared light video. Fig. 2 illustrates an external interface of a display control system for a portable manipulation terminal of an on-board electro-optical device according to an embodiment of the present invention.
As shown in fig. 2, the external interface of the display control system for the portable manipulation terminal 8 of the on-board electro-optical device provided by the present invention includes: the serial communication interface with the onboard photoelectric device 7, the UDP multicast network interface with the onboard photoelectric device 7 and the UDP multicast network interface with the flight control system 9 are used for sending instruction parameters to the onboard photoelectric device 7 and receiving the load state and data information of the onboard photoelectric device 7; the UDP multicast network interface with the onboard photoelectric device 7 is used for receiving the visible light video information and the infrared light video information of the onboard photoelectric device 7; the UDP multicast network interface with the flight control system 9 is used for receiving the instruction parameters sent by the portable control terminal control system, and forwarding the load state and data information of the airborne optoelectronic device 7 to the flight control system 9.
The portable operating terminal 8 for the on-board photovoltaic device is connected to the on-board photovoltaic device 7 and the flight control system 9 via a serial communication interface with the on-board photovoltaic device 7 and a UDP multicast network interface with the flight control system 9, respectively, and the on-board photovoltaic device 7 is unidirectionally connected to the portable operating terminal 8 for the on-board photovoltaic device via the on-board photovoltaic device 7.
Fig. 3 shows a control flow of a display control system for a portable manipulation terminal of an on-board electro-optical device according to an embodiment of the present invention.
As shown in fig. 3, a display control system of a portable control terminal 8 for an onboard photoelectric device is initialized, an interface of the onboard photoelectric device 7 is connected, data playback, communication with the onboard photoelectric device 7 and communication with a flight control system 9 are selected according to practical application, and if a data playback mode is adopted, visible light video information or infrared light video information, load state information at the same moment and data information are directly played, so that the video playback simultaneity is ensured; if the real-time playing mode is adopted, the real-time playing mode is converted into a communication mode with the airborne photoelectric equipment 7, and after the real-time playing mode is connected with the airborne photoelectric equipment 7 through a serial communication interface, the real-time loading state, the data information and the like of the information data display area are displayed, and control instructions and parameters are sent; after establishing UDP multicast network connection with the airborne photoelectric device 7, the video display area displays a single-path visible light video, a single-path infrared light video, a picture-in-picture video, a spliced image and a fused image of the airborne photoelectric device 7 in real time according to the data information; after the connection with the flight control system 9 is established, the instruction control can be carried out on the airborne photoelectric equipment 7 through a hard key of the portable control terminal, a physical single rod, a keyboard and the like can be also connected, and after the connection with the flight control system 9 is established, the display control system of the portable control terminal 8 for the airborne photoelectric equipment can forward the load state, data information and the like of the airborne photoelectric equipment 7 to the flight control system 9 in real time; when the user needs to exit, the software can automatically disconnect all interfaces of the equipment and directly close the software.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (10)

1. A display control system for a portable manipulation terminal of an onboard electro-optical device is characterized by comprising a video display unit, an information data display unit, a virtual single-lever operation unit, a data communication unit, an instruction control unit, and a data playback unit, wherein,
the information data display unit is used for displaying multiplexing data acquired by the airborne photoelectric equipment; the virtual single-rod operation unit is used for controlling the movement direction, the pitching direction and the movement speed of the airborne photoelectric equipment; the data communication unit is used for communicating with the airborne photoelectric equipment and the flight control system; the instruction control unit is used for sending an instruction to the airborne photoelectric equipment; the video display unit is used for receiving, decoding and displaying the multiplexing data, the visible light video and the infrared light video which are acquired by the airborne photoelectric equipment; the video playback unit is used for storing and playing back the multiplexed data, the visible light video and the infrared light video which are acquired by the airborne photoelectric equipment.
2. The display control system for a portable manipulation terminal of an on-board optoelectronic device of claim 1, wherein the video display unit comprises a video receiving module, a video parsing module, an image processing module, and a video display module, wherein,
the video receiving module is used for receiving the visible light video, the infrared light video and the multiplexing data acquired by the airborne photoelectric equipment; the video analysis module is used for separating the visible light video, the infrared light video and the multiplexing data, decoding the visible light video and the infrared light video and analyzing the multiplexing data; the image processing module is used for splicing, fusing and picture-in-picture processing the visible light video and the infrared light video which are acquired by the airborne photoelectric equipment; the video display module is used for displaying a single-path visible light video, a single-path infrared light video, a picture-in-picture video, a spliced video or a fusion video.
3. The display control system for a portable manipulation terminal of an on-board optoelectronic device of claim 2, wherein the image processing module comprises a picture-in-picture sub-module, an image stitching sub-module, and an image fusion sub-module, wherein the picture-in-picture sub-module is configured to output a picture-in-picture video in which the visible light video and the infrared light video are superimposed in combination with the visible light video, the infrared light video, and the multiplexed data; the image stitching sub-module is used for outputting a visible light stitching image or an infrared stitching image by utilizing the visible light video or the infrared light video and combining the multiplexing data; the image fusion submodule is used for outputting a fusion image by combining the visible light video, the infrared light video and the output type of image fusion.
4. The display control system for a portable manipulation terminal of an on-board optoelectronic device according to claim 1, wherein the information data display unit comprises a status display module and a data display module, wherein the status display module is configured to parse and display a load status in the multiplexed data; the data display module is used for analyzing and displaying other data except the load state in the multiplexing data.
5. The display control system for a portable manipulation terminal of an on-board optoelectronic device according to claim 1, wherein the data communication unit comprises an on-board optoelectronic device communication module and a flight control communication module, wherein the on-board optoelectronic device communication module is configured to input control instructions and control parameters to the on-board optoelectronic device according to received multiplexed data of the on-board optoelectronic device; the flight control communication module is used for inputting control instructions and control parameters to the flight control system through the key input of the portable operation terminal.
6. The display control system for a portable manipulation terminal of an on-board optoelectronic device of claim 1, wherein the video playback unit comprises a data storage module, a data information playback module, and a video information playback module, wherein,
the data storage module is used for storing the visible light video, the infrared light video and the multiplexing data acquired by the airborne photoelectric equipment; the data information playback module is used for analyzing and playing back the multiplexed data; the video information playback module is used for decoding and playing back the visible light video and the infrared light video.
7. The display control system for a portable manipulation terminal of an on-board optoelectronic device according to claim 5, wherein the on-board optoelectronic device communication module comprises a command parameter transmission sub-module for inputting command parameters to the on-board optoelectronic device and a data information receiving sub-module for receiving multiplexed data of the on-board optoelectronic device;
the flight control communication module comprises an operation terminal control sub-module and a state forwarding sub-module, wherein the operation terminal control sub-module is used for sending an instruction to the flight control system by operating the portable operation terminal, and the state forwarding sub-module is used for forwarding multiplexing data of the airborne photoelectric equipment to the flight control system according to the received multiplexing data of the airborne photoelectric equipment.
8. The display control system for a portable manipulation terminal of an on-board optoelectronic device of claim 1, wherein the multiplexed data includes a loading state, a loading focal length, a laser ranging value, an azimuth angle, a pitch angle, an operation mode, a command report of the on-board optoelectronic device, and a longitude, latitude, and altitude of a target collected by the on-board optoelectronic device.
9. The display control system for a portable manipulation terminal of an on-board optoelectronic device of claim 3, wherein the real-time stitching process of the image stitching sub-module is:
s1, selecting two frames of visible light videos or two frames of infrared light videos, introducing a rapid guiding filter into a splicing algorithm, and carrying out filtering treatment on one frame of the visible light videos or the infrared light videos:
wherein I is the pixel index of the visible light video or the infrared light video, k is the window size, I is the image, ω is the selected local square window, a k And b k Is a constant in the partial square window, mu k Sum sigma k Mean and variance of the visible light video or the infrared light video in the local square window, respectively, q i For the output pixel value, p, of the visible light video or the infrared light video i To guide the pixel values of the image, P k Representing a guide image mean;
s2, performing feature matching on the visible light video or the infrared light video filtered in the step S1 and SURF feature points of the other frame of the visible light video or the infrared light video in the step S1, and eliminating mismatching point pairs by using a RANSAC algorithm:
wherein x 'is the abscissa of the output pixel, y' is the ordinate of the output pixel, x is the abscissa of the input pixel, y is the ordinate of the input pixel,a conversion matrix of pixel points, h 11 、h 12 、h 13 、h 21 、h 22 、h 23 、h 31 、h 32 、h 33 Are parameters of the transformation matrix;
s3, calculating a loss function f according to the formula (4) loss ：
Wherein x is i The' is the abscissa, y of the pixel point of the ith output i The' is the ordinate, x of the pixel point of the ith output i The abscissa, y, of the pixel point input as the ith i The ordinate of the pixel point input for the ith;
and S4, performing distortion deformation on the two frames of visible light videos or the two frames of infrared light videos according to the characteristic points obtained in the formula (5), and fusing the distorted two frames of visible light videos or the two frames of infrared light videos to obtain a spliced image.
10. The display control system for a portable manipulation terminal of an on-board optoelectronic device of claim 3, wherein the image fusion process is: registering the visible light video and the infrared light video, drawing a fitting curve according to the characteristic points of the visible light video and the infrared light video, and carrying out interpolation calculation on the fitting curve by combining the output type of image fusion to obtain the fusion image.
CN202311062405.3A 2023-08-22 2023-08-22 Display control system for portable operation terminal of airborne photoelectric equipment Pending CN117061808A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311062405.3A CN117061808A (en) 2023-08-22 2023-08-22 Display control system for portable operation terminal of airborne photoelectric equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311062405.3A CN117061808A (en) 2023-08-22 2023-08-22 Display control system for portable operation terminal of airborne photoelectric equipment

Publications (1)

Publication Number Publication Date
CN117061808A true CN117061808A (en) 2023-11-14

Family

ID=88660522

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311062405.3A Pending CN117061808A (en) 2023-08-22 2023-08-22 Display control system for portable operation terminal of airborne photoelectric equipment

Country Status (1)

Country Link
CN (1) CN117061808A (en)

Similar Documents

Publication Publication Date Title
CN104580858B (en) A kind of unmanned plane is imaged load with multi-source
WO2017206917A1 (en) Video management system, multi-screen display card and monitoring all-in-one machine
US20060284786A1 (en) Display control apparatus, system, and display control method
CN1251235A (en) Real-time method of digitally altering video data stream to remove portions of the original image
CN110233998A (en) A kind of method of transmitting video data, device, equipment and storage medium
US6941343B2 (en) System and method for point to point integration of personal computers with videoconferencing systems
CN101086834A (en) A method for controlling display effect of caption and control device
CN211184115U (en) Vehicle-mounted display control terminal with multi-channel video display function
CN110519531B (en) Multipath high-definition video distributed processing equipment
US20080256591A1 (en) Virtual webcam and method thereof
CN103090976B (en) IR image display device, thermal image display system and IR image display method
CN105183269A (en) Method for automatically identifying screen where cursor is located
CN117061808A (en) Display control system for portable operation terminal of airborne photoelectric equipment
CN113949855A (en) Image data transmission method and device and nonvolatile storage medium
CN113890977A (en) Airborne video processing device and unmanned aerial vehicle with same
CN112312040B (en) Video processor and display system
CN210518587U (en) Multi-channel high-definition video distributed processing equipment
CN202026402U (en) Monitoring camera with functions of automatic focusing and image correcting
CN111459068A (en) Ground comprehensive display control system and method for unmanned aerial vehicle-mounted day and night photoelectric system
CN109951736A (en) A kind of filter method and system of online real-time video
CN112261430A (en) Live broadcast system with more than one mobile camera device and live broadcast method thereof
KR20200053879A (en) Apparatus and method for simultaneous control of heterogeneous cameras, and camera control client
CN103674269A (en) Thermal image measuring device and thermal image measuring method
CN114199464A (en) SF6 gas leakage detection handheld equipment for realizing video display control
CN113132783A (en) Multi-machine shared multi-screen display control system and method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination