CN116797716A

CN116797716A - Real-time acquisition method, device and system for mapping model

Info

Publication number: CN116797716A
Application number: CN202210266976.8A
Authority: CN
Inventors: 赵岩; 谭兴; 马维峰; 吴昊
Original assignee: Wuhan Infoearth Information Co ltd
Current assignee: Wuhan Infoearth Information Co ltd
Priority date: 2022-03-17
Filing date: 2022-03-17
Publication date: 2023-09-22

Abstract

The invention provides a method, a device and a system for acquiring a mapping model in real time, wherein the method comprises the following steps: receiving video signals acquired in real time by a aerial camera carried by an aircraft and target information corresponding to the period in each period; writing the target information into the supplemental enhancement information of the current frame in the video signal to generate a target video; the target video is sent to first equipment in real time, so that the first equipment intercepts modeling candidate images from the target video, models the modeling candidate images by using a modeling engine based on the modeling candidate images, and acquires a mapping model; wherein the target information includes at least location information of the aircraft. The method, the device and the system for acquiring the mapping model in real time can improve modeling efficiency and have higher behavior of the mapping model.

Description

Real-time acquisition method, device and system for mapping model

Technical Field

The present invention relates to the field of computer vision technologies, and in particular, to a method, an apparatus, and a system for acquiring a mapping model in real time.

Background

Mapping refers to measuring, collecting and modeling the shape, size, spatial position, attributes and the like of natural geographic elements or surface artificial facilities. With the development of technology, digital mapping has been widely used because of its characteristics of automation, intelligence and specialized rapid acquisition of required data information.

The data mapping at the present stage mostly utilizes an aircraft to carry a cradle head, flies in a flight operation area according to a preset flight route, and executes corresponding aerial photography operation by the cradle head so as to complete mapping work of terrain exploration. In the prior art, pictures acquired during aerial photography operation are stored in a secure digital card (Secure Digital Memory Card, SD card), and after an aircraft lands, the pictures are copied out of the SD card, so that a mapping model can be built according to picture information. When the aircraft flies, the photos inside the SD card cannot be taken out for real-time processing, and the efficiency of building the mapping model is low.

Disclosure of Invention

The invention provides a method, a device and a system for acquiring a mapping model in real time, which are used for solving the defect that in the prior art, only after the flying is finished, the efficiency is low due to the fact that the mapping model is constructed by using pictures, and improving the efficiency and the behavior of the mapping model is realized.

The invention provides a real-time acquisition method of a mapping model, which comprises the following steps:

receiving video signals acquired in real time by a aerial camera carried by an aircraft and target information corresponding to the period in each period;

writing the target information into the supplemental enhancement information of the current frame in the video signal to generate a target video;

The target video is sent to first equipment in real time, so that the first equipment intercepts modeling candidate images from the target video, models the modeling candidate images by using a modeling engine based on the modeling candidate images, and acquires a mapping model;

wherein the target information includes at least location information of the aircraft.

According to the real-time acquisition method of the mapping model provided by the invention, the target information also comprises the attitude information of the aerial camera.

The invention also provides a real-time acquisition method of the mapping model, which comprises the following steps:

receiving a target video sent by a second device in real time;

under the condition that the supplemental enhancement information of each current frame is analyzed from the target video to meet target conditions, converting the current frame into a modeling candidate image;

modeling by using a modeling engine based on the modeling candidate image and target information carried in the supplementary enhancement information corresponding to the modeling candidate image to obtain a mapping model;

According to the method for acquiring the mapping model in real time provided by the invention, the method for acquiring the mapping model by modeling by using a modeling engine based on the modeling candidate image and the target information carried in the supplementary enhancement information corresponding to the modeling candidate image comprises the following steps:

writing the position information of the aircraft carried in the supplemental enhancement information corresponding to the modeling candidate image into first information of the modeling candidate image;

writing the attitude information of the aerial camera carried in the supplementary enhancement information into second information of the modeling candidate image;

and inputting the modeling candidate image into the modeling engine for modeling, and obtaining a mapping model.

According to the method for acquiring the mapping model in real time provided by the invention, the target condition comprises that the target content is stored in the supplementary enhancement information, and the verification content of the target condition is true.

The invention also provides a real-time acquisition device of the mapping model, which comprises the following steps:

the information receiving module is used for receiving video signals acquired in real time by the aerial camera carried by the aircraft and target information corresponding to the period in each period;

The video coding module is used for writing the target information into the supplemental enhancement information of the current frame in the video signal to generate a target video;

the video sending module is used for sending the target video to first equipment in real time, so that the first equipment intercepts modeling candidate images from the target video, models the modeling candidate images by using a modeling engine based on the modeling candidate images, and acquires a mapping model;

the video receiving module is used for receiving the target video sent by the second equipment in real time;

the image acquisition module is used for converting the current frame into a modeling candidate image under the condition that the supplemental enhancement information of each current frame is analyzed from the target video to meet the target condition;

the modeling module is used for modeling by using a modeling engine based on the modeling candidate image and target information carried in the supplementary enhancement information corresponding to the modeling candidate image to obtain a mapping model;

The invention also provides a real-time acquisition system of the mapping model, which comprises one or more real-time acquisition devices of the mapping model and one or more real-time acquisition devices of the mapping model.

The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method for acquiring a mapping model in real time as described in any one of the above when executing the program.

The invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of a method of real-time acquisition of a mapping model as described in any of the above.

The invention also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of a method of real-time acquisition of a mapping model as described in any one of the above.

According to the method, the device and the system for acquiring the mapping model in real time, the target video is acquired by writing the target information into the current frame in the video signal in the form of the supplementary enhancement information, and the target video is synchronously transmitted to the first equipment in real time, so that the first equipment intercepts out the modeling candidate image according to the target video, and the mapping model is built in real time by combining the position information of the aircraft carried by the target video. The modeling efficiency can be improved, and the behavior of the mapping model is higher.

Drawings

In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for acquiring a mapping model in real time according to the present invention;

FIG. 2 is a second flow chart of a method for acquiring a mapping model in real time according to the present invention;

FIG. 3 is a schematic structural diagram of a real-time acquisition device for mapping models provided by the invention;

FIG. 4 is a second schematic structural diagram of the real-time acquisition device of mapping model according to the present invention;

FIG. 5 is a schematic diagram of a real-time acquisition system for mapping models provided by the present invention;

fig. 6 is a schematic structural diagram of an electronic device provided by the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The terms first, second and the like in the description and in the claims, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, as appropriate, such that embodiments of the present application may be implemented in sequences other than those illustrated or described herein, and that the objects identified by "first," "second," etc. are generally of a type, and are not limited to the number of objects, such as the first object may be one or more.

It is to be understood that the terminology used in the description of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

The terms "comprises" and "comprising" indicate the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

Fig. 1 is a schematic flow chart of a method for acquiring a mapping model in real time. As shown in fig. 1, a method for acquiring a mapping model in real time according to an embodiment of the present invention includes: and step 101, receiving video signals acquired in real time by the aerial camera carried by the aircraft and target information corresponding to the period in each period.

Wherein the target information comprises at least position information of the aircraft.

The real-time acquisition device of the mapping model may be a device installed on an aircraft. The application scene of the real-time acquisition device of the mapping model is that a second device is arranged on the aircraft, and the real-time acquisition device of the mapping model provides videos shot by the aircraft in a flight route in real time for any first device through the second device. The first equipment captures the images from the received real-time video, so that the real-time pictures are obtained in the flight process of the aircraft, and a real-time mapping model is built.

The application scene of the real-time acquisition device for the mapping model in the embodiment of the invention does not influence the operation of carrying out the production of the aerial remote sensing data and the information extraction according to the normal program.

The types and layout positions of the aircrafts and the aerial cameras are not particularly limited in the embodiment of the invention.

Illustratively, the aircraft includes, but is not limited to, an aircraft such as an unmanned aerial vehicle, etc., as embodiments of the invention are not specifically limited thereto.

Illustratively, aerial cameras include, but are not limited to, area array optical instruments, linear array optical instruments, spectroscopic instruments, or radar instruments, and the like, to which embodiments of the present invention are not limited in detail.

Preferably, the unmanned aerial vehicle carrier has a cradle head in the form of any one of a single lens, a double lens, a triple lens or a five lens, and performs an aerial photography task of earth observation. Taking a single-lens holder as an example, the camera holder is mainly used for shooting an orthographic image, and the effect of oblique photogrammetry can be achieved by adjusting the angle of the holder. Meanwhile, an airborne computer (namely second equipment) is arranged on the unmanned aerial vehicle, an interface of the computer is connected with a camera cradle head stably arranged on the unmanned aerial vehicle, and the interface is used for transmitting video signals collected by the camera cradle head in real time.

An interface of the computer is in communication connection with a positioning system arranged in the unmanned aerial vehicle, and the interface is used for transmitting target information in real time.

The target information refers to state information corresponding to the aircraft while the video signals are acquired by the aerial camera in real time. The content of the target information is not particularly limited in the embodiment of the present invention. Illustratively, the target information includes at least location information of the aircraft.

The position information of the aircraft refers to the geographic position information corresponding to the aircraft while the video signals are acquired in real time by the aerial camera. The position information of the aircraft is used for unifying the image information acquired in the flight process to the same geographic frame.

Specifically, in step 101, after the aircraft takes off, the aerial camera carried by the aircraft monitors the ground condition. In each period, the real-time acquisition device of the mapping model corresponding to the second equipment receives the collected video signals from the aerial camera and observes the collected video signals to the ground in real time, and records the position information of the corresponding aircraft at the moment of collecting each video signal.

The process of acquiring the position information of the aircraft is not particularly limited in the embodiment of the invention.

Illustratively, the real-time acquisition device of the mapping model acquires position information of the aircraft corresponding to the time information from a global positioning system (Global Positioning System, GPS) built in the aircraft by acquiring the time information of the video signal. The location information of the aircraft includes, but is not limited to, geographic information such as aircraft longitude, aircraft latitude, and aircraft altitude.

And 102, writing the target information into the supplemental enhancement information of the current frame in the video signal to generate a target video.

Specifically, in step 102, the real-time acquisition device of the mapping model corresponding to the second device encodes the video RAW data (i.e., the video signal) acquired in step 101 into a compressed format h.264 or h.265, and the encoder writes the target information into supplemental enhancement information (Supplemental Enhancement Information, SEI) of the video stream, so as to obtain the compressed target video.

The process of compression encoding the video signal into the target video according to the embodiment of the invention is not particularly limited.

Illustratively, in each cycle, the aircraft longitude, aircraft latitude, and aircraft altitude are written into the SEI information of the current frame in that cycle. A specific embodiment of a format of writing the target information into the SEI information is given below:

at bytes 1-8, the aircraft longitude is written, the data type of which is double precision floating point.

At the 9 th-16 th byte, the aircraft latitude is written, and the data type is double-precision floating point type.

At 17-24 bytes, the aircraft altitude is written, and the data type is double-precision floating point type.

At bytes 25-28, check bits are written, the data type of which is integer.

Step 103, the target video is sent to the first device in real time, so that the first device intercepts modeling candidate images from the target video, models the modeling candidate images by using a modeling engine based on the modeling candidate images, and acquires a mapping model.

The first device is a main body for performing aerial photography operation in the air.

Correspondingly, the second device refers to a party subject who performs building of the mapping model on the ground.

Specifically, in step 103, the real-time acquisition device of the mapping model corresponding to the second device sends the target video generated in step 102 to the first device through a wireless communication technology.

Among other wireless communication technologies, but not limited to, WIFI wireless cellular signals (2G, 3G, 4G, 5G).

A real-time acquisition device of a mapping model operated by a first device on the ground is used for receiving a target video sent by a real-time acquisition device of the mapping model operated by a second device for performing aerial photography tasks, then a modeling candidate image is selected from the target video, and the modeling candidate image is input into a modeling engine to establish a real-time mapping model.

The embodiment of the present invention is not particularly limited to this process.

Preferably, each frame of the target video is received, SEI information of the current frame is read, if the corresponding byte of the SEI information stores the target information, and meanwhile, the SEI information length and the check bit accord with preset rules, the current frame is stored into a modeling candidate image, and meanwhile, the aircraft longitude, the aircraft latitude and the aircraft altitude carried by the SEI information are written into the image, so that a modeling engine can establish a real-time mapping model according to the modeling candidate image.

In the prior art, most of camera technologies carried by aircrafts are foreign equipment and technologies, and manufacturers of aircrafts cannot take out photos inside an SD card during the flight of the aircrafts, so that mapping models can only be built by utilizing the photos inside the SD card after the aircrafts land. The two images are continuously shot by the carried camera, which is far apart in the actual geographic space, and the complete landform of the region is represented by simply relying on the two images, so that the flight route of the aircraft needs to be provided with the condition of turning back or layering, and the situation of traversing to all image acquisition points is ensured to construct the complete mapping model of the landform of the region, but the modeling efficiency is slower and the behavior is lower.

In the embodiment of the invention, video signals acquired by a camera carried by an aircraft are utilized, after being encoded and processed by utilizing SEI information carried by target information, the generated target video is transmitted to first equipment on the ground in real time, so that a ground worker can perform real-time screenshot from the target video, and after the target video is saved as a picture, SEI information decoded from the video is written into the picture to establish a mapping model. In the process, the flight route of the aircraft does not need to be set in a complex mode, the modeling efficiency is high, and the appearance is high.

According to the embodiment of the invention, the target information is written into the current frame in the video signal in the form of the supplemental enhancement information, the target video is acquired, and the target video is synchronously transmitted to the first device in real time, so that the first device intercepts the modeling candidate image according to the target video, and a mapping model is built in real time by combining the position information of the aircraft carried by the target video. The modeling efficiency can be improved, and the behavior of the mapping model is higher.

On the basis of any of the above embodiments, the target information further includes attitude information of the aerial camera.

Specifically, the target information in step 101 may include attitude information of the aerial camera in addition to the position information of the aircraft.

The attitude information of the aerial camera refers to direction information for acquiring video signals in the aerial camera, and the own characteristics of the attitude information are related. The attitude information of the aerial camera is used for associating the pixel coordinate system of the image information acquired in the flight process with the corresponding camera coordinate system.

Further, in each period, the real-time acquisition device of the mapping model corresponding to the second device receives the collected video signals from the aerial camera and observes the collected video signals to the ground in real time, and records the position information of the corresponding aircraft and the attitude information of the aerial camera at the moment of collecting each video signal.

The attitude information of the aerial camera comprises, but is not limited to, camera pan-tilt yaw angle, camera pan-tilt pitch angle, camera pan-tilt roll angle and other camera external parameter information.

Correspondingly, the specific implementation of the format of the target information writing SEI information:

And writing a yaw angle of the camera cradle head in the 25 th to 32 th bytes, wherein the data type is double-precision floating point type.

And writing the 33 th to 40 th bytes into the pitching angle of the camera cradle head, wherein the data type is double-precision floating point type.

And writing the rolling angle of the camera cradle head in the 41 st-48 th byte, wherein the data type is double-precision floating point type.

At bytes 49-52, check bits are written, the data type of which is integer.

According to the embodiment of the invention, the target information is written into the current frame in the video signal in the form of supplementary enhancement information, the target video is acquired, and the target video is synchronously transmitted to the first equipment in real time, so that the first equipment intercepts the modeling candidate image according to the target video, and a mapping model is built in real time by combining the position information of the aircraft carried by the target video and the attitude information of the aerial camera. The modeling efficiency can be improved, and the accuracy and the behavior of the mapping model are higher.

FIG. 2 is a second flow chart of a method for acquiring a mapping model in real time according to the present invention. As shown in fig. 2, the method for acquiring a mapping model in real time according to the embodiment of the present invention includes: step 201, receiving a target video sent by a second device in real time.

The real-time acquisition device of the mapping model may be a device disposed on the ground receiving end. The application scene of the real-time acquisition device of the mapping model is that a first device is arranged on the ground, the real-time acquisition device of the mapping model receives videos of an aircraft provided by a second device in real time in a flight route through the first device, intercepts the received videos in real time, acquires real-time pictures in the flight process of the aircraft, and establishes the real-time mapping model.

Specifically, in step 201, the real-time acquisition device of the mapping model corresponding to the first device receives, through a wireless communication technology, the target video from the real-time acquisition device of the mapping model corresponding to the second device.

Step 202, converting the current frame into a modeling candidate image under the condition that the supplemental enhancement information of each current frame is analyzed from the target video to meet the target condition.

Specifically, in step 202, the real-time acquisition device of the mapping model corresponding to the first device sequentially reads each frame of the target video in order, parses the currently read frame as the current frame, and determines the parsed supplemental enhancement information and the preset target condition, where the determination results are two: success of the determination and failure of the determination.

And if the judgment success is that the supplemental enhancement information of each current frame meets the target condition, indicating that the supplemental enhancement information obtained after the current frame is decoded is effective information, saving the current frame as a picture, and enabling the picture to be used as a modeling candidate image.

The failure judgment is the condition that the supplemental enhancement information of each current frame does not meet the target condition, and the obtained supplemental enhancement information is invalid information after the current frame is decoded, the current frame is discarded, and a real-time acquisition device of the mapping model corresponding to the first device sequentially reads the next frame of the current frame, so as to execute the judgment process.

It is understood that the target condition refers to a judgment condition that is set according to the supplemental enhancement information being valid and being readable.

The target condition may be a comprehensive judgment condition for a plurality of kinds of information carried by the supplemental enhancement information, or an independent judgment condition for a certain kind of information carried by the supplemental enhancement information. The setting of the target condition in the embodiment of the invention is not particularly limited.

The method includes the steps of reading SEI information of a current frame, if bytes stored in each section of memory space in the SEI information are converted into corresponding data through binary conversion, and if the data are larger than a certain preset threshold value, judging that the judgment is successful, namely, the supplemental enhancement information of the current frame accords with a target condition, and storing the current frame as a modeling candidate image.

Illustratively, the SEI information of the current frame is read, and if the number of bits of the SEI information is equal to another preset threshold, the determination is considered to be successful, that is, the supplemental enhancement information indicating the current frame meets the target condition, the current frame is saved as a modeling candidate image. Taking the example that the SEI information contains 52 bytes, if the number of bits of the SEI information is 52×8=416 bits, the SEI information is considered valid.

And 203, modeling by using a modeling engine based on the modeling candidate image and target information carried in the supplemental enhancement information corresponding to the modeling candidate image to obtain a mapping model.

It should be noted that, the modeling engine refers to a development environment that uses an underlying function library to create a mathematical representation of a three-dimensional object or shape. The modeling engine includes, but is not limited to, context Capture, pix4Dmapper, photoscan, or inp, etc. live-action three-dimensional modeling software.

Specifically, in step 203, the real-time acquisition device of the mapping model corresponding to the first device uses the modeling candidate images acquired in real time in step 202 and the target information carried in the supplemental enhancement information of each modeling candidate image as input information, selects a modeling engine corresponding to the input information, inputs the modeling engine into the modeling engine for modeling, and builds and expands the complete mapping model in real time.

The content of the target information is not particularly limited in the embodiment of the present invention.

Optionally, the real-time acquisition device of the mapping model corresponding to the first device analyzes the modeling candidate image supplementary enhancement information while acquiring the modeling candidate image in real time in step 202, and acquires the aircraft longitude, the aircraft latitude and the aircraft altitude of the corresponding position. The modeling candidate images, as well as the aircraft longitude, aircraft latitude, and aircraft altitude, are then input to a modeling engine to build a mapping model in real-time.

The modeling process of the embodiment of the invention is not particularly limited.

Illustratively, in step 202, SEI information meeting the target condition is parsed, and contents stored in 1 st to 8 th bytes of the SEI information are binary-converted to obtain aircraft longitude data, which are sequentially stored under an "aircraft longitude" column field in an independently maintained database. Similarly, the contents stored in the 9 th to 16 th bytes and the 17 th to 24 th bytes of SEI information are processed and stored under the "aircraft latitude" and "aircraft altitude" column fields in the database respectively.

The current frame corresponding to the SEI information is then stored as a modeling candidate image into another independently maintained database. Wherein the two databases are associated with a read row number.

In step 203, the two databases are input to a modeling engine, which correlates the modeling candidate image with the aircraft longitude, aircraft latitude and aircraft altitude at the corresponding locations using the number of reading lines of the two databases, and invokes an algorithm in the engine to perform modeling to obtain a mapping model.

The embodiment of the invention is based on receiving a target video, converting the current frame into a modeling candidate image under the condition that the supplemental enhancement information of the current frame decoded by the target video accords with a target condition, and establishing a mapping model in real time by inputting the modeling candidate image and the position information of a corresponding aircraft into a modeling engine. The modeling efficiency can be improved, and the behavior of the mapping model is higher.

Specifically, the real-time acquisition device of the mapping model corresponding to the first device analyzes the modeling candidate image supplementary enhancement information according to the method as described above while acquiring the modeling candidate image in real time in step 202, and may acquire attitude information of aerial cameras such as a yaw angle of a camera pan, a pitch angle of the camera pan, a roll angle of the camera pan, and the like, in addition to acquiring the aircraft longitude, the aircraft latitude, and the aircraft altitude in the corresponding position. Then, the modeling candidate image, the position information of the aircraft and the attitude information of the aerial camera are input into a modeling engine to establish a mapping model in real time.

The embodiment of the invention is based on receiving a target video, converting the current frame into a modeling candidate image under the condition that the supplemental enhancement information of the current frame decoded by the target video accords with a target condition, and inputting the modeling candidate image, the position information of a corresponding aircraft and the gesture information of a aerial camera into a modeling engine to establish a mapping model in real time. The modeling efficiency can be improved, and the accuracy and the behavior of the mapping model are higher.

On the basis of any of the above embodiments, modeling is performed by using a modeling engine based on the modeling candidate image and target information carried in supplemental enhancement information corresponding to the modeling candidate image, to obtain a mapping model, including: and writing the position information of the aircraft carried in the supplementary enhancement information into the first information of the modeling candidate image.

Specifically, after the current frame is stored as a modeling candidate image, the real-time acquisition device of the mapping model corresponding to the first device writes the position information of the aircraft carried in the supplemental enhancement information of the current frame into the modeling candidate image in the form of first information so as to record attribute information and shooting data of the image.

The first information includes, but is not limited to, exchangeable image file format (Exchangeable image file format, exif) information modeling the candidate image.

Preferably, after the current frame is stored as the modeling candidate image, the aircraft longitude, the aircraft latitude and the aircraft altitude carried by the SEI information of the current frame are written into the Exif information of the modeling candidate image, so that after receiving the modeling candidate image carrying the Exif information, various modeling engines can acquire the Exif information in the image according to a general analytic function.

And writing the attitude information of the aerial camera carried in the supplementary enhancement information into the second information of the modeling candidate image.

Specifically, after the current frame is stored as a modeling candidate image, the real-time acquisition device of the mapping model corresponding to the first device writes the position information of the aircraft carried in the supplemental enhancement information of the current frame into the modeling candidate image in the form of first information, and writes the attitude information of the aerial camera carried in the supplemental enhancement information of the current frame into the modeling candidate image in the form of second information, so as to record the attribute information and shooting data of the image.

The second information includes, but is not limited to, extensible metadata platform (Extensible Metadata Platform, XMP) information modeling the candidate image. The XMP information is used to store metadata of a certain file as a template and then import the metadata into a modeling candidate image.

Preferably, after the current frame is stored as a modeling candidate image, after the aircraft longitude, the aircraft latitude and the aircraft altitude carried by the SEI information of the current frame are written into the Exif information of the modeling candidate image, the camera pan yaw angle, the camera pan pitch angle and the camera pan roll angle are also written into the XMP information of the modeling candidate image, and the modeling engine is selected according to actual modeling requirements, so that the modeling engine can acquire the Exif information in the image according to a general analytic function and can acquire the XMP information in the image by utilizing a built-in specific analytic function.

And inputting the modeling candidate images into a modeling engine for modeling, and obtaining a mapping model.

Before step 203, the user needs to select a corresponding modeling engine with the modeling task as a guide.

Specifically, the real-time acquisition device of the mapping model corresponding to the first device inputs modeling candidate images carrying the first information and the second information into a pre-selected modeling engine, and the modeling engine outputs the mapping model.

Optionally, if the modeling task is to build a mapping model under a standard geographic space, any modeling engine may be selected, exif information in the modeling candidate images is obtained through a general analytical function in the modeling engine, the corresponding modeling candidate images are matched and spliced, and the longitude of the aircraft, the latitude of the aircraft and the altitude of the aircraft are resolved from the Exif information to be incorporated into a unified geographic space.

Optionally, if the modeling task is to build a live-action 3D mapping model, a specific modeling engine may be selected, exif information in a modeling candidate image is obtained through a general analytical function in the modeling engine, XMP information in the modeling candidate image is obtained through the specific analytical function in the modeling engine, the corresponding modeling candidate image is matched and spliced, affine transformation is performed by using a camera pan-tilt yaw angle, a camera pan-tilt angle and a camera pan-tilt roll angle which are resolved in the XMP information, and aircraft longitude, aircraft latitude and aircraft altitude are resolved by using the Exif information, so that the modeling candidate image is incorporated into a unified geographic space.

It can be appreciated that, according to the selected underlying analytical function of the modeling engine, the second information of the modeling candidate image may be correspondingly set to be other information besides XMP information.

The embodiment of the invention is based on the fact that the position information of the aircraft and the attitude information of the aerial camera are respectively written into the first information and the second information of the modeling candidate image, and the modeling candidate image is input into a modeling engine to obtain a mapping model. The modeling process steps can be simplified, and the expandability is improved.

On the basis of any one of the above embodiments, the target condition includes that the target content is stored in the supplemental enhancement information, and that the verification content of the target condition is true.

Specifically, before step 202, the real-time acquisition device of the mapping model corresponding to the first device receives input information in response to a user, and preset target conditions.

The target conditions are not particularly limited in the embodiment of the present invention.

Alternatively, the target condition may be that the bytes stored in the respective pieces of memory space of the SEI information are not empty.

Illustratively, "0" or "1" is stored at each of bytes 1-52 of the SEI information.

Optionally, the target condition may be that the logical comparison result of the storage contents of the memory space corresponding to the check bit of the SEI information is true.

Illustratively, the check means corresponding to the check contents stored in the 49 th-52 th byte of the SEI information includes, but is not limited to, any one of an odd check, an even check, an accumulation and check, or a cyclic code redundancy check.

Alternatively, the target condition may be that the logical comparison result of the information length of the SEI information and the preset threshold is true.

The embodiment of the invention sets the target condition based on the fact that the target content is stored in the supplementary enhancement information and the verification content of the target condition is true, further converts the current frame meeting the target condition into a modeling candidate image, and establishes a mapping model in real time by inputting the modeling candidate image, the position information of the corresponding aircraft and the gesture information of the aerial camera into a modeling engine. The modeling efficiency can be improved, and the accuracy and the behavior of the mapping model are higher.

Fig. 3 is a schematic structural diagram of a real-time acquisition device of a mapping model provided by the invention. As shown in fig. 3, on the basis of any one of the above embodiments, the real-time acquisition device of a mapping model includes: a receive information module 310, a video encoding module 320, and a video transmitting module 330, wherein:

the receiving information module 310 is configured to receive, in each period, a video signal collected in real time by a aerial camera mounted on the aircraft, and target information corresponding to the period.

The video encoding module 320 is configured to write the target information into the supplemental enhancement information of the current frame in the video signal, and generate a target video.

The video sending module 330 is configured to send the target video to the first device in real time, so that the first device intercepts the modeling candidate image from the target video, and performs modeling by using the modeling engine based on the modeling candidate image, to obtain the mapping model.

Specifically, the reception information module 310, the video encoding module 320, and the video transmitting module 330 are electrically connected in sequence.

The receiving information module 310 receives the video signals collected from the aerial camera in real time for earth observation at each period, and records the position information of the corresponding aircraft at the moment of collecting each video signal.

The video encoding module 320 encodes the video RAW data (i.e., video signal) acquired in the reception information module 310 into a compressed format h.264 or h.265, and an encoder thereof writes target information into supplemental enhancement information (Supplemental Enhancement Information, SEI) of the video stream, resulting in a compressed target video.

The video transmitting module 330 transmits the target video generated by the video encoding module 320 to the first device through a wireless communication technology.

Optionally, the target information further includes attitude information of the aerial camera.

Preferably, the real-time acquisition means of the mapping model is a second device performing aerial photography tasks in air.

The real-time acquisition device for the mapping model provided by the embodiment of the invention is used for executing the real-time acquisition method for the mapping model, the implementation mode of the real-time acquisition device for the mapping model is consistent with the implementation mode of the real-time acquisition method for the mapping model provided by the invention, and the same beneficial effects can be achieved, and the description is omitted here.

Fig. 4 is a second schematic structural diagram of the real-time acquisition device for mapping models provided by the present invention. As shown in fig. 4, on the basis of any one of the above embodiments, the real-time acquisition device of a mapping model includes: a video receiving module 410, an image acquisition module 420 and a modeling module 430,

Wherein:

the video receiving module 410 is configured to receive the target video sent by the second device in real time.

The image obtaining module 420 is configured to convert the current frame into a modeling candidate image if it is analyzed from the target video that the supplemental enhancement information of each current frame meets the target condition.

The modeling module 430 is configured to perform modeling by using a modeling engine based on the modeling candidate image and target information carried in the supplemental enhancement information corresponding to the modeling candidate image, and obtain a mapping model.

Specifically, the video receiving module 410, the image obtaining module 420, and the modeling module 430 are electrically connected in sequence.

The video receiving module 410 receives, through a wireless communication technology, a target video from a real-time acquisition device of a mapping model corresponding to the second device.

The image acquisition module 420 sequentially reads each frame of the target video in order, analyzes the currently read frame as a current frame, judges the analyzed supplemental enhancement information and preset target conditions, and stores the current frame as a picture under the condition that judgment is successful, so that the picture is used as a modeling candidate image.

The modeling module 430 uses the modeling candidate images acquired in real time in the image acquisition module 420 and the target information carried in the supplemental enhancement information of each modeling candidate image as input information, selects a modeling engine corresponding to the input information, inputs the modeling engine into the modeling engine for modeling, and builds and expands a complete mapping model in real time.

Optionally, the modeling module 430 includes: the device comprises a first writing unit, a second writing unit and a modeling unit, wherein:

and the first writing unit is used for writing the position information of the aircraft carried in the supplementary enhancement information into the first information of the modeling candidate image.

And the second writing unit is used for writing the gesture information of the aerial camera carried in the supplementary enhancement information into the second information of the modeling candidate image.

And the modeling unit is used for inputting the modeling candidate image into a modeling engine to perform modeling and obtain a mapping model.

Optionally, the target condition includes that the target content is stored in the supplemental enhancement information, and that the verification content of the target condition is true.

Preferably, the real-time acquisition means of the mapping model is a first device performing modeling tasks on the ground.

Fig. 5 is a schematic structural diagram of a real-time acquisition system for a mapping model provided by the invention. As shown in fig. 5, the real-time acquisition system of a mapping model according to any of the above embodiments includes one or more real-time acquisition devices 510 of a mapping model and one or more real-time acquisition devices 520 of a mapping model.

Specifically, the real-time acquisition system of the mapping model includes a real-time acquisition device 510 (i.e., the second apparatus) of the mapping model on board the aircraft, and a real-time acquisition device 520 (i.e., the first apparatus) of the mapping model on board the aircraft.

The number of real-time acquisition devices 510 and 520 for the mapping model on the aircraft and the mapping model on the ground in the embodiments of the present invention is not particularly limited.

Optionally, in the actual geographic space, a real-time acquisition device 520 (i.e., the first apparatus) of the mapping model is disposed in a region, around which a ground control center is formed. The mapping model is obtained by receiving real-time target video of the geographic area that is observed in real-time by the real-time acquisition device 510 (i.e., the second device) of one or more mapping models, so as to intercept modeling candidate images, and inputting the modeling candidate images to the modeling engine.

Optionally, in the actual geographic space, a plurality of real-time acquisition devices 520 (i.e., the first apparatus) of mapping models are correspondingly disposed in a plurality of regions. The real-time acquisition device 510 (i.e. the second device) for receiving different mapping models performs real-time earth observation on the target video, so as to intercept modeling candidate images belonging to the area, and inputs the modeling candidate images to the modeling engine to acquire the mapping models.

Fig. 6 illustrates a physical schematic diagram of an electronic device, as shown in fig. 6, which may include: processor 610, communication interface (Communications Interface) 620, memory 630, and communication bus 640, wherein processor 610, communication interface 620, and memory 630 communicate with each other via communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform a method of real-time acquisition of a mapping model, the method comprising: receiving video signals acquired in real time by a aerial camera carried by an aircraft and target information corresponding to the period in each period; writing the target information into the supplemental enhancement information of the current frame in the video signal to generate a target video; the method comprises the steps of sending a target video to first equipment in real time, enabling the first equipment to intercept modeling candidate images from the target video, and modeling by using a modeling engine based on the modeling candidate images to obtain a mapping model; wherein the target information comprises at least position information of the aircraft. A method of acquiring a mapping model in real time may also be performed, the method comprising: receiving a target video sent by a second device in real time; under the condition that the supplemental enhancement information of each current frame is analyzed from the target video to meet the target condition, converting the current frame into a modeling candidate image; modeling is carried out by using a modeling engine based on the modeling candidate image and target information carried in the supplementary enhancement information corresponding to the modeling candidate image, so as to obtain a mapping model; wherein the target information comprises at least position information of the aircraft.

Further, the logic instructions in the memory 630 may be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In another aspect, the present invention also provides a computer program product comprising a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of executing a method of acquiring a mapping model provided by the methods described above, the method comprising: receiving video signals acquired in real time by a aerial camera carried by an aircraft and target information corresponding to the period in each period; writing the target information into the supplemental enhancement information of the current frame in the video signal to generate a target video; the method comprises the steps of sending a target video to first equipment in real time, enabling the first equipment to intercept modeling candidate images from the target video, and modeling by using a modeling engine based on the modeling candidate images to obtain a mapping model; wherein the target information comprises at least position information of the aircraft. A method of acquiring a mapping model in real time may also be performed, the method comprising: receiving a target video sent by a second device in real time; under the condition that the supplemental enhancement information of each current frame is analyzed from the target video to meet the target condition, converting the current frame into a modeling candidate image; modeling is carried out by using a modeling engine based on the modeling candidate image and target information carried in the supplementary enhancement information corresponding to the modeling candidate image, so as to obtain a mapping model; wherein the target information comprises at least position information of the aircraft.

In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform a method of real-time acquisition of a mapping model provided by the methods described above, the method comprising: receiving video signals acquired in real time by a aerial camera carried by an aircraft and target information corresponding to the period in each period; writing the target information into the supplemental enhancement information of the current frame in the video signal to generate a target video; the method comprises the steps of sending a target video to first equipment in real time, enabling the first equipment to intercept modeling candidate images from the target video, and modeling by using a modeling engine based on the modeling candidate images to obtain a mapping model; wherein the target information comprises at least position information of the aircraft. A method of acquiring a mapping model in real time may also be performed, the method comprising: receiving a target video sent by a second device in real time; under the condition that the supplemental enhancement information of each current frame is analyzed from the target video to meet the target condition, converting the current frame into a modeling candidate image; modeling is carried out by using a modeling engine based on the modeling candidate image and target information carried in the supplementary enhancement information corresponding to the modeling candidate image, so as to obtain a mapping model; wherein the target information comprises at least position information of the aircraft.

The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for acquiring a mapping model in real time, comprising:

2. The method of claim 1, wherein the target information further comprises pose information of the aerial camera.

3. A method for acquiring a mapping model in real time, comprising:

receiving a target video sent by a second device in real time;

4. The method of real-time acquisition of a mapping model of claim 3, wherein the target information further comprises attitude information of a aerial camera.

5. The method according to claim 4, wherein the obtaining a mapping model based on the modeling candidate image and the target information carried in the supplemental enhancement information corresponding to the modeling candidate image by modeling using a modeling engine includes:

6. The method of claim 3, wherein the target condition includes the target content being present in the supplemental enhancement information, and the verification content of the target condition is true.

7. A real-time acquisition device of a mapping model, characterized by comprising:

8. A real-time acquisition device of a mapping model, characterized by comprising:

9. A real-time acquisition system of a mapping model, characterized by comprising one or more real-time acquisition devices of a mapping model according to claim 7 and one or more real-time acquisition devices of a mapping model according to claim 8.

10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method for real-time acquisition of a mapping model as claimed in any one of claims 1 to 6 when the program is executed by the processor.

11. A non-transitory computer readable storage medium having stored thereon a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the real-time acquisition method of a mapping model according to any one of claims 1 to 6.

12. Computer program product comprising a computer program, characterized in that the computer program, when executed by a processor, implements the steps of a method for real-time acquisition of a mapping model according to any one of claims 1 to 6.