CN114244904A

CN114244904A - Remote sensing data transmission method, device, equipment and medium

Info

Publication number: CN114244904A
Application number: CN202111539167.1A
Authority: CN
Inventors: 徐伟; 包承鸣; 姜凯英
Original assignee: Ping An Technology Shenzhen Co Ltd
Current assignee: Ping An Technology Shenzhen Co Ltd
Priority date: 2021-12-15
Filing date: 2021-12-15
Publication date: 2022-03-25
Anticipated expiration: 2041-12-15
Also published as: CN114244904B

Abstract

The invention relates to the technical field of data transmission, and provides a method, a device, equipment and a medium for transmitting remote sensing data, wherein the method determines the request priority of a plurality of remote sensing data requests, determines the current request to be processed according to the request priority of each remote sensing data request, and acquires the remote sensing data to be transmitted according to the remote sensing data identifier in the current request to be processed and transmits the remote sensing data.

Description

Remote sensing data transmission method, device, equipment and medium

Technical Field

The invention relates to the technical field of data transmission, and provides a method, a device, equipment and a medium for transmitting remote sensing data.

Background

With the development of remote sensing satellite technology, remote sensing data has become an important data source for acquiring and analyzing geographic information, and the ecological state of a certain area, such as the growth of crops, the vegetation coverage and the like, is obtained through analysis of the remote sensing data. The processing of the remote sensing data often depends on a plurality of remote sensing models, and the remote sensing data is processed through the plurality of remote sensing models to obtain a required result.

However, the remote sensing model usually has ultra-large scale data processing requirements, and the data scale reaches hundreds of T or even larger. The method is limited by the requirements of algorithm environment and computing power, the design idea of mobile computing in a general big data system cannot be adopted in some systems, and only data can be transmitted to a computing host for computing. Because the bandwidth of data transmission is usually limited, the transmission sequence of remote sensing data has a great influence on the service efficiency of a plurality of remote sensing models and the overall data processing efficiency, and data transmission scheduling of the remote sensing data in the related technology often causes some remote sensing models to suspend data processing due to the fact that new remote sensing data are not obtained in time, the service processing efficiency of the overall remote sensing model is influenced, the service processing efficiency is low, and the real-time performance of data transmission is low.

Disclosure of Invention

The invention provides a method, a device, equipment and a medium for transmitting remote sensing data, and mainly aims to provide a method for solving the technical problems that data transmission scheduling of the remote sensing data in the related technology often causes data processing to be suspended due to the fact that some remote sensing models do not acquire new remote sensing data in time, service processing efficiency of the whole remote sensing model is affected, service processing efficiency is low, and data transmission real-time performance is low.

In order to achieve the above object, the present invention provides a method for transmitting remote sensing data, comprising:

obtaining a plurality of remote sensing data requests of a plurality of remote sensing models, wherein the remote sensing data requests comprise request time, request data size, request influence factors and remote sensing data identifiers;

evaluating the request priority of each remote sensing request according to the current time and the remote sensing data request;

processing the remote sensing data requests according to the request priority in sequence to obtain to-be-processed data identifications of the current to-be-processed requests, wherein the to-be-processed data identifications comprise the remote sensing data identifications of the current to-be-processed requests;

and acquiring remote sensing data to be transmitted corresponding to the data identifier to be processed, and executing transmission.

Optionally, before the request priority of each remote sensing request is evaluated according to the current time and the remote sensing data request, the method further includes:

determining a plurality of suspected repeated request sets from each remote sensing data request, wherein the suspected repeated request sets comprise at least two remote sensing data requests with the same remote sensing data identifier;

respectively acquiring a remote sensing model corresponding to each remote sensing data request in the suspected repeated request set;

and if at least two remote sensing data requests in the suspected repeated request set correspond to the same remote sensing model, deleting other remote sensing data requests except a target request, wherein the target request is one remote sensing data request with the earliest request time in each remote sensing data request in the suspected repeated request set.

Optionally, the generating manner of the remote sensing data request includes:

obtaining an initial data request sent by a remote sensing model, wherein the initial data request comprises request time and a remote sensing data identifier, determining the size of request data according to the remote sensing data identifier, determining the request influence factor according to at least one of the remote sensing data identifier and the remote sensing model, and generating the remote sensing data request;

or the like, or, alternatively,

the method comprises the steps of obtaining a remote sensing data identifier of remote sensing data required by a remote sensing model, determining the size of request data according to the remote sensing data identifier, obtaining a preset data influence factor corresponding to the remote sensing data identifier and a preset model influence factor corresponding to the remote sensing model, determining the request influence factor according to at least one of the preset data influence factor and the preset model influence factor, and generating a remote sensing data request by the remote sensing model according to the remote sensing data identifier, request time, the request influence factor and the request data size. Optionally, the determining manner of the request priority includes:

wherein, P is the request priority, I is the request influence factor, the value is [0,1], T is the request time, S is the request data size, and T is the current time.

Optionally, the method further includes:

acquiring the occurrence frequency of each remote sensing data identifier in each remote sensing data request;

determining the request priority according to the occurrence times, the current time, the request data size and the request influence factor;

in this case, the determining of the request priority includes:

wherein, P is the request priority, I is the request influence factor, the value is [0,1], T is the request time, S is the request data size, T is the current time, and F is the occurrence number.

Optionally, the determining manner of the current pending request includes:

acquiring the current data transmission speed of a current data transmission channel, and determining the size of the current transmittable data according to preset time and the current data transmission speed;

generating a request queue according to the request priority from high to low for each remote sensing data request;

according to the sequence of each remote sensing data request in the request queue, accumulating the request data size corresponding to each remote sensing data request one by one to obtain the sum of a plurality of request data sizes in sequence, taking the sum of the request data sizes corresponding to the remote sensing data request with the largest quantity as the sum of the current request data sizes, and taking the sum of the request data sizes which is one less than the quantity of the remote sensing data requests corresponding to the sum of the current request data sizes as the sum of the previous request data sizes;

and if the sum of the current request data size exceeds the current transmittable data size, taking the remote sensing data request corresponding to the sum of the previous request data size as the current request to be processed.

Optionally, the determining manner of the current pending request includes:

determining a priority processing remote sensing data request set, wherein the priority processing remote sensing data request set comprises the first N remote sensing data requests which are sequenced from high to low according to the request priority, and the sum of the request data sizes of the N remote sensing data requests is determined;

acquiring a first data size and a second data size, wherein the first data size is a request data size corresponding to the remote sensing data request with the lowest request priority in the priority processing remote sensing data request set, and the second data size is a request data size of an N +1 th remote sensing data request which is ordered from high to low according to the request priority;

if the sum of the request data size is larger than the current transmittable data size, determining a transmission data balance, if the transmission data balance is larger than the first data size, removing the Nth remote sensing data request in the priority processing remote sensing data request set and the (N-1) th remote sensing data request in the priority processing remote sensing data request set, taking the residual remote sensing data requests in the priority processing remote sensing data request set as a new priority processing remote sensing data request set, determining the sum of new request data until the sum of the request data is smaller than or equal to the current transmittable data size, and taking the remote sensing data requests in the priority processing remote sensing data request set as current requests to be processed;

if the sum of the request data is smaller than or equal to the size of the current transmittable data, determining a transmission data balance, if the transmission data balance is larger than the second data size, determining an inserting data request from the remote sensing data requests according to the sequence of request priorities from high to low and according to the transmission data balance, wherein the ratio of the size of the request data of the inserting data request to the transmission data balance is higher than a preset ratio, and taking the inserting data request and the remote sensing data request in a remote sensing data request set to be processed preferentially as the current request to be processed;

if the sum of the request data is smaller than or equal to the size of the current transmittable data, determining the balance of the transmission data, if the balance of the transmission data is smaller than the size of the second request data, determining an empty insertion data request from other remote sensing data requests except the remote sensing data request in the remote sensing data request set according to the transmission data balance according to the request priority sequence, wherein the ratio of the size of the request data of the empty insertion data request to the balance of the transmission data is higher than the preset ratio, and taking the (N + 1) th remote sensing data request, the empty insertion data request and the remote sensing data request in the remote sensing data request set which is processed preferentially as the current request to be processed.

Optionally, the method further includes:

acquiring remote sensing models corresponding to the remote sensing data requests, and acquiring data processing average duration corresponding to the remote sensing models;

obtaining the last request time which is the request time closest to the current time in each remote sensing data request corresponding to the remote sensing model, and determining the unrequested duration according to the last request time and the current time;

and if the unsolicited time exceeds a preset time threshold, sending early warning information, wherein the early warning information comprises the unsolicited time and the model identification of the remote sensing model.

In addition, to achieve the above object, the present invention also provides a remote sensing data transmission apparatus, including:

the remote sensing data acquisition module is used for acquiring a plurality of remote sensing data requests of a plurality of remote sensing models, and the remote sensing data requests comprise request time, request data size, request influence factors and remote sensing data identifiers;

the priority evaluation module is used for evaluating the request priority of each remote sensing request according to the current time and the remote sensing data request;

a to-be-processed data identifier obtaining module, configured to sequentially process the remote sensing data requests according to the request priority, and obtain a to-be-processed data identifier of a current to-be-processed request, where the to-be-processed data identifier includes the remote sensing data identifier of the current to-be-processed request;

and the transmission module is used for acquiring the remote sensing data to be transmitted corresponding to the data identifier to be processed and executing transmission.

Furthermore, to achieve the above object, the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the method according to any one of the above embodiments.

Furthermore, to achieve the above object, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the method according to any one of the above embodiments.

The method comprises the steps of obtaining a plurality of remote sensing data requests of a plurality of remote sensing models, evaluating request priorities of the plurality of remote sensing data requests according to current time and the remote sensing data requests, processing the remote sensing data requests according to the request priorities in sequence, obtaining remote sensing data to be transmitted according to remote sensing data identification in the current request to be processed and transmitting the remote sensing data, and selecting more urgent remote sensing data requests to respond under limited bandwidth, so that the service efficiency of the remote sensing models and the service processing efficiency of the whole remote sensing models can be improved, and the real-time performance of data transmission is improved.

Drawings

FIG. 1 is a schematic flow chart of a method for remote sensing data transmission according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating a method for transmitting remote sensing data according to an embodiment of the present invention;

FIG. 3 is a schematic flow chart illustrating another embodiment of a method for remote sensing data transmission according to the present invention;

FIG. 4 is a schematic flow chart illustrating another embodiment of a method for remote sensing data transmission according to the present invention;

FIG. 5 is a schematic diagram of an embodiment of a remote sensing data transmission device;

FIG. 6 is a schematic diagram of an architecture of a computing device according to an embodiment of the invention;

the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

In one embodiment, there is provided a method for remote sensing data transmission, as shown in fig. 1, the method comprising the steps of:

step S101: and obtaining a plurality of remote sensing data requests of a plurality of remote sensing models.

The remote sensing data request at least comprises request time, request data size, request influence factors and remote sensing data identification.

The multiple remote sensing data requests can be sent by the same remote sensing model or can be sent by multiple remote sensing models. When multiple remote sensing data requests are sent out by multiple remote sensing models, each remote sensing model can send out one or more remote sensing data requests.

In one embodiment, the manner of generating the remote sensing data request includes:

obtaining an initial data request sent by a remote sensing model, wherein the initial data request comprises request time and a remote sensing data identifier, determining the size of request data according to the remote sensing data identifier, determining a request influence factor according to at least one of the remote sensing data identifier and the remote sensing model, and generating the remote sensing data request;

or the like, or, alternatively,

the method comprises the steps of obtaining a remote sensing data identifier of remote sensing data required by a remote sensing model, determining the size of request data according to the remote sensing data identifier, obtaining a preset data influence factor corresponding to the remote sensing data identifier and a preset model influence factor corresponding to the remote sensing model, determining a request influence factor according to at least one of the preset data influence factor and the preset model influence factor, and generating a remote sensing data request by the remote sensing model according to the remote sensing data identifier, request time, the request influence factor and the request data size.

The remote sensing data request can be generated according to an initial data request sent by a remote sensing model, the initial data request comprises request time and a remote sensing data identifier, after the initial data request is received, the size of the request data is determined according to the remote sensing data identifier, and a request influence factor is determined according to at least one of the remote sensing data identifier and the remote sensing model. And generating a remote sensing data request according to the initial data request, the size of the request data, the remote sensing data identifier and the request influence factor.

The request time is the generation time of the remote sensing data request and can be determined by means of a timestamp of the remote sensing data request, the request data size is the data size of the remote sensing data targeted by the remote sensing data request, and if the data size of a certain remote sensing data is not a fixed size, the request data size can be determined by other rules specified by those skilled in the art according to needs, such as the maximum data size or the average data size of the data format.

The request influence factor can be determined according to the remote sensing data identification, the mapping relation between the category influence factor and the remote sensing data to be transmitted is formed by classifying the remote sensing data to be transmitted in advance and endowing the remote sensing data to be transmitted with the corresponding category influence factor, and the mapping relation between the remote sensing data identification and the category influence factor can be obtained at the moment because the remote sensing data identification and the remote sensing data to be transmitted are in one-to-one correspondence. The category impact factor may then be used as the data impact factor.

The category influence factor may also be determined according to a data source of the remote sensing data to be transmitted (which satellite collected), a geographic location corresponding to the data (which geographic area is targeted), and the like. When the request influence factor is determined according to the geographical position corresponding to the remote sensing data to be transmitted, the request influence factor can be determined through the preset weight of the geographical position and the last-period data expression of the geographical position. The higher the preset weight of the geographical location where the relevant staff is more concerned, such as when analyzing the growth of crops through remote sensing data, the preset weight of the relevant area of the northeast plain may be higher than the preset weight of the southwest mountain land. If the growth vigor of the crops in the previous period in a certain area is not good and does not reach the expected growth vigor, the data in the previous period in the geographical position of the area is not good in performance, and the data influence factor is increased. The specific values can be set by those skilled in the art as desired.

The request influence factor can also be determined according to the source (remote sensing model) of the remote sensing data request, different model influence factors are given to different remote sensing models, and the model influence factor is used as the request influence factor of the remote sensing data request sent by the remote sensing model. Some remote sensing models are basic models, a plurality of subsequent remote sensing models need to be further processed by means of data processed by the remote sensing models, and the request influence factor of the remote sensing models can be set higher, so that the remote sensing data requests of the remote sensing models can be processed more quickly.

The request impact factor can also be determined by combining the remote sensing model and the remote sensing data identification, and the data impact factor based on the remote sensing data identification and the model impact factor of the remote sensing model can be determined respectively, for example, the data impact factor and the model impact factor are weighted and averaged, so as to obtain the final request impact factor.

The telemetry data request may be generated directly from the telemetry model. The mapping relation between the category influence factor and the remote sensing data to be transmitted can be obtained for each remote sensing model, when the remote sensing model generates a remote sensing data request, the request influence factor is obtained in advance according to the required remote sensing data identification, and then the remote sensing data request is generated based on the request influence factor, the remote sensing data identification, the request time and the request data size.

Note that the request impact factor is greater than zero and less than or equal to 1.

In some embodiments, after step S101, before step S102, see fig. 2, the method further comprises:

step S201: determining a plurality of suspected repeated request sets from each remote sensing data request;

the suspected repeated request set comprises at least two remote sensing data requests with the same remote sensing data identifier;

step S202: respectively obtaining a remote sensing model corresponding to each remote sensing data request in the suspected repeated request set;

step S203: and if at least two remote sensing data requests in the suspected repeated request set correspond to the same remote sensing model, deleting other remote sensing data requests except the target request.

The target request is a remote sensing data request with the earliest request time in all remote sensing data requests in the suspected repeated request set.

Therefore, resource waste caused by repeated data transmission or repeated data processing due to repeated remote sensing data request transmission of the remote sensing model can be avoided.

And if at least two remote sensing data requests do not exist in the suspected repeated request set and correspond to the same remote sensing model, reserving each remote sensing data request.

Step S102: and evaluating the request priority of each remote sensing request according to the current time and the remote sensing data request.

In one embodiment, the request priority is determined by:

It is understood that, in this case, the priority of the request is considered to be the influence weight of the remote sensing data request itself, the waiting time of the request, and the size of the requested data, based on which the processing order of the plurality of remote sensing data requests is determined. That is, the smaller the request data size is, the longer the waiting time is, and the larger the request impact factor is, the priority will be given to the remote sensing data request.

In another embodiment, the request priority is determined by:

determining request priority according to the occurrence times, the current time, the request data size and the request influence factor;

in this case, the determination of the request priority includes:

Because a plurality of remote sensing models possibly need the same remote sensing data, the request priority of the remote sensing data request can be judged together according to the occurrence frequency of each remote sensing data identifier in different remote sensing data requests, the current time, the request data size and the request influence factor. Therefore, if a plurality of remote sensing models need the same remote sensing data to be transmitted at a certain moment, the remote sensing data to be transmitted can be transmitted preferentially, and the data transmission efficiency is improved.

By the implementation mode, the remote sensing data requests can be sorted based on the request priority, and the current to-be-processed request can be determined conveniently.

Step S103: and processing the remote sensing data requests according to the request priority in sequence to obtain the to-be-processed data identification of the current to-be-processed request.

The to-be-processed data identification comprises a remote sensing data identification of the current to-be-processed request.

By the method, the part of the remote sensing data requests which are processed preferentially can be determined from the plurality of remote sensing data requests through the order of the request priority, so that the normal operation of most remote sensing models can be met, and when the data transmission capacity cannot meet the data requirements of all the remote sensing models, the data requirements of more and more important remote sensing models can be met as far as possible.

The current to-be-processed requests can be one or more, the remote sensing data identification of the remote sensing data request corresponding to each current to-be-processed request is used as the to-be-processed data identification, corresponding remote sensing data can be obtained based on the to-be-processed data identification, data transmission is carried out, and the data transmission is transmitted to the corresponding remote sensing model.

In one embodiment, referring to fig. 3, determining the current pending request from the requests for telemetry data according to the request priority comprises:

step S301: acquiring the size of currently transmittable data;

wherein, the determination mode of the size of the current transmittable data comprises the following steps:

and acquiring the current data transmission speed of the current data transmission channel, and determining the size of the current transmittable data according to the preset time and the current data transmission speed. The preset time is a time set by a person skilled in the art. If the transmission speed at the current time is too low, for some larger data, the transmission time is too long, the working efficiency of other waiting remote sensing data models is greatly influenced, when the transmission speed of the current data is lower, the smaller data can be transmitted firstly to meet the data requirements of more remote sensing models, and after the transmission condition is improved, the larger remote sensing data is transmitted when the transmission speed is higher. Step S302: each remote sensing data request generates a request queue from high to low according to the request priority;

step S303: accumulating the request data sizes corresponding to the remote sensing data requests one by one according to the sequence of the remote sensing data requests in the request queue to obtain the sum of the sizes of a plurality of request data in sequence;

step S304: taking the sum of the request data with the largest number of corresponding remote sensing data requests as the sum of the current request data sizes;

step S305: taking the sum of the request data sizes which is one less than the number of the remote sensing data requests corresponding to the sum of the current request data sizes as the sum of the previous request data sizes;

step S306: and if the sum of the current request data size exceeds the current transmittable data size, taking the remote sensing data request corresponding to the sum of the previous request data size as the current request to be processed.

The size of the currently transmittable data can be determined according to the amount of data that can be transmitted by the current system, or according to the size of the transmitted data preset by a person skilled in the art.

The request queue may include a plurality of requests for remote sensing data with the same request priority, and in this case, the requests may be sorted randomly or simply sorted according to the request time or the occurrence number or the order (from large to small) of the request influence factors.

In the request queue, the higher the request priority. And sequentially accumulating the request data sizes corresponding to the remote sensing data requests with higher request priority levels until the sum of the obtained current request data sizes exceeds the current transmittable data size, and taking the remote sensing data request corresponding to the sum of the previous request data sizes as the current request to be processed. In other words, if multiple pieces of remote sensing data can be transmitted simultaneously, at this time, multiple pieces of remote sensing data transmitted preferentially can be determined and transmitted according to the request priority and the occupied bandwidth of the remote sensing data. If the current transmittable data size is 100%, the remote sensing data requests with request data sizes of 30%, 25%, 15% and 10% are sequentially included in the request queue, and when the sum of the current request data sizes is (30% + 30% + 25% + 15% + 10% + 110%), and the sum of the previous request data sizes is (30% + 30% + 25% + 15% + 100%), the remote sensing data requests with request data sizes of 30%, 25% and 15% are respectively used as the current to-be-processed requests.

And accumulating the request data size of each remote sensing request data in sequence according to the sequence of the request queue, and at the moment, obtaining the sum of the new request data size when the request data size of one remote sensing request data is accumulated. And once the sum of the sizes of the obtained current request data is larger than the size of the current transmittable data, the sum of the sizes of the previous request data is a numerical value closest to the size of the current transmittable data, so that the remote sensing data request forming the sum of the sizes of the previous request data is directly used as the current request to be processed to process the requests preferentially.

And if the sum of the sizes of the previous request data is smaller than the size of the current transmittable data, determining the balance of the transmission data (the balance of the transmission data is the sum of the size of the current transmittable data and the size of the previous request data), searching the remote sensing data requests with the balance of the transmission data equal to or smaller than the balance of the transmission data from other remote sensing data requests except the current request to be processed according to the balance of the transmission data, and selecting one remote sensing data request with the highest request priority as the current request to be processed. Therefore, the bandwidth waste can be avoided, the transmission efficiency is improved, and more data requirements of the remote sensing model are met.

Optionally, in step S302, generating a request queue according to the request priority from high to low for each remote sensing data request includes:

if a newly-increased remote sensing data request is obtained, determining the newly-increased priority of the newly-increased remote sensing data request;

and inserting the newly-added remote sensing data request into the request queue according to the newly-added priority, and updating the request queue.

Through the mode, the method mainly embodies that the request queue is directly updated for the newly added remote sensing data request, so that the current to-be-processed request determined each time is selected according to a uniform standard. And when a new remote sensing data request exists, calculating the request priority of the remote sensing data request, inserting the new remote sensing data request into the corresponding position of the request queue according to the request priority, and updating the request queue to ensure the consistency of the sequencing standards in the request queue. Specifically, the newly added remote sensing data request and other currently existing remote sensing data requests may be reordered together to obtain a request queue, or the remote sensing data request may be inserted into the current request queue to generate a new request queue.

The implementation of the above approach requires a separate request queue to determine the current pending request. That is, an additional request queue needs to be maintained, and the maintenance of the current pending request can be completed only by updating the request queue.

Optionally, when there is a new remote sensing data request, the request queue may not be updated in real time, and only the current to-be-processed request is determined on the basis of the original queue, and the request priority ratio between the current to-be-processed request determined at this time and the new remote sensing data request is performed, and if the request priority of the current to-be-processed request is higher than the request priority of the new remote sensing data request, the current to-be-processed request is directly transmitted. And if the request priority of at least one current request to be processed is lower than the request priority of the newly added remote sensing data request, updating the request queue, and determining the current request to be processed through the updated request queue. Therefore, the calculation force can be further saved, and the execution efficiency is improved. The request queue is updated only when the request queue is necessary to be updated, otherwise, the request priority of the newly added remote sensing data request is low, and the request queue is updated once without adding a (preset number or preset time) remote sensing data request, which is not cost-effective.

In some embodiments, the determination of the current pending request may also be performed directly according to the request priority without making a request queue, which may specifically be seen in the following manner:

one way of determining the current pending request includes:

determining at least one priority request according to the request priority from high to low, and determining the sum of the request data of each priority request;

and if the sum of the request data is smaller than or equal to the size of the transmittable data and the balance of the transmission data is smaller than a preset balance threshold value, taking the priority request as the current request to be processed, wherein the balance of the transmission data is determined according to the sum of the request data and the size of the transmittable data.

The currently transmittable data size may be the maximum bandwidth that the current system can support to transmit data.

In this way, an appropriate current pending request may be determined based on the current transmittable data size.

In another embodiment, the determination of the current pending request comprises:

acquiring the size of currently transmittable data;

determining a priority processing remote sensing data request set, wherein the priority processing remote sensing data request set comprises the first N remote sensing data requests which are sequenced from high to low according to request priority, and determining the sum of the request data sizes of the N remote sensing data requests;

acquiring a first data size and a second data size, wherein the first data size is a request data size corresponding to a remote sensing data request with the lowest request priority in a remote sensing data request set to be processed preferentially, and the second data size is a request data size of an N +1 th remote sensing data request which is ordered from high to low according to the request priority;

if the sum of the request data size is larger than the current transmittable data size, determining a transmission data balance, if the transmission data balance is larger than the first data size, removing the Nth remote sensing data request in the remote sensing data request set and the (N-1) th remote sensing data request in the remote sensing data request set which are processed preferentially, taking the rest remote sensing data requests in the remote sensing data request set which are processed preferentially as a new remote sensing data request set which is processed preferentially, and determining the sum of new request data until the sum of the request data is smaller than or equal to the current transmittable data size, and taking the remote sensing data requests in the remote sensing data request set which are processed preferentially as a current request to be processed;

if the sum of the request data is smaller than or equal to the size of the current transmittable data, determining the balance of the transmission data, if the balance of the transmission data is larger than the size of the second data, determining an inserting data request from the remote sensing data removing request according to the balance of the transmission data from high to low according to the request priority, wherein the ratio of the size of the request data of the inserting data request to the balance of the transmission data is higher than the preset ratio, and taking the remote sensing data request in the inserting data request and the remote sensing data request set for priority processing as the current request to be processed;

Wherein N is greater than 0 and less than or equal to the number of requests for telemetry data. The value of N is sequentially taken from 1.

The value of N may be preset by a person skilled in the art, such as 1 or 2. The preset ratio may be a ratio set by a person skilled in the art, which is greater than 0 and less than or equal to 1.

In one embodiment, if the request data size is smaller than the current transmittable data size when N is 1, after the previous remote sensing data request is removed, the previous N remote sensing data requests ranked from high to low are determined again, and the sum of the request sizes of the N remote sensing data requests is determined until the sum of the request data sizes is larger than or equal to the current transmittable data size position. In other words, when N-1 is equal to 0, and the request data size of the previous remote sensing data request is larger than the current transmittable data size, then, starting from the previous 2 remote sensing data requests, the remote sensing data requests with the request data size smaller than the current transmittable data size are sequentially searched until a remote sensing data request position is found, if the remote sensing data request is the previous M remote sensing data, a certain number of remote sensing data requests including the previous M remote sensing data and the previous M remote sensing data are generated into a priority processing remote sensing data request set, the sum of the request data sizes of the N remote sensing data requests is sequentially obtained, if the sum of the request data sizes is larger than the current transmittable data size, the balance of the transmission data is determined, and if the balance of the transmission data is larger than the first data size, the nth remote sensing data request in the priority processing remote sensing data request set and the nth remote sensing data request in the priority processing remote sensing data request set are sequentially obtained Removing 1 remote sensing data request, taking the rest remote sensing data requests in the remote sensing data request set with priority processing as a new remote sensing data request set with priority processing, determining the sum of new request data until the sum of the request data is smaller than or equal to the size of currently transmittable data, and taking the remote sensing data requests in the remote sensing data request set with priority processing as current requests to be processed.

For example, the size of the currently transmittable data is 100%, and the sizes of the currently acquired remote sensing data request and the request data thereof are respectively a remote sensing data request a 105%, a remote sensing data request B101%, a remote sensing data request C80%, a remote sensing data request D60%, a remote sensing data request E15%, and a remote sensing data request F20%. Then, sorting the remote sensing data requests according to the request data size to obtain sorting results, namely a remote sensing data request A105%, a remote sensing data request B101%, a remote sensing data request C80%, a remote sensing data request D60%, a remote sensing data request F20% and a remote sensing data request E15%, wherein N is equal to 1, the sum of the request data sizes is 105%, the data is larger than the current transmittable data size by 100%, the remote sensing data request A is removed, N is equal to 1 again, the sum of the obtained new request data sizes is 101%, and the data is still larger than the current transmittable data size by 100%. And removing the remote sensing data request B, taking N as 1 again, obtaining a new request data with the sum of 80% (remote sensing data request C), wherein the data is smaller than 100% of the current transmittable data size, determining the balance of the transmitted data, (100% -80% ═ 20%), the second data size is 60%, the balance of the transmitted data (20%) is smaller than the second data size (60%), determining an empty data insertion request, and taking the remote sensing data request F as the empty data insertion request and taking the remote sensing data request F and the remote sensing data request C as the current to-be-processed requests, wherein the ratio of the remote sensing data request F (20%/100%) is larger than the ratio of the remote sensing data request E (15%/20%/75%). Therefore, the available bandwidth can be applied with the maximum efficiency by the mode, the data transmission efficiency is improved, more remote sensing models can obtain the required remote sensing data as much as possible in time, the working efficiency of the remote sensing models is improved, and the analysis of all the remote sensing data is completed more quickly.

For another example, the currently transmittable data size is 100%, and the currently acquired remote sensing data request and the requested data size thereof are respectively a remote sensing data request a 80%, a remote sensing data request B15%, a remote sensing data request C10%, and a remote sensing data request D5%. If N is 2, the remote sensing data request set which is preferentially processed at the moment comprises a remote sensing data request A and a remote sensing data request B, the sum of the sizes of the request data is 95%, the sum (95%) of the sizes of the request data is smaller than the size (100%) of the currently transmittable data, the balance (100% -95% ═ 5%) of the transmitted data is 15% (remote sensing data request B), the size of the second data is 10% (remote sensing data request C), the size of the balance of the transmitted data is smaller than the size of the second data, and the size of the request data of none of the remote sensing data requests is smaller than or equal to the balance of the transmitted data, the remote sensing data request A and the remote sensing data request B are directly used as the current request to be processed.

For another example, the currently transmittable data size is 100%, and the currently acquired remote sensing data requests and the requested data sizes thereof are respectively a 86% remote sensing data request, B5% remote sensing data request, C5% remote sensing data request and D3% remote sensing data request. If N is 3, the remote sensing data request set which is processed preferentially at the moment comprises a remote sensing data request A, a remote sensing data request B and a remote sensing data request C, the sum of the sizes of the request data is 96%, the sum (96%) of the sizes of the request data is smaller than the size (100%) of the current transmittable data, the balance (100% -96% ═ 4%) of the transmission data is 5% (remote sensing data request C) of the first data, the size of the second data is 3% (remote sensing data request D), and the size of the balance of the transmission data is larger than the size of the second data, so that the remote sensing data request A, the remote sensing data request B, the remote sensing data request C and the remote sensing data request D are directly used as the current request to be processed.

By the method, the current request to be processed can be determined more simply and conveniently. Compared with the method that the request data size with the highest request priority level is accumulated in an attempt mode one by one, the method has the advantages that the current to-be-processed request is obtained in a few operation modes by presetting the N value. In addition, when the residual capacity is not enough to meet the next remote sensing data request, other remote sensing data requests meeting the residual capacity are searched for being transmitted together, and waste of transmission resources can be reduced.

It should be noted that, if there are at least two requests with the same request priority, the requests may be sorted again based on any one of the request data size, the request impact factor, and the waiting time. For example, with the size of the request data as a dimension, the larger the request data is, the higher the priority of the processing order is. If the remote sensing data requests with the same request priority and the same request data size still exist, the waiting duration is continuously taken as the dimensionality, and the longer the waiting duration is, the more the processing sequence is prioritized. The specific sorting rule can be set by those skilled in the art, and is not limited herein. Wherein the waiting duration is the current time-the request time.

Through the implementation mode, the current request to be processed can be determined from the multiple remote sensing data requests through multiple modes, subsequent data transmission is facilitated, and the data requirement of a more important remote sensing algorithm can be met through adjustment of a certain data transmission rule when the data transmission capacity is insufficient.

Step S104: and acquiring remote sensing data to be transmitted corresponding to the data identifier to be processed, and executing transmission.

The remote sensing data to be transmitted can be obtained by searching in a preset database according to the remote sensing data identification and transmitted to the corresponding remote sensing model. The specific transmission mode can be realized by adopting the existing technical means. For example, according to the remote sensing data identification of the remote sensing data to be transmitted, the remote sensing data to be transmitted is distributed to the corresponding remote sensing model, and the like.

In one embodiment, referring to fig. 4, the method further comprises:

step S401: acquiring remote sensing models corresponding to the remote sensing data requests, and acquiring data processing average duration corresponding to the remote sensing models;

step S402: obtaining the last request time which is the request time closest to the current time in each remote sensing data request corresponding to the remote sensing model, and determining the unsolicited duration according to the last request time and the current time;

step S403: and if the unsolicited time length exceeds a preset time length threshold value, sending early warning information, wherein the early warning information comprises the unsolicited time length and a model identifier of the remote sensing model.

The remote sensing model corresponding to the remote sensing data request can be added with a remote sensing model identifier when the remote sensing model sends the remote sensing data request so as to determine the remote sensing model corresponding to the remote sensing data request.

And if the duration of the unsolicited data is less than the preset duration threshold, each remote sensing data request is processed normally.

The preset duration threshold may be a determined value, may be a corresponding preset duration threshold set for different remote sensing models, or may be a duration value set for all remote sensing models in a unified manner. The preset time threshold value can also be determined according to a certain rule (a preset formula) through the average time of data processing corresponding to each remote sensing model.

The determination mode of the average processing time can determine the average processing time according to the sizes of multiple groups of request data and the request time by acquiring remote sensing data requests which correspond to the remote sensing model within a preset time period and comprise different remote sensing data identifications and the sizes of the request data in each remote sensing data request.

In step S402, the latest request time may be a request time that is closest to the current time in the remote sensing data request and can be found after the current time is advanced by a preset duration. The preset duration may be a duration set by a person skilled in the art, or may be obtained according to several times of the maximum average duration in the average processing durations of the current remote sensing data. If a certain remote sensing model cannot obtain the latest request time, the model unsolicited duration can be set as a certain preset threshold to obtain a preset identifier, and the remote sensing model also needs to send early warning information.

Optionally, if the unsolicited duration exceeds the preset duration threshold, the sending the warning information further includes:

determining the overtime length according to the model unsolicited time length and the preset time length threshold, determining the early warning level according to the overtime length, and adding the early warning level into the early warning information.

The early warning level can be determined according to the overtime duration by setting the mapping relation of the early warning levels corresponding to different overtime durations.

Aiming at the steps S401-S403, the following of the data processing condition of the remote sensing model is mainly embodied, if the remote sensing model does not send a new remote sensing data request for a long time, the remote sensing model can be abnormal in operation, and early warning information can be sent to enable relevant workers to timely remove faults.

According to the remote sensing data transmission method provided by the embodiment, the request priority of the plurality of remote sensing data requests is determined, the current to-be-processed request is determined according to the request priority of each remote sensing data request, the remote sensing data to be transmitted is obtained according to the remote sensing data identification in the current to-be-processed request and is transmitted, and therefore the more urgent remote sensing data request can be selected to respond under the limited bandwidth, the service efficiency of the remote sensing model and the service processing efficiency of the whole remote sensing model can be improved, and the real-time performance of data transmission is improved.

By the method, the data transmission request with low data requirement and high real-time requirement can be processed preferentially, and other data requests cannot be executed all the time due to the fact that a large number of data requests with low real-time requirement are transmitted for a long time.

Remote sensing algorithms (remote sensing models) usually have ultra-large-scale data processing requirements, and the data scale reaches hundreds of T or even larger. The method is limited by the requirements of algorithm environment and computing power, the design idea of mobile computing in a general big data system cannot be adopted in some systems, and only data can be transmitted to a computing host for computing. Since the bandwidth of data transmission is usually limited, how to schedule the transmission sequence of data in this case has a great influence on the overall business efficiency of multiple remote sensing algorithms. If not scheduled properly, some algorithms may be caused to wait until they are late to read the data. The embodiment provides a remote sensing data transmission method, which comprehensively considers the data transmission sequence from multiple dimensions such as the response requirement of a business algorithm, the size of data required by the business algorithm (the size of requested data), the waiting time (the request time) and the like, thereby achieving the aim of improving the efficiency of the comprehensive algorithm. Specifically, an exemplary embodiment is as follows:

when a service algorithm (remote sensing model) needs a data request (remote sensing data request), the scheduling system determines the request priority of the remote sensing data request through the manner provided by the embodiment, the scheduling system additionally maintains a data request queue L, the data requests are stored in the data request queue L in the sequence of P on the basis of the current time, and the request with the high priority is at the front end of the queue. The scheduling system processes the data requests at the front end of the queue in sequence. When a new data request exists, sequentially recalculating the data requests (request priorities) in the queue according to the current time reference, and sequencing. After each data request is completed, the request priorities of all remote sensing data requests are recalculated by the method according to the current time reference, and are reordered into a queue. And the scheduling of the data request processing is realized through the control of the data request in the queue.

By the method, the data transmission request with low data requirement and high real-time requirement can be processed preferentially, and the problem that other data requests cannot be executed all the time due to the fact that a large amount of data requests with low real-time requirement are transmitted for a long time and the processing efficiency of the remote sensing data model is influenced is avoided.

In one embodiment, the present invention further provides a remote sensing data transmission apparatus 500, referring to fig. 5, comprising:

the obtaining module 501 is configured to obtain multiple remote sensing data requests of multiple remote sensing models, where the remote sensing data requests include request time, request data size, request impact factor, and remote sensing data identifier;

a priority evaluation module 502, configured to evaluate a request priority of each remote sensing request according to a current time and the remote sensing data request;

a to-be-processed data identifier obtaining module 503, configured to sequentially process the remote sensing data requests according to the request priority, and obtain a to-be-processed data identifier of a current to-be-processed request, where the to-be-processed data identifier includes the remote sensing data identifier of the current to-be-processed request;

and the transmission module 504 is configured to acquire the remote sensing data to be transmitted corresponding to the to-be-processed data identifier, and execute transmission. In one embodiment, the apparatus further comprises a duplication checking module comprising:

the repeated determining module is used for determining a plurality of suspected repeated request sets from the remote sensing data requests before determining the request priority of each remote sensing request according to the current time, the request data size and the request influence factor, wherein the suspected repeated request sets comprise at least two remote sensing data requests with the same remote sensing data identifier;

the corresponding model acquisition module is used for respectively acquiring the remote sensing models corresponding to the remote sensing data requests in the suspected repeated request set;

and the duplication deletion module is used for deleting other remote sensing data requests except the target request if at least two remote sensing data requests in the suspected duplication request set correspond to the same remote sensing model, wherein the target request is one remote sensing data request with the earliest request time in all the remote sensing data requests in the suspected duplication request set.

In one embodiment, the current pending request determination module includes:

the current transmissible data size obtaining module is used for obtaining the current transmissible data size;

the queue generating module is used for generating a request queue from high to low according to the request priority of each remote sensing data request;

the summation module is used for accumulating the request data sizes corresponding to the remote sensing data requests one by one according to the sequence of the remote sensing data requests in the request queue to obtain the sum of the sizes of a plurality of request data in sequence, taking the sum of the request data sizes corresponding to the remote sensing data request with the largest quantity as the sum of the current request data sizes, and taking the sum of the request data sizes which is one less than the quantity of the remote sensing data requests corresponding to the sum of the current request data sizes as the sum of the previous request data sizes;

and the current to-be-processed request determining submodule is used for taking the remote sensing data request corresponding to the sum of the sizes of the previous request data as the current to-be-processed request if the sum of the sizes of the current request data exceeds the size of the current transmittable data.

In another embodiment, the current pending request determination module includes:

the request set determining module is used for determining a remote sensing data request set to be processed preferentially, wherein the remote sensing data request set to be processed preferentially comprises the first N remote sensing data requests which are sequenced from high to low according to the request priority, and the sum of the request data sizes of the N remote sensing data requests is determined;

the data size obtaining module is used for obtaining a first data size and a second data size, wherein the first data size is a request data size corresponding to a remote sensing data request with the lowest request priority in a priority processing remote sensing data request set, and the second data size is a request data size of an N +1 th remote sensing data request which is ordered from high to low according to the request priority;

a transmission data balance determination first sub-module, which is used for determining the transmission data balance if the sum of the sizes of the request data is larger than the size of the current transmittable data, removing the Nth remote sensing data request in the remote sensing data request set and the (N-1) th remote sensing data request in the remote sensing data request set which are processed preferentially if the transmission data balance is larger than the size of the first data, taking the rest remote sensing data requests in the remote sensing data request set which are processed preferentially as a new remote sensing data request set which is processed preferentially, determining the sum of new request data until the sum of the request data is smaller than or equal to the size of the current transmittable data, and taking the remote sensing data requests in the remote sensing data request set which are processed preferentially as the current request to be processed;

a second transmission data balance determining submodule, configured to determine a transmission data balance if the sum of the request data is smaller than or equal to the size of the current transmittable data, determine an empty insertion data request from the remote sensing data request according to the transmission data balance in the order from high to low according to the request priority if the transmission data balance is larger than the second data size, where the proportion of the size of the request data of the empty insertion data request to the transmission data balance is higher than a preset proportion, and use the empty insertion data request and the remote sensing data request in the remote sensing data request set to be preferentially processed as a current request to be processed;

and the transmission data balance determination third sub-module is used for determining the transmission data balance if the sum of the request data is smaller than or equal to the size of the current transmittable data, determining an inserting empty data request from other remote sensing data requests except the remote sensing data request in the remote sensing data request set according to the transmission data balance according to the request priority order if the transmission data balance is smaller than the size of the second request data, wherein the ratio of the size of the request data of the inserting empty data request to the size of the transmission data balance is higher than the preset ratio, and taking the (N + 1) th remote sensing data request, the inserting empty data request and the remote sensing data request in the remote sensing data request set which is processed preferentially as the current request to be processed.

In one embodiment, the apparatus further comprises an early warning module comprising:

the time length obtaining module is used for obtaining the remote sensing models corresponding to the remote sensing data requests and obtaining the average time length of data processing corresponding to the remote sensing models;

the last request time obtaining module is used for obtaining the last request time which is the request time closest to the current time in each remote sensing data request corresponding to the remote sensing model, and determining the duration of the unsolicited time according to the last request time and the current time;

and the early warning information sending module is used for sending early warning information if the unsolicited time exceeds a preset time threshold, wherein the early warning information comprises the unsolicited time and the model identification of the remote sensing model.

The remote sensing data transmission device in the embodiment determines the request priority of the plurality of remote sensing data requests, determines the current to-be-processed request according to the request priority of each remote sensing data request, obtains the remote sensing data to be transmitted according to the remote sensing data identifier in the current to-be-processed request and transmits the remote sensing data, and can select the more urgent remote sensing data request to respond under limited bandwidth, so that the service efficiency of the remote sensing model and the service processing efficiency of the whole remote sensing model can be improved, and the real-time performance of data transmission is improved.

It should be understood that the message conversion apparatus is substantially provided with a plurality of modules for executing the remote sensing data transmission method in any of the embodiments, and specific functions and technical effects may be obtained by referring to the embodiments, which are not described herein again.

In an embodiment, referring to fig. 6, the embodiment further provides a computer device 700, which includes a memory 701, a processor 702, and a computer program stored on the memory and executable on the processor, and when the processor 702 executes the computer program, the steps of the method according to any one of the above embodiments are implemented.

In an embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any of the above embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, apparatus, article, or method that includes the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments. Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims

1. A method for remote sensing data transmission, the method comprising:

2. The method of remote sensing data transmission according to claim 1, wherein prior to said assessing a request priority of each of said remote sensing requests based on a current time and said remote sensing data request, said method further comprises:

3. The method for remote sensing data transmission according to claim 1, wherein said remote sensing data request is generated in a manner comprising:

or the like, or, alternatively,

the method comprises the steps of obtaining a remote sensing data identifier of remote sensing data required by a remote sensing model, determining the size of request data according to the remote sensing data identifier, obtaining a preset data influence factor corresponding to the remote sensing data identifier and a preset model influence factor corresponding to the remote sensing model, determining the request influence factor according to at least one of the preset data influence factor and the preset model influence factor, and generating a remote sensing data request by the remote sensing model according to the remote sensing data identifier, request time, the request influence factor and the request data size.

4. The method for remote sensing data transmission according to claim 3, wherein said request priority is determined by:

5. The method for remote sensing data transmission according to claim 1, further comprising:

in this case, the determining of the request priority includes:

6. The method for remote sensing data transmission according to any of claims 1-5, wherein the determination of the current pending request comprises:

7. The method for remote sensing data transmission according to any of claims 1-5, wherein the determination of the current pending request comprises:

8. The method for remote sensing data transmission according to any of claims 1-5, wherein the method further comprises:

9. A remote sensing data transmission apparatus, the apparatus comprising:

10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the method of any of claims 1 to 8 are implemented by the processor when executing the computer program.

11. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.