JP2021532497A - Visualization method, equipment and recording medium for multi-source earth observation image processing - Google Patents

Abstract

The present invention relates to a visualization method for multi-source earth observation image processing, an electronic device, and a recording medium. The method includes a step (S1) of acquiring a plurality of sub-algorithms corresponding to the processing flow based on the processing flow of the multi-source earth observation image, and a graphic object corresponding to each sub-algorithm based on the processing flow. A step (S2) for generating a graphic object model by associating the sub-algorithm with the graphic object by a model generator, and a step (S3) for generating a graphic object model, and connecting a plurality of graphic object models according to the processing flow. , A step (S4) of generating a visual model of the multi-source earth observation image, and a step (S5) of visualizing the processing process of the multi-source earth observation image by the visual model. The method realizes visualization of the processing process of the multi-source earth observation image, intuitively displays each processing step, and satisfies the requirement of visualization integration for the processing process of the multi-source earth observation image by the WYSIWYG method. [Selection diagram] Fig. 1

Description

This application is based on the Chinese patent application CN201910290454 filed on April 11, 2019. It claims the priority of X, and all the contents of the above-mentioned matter shall be incorporated into the present application by reference.

The present application relates to the technical field of big data, and particularly to visualization methods, instruments and recording media of multi-source earth observation image processing.

Visual modeling (VISUAL MODELING) is a method of thinking about problems by organizing a model centered on a realistic way of thinking, and is a process of explaining the developed system in a graphic manner. Visual modeling presents the required detail of a complex problem and allows you to filter out the unwanted detail. Visual modeling also provides a mechanism for observing the developed system from different perspectives.

Currently, common visual modeling software includes Unified Modeling Language (UML), VISIO, Simulink, Model Maker and Model Builder, and conventional visual platforms provide a graphical modeling user interface with visual modeling software. Although it can be done, conventional visual platforms use the WYSIWYG method for visuals of data, algorithms, flows and hidden information streams in the processing and mining process of earth observation data (including satellite images, structured data, etc.). It is difficult to integrate the processing, it is difficult to realize serial or parallel processing for the algorithm in the observation data processing process, and it is difficult to meet the demand for improving the debugging and development efficiency of visual processing.

The present application provides visualization methods, equipment and recording media for multi-source earth observation image processing in order to solve the problem that it is difficult to deal with the visual processing integration for the processing of earth observation data and the mining process in the prior art.

In order to achieve the above object, the visualization method of multi-source earth observation image processing according to one aspect of the present application acquires a plurality of sub-algorithms corresponding to the processing flow based on the processing flow of the multi-source earth observation image. A step of generating a graphic object corresponding to each sub-algorithm based on the processing flow, a step of associating the sub-algorithm with the graphic object by a model generator, and a step of generating a graphic object model. A step of connecting a plurality of graphic object models according to a processing flow to generate a visual model of the multi-source earth observation image, and a step of visualizing the processing process of the multi-source earth observation image by the visual model are included. ..

To achieve the above object, the electronic device according to another aspect of the present application includes a processor and a memory, which, when executed by the processor, is a step in the visualization method of the multi-source earth observation image processing. Includes a visualization program for multi-source earth observation image processing to achieve this.

In order to achieve the above object, a computer-readable recording medium according to a further aspect of the present application, when executed by a processor, realizes the steps of the visualization method of the multi-source earth observation image processing. A process visualization program is included.

Compared with the prior art, the present application has the following advantages and beneficial effects.

In this application, the visual model generated based on the processing flow of the multi-source earth observation image realizes the visualization of the processing process of the multi-source earth observation image, and the graphic object model in the visual model makes each step in the image processing intuitive. The WYSIWYG method meets the requirements for visualization integration for the processing process of multi-source earth observation images by improving the visibility of data processing.

In the present application, by associating a sub-algorithm with a graphic object, the editability for each sub-algorithm in the image processing process is improved, the flexibility of the combination of each graphic object model in the visual model is improved, and the development debugging efficiency is improved. It realizes design visualization, intermediate call visualization, and processing result visualization for image processing research, and can effectively meet the user's demands for high-level development and application.

It is a flowchart of the visualization method of the multi-source earth observation image processing which concerns on this application. It is a module schematic diagram of the visualization program of the multi-source earth observation image processing which concerns on this application.

The realization, functional features and advantages of the present application will be further described in combination with examples with reference to the drawings.

Hereinafter, examples of the present application will be described with reference to the drawings. One of ordinary skill in the art can modify the embodiments described by various different methods or combinations thereof without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are exemplary in nature and are intended only to illustrate the present application, not to limit the scope of the claims. Also, in this specification, the drawings are not drawn in proportion to the actual size, and the same reference numerals indicate the same parts.

FIG. 1 is a flowchart of a method for visualizing multi-source earth observation image processing according to the present application, and as shown in FIG. 1, the method for visualizing multi-source earth observation image processing according to the present application includes the following steps S1 to S5. ..

In step S1, a plurality of sub-algorithms corresponding to the processing flow are acquired based on the processing flow of the multi-source earth observation image.

The processing flow of a multi-source earth observation image can be conceptualized into a sequential instruction set formed by a series of modular, continuous and different image processing sets. Taking the remote sensing image as an example, the processing flow of the remote sensing image includes data input / output, image preprocessing (including geometric correction, fusion, mosaic, etc.), video information extraction (artificial interpretation, automatic classification, feature extraction, etc.). Processing steps such as (including dynamic detection, etc.), subject mapping / 3D visualization analysis (including integration of conventional data of geographic information system, etc.), and result report (including analysis and sharing of geographic information system, etc.) include. Each processing step in the processing flow corresponds to one or more sub-algorithms, and the combination of the sub-algorithms performs related processing and information extraction for the remote sensing image. Sub-algorithms can be obtained by querying the open source image processing algorithm repository.

In step S2, a graphic object corresponding to each sub-algorithm is generated based on the processing flow. Each graphic object represents one sub-algorithm and can use the same or different shapes. Specifically, the Graphics View framework in the cross-platform C ++ graphic user interface application framework (Qt) is used as the graphic user interface to generate a graphic object. The Graphics View framework includes a Diagram Scene, Diagram View and Diagram Item, a Diagram Scene is a visualization work area, a Diagram Item is a two-dimensional graphic box that can be placed in a work area, and a Diagram is a two-dimensional graphic box that can be placed in the work area. In the Diagram Scene work area, you can add shapes and texts, create connection direction lines, and the generated graphic object is a 2D graphic box added to the work area, 2D graphic. It is also possible to adjust the shape of the box and use the Binary Space Partitioning tree (BSP tree) to store the generated graphic objects.

In step S3, the model generator associates the sub-algorithm with the graphic object to generate a graphic object model. In a model language environment for a target, multiple flowcharts are intuitively drawn on one page scene, and each graphic object in the flowchart is associated with a sub-algorithm that represents data input, operation function, arithmetic rule, data output, etc. You can generate a spatial object model for your image processing steps.

The graphic object model can be executed, edited or stored in the model library and can be further edited by integrating different processing module sets. Different processing modules contain different sub-algorithms, providing great convenience for research testing and subsequent use. The graphic object model can be printed out as a flowchart or presented as an explanation for a research report. The model generator is provided with multiple operational operators such as data type conversion, image space area time area basic processing, image conversion, projection correction, feature extraction, change monitoring, etc., and raster data is provided by the graphic object model. , Vector data, classification data, etc. can be operated. In one embodiment, a visualization method for multi-source earth observation image processing is implemented based on a Qt cross-platform graphical interface and a spatial graphic object model is generated by using panel tools.

In step S4, a plurality of graphic object models are connected according to the processing flow, and a visual model of the multi-source earth observation image is generated. One visual model is composed of a series of graphic object models, and each graphical object model generated in step S3 is an individual independent of each other, and is one spatial model element, and each graphic object model is used. Only organic connections can form a complete visual model. Since the generated visual model processes the multi-source earth observation image, it is necessary to make regular connections between each graphic object model, and by connecting according to the processing flow of the multi-source earth observation image. , When the visual model is executed, the sub-algorithms in each graphic object model are sequentially executed according to the processing flow, and the operation functions of spatial geographic information and image processing are executed by each graphic object model. Connections can be achieved by placing connecting lines with arrows between each graphic object model, and the arrows indicate the direction of the data stream.

In step S5, the processing process of the multi-source earth observation image is visualized by the visual model.

The present application realizes visualization of the processing process of the multi-source earth observation image by the visual model generated based on the processing flow of the multi-source earth observation image, and each step of processing the image by the graphic object model in the visual model. It can be shown intuitively, the visibility of data processing can be improved, and the demand for visualization integration for the processing process of multi-source earth observation images can be satisfied by the WYSIWYG method.

In the present application, by associating a sub-algorithm with a graphic object, the editability for each sub-algorithm in the image processing process is improved, and by improving the flexibility of the combination of each graphic object model in the visual model, the development debugging efficiency is improved. It can realize design visualization, intermediate call visualization, and processing result visualization for image processing research, and can effectively meet the user's demand for high-level development and application.

In one preferred embodiment of the present application, the step of acquiring a plurality of sub-algorithms corresponding to the processing flow based on the processing flow of the multi-source earth observation image is processed based on the processing purpose of the multi-source earth observation image. Flow and processing A clear processing purpose includes a step of determining each substep in the flow and a step of querying the algorithm knowledge base and acquiring the subalgorithm corresponding to the substep based on the substep. The finalized processing flow, including preprocessing, image feature extraction, change monitoring, etc., is a specific method and necessary data support to achieve the processing objectives. For example, when the processing purpose of the multi-source earth observation image is the extraction of image features (roads, buildings, water bodies, etc.), the processing flow for extracting the image features is acquired based on this processing purpose. Taking building feature extraction in a remote sensing image as an example, the processing flow preprocesses the remote sensing image, performs edge detection on the remote sensing image acquired by the preprocessing, and performs linear information from the detected edge information. Including the step of acquiring the building features in the remote sensing image by extracting and extracting the rectangular building structure based on the linear information, preprocessing for the remote sensing image, edge detection, linear information extraction, and rectangular structure in the processing flow. Each extraction is one sub-step, and the sub-algorithm corresponding to the sub-algorithm is queried by querying the algorithm knowledge base, and the related processing function is realized by the sub-algorithm.

Preferably, the step of associating the sub-algorithm with the graphic object sets a step of determining the filter type of the graphic object based on the sub-algorithm and an attribute parameter of the graphic object based on the filter type. , A step of determining a filter corresponding to the graphic object, and a step of associating the sub-algorithm with the graphic object by the filter. Here, a filter is one in which manipulating data may cause changes in the data or generate new data processing, and the independent operation of the sub-algorithm is encapsulated in a filter by encapsulation technology and sub-algorithm. It realizes the function of the algorithm.

The filter type includes one or more of a basic filter or a composite filter such as image noise removal, image conversion, image analysis, image division, image compression, image enhancement, image blurring, and image registration in image processing.

Further, the step of connecting a plurality of graphic object models according to the processing flow is a step of constructing a filter connection channel based on the processing flow and connecting a filter in the graphic object model by the filter connection channel. , Each filter is connected according to the processing flow of the multi-source earth observation image, and the step of processing the multi-source earth observation image is included.

A filter connection channel connects a filter that performs various processing steps to a flow that performs a series of processing. For example, in a remote sensing image processing library (ORFEO ToolBox, OTB), the SetInput () and GetAutoput () functions can be used to realize a step of connecting a filter in a graphic object model by the filter connection channel. ORFEO is an abbreviation for Optical and Radar Federated Earth Observation. When executing the processing process of the filter connection channel, after the step of connecting the filter in the graphic object model by the filter connection channel, the Update () function is called and executed at a certain stage of the filter connection channel to execute the filter in the graphic object model. By updating the data in, it further includes the step of sequentially executing each filter operation in the channel with the latest data from the start point position of the filter connection channel to the end point position of the connection channel. Taking one processing flow including input, first processing, second processing, and output as an example, each processing step of the processing flow is constructed in one filter connection channel, and the filter connection channel is an input. It includes a filter, a first processing filter, a second processing filter, and an output filter. At the time of operation, first, image data is acquired from the image file by the input filter, and the first processing is performed by the first processing filter. The first processing result is transmitted to the second processing filter to perform the second processing, then the second processing result is transmitted to the output filter, and the final processing result is output.

Further, after the step of determining the filter corresponding to the graphic object, a step of setting parameters for the filter and setting parameters of the sub-algorithm is further included.

Further, after the step of determining the filter corresponding to the graphic object, a step of modifying the parameters of the filter and modifying the parameters of the sub-algorithm is further included. The parameters of the sub-algorithm are continuously modified and the filtering result is inspected until the desired processing result is achieved. If the filtering result does not meet the requirement, go back and re-modify the sub-algorithm parameters or adjust the combination of filter types until the filtering result meets the requirement.

The visualization method can realize visualization of a simple processing process of the multi-source earth observation image, and can also realize visualization of a complicated processing process of the multi-source earth observation image. Preferably, the visual model includes a graphic object input model, a graphic object processing model, and a graphic object output model in order to realize visualization of a simple processing process of an image. Preferably, the visual model comprises one or more graphic object processing models, each of which is complex by connecting to one or more graphic object input models and one or more graphic object output models. It is possible to realize visualization of various processing processes, for example, to integrate and analyze feature information of multiple multi-source earth observation images, and to output analysis reports of multiple different angles.

Preferably, the step of visualizing the processing process of the multi-source earth observation image by the visual model includes a step of inputting the multi-source earth observation image into the corresponding graphic object model based on the processing flow and the visual. It includes a step of executing the model and a step of confirming the corresponding processing step by the graphic object model and outputting the processing result of the multi-source earth observation image.

Further, after the step of outputting the processing result of the multi-source earth observation image, a step of determining whether or not the processing result satisfies the processing request according to the processing purpose of the multi-source earth observation image is further included. For example, if the processing purpose is to extract the target features of the building in the remote sensing image, the processing request is that the target features of all the buildings in the processed remote sensing image are extracted and the extracted building targets are. Including being accurate. If the processing request is not satisfied, the parameters of one or more graphic object models in the visual model may be modified (for example, the combination of filter types in the visual model may be modified), and if the processing request is satisfied, the processing result and the processing result and The visual model is stored. By continuously modifying the parameters, the final output processing result can meet the processing purpose of the multi-source earth observation image, and by continuously modifying the parameters, the optimum processing result can be obtained. You can get it and finally memorize a good visual model.

The visualization method of the multi-source earth observation image processing includes a step of acquiring a reading command of a visual model based on the processing flow of the multi-source earth observation image before acquiring a plurality of sub-algorithms corresponding to the processing flow. , A step to determine whether the visual model corresponding to the read instruction is stored, and a step to read the stored visual model and generate a visual model if the visual model corresponding to the read instruction is stored. Further includes a step of not executing and a step of visualizing the processing process of the multi-source earth observation image based on the read visual model. If the visual model corresponding to the read instruction is not stored, the corresponding sub-algorithms are acquired based on the processing flow, the graphic object model and the visual model are generated, and the multi-source earth observation is performed based on the generated visual model. Visualize the image processing process.

After reading the stored visual model, the read visual model is modified based on the processing flow of the multi-source earth observation image, and the processing process of the multi-source earth observation image is performed based on the modified visual model. It further includes steps to realize visualization of multi-source earth observation image processing by visualization. When the stored visual model file is read, graphic arrows are added and each attribute is automatically defined, which is the process of reusing the visual model of image processing. The step of reading the stored visual model includes two steps of reading the shape and reading the arrow, the shape refers to the shape of the graphic object model in the visual model, and the arrow refers to the visual model. Point to the connecting arrow between multiple graphic object models in. Preferably, the shape reading step creates a graphic box with the corresponding position coordinates in the graphic scene form and the position coordinates of the graphic box of the graphic object and the corresponding filter type, and sets the filter type. A step, a step of reading an attribute value in the form of "attribute-value", and a setting attribute interface filtering algorithm function predefined in the algorithm knowledge base are sequentially called based on the attribute value, and the corresponding attribute item is supported. Includes steps to set the attribute value to be. The step of reading the arrow is a step of acquiring the start point coordinates of the arrow, a step of searching for the shape of the position of the arrow according to the start point coordinates of the arrow, and an arrow connecting to the start point in the same manner as the process of manually drawing an arrow. Includes steps to create and. By the above reading step, the graphic object model stored in the model library can be read and edited, and further editing and development can be performed by integrating different graphic object models. The visual model thus formed can encapsulate the validated processing flow by recording the input / output interfaces between the series of processing processes and the parameter settings of different processing processes or filtering operators. .. The visual model may be quoted by oneself or others, and with the continuous formation of modules that realize different processing flows, the modules are further integrated to form a processing process of a certain scale, and a complicated image processing function is performed. At the same time, the time cost and labor cost of intermediate man-machine interaction can be omitted.

The flow of the visualization method will be further described below by combining the interface part and the corresponding background processing part:
First, in the graphic scene area, the shape of the filter in a typical remote sensing image processing library (ORFEO ToolBox, OTB) is created and connected with an arrow, or the file in "* .mdl" format is read completely. Arrange the processing flow. At this time, the background completes the creation of the corresponding OTB filter and at the same time displays the adjustable parameter items contained in the OTB filter corresponding to the shape of the selected filter in the attribute box;
Then, by selecting different filter shapes in the graphic scene, the corresponding adjustable parameter items are displayed in the attribute box, and each parameter item value that can be set in the displayed parameter item cell is clicked. Adjust the desired parameters for the OTB filter;
Next, the filter corresponding to a certain processing step in the processing flow is selected, selectively executed or executed, the execution result is displayed, and the operation is completed;
Furthermore, the processing flow is changed by modifying the position of each graphic object model, or the experimental parameters are adjusted by modifying the parameter items, and the experimental results are compared.

By comparing the experimental results and finally determining the parameters of each graphic object in the visual model, the accuracy of the stored visual model becomes higher.

The visualization method of the present application is applied to the processing and visualization analysis of big data such as dozens of satellite images and spatial geographic information.

The method for visualizing multi-source earth observation image processing according to the present application is applied to electronic devices as terminals such as televisions, smartphones, tablet computers, and computers.

The electronic device includes a processor and a memory for storing a visualization program for multi-source earth observation image processing, and the processor executes the visualization program for multi-source earth observation image processing.
Based on the processing flow of the multi-source earth observation image, a step of acquiring a plurality of sub-algorithms corresponding to the processing flow, a step of generating a graphic object corresponding to each sub-algorithm based on the processing flow, and a model. A step of associating the sub-algorithm with the graphic object by a generator to generate a graphic object model, and a step of connecting a plurality of graphic object models according to the processing flow to generate a visual model of the multi-source earth observation image. And, the visual model realizes a visualization method of multi-source earth observation image processing, which comprises a step of visualizing the processing process of the multi-source earth observation image.

The electronic device further includes a network interface, a communication bus, and the like. The network interface includes standard wired and wireless interfaces, and the communication bus provides connection communication between each component.

The memory includes at least one type of readable recording medium, and may be a non-volatile recording medium such as a flash memory, a hard disk, an optical disk, or a plug-in hard disk, but is not limited thereto. , Any device that non-volatilely stores instructions or software and any related data files and provides instructions or software programs to the processor so that the processor can execute the instructions or software programs. In the present application, the software program stored in the memory includes a visualization program for multi-source earth observation image processing, and by providing the processor with the visualization program for the multi-source earth observation image processing, the multi-source ground observation image is provided to the processor. The processing visualization program may be executed to realize the steps of the visualization method of the multi-source ground observation image processing.

The processor may be a central processing unit, microprocessor or other data processing chip, etc., and may execute a storage program in memory, for example, a visualization program for multi-source earth observation image processing in the present application.

The electronic device may further include a display, also referred to as a display screen or display unit. In some embodiments, the display may be an LED display, a liquid crystal display, a touch liquid crystal display, an organic light-emitting diode (OLED) touch device, and the like. The display displays a working interface for information and visualization processed by the electronic device.

The electronic device may further include a user interface such as an input unit (eg, keyboard), audio output device (eg, audio, headphones).
In another embodiment, the visualization program for multi-source earth observation image processing may be divided into one or more modules, embodying the present application by storing one or more modules in memory and executing them by a processor. It is possible. A module as used herein refers to a set of computer program instruction segments capable of performing a particular function. FIG. 2 is a modular schematic diagram of the visualization program for the multi-source earth observation image processing according to the present application, and as shown in FIG. 2, the visualization program for the multi-source earth observation image processing includes the acquisition module 1 and the graphic object generation module. 2. It can be divided into an association module 3, a model generation module 4, and a visual module 5. Since all the functions or operation steps realized by the above modules are the same as the above, they will not be described in detail here.

Illustratively, the acquisition module 1 acquires a plurality of sub-algorithms corresponding to the processing flow based on the processing flow of the multi-source earth observation image, and the graphic object generation module 2 acquires each of the plurality of sub-algorithms based on the processing flow. A graphic object corresponding to the sub-algorithm is generated, the association module 3 associates the sub-algorithm with the graphic object by the model generator to generate a graphic object model, and the model generation module 4 generates a plurality of graphic objects according to the processing flow. A graphic object model is connected to generate a visual model of the multi-source earth observation image, and the visual module 5 visualizes the processing process of the multi-source earth observation image by the visual model.

The processing flow of a multi-source earth observation image can be conceptualized into a sequential instruction set formed by a series of modular, continuous and different image processing sets. Taking the remote sensing image as an example, the processing flow of the remote sensing image includes data input / output, image preprocessing (including geometric correction, fusion, mosaic, etc.), and video information extraction (artificial interpretation, automatic classification, feature extraction). , Including dynamic detection, etc.), Subject mapping / 3D visualization analysis (including integration of conventional data of geographic information system, etc.), and result report (including analysis and sharing of geographic information system, etc.) including. Each processing step in the processing flow corresponds to one or more sub-algorithms, and the combination of the sub-algorithms realizes related processing and information extraction for the remote sensing image. Sub-algorithms can be obtained by querying the open source image processing algorithm repository.

Further editing can be done by executing, editing or storing the graphic object model in the model library and integrating different processing module sets. Different processing modules contain different sub-algorithms, providing great convenience for research testing and subsequent use. The graphic object model can be printed out as a flowchart or presented as a lecture in a research report. The model generator is provided with multiple operational operators such as data type conversion, image space area time area basic processing, image conversion, projection correction, feature extraction, change monitoring, and raster data for the graphic object model. It is possible to perform operations such as vector data and classification data.

In this application, the visual model generated based on the processing flow of the multi-source earth observation image realizes the visualization of the processing process of the multi-source earth observation image, and the graphic object model in the visual model makes each step of the image processing intuitive. It is possible to improve the visibility of data processing and satisfy the requirements for visualization integration for the processing process of multi-source earth observation images by the WYSIWYG method.

The acquisition module 1 queries the algorithm knowledge base based on the first decision unit that determines the processing flow and each substep in the processing flow based on the processing purpose of the multi-source earth observation image, and the substep. A query unit that acquires a sub-algorithm corresponding to the sub-step is included, for example, the clear processing purpose includes image pre-processing, image feature extraction, change monitoring, etc., and the determined processing flow achieves the processing purpose. Specific methods and necessary data support.

Preferably, the association module 3 includes a second determination unit that determines the filter type of the graphic object based on the sub-algorithm, and a setting unit that sets attribute parameters of the graphic object based on the filter type. , A filter determination unit that determines the filter corresponding to the graphic object, and an association unit that associates the sub-algorithm with the graphic object by the filter. A filter is one in which manipulating data can cause data changes or generate new data processing, and the sub-algorithm's function is to encapsulate the independent operations of the sub-algorithm into a filter using encapsulation technology. Has been realized.

The filter type includes one or more of a basic filter or a composite filter such as image noise removal, image conversion, image analysis, image division, image compression, image enhancement, image blurring, and image registration in image processing.

Further, the model generation module includes a channel construction unit that constructs a filter connection channel based on the processing flow, and a connection unit that connects the filters in the graphic object model by the filter connection channel.

A filter connection channel connects a filter that performs various processing steps to a flow that performs a series of processing, and when the processing process of the filter connection channel is executed, the channel is from the start point position of the connection channel to the end point position of the connection channel. Perform each of the filter operations in Sequentially with the latest data. Taking one processing flow including input, first processing, second processing, and output as an example, each processing step of the processing flow is constructed in one filter connection channel, and the filter connection channel is an input. It includes a filter, a first processing filter, a second processing filter, and an output filter. At the time of operation, first, image data is acquired from the image file by the input filter, and the first processing is performed by the first processing filter. The first processing result is transmitted to the second processing filter to perform the second processing, then the second processing result is transmitted to the output filter, and the final processing result is output.

Further, the association module 3 further includes a parameter setting unit that sets parameters for the filter and sets parameters for the sub-algorithm.

Further, the association module 3 further includes a parameter modification unit that modifies the parameters of the filter and modifies the parameters of the sub-algorithm. The parameters of the sub-algorithm are continuously modified and the filtering result is inspected until the desired processing result is achieved. If it does not meet the requirement, it goes back and re-modifies the parameters of the sub-algorithm or adjusts the combination of filter types until the obtained processing result meets the requirement.

The electronic device can realize a simple visualization of the processing process of the multi-source earth observation image, and can also realize visualization of a complicated processing process of the multi-source earth observation image. Preferably, the visual model includes a graphic object input model, a graphic object processing model, and a graphic object output model in order to realize visualization of a simple processing process of an image. Preferably, the visual model comprises a plurality of graphic object processing models, each of which is connected to one or more graphic object input models and one or more graphic object output models to perform complex processing. It is possible to realize process visualization, for example, it is possible to integrate and analyze the feature information of multiple multi-source earth observation images and output analysis reports of multiple different angles.

Preferably, the visual module 5 corresponds to the input unit for inputting the multi-source earth observation image into the corresponding graphic object model, the execution unit for executing the visual model, and the graphic object model based on the processing flow. It includes an output unit that confirms the processing step and outputs the processing result of the multi-source earth observation image.

Further, the visual module 5 determines whether or not the processing result satisfies the processing request of the multi-source earth observation image, and if the processing request is not satisfied, modifies the parameters of one or more graphic object models in the visual model. (For example, the combination of filter types in the visual model may be modified), and when the processing request is satisfied, the request determination unit for storing the processing result and the visual model is further included.

The electronic device acquires a reading command of the visual model, determines whether or not the visual model corresponding to the reading command is stored, and if the visual model corresponding to the reading command is stored, it is stored. It further includes a judgment module that reads the visual model and visualizes the processing process of the multi-source earth observation image based on the read visual model.

After reading the stored visual model, modify the read visual model based on the processing flow of the multi-source earth observation image, and realize visualization of multi-source earth observation image processing by the modified visual model. Includes more steps. When the stored visual model file is read, the process of adding graphic arrows and defining each attribute is automatically executed, which is the process of reusing the visual model of image processing. The step of reading the stored visual model includes two steps, reading the shape and reading the arrow, and when reading the shape, first obtain the position coordinates of the graphic box of the graphic object and the corresponding filter type. After creating a graphic box with the corresponding position coordinates in the graphic scene form and setting the filter type, the attribute values are read in the "attribute-value" format and predefined in the algorithm knowledge base based on the attribute values. The setting attribute interface filtering algorithm function is called sequentially, and the corresponding attribute item is set to the corresponding attribute value. When reading an arrow, first, the shape of the position is searched according to the coordinates of the start point of the arrow, and then an arrow having a shape connected to the start point is created in the same manner as in the process of manually drawing an arrow. By the above reading step, the graphic object model stored in the model library can be read and edited, and further editing and development can be performed by integrating different graphic object models. The visual model thus formed can encapsulate the validated processing flow by recording the input / output interfaces between the series of processing processes and the parameter settings of different processing processes or filtering operators. The visual model is often cited by oneself or others, and with the continuous formation of modules that realize different processing flows, the modules are further integrated to form a processing process of a certain scale, and a complicated image processing function is performed. At the same time, it is possible to omit the time cost and labor cost of the intermediate man-machine interaction.

The electronic device further includes a display module including a graphic scene form, an attribute box form and a result display form, wherein the graphic scene form includes one or more graphic boxes (eg, a rectangular box, a triangular box, a circular box, etc.). Arranged, each graphic box indicates the corresponding processing data and steps, and the plurality of graphic boxes are connected by a connection line for indicating the flow of data (for example, placing an arrow on the connection line). Indicates the flow before and after processing the data, the graphic box and the connecting line can always be dragged, and the connecting line can be dragged to correct the start point and end point. The attribute box form displays each attribute of the processing data or step indicated by the currently selected graphic box, and the attribute box form can be used to modify the attribute value corresponding to the graphic object. For example, each attribute value can be conveniently modified by clicking on the corresponding mass item in the attribute box. The result display form is for displaying the execution result, includes one or more sub-windows, and displays the intermediate processing result obtained by executing the processing flow up to the graphic object model by the sub-window. In each subwindow, the subwindow as the main program window is always displayed above the other subwindows.

By encapsulating each stage of the image processing process in a visualized graphic object model, the data processing process can be intuitively shown, and parameters and each stage can be easily adjusted in the graphical interface. Since the processing results can be displayed in a sub-window, it is easy to compare each processing result.

Furthermore, the entire processing flow in the graphic scene can be stored as a file in the "* .mdl" format, and unfinished experiments can be continued after reading the previously stored file in the "* .mdl" format. It is possible to complete the processing flow based on the file.

In one embodiment of the present application, the computer-readable recording medium is any tangible medium containing or storing a program or instruction, the program in which the program is executed, and the stored program corresponds to the related hardware. It is possible to realize the function to be performed. For example, the computer-readable recording medium may be a computer magnetic disk, a hard disk, a random access memory, a read-only memory, or the like. The present application is not limited thereto, and any instruction or software and any related data file or data structure can be stored non-temporarily and provided to the processor so that the processor can execute a program or instruction in the instruction. It may be the device of. The computer-readable recording medium includes a visualization program for multi-source earth observation image processing, and when the visualization program for multi-source earth observation image processing is executed by the processor, the processing flow of the multi-source earth observation image is entered. Based on this, a step of acquiring a plurality of sub-algorithms corresponding to the processing flow, a step of generating a graphic object corresponding to each sub-algorithm based on the processing flow, and a model generator to obtain the sub-algorithm. A step of generating a graphic object model in association with a graphic object, a step of connecting a plurality of graphic object models according to the processing flow to generate a visual model of the multi-source earth observation image, and the above-mentioned visual model. We will realize a visualization method for multi-source earth observation image processing, which consists of steps to visualize the processing process of multi-source earth observation images.

Since the specific embodiment of the computer-readable recording medium of the present application is almost the same as the visualization method of the multi-source earth observation image processing and the specific embodiment of the electronic device, the description thereof will be omitted here.

It should be noted that in the present specification, the terms "include", "contains" or other variations are intended to cover non-exclusive inclusion, and a process, apparatus, article or method comprising a series of elements shall be used. In addition to including those elements, it also includes other elements not explicitly listed, or further includes elements specific to such processes, devices, articles or methods. In the absence of more limitations, an element limited by the phrase "contains one ..." excludes the further presence of another identical element in the process, device, product or method containing the element. is not it.

The order of the above embodiments in the present application is for illustration purposes only and does not indicate superiority or inferiority of the embodiments. From the above description of the embodiment, as can be clearly understood by those skilled in the art, the method of the above embodiment can be realized by a method using software and a necessary general-purpose hardware platform in combination, and of course, it can also be realized by hardware. However, in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present application may be embodied in the form of a software product in which an essential or contributory part to the prior art is embodied, and the computer software product is the above-mentioned one recording medium (for example, the above-mentioned one recording medium (for example). , ROM / RAM, magnetic disk, optical disk) to cause one terminal device (which may be a mobile phone, computer, server, network device, etc.) to execute the method described in each embodiment of the present application. Contains multiple instructions.

The above is merely a preferred embodiment of the present application, and the scope of protection of the present application is not limited thereto, and the exchange of the same price structure or the same price flow performed by utilizing the contents shown in the specification and the drawings of the present invention. , Directly or indirectly, used in other related technical fields are included in the scope of protection of the present invention.

Claims (20)

A visualization method for multi-source earth observation image processing applied to electronic devices.
Based on the processing flow of the multi-source earth observation image, the step of acquiring a plurality of sub-algorithms corresponding to the processing flow, and
A step to generate a graphic object corresponding to each sub-algorithm based on the processing flow,
A step of associating the sub-algorithm with the graphic object to generate a graphic object model by the model generator.
A step of connecting a plurality of graphic object models according to the processing flow to generate a visual model of the multi-source earth observation image, and
A method for visualizing multi-source earth observation image processing, which comprises a step of visualizing the processing process of the multi-source earth observation image by the visual model. Based on the processing flow of the multi-source earth observation image, the step of acquiring multiple sub-algorithms corresponding to the processing flow is
A step to determine each substep in the processing flow and processing flow based on the processing purpose of the multi-source earth observation image, and
The visualization method for multi-source earth observation image processing according to claim 1, further comprising a step of querying an algorithm knowledge base and acquiring a sub-algorithm corresponding to the sub-step based on the sub-step. The step of associating the sub-algorithm with the graphic object is
A step of determining the filter type of the graphic object based on the sub-algorithm,
A step of setting attribute parameters of the graphic object based on the filter type and determining a filter corresponding to the graphic object.
The visualization method for multi-source earth observation image processing according to claim 1, comprising the step of associating the sub-algorithm with the graphic object by the filter. The multi-source earth observation image processing according to claim 3, further comprising a step of setting parameters for the filter and setting parameters of the sub-algorithm after the step of determining the filter corresponding to the graphic object. Visualization method. After the step of determining the filter corresponding to the graphic object, the parameter is modified for the filter, the parameter of the sub-algorithm is modified, and it is determined whether the filtering result satisfies the requirement, and the requirement is satisfied. If not, the visualization of the multi-source earth observation image processing according to claim 3, further comprising the step of revising the parameters of the sub-algorithm or adjusting the combination of filter types until the filtering result meets the requirements. Method. The step of connecting multiple graphic object models according to the above processing flow is
Steps to build a filter connection channel based on the processing flow,
The visualization method for multi-source earth observation image processing according to claim 3, comprising the step of connecting the filters in the graphic object model by the filter connection channel. The visualization method for multi-source earth observation image processing according to claim 6, wherein the step of connecting the filter in the graphic object model by the filter connection channel is realized by the SetInput () and GetAutot () functions in the remote sensing image processing library. .. After the step of connecting the filter in the graphic object model by the filter connection channel, the Update () function is called and executed to update the filter data in the graphic object model, and the filter connection is performed from the start point position of the filter connection channel. The visualization method for multi-source earth observation image processing according to claim 6, further comprising a step of sequentially executing each filter operation in the filter connection channel with the updated data up to the end point position of the channel. The step of visualizing the processing process of the multi-source earth observation image by the visual model is
Based on the processing flow, the step of inputting the multi-source earth observation image into the corresponding graphic object model, and
The steps to execute the visual model and
The visualization method for multi-source earth observation image processing according to claim 1, further comprising a step of confirming a corresponding processing step by a graphic object model and outputting a processing result of the multi-source earth observation image. After the step of outputting the processing result of the multi-source earth observation image,
It is determined whether or not the processing result satisfies the processing request of the multi-source earth observation image, and if the processing request is not satisfied, the parameters of one or more graphic object models in the visual model are modified and the processing request is satisfied. The visualization method for multi-source earth observation image processing according to claim 9, further comprising the step of storing the processing result and the visual model. Based on the processing flow of the multi-source earth observation image, before acquiring multiple sub-algorithms corresponding to the processing flow,
The step to get the read instruction of the visual model and
A step to determine if the visual model corresponding to the read instruction is stored,
If the visual model corresponding to the read instruction is stored, the step to read the stored visual model and
The visualization method for multi-source earth observation image processing according to claim 1, further comprising a step of visualizing the processing process of the multi-source earth observation image by a read visual model. After reading the stored visual model, the read visual model is modified based on the processing flow of the multi-source earth observation image, and the modified visual model visualizes the processing process of the multi-source earth observation image. The visualization method of the multi-source earth observation image processing according to claim 11, further comprising the step of performing. The step of reading the stored visual model includes two steps of reading the shape and reading the arrow, the shape refers to the shape of the graphic object model in the visual model, and the arrow refers to the visual model. The visualization method for multi-source earth observation image processing according to claim 11, which points to a connecting arrow between a plurality of graphic object models in. The step of reading the shape is
Steps to get the position coordinates of the graphic box of the graphic object and the corresponding filter type,
Steps to create a graphic box with the corresponding position coordinates in the graphic scene form and set the filter type,
A step to read an attribute value in the "attribute-value" format,
The thirteenth aspect of claim 13, comprising a step of sequentially calling a pre-defined set attribute interface filtering algorithm function in the algorithm knowledge base based on the attribute value and setting the corresponding attribute item to the corresponding attribute value. How to visualize multi-source earth observation image processing. The step of reading the arrow is
The step to get the coordinates of the start point of the arrow,
A step of searching for the shape of the location of the arrow according to the coordinates of the start point of the arrow, and
The visualization method for multi-source earth observation image processing according to claim 13, comprising the step of creating an arrow connecting to the start point. The visual model includes a graphic object input model, a graphic object processing model, and a graphic object output model, the graphic object processing model is one or more, and each graphic object processing model has one or more graphic object inputs. The visualization method for multi-source earth observation image processing according to claim 1, which is connected to a model and one or more graphic object output models. The step of generating the graphic object corresponding to each sub-algorithm based on the processing flow is realized by using the Graphics View framework in the cross-platform C ++ graphic user interface application framework as the graphic user interface. Visualization method of multi-source earth observation image processing described in. Including processor and memory
The memory includes a visualization program for multi-source earth-observed image processing that is executed by the processor and realizes the steps of the visualization method for multi-source earth-observed image processing according to any one of claims 1 to 17. Electronic equipment. Further includes display modules including graphic scene forms, attribute box forms and result display forms,
The graphic scene form arranges one or more graphic boxes, each graphic box shows a corresponding processing data and a step, and the plurality of graphic boxes are connected by a connecting line for showing the flow of data. , The attribute box form displays each attribute of the processing data or step indicated by the currently selected graphic box, and the result display form is for displaying the execution result and includes one or more subwindows. The electronic device according to claim 18, wherein the sub-window displays the intermediate processing results obtained by executing the processing flow up to the corresponding graphic object model. A visualization program for multi-source earth-observation image processing that, when executed by a processor, implements the steps of the visualization method for multi-source earth-observation image processing according to any one of claims 1-17.

Images