WO2023050745A1 - Image processing method and apparatus, device, medium, and program - Google Patents

Image processing method and apparatus, device, medium, and program Download PDF

Info

Publication number
WO2023050745A1
WO2023050745A1 PCT/CN2022/083772 CN2022083772W WO2023050745A1 WO 2023050745 A1 WO2023050745 A1 WO 2023050745A1 CN 2022083772 W CN2022083772 W CN 2022083772W WO 2023050745 A1 WO2023050745 A1 WO 2023050745A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
image
execution
computing node
execution process
Prior art date
Application number
PCT/CN2022/083772
Other languages
French (fr)
Chinese (zh)
Inventor
罗均
徐翔
许鑫汉
Original Assignee
上海商汤智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海商汤智能科技有限公司 filed Critical 上海商汤智能科技有限公司
Publication of WO2023050745A1 publication Critical patent/WO2023050745A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches

Definitions

  • the present disclosure relates to the technical field of image processing, and in particular to an image processing method, device, equipment, medium and program.
  • AI remote sensing detection models such as target detection, image segmentation, change detection, etc.
  • each AI remote sensing detection model corresponds to at least one set
  • the post-processing algorithm will generate more customized post-processing functions when responding to different industry application requirements.
  • the post-processing of the building extraction model can include removing small image spots, regularizing the building extraction results, vector
  • the post-processing of the water body recognition model can include functions such as removing small patches, blurring boundaries, growing boundaries, and fusion of multi-model results. According to the task requirements of different industry applications, it is often necessary to make appropriate adjustments to the post-processing algorithm, such as adding functions such as vectorization or removing small plaques in the post-processing algorithm.
  • post-processing codes when post-processing codes are often modified, many versions of post-processing algorithms for different tasks are formed.
  • Different versions of post-processing algorithms may include codes with repeated functions, resulting in support for AI remote sensing detection models and post-processing algorithms.
  • the platform where the processing algorithm runs stores excessive redundant codes, resulting in a large storage load on the platform.
  • R&D personnel are required to modify and maintain it. The cost is high, which in turn leads to a decrease in user experience.
  • Embodiments of the present disclosure at least provide an image processing method, device, device, medium, and program.
  • an embodiment of the present disclosure provides an image processing method, the method is executed by an electronic device, including:
  • calculation graph information is generated, and the image to be processed is processed according to the calculation graph information to obtain an image processing result.
  • the post-processing algorithm can be preliminarily abstracted into a hierarchical structure, that is, computing nodes and execution processes, by using the execution process generated by abstracting the computing node information and execution process information corresponding to computing nodes with different preset functions, and then, The image to be processed is processed by using the calculation graph information composed of the execution flow, and the post-processing algorithm is further abstracted into a three-layer architecture system, namely, the calculation node (first layer), the execution flow (second layer) and the calculation graph (third layer). ).
  • the existing computing nodes are used to flexibly construct the post-processing calculation graph to meet the processing conditions of the target task , for different tasks, there is no need to repeatedly store code blocks with the same function (corresponding to the above-mentioned computing nodes), which reduces redundant code storage, reduces the storage load of the platform, improves the flexibility of using computing nodes, and reduces the Difficulty managing post-processing algorithms.
  • processing algorithms no longer require R&D personnel to adjust the code, but only need the staff to configure the information required for the calculation graph, and then construct the calculation graph based on the generated calculation graph information to process the image to be processed, which improves the image quality. processing efficiency.
  • an embodiment of the present disclosure further provides an image processing device, including:
  • An image acquisition module configured to acquire images to be processed
  • the information determination module is configured to determine a plurality of computing node information and execution process information based on the target task corresponding to the image to be processed; wherein, the computing nodes corresponding to the plurality of computing node information are used to complete different preset functions ;
  • a process generation module configured to generate an execution process corresponding to the execution process information based on the plurality of computing node information and the execution process information; wherein the execution process includes at least one computing node information;
  • the image processing module is configured to generate calculation graph information based on the execution flow, and process the image to be processed according to the calculation graph information to obtain an image processing result.
  • an embodiment of the present disclosure further provides a computer device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processing The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps of any possible image processing method in the first aspect are executed.
  • embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned first aspect, or any of the first aspects of the first aspect, may be executed. Steps of a possible image processing method.
  • An embodiment of the present disclosure also provides a computer program, where the computer program includes computer readable codes, and when the computer readable codes run in an electronic device, the processor of the electronic device executes any of the above embodiments The image processing method described.
  • FIG. 1 shows a flowchart of an image processing method provided by an embodiment of the present disclosure
  • FIG. 2 shows a schematic diagram of a system architecture applying an image processing method according to an embodiment of the present disclosure
  • FIG. 3 shows a schematic diagram of determining dependency information between computing nodes based on input information and output information of each computing node provided by an embodiment of the present disclosure
  • FIG. 4 shows a schematic diagram of a calculation graph provided by an embodiment of the present disclosure
  • FIG. 5 shows an execution flowchart of an application scenario of an image processing method provided by an embodiment of the present disclosure
  • FIG. 6 shows a schematic diagram of an image processing device provided by an embodiment of the present disclosure
  • FIG. 7 shows a schematic structural diagram of a computer device provided by an embodiment of the present disclosure.
  • Multiple or multiple in the embodiments of the present disclosure may refer to at least two or at least two, respectively.
  • the embodiment of the present disclosure provides an image processing method, which can preliminarily abstract the post-processing algorithm into a hierarchical architecture, that is, the computing nodes and the execution process, and then use the calculation graph information composed of the execution process to process the image to be processed, and further abstract the post-processing algorithm into a three-layer architecture system, namely, the computing node (first layer), the execution process (second layer) and computational graph (third layer).
  • the existing computing nodes are used to flexibly construct the post-processing calculation graph to meet the processing conditions of the target task , for different tasks, there is no need to repeatedly store code blocks with the same function (corresponding to the above-mentioned computing nodes), which reduces redundant code storage, reduces the storage load of the platform, improves the flexibility of using computing nodes, and reduces the Difficulty managing post-processing algorithms.
  • processing algorithms no longer require R&D personnel to adjust the code, but only need the staff to configure the information required for the calculation graph, and then construct the calculation graph based on the generated calculation graph information to process the image to be processed, which improves the image quality. processing efficiency.
  • Post-processing After the image is reasoned and recognized by the AI remote sensing detection model, it is then subjected to some processing such as splicing, fusion, edge blurring, and refinement, so that the accuracy of the results and the display effect are further improved.
  • Computation graph an abstract data structure composed of computing nodes (operators) and lines (operator dependencies).
  • an image processing method disclosed in the embodiment of the present disclosure is firstly introduced in detail.
  • the image processing method provided in the embodiment of the present disclosure is generally executed by a computer device with certain computing capabilities.
  • the image processing method may be implemented by a processor invoking computer-readable instructions stored in a memory.
  • the application scenario of an image processing method disclosed in the embodiment of the present disclosure is firstly introduced.
  • the image processing method provided in the embodiment of the present invention can be applied to post-processing of image recognition in the field of remote sensing image interpretation , which is the post-processing for the recognition and analysis of remote sensing images.
  • the post-processing for the building extraction model may include functions such as removal of small image spots, regularization of building extraction results, and vectorization;
  • the post-processing for the water body identification model may include removal of small patches, boundary blurring, boundary growth, Multi-model result fusion and other functions.
  • the image processing method provided by the embodiment of the present disclosure is used to meet the task requirements of different industry applications, and the post-processing algorithm of the remote sensing AI detection model is appropriately adjusted.
  • FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present disclosure
  • the method is executed by an electronic device, and the method includes steps S101 to S104, wherein:
  • S101 Acquire images to be processed.
  • the image to be processed may include the category image that has been processed by the AI remote sensing detection model, that is, the result of remote sensing interpretation, and an image representing the category information of the AI remote sensing detection.
  • the image to be processed is a result image that needs to be post-processed, and the result image may or may not be a remote sensing image.
  • S102 Based on the target task corresponding to the image to be processed, determine multiple computing node information and execution process information.
  • the computing nodes corresponding to the multiple computing node information are used to implement different preset functions.
  • the target task may be a task of post-processing the image to be processed.
  • the target tasks are different, and the target functions contained in the corresponding post-processing algorithms are different.
  • the target task includes a water body detection task.
  • the water body detection task may include pre-remote sensing interpretation processing and post-processing; wherein, the post-processing may be performed on the initial water body detection results, that is, remote sensing interpretation for images containing water bodies
  • the result (that is, the first remote sensing image) is an algorithm for further processing;
  • the target function included in the algorithm can be functions such as removing small patches, boundary blurring, boundary growth, and multi-model result fusion; or, exemplary, the target task Including the building detection task, for example, the building detection task may include pre-remote sensing interpretation processing and post-processing; where the post-processing may be for the initial building detection result, that is, the result of remote sensing interpretation for the image containing the building ( That is, the second remote sensing image), an algorithm for
  • the computing nodes corresponding to the multiple computing node information are used to implement different preset functions, wherein the preset function may include a single function, which is the minimum function among the target functions. For example, if the target function includes regularization of building extraction results, then the minimum function in the target function includes extraction function, result regularization function and so on.
  • a compute node may include function code capable of performing a single predetermined function.
  • the computing node information may include identification information of the computing node, such as name, serial number, and the like.
  • the target task as an example of the post-processing task of removing small image spots, boundary blurring and vectorization of the image to be processed, based on the first subtask in the target task, that is, removing small image spots, determine at least one corresponding to the removal of small image spots Computing node information; based on the second subtask in the target task, that is, boundary blurring, determine at least one computing node information corresponding to boundary blurring; based on the third subtask in the target task, namely vectorization, determine at least one corresponding to vectorization Computing node information, here, can be the name of computing node A and computing node B corresponding to the removal of small image spots; the name of computing node C corresponding to boundary blur; the name of computing node D corresponding to vectorization, computing node The name of E and the name of computing node F, etc.
  • each execution flow information corresponds to one execution flow, therefore, each execution flow information includes at least one computing node information included in its corresponding execution flow.
  • the three subtasks correspond to three execution task information, that is, there are three execution processes correspondingly, where the first execution process information includes the computing node A information and computing node B information included in the corresponding first execution process ; The second execution process information includes the information of computing node C included in the corresponding second execution process; the third execution process information includes the information of computing node D included in the corresponding third execution process, computing node E information and computing node F information.
  • S103 Generate an execution process corresponding to the execution process information based on a plurality of computing node information and execution process information; wherein, the execution process includes at least one computing node information.
  • the execution process corresponding to the execution process information can be generated according to at least one computing node information in the execution process information, and the dependency relationship information between each computing node corresponding to each computing node information, for example, the computing node A depends on computing node B, so it can be determined that the execution sequence of the execution process is to execute computing node B first, and then execute computing node A.
  • the execution process includes the name of computing node A, the name of computing node B, and the relationship between computing nodes. The sequence of execution does not include the function codes of computing node A and computing node B.
  • the computing node information also includes input information and output information of the corresponding computing node.
  • Determine the dependency relationship information between each computing node for example, determine the dependency relationship information between multiple computing nodes based on the input information and output information of each computing node; based on the determined dependency relationship information and execution process information, An execution flow corresponding to the execution flow information is generated.
  • the input information/output parameter may include at least one of the following: the identification of the input parameter/output parameter of the calculation node, the type of the input parameter/output parameter; wherein, the type of the input parameter/output parameter, for example, may be an image, a digital Or parameter types such as pointers.
  • the target task may include multiple subtasks, each subtask corresponds to at least one piece of execution flow information, and each piece of execution flow information corresponds to one execution flow, therefore, the target task may have multiple execution flows.
  • FIG. 2 it is a schematic diagram of determining the dependency information between computing nodes based on the input information and output information of each computing node; wherein, the identification of the input parameter in the information of computing node A is a, and the identification of the output parameter is The identification is c; the identification of input parameters in the information of computing node B is b, and the identification of output parameters is d; the identification of input parameters in information of computing node C is c and d, and the identification of output parameters is e; information of computing node D
  • the identifier of the input parameter in is d, the identifier of the output parameter is f; the identifiers of the input parameters in the computing node E information are e and f, and the identifier of the output parameter is g.
  • computing node C depends on computing node A, that is, Determine dependency information between computing node C and computing node A.
  • dependency relationship information between computing nodes includes that computing node E depends on computing nodes C and computing nodes D, computing node C depends on computing nodes A and computing nodes B, and computing node D depends on computing nodes B.
  • the execution flow corresponding to the execution flow information is generated, which may include execution flow A, B ⁇ C, execution flow B ⁇ D, and execution flow C, D ⁇ E.
  • the above-mentioned input information and output information of computing nodes can be used to automatically determine the dependency information between computing nodes, and the execution process corresponding to the execution process information can be generated without specifying the execution sequence between computing nodes, which improves the processing of target tasks. efficiency.
  • S104 Generate calculation graph information based on the execution flow, and process the image to be processed according to the calculation graph information to obtain an image processing result.
  • the execution flow information may include the execution sequence of the corresponding execution flow.
  • calculation graph information can be generated based on the multiple execution processes and the execution order of each execution process; then, based on the execution order, the first execution process is determined; Input the image to be processed into the first computing node corresponding to the first execution process, and process it according to the first execution process and other execution processes corresponding to the execution order, until all the execution processes are executed, and the image processing result is obtained.
  • other execution processes include multiple execution processes corresponding to the target task except the first execution process.
  • each execution process includes computing node information corresponding to at least one computing node to be executed by the execution process.
  • the calculation graph information includes the calculation graph and the description information of the calculation graph.
  • the description information of the calculation graph may include information describing at least one execution process and at least one calculation node to be executed by each execution process, and the description information of the calculation graph is used to construct the calculation graph.
  • the calculation graph can be constructed according to the description information of the calculation graph, and then each execution process in the calculation graph is run, and each calculation node is calculated in the execution process according to the execution order (that is, the execution interface of each calculation node is called, and the Calculation), so that the image processing can be completed in an orderly manner according to the execution order, and the image processing result can be obtained.
  • the execution order that is, the execution interface of each calculation node is called, and the Calculation
  • FIG. 3 it is a schematic diagram of the calculation graph, including three execution processes, where the first execution process includes three computing nodes (S_1, S_2, and S_3, of which the first computing node is S_1) and their dependency information , that is, S_1 ⁇ S_2 ⁇ S_3, the second execution process includes two computing nodes (S_4 and S_5) and their dependency information, that is, S_4 ⁇ S_5, and the third execution process includes two computing nodes (S_6 and S_7) and Its dependency information, namely S_6 ⁇ S_7.
  • the execution sequence is to first execute S_1 ⁇ S_2 ⁇ S_3, then execute S_4 ⁇ S_5, and finally execute S_6 ⁇ S_7.
  • the execution process is the minimum execution sequence of the calculation graph, which can complete at least one target function (such as the regularization of building extraction results), each calculation graph has at least one execution process, and the calculation node can complete the minimum preset function in the calculation graph (such as extract function).
  • Executing the target task for example, as shown in FIG. 3 , includes three subtasks, that is, each subtask corresponds to an execution process.
  • the interface of input parameters/output parameters of each computing node and the execution interface of the function code of the computing node can be respectively called according to the calculation graph, and the function code of the computing node can be called to execute the target task.
  • call the function code of the computing node starting from the first computing node of the first execution process, call the function code of the computing node to process the input parameters, obtain the output result, store the output result through the output interface of the computing node, and wait for the input of the next computing node Called when parameters are entered.
  • the image to be processed may include the first remote sensing image corresponding to the initial water body detection result obtained after the early remote sensing interpretation processing. Afterwards, the first remote sensing image is processed according to the execution flow in the calculation graph information, and the target water body detection result corresponding to the water body detection task is obtained.
  • each execution process in the calculation graph can be run, the function code of each calculation node in the execution process can be called according to the execution order, and the post-processing task in the water body detection task can be executed to obtain the target water body detection result.
  • the image to be processed may include the second remote sensing image corresponding to the initial building detection result obtained after the previous remote sensing interpretation processing. Afterwards, the second remote sensing image is processed according to the execution flow in the calculation graph information, and the target building detection result corresponding to the building detection task is obtained.
  • each execution process in the calculation graph can be run, the function code of each calculation node in the execution process can be called according to the execution order, and the post-processing task in the building detection task can be executed to obtain the detection result of the target building.
  • the post-processing algorithm can be preliminarily abstracted into a hierarchical architecture by using the computing node information corresponding to the computing node abstracted as a single preset function and the execution process information to generate the execution process, that is, the computing node and the execution process , after that, use the calculation graph information composed of the execution process to process the image to be processed, and further abstract the post-processing algorithm into a three-layer architecture system, namely, the calculation node (the first layer), the execution process (the second layer) and the calculation graph ( the third floor).
  • the existing computing nodes are used to flexibly construct the post-processing calculation graph to meet the processing conditions of the target task, and different tasks do not use
  • Repeated storage of code blocks with the same function reduces redundant redundant codes, reduces platform storage load, improves the flexibility of computing nodes, and reduces the management of post-processing algorithms difficulty.
  • the construction and adjustment of processing algorithms no longer require R&D personnel to adjust the code, but only need the staff to configure the information required for the calculation graph, and then construct the calculation graph based on the generated calculation graph information to process the image to be processed, which improves the image quality. processing efficiency.
  • FIG. 4 shows a schematic diagram of a system architecture applying an image processing method according to an embodiment of the present disclosure
  • the system architecture includes: an image acquisition terminal 201 , a network 202 and a control terminal 203 .
  • the acquisition terminal 201 and the control terminal 203 establish a communication connection through the network 202
  • the image acquisition terminal 201 reports the image to be processed to the control terminal 203 through the network 202
  • the control terminal 203 acquires the image to be processed , based on the target task corresponding to the image to be processed, determine a plurality of computing node information and execution process information; wherein, the computing nodes corresponding to the plurality of computing node information are used to complete different preset functions; based on the multiple computing node information and the execution process information, generating an execution process corresponding to the execution process information; wherein, the execution process includes at least one computing node information; based on the execution process, generating calculation graph information, and according to the The calculation graph information processes
  • the image acquisition terminal 201 may include an image acquisition device, and the control terminal 203 may include a vision processing device capable of processing visual information or a remote server.
  • the network 202 may be connected in a wired or wireless manner.
  • the control terminal 203 is a visual processing device
  • the image acquisition terminal 201 can communicate with the visual processing device through a wired connection, such as performing data communication through a bus;
  • the control terminal 203 is a remote server, the current scene image acquisition terminal 201 can perform data interaction with a remote server through a wireless network.
  • the image acquisition terminal 201 may be a vision processing device with a video capture module, or a host with a camera.
  • the display method in the augmented reality scene of the embodiment of the present disclosure may be executed by the image acquisition terminal 201 , and the above-mentioned system architecture may not include the network 202 and the control terminal 203 .
  • the undefined computing nodes need to be generated during the process of building the computing graph.
  • the corresponding computing nodes are generated based on the computing node information of the undefined computing nodes; call The computing node corresponding to each computing node information processes the image to be processed to obtain an image processing result.
  • An undefined computing node may be an algorithm that has not been implemented in the software, for example, the function code of the computing node is not added. Defining a compute node may be adding the algorithm (function code) to the software.
  • an undefined computing node may have already stored the function code of the computing node, but has not been configured with an execution interface for calling the function code of the computing node.
  • an execution interface for calling the function code of the computing node.
  • the user can configure the function code corresponding to the extraction function.
  • An execution interface (configurable through a configuration file), and an execution interface for configuring the function code corresponding to the call result regularization function.
  • an undefined computing node may also be that the function code of the computing node is not stored, and the execution interface for calling the function code of the computing node is not configured.
  • the function code of building extraction result regularization can be determined from the post-processing algorithm of the historical task, and the function code corresponding to the execution extraction function and the function code corresponding to the result regularization function can be determined from it; after that, the extraction function is configured and called
  • the execution interface of the corresponding function code (which can be configured through the configuration file), and the execution interface of the function code corresponding to the call result regularization function, can realize the definition of the undefined computing node and generate the corresponding calculate node.
  • the input information/output information of a certain execution process occupies a large amount of memory
  • the input information of the execution process can also be processed in batches to obtain processing result information with a smaller memory, and batch processing
  • the execution flow needs to be executed cyclically.
  • the execution flow includes a cyclic execution flow
  • the output information of the previous execution flow of the cyclic execution flow is obtained, and the output information is divided into multiple sub-information; the multiple sub-information are respectively Enter the cyclic execution process for processing, and obtain the processing result information corresponding to each sub-information; determine the target result information based on the processing result information corresponding to each sub-information, and input the target result information into the next execution process of the cyclic execution process until execution After completing all the execution processes, the image processing result is obtained.
  • the cyclic execution process is a cyclic execution process, and the number of cyclic executions can be set according to the memory size occupied by the input information/output information of the cyclic execution process.
  • the output information of the previous execution process of the cyclic execution process is the input information of the cyclic execution process, and the input information is divided into multiple sub-information. Processing the sub-information occupying a small memory can obtain the processing result information (that is, part of the information in the output information of the cyclic execution process) occupying a small memory.
  • the preset maximum memory is the third-generation mobile communication technology (3rd-Generation, 3G).
  • the memory occupied by the input information of the cyclic execution flow is 3G
  • the processing process of the cyclic execution flow also includes There is a lot of intermediate information, so the memory occupied by the output information of the cyclic execution process is larger than 3G. Therefore, the output information of the previous execution process can be divided into 2 sub-information, and the number of cyclic executions of the cyclic execution process is 2 times. If the performance impact of device operation is also considered, the output information of the previous execution process can be divided into 3 sub-information, and the number of cyclic executions of the cyclic execution process is 3 times.
  • the output information of the previous execution process may be divided according to the memory occupied by each sub-information is less than the preset maximum memory amount , divided into multiple sub-information.
  • the processing result is divided so that the memory occupied by it is less than the preset maximum memory amount, Afterwards, according to the number of divisions, the output information of the previous execution process is evenly divided into multiple sub-information.
  • the associated two sub-information can be used as adjacent input sub-information to determine the input sequence during loop execution; after that, according to the processed
  • the output sequence continuously integrates the processing result information corresponding to each sub-information to obtain the target processing result.
  • input multiple sub-information out of order for example, input multiple sub-information respectively into the loop execution process for processing, you can get the processing result information corresponding to each sub-information, after that, process the processing result information corresponding to each sub-information
  • the integrated target processing results can be obtained.
  • the data integration process is different, which is not specifically limited in the embodiments of the present disclosure.
  • the sub-information after the batch processing information of the cyclic execution flow can be used to fuse the processing result information corresponding to each sub-information obtained by the cyclic execution flow processing, and it can be obtained by using Target result information representing the processing result of the entire information, so that the process can continue to be executed.
  • the sub-information is processed in batches by means of a cyclic execution flow, which can reduce the computer resources invoked by each processing process, thereby reducing equipment loss.
  • the user can pre-configure the configuration file corresponding to the target task, and then download the configuration file to the computer device to prepare for executing the post-processing function corresponding to the target task.
  • the configuration file includes a plurality of computing node information, at least one execution process information, and execution interface information of each computing node called when running the computing graph, and the configuration file is used to configure information for generating the computing graph information.
  • At least one execution process information can be obtained from the configuration file, and then at least one execution process item is created, and at least one computing node information corresponding to the execution process item is added to each execution process item, according to The dependency information between computing nodes corresponding to each computing node information generates an execution process.
  • the target task is to perform boundary blurring and post-processing for removing small plaques.
  • the configuration file includes computing node information Q_1 corresponding to the boundary blur function, computing node information Q_2 and Q_3 corresponding to removing small plaques, execution process information P_1 is post-processing of boundary blurring, and execution process information P_2 is post-processing of removing small plaques.
  • the execution process information P_1 in the configuration file determine the execution process P_1 ⁇ ' as the computing node corresponding to the execution computing node information Q_1; according to the execution process information P_2 in the configuration file and the dependency information between Q_2 and Q_3, determine The execution process P_2 ⁇ ' is to first execute the computing node corresponding to the computing node information Q_2, and then execute the computing node corresponding to the computing node information Q_3.
  • the calculation graph information is generated according to at least one execution process, and the function code corresponding to the computing node can be called according to the calculation graph information to execute the post-processing algorithm, complete the post-processing process of the image to be processed, and obtain the image processing result.
  • using the configuration file to store multiple computing node information and at least one execution process information can ensure that the above-mentioned computing node information and execution process information used to generate the calculation graph information can be quickly obtained according to the configuration file, and the calculation graph information is improved. Efficiency and accuracy of information generation.
  • the standard detection result of the object to be detected in the target task stored in the history can also be obtained; according to the calculation graph information and the standard detection result, the image to be processed is processed to obtain the image processing result. It uses the pre-set task threshold information to obtain accurate image processing results.
  • the target task as a water body detection task as an example, that is, the object to be detected is a water body
  • the historically stored standard detection results for water bodies can be obtained, and then the first remote sensing image is processed according to the calculation map information to obtain the water body
  • the detection result of the target water body corresponding to the detection task and then compare the detection result of the target water body with the standard detection result for the water body stored in history. detection.
  • FIG. 5 it is an execution flowchart of an application scenario of an image processing method provided by an embodiment of the present disclosure, including:
  • the calculation graph item is used to receive the execution process and calculation node information, and generate calculation graph information.
  • the purpose of adding an execution process is to establish an execution process.
  • To add an execution process item for example, at least one execution process item may be added according to the execution process information in the configuration file. Execution process items are used to receive compute node information.
  • S405 Determine the execution process corresponding to each execution process item according to the computing node information added to each execution process item and the dependency relationship information between computing nodes corresponding to each computing node information.
  • the execution process item is completed, and then, the execution process is generated according to the dependency relationship information between computing nodes corresponding to each computing node information.
  • the execution process can call the computing node corresponding to the computing node information according to its internal computing node information.
  • S406 Generate calculation graph information according to at least one execution flow
  • S407 Call the function code corresponding to the computing node according to the computing graph information, execute the post-processing algorithm, complete the post-processing process of the image to be processed, obtain the image processing result, and end the process.
  • the simplification of the preset functions that the computing nodes can complete is maintained.
  • the post-processing algorithm needs to be adjusted, there is no need to modify the existing
  • the preset function does not need to modify the code, just change the combination of calculation node information in the post-processing calculation graph information to meet different post-processing requirements.
  • the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process.
  • the specific execution order of each step should be based on its function and possible
  • the inner logic is OK.
  • the embodiment of the present disclosure also provides an image processing device corresponding to the image processing method. Since the problem-solving principle of the device in the embodiment of the present disclosure is similar to the above-mentioned image processing method in the embodiment of the present disclosure, the implementation of the device Reference can be made to the implementation of the method, and repeated descriptions will not be repeated.
  • FIG. 6 it is a schematic diagram of an image processing device provided by an embodiment of the present disclosure, and the device includes: an image acquisition module 501, an information determination module 502, a flow generation module 503, and an image processing module 504; wherein,
  • An image acquisition module 501 configured to acquire an image to be processed
  • the information determination module 502 is configured to determine a plurality of computing node information and execution process information based on the target task corresponding to the image to be processed; wherein, the computing nodes corresponding to the plurality of computing node information are used to complete different presets Function;
  • the process generation module 503 is configured to generate an execution process corresponding to the execution process information based on the plurality of computing node information and the execution process information; wherein the execution process includes at least one computing node information;
  • the image processing module 504 is configured to generate calculation graph information based on the execution process, and process the image to be processed according to the calculation graph information to obtain an image processing result.
  • the computing node information includes input information and output information of corresponding computing nodes
  • the process generating module 503 is configured to determine dependency information among multiple computing nodes based on the input information and output information of each computing node; and generate the determined dependency information and the execution process information based on the determined Execution process corresponding to the execution process information described above.
  • the device further includes a node generating module 505 configured to determine a plurality of computing node information based on the target task corresponding to the image to be processed, and determine the computing nodes corresponding to the multiple computing node information When the nodes include undefined computing nodes, generating corresponding computing nodes based on computing node information of the undefined computing nodes;
  • the image processing module 504 is configured to call a computing node corresponding to each computing node information to process the image to be processed according to the computing graph information, and obtain an image processing result.
  • the execution process information includes the execution order of the corresponding execution process
  • the image processing module 504 is configured to generate calculation graph information based on the multiple execution processes and the execution order of each execution process when it is determined that the target task has multiple execution process information; The above execution order is used to determine the first execution process; the image to be processed is input to the first computing node corresponding to the first execution process, and the process is performed according to the first execution process and other execution processes corresponding to the execution order processing, until all the execution processes are executed, and the image processing result is obtained.
  • the execution process includes a cyclic execution process
  • the image processing module 504 is configured to obtain the output information of the previous execution flow of the cyclic execution flow, and divide the output information into a plurality of sub-information; respectively input the plurality of sub-information into the cyclic execution flow to perform processing, respectively obtaining the processing result information corresponding to each sub-information; determining the target result information based on the processing result information corresponding to each sub-information, and inputting the target result information into the next execution process of the cyclic execution process until the execution is completed All execution processes are performed to obtain the image processing results.
  • the information determining module 502 is configured to acquire a configuration file corresponding to the target task; and acquire multiple computing node information and at least one execution process information corresponding to the target task from the configuration file.
  • the image processing module 504 is configured to obtain a standard detection result of the object to be detected in the target task stored in history; perform calculation on the image to be processed according to the calculation graph information and the standard detection result processing to get the image processing result.
  • the image to be processed includes the first remote sensing image corresponding to the initial water body detection result;
  • the target task includes a water body detection task;
  • the image processing module 504 is configured to process the first remote sensing image according to the execution flow in the calculation graph information, and obtain the target water body detection result corresponding to the water body detection task.
  • the image to be processed includes a second remote sensing image corresponding to the initial building detection result;
  • the target task includes a building detection task;
  • the image processing module 504 is configured to process the second remote sensing image according to the execution flow in the calculation graph information to obtain the target building detection result corresponding to the building detection task.
  • FIG. 7 it is a schematic structural diagram of a computer device provided by an embodiment of the present disclosure, including:
  • the processor 61 executes The following steps: S101: Obtain the image to be processed; S102: Based on the target task corresponding to the image to be processed, determine the information of multiple computing nodes and the execution process information; wherein, the computing nodes corresponding to the multiple computing node information are used to complete different Preset function; S103: Based on multiple computing node information and execution process information, generate an execution process corresponding to the execution process information; wherein, the execution process includes at least one computing node information; S104: Based on the execution process, generate calculation graph information, and follow The calculation graph information processes the image to be processed to obtain the image processing result.
  • memory 62 comprises memory 621 and external memory 622;
  • Memory 621 here is also called internal memory, is used for temporarily storing the operation data in processor 61, and the data exchanged with external memory 622 such as hard disk, processor 61 communicates with memory 621 through memory 621.
  • the external memory 622 performs data exchange.
  • the processor 61 communicates with the memory 62 through the bus 63, so that the processor 61 executes the execution instructions mentioned in the above method embodiments.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the image processing method described in the foregoing method embodiments are executed.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • An embodiment of the present disclosure also provides a computer program, where the computer program includes computer readable codes, and when the computer readable codes run in an electronic device, the processor of the electronic device executes any of the above embodiments The image processing method described.
  • An embodiment of the present disclosure also provides another computer program product, including computer instructions, and when the computer instructions are executed by a processor, the steps of the above-mentioned image processing method are implemented.
  • the computer program product may be any product capable of realizing the above-mentioned image processing method, and part or all of the solutions in the computer program product that contribute to the prior art may be implemented as a software product (such as a software development kit (Software Development Kit, SDK) ), the software product can be stored in a storage medium, and the computer instructions contained therein make relevant devices or processors execute some or all of the steps of the above-mentioned image processing method.
  • the device involved in the embodiments of the present disclosure may be at least one of a system, a method, and a computer program product.
  • a computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present disclosure.
  • a computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device.
  • a computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Examples of computer-readable storage media include: portable computer disks, hard disks, Random Access Memory (RAM), Read-Only Memory (ROM), erasable Electrical Programmable Read Only Memory (EPROM) or flash memory, Static Random-Access Memory (Static Random-Access Memory, SRAM), Portable Compact Disc Read-Only Memory (CD-ROM), Digital Video Discs (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punched cards or raised structures in grooves with instructions stored thereon, and any suitable combination of the foregoing.
  • RAM Random Access Memory
  • ROM Read-Only Memory
  • EPROM erasable Electrical Programmable Read Only Memory
  • flash memory Static Random-Access Memory
  • SRAM Static Random-Access Memory
  • CD-ROM Portable Compact Disc Read-Only Memory
  • DVDs Digital Video Discs
  • memory sticks floppy disks, mechanically encoded devices such as punched cards or raised structures in grooves with instructions stored thereon, and any suitable combination of the foregoing.
  • computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.
  • the computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device over at least one of a network, such as the Internet, a local area network, a wide area network, and a wireless network.
  • the network may include at least one of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers.
  • a network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
  • Computer program instructions for performing the operations of the present disclosure may be assembly instructions, Industry Standard Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the “C” language or similar programming languages.
  • Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement.
  • the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it may be connected to an external computer (for example, using Internet Service Provider to connect via the Internet).
  • LAN Local Area Network
  • WAN Wide Area Network
  • electronic circuits such as programmable logic circuits, FPGAs, or programmable logic arrays (Programmable Logic Arrays, PLAs), can be customized by using state information of computer-readable program instructions, which can execute computer-readable Read program instructions, thereby implementing various aspects of the present disclosure.
  • the disclosed devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the modules is only a logical function division.
  • multiple modules or components can be combined.
  • some features can be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or modules may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional module in each embodiment of the present disclosure may be integrated into one processing module, each module may exist separately physically, or two or more modules may be integrated into one module.
  • the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor.
  • the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .
  • Embodiments of the present disclosure provide an image processing method, device, device, medium, and program, wherein the method includes: acquiring an image to be processed; determining multiple computing node information and executing Process information; wherein, the computing nodes corresponding to the multiple computing node information are used to complete different preset functions; based on the multiple computing node information and the execution process information, the execution process information corresponding to the execution process is generated A process; wherein, the execution process includes at least one computing node information; based on the execution process, calculation graph information is generated, and the image to be processed is processed according to the calculation graph information to obtain an image processing result.

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)

Abstract

Embodiments of the present disclosure provide an image processing method and apparatus, a device, a medium, and a program. The method is performed by an electronic device. The method comprises: acquiring an image to be processed; determining a plurality of computing node information and execution flow information on the basis of a target task corresponding to said image, wherein computing nodes respectively corresponding to the plurality of computing node information are used for completing different preset functions; generating, on the basis of the plurality of computing node information and the execution flow information, an execution flow corresponding to the execution flow information, wherein the execution flow comprises at least one computing node information; and generating calculation graph information on the basis of the execution flow, and processing said image according to the calculation graph information to obtain an image processing result. Thus, repeated storage of excessive redundant codes is reduced, the platform storage load is reduced, the use flexibility of the computing nodes is improved, and the management difficulty of a postprocessing algorithm is reduced.

Description

一种图像处理方法、装置、设备、介质及程序An image processing method, device, equipment, medium and program
相关申请的交叉引用Cross References to Related Applications
本专利申请要求2021年09月29日提交的中国专利申请号为202111152501.8、申请人为上海商汤智能科技有限公司,申请名称为“一种图像处理方法、装置、设备、介质及程序”的优先权,该申请的全文以引用的方式并入本申请中。This patent application claims the priority of the Chinese patent application number 202111152501.8 submitted on September 29, 2021, the applicant is Shanghai Shangtang Intelligent Technology Co., Ltd., and the application name is "an image processing method, device, equipment, medium and program" , which is incorporated by reference in its entirety into this application.
技术领域technical field
本公开涉及图像处理技术领域,尤其涉及一种图像处理方法、装置、设备、介质及程序。The present disclosure relates to the technical field of image processing, and in particular to an image processing method, device, equipment, medium and program.
背景技术Background technique
基于人工智能(Artificial Intelligence,AI)进行遥感图像解译的算法模型(即AI遥感检测模型)有多种,比如目标检测、影像分割、变化检测等,每种AI遥感检测模型对应有至少一套后处理算法,在应对不同的行业应用需求时,又会衍生成更多定制化的后处理功能,比如,建筑物提取模型的后处理可以包括去除小图斑、建筑物提取结果规则化、矢量化等功能,水体识别模型的后处理可以包括去除小斑块、边界模糊、边界生长、多模型结果融合等功能。针对不同的行业应用的任务需求,经常需要对后处理算法进行适当的调整,如在后处理算法中能够增加矢量化或去除小斑块等功能。There are many algorithm models for remote sensing image interpretation based on artificial intelligence (AI) (ie, AI remote sensing detection models), such as target detection, image segmentation, change detection, etc., and each AI remote sensing detection model corresponds to at least one set The post-processing algorithm will generate more customized post-processing functions when responding to different industry application requirements. For example, the post-processing of the building extraction model can include removing small image spots, regularizing the building extraction results, vector The post-processing of the water body recognition model can include functions such as removing small patches, blurring boundaries, growing boundaries, and fusion of multi-model results. According to the task requirements of different industry applications, it is often necessary to make appropriate adjustments to the post-processing algorithm, such as adding functions such as vectorization or removing small plaques in the post-processing algorithm.
相关技术中,在后处理代码经常被修改的情况下,形成了很多不同任务的后处理算法的版本,不同版本的后处理算法之间可能包括具有重复功能代码,导致支持AI遥感检测模型以及后处理算法运行的平台存储过量的冗余代码,造成平台存储负荷量大,同时不同版本的后处理算法也难以统一管理;另外每次用户需要调整后处理算法时,都需要研发人员进行修改,维护代价较高,进而导致用户体验感降低。In related technologies, when post-processing codes are often modified, many versions of post-processing algorithms for different tasks are formed. Different versions of post-processing algorithms may include codes with repeated functions, resulting in support for AI remote sensing detection models and post-processing algorithms. The platform where the processing algorithm runs stores excessive redundant codes, resulting in a large storage load on the platform. At the same time, it is difficult to uniformly manage different versions of the post-processing algorithm. In addition, every time the user needs to adjust the post-processing algorithm, R&D personnel are required to modify and maintain it. The cost is high, which in turn leads to a decrease in user experience.
发明内容Contents of the invention
本公开实施例至少提供一种图像处理方法、装置、设备、介质及程序。Embodiments of the present disclosure at least provide an image processing method, device, device, medium, and program.
第一方面,本公开实施例提供了一种图像处理方法,所述方法由电子设备执行,包括:In a first aspect, an embodiment of the present disclosure provides an image processing method, the method is executed by an electronic device, including:
获取待处理图像;Get the image to be processed;
基于所述待处理图像对应的目标任务,确定多个计算节点信息和执行流程信息;其中,所述多个计算节点信息分别对应的计算节点用于完成不同的预设功能;Based on the target task corresponding to the image to be processed, determine a plurality of computing node information and execution process information; wherein, the computing nodes corresponding to the plurality of computing node information are used to complete different preset functions;
基于所述多个计算节点信息和所述执行流程信息,生成所述执行流程信息对应的执行流程;其中,所述执行流程包括至少一个计算节点信息;Based on the plurality of computing node information and the execution process information, generate an execution process corresponding to the execution process information; wherein the execution process includes at least one computing node information;
基于所述执行流程,生成计算图信息,并按照所述计算图信息对所述待处理图像进行处理,得到图像处理结果。Based on the execution flow, calculation graph information is generated, and the image to be processed is processed according to the calculation graph information to obtain an image processing result.
本方面中,利用抽象为不同预设功能的计算节点对应的计算节点信息和执行流程信息生成的执行流程,能够将后处理算法初步抽象为具有层级的架构,即计算节点和执行流程,之后,利用执行流程组成的计算图信息对待处理图像进行处理,进一步将后处理算法抽象为具有三层架构体系,即计算节点(第一层)、执行流程(第二层)和计算图(第三层)。针对具有层级架构的后处理算法,由于计算节点用于完成预设功能(比如单一的某一功能),因此利用已有的计算节点灵活的构建后处理计算图,使其满足目标任务的处理条件,针对不同的任务不用重复存储相同功能的代码块(对应于上述计算节点),减少了重复存储过量的冗余代码,降低了平台存储负荷量,提高了计算节点使用的灵活性,以及降低了对后处理算法的管理难度。另外,构建和调整处理算法都不再需要研发人员进行代码调整,只需要工作人员配置计算图所需的信息,之后根据生成的计算图信息构建计算图对待处理图像进行处理即可,提高了图像处理的效率。In this aspect, the post-processing algorithm can be preliminarily abstracted into a hierarchical structure, that is, computing nodes and execution processes, by using the execution process generated by abstracting the computing node information and execution process information corresponding to computing nodes with different preset functions, and then, The image to be processed is processed by using the calculation graph information composed of the execution flow, and the post-processing algorithm is further abstracted into a three-layer architecture system, namely, the calculation node (first layer), the execution flow (second layer) and the calculation graph (third layer). ). For the post-processing algorithm with hierarchical architecture, since the computing nodes are used to complete the preset function (such as a single function), the existing computing nodes are used to flexibly construct the post-processing calculation graph to meet the processing conditions of the target task , for different tasks, there is no need to repeatedly store code blocks with the same function (corresponding to the above-mentioned computing nodes), which reduces redundant code storage, reduces the storage load of the platform, improves the flexibility of using computing nodes, and reduces the Difficulty managing post-processing algorithms. In addition, the construction and adjustment of processing algorithms no longer require R&D personnel to adjust the code, but only need the staff to configure the information required for the calculation graph, and then construct the calculation graph based on the generated calculation graph information to process the image to be processed, which improves the image quality. processing efficiency.
第二方面,本公开实施例还提供一种图像处理装置,包括:In a second aspect, an embodiment of the present disclosure further provides an image processing device, including:
图像获取模块,配置为获取待处理图像;An image acquisition module configured to acquire images to be processed;
信息确定模块,配置为基于所述待处理图像对应的目标任务,确定多个计算节点信息和执行流程信息;其中,所述多个计算节点信息分别对应的计算节点用于完成不同的预设功能;The information determination module is configured to determine a plurality of computing node information and execution process information based on the target task corresponding to the image to be processed; wherein, the computing nodes corresponding to the plurality of computing node information are used to complete different preset functions ;
流程生成模块,配置为基于所述多个计算节点信息和所述执行流程信息,生成所述执行流程信息对应的执行流程;其中,所述执行流程包括至少一个计算节点信息;A process generation module configured to generate an execution process corresponding to the execution process information based on the plurality of computing node information and the execution process information; wherein the execution process includes at least one computing node information;
图像处理模块,配置为基于所述执行流程,生成计算图信息,并按照所述计算图信息对所述待处理图像进行处理,得到图像处理结果。The image processing module is configured to generate calculation graph information based on the execution flow, and process the image to be processed according to the calculation graph information to obtain an image processing result.
第三方面,本公开实施例还提供一种计算机设备,包括:处理器、存储器和总线,所述存储器存储有所述处理器可执行的机器可读指令,当计算机设备运行时,所述处理器与所述存储器之间通过总线通信,所述机器可读指令被所述处理器执行时执行上述第一方面,或第一方面中任一种可能的图像处理方法的步骤。In a third aspect, an embodiment of the present disclosure further provides a computer device, including: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processing The processor communicates with the memory through a bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps of any possible image processing method in the first aspect are executed.
第四方面,本公开实施例还提供一种计算机可读存储介质,该计算机可读存储介质上存储有计算机程序,该计算机程序被处理器运行时执行上述第一方面,或第一方面中任一种可能的图像处理方法的步骤。In a fourth aspect, embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned first aspect, or any of the first aspects of the first aspect, may be executed. Steps of a possible image processing method.
本公开实施例还提供一种计算机程序,所述计算机程序包括计算机可读代码,在所述计算机可读代码在电子设备中运行的情况下,所述电子设备的处理器执行上述任一实施例所述的图像处理方法。An embodiment of the present disclosure also provides a computer program, where the computer program includes computer readable codes, and when the computer readable codes run in an electronic device, the processor of the electronic device executes any of the above embodiments The image processing method described.
关于上述图像处理装置、计算机设备和存储介质的效果描述参见上述图像处理方法的说明,这里不再赘述。For the effect description of the above image processing apparatus, computer equipment and storage medium, please refer to the description of the above image processing method, which will not be repeated here.
为使本公开的上述目的、特征和优点能更明显易懂,下文特举较佳实施例,并配合所附附图,作详细说明如下。In order to make the above-mentioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.
附图说明Description of drawings
为了更清楚地说明本公开实施例的技术方案,下面将对实施例中所需要使用的附图作简单地介绍,此处的附图被并入说明书中并构成本说明书中的一部分,这些附图示出了符合本公开的实施例,并与说明书一起用于说明本公开的技术方案。应当理解,以下附图仅示出了本公开的某些实施例,因此不应被看作是对范围的限定,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他相关的附图。In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following will briefly introduce the accompanying drawings used in the embodiments. The accompanying drawings here are incorporated into the specification and constitute a part of the specification. The drawings show the embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those skilled in the art, they can also make From these drawings other related drawings are obtained.
图1示出了本公开实施例所提供的一种图像处理方法的流程图;FIG. 1 shows a flowchart of an image processing method provided by an embodiment of the present disclosure;
图2示出了应用本公开实施例的一种图像处理方法的一种系统架构示意图;FIG. 2 shows a schematic diagram of a system architecture applying an image processing method according to an embodiment of the present disclosure;
图3示出了本公开实施例所提供的基于每个计算节点的输入信息和输出信息,确定计算节点之间的依赖关系信息的展示示意图;FIG. 3 shows a schematic diagram of determining dependency information between computing nodes based on input information and output information of each computing node provided by an embodiment of the present disclosure;
图4示出了本公开实施例所提供的计算图的展示示意图;FIG. 4 shows a schematic diagram of a calculation graph provided by an embodiment of the present disclosure;
图5示出了本公开实施例所提供的一种图像处理方法的应用场景的执行流程图;FIG. 5 shows an execution flowchart of an application scenario of an image processing method provided by an embodiment of the present disclosure;
图6示出了本公开实施例所提供的一种图像处理装置的示意图;FIG. 6 shows a schematic diagram of an image processing device provided by an embodiment of the present disclosure;
图7示出了本公开实施例所提供的一种计算机设备的结构示意图。FIG. 7 shows a schematic structural diagram of a computer device provided by an embodiment of the present disclosure.
具体实施方式Detailed ways
为使本公开实施例的目的、技术方案和优点更加清楚,下面将结合本公开实施例中附图,对本公开实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本公开一部分实施例,而不是全部的实施例。通常在此处附图中描述和示出的本公开实施例的组件可以以各种不同的配置来布置和设计。因此,以下对在附图中提供的本公开的实施例的详细描述并非旨在限制要求保护的本公开的范围,而是仅仅表示本公开的选定实施例。基于本公开的实施例,本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例,都属于本公开保护的范围。In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only It is a part of the embodiments of the present disclosure, but not all of them. The components of the disclosed embodiments generally described and illustrated in the figures herein may be arranged and designed in a variety of different configurations. Accordingly, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort shall fall within the protection scope of the present disclosure.
另外,本公开实施例中的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换,以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。In addition, the terms "first", "second" and the like in the description and claims in the embodiments of the present disclosure and the above drawings are used to distinguish similar objects, and not necessarily used to describe a specific order or sequence. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments described herein can be practiced in sequences other than those illustrated or described herein.
在本文中提及的“多个或者若干个”是指两个或两个以上。“和/或”,描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。字符“/”一般表示前后关联对象是一种“或”的关系。"Plural or several" mentioned herein means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently. The character "/" generally indicates that the contextual objects are an "or" relationship.
本公开实施例中的多个或者多种可以分别指的是至少两个或者至少两种。Multiple or multiple in the embodiments of the present disclosure may refer to at least two or at least two, respectively.
经研究发现,针对AI遥感图像解译模型的后处理算法,传统技术在应用不同任务,对后处理算法进行临时调整时,需要研发人员修改后处理算法的代码。相关技术中,在后处理代码经常被修改的情况下,形成了很多不同任务的后处理算法的版本,不同版本的后处理算法之间可能包括具有重复功能代码,导致支持AI遥感检测模型以及后处理算法运行的平台存储过量的冗余代码,造成平台存储负荷量大,同时不同版本的后处理算法也难以统一管理;另外每次用户需要调整后处理算法时,都需要研发人员进行修改,维护代价较高,进而导致用户体验感降低。After research, it is found that for the post-processing algorithm of the AI remote sensing image interpretation model, when the traditional technology is applied to different tasks and the post-processing algorithm is temporarily adjusted, the R&D personnel need to modify the code of the post-processing algorithm. In related technologies, when post-processing codes are often modified, many versions of post-processing algorithms for different tasks are formed. Different versions of post-processing algorithms may include codes with repeated functions, resulting in support for AI remote sensing detection models and post-processing algorithms. The platform where the processing algorithm runs stores excessive redundant codes, resulting in a large storage load on the platform. At the same time, it is difficult to uniformly manage different versions of the post-processing algorithm. In addition, every time the user needs to adjust the post-processing algorithm, R&D personnel are required to modify and maintain it. The cost is high, which in turn leads to a decrease in user experience.
基于上述研究,本公开实施例提供了一种图像处理方法,利用抽象为不同预设功能的计算节点对应的计算节点信息和执行流程信息生成的执行流程,能够将后处理算法初步抽象为具有层级的架构,即计算节点和执行流程,之后,利用执行流程组成的计算图信息对待处理图像进行处理,进一步将后处理算法抽象为具有三层架构体系,即计算节点(第一层)、执行流程(第二层)和计算图(第三层)。针对具有层级架构的后处理算法,由于计算节点用于完成预设功能(比如单一的某一功能),因此利用已有的计算节点灵活的构建后处理计算图,使其满足目标任务的处理条件,针对不同的任务不用重复存储相同功能的代码块(对应于上述计算节点),减少了重复存储过量的冗余代码,降低了平台存储负荷量,提高了计算节点使用的灵活性,以及降低了对后处理算法的管理难度。另外,构建和调整处理算法都不再需要研发人员进行代码调整,只需要工作人员配置计算图所需的信息,之后根据生成的计算图信息构建计算图对待处理图像进行处理即可,提高了图像处理的效率。Based on the above research, the embodiment of the present disclosure provides an image processing method, which can preliminarily abstract the post-processing algorithm into a hierarchical architecture, that is, the computing nodes and the execution process, and then use the calculation graph information composed of the execution process to process the image to be processed, and further abstract the post-processing algorithm into a three-layer architecture system, namely, the computing node (first layer), the execution process (second layer) and computational graph (third layer). For the post-processing algorithm with hierarchical architecture, since the computing nodes are used to complete the preset function (such as a single function), the existing computing nodes are used to flexibly construct the post-processing calculation graph to meet the processing conditions of the target task , for different tasks, there is no need to repeatedly store code blocks with the same function (corresponding to the above-mentioned computing nodes), which reduces redundant code storage, reduces the storage load of the platform, improves the flexibility of using computing nodes, and reduces the Difficulty managing post-processing algorithms. In addition, the construction and adjustment of processing algorithms no longer require R&D personnel to adjust the code, but only need the staff to configure the information required for the calculation graph, and then construct the calculation graph based on the generated calculation graph information to process the image to be processed, which improves the image quality. processing efficiency.
针对以上方案所存在的缺陷,均是发明人在经过实践并仔细研究后得出的结果,因此,上述问题的发现过程以及下文中本公开针对上述问题所提出的解决方案,都应该是发明人在本公开过程中对本公开做出的贡献。The defects in the above solutions are all the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure below for the above problems should be the result of the inventor Contributions made to this disclosure during the course of this disclosure.
应注意到:相似的标号和字母在下面的附图中表示类似项,因此,一旦某一项在一个附图中被定义,则在随后的附图中不需要对其进行进一步定义和解释。It should be noted that like numerals and letters denote similar items in the following figures, therefore, once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.
下面对本公开实施例中涉及到的特殊名词做进一步的解释:The special nouns involved in the embodiments of the present disclosure are further explained below:
1、后处理:图像经过AI遥感检测模型进行推理识别后,再进行一些拼接、融合、边缘模糊、细化等处理,使结果精度、展示效果得到进一步提升。1. Post-processing: After the image is reasoned and recognized by the AI remote sensing detection model, it is then subjected to some processing such as splicing, fusion, edge blurring, and refinement, so that the accuracy of the results and the display effect are further improved.
2、计算图:由计算节点(算子)和线(算子依赖关系)组成的一种抽象数据结构。2. Computation graph: an abstract data structure composed of computing nodes (operators) and lines (operator dependencies).
3、遥感解译,遥感图像的识别解析。3. Remote sensing interpretation, recognition and analysis of remote sensing images.
为便于对本实施例进行理解,首先对本公开实施例所公开的一种图像处理方法进行详细介绍,本公开实施例所提供的图像处理方法的执行主体一般为具有一定计算能力的计算机设备。在一些可能的实现方式中,该图像处理方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。In order to facilitate the understanding of this embodiment, an image processing method disclosed in the embodiment of the present disclosure is firstly introduced in detail. The image processing method provided in the embodiment of the present disclosure is generally executed by a computer device with certain computing capabilities. In some possible implementation manners, the image processing method may be implemented by a processor invoking computer-readable instructions stored in a memory.
下面以执行主体为计算机设备为例对本公开实施例提供的图像处理方法加以说明。The image processing method provided by the embodiment of the present disclosure will be described below by taking the execution subject as a computer device as an example.
为便于对本实施例进行理解,首先对本公开实施例所公开的一种图像处理方法的应用场景进行介绍,本发明实施例提供的图像处理方法可以应用于遥感图像解译领域的图像识别的后处理,即为针对遥感图像的识别解析的后处理。示例性的,针对建筑物提取模型后处理可以包括去除小图斑、建筑物提取结果规则化、矢量化等功能;针对水体识别模型的后处理可以包括去除小斑块、边界模糊、边界生长、多模型结果融合等功能。In order to facilitate the understanding of this embodiment, the application scenario of an image processing method disclosed in the embodiment of the present disclosure is firstly introduced. The image processing method provided in the embodiment of the present invention can be applied to post-processing of image recognition in the field of remote sensing image interpretation , which is the post-processing for the recognition and analysis of remote sensing images. Exemplarily, the post-processing for the building extraction model may include functions such as removal of small image spots, regularization of building extraction results, and vectorization; the post-processing for the water body identification model may include removal of small patches, boundary blurring, boundary growth, Multi-model result fusion and other functions.
下面通过本公开实施例提供的图像处理方法,以应对不同的行业应用的任务需求,对遥感AI检测模型的后处理算法进行适当的调整。In the following, the image processing method provided by the embodiment of the present disclosure is used to meet the task requirements of different industry applications, and the post-processing algorithm of the remote sensing AI detection model is appropriately adjusted.
参见图1所示,为本公开实施例提供的图像处理方法的流程图,所述方法由电子设备执行,所述方法包括步骤S101至S104,其中:Referring to FIG. 1 , which is a flow chart of an image processing method provided by an embodiment of the present disclosure, the method is executed by an electronic device, and the method includes steps S101 to S104, wherein:
S101:获取待处理图像。S101: Acquire images to be processed.
本步骤中,待处理图像可以包括已经经过AI遥感检测模型处理后输出的类别图像,即遥感解译的结果,表示AI遥感检测类别信息的图像。该待处理图像为需要进行后处理的结果图像,该结果图像可以是遥感图像,也可以不是遥感图像。In this step, the image to be processed may include the category image that has been processed by the AI remote sensing detection model, that is, the result of remote sensing interpretation, and an image representing the category information of the AI remote sensing detection. The image to be processed is a result image that needs to be post-processed, and the result image may or may not be a remote sensing image.
S102:基于待处理图像对应的目标任务,确定多个计算节点信息和执行流程信息。S102: Based on the target task corresponding to the image to be processed, determine multiple computing node information and execution process information.
在一些实施例中,多个计算节点信息分别对应的计算节点用于完成不同的预设功能。In some embodiments, the computing nodes corresponding to the multiple computing node information are used to implement different preset functions.
本步骤中,目标任务可以为对待处理图像进行后处理的任务。目标任务不同,对应的后处理算法中包含的目标功能不同。示例性的,目标任务包括水体检测任务,例如,水体检测任务可以包括前期遥感解译处理和后处理;其中,后处理可以是针对初始水体检测结果,即针对包含水体的图像进行遥感解译的结果(也即第一遥感影像),进行进一步处理的算法;该算法包含的目标功能可以为去除小斑块、边界模糊、边界生长、多模型结果融合等功能;或者,示例性的,目标任务包括建筑物检测任务,例如,建筑物检测任务可以包括前期遥感解译处理和后处理;其中,后处理可以是针对初始建筑物检测结果,即针对包含建筑物的图像进行遥感解译的结果(也即第二遥感影像),进行进一步处理的算法;该算法包含的目标功能可以为去除小图斑、建筑物提取结果规则化、矢量化等功能。In this step, the target task may be a task of post-processing the image to be processed. The target tasks are different, and the target functions contained in the corresponding post-processing algorithms are different. Exemplarily, the target task includes a water body detection task. For example, the water body detection task may include pre-remote sensing interpretation processing and post-processing; wherein, the post-processing may be performed on the initial water body detection results, that is, remote sensing interpretation for images containing water bodies The result (that is, the first remote sensing image) is an algorithm for further processing; the target function included in the algorithm can be functions such as removing small patches, boundary blurring, boundary growth, and multi-model result fusion; or, exemplary, the target task Including the building detection task, for example, the building detection task may include pre-remote sensing interpretation processing and post-processing; where the post-processing may be for the initial building detection result, that is, the result of remote sensing interpretation for the image containing the building ( That is, the second remote sensing image), an algorithm for further processing; the target functions included in the algorithm can be functions such as removing small image spots, regularizing building extraction results, and vectorizing.
这里,多个计算节点信息分别对应的计算节点用于完成不同的预设功能,其中,该预设功能可以包括单一的功能,即为目标功能中的最小功能。例如,目标功能包括建筑物提取结果规则化,则该目标功能中的最小功能包括提取功能、结果规则化功能等。计算节点可以包括能够执行单一的预设功能的功能代码。计算节点信息可以包括计算节点的标识信息,比如名称、编号等。Here, the computing nodes corresponding to the multiple computing node information are used to implement different preset functions, wherein the preset function may include a single function, which is the minimum function among the target functions. For example, if the target function includes regularization of building extraction results, then the minimum function in the target function includes extraction function, result regularization function and so on. A compute node may include function code capable of performing a single predetermined function. The computing node information may include identification information of the computing node, such as name, serial number, and the like.
以目标任务为对待处理图像进行去除小图斑、边界模糊和矢量化的后处理任务为例,基于目标任务中的第一子任务,即去除小图斑,确定去除小图斑对应的至少一个计算节点信息;基于目标任务中的第二子任务,即边界模糊,确定边界模糊对应的至少一个计算节点信息;基于目标任务中的第三子任务,即矢量化,确定矢量化对应的至少一个计算节点信息,这里,可以为确定出去除小图斑对应的计算节点A的名称和计算节点B的名称;边界模糊对应的计算节点C的名称;矢量化对应的计算节点D的名称、计算节点E的名称和计算节点F的名称等。Taking the target task as an example of the post-processing task of removing small image spots, boundary blurring and vectorization of the image to be processed, based on the first subtask in the target task, that is, removing small image spots, determine at least one corresponding to the removal of small image spots Computing node information; based on the second subtask in the target task, that is, boundary blurring, determine at least one computing node information corresponding to boundary blurring; based on the third subtask in the target task, namely vectorization, determine at least one corresponding to vectorization Computing node information, here, can be the name of computing node A and computing node B corresponding to the removal of small image spots; the name of computing node C corresponding to boundary blur; the name of computing node D corresponding to vectorization, computing node The name of E and the name of computing node F, etc.
这里,每个执行流程信息对应一条执行流程,因此,每个执行流程信息包括其对应的执行流程中所包括的至少一个计算节点信息。延续上例,三个子任务对应有三个执行任务信息,即对应有三条执行流程,其中,第一执行流程信息包括其对应的第一条执行流程中所包括的计算节点A信息和计算节点B信息;第二执行流程信息包括其对应的第二条执行流程中所包括的计算节点C信息;第三执行流程信息包括其对应的第三条执行流程中所包括的计算节点D信息,计算节点E信息和计算节点F信息。Here, each execution flow information corresponds to one execution flow, therefore, each execution flow information includes at least one computing node information included in its corresponding execution flow. Continuing the above example, the three subtasks correspond to three execution task information, that is, there are three execution processes correspondingly, where the first execution process information includes the computing node A information and computing node B information included in the corresponding first execution process ; The second execution process information includes the information of computing node C included in the corresponding second execution process; the third execution process information includes the information of computing node D included in the corresponding third execution process, computing node E information and computing node F information.
S103:基于多个计算节点信息和执行流程信息,生成执行流程信息对应的执行流程;其中,执行流程包括至少一个计算节点信息。S103: Generate an execution process corresponding to the execution process information based on a plurality of computing node information and execution process information; wherein, the execution process includes at least one computing node information.
具体实施时,可以根据执行流程信息中的至少一个计算节点信息,以及每个计算节点信息对应的每个计算节点之间的依赖关系信息,生成该执行流程信息对应的执行流程,比如,计算节点A依赖计算节点B,则可以确定执行流程的执行顺序为先执行计算节点B,后执行计算节点A,这里,执行流程包括计算节点A的名称、计算节点B的名称以及各个计算节点之间的先后执行顺序,不包括计算节点A和计算节点B的功能代码。During specific implementation, the execution process corresponding to the execution process information can be generated according to at least one computing node information in the execution process information, and the dependency relationship information between each computing node corresponding to each computing node information, for example, the computing node A depends on computing node B, so it can be determined that the execution sequence of the execution process is to execute computing node B first, and then execute computing node A. Here, the execution process includes the name of computing node A, the name of computing node B, and the relationship between computing nodes. The sequence of execution does not include the function codes of computing node A and computing node B.
这里,计算节点信息还包括对应的计算节点的输入信息和输出信息。确定每个计算节点之间的依赖关系信息,例如,可以基于每个计算节点的输入信息和输出信息,确定多个计算节点之间的依赖关系信息;基于确定的依赖关系信息和执行流程信息,生成执行流程信息对应的执行流程。其中,输入信息/输出参数可以包括以下至少之一:计算节点的输入参数/输出参数的标识、输入参数/输出参数的类型;其中,输入参数/输出参数的类型,比如,可以为图像、数字或指针等参数类型。Here, the computing node information also includes input information and output information of the corresponding computing node. Determine the dependency relationship information between each computing node, for example, determine the dependency relationship information between multiple computing nodes based on the input information and output information of each computing node; based on the determined dependency relationship information and execution process information, An execution flow corresponding to the execution flow information is generated. Wherein, the input information/output parameter may include at least one of the following: the identification of the input parameter/output parameter of the calculation node, the type of the input parameter/output parameter; wherein, the type of the input parameter/output parameter, for example, may be an image, a digital Or parameter types such as pointers.
示例性的,目标任务可以包括多个子任务,每个子任务分别对应至少一个执行流程信息,每个执行流程信息分别对应一个执行流程,因此,目标任务可以具有多个执行流程。Exemplarily, the target task may include multiple subtasks, each subtask corresponds to at least one piece of execution flow information, and each piece of execution flow information corresponds to one execution flow, therefore, the target task may have multiple execution flows.
如图2所示,为基于每个计算节点的输入信息和输出信息,确定计算节点之间的依赖关系信息的展示示意图;其中,计算节点A信息中的输入参数的标识为a,输出参数的标识为c;计算节点B信息中的输入参数的标识为b,输出参数的标识为d;计算节点C信息中的输入参数的标识为c和d,输出参数的标识为e;计算节点D信息中的输入参数的标识为d,输出参数的标识为f;计算节点E信息中的输入参数的标识为e和f,输出参数的标识为g。之后,在确定了每个计算节点的输入信息和输出信息的情况下,由于计算节点C的输入参数的标识为计算节点A的输出参数的标识,则可以 确定计算节点C依赖计算节点A,即确定计算节点C与计算节点A之间的依赖关系信息。同理,能够确定出计算节点之间的依赖关系信息包括计算节点E依赖计算节点C和计算节点D,计算节点C依赖计算节点A和计算节点B,计算节点D依赖计算节点B。As shown in Figure 2, it is a schematic diagram of determining the dependency information between computing nodes based on the input information and output information of each computing node; wherein, the identification of the input parameter in the information of computing node A is a, and the identification of the output parameter is The identification is c; the identification of input parameters in the information of computing node B is b, and the identification of output parameters is d; the identification of input parameters in information of computing node C is c and d, and the identification of output parameters is e; information of computing node D The identifier of the input parameter in is d, the identifier of the output parameter is f; the identifiers of the input parameters in the computing node E information are e and f, and the identifier of the output parameter is g. Afterwards, when the input information and output information of each computing node are determined, since the identifier of the input parameter of computing node C is the identifier of the output parameter of computing node A, it can be determined that computing node C depends on computing node A, that is, Determine dependency information between computing node C and computing node A. Similarly, it can be determined that the dependency relationship information between computing nodes includes that computing node E depends on computing nodes C and computing nodes D, computing node C depends on computing nodes A and computing nodes B, and computing node D depends on computing nodes B.
之后,基于上述确定的依赖关系信息和执行流程信息,生成执行流程信息对应的执行流程,即可以包括执行流程A、B→C,执行流程B→D和执行流程C、D→E。Afterwards, based on the determined dependency information and execution flow information, the execution flow corresponding to the execution flow information is generated, which may include execution flow A, B→C, execution flow B→D, and execution flow C, D→E.
上述利用计算节点的输入信息和输出信息,能够自动确定计算节点之间的依赖关系信息,无需指定计算节点之间的执行顺序,就能够生成执行流程信息对应的执行流程,提高了目标任务的处理效率。The above-mentioned input information and output information of computing nodes can be used to automatically determine the dependency information between computing nodes, and the execution process corresponding to the execution process information can be generated without specifying the execution sequence between computing nodes, which improves the processing of target tasks. efficiency.
S104:基于执行流程,生成计算图信息,并按照计算图信息对待处理图像进行处理,得到图像处理结果。S104: Generate calculation graph information based on the execution flow, and process the image to be processed according to the calculation graph information to obtain an image processing result.
执行流程信息可以包括对应的执行流程的执行次序。具体实施时,在确定目标任务存在多个执行流程信息的情况下,可以基于多个执行流程和每个执行流程的执行次序,生成计算图信息;之后,基于执行次序,确定首个执行流程;将待处理图像输入首个执行流程对应的首个计算节点,并按照首个执行流程、以及执行次序对应的其它执行流程进行处理,直到执行完所有的执行流程,得到图像处理结果。其中,其它执行流程包括除首个执行流程以外的,目标任务对应的多个执行流程。The execution flow information may include the execution sequence of the corresponding execution flow. During specific implementation, when it is determined that there are multiple execution process information for the target task, calculation graph information can be generated based on the multiple execution processes and the execution order of each execution process; then, based on the execution order, the first execution process is determined; Input the image to be processed into the first computing node corresponding to the first execution process, and process it according to the first execution process and other execution processes corresponding to the execution order, until all the execution processes are executed, and the image processing result is obtained. Wherein, other execution processes include multiple execution processes corresponding to the target task except the first execution process.
这里,每个执行流程包括该执行流程所要执行的至少一个计算节点对应的计算节点信息。计算图信息包括计算图以及对该计算图的描述信息。其中,计算图的描述信息可以包括描述有至少一个执行流程和每个执行流程所要执行的至少一个计算节点信息,所述计算图的描述信息用于构建计算图。Here, each execution process includes computing node information corresponding to at least one computing node to be executed by the execution process. The calculation graph information includes the calculation graph and the description information of the calculation graph. Wherein, the description information of the calculation graph may include information describing at least one execution process and at least one calculation node to be executed by each execution process, and the description information of the calculation graph is used to construct the calculation graph.
示例性的,可以按照计算图的描述信息构建计算图,之后,运行计算图中的每个执行流程,按照执行次序在执行流程中计算每个计算节点(即调用各个计算节点的执行接口,完成计算),使得图像处理能够按照执行次序有条不紊的处理完毕,得到图像处理结果。Exemplarily, the calculation graph can be constructed according to the description information of the calculation graph, and then each execution process in the calculation graph is run, and each calculation node is calculated in the execution process according to the execution order (that is, the execution interface of each calculation node is called, and the Calculation), so that the image processing can be completed in an orderly manner according to the execution order, and the image processing result can be obtained.
如图3所示,为计算图的展示示意图,包括3个执行流程,其中,首个执行流程包括三个计算节点(S_1,S_2和S_3,其中首个计算节点为S_1)及其依赖关系信息,即S_1→S_2→S_3,第二个执行流程包括两个计算节点(S_4和S_5)及其依赖关系信息,即S_4→S_5,第三个执行流程包括两个计算节点(S_6和S_7)及其依赖关系信息,即S_6→S_7。其中,执行次序分别为首先执行S_1→S_2→S_3,之后执行S_4→S_5,最后执行S_6→S_7。As shown in Figure 3, it is a schematic diagram of the calculation graph, including three execution processes, where the first execution process includes three computing nodes (S_1, S_2, and S_3, of which the first computing node is S_1) and their dependency information , that is, S_1→S_2→S_3, the second execution process includes two computing nodes (S_4 and S_5) and their dependency information, that is, S_4→S_5, and the third execution process includes two computing nodes (S_6 and S_7) and Its dependency information, namely S_6→S_7. Wherein, the execution sequence is to first execute S_1→S_2→S_3, then execute S_4→S_5, and finally execute S_6→S_7.
这里,执行流程为计算图的最小执行序列,能够完成至少一个目标功能(比如建筑物提取结果规则化),每个计算图至少具有一个执行流程,计算节点能够完成计算图中的最小预设功能(比如提取功能)。Here, the execution process is the minimum execution sequence of the calculation graph, which can complete at least one target function (such as the regularization of building extraction results), each calculation graph has at least one execution process, and the calculation node can complete the minimum preset function in the calculation graph (such as extract function).
执行目标任务,例如,如图3所示,包括三个子任务,即每个子任务对应一个执行流程。可以按照计算图分别调取每个计算节点的输入参数/输出参数的接口,以及该计算节点的功能代码的执行接口,调用计算节点的功能代码执行目标任务。具体实施时,从首个执行流程的首个计算节点开始,调用该计算节点的功能代码处理输入参数,得到输出结果,将输出结果通过该计算节点的输出接口进行存储,等待下一计算节点输入输入参数时调用。示例性的,调用首个计算节点S_1的功能代码,以及输入参数,即待处理图像,并利用S_1的功能代码对该待处理图像进行处理,将输出结果存储到内存1中,之后,从内存1中调用计算节点S_2的输入参数(即S_1的输出结果),并对上一计算节点S_1的输出结果进行进一步处理,并将处理结果存储到内存2中,依次循环,直到执行完所有的执行流程,得到图像处理结果。Executing the target task, for example, as shown in FIG. 3 , includes three subtasks, that is, each subtask corresponds to an execution process. The interface of input parameters/output parameters of each computing node and the execution interface of the function code of the computing node can be respectively called according to the calculation graph, and the function code of the computing node can be called to execute the target task. In specific implementation, starting from the first computing node of the first execution process, call the function code of the computing node to process the input parameters, obtain the output result, store the output result through the output interface of the computing node, and wait for the input of the next computing node Called when parameters are entered. Exemplarily, call the function code of the first computing node S_1, and the input parameters, that is, the image to be processed, and use the function code of S_1 to process the image to be processed, store the output result in the memory 1, and then, from the memory In 1, call the input parameters of computing node S_2 (that is, the output result of S_1), and further process the output result of the previous computing node S_1, and store the processing result in memory 2, and cycle in turn until all executions are executed process to obtain image processing results.
以目标任务包括水体检测任务为例,此时,待处理图像可以包括前期遥感解译处理后得到的初始水体检测结果对应的第一遥感影像。之后,按照计算图信息中的执行流程对第一遥感影像进行处理,得到水体检测任务对应的目标水体检测结果。Taking the target task including the water body detection task as an example, at this time, the image to be processed may include the first remote sensing image corresponding to the initial water body detection result obtained after the early remote sensing interpretation processing. Afterwards, the first remote sensing image is processed according to the execution flow in the calculation graph information, and the target water body detection result corresponding to the water body detection task is obtained.
具体实施时,可以运行计算图中的每个执行流程,按照执行次序调取执行流程中的每个计算节点的功能代码,执行水体检测任务中的后处理任务,得到目标水体检测结果。During specific implementation, each execution process in the calculation graph can be run, the function code of each calculation node in the execution process can be called according to the execution order, and the post-processing task in the water body detection task can be executed to obtain the target water body detection result.
以目标任务包括建筑物检测任务为例,此时,待处理图像可以包括前期遥感解译处理后得到的初始建筑物检测结果对应的第二遥感影像。之后,按照计算图信息中的执行流程对第二遥感影像进行处理,得到建筑物检测任务对应的目标建筑物检测结果。Taking the target task including building detection task as an example, at this time, the image to be processed may include the second remote sensing image corresponding to the initial building detection result obtained after the previous remote sensing interpretation processing. Afterwards, the second remote sensing image is processed according to the execution flow in the calculation graph information, and the target building detection result corresponding to the building detection task is obtained.
具体实施时,可以运行计算图中的每个执行流程,按照执行次序调取执行流程中的每个计算节点的功能代码,执行建筑物检测任务中的后处理任务,得到目标建筑物检测结果。During specific implementation, each execution process in the calculation graph can be run, the function code of each calculation node in the execution process can be called according to the execution order, and the post-processing task in the building detection task can be executed to obtain the detection result of the target building.
在本公开实施例中,利用抽象为单一预设功能的计算节点对应的计算节点信息和执行流程信息生成的执行流程,能够将后处理算法初步抽象为具有层级的架构,即计算节点和执行流程,之后,利用执行流程组成的计算图信息对待处理图像进行处理,进一步将后处理算法抽象为具有三层架构体系,即计算节点(第一层)、执行流程(第二层)和计算图(第三层)。针对具有层级架构的后处理算法,由于计算节点用于完成单一的预设功能,因此利用已有的计算节点灵活的构建后处理计算图,使其满足目标任务的处理条件,针对不同的任务不用重复存储相同功能的代码块(对应于上述计算节点),减少了重复存储过量的冗余代码,降低了平台存储负荷量,提高了计算节点使用的灵活性,以及降低了对后处理算法的管理难度。另外,构建和调整处理算法都不再需要研发人员进行代码调整,只需要工作人员配置计算图所需的信息,之后根据生成的计算图信息构建计算图对待处理图像进行处理即可,提高了图像处理的效率。In the embodiment of the present disclosure, the post-processing algorithm can be preliminarily abstracted into a hierarchical architecture by using the computing node information corresponding to the computing node abstracted as a single preset function and the execution process information to generate the execution process, that is, the computing node and the execution process , after that, use the calculation graph information composed of the execution process to process the image to be processed, and further abstract the post-processing algorithm into a three-layer architecture system, namely, the calculation node (the first layer), the execution process (the second layer) and the calculation graph ( the third floor). For the post-processing algorithm with a hierarchical structure, since the computing nodes are used to complete a single preset function, the existing computing nodes are used to flexibly construct the post-processing calculation graph to meet the processing conditions of the target task, and different tasks do not use Repeated storage of code blocks with the same function (corresponding to the above-mentioned computing nodes) reduces redundant redundant codes, reduces platform storage load, improves the flexibility of computing nodes, and reduces the management of post-processing algorithms difficulty. In addition, the construction and adjustment of processing algorithms no longer require R&D personnel to adjust the code, but only need the staff to configure the information required for the calculation graph, and then construct the calculation graph based on the generated calculation graph information to process the image to be processed, which improves the image quality. processing efficiency.
图4示出了应用本公开实施例的一种图像处理方法的一种系统架构示意图;如图4所示,该系统架构中包括:图像获取终端201、网络202和控制终端203。为实现支撑一个示例性应用,获取终端201和控制终端203通过网络202建立通信连接,图像获取终端201通过网络202向控制终端203上报待处理图像,控制终端203在获取到待处理图像的情况下,基于所述待处理图像对应的目标任务,确定多个计算节点信息和执行流程信息;其中,所述多个计算节点信息分别对应的计算节点用于完成不同的预设功能;基于所述多个计算节点信息和所述执行流程信息,生成所述执行流程信息对应的执行流程;其中,所述执行流程包括至少一个计算节点信息;基于所述执行流程,生成计算图信息,并按照所述计算图信息对所述待处理图像进行处理,得到图像处理结果。最后,控制终端203将图像处理结果上传至网络202,并通过网络202发送给图像获取终端201。FIG. 4 shows a schematic diagram of a system architecture applying an image processing method according to an embodiment of the present disclosure; as shown in FIG. 4 , the system architecture includes: an image acquisition terminal 201 , a network 202 and a control terminal 203 . In order to support an exemplary application, the acquisition terminal 201 and the control terminal 203 establish a communication connection through the network 202, the image acquisition terminal 201 reports the image to be processed to the control terminal 203 through the network 202, and when the control terminal 203 acquires the image to be processed , based on the target task corresponding to the image to be processed, determine a plurality of computing node information and execution process information; wherein, the computing nodes corresponding to the plurality of computing node information are used to complete different preset functions; based on the multiple computing node information and the execution process information, generating an execution process corresponding to the execution process information; wherein, the execution process includes at least one computing node information; based on the execution process, generating calculation graph information, and according to the The calculation graph information processes the image to be processed to obtain an image processing result. Finally, the control terminal 203 uploads the image processing result to the network 202 and sends it to the image acquisition terminal 201 through the network 202 .
作为示例,图像获取终端201可以包括图像采集设备,控制终端203可以包括具有视觉信息处理能力的视觉处理设备或远程服务器。网络202可以采用有线或无线连接方式。其中,当控制终端203为视觉处理设备时,图像获取终端201可以通过有线连接的方式与视觉处理设备通信连接,例如通过总线进行数据通信;当控制终端203为远程服务器时,当前场景图像获取终端201可以通过无线网络与远程服务器进行数据交互。As an example, the image acquisition terminal 201 may include an image acquisition device, and the control terminal 203 may include a vision processing device capable of processing visual information or a remote server. The network 202 may be connected in a wired or wireless manner. Wherein, when the control terminal 203 is a visual processing device, the image acquisition terminal 201 can communicate with the visual processing device through a wired connection, such as performing data communication through a bus; when the control terminal 203 is a remote server, the current scene image acquisition terminal 201 can perform data interaction with a remote server through a wireless network.
或者,在一些场景中,图像获取终端201可以是带有视频采集模组的视觉处理设备,可以是带有摄像头的主机。这时,本公开实施例的增强现实场景下的展示方法可以由图像获取终端201执行,上述系统架构可以不包含网络202和控制终端203。Or, in some scenarios, the image acquisition terminal 201 may be a vision processing device with a video capture module, or a host with a camera. At this time, the display method in the augmented reality scene of the embodiment of the present disclosure may be executed by the image acquisition terminal 201 , and the above-mentioned system architecture may not include the network 202 and the control terminal 203 .
在一些实施例中,如果S102中确定出的多个计算节点信息对应的计算节点中包括未被定义的计算节点,则在构建计算图的过程中,还需要生成该未被定义的计算节点。In some embodiments, if the computing nodes corresponding to the plurality of computing node information determined in S102 include undefined computing nodes, the undefined computing nodes need to be generated during the process of building the computing graph.
具体实施时,在确定多个计算节点信息对应的计算节点中包括未被定义的计算节点的情况下,基于未被定义的计算节点的计算节点信息,生成对应的计算节点;按照计算图信息调用每个计算节点信息对应的计算节点对待处理图像进行处理,得到图像处理结果。During specific implementation, when it is determined that the computing nodes corresponding to the multiple computing node information include undefined computing nodes, the corresponding computing nodes are generated based on the computing node information of the undefined computing nodes; call The computing node corresponding to each computing node information processes the image to be processed to obtain an image processing result.
未被定义的计算节点可以为软件中未曾实现的算法,比如,未添加该计算节点的功能代码。定义计算节点可以是将该算法(功能代码)添加到软件中。An undefined computing node may be an algorithm that has not been implemented in the software, for example, the function code of the computing node is not added. Defining a compute node may be adding the algorithm (function code) to the software.
示例性的,未被定义的计算节点,可以是已经存储有该计算节点的功能代码,但还未配置有调用该计算节点的功能代码的执行接口,例如,在确定未被定义的计算节点未配置有调用建筑物提取结果规则化目标功能中的提取功能对应的功能代码的执行接口和结果规则化功能对应的功能代码的接口的情况下,用户可以为其配置调用提取功能对应的功能代码的执行接口(可以通过配置文件进行配置),以及为其配置调用结果规则化功能对应的功能代码的执行接口。之后,按照计算图的描述信息构建计算图(包括新定义的计算节点),之后,运行计算图中的每个执行流程,在执行流程中计算每个计算节点(即调用各个计算节点的执行接口,完成计算),直到得到图像处理结果。Exemplarily, an undefined computing node may have already stored the function code of the computing node, but has not been configured with an execution interface for calling the function code of the computing node. For example, when it is determined that the undefined computing node is not In the case where the execution interface of the function code corresponding to the extraction function and the interface of the function code corresponding to the result regularization function are configured to call the building extraction result regularization target function, the user can configure the function code corresponding to the extraction function. An execution interface (configurable through a configuration file), and an execution interface for configuring the function code corresponding to the call result regularization function. Afterwards, construct a calculation graph (including newly defined computing nodes) according to the description information of the computing graph, and then run each execution process in the calculation graph, and calculate each computing node in the execution process (that is, call the execution interface of each computing node , to complete the calculation), until the image processing result is obtained.
示例性的,未被定义的计算节点,还可以是未存储有该计算节点的功能代码,以及未配置有调用该计算节点的功能代码的执行接口。例如,可以从历史任务的后处理算法中确定建筑物提取结果规则 化的功能代码,并从中确定执行提取功能对应的功能代码和结果规则化功能对应的功能代码;之后,为其配置调用提取功能对应的功能代码的执行接口(可以通过配置文件进行配置),以及为其配置调用结果规则化功能对应的功能代码的执行接口,即可实现对该未被定义的计算节点的定义,生成对应的计算节点。Exemplarily, an undefined computing node may also be that the function code of the computing node is not stored, and the execution interface for calling the function code of the computing node is not configured. For example, the function code of building extraction result regularization can be determined from the post-processing algorithm of the historical task, and the function code corresponding to the execution extraction function and the function code corresponding to the result regularization function can be determined from it; after that, the extraction function is configured and called The execution interface of the corresponding function code (which can be configured through the configuration file), and the execution interface of the function code corresponding to the call result regularization function, can realize the definition of the undefined computing node and generate the corresponding calculate node.
上述由于计算节点信息对应的计算节点中可能存在未被定义的计算节点,因此,需要根据未被定义的计算节点的计算节点信息,生成对应的计算节点,避免后续根据计算图进行任务执行时,因为调用未被定义的计算节点而出现图像处理的中断,从而提高了图像处理的效率。As mentioned above, there may be undefined computing nodes in the computing nodes corresponding to the computing node information. Therefore, it is necessary to generate corresponding computing nodes based on the computing node information of the undefined computing nodes, so as to avoid subsequent task execution based on the computing graph. Image processing is interrupted due to calling undefined computing nodes, thereby improving the efficiency of image processing.
在一些实施例中,在某一执行流程的输入信息/输出信息所占内存较大的情况下,还可以分批处理该执行流程的输入信息,得到较小内存的处理结果信息,分批处理需要循环执行该执行流程,具体实施时,在执行流程包括循环执行流程的情况下,获取循环执行流程的上一个执行流程的输出信息,并将输出信息分割为多个子信息;将多个子信息分别输入循环执行流程进行处理,分别得到每个子信息对应的处理结果信息;基于每个子信息对应的处理结果信息,确定目标结果信息,并将目标结果信息输入循环执行流程的下一个执行流程,直到执行完所有的执行流程,得到图像处理结果。In some embodiments, when the input information/output information of a certain execution process occupies a large amount of memory, the input information of the execution process can also be processed in batches to obtain processing result information with a smaller memory, and batch processing The execution flow needs to be executed cyclically. In specific implementation, when the execution flow includes a cyclic execution flow, the output information of the previous execution flow of the cyclic execution flow is obtained, and the output information is divided into multiple sub-information; the multiple sub-information are respectively Enter the cyclic execution process for processing, and obtain the processing result information corresponding to each sub-information; determine the target result information based on the processing result information corresponding to each sub-information, and input the target result information into the next execution process of the cyclic execution process until execution After completing all the execution processes, the image processing result is obtained.
这里,循环执行流程为循环执行的执行流程,其循环执行次数可以根据循环执行流程的输入信息/输出信息所占内存大小进行设定。循环执行流程的上一个执行流程的输出信息,即为循环执行流程的输入信息,将该输入信息分割为多个子信息。对所占内存较小的子信息进行处理,能够得到占用内存较小的处理结果信息(即循环执行流程的输出信息中的部分信息)。示例性的,已知预设最大内存为第三代移动通信技术(3rd-Generation,3G),在循环执行流程的输入信息所占内存为3G的情况下,由于循环执行流程的处理过程还包括很多中间信息,因此循环执行流程的输出信息所占内存要大于3G,所以,可以将上一个执行流程的输出信息分割为2个子信息,其循环执行流程的循环执行次数为2次。如果还考虑设备运行的性能影响的情况,则可以将上一个执行流程的输出信息分割为3个子信息,其循环执行流程的循环执行次数为3次。Here, the cyclic execution process is a cyclic execution process, and the number of cyclic executions can be set according to the memory size occupied by the input information/output information of the cyclic execution process. The output information of the previous execution process of the cyclic execution process is the input information of the cyclic execution process, and the input information is divided into multiple sub-information. Processing the sub-information occupying a small memory can obtain the processing result information (that is, part of the information in the output information of the cyclic execution process) occupying a small memory. Exemplarily, it is known that the preset maximum memory is the third-generation mobile communication technology (3rd-Generation, 3G). In the case that the memory occupied by the input information of the cyclic execution flow is 3G, since the processing process of the cyclic execution flow also includes There is a lot of intermediate information, so the memory occupied by the output information of the cyclic execution process is larger than 3G. Therefore, the output information of the previous execution process can be divided into 2 sub-information, and the number of cyclic executions of the cyclic execution process is 2 times. If the performance impact of device operation is also considered, the output information of the previous execution process can be divided into 3 sub-information, and the number of cyclic executions of the cyclic execution process is 3 times.
分割方式,示例性的,在上一个执行流程的输出信息所占内存大于预设值的情况下,可以是将上一个执行流程的输出信息,按照每个子信息所占内存小于预设最大内存量,均分为多个子信息。示例性的,在上一个执行流程的输出信息所占内存小于预设值,并且处理结果所占内存大于预设值的情况下,分割处理结果,使其所占内存小于预设最大内存量,之后,根据分割数量,将上一个执行流程的输出信息均分为多个子信息。Segmentation, for example, in the case where the memory occupied by the output information of the previous execution process is greater than the preset value, the output information of the previous execution process may be divided according to the memory occupied by each sub-information is less than the preset maximum memory amount , divided into multiple sub-information. Exemplarily, when the memory occupied by the output information of the previous execution process is less than the preset value, and the memory occupied by the processing result is greater than the preset value, the processing result is divided so that the memory occupied by it is less than the preset maximum memory amount, Afterwards, according to the number of divisions, the output information of the previous execution process is evenly divided into multiple sub-information.
之后,在一种情况下,可以基于分割得到的多个子信息之间的关联关系,将相关联的两个子信息作为相邻输入子信息,确定循环执行时的输入顺序;之后,按照处理后的输出顺序不断整合每个子信息对应的处理结果信息,得到目标处理结果。在另一种情况下,无顺序输入多个子信息,例如,将多个子信息分别输入循环执行流程进行处理,可以得到每个子信息对应的处理结果信息,之后,将每个子信息对应的处理结果信息做最后的整合,能够得到整合后的目标处理结果。在不同的应用场景下,其数据整合过程不同,本公开实施例不进行具体限定。Afterwards, in one case, based on the association relationship between multiple sub-information obtained by segmentation, the associated two sub-information can be used as adjacent input sub-information to determine the input sequence during loop execution; after that, according to the processed The output sequence continuously integrates the processing result information corresponding to each sub-information to obtain the target processing result. In another case, input multiple sub-information out of order, for example, input multiple sub-information respectively into the loop execution process for processing, you can get the processing result information corresponding to each sub-information, after that, process the processing result information corresponding to each sub-information After the final integration, the integrated target processing results can be obtained. In different application scenarios, the data integration process is different, which is not specifically limited in the embodiments of the present disclosure.
这里,由于需要处理的信息数据量较大,因此,可以利用循环执行流程分批处理信息分割后的子信息,对循环执行流程处理得到的每个子信息对应的处理结果信息进行融合,能够得到用于表征整个信息的处理结果的目标结果信息,从而能够使得流程继续执行。该实施方式利用循环执行流程分批处理子信息,能够降低每次处理过程所调用的计算机资源,进而降低设备损耗。Here, since the amount of information data to be processed is relatively large, the sub-information after the batch processing information of the cyclic execution flow can be used to fuse the processing result information corresponding to each sub-information obtained by the cyclic execution flow processing, and it can be obtained by using Target result information representing the processing result of the entire information, so that the process can continue to be executed. In this embodiment, the sub-information is processed in batches by means of a cyclic execution flow, which can reduce the computer resources invoked by each processing process, thereby reducing equipment loss.
在一些实施例中,用户可以预先配置目标任务对应的配置文件,之后,将配置文件下载到计算机设备中,以为执行目标任务对应的后处理功能做准备。In some embodiments, the user can pre-configure the configuration file corresponding to the target task, and then download the configuration file to the computer device to prepare for executing the post-processing function corresponding to the target task.
这里,配置文件包括多个计算节点信息、至少一个执行流程信息和运行计算图时所调用的各个计算节点的执行接口信息等,该配置文件用于为生成计算图信息配置信息。Here, the configuration file includes a plurality of computing node information, at least one execution process information, and execution interface information of each computing node called when running the computing graph, and the configuration file is used to configure information for generating the computing graph information.
针对S102,基于待处理图像对应的目标任务,确定多个计算节点信息和至少一个执行流程信息,可以通过以下过程实现:首先,获取目标任务对应的配置文件;其次,从配置文件中获取目标任务对应的多个计算节点信息和至少一个执行流程信息。For S102, based on the target task corresponding to the image to be processed, determine multiple computing node information and at least one execution process information, which can be achieved through the following process: first, obtain the configuration file corresponding to the target task; second, obtain the target task from the configuration file Corresponding pieces of computing node information and at least one execution process information.
示例性的,针对目标任务,可以从配置文件中获取至少一个执行流程信息,之后,创建至少一个 执行流程事项,分别为每个执行流程事项添加该执行流程事项对应的至少一个计算节点信息,根据每个计算节点信息对应的计算节点之间的依赖关系信息,生成执行流程。Exemplarily, for the target task, at least one execution process information can be obtained from the configuration file, and then at least one execution process item is created, and at least one computing node information corresponding to the execution process item is added to each execution process item, according to The dependency information between computing nodes corresponding to each computing node information generates an execution process.
示例性的,目标任务为进行边界模糊和去除小斑块后处理。例如,配置文件包括边界模糊功能对应的计算节点信息Q_1、去除小斑块对应的计算节点信息Q_2和Q_3、执行流程信息P_1为边界模糊后处理、执行流程信息P_2为去除小斑块后处理。之后,根据配置文件中的执行流程信息P_1,确定执行流程P_1^'为执行计算节点信息Q_1对应的计算节点;根据配置文件中的执行流程信息P_2以及Q_2与Q_3之间的依赖关系信息,确定执行流程P_2^'为先执行计算节点信息Q_2对应的计算节点,之后,执行计算节点信息Q_3对应的计算节点。之后,根据至少一个执行流程生成计算图信息,可以根据计算图信息调用计算节点对应的功能代码,即可执行后处理算法,完成对待处理图像的后处理过程,得到图像处理结果。Exemplarily, the target task is to perform boundary blurring and post-processing for removing small plaques. For example, the configuration file includes computing node information Q_1 corresponding to the boundary blur function, computing node information Q_2 and Q_3 corresponding to removing small plaques, execution process information P_1 is post-processing of boundary blurring, and execution process information P_2 is post-processing of removing small plaques. Afterwards, according to the execution process information P_1 in the configuration file, determine the execution process P_1^' as the computing node corresponding to the execution computing node information Q_1; according to the execution process information P_2 in the configuration file and the dependency information between Q_2 and Q_3, determine The execution process P_2^' is to first execute the computing node corresponding to the computing node information Q_2, and then execute the computing node corresponding to the computing node information Q_3. Afterwards, the calculation graph information is generated according to at least one execution process, and the function code corresponding to the computing node can be called according to the calculation graph information to execute the post-processing algorithm, complete the post-processing process of the image to be processed, and obtain the image processing result.
这里,利用配置文件存储多个计算节点信息和至少一个执行流程信息,能够保证根据该配置文件能够较快的获取到用于生成计算图信息的上述计算节点信息和执行流程信息,提高了计算图信息生成的高效性和准确性。Here, using the configuration file to store multiple computing node information and at least one execution process information can ensure that the above-mentioned computing node information and execution process information used to generate the calculation graph information can be quickly obtained according to the configuration file, and the calculation graph information is improved. Efficiency and accuracy of information generation.
在一些实施例中,还可以获取历史存储的目标任务中待检测对象的标准检测结果;按照计算图信息和标准检测结果,对待处理图像进行处理,得到图像处理结果。其利用预先设置好的任务阈值信息能够得到精准的图像处理结果。In some embodiments, the standard detection result of the object to be detected in the target task stored in the history can also be obtained; according to the calculation graph information and the standard detection result, the image to be processed is processed to obtain the image processing result. It uses the pre-set task threshold information to obtain accurate image processing results.
示例性的,以目标任务为水体检测任务为例,即待检测对象为水体,可以获取历史存储的、针对水体的标准检测结果,之后,按照计算图信息对第一遥感影像进行处理,得到水体检测任务对应的目标水体检测结果,之后,将目标水体检测结果与历史存储的、针对水体的标准检测结果进行对比,如果不匹配,则确定本次检测到的目标水体检测结果异常,需要重新进行检测。Exemplarily, taking the target task as a water body detection task as an example, that is, the object to be detected is a water body, the historically stored standard detection results for water bodies can be obtained, and then the first remote sensing image is processed according to the calculation map information to obtain the water body The detection result of the target water body corresponding to the detection task, and then compare the detection result of the target water body with the standard detection result for the water body stored in history. detection.
这里,不同的应用场景,任务阈值信息不同,本公开实施例不进行具体限定,不脱离本公开的范围,本领域技术人员可以针对任务阈值信息做出多种代替和修改,这些代替和修改都应落在本公开的范围内。由于任务阈值信息可以根据具体的应用场景灵活设定,因此,上述利用预先设置好的任务阈值信息来定图像处理结果,能够提高图像处理方法的实用性,以及提高图像处理结果的准确性。Here, different application scenarios have different task threshold information, and the embodiment of the present disclosure does not specifically limit it, without departing from the scope of the present disclosure. Those skilled in the art can make various substitutions and modifications for the task threshold information, and these substitutions and modifications are all should fall within the scope of this disclosure. Since the task threshold information can be flexibly set according to specific application scenarios, the use of the pre-set task threshold information to determine the image processing result can improve the practicability of the image processing method and the accuracy of the image processing result.
如图5所示,为本公开实施例提供的一种图像处理方法的应用场景的执行流程图,包括:As shown in FIG. 5, it is an execution flowchart of an application scenario of an image processing method provided by an embodiment of the present disclosure, including:
S401:创建后处理计算图事项。S401: Create post-processing calculation graph items.
该计算图事项用于接收执行流程和计算节点信息,生成计算图信息。The calculation graph item is used to receive the execution process and calculation node information, and generate calculation graph information.
S402:添加执行流程事项。S402: Add an execution flow item.
添加执行流程实现目的为建立执行流程。添加执行流程事项,例如,可以为根据配置文件中的执行流程信息,添加至少一个执行流程事项。执行流程事项用于接收计算节点信息。The purpose of adding an execution process is to establish an execution process. To add an execution process item, for example, at least one execution process item may be added according to the execution process information in the configuration file. Execution process items are used to receive compute node information.
S403:为每个执行流程事项添加计算节点信息。S403: Add computing node information for each execution process item.
S404:判断每个执行流程中的计算节点信息对应的计算节点是否被定义;如果已定义,则执行S405,如果未定义,则执行S408。S404: Determine whether the computing node corresponding to the computing node information in each execution process is defined; if it is defined, execute S405; if not, execute S408.
S405:根据每个执行流程事项添加的计算节点信息,以及每个计算节点信息对应的计算节点之间的依赖关系信息,确定每个执行流程事项对应的执行流程。S405: Determine the execution process corresponding to each execution process item according to the computing node information added to each execution process item and the dependency relationship information between computing nodes corresponding to each computing node information.
添加完计算节点信息后,该执行流程事项完成,之后,根据每个计算节点信息对应的计算节点之间的依赖关系信息,生成执行流程。该执行流程可以根据其内部的计算节点信息,调用该计算节点信息对应的计算节点。After the computing node information is added, the execution process item is completed, and then, the execution process is generated according to the dependency relationship information between computing nodes corresponding to each computing node information. The execution process can call the computing node corresponding to the computing node information according to its internal computing node information.
S406:根据至少一个执行流程生成计算图信息;S406: Generate calculation graph information according to at least one execution flow;
S407:根据计算图信息调用计算节点对应的功能代码,执行后处理算法,完成对待处理图像的后处理过程,得到图像处理结果,结束流程。S407: Call the function code corresponding to the computing node according to the computing graph information, execute the post-processing algorithm, complete the post-processing process of the image to be processed, obtain the image processing result, and end the process.
S408:定义未定义的计算节点。S408: Define an undefined computing node.
在本公开实施例中,通过将遥感解译后处理算法分解成很多个计算节点,保持了计算节点能够完成的预设功能的单一化,后处理算法需要调整时,既不需要修改已有的预设功能,也不需要修改代码,仅仅改变后处理计算图信息中计算节点信息的组合方式,即可满足不同的后处理需求。In the embodiment of the present disclosure, by decomposing the remote sensing interpretation post-processing algorithm into many computing nodes, the simplification of the preset functions that the computing nodes can complete is maintained. When the post-processing algorithm needs to be adjusted, there is no need to modify the existing The preset function does not need to modify the code, just change the combination of calculation node information in the post-processing calculation graph information to meet different post-processing requirements.
本领域技术人员可以理解,在具体实施方式的上述方法中,各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定,各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。Those skilled in the art can understand that in the above method of specific implementation, the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possible The inner logic is OK.
基于同一发明构思,本公开实施例中还提供了与图像处理方法对应的图像处理装置,由于本公开实施例中的装置解决问题的原理与本公开实施例上述图像处理方法相似,因此装置的实施可以参见方法的实施,重复之处不再赘述。Based on the same inventive concept, the embodiment of the present disclosure also provides an image processing device corresponding to the image processing method. Since the problem-solving principle of the device in the embodiment of the present disclosure is similar to the above-mentioned image processing method in the embodiment of the present disclosure, the implementation of the device Reference can be made to the implementation of the method, and repeated descriptions will not be repeated.
如图6所示,为本公开实施例提供的一种图像处理装置的示意图,所述装置包括:图像获取模块501、信息确定模块502、流程生成模块503和图像处理模块504;其中,As shown in FIG. 6 , it is a schematic diagram of an image processing device provided by an embodiment of the present disclosure, and the device includes: an image acquisition module 501, an information determination module 502, a flow generation module 503, and an image processing module 504; wherein,
图像获取模块501,配置为获取待处理图像;An image acquisition module 501 configured to acquire an image to be processed;
信息确定模块502,配置为基于所述待处理图像对应的目标任务,确定多个计算节点信息和执行流程信息;其中,所述多个计算节点信息分别对应的计算节点用于完成不同的预设功能;The information determination module 502 is configured to determine a plurality of computing node information and execution process information based on the target task corresponding to the image to be processed; wherein, the computing nodes corresponding to the plurality of computing node information are used to complete different presets Function;
流程生成模块503,配置为基于所述多个计算节点信息和所述执行流程信息,生成所述执行流程信息对应的执行流程;其中,所述执行流程包括至少一个计算节点信息;The process generation module 503 is configured to generate an execution process corresponding to the execution process information based on the plurality of computing node information and the execution process information; wherein the execution process includes at least one computing node information;
图像处理模块504,配置为基于所述执行流程,生成计算图信息,并按照所述计算图信息对所述待处理图像进行处理,得到图像处理结果。The image processing module 504 is configured to generate calculation graph information based on the execution process, and process the image to be processed according to the calculation graph information to obtain an image processing result.
在一些实施例中,所述计算节点信息包括对应的计算节点的输入信息和输出信息;In some embodiments, the computing node information includes input information and output information of corresponding computing nodes;
所述流程生成模块503,配置为基于每个计算节点的输入信息和输出信息,确定多个计算节点之间的依赖关系信息;基于确定的所述依赖关系信息和所述执行流程信息,生成所述执行流程信息对应的执行流程。The process generating module 503 is configured to determine dependency information among multiple computing nodes based on the input information and output information of each computing node; and generate the determined dependency information and the execution process information based on the determined Execution process corresponding to the execution process information described above.
在一些实施例中,所述装置还包括节点生成模块505,配置为在基于所述待处理图像对应的目标任务,确定多个计算节点信息之后,在确定所述多个计算节点信息对应的计算节点中包括未被定义的计算节点的情况下,基于所述未被定义的计算节点的计算节点信息,生成对应的计算节点;In some embodiments, the device further includes a node generating module 505 configured to determine a plurality of computing node information based on the target task corresponding to the image to be processed, and determine the computing nodes corresponding to the multiple computing node information When the nodes include undefined computing nodes, generating corresponding computing nodes based on computing node information of the undefined computing nodes;
所述图像处理模块504,配置为按照所述计算图信息调用每个计算节点信息对应的计算节点对所述待处理图像进行处理,得到图像处理结果。The image processing module 504 is configured to call a computing node corresponding to each computing node information to process the image to be processed according to the computing graph information, and obtain an image processing result.
在一些实施例中,所述执行流程信息包括对应的执行流程的执行次序;In some embodiments, the execution process information includes the execution order of the corresponding execution process;
所述图像处理模块504,配置为在确定所述目标任务存在多个所述执行流程信息的情况下,基于多个所述执行流程和每个执行流程的执行次序,生成计算图信息;基于所述执行次序,确定首个执行流程;将所述待处理图像输入所述首个执行流程对应的首个计算节点,并按照所述首个执行流程、以及所述执行次序对应的其它执行流程进行处理,直到执行完所有的执行流程,得到所述图像处理结果。The image processing module 504 is configured to generate calculation graph information based on the multiple execution processes and the execution order of each execution process when it is determined that the target task has multiple execution process information; The above execution order is used to determine the first execution process; the image to be processed is input to the first computing node corresponding to the first execution process, and the process is performed according to the first execution process and other execution processes corresponding to the execution order processing, until all the execution processes are executed, and the image processing result is obtained.
在一些实施例中,所述执行流程包括循环执行流程;In some embodiments, the execution process includes a cyclic execution process;
所述图像处理模块504,配置为获取所述循环执行流程的上一个执行流程的输出信息,并将所述输出信息分割为多个子信息;将所述多个子信息分别输入所述循环执行流程进行处理,分别得到每个子信息对应的处理结果信息;基于每个子信息对应的处理结果信息,确定目标结果信息,并将所述目标结果信息输入所述循环执行流程的下一个执行流程,直到执行完所有的执行流程,得到所述图像处理结果。The image processing module 504 is configured to obtain the output information of the previous execution flow of the cyclic execution flow, and divide the output information into a plurality of sub-information; respectively input the plurality of sub-information into the cyclic execution flow to perform processing, respectively obtaining the processing result information corresponding to each sub-information; determining the target result information based on the processing result information corresponding to each sub-information, and inputting the target result information into the next execution process of the cyclic execution process until the execution is completed All execution processes are performed to obtain the image processing results.
在一些实施例中,所述信息确定模块502,配置为获取所述目标任务对应的配置文件;从所述配置文件中获取所述目标任务对应的多个计算节点信息和至少一个执行流程信息。In some embodiments, the information determining module 502 is configured to acquire a configuration file corresponding to the target task; and acquire multiple computing node information and at least one execution process information corresponding to the target task from the configuration file.
在一些实施例中,所述图像处理模块504,配置为获取历史存储的所述目标任务中待检测对象的标准检测结果;按照计算图信息和所述标准检测结果,对所述待处理图像进行处理,得到图像处理结果。In some embodiments, the image processing module 504 is configured to obtain a standard detection result of the object to be detected in the target task stored in history; perform calculation on the image to be processed according to the calculation graph information and the standard detection result processing to get the image processing result.
在一些实施例中,所述待处理图像包括初始水体检测结果对应的第一遥感影像;所述目标任务包括水体检测任务;In some embodiments, the image to be processed includes the first remote sensing image corresponding to the initial water body detection result; the target task includes a water body detection task;
所述图像处理模块504,配置为按照所述计算图信息中的执行流程对所述第一遥感影像进行处理,得到所述水体检测任务对应的目标水体检测结果。The image processing module 504 is configured to process the first remote sensing image according to the execution flow in the calculation graph information, and obtain the target water body detection result corresponding to the water body detection task.
在一些实施例中,所述待处理图像包括初始建筑物检测结果对应的第二遥感影像;所述目标任务包括建筑物检测任务;In some embodiments, the image to be processed includes a second remote sensing image corresponding to the initial building detection result; the target task includes a building detection task;
所述图像处理模块504,配置为按照所述计算图信息中的执行流程对所述第二遥感影像进行处理,得到所述建筑物检测任务对应的目标建筑物检测结果。The image processing module 504 is configured to process the second remote sensing image according to the execution flow in the calculation graph information to obtain the target building detection result corresponding to the building detection task.
关于图像处理装置中的各模块的处理流程、以及各模块之间的交互流程的描述可以参照上述图像处理方法实施例中的相关说明,这里不再详述。For the description of the processing flow of each module in the image processing device and the interaction flow between the modules, reference may be made to the relevant description in the above embodiment of the image processing method, which will not be described in detail here.
基于同一技术构思,本公开实施例还提供了一种计算机设备。参照图7所示,为本公开实施例提供的计算机设备的结构示意图,包括:Based on the same technical idea, the embodiment of the present disclosure also provides a computer device. Referring to FIG. 7 , it is a schematic structural diagram of a computer device provided by an embodiment of the present disclosure, including:
处理器61、存储器62和总线63。其中,存储器62存储有处理器61可执行的机器可读指令,处理器61用于执行存储器62中存储的机器可读指令,所述机器可读指令被处理器61执行时,处理器61执行下述步骤:S101:获取待处理图像;S102:基于待处理图像对应的目标任务,确定多个计算节点信息和执行流程信息;其中,多个计算节点信息分别对应的计算节点用于完成不同的预设功能;S103:基于多个计算节点信息和执行流程信息,生成执行流程信息对应的执行流程;其中,执行流程包括至少一个计算节点信息;S104:基于执行流程,生成计算图信息,并按照计算图信息对待处理图像进行处理,得到图像处理结果。 processor 61 , memory 62 and bus 63 . Wherein, the memory 62 stores machine-readable instructions executable by the processor 61, and the processor 61 is used to execute the machine-readable instructions stored in the memory 62. When the machine-readable instructions are executed by the processor 61, the processor 61 executes The following steps: S101: Obtain the image to be processed; S102: Based on the target task corresponding to the image to be processed, determine the information of multiple computing nodes and the execution process information; wherein, the computing nodes corresponding to the multiple computing node information are used to complete different Preset function; S103: Based on multiple computing node information and execution process information, generate an execution process corresponding to the execution process information; wherein, the execution process includes at least one computing node information; S104: Based on the execution process, generate calculation graph information, and follow The calculation graph information processes the image to be processed to obtain the image processing result.
上述存储器62包括内存621和外部存储器622;这里的内存621也称内存储器,用于暂时存放处理器61中的运算数据,以及与硬盘等外部存储器622交换的数据,处理器61通过内存621与外部存储器622进行数据交换,当计算机设备运行时,处理器61与存储器62之间通过总线63通信,使得处理器61在执行上述方法实施例中所提及的执行指令。Above-mentioned memory 62 comprises memory 621 and external memory 622; Memory 621 here is also called internal memory, is used for temporarily storing the operation data in processor 61, and the data exchanged with external memory 622 such as hard disk, processor 61 communicates with memory 621 through memory 621. The external memory 622 performs data exchange. When the computer device is running, the processor 61 communicates with the memory 62 through the bus 63, so that the processor 61 executes the execution instructions mentioned in the above method embodiments.
本公开实施例还提供一种计算机可读存储介质,该计算机可读存储介质上存储有计算机程序,该计算机程序被处理器运行时执行上述方法实施例中所述的图像处理方法的步骤。其中,该存储介质可以是易失性或非易失的计算机可读取存储介质。Embodiments of the present disclosure further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the steps of the image processing method described in the foregoing method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.
本公开实施例还提供一种计算机程序,所述计算机程序包括计算机可读代码,在所述计算机可读代码在电子设备中运行的情况下,所述电子设备的处理器执行上述任一实施例所述的图像处理方法。An embodiment of the present disclosure also provides a computer program, where the computer program includes computer readable codes, and when the computer readable codes run in an electronic device, the processor of the electronic device executes any of the above embodiments The image processing method described.
本公开实施例还提供另一种计算机程序产品,包括计算机指令,所述计算机指令被处理器执行时实现上述的图像处理方法的步骤。其中,计算机程序产品可以是任何能实现上述图像处理方法的产品,该计算机程序产品中对现有技术做出贡献的部分或全部方案可以以软件产品(例如软件开发包(Software Development Kit,SDK))的形式体现,该软件产品可以被存储在一个存储介质中,通过包含的计算机指令使得相关设备或处理器执行上述图像处理方法的部分或全部步骤。An embodiment of the present disclosure also provides another computer program product, including computer instructions, and when the computer instructions are executed by a processor, the steps of the above-mentioned image processing method are implemented. Wherein, the computer program product may be any product capable of realizing the above-mentioned image processing method, and part or all of the solutions in the computer program product that contribute to the prior art may be implemented as a software product (such as a software development kit (Software Development Kit, SDK) ), the software product can be stored in a storage medium, and the computer instructions contained therein make relevant devices or processors execute some or all of the steps of the above-mentioned image processing method.
本公开实施例中涉及的设备可以是系统、方法和计算机程序产品中的至少之一。计算机程序产品可以包括计算机可读存储介质,其上载有用于使处理器实现本公开的各个方面的计算机可读程序指令。The device involved in the embodiments of the present disclosure may be at least one of a system, a method, and a computer program product. A computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present disclosure.
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是但不限于电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的例子(非穷举的列表)包括:便携式计算机盘、硬盘、随机存取存储器(Random Access Memory,RAM)、只读存储器(Read-Only Memory,ROM)、可擦除可编程只读存储器(Electrical Programmable Read Only Memory,EPROM)或闪存、静态随机存取存储器(Static Random-Access Memory,SRAM)、便携式压缩盘只读存储器(Compact Disc Read-Only Memory,CD-ROM)、数字多功能盘(Digital Video Disc,DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身,诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如,通过光纤电缆的光脉冲)、或者通过电线传输的电信号。A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. Examples of computer-readable storage media (a non-exhaustive list) include: portable computer disks, hard disks, Random Access Memory (RAM), Read-Only Memory (ROM), erasable Electrical Programmable Read Only Memory (EPROM) or flash memory, Static Random-Access Memory (Static Random-Access Memory, SRAM), Portable Compact Disc Read-Only Memory (CD-ROM), Digital Video Discs (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punched cards or raised structures in grooves with instructions stored thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备,或者通过网络、例如因特网、局域网、广域网和无线网中的至少之一下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和边缘服务器中的至少之一。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令,并转发该计算机可读程序指令,以供存储在各个计算/处理设备中的计算机可读存储介质中。The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device over at least one of a network, such as the Internet, a local area network, a wide area network, and a wireless network. . The network may include at least one of copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .
用于执行本公开操作的计算机程序指令可以是汇编指令、指令集架构(Industry Standard Architecture,ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码,所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等,以及常规的过程式编程语言,诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中,远程计算机可以通过任意种类的网络,包括局域网(Local Area Network,LAN)或广域网(Wide Area Network,WAN)连接到用户计算机,或者,可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中,通过利用计算机可读程序指令的状态信息来个性化定制电子电路,例如可编程逻辑电路、FPGA或可编程逻辑阵列(Programmable Logic Arrays,PLA),该电子电路可以执行计算机可读程序指令,从而实现本公开的各个方面。Computer program instructions for performing the operations of the present disclosure may be assembly instructions, Industry Standard Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages, such as the “C” language or similar programming languages. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In cases involving a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it may be connected to an external computer (for example, using Internet Service Provider to connect via the Internet). In some embodiments, electronic circuits, such as programmable logic circuits, FPGAs, or programmable logic arrays (Programmable Logic Arrays, PLAs), can be customized by using state information of computer-readable program instructions, which can execute computer-readable Read program instructions, thereby implementing various aspects of the present disclosure.
所属领域的技术人员可以清楚地了解到,为描述的方便和简洁,上述描述的装置的具体工作过程,可以参考前述方法实施例中的对应过程,在此不再赘述。在本公开所提供的几个实施例中,应该理解到,所揭露的装置和方法,可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的,例如,所述模块的划分,仅仅为一种逻辑功能划分,实际实现时可以有另外的划分方式,又例如,多个模块或组件可以结合,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口,装置或模块的间接耦合或通信连接,可以是电性,机械或其它的形式。Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the device described above can refer to the corresponding process in the foregoing method embodiment, and details are not repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods may be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined. Or some features can be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interfaces, and the indirect coupling or communication connection of devices or modules may be in electrical, mechanical or other forms.
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
另外,在本公开各个实施例中的各功能模块可以集成在一个处理模块中,也可以是各个模块单独物理存在,也可以两个或两个以上模块集成在一个模块中。In addition, each functional module in each embodiment of the present disclosure may be integrated into one processing module, each module may exist separately physically, or two or more modules may be integrated into one module.
所述功能如果以软件功能模块的形式实现并作为独立的产品销售或使用时,可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解,本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、移动硬盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(Random Access Memory,RAM)、磁碟或者光盘等各种可以存储程序代码的介质。If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disc and other media that can store program codes. .
最后应说明的是:以上所述实施例,仅为本公开的具体实施方式,用以说明本公开的技术方案,而非对其限制,本公开的保护范围并不局限于此,尽管参照前述实施例对本公开进行了详细的说明,本领域的普通技术人员应当理解:任何熟悉本技术领域的技术人员在本公开揭露的技术范围内,其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化,或者对其中部分技术特征进行等同替换;而这些修改、变化或者替换,并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围,都应涵盖在本公开的保护范围之内。因此,本公开的保护范围应所述以权利要求的保护范围为准。Finally, it should be noted that: the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, rather than limit them, and the protection scope of the present disclosure is not limited thereto, although referring to the aforementioned The embodiments have described the present disclosure in detail, and those skilled in the art should understand that any person familiar with the technical field can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present disclosure Changes can be easily imagined, or equivalent replacements can be made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in this disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be defined by the protection scope of the claims.
工业实用性Industrial Applicability
本公开实施例提供了一种图像处理方法、装置、设备、介质及程序,其中,该方法包括:获取待处理图像;基于所述待处理图像对应的目标任务,确定多个计算节点信息和执行流程信息;其中,所述多个计算节点信息分别对应的计算节点用于完成不同的预设功能;基于所述多个计算节点信息和所述执行流程信息,生成所述执行流程信息对应的执行流程;其中,所述执行流程包括至少一个计算节点信息;基于所述执行流程,生成计算图信息,并按照所述计算图信息对所述待处理图像进行处理,得到图像处理结果。Embodiments of the present disclosure provide an image processing method, device, device, medium, and program, wherein the method includes: acquiring an image to be processed; determining multiple computing node information and executing Process information; wherein, the computing nodes corresponding to the multiple computing node information are used to complete different preset functions; based on the multiple computing node information and the execution process information, the execution process information corresponding to the execution process is generated A process; wherein, the execution process includes at least one computing node information; based on the execution process, calculation graph information is generated, and the image to be processed is processed according to the calculation graph information to obtain an image processing result.

Claims (12)

  1. 一种图像处理方法,所述方法由电子设备执行,包括:An image processing method, the method being executed by an electronic device, comprising:
    获取待处理图像;Get the image to be processed;
    基于所述待处理图像对应的目标任务,确定多个计算节点信息和执行流程信息;其中,所述多个计算节点信息分别对应的计算节点用于完成不同的预设功能;Based on the target task corresponding to the image to be processed, determine a plurality of computing node information and execution process information; wherein, the computing nodes corresponding to the plurality of computing node information are used to complete different preset functions;
    基于所述多个计算节点信息和所述执行流程信息,生成所述执行流程信息对应的执行流程;其中,所述执行流程包括至少一个计算节点信息;Based on the plurality of computing node information and the execution process information, generate an execution process corresponding to the execution process information; wherein the execution process includes at least one computing node information;
    基于所述执行流程,生成计算图信息,并按照所述计算图信息对所述待处理图像进行处理,得到图像处理结果。Based on the execution flow, calculation graph information is generated, and the image to be processed is processed according to the calculation graph information to obtain an image processing result.
  2. 根据权利要求1所述的方法,其中,所述计算节点信息包括对应的计算节点的输入信息和输出信息;The method according to claim 1, wherein the computing node information includes input information and output information of a corresponding computing node;
    所述基于多个计算节点信息和所述执行流程信息,生成所述执行流程信息对应的执行流程,包括:The generating the execution flow corresponding to the execution flow information based on the multiple computing node information and the execution flow information includes:
    基于每个计算节点的输入信息和输出信息,确定多个计算节点之间的依赖关系信息;Determine dependency information among multiple computing nodes based on input information and output information of each computing node;
    基于确定的所述依赖关系信息和所述执行流程信息,生成所述执行流程信息对应的执行流程。Based on the determined dependency relationship information and the execution flow information, an execution flow corresponding to the execution flow information is generated.
  3. 根据权利要求1或2所述的方法,其中,在基于所述待处理图像对应的目标任务,确定多个计算节点信息之后,还包括:The method according to claim 1 or 2, wherein, after determining a plurality of computing node information based on the target task corresponding to the image to be processed, further comprising:
    在确定所述多个计算节点信息对应的计算节点中包括未被定义的计算节点的情况下,基于所述未被定义的计算节点的计算节点信息,生成对应的计算节点;When it is determined that the computing nodes corresponding to the plurality of computing node information include undefined computing nodes, generating corresponding computing nodes based on the computing node information of the undefined computing nodes;
    所述按照所述计算图信息对所述待处理图像进行处理,得到图像处理结果,包括:The processing of the image to be processed according to the calculation graph information to obtain an image processing result includes:
    按照所述计算图信息调用每个计算节点信息对应的计算节点对所述待处理图像进行处理,得到图像处理结果。The computing node corresponding to each computing node information is invoked according to the computing graph information to process the image to be processed to obtain an image processing result.
  4. 根据权利要求1至3中任意一项所述的方法,其中,所述执行流程信息包括对应的执行流程的执行次序;The method according to any one of claims 1 to 3, wherein the execution process information includes the execution order of the corresponding execution process;
    所述基于所述执行流程,生成计算图信息,包括:The calculation graph information is generated based on the execution process, including:
    在确定所述目标任务存在多个所述执行流程信息的情况下,基于多个所述执行流程和每个执行流程的执行次序,生成计算图信息;When it is determined that there are multiple execution process information for the target task, generating calculation graph information based on the multiple execution processes and the execution order of each execution process;
    所述按照所述计算图信息对所述待处理图像进行处理,得到图像处理结果,包括:The processing of the image to be processed according to the calculation graph information to obtain an image processing result includes:
    基于所述执行次序,确定首个执行流程;Determining a first execution flow based on the execution sequence;
    将所述待处理图像输入所述首个执行流程对应的首个计算节点,并按照所述首个执行流程、以及所述执行次序对应的其它执行流程进行处理,直到执行完所有的执行流程,得到所述图像处理结果。Input the image to be processed into the first computing node corresponding to the first execution process, and process it according to the first execution process and other execution processes corresponding to the execution order until all the execution processes are executed, Obtain the image processing result.
  5. 根据权利要求4所述的方法,其中,所述执行流程包括循环执行流程;The method according to claim 4, wherein the execution process comprises a cyclic execution process;
    所述按照所述对所述待处理图像进行处理,得到图像处理结果,包括:The processing of the image to be processed according to the description to obtain an image processing result includes:
    获取所述循环执行流程的上一个执行流程的输出信息,并将所述输出信息分割为多个子信息;Obtaining the output information of the previous execution process of the cyclic execution process, and dividing the output information into a plurality of sub-information;
    将所述多个子信息分别输入所述循环执行流程进行处理,分别得到每个子信息对应的处理结果信息;Input the plurality of sub-information respectively into the loop execution flow for processing, and respectively obtain the processing result information corresponding to each sub-information;
    基于每个子信息对应的处理结果信息,确定目标结果信息,并将所述目标结果信息输入所述循环执行流程的下一个执行流程,直到执行完所有的执行流程,得到所述图像处理结果。Determine the target result information based on the processing result information corresponding to each sub-information, and input the target result information into the next execution process of the cyclic execution process until all the execution processes are executed to obtain the image processing result.
  6. 根据权利要求1至5中任意一项所述的方法,其中,所述基于所述待处理图像对应的目标任务,确定多个计算节点信息和至少一个执行流程信息,包括:The method according to any one of claims 1 to 5, wherein said determining a plurality of computing node information and at least one execution process information based on the target task corresponding to the image to be processed includes:
    获取所述目标任务对应的配置文件;Obtain a configuration file corresponding to the target task;
    从所述配置文件中获取所述目标任务对应的多个计算节点信息和至少一个执行流程信息。Acquiring multiple computing node information and at least one execution process information corresponding to the target task from the configuration file.
  7. 根据权利要求1至6中任意一项所述的方法,其中,所述按照计算图信息对所述待处理图像 进行处理,得到图像处理结果,包括:The method according to any one of claims 1 to 6, wherein, the processing of the image to be processed according to the calculation graph information to obtain an image processing result includes:
    获取历史存储的所述目标任务中待检测对象的标准检测结果;Obtaining the standard detection results of the object to be detected in the target task stored in history;
    按照计算图信息和所述标准检测结果,对所述待处理图像进行处理,得到图像处理结果。According to the calculation graph information and the standard detection result, the image to be processed is processed to obtain an image processing result.
  8. 根据权利要求1至7中任意一项所述的方法,其中,所述待处理图像包括初始水体检测结果对应的第一遥感影像和初始建筑物检测结果对应的第二遥感影像中的至少一项;The method according to any one of claims 1 to 7, wherein the image to be processed includes at least one of the first remote sensing image corresponding to the initial water body detection result and the second remote sensing image corresponding to the initial building detection result ;
    所述按照所述计算图信息对所述待处理图像进行处理,得到图像处理结果,包括以下至少一项:The processing of the image to be processed according to the calculation graph information to obtain an image processing result includes at least one of the following:
    按照所述计算图信息中的执行流程对所述第一遥感影像进行处理,得到所述目标任务中水体检测任务对应的目标水体检测结果;Process the first remote sensing image according to the execution flow in the calculation graph information, and obtain the target water body detection result corresponding to the water body detection task in the target task;
    按照所述计算图信息中的执行流程对所述第二遥感影像进行处理,得到所述目标任务中建筑物检测任务对应的目标建筑物检测结果。The second remote sensing image is processed according to the execution flow in the calculation graph information to obtain a target building detection result corresponding to the building detection task in the target task.
  9. 一种图像处理装置,包括:An image processing device, comprising:
    图像获取模块,配置为获取待处理图像;An image acquisition module configured to acquire images to be processed;
    信息确定模块,配置为基于所述待处理图像对应的目标任务,确定多个计算节点信息和执行流程信息;其中,所述多个计算节点信息分别对应的计算节点用于完成不同的预设功能;The information determination module is configured to determine a plurality of computing node information and execution process information based on the target task corresponding to the image to be processed; wherein, the computing nodes corresponding to the plurality of computing node information are used to complete different preset functions ;
    流程生成模块,配置为基于所述多个计算节点信息和所述执行流程信息,生成所述执行流程信息对应的执行流程;其中,所述执行流程包括至少一个计算节点信息;A process generation module configured to generate an execution process corresponding to the execution process information based on the plurality of computing node information and the execution process information; wherein the execution process includes at least one computing node information;
    图像处理模块,配置为基于所述执行流程,生成计算图信息,并按照所述计算图信息对所述待处理图像进行处理,得到图像处理结果。The image processing module is configured to generate calculation graph information based on the execution flow, and process the image to be processed according to the calculation graph information to obtain an image processing result.
  10. 一种计算机设备,包括:处理器、存储器和总线,所述存储器存储有所述处理器可执行的机器可读指令,当计算机设备运行时,所述处理器与所述存储器之间通过总线通信,所述机器可读指令被所述处理器执行时执行如权利要求1至8中任意一项所述的图像处理方法的步骤。A computer device, comprising: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory through the bus , when the machine-readable instructions are executed by the processor, the steps of the image processing method according to any one of claims 1 to 8 are executed.
  11. 一种计算机可读存储介质,该计算机可读存储介质上存储有计算机程序,该计算机程序被处理器运行时执行如权利要求1至8中任意一项所述的图像处理方法的步骤。A computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 8 are executed.
  12. 一种计算机程序,所述计算机程序包括计算机可读代码,在所述计算机可读代码在电子设备中运行的情况下,所述电子设备的处理器执行用于实现如权利要求1至8任一所述的图像处理方法。A computer program, the computer program comprising computer readable code, in the case of the computer readable code running in an electronic device, the processor of the electronic device executes to implement any one of claims 1 to 8 The image processing method described.
PCT/CN2022/083772 2021-09-29 2022-03-29 Image processing method and apparatus, device, medium, and program WO2023050745A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111152501.8 2021-09-29
CN202111152501.8A CN113887396A (en) 2021-09-29 2021-09-29 Image processing method and device, computer equipment and storage medium

Publications (1)

Publication Number Publication Date
WO2023050745A1 true WO2023050745A1 (en) 2023-04-06

Family

ID=79008162

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/083772 WO2023050745A1 (en) 2021-09-29 2022-03-29 Image processing method and apparatus, device, medium, and program

Country Status (2)

Country Link
CN (1) CN113887396A (en)
WO (1) WO2023050745A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116985830A (en) * 2023-07-26 2023-11-03 小米汽车科技有限公司 Vehicle mode operation method and device, vehicle and storage medium

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113887396A (en) * 2021-09-29 2022-01-04 上海商汤智能科技有限公司 Image processing method and device, computer equipment and storage medium
CN114647472B (en) * 2022-03-24 2023-08-15 北京字跳网络技术有限公司 Picture processing method, apparatus, device, storage medium, and program product

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106815070A (en) * 2016-12-30 2017-06-09 中国科学院计算技术研究所 A kind of high-performance calculation frame method and system for graphical analysis
CN107967166A (en) * 2017-10-18 2018-04-27 东莞理工学院 Remote sensing image processing service flow implementation method under a kind of cloud environment
CN110968321A (en) * 2019-10-25 2020-04-07 浙江省北大信息技术高等研究院 Tensor calculation code optimization method, device, equipment and medium
CN111831285A (en) * 2020-06-23 2020-10-27 西安电子科技大学 Code conversion method, system and application for memory computing platform
US10884755B1 (en) * 2019-07-31 2021-01-05 International Business Machines Corporation Graph rewriting for large model support using categorized topological sort
CN113887396A (en) * 2021-09-29 2022-01-04 上海商汤智能科技有限公司 Image processing method and device, computer equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106815070A (en) * 2016-12-30 2017-06-09 中国科学院计算技术研究所 A kind of high-performance calculation frame method and system for graphical analysis
CN107967166A (en) * 2017-10-18 2018-04-27 东莞理工学院 Remote sensing image processing service flow implementation method under a kind of cloud environment
US10884755B1 (en) * 2019-07-31 2021-01-05 International Business Machines Corporation Graph rewriting for large model support using categorized topological sort
CN110968321A (en) * 2019-10-25 2020-04-07 浙江省北大信息技术高等研究院 Tensor calculation code optimization method, device, equipment and medium
CN111831285A (en) * 2020-06-23 2020-10-27 西安电子科技大学 Code conversion method, system and application for memory computing platform
CN113887396A (en) * 2021-09-29 2022-01-04 上海商汤智能科技有限公司 Image processing method and device, computer equipment and storage medium

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116985830A (en) * 2023-07-26 2023-11-03 小米汽车科技有限公司 Vehicle mode operation method and device, vehicle and storage medium

Also Published As

Publication number Publication date
CN113887396A (en) 2022-01-04

Similar Documents

Publication Publication Date Title
WO2023050745A1 (en) Image processing method and apparatus, device, medium, and program
US10430250B2 (en) Decomposing monolithic application into microservices
US10831733B2 (en) Interactive adjustment of decision rules
CN108829518B (en) Method and device for pushing information
US20170090990A1 (en) Modeling resource usage for a job
US9569203B2 (en) Service-based integration of application patterns
CN108885686B (en) Cloud-based active debugging system for video analysis
CN110633959A (en) Method, device, equipment and medium for creating approval task based on graph structure
CN115392501A (en) Data acquisition method and device, electronic equipment and storage medium
CN113449842A (en) Distributed automatic differentiation method and related device
CN116569141A (en) Workflow repair
JP2023553220A (en) Process mining for multi-instance processes
CN114091672A (en) Distributed model reasoning method and device, electronic device and medium
US10671891B2 (en) Reducing computational costs of deep reinforcement learning by gated convolutional neural network
CN111626401B (en) Operation method and device
CN112753016A (en) Management method and device for computing resources in data preprocessing stage in neural network
CN114548407A (en) Hierarchical target oriented cause and effect discovery method and device and electronic equipment
CN114595047A (en) Batch task processing method and device
CN110825461B (en) Data processing method and device
US20190377968A1 (en) Image cataloger based on gridded color histogram analysis
CN111815654A (en) Method, apparatus, device and computer readable medium for processing image
CN116483482B (en) Deep learning task processing method, system, device, equipment and medium
CN113469364B (en) Reasoning platform, method and device
US11120313B2 (en) Generating search determinations for assortment planning using visual sketches
WO2023184810A1 (en) Application definition method and apparatus, electronic device, storage medium, and computer program product

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE