CN114511483A

CN114511483A - Image processing method, device, equipment, system and storage medium

Info

Publication number: CN114511483A
Application number: CN202111628214.XA
Authority: CN
Inventors: 黄敬易
Original assignee: Guangzhou Xaircraft Technology Co Ltd
Current assignee: Guangzhou Xaircraft Technology Co Ltd
Priority date: 2021-12-27
Filing date: 2021-12-27
Publication date: 2022-05-17

Abstract

The application provides a method, a device, equipment, a system and a storage medium for image processing, comprising the following steps: acquiring an image to be processed, wherein the image to be processed comprises a plurality of sub-images, and an overlapping area exists between adjacent sub-images; processing each sub-image to obtain a sub-tensor of each sub-image; for each overlapped area, processing the overlapped area based on the sub tensors of at least two sub images corresponding to the overlapped area so as to update and obtain an overlapped area fusion tensor of the overlapped area; updating an output tensor of the image to be processed based on the fold fusion tensor. The method in the embodiment of the application can effectively reduce the splicing trace and the grid effect, thereby improving the image processing effect.

Description

Image processing method, device, equipment, system and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, a system, and a storage medium for image processing.

Background

In recent years, with the rapid development of mechanization and intellectualization, many countries begin to research unmanned aerial vehicles, and as one of the most extensive applications of unmanned aerial vehicles, unmanned aerial vehicles are also more and more widely applied in the agricultural field.

For example, can utilize unmanned aerial vehicle to carry out the aerial photography to large tracts of land farmland, soil to image to taking photo by plane is handled and is analyzed, with farmland information monitoring such as realization sick and pest monitoring, irrigation condition monitoring and crops growth condition monitoring, thereby the peasant of being convenient for carries out field management better. Compare with conventional monitoring means, use unmanned aerial vehicle can reduce the reliance to artifical monitoring on the spot, promote agricultural production's efficiency.

When the aerial image is processed, the aerial image can be subjected to image segmentation so as to segment different types of crops or regions in the aerial image, such as corns, flue-cured tobaccos, coix seeds, buildings and the like. However, at present, the image segmentation processing and then the splicing of the aerial images are not ideal in the splicing effect of the obtained images.

Disclosure of Invention

In view of this, embodiments of the present application provide a method, an apparatus, a device, a system, and a storage medium for image processing, which can improve the effect of image processing.

In a first aspect, a method for image processing is provided, including: acquiring an image to be processed, wherein the image to be processed comprises a plurality of sub-images, and an overlapping area exists between adjacent sub-images; processing each sub-image to obtain a sub-tensor of each sub-image; for each overlapped area, processing the overlapped area based on the sub tensors of at least two sub images corresponding to the overlapped area so as to update and obtain an overlapped area fusion tensor of the overlapped area; updating an output tensor of the image to be processed based on the fold fusion tensor.

Optionally, the method further comprises: and carrying out normalization processing on the output tensor to obtain a processing result of the image to be processed.

Optionally, for each overlap region, processing the overlap region based on the sub-tensors of the at least two sub-images corresponding to the overlap region to update the overlap region fusion tensor of the overlap region includes: for a current overlap area needing to be processed currently, determining a first overlap area tensor and a second overlap area tensor which respectively correspond to the current overlap area in at least two sub-images forming the current overlap area; and weighting the first folding tensor and the second folding tensor to obtain the folding fusion tensor.

Optionally, the weighting the first stack tensor and the second stack tensor to obtain the stack fusion tensor includes: determining a first product of the first stack tensor and a set first weight matrix; determining a second product of the second stack tensor and a set second weight matrix; taking a sum of the first product and the second product as the fold fusion tensor.

Optionally, the sum of the weight coefficient values of the first weight matrix and the second weight matrix corresponding to the same position point in the overlapping region is 1.

Optionally, in any weight matrix, the weight coefficient of the weight matrix shows a decay trend; the attenuation trend is in a negative correlation relation with the distance between the position point corresponding to the weight coefficient and the center line of the corresponding sub-image.

Optionally said updating an output tensor of the image to be processed based on the fold fusion tensor comprises: updating tensors corresponding to the overlap region fusion tensor in the sub tensors of the sub images forming the corresponding overlapping region based on the overlap region fusion tensor to obtain updated sub tensors of the sub images; and obtaining the output tensor of the image to be processed based on all the updated sub tensors.

Optionally, the normalizing the output tensor to obtain the processing result of the image to be processed includes: and normalizing the output tensor through a sigmoid function and/or a softmax function to obtain a processing result of the image to be processed.

In a second aspect, an apparatus for image processing is provided, including: the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring an image to be processed, the image to be processed comprises a plurality of sub-images, and an overlapping area exists between adjacent sub-images; the first processing unit is used for processing each sub-image to obtain a sub-tensor of each sub-image; the second processing unit is used for processing each overlapped area based on the sub tensors of at least two sub images corresponding to the overlapped area so as to update and obtain the overlapped area fusion tensor of the overlapped area; and the updating unit is used for updating the output tensor of the image to be processed based on the overlapped region fusion tensor.

In a third aspect, an intelligent agricultural system is provided, which includes the image processing device of the second aspect.

In a fourth aspect, an electronic device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method according to any of the embodiments of the first aspect when executing the program.

In a fifth aspect, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a computer processor, performs the steps of the method according to any of the embodiments of the first aspect.

In the embodiment of the application, the overlap region is processed based on the sub-tensors of the at least two sub-images corresponding to the overlap region, the overlap region fusion tensor of the overlap region is obtained, the output tensor of the image to be processed is updated based on the overlap region fusion tensor, so that transition of the overlap region in the image to be processed is smooth, splicing traces and a grid effect can be effectively reduced, and the image processing effect can be improved.

Drawings

Fig. 1 is a diagram illustrating an application scenario applicable to the embodiment of the present application.

FIG. 2 is a schematic block diagram of a method of image processing in one embodiment of the present application.

Fig. 3 is a schematic block diagram of an apparatus for image processing according to an embodiment of the present application.

Fig. 4 is a schematic block diagram of an apparatus for image processing according to another embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

In the description of the present application, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. are used for indicating the orientation or positional relationship based on the orientation or positional relationship shown in the drawings or the orientation or positional relationship which the present invention product is usually put into use, it is only for convenience of describing the present application and simplifying the description, but it is not intended to indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation and be operated, and thus, should not be construed as limiting the present application.

Furthermore, the appearances of the terms "first," "second," and the like, if any, are only used to distinguish one description from another and are not to be construed as indicating or implying relative importance. It should be noted that the features of the embodiments of the present application may be combined with each other without conflict.

The method in the embodiment of the present application may be applied to various aerial photographs and other scenes requiring image processing, and is not limited in the embodiment of the present application.

Fig. 1 is a diagram of an application scenario applicable to the embodiment of the present application.

It should be noted that the application scenario shown in fig. 1 is only an example and is not limited, and more or fewer devices or apparatuses may be included in the application scenario shown in fig. 1, which is not limited in the embodiment of the present application.

The application scenario 100 in fig. 1 may include an image acquisition device 110 and an image processing device 120.

The image acquiring device 110 may be a shooting device for shooting an image to be processed, such as a camera; alternatively, the image capturing apparatus 110 may also be a device or an apparatus including a shooting apparatus, such as an unmanned aerial vehicle (the unmanned aerial vehicle can perform aerial photography), a terminal device (e.g., a mobile phone, a remote controller of the unmanned aerial vehicle, a smart terminal), or other devices capable of taking pictures; alternatively, the image acquisition apparatus 110 may also store a device or apparatus to be processed with the image, such as a server (or a cloud server).

The image processing apparatus 120 may be a computer, a terminal device, a drone, a server (e.g., a cloud server), or other apparatus or device capable of image processing.

Optionally, the image processing apparatus 120 may include one or more neural network models (e.g., an image segmentation model, an object detection model, and other image processing models), and perform image processing on the image to be processed acquired by the image acquisition apparatus 110 to obtain a processing result of the image to be processed.

For example, the image acquiring device 110 may be an unmanned aerial vehicle, and the image processing device 120 may be a terminal device. For example, the drone may take a ground image (i.e., an image to be processed) and transmit the ground image to the terminal device; correspondingly, the terminal equipment can receive the ground image shot by the unmanned aerial vehicle and perform image processing on the ground image. Alternatively, the image capturing device 110 and the image processing device 120 may both be unmanned aerial vehicles. For example, the drone may take a ground image and perform image processing on the ground image. Alternatively, the image capturing device 110 may be a drone, and the image processing device 120 may be a server (or cloud server). For example, the drone may take a ground image (i.e., an image to be processed) and transmit the ground image to a server, which performs image processing on the ground image.

For another example, the image capturing device 110 may be a terminal device, and the image processing device 120 may be an unmanned aerial vehicle. For example, the terminal device may capture a ground image (i.e., an image to be processed) and transmit the ground image to the drone, which performs image processing on the ground image. Alternatively, the image acquisition apparatus 110 and the image processing apparatus 120 may be both terminal devices. For example, the terminal device may capture a ground image and perform image processing on the ground image. Alternatively, the image acquiring device 110 may be a terminal device, and the image processing device 120 may be a server. For example, the terminal device may capture a ground image (i.e., an image to be processed) and transmit the ground image to a server, which performs image processing on the ground image.

For another example, the image capturing device 110 may be a server, and the image processing device 120 may be a drone. For example, a to-be-processed image (e.g., a ground image captured in advance) is stored in the server, and the server transmits the to-be-processed image to the drone, and the drone performs image processing on the to-be-processed image. Alternatively, the image acquisition device 110 and the image processing device 120 may both be servers. For example, the server stores an image to be processed, and performs image processing on the image to be processed. Alternatively, the image acquisition apparatus 110 may be a server, and the image processing apparatus 120 may be a terminal device. For example, a to-be-processed image is stored in a server, the server transmits the to-be-processed image to a terminal device, and the terminal device performs image processing on the to-be-processed image.

In general, the aerial ground image has a high resolution, and the ground image cannot be directly input to a neural network for image processing. At present, a common solution is to perform sliding window processing on an image to be processed. The first mode is to perform non-overlapping sliding window processing on the image to be processed and splice the image, and no overlapping area (namely overlapping area) exists between two (or two) adjacent sliding windows in the mode, so that the splicing trace of the processing result obtained by the mode is obvious; the second method is to perform overlapping sliding window processing and splicing on the image to be processed, wherein an overlapping area exists between two (or two) adjacent sliding windows in the method, the splicing trace of the processing result obtained by the method is obvious, and the image presents a checkered effect (namely a 'checkered' edge exists around an area corresponding to each sliding window in the processing result) due to the obvious splicing trace. In the second method, if the checkered effect is to be reduced, the area of the overlap between the sliding windows needs to be increased, which, however, reduces the ratio of the effective area in the processing result, and thus reduces the efficiency of image processing.

Based on the above problems, the embodiments of the present application provide an image processing method and apparatus, which can effectively remove the stitching trace and the checkered effect under the condition of a low overlapping ratio (i.e., the area of the overlapping area is small), so as to improve the image processing effect. The method may be performed by a device with an image capturing device, such as an unmanned aerial vehicle or an image processing module, or may be performed by a device without an image capturing device, such as a cloud server or a remote controller, but is not limited thereto.

The method in the embodiment of the present application is described in detail below with reference to fig. 2.

FIG. 2 is a schematic block diagram of a method 200 of image processing according to one embodiment of the present application. The method 200 may be performed by the image processing apparatus 120 of fig. 1.

It should be understood that fig. 2 shows the steps or operations of the method 200, but these steps or operations are only examples, and other operations or variations of the individual operations of the method 200 in fig. 2 may be performed by embodiments of the present application, or not all of the steps need to be performed, or the steps may be performed in other orders.

And S210, acquiring an image to be processed.

The image to be processed may include a plurality of sub-images, and an overlap region exists between adjacent sub-images. Alternatively, the image to be processed may be an image captured by the image capturing device 110 in fig. 1 or a stored image. For example, the image to be processed may be an aerial high-resolution image. Optionally, the sub-image may be obtained by cutting according to a set size based on the image to be processed, where the set size may be preset according to an actual requirement, and this is not limited in this application.

And S220, processing each sub-image to obtain the sub-tensor of each sub-image.

Optionally, the sub-images may be processed using a neural network model to obtain sub-tensors of the sub-images. Wherein the sub-tensor can be logits tensor.

In some embodiments, a neural network model may be used, and a sliding window strategy is adopted to process a plurality of sub-images in the image to be processed, so as to obtain the plurality of sub-tensors.

In some embodiments, a sliding window strategy may be used to perform dilation prediction on a plurality of sub-images in the image to be processed, so as to obtain the plurality of sub-tensors. For example, in image processing, only the central region of the prediction result corresponding to each sub-image is retained, and the image edge with incorrect prediction is discarded, so as to obtain the prediction result corresponding to the sub-image (i.e., the sub-tensor corresponding to the sub-image).

For example, if the step size of the sliding window is 512 and the size of the sliding window is 1024 × 1024, for an image to be processed whose resolution does not satisfy the integral multiple of the size of the sliding window, the right boundary and the lower boundary of the image to be processed may be filled first, so that the resolution of the image to be processed is the integral multiple of the size of the sliding window; the resolution of the first sub-image in the image to be processed is 512x512, and the outer frame with the size of sliding window step size can be filled 1/2 for the first sub-image (the resolution after the filling of the first sub-image is 1024x 1024). Only 512x512 results at the center of the sliding window may be retained for each prediction.

Optionally, the sliding window policy may be an overlapping sliding window policy, that is, during the image processing of the image to be processed, an overlapping region exists between two (or two) adjacent sliding windows.

And S230, for each overlapped area, processing the overlapped area based on the sub tensors of at least two sub images corresponding to the overlapped area so as to update and obtain the overlapped area fusion tensor of the overlapped area.

Optionally, for a current overlap area which needs to be processed currently, a first overlap tensor and a second overlap tensor which respectively correspond to the current overlap area in at least two sub-images forming the current overlap area can be determined; and processing the first fold region tensor and the second fold region tensor to obtain the fold region fusion tensor.

The processing of the first stack tensor and the second stack tensor (to obtain the stack fusion tensor) may be weighting, averaging, smoothing, or the like, of the first stack tensor and the second stack tensor. Next, an example of weighting the first stack tensor and the second stack tensor will be described.

For example, the image to be processed may include a first sub-image and a second sub-image, a first overlap region exists between the first sub-image and the second sub-image, when the first overlap region is a current overlap region that needs to be processed currently, a first overlap region tensor of the first sub-image in the first overlap region and a second overlap region tensor of the second sub-image in the second overlap region may be determined, and the first overlap region tensor and the second overlap region tensor are subjected to weighted addition to obtain a overlap region fusion tensor of the first overlap region. Further, the tensor (or sub-tensor) corresponding to the first overlapping region can be updated by using the overlapping region fusion tensor.

Further, a first product of the first stack tensor and the set first weight matrix may be determined, a second product of the second stack tensor and the set second weight matrix may be determined, and a sum of the first product and the second product may be used as the stack fusion tensor.

For example, the first stack tensor may correspond to a first weight matrix and the second stack tensor may correspond to a second weight matrix. In determining the overlap fusion tensor for the first overlap region, a first product of the first overlap tensor and a first weight matrix and a second product of the second overlap tensor and a second weight matrix may be determined, and the first product and the second product may be added to obtain the overlap fusion tensor for the first overlap region.

Alternatively, the sum of the weight coefficient values at the same position point in the overlapping area in the first weight matrix and the second weight matrix may be 1.

For example, a first overlap region exists between the first sub-image and the second sub-image, a first overlap region tensor of the first sub-image in the first overlap region may be denoted as a matrix a, a second overlap region tensor of the second sub-image in the first overlap region may be denoted as a matrix B, a first weight matrix corresponding to the matrix a is W, a second weight matrix corresponding to the matrix a is W ', and a final numerical matrix of the first overlap region (i.e., an output tensor of the overlap region) is denoted as C, where C is W + W'. B. Wherein a and B are two elements located at the same position in the matrix a and the matrix B, respectively, w is a weight corresponding to the element a in the first weight matrix, w ' is a weight corresponding to the element B in the second weight matrix, and c is a tensor corresponding to the final value matrix element a and the element B of the overlap region, that is, c ═ wa + w ' B, where w + w ' is 1.

Optionally, in any weight matrix, the weight coefficient of the weight matrix may show a decay trend. Alternatively, the attenuation trend may be inversely related to the distance of the position point corresponding to the weight coefficient from the center line of the corresponding sub-image.

For example, the value of the weight corresponding to the element a is smaller as the element a is farther from the geometric center of the matrix a, and the value of the weight corresponding to the element a is larger as the element a is closer to the geometric center of the matrix a. Wherein the attenuation of the value of the weight corresponding to element a with the distance (the distance from the geometric center of matrix a) satisfies, but is not limited to, a linear function.

It should be noted that, in the above embodiment, only one overlapping area of the first sub-image is described as an example, and an overlapping area may exist between the sub-image in the image to be processed and the plurality of sub-images at the same time. For example, if the first sub-image is the sub-image at the upper left corner of the image to be processed, an overlap area exists between the first sub-image and two sub-images at the right and lower sides of the first sub-image; if the first sub-image is a middle sub-image of the image to be processed, the first sub-image may have an overlap area with the four sub-images on the left, right, top, and bottom sides of the first sub-image. That is, the second sub-image may include at least one of a sub-image having an upper adjacent overlap region with the first sub-image, a sub-image having a lower adjacent overlap region with the first sub-image, a sub-image having a left adjacent overlap region with the first sub-image, and a sub-image having a right adjacent overlap region with the first sub-image.

Under the condition that the first sub-image and a plurality of sub-images on the left side, the right side, the upper side and/or the lower side of the first sub-image have overlapped areas, the sub-tensor domains corresponding to the overlapped areas can be sequentially processed, and overlapped area fusion tensors of the overlapped areas are respectively obtained. Of course, the image to be processed may also include an overlap region, and other overlap regions may also be processed by using the above method. For example, the image to be processed may further include a second overlapping area. The second overlap region may be an overlap region between the second sub-image and the third sub-image, and the overlap fusion tensor of the second overlap region may be determined by using the method in the above embodiment.

Subsequently, other overlapping regions in the image to be processed may be processed until all the overlapping regions in the image to be processed are traversed.

S240, updating the output tensor of the image to be processed based on the overlapped region fusion tensor.

Optionally, a tensor corresponding to the overlap region fusion tensor in the sub-tensors of the sub-images forming the corresponding overlapping region may be updated based on the overlap region fusion tensor to obtain an updated sub-tensor of the sub-image, and an output tensor of the image to be processed may be obtained based on all the updated sub-tensors.

For example, all updated sub-tensors may be stitched to obtain an output tensor of the image to be processed. Optionally, the output tensor of the image to be processed is a full frame logits tensor.

The following example illustrates the processing of an overlap region:

suppose that the image to be processed includes 9 sub-images, and the 9 sub-images are arranged in a squared form. For convenience of description, the 9 sub-images are sequentially marked from left to right as a first sub-image and a ninth sub-image … … respectively; the first sub-image is the first sub-image, and the sub-images adjacent to the first sub-image are the second sub-image, the fourth sub-image and the fifth sub-image respectively; adjacent to the second sub-image are the first sub-image, the third sub-image, the fourth sub-image, the fifth sub-image and the sixth sub-image, respectively, and so on, the sub-images adjacent to the sub-images can be clearly known. Based on this, it is understood that other sub-images adjacent to a certain sub-image may refer to sub-images having an overlapping area with the sub-image.

As can be seen from the above, there are 3 adjacent sub-images for the sub-image located at the corner, and 8 adjacent sub-images for the sub-image located at the middle position (e.g., the fifth sub-image). Based on this, in the process of processing the overlap regions, the overlap regions may be sequentially processed according to the arrangement order of the sub-images, for example, for the first sub-image, the overlap region thereof includes 3 overlap regions, which are respectively the first overlap region formed by overlapping with only the second sub-image, the second overlap region formed by overlapping with only the fourth sub-image, and the third overlap region formed by overlapping with both the second sub-region and the fourth sub-region. Therefore, the first overlap region may be processed, for example, the first overlap region tensor corresponding to the first overlap region in the first sub-image and the first overlap region tensor corresponding to the first overlap region in the second sub-image are weighted and then added to obtain the first overlap region fusion tensor, and the first overlap region fusion tensor is updated to the data matrix of the first overlap region in the first sub-image and the second sub-image. Similarly, the second overlap region may be processed to obtain a second overlap region fusion tensor of the second overlap region, and the second overlap region fusion tensor is updated to the data matrix of the second overlap region in the first sub-image and the second sub-image. For the third overlapping area, the processing principle is the same, but since the third overlapping area is formed by overlapping three images, the tensor of the third overlapping area in the three sub-images can be used for weighting and then adding, so as to obtain the tensor of the third overlapping area in the third overlapping area, and the tensor of the third overlapping area is updated to the data matrix of the third overlapping area in the three sub-images. Thereby completing the processing of the overlap area in the first sub-image.

Then, the overlap regions in the second sub-image may be processed according to the same principle, but it should be noted that, since the same overlap regions as those in the first sub-image, i.e., the first overlap region and the third overlap region, are included in the second sub-image, and these overlap regions are already processed in the first sub-image, the already processed overlap regions may not be reprocessed to reduce the image data processing amount and further improve the processing efficiency. As for other unprocessed overlapped regions in the second sub-image, processing may be performed based on the above processing manner for the overlapped regions, which is not described herein again.

And so on until the processing of the overlap areas of all sub-images is completed. After the processing is finished, the overlapped region fusion tensor corresponding to the overlapped region obtained by the processing can be updated to the corresponding data matrix in real time in the processing process, so that the output tensor corresponding to the whole image to be processed can be obtained when the processing is finished, at the moment, when the target image is displayed based on the output tensor, the grid effect does not exist in the target image, and the influence of splicing trace on the display effect of the whole image can be well reduced.

In an embodiment, the method 200 may further include step S250, which is specifically as follows:

and S250, carrying out normalization processing on the output tensor to obtain a processing result of the image to be processed.

Wherein the normalization process can be realized by a sigmoid function and/or a softmax function.

For example, taking the semantic segmentation task as an example, the normalization process of a single class may use a sigmoid function, and the normalization process of two or more classes may use a softmax function.

In the prior art, when an image to be processed is processed, normalization processing is directly performed on an output tensor (i.e., logits tensor) obtained by each sliding window to obtain a prediction matrix (i.e., a prediction result corresponding to a sub-image), and the prediction matrices are spliced to obtain a prediction result of the image to be processed, i.e., a spliced object in the image processing process is the prediction result corresponding to each sliding window. Usually, the prediction matrix corresponding to each sliding window is directly subjected to One-Hot (One-Hot) processing, that is, the prediction result corresponding to each sliding window is already the One-Hot matrix. That is, the object to be stitched in the image processing process in the prior art is often already the unique heat matrix corresponding to each sliding window.

Different from the method in the prior art, in the embodiment of the application, the overlap region fusion tensor of the overlap region is obtained by processing the overlap region between the plurality of sub-images in the image to be processed, and the pixels corresponding to each overlap region in the image to be processed are adjusted based on the overlap region fusion tensor, so that the transition of the overlap region in the plurality of sub-images is smooth, the grid effect caused by the obvious splicing trace in the image can be effectively reduced, and the image processing effect can be improved. In addition, according to the present invention, after obtaining the overall output tensor of the image to be processed (for example, the full frame logits tensor of the image to be processed), the overall output tensor of the image to be processed is normalized, so that the stitching trace and the checkered effect can be further reduced, and the image processing effect can be improved.

Fig. 3 is a schematic block diagram of an apparatus 300 for image processing according to an embodiment of the present application. It should be understood that the apparatus 300 shown in fig. 3 is only an example, and the apparatus 300 of the present embodiment may further include other modules or units.

It should be understood that the apparatus 300 is capable of performing the various steps in the method of fig. 2 and, to avoid repetition, will not be described in detail herein.

In one possible implementation manner of the present application, the apparatus 300 includes:

an obtaining unit 310, configured to obtain an image to be processed, where the image to be processed includes a plurality of sub-images, and an overlap area exists between adjacent sub-images;

the first processing unit 320 is configured to process each sub-image to obtain a sub-tensor of each sub-image;

the second processing unit 330 is configured to, for each overlap area, process the overlap area based on the sub-tensors of the at least two sub-images corresponding to the overlap area, so as to update a fold fusion tensor of the overlap area;

an updating unit 340, configured to update an output tensor of the image to be processed based on the overlap fusion tensor.

And obtaining a processing result of the image to be processed. Optionally, the apparatus further comprises a normalization unit 350 for: the output tensor is normalized by a normalization process,

optionally, the second processing unit 330 is specifically configured to: for a current overlap area needing to be processed currently, determining a first overlap area tensor and a second overlap area tensor which respectively correspond to the current overlap area in at least two sub-images forming the current overlap area; and weighting the first folding tensor and the second folding tensor to obtain the folding fusion tensor.

Optionally, the second processing unit 330 is specifically configured to: determining a first product of the first stack tensor and a set first weight matrix; determining a second product of the second stack tensor and a set second weight matrix; taking a sum of the first product and the second product as the fold fusion tensor.

Optionally, in any weight matrix, the weight coefficient of the weight matrix shows a decay trend; the attenuation trend is in a negative correlation with the distance between the position point corresponding to the weight coefficient and the central line of the corresponding sub-image.

Optionally, the updating unit 340 is specifically configured to: updating a tensor corresponding to the overlap region fusion tensor in the sub-tensors of the sub-images forming the corresponding overlapped area based on the overlap region fusion tensor to obtain an updated sub-tensor of the sub-image; and obtaining the output tensor of the image to be processed based on all the updated sub tensors.

Optionally, the normalization unit 350 is specifically configured to: and normalizing the output tensor through a sigmoid function and/or a softmax function to obtain a processing result of the image to be processed.

It should be understood that the apparatus 300 for image processing herein is embodied in the form of functional modules. The term "module" herein may be implemented in software and/or hardware, and is not particularly limited thereto. For example, a "module" may be a software program, a hardware circuit, or a combination of both that implements the functionality described above. The hardware circuitry may include an Application Specific Integrated Circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group of processors) and memory that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable components that support the described functionality.

As an example, the apparatus 300 for image processing provided in the embodiment of the present application may be a processor or a chip, and is configured to perform the method described in the embodiment of the present application.

Fig. 4 is a schematic block diagram of an apparatus 400 for image processing according to an embodiment of the present application. The apparatus 400 shown in fig. 4 comprises a memory 401, a processor 402, a communication interface 403 and a bus 404. The memory 401, the processor 402 and the communication interface 403 are connected to each other by a bus 404.

The memory 401 may be a Read Only Memory (ROM), a static memory device, a dynamic memory device, or a Random Access Memory (RAM). The memory 401 may store a program, and when the program stored in the memory 401 is executed by the processor 402, the processor 402 is configured to perform the steps of the method of image processing according to the embodiment of the present application, for example, the steps of the embodiments shown in fig. 3 and 4 may be performed.

The processor 402 may be a general-purpose Central Processing Unit (CPU), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the image processing method according to the embodiment of the present application.

The processor 402 may also be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the image processing method of the embodiment of the present application may be implemented by integrated logic circuits of hardware in the processor 402 or instructions in the form of software.

The processor 402 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, or discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory 401, and a processor 402 reads information in the memory 401, and performs, in combination with hardware thereof, functions required to be performed by units included in the apparatus for image processing in the embodiment of the present application, or performs the method for image processing in the embodiment of the method of the present application, for example, the steps/functions in the embodiment shown in fig. 2 may be performed.

The communication interface 403 may use transceiver means, such as, but not limited to, a transceiver, to enable communication between the apparatus 400 and other devices or communication networks.

Bus 404 may include a path that transfers information between various components of apparatus 400 (e.g., memory 401, processor 402, communication interface 403).

It should be understood that the apparatus 400 shown in the embodiment of the present application may be a processor or a chip for performing the method of image processing described in the embodiment of the present application.

It should be understood that the processor in the embodiments of the present application may be a Central Processing Unit (CPU), and the processor may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

It should be understood that in the embodiment of the present application, "B corresponding to a" means that B is associated with a, from which B can be determined. It should also be understood that determining B from a does not mean determining B from a alone, but may be determined from a and/or other information.

It should be understood that the term "and/or" herein is merely one type of association relationship that describes an associated object, meaning that three relationships may exist, e.g., a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be read by a computer or a data storage device including one or more available media integrated servers, data centers, and the like. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Versatile Disk (DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), among others.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method of image processing, comprising:

acquiring an image to be processed, wherein the image to be processed comprises a plurality of sub-images, and an overlapping area exists between adjacent sub-images;

processing each sub-image to obtain a sub-tensor of each sub-image;

for each overlapped area, processing the overlapped area based on the sub tensors of at least two sub images corresponding to the overlapped area so as to update and obtain an overlapped area fusion tensor of the overlapped area;

updating an output tensor of the image to be processed based on the fold fusion tensor.

2. The method of claim 1, further comprising:

and carrying out normalization processing on the output tensor to obtain a processing result of the image to be processed.

3. The method of claim 2, wherein for each overlap region, processing the overlap region based on the sub-tensors of the at least two sub-images corresponding to the overlap region to update a fold fusion tensor of the overlap region comprises:

for a current overlap area needing to be processed currently, determining a first overlap area tensor and a second overlap area tensor which respectively correspond to the current overlap area in at least two sub-images forming the current overlap area;

and weighting the first folding tensor and the second folding tensor to obtain the folding fusion tensor.

4. The method of claim 3, wherein weighting the first stack tensor and the second stack tensor to obtain the stack fusion tensor comprises:

determining a first product of the first stack tensor and a set first weight matrix;

determining a second product of the second stack tensor and a set second weight matrix;

taking a sum of the first product and the second product as the fold fusion tensor.

5. The method according to claim 4, wherein the sum of the weight coefficient values in the first weight matrix and the second weight matrix corresponding to the same position point in the overlap region is 1.

6. The method according to claim 4 or 5, wherein in any weight matrix, the weight coefficient of the weight matrix shows a decay trend; the attenuation trend is in a negative correlation with the distance between the position point corresponding to the weight coefficient and the central line of the corresponding sub-image.

7. The method of any of claims 1-6, wherein updating the output tensor for the image to be processed based on the fold fusion tensor comprises:

updating tensors corresponding to the overlap region fusion tensor in the sub tensors of the sub images forming the corresponding overlapping region based on the overlap region fusion tensor to obtain updated sub tensors of the sub images;

and obtaining the output tensor of the image to be processed based on all the updated sub tensors.

8. The method according to any one of claims 2 to 7, wherein the normalizing the output tensor to obtain the processing result of the image to be processed comprises:

and normalizing the output tensor through a sigmoid function and/or a softmax function to obtain a processing result of the image to be processed.

9. An apparatus for image processing, comprising:

the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring an image to be processed, the image to be processed comprises a plurality of sub-images, and an overlapping area exists between adjacent sub-images;

the first processing unit is used for processing each sub-image to obtain a sub-tensor of each sub-image;

the second processing unit is used for processing each overlapped area based on the sub tensors of at least two sub images corresponding to the overlapped area so as to update and obtain the overlapped area fusion tensor of the overlapped area;

and the updating unit is used for updating the output tensor of the image to be processed based on the overlapped region fusion tensor.

10. An intelligent agricultural system, comprising the image processing apparatus of claim 9.

11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 8 when executing the program.

12. A computer storage medium, characterized in that the storage medium has stored thereon a computer program which, when executed by a computer processor, implements the method according to any one of claims 1 to 8.