CN112967211A

CN112967211A - Image processing method and device, computer equipment and storage medium

Info

Publication number: CN112967211A
Application number: CN202110132579.7A
Authority: CN
Inventors: 周军; 周亮; 常亮; 王文强; 吴飞; 徐宁仪
Original assignee: University of Electronic Science and Technology of China; Chengdu Sensetime Technology Co Ltd
Current assignee: University of Electronic Science and Technology of China; Chengdu Sensetime Technology Co Ltd
Priority date: 2021-01-31
Filing date: 2021-01-31
Publication date: 2021-06-15
Anticipated expiration: 2041-01-31
Also published as: WO2022160704A1; CN112967211B

Abstract

The present disclosure provides an image processing method, apparatus, computer device, and storage medium, wherein the method comprises: carrying out pixel combination processing on a plurality of pieces of image data to be processed to obtain a target image; processing the target image by utilizing a multiplier-adder array; and determining the processing result of each piece of image data to be processed based on the processing results obtained by the multipliers and adders respectively corresponding to the plurality of pieces of image data to be processed in the multiplier and adder array. The method and the device can simultaneously process a plurality of pieces of image data to be processed by using the same multiplier-adder array, and can determine the processing result of each piece of image data to be processed based on the processing results of the plurality of pieces of image data to be processed obtained by the multiplier-adder respectively corresponding to the multiplier-adder in the multiplier-adder array, thereby improving the utilization rate of the multiplier-adder array, reducing the waste of computing resources and improving the processing efficiency of the image data to be processed.

Description

Image processing method and device, computer equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to an image processing method and apparatus, a computer device, and a storage medium.

Background

At present, a convolutional neural network mainly depends on a multiplier-adder array to perform convolution processing, the multiplier-adder array stores data to be processed in a data matrix in a corresponding register array, and the data to be processed moves in the register array in different data processing periods, but the current data processing method has the problems of low utilization rate of the multiplier-adder array and waste of computing resources.

Disclosure of Invention

The embodiment of the disclosure at least provides an image processing method, an image processing device, computer equipment and a storage medium.

In a first aspect, an embodiment of the present disclosure provides an image processing method, including:

carrying out pixel combination processing on a plurality of pieces of image data to be processed to obtain a target image; processing the target image by utilizing a multiplier-adder array; and determining the processing result of each piece of image data to be processed based on the processing results obtained by the multipliers and adders respectively corresponding to the plurality of pieces of image data to be processed in the multiplier and adder array.

Therefore, based on the fact that a plurality of pieces of image data to be processed are combined into one target image, the size of the obtained target image is larger than that of each piece of image data to be processed in the plurality of pieces of image data to be processed, the multiplier-adder array is utilized to process the target image in the process of actually processing the plurality of pieces of image data to be processed simultaneously, more multipliers and adders in the multiplier-adder array can be utilized, the utilization rate of the multiplier-adder array is improved, the waste of computing resources is reduced, and the processing efficiency of the image data to be processed is improved.

In a possible implementation manner, in the same row of the target image, the number of pixels between two adjacent pixels of the same image data to be processed is the same and is not zero, and in the same column of the target image, the number of pixels between two adjacent pixels of the same image data to be processed is the same and is not zero.

Therefore, the arrangement rule of the pixel points of the image data to be processed based on the target image can ensure that when the multiplier-adder array processes the target image, a correct processing result corresponding to each image data to be processed is obtained, and the processing efficiency of the image data to be processed is improved.

In one possible embodiment, the method further comprises: determining the number of the image data to be processed based on the size information of the multiplier-adder array and the size information of the image data to be processed.

Therefore, the number of the image data to be processed which can be processed by the multiplier-adder array can be determined based on the size information of the multiplier-adder array and the size information of each image data to be processed, the utilization efficiency of the multiplier-adder array is improved, and the accuracy of the processing result of the multiplier-adder array on the image data to be processed is ensured.

In one possible embodiment, the size information includes a number of rows and a number of columns; the determining the number of the image data to be processed based on the size information of the multiplier-adder array and the size information of the image data to be processed includes: determining a first value based on the number of rows of the multiplier-adder array and the number of rows of the image data to be processed, and determining a second value based on the number of columns of the multiplier-adder array and the number of columns of the image data to be processed; and determining the quantity of the image data to be processed based on the first value and the second value.

In a possible implementation manner, the performing pixel combination processing on multiple pieces of image data to be processed to obtain a target image includes: sequentially taking each image data to be processed in the plurality of image data to be processed as a current image, and determining a target position of a first pixel point of the current image in the target image; and determining the target positions of other pixel points in the current image except the first pixel point in the target image based on the target position of the first pixel point in the target image to obtain the target image.

Therefore, after the target position of the first pixel point of each piece of image data to be processed in the target image is determined, the target positions of other pixel points except the first pixel point in each piece of image data to be processed in the target image can be determined based on the target position of the first pixel point of each piece of image data to be processed in the target image, and the efficiency of obtaining the target image is improved.

In a possible implementation manner, the determining, based on the target position of the first pixel point in the target image, the target positions of other pixel points in the current image, except for the first pixel point, in the target image includes: determining a first position relation of each pixel point in each row of the current image except for a first pixel point in the row and an adjacent previous pixel point of the pixel point in the target image based on a target position of the first pixel point of the current image in the target image, the number of columns of the multiplier-adder array and the number of columns of the current image; determining a second position relation of each pixel point in each column of the current image except for the first pixel point in the column and an adjacent previous pixel point of the pixel point in the target image based on the target position of the first pixel point of the current image in the target image, the row number of the multiplier-adder array and the row number of the current image; and determining the target positions of other pixel points in the current image except the first pixel point in the target image based on the first position relation and/or the second position relation.

In a possible embodiment, the determining a target position of a first pixel point of the current image in the target image includes: determining a target pixel matrix based on the number of rows and columns of the multiplier-adder array and the number of rows and columns of the image data to be processed; determining the target position of the first pixel point of each image data to be processed in the target image according to the matrix element value of the target pixel matrix; and the target position of the first pixel point of the current image in the target image is any one of the determined target positions.

In one possible embodiment, the processing the target image by using the multiplier-adder array includes: storing the target image into a register array based on the corresponding relation between the multiplier-adder array and the register array; wherein the adjacent relation of each pixel point of the target image stored in the register array is unchanged; and in each data processing period of a plurality of data processing periods, reading pixel information corresponding to each data processing period from the register array by using the multiplier-adder array, and processing the read pixel information to obtain a data processing result corresponding to each processing period.

In this way, the multiplier-adder array ensures the validity of the processing result for each piece of image data to be processed by reading the corresponding pixel information at different data processing cycles.

In one possible implementation, the reading, by the multiplier-adder array, pixel information corresponding to each data processing cycle from the register array in each data processing cycle of the plurality of data processing cycles, and processing the read pixel information to obtain a data processing result corresponding to each data processing cycle includes: in the first data processing period of processing the target image, controlling each multiplier-adder with the target image pixel information stored in the corresponding register in the multiplier-adder array, and reading the pixel value of the pixel point corresponding to the first data processing period of each multiplier-adder from the register connected with each multiplier-adder as a first operand; determining the weight value corresponding to each multiplier-adder in the first data processing period as a second operand; determining the product of the first operand and the second operand of each multiplier-adder in the first data processing cycle respectively; for each non-first data processing period of processing the target image, controlling each pixel of the target image to move by a preset step length in the register array according to a preset data movement mode corresponding to the data processing period; controlling each multiplier-adder, and reading the pixel value of the pixel point corresponding to each multiplier-adder in the data processing period from a register connected with each multiplier-adder as a first operand; determining the weight value corresponding to each multiplier-adder in the data processing period as a second operand; the product of the first operand and the second operand of each multiplier-adder in the data processing period is respectively determined.

Therefore, the pixel information of the target characteristic diagram is orderly displaced in the register array along with the transformation of the data processing period based on the preset step length and the preset data moving mode, so that the corresponding multiplier-adder in the multiplier-adder array can be ensured to obtain effective data, and the effectiveness of the processing result of the target image is ensured.

In a possible implementation manner, the determining a processing result of each image data to be processed based on processing results obtained by multipliers and adders respectively corresponding to the image data to be processed in the multiplier and adder array includes: for each effective multiplier-adder in each multiplier-adder, adding products obtained by the effective multiplier-adder in each data processing period to obtain a sum; determining effective multiplier-adders corresponding to each piece of image data to be processed from each effective multiplier-adder; and determining the processing result of each image data to be processed based on the sum value respectively obtained by the effective multiplier-adder corresponding to each image data to be processed.

Therefore, the processing result of each image data to be processed can be obtained based on the sum value respectively obtained by the effective multiplier-adder respectively corresponding to each image data to be processed, and the processing efficiency of the image data to be processed is improved.

In one possible embodiment, the effective multiplier-adder corresponding to each piece of image data to be processed includes: the register connected with each data processing cycle is provided with a multiplier-adder for inputting the effective pixel value of the piece of image data to be processed.

In one possible implementation, the image processing method further includes: and determining the preset step length based on the distance between two adjacent pixel points belonging to the same image data to be processed in the target image.

Therefore, when the target image contains different amounts of to-be-processed image data, the validity of the processing result of the multiplier-adder array on each to-be-processed image data contained in the target image can be ensured based on the setting of the preset step length, and the flexibility of processing the target image by the multiplier-adder array is improved.

In a possible embodiment, the acquiring the plurality of pieces of image data to be processed includes: acquiring a plurality of original images; taking a first original image in the plurality of original images as the image data to be processed; and/or performing data filling processing on a second original image in the plurality of original images to obtain to-be-processed image data corresponding to the second original image.

Therefore, the data of the image to be processed with the applicable size is obtained based on the data filling of the second original image, and the method is more flexible.

In a second aspect, an embodiment of the present disclosure further provides an image processing apparatus, including: a controller; the controller is configured to:

acquiring a plurality of pieces of image data to be processed; performing fusion processing on the multiple pieces of image data to be processed to obtain a target image; and processing the target image by utilizing a multiplier-adder array to obtain a processing result of each piece of to-be-processed image data in the plurality of pieces of to-be-processed image data.

In one possible embodiment, the controller is further configured to: determining the number of the image data to be processed based on the size information of the multiplier-adder array and the size information of the image data to be processed.

In one possible embodiment, the size information includes a number of rows and a number of columns; when the number of the image data to be processed is determined based on the size information of the multiplier-adder array and the size information of the image data to be processed, the controller is specifically configured to determine a first value based on the number of rows of the multiplier-adder array and the number of rows of the image data to be processed, and determine a second value based on the number of columns of the multiplier-adder array and the number of columns of the image data to be processed; and determining the quantity of the image data to be processed based on the first value and the second value.

In a possible implementation manner, when pixel combination processing is performed on a plurality of pieces of image data to be processed to obtain a target image, the controller is specifically configured to sequentially use each piece of image data to be processed in the plurality of pieces of image data to be processed as a current image, and determine a target position of a first pixel point of the current image in the target image; and determining the target positions of other pixel points in the current image except the first pixel point in the target image based on the target position of the first pixel point in the target image to obtain the target image.

In a possible implementation manner, when determining, based on a target position of the first pixel point in the target image, a target position of another pixel point in the target image except the first pixel point in the current image, the controller is specifically configured to determine, based on the target position of the first pixel point in the target image in the current image, the number of columns of the multiplier-adder array, and the number of columns of the current image, a first positional relationship between each pixel point in each row of the current image except the first pixel point in the row and an adjacent previous pixel point of the pixel point in the target image; determining a second position relation of each pixel point in each column of the current image except for the first pixel point in the column and an adjacent previous pixel point of the pixel point in the target image based on the target position of the first pixel point of the current image in the target image, the row number of the multiplier-adder array and the row number of the current image; and determining the target positions of other pixel points in the current image except the first pixel point in the target image based on the first position relation and/or the second position relation.

In a possible implementation manner, when determining a target position of a first pixel point of the current image in the target image, the controller is specifically configured to determine a target pixel matrix based on the number of rows and columns of the multiplier-adder array and the number of rows and columns of the image data to be processed; determining the target position of the first pixel point of each image data to be processed in the target image according to the matrix element value of the target pixel matrix; and the target position of the first pixel point of the current image in the target image is any one of the determined target positions.

In a possible implementation manner, when the target image is processed by using a multiplier-adder array, the controller is specifically configured to store the target image into a register array based on a correspondence between the multiplier-adder array and the register array; wherein the adjacent relation of each pixel point of the target image stored in the register array is unchanged; and in each data processing period of a plurality of data processing periods, reading pixel information corresponding to each data processing period from the register array by using the multiplier-adder array, and processing the read pixel information to obtain a data processing result corresponding to each processing period.

In one possible embodiment, when the multiplier-adder array reads pixel information corresponding to the data processing cycle from the register array in each of a plurality of data processing cycles, and processes the read pixel information to obtain a data processing result corresponding to each of the processing cycles, the controller is specifically configured to control each multiplier-adder having the target image pixel information stored in a corresponding register in the multiplier-adder array in a first data processing cycle in which the target image is processed, and read a pixel value of a pixel point corresponding to each multiplier-adder in the first data processing cycle from a register connected to each multiplier-adder as a first operand; determining the weight value corresponding to each multiplier-adder in the first data processing period as a second operand; determining the product of the first operand and the second operand of each multiplier-adder in the first data processing cycle respectively; for each non-first data processing period of processing the target image, controlling each pixel of the target image to move by a preset step length in the register array according to a preset data movement mode corresponding to the data processing period; controlling each multiplier-adder, and reading the pixel value of the pixel point corresponding to each multiplier-adder in the data processing period from a register connected with each multiplier-adder as a first operand; determining the weight value corresponding to each multiplier-adder in the data processing period as a second operand; the product of the first operand and the second operand of each multiplier-adder in the data processing period is respectively determined.

In one possible embodiment, when determining the processing result of each to-be-processed image data based on the processing results obtained by the multipliers and adders corresponding to the to-be-processed image data in the multiplier and adder array, the controller is specifically configured to add the products obtained by the effective multipliers and adders in each data processing cycle to obtain a sum for each effective multiplier and adder in each multiplier and adder; determining effective multiplier-adders corresponding to each piece of image data to be processed from each effective multiplier-adder; and determining the processing result of each image data to be processed based on the sum value respectively obtained by the effective multiplier-adder corresponding to each image data to be processed.

In a possible implementation manner, the controller is further configured to determine the preset step length based on a distance between two adjacent pixel points in the target image that belong to the same image data to be processed.

In a possible implementation manner, when acquiring a plurality of pieces of image data to be processed, the controller is specifically configured to acquire a plurality of original images; taking a first original image in the plurality of original images as the image data to be processed; and/or performing data filling processing on a second original image in the plurality of original images to obtain to-be-processed image data corresponding to the second original image.

In a third aspect, this disclosure also provides a computer device, a controller, and a memory, where the memory stores machine-readable instructions executable by the controller, and the controller is configured to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the controller, the machine-readable instructions are executed by the controller to perform the steps in the first aspect or any one of the possible implementations of the first aspect.

In a fourth aspect, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, where the computer program is executed to perform the steps in the first aspect or any one of the possible implementation manners of the first aspect.

For the description of the effects of the image processing apparatus, the computer device, and the computer-readable storage medium, reference is made to the description of the image processing method, which is not repeated here.

In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

Fig. 1 shows a flowchart of an image processing method provided by an embodiment of the present disclosure;

FIG. 2 illustrates an example diagram of image data to be processed provided by an embodiment of the disclosure;

fig. 3 illustrates an example diagram of a multiplier-adder array provided by an embodiment of this disclosure;

FIG. 4 illustrates an example diagram of a target image provided by an embodiment of the disclosure;

FIG. 5 is a diagram illustrating an example of a matrix for determining a target position of a first pixel point in each image data to be processed in a target image according to an embodiment of the present disclosure;

FIG. 6 illustrates an example diagram of a target image provided by an embodiment of the disclosure;

FIG. 7 illustrates an example diagram of a register array provided by an embodiment of this disclosure;

FIG. 8 illustrates an example diagram of a register array after a pixel value is shifted by a preset step size provided by an embodiment of the present disclosure;

fig. 9 shows a schematic diagram of an image processing apparatus provided by an embodiment of the present disclosure;

fig. 10 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of embodiments of the present disclosure, as generally described and illustrated herein, may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the disclosure, as claimed, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

It has been found that convolutional neural networks rely primarily on multiplier-adder arrays for convolution processing. When convolution processing is carried out, data to be processed is stored in a register array connected with the multiplier-adder array; the data to be processed stored in the register array can move in the register array in different data processing periods; the multiplier-adder array reads the operands of the data processing cycle from the registers (belonging to the register array) connected to the multiplier-adder array and performs multiplication and/or addition operations per data processing cycle. After processing in a plurality of data processing cycles, the effective multiplier-adder in the multiplier-adder array outputs a partial result after convolution processing is carried out on the data to be processed. And combining the results output by the effective multiply-add devices to obtain the final result of performing convolution processing on the data to be processed. The size of the multiplier-adder array is usually fixed. When the size of the data matrix corresponding to the data to be processed is smaller than the size of the multiplier-adder array, only part of the multiplier-adder in the multiplier-adder array needs to be used when the data matrix is subjected to convolution calculation by the multiplier-adder array, that is, only part of the multiplier-adder in the multiplier-adder array is an effective multiplier-adder, so that the current data processing method has the problems of low utilization rate of the multiplier-adder array and waste of calculation resources.

Based on the above research, the present disclosure provides an image processing method, an image processing apparatus, a computer device, and a storage medium, in which a target image is obtained by performing pixel combination on a plurality of pieces of image data to be processed, and the target image is processed by using a multiplier-adder array, so that when the size of the image data to be processed is smaller than that of the multiplier-adder array, the plurality of pieces of image data to be processed can be simultaneously processed by using the same multiplier-adder array, and based on processing results obtained by multipliers and adders corresponding to the plurality of pieces of image data to be processed in the multiplier-adder array, a processing result of each piece of image data to be processed can be determined, thereby improving the utilization rate of the multiplier-adder array, reducing the waste of computing resources, and improving the efficiency of data processing.

The above-mentioned drawbacks are the results of the inventor after practical and careful study, and therefore, the discovery process of the above-mentioned problems and the solutions proposed by the present disclosure to the above-mentioned problems should be the contribution of the inventor in the process of the present disclosure.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

To facilitate understanding of the present embodiment, first, an image processing method disclosed in the embodiments of the present disclosure is described in detail, where an execution subject of the image processing method provided in the embodiments of the present disclosure is generally a computer device with certain computing capability, and the computer device includes, for example: a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle mounted device, a wearable device, or a server or other processing device. In some possible implementations, the image processing method may be implemented by a processor calling computer readable instructions stored in a memory.

The following describes an image processing method provided by the embodiment of the present disclosure.

Referring to fig. 1, a flowchart of an image processing method provided by the embodiment of the present disclosure is shown, where the method includes steps S101 to S103, where:

s101: carrying out pixel combination processing on a plurality of pieces of image data to be processed to obtain a target image;

s102: processing the target image by utilizing the multiplier-adder array;

s103: and determining the processing result of each piece of image data to be processed based on the processing results obtained by the multipliers and adders corresponding to the plurality pieces of image data to be processed in the multiplier and adder array respectively.

The method comprises the steps of obtaining a target image by carrying out pixel combination on a plurality of pieces of image data to be processed, and processing the target image by utilizing a multiplier-adder array, so that when the size of the image data to be processed is smaller than that of the multiplier-adder array, the multiplier-adder array can process the target image formed by combining a plurality of special pictures to be processed, and further realize simultaneous processing of the plurality of pieces of image data to be processed; determining a processing result of each piece of image data to be processed based on processing results obtained by multipliers and adders corresponding to the plurality of pieces of image data to be processed in the multiplier and adder array respectively; the utilization rate of the multiplier-adder array is improved, the waste of computing resources is reduced, and the processing efficiency of the image data to be processed is improved.

The following describes details of S101 to S103.

For the above S101, the image data to be processed includes at least one of the following:

an original image to be processed;

a subgraph corresponding to any color channel in an original image to be processed;

carrying out feature extraction on the original image to obtain a feature map;

performing feature extraction on the original image to obtain a feature subgraph corresponding to at least one channel in a feature graph;

performing data filling processing on a subgraph corresponding to at least one color channel in the original image to obtain;

and performing data filling processing on the characteristic subgraph corresponding to at least one channel of the characteristic graph.

Illustratively, in the case of obtaining to-be-processed image data after performing data padding processing on the second original image, if the size information of the second original image in the original image is 7 rows and 7 columns, and the size of the desired to-be-processed image data is 7 rows and 8 columns, then performing 0-complementing processing on the second original feature map to obtain a 7-row and 8-column image, where the pixel values of the 8 th column of the image are all 0, and taking the image as the to-be-processed image data.

Before the pixel combination processing is performed on a plurality of pieces of image data to be processed, the number of pieces of image data to be processed to be pixel combination processed may be determined first.

The number of image data to be processed may be determined, for example, in the following manner: the number of pieces of image data to be processed is determined based on the size information of the multiplier-adder array and the size information of the image data to be processed.

Here, the size information of the image data to be processed includes the number of rows and the number of columns of pixel points included in each image data to be processed; the size information of the image data to be processed is consistent, namely the number of rows and the number of columns of pixel points contained in each image data to be processed are the same; illustratively, as shown in fig. 2, an exemplary diagram of image data to be processed provided by the present disclosure includes four pieces of image data to be processed with identical size information, which is 7 rows and 7 columns. The size information of the multiplier-adder array includes the number of rows and the number of columns of the multiplier-adder included in the multiplier-adder array; illustratively, fig. 3 is a diagram illustrating an exemplary multiplier-adder array provided by the present disclosure, which includes 14 rows and 14 columns of multiplier-adders.

In specific implementation, the number of image data to be processed may be determined based on the following method, for example:

determining a first value based on the number of rows of the multiplier-adder array and the number of rows of the image data to be processed, and determining a second value based on the number of columns of the multiplier-adder array and the number of columns of the image data to be processed; and determining the quantity of the image data to be processed based on the first value and the second value.

Exemplary, column number A of the multiplier-adder array_xNumber of rows A of 14_y14, the number of columns F of image data to be processed_xIs 7, the number of rows F_yTo 7, a first value, i.e. rounding down, is determined using the number of rows based on the multiplier-adder array and the number of rows of the image data to be processed

Determining a second value, i.e., a second value, using the number of columns of the multiplier-adder array and the number of columns of the image data to be processed

Determining the number of the image data to be processed based on the first value and the second value: namely, it is

The number of image data to be processed is 4. That is, the product of the number of columns of the multiplier-adder array and the result of rounding down the number of columns of the image data to be processed and the product of the number of rows of the multiplier-adder array and the result of rounding down the number of rows of the image data to be processed determines the number of image data to be processed that can be processed at any one time.

After the number of the image data to be processed is determined, the image data to be processed can be obtained.

And after the image data to be processed is obtained, carrying out pixel combination processing on the obtained image data to be processed in the quantity to obtain a target image.

The embodiment of the present disclosure provides a specific method for performing pixel combination processing on a plurality of pieces of image data to be processed, including:

sequentially taking each image data to be processed in the plurality of image data to be processed as a current image, and determining a target position of a first pixel point of the current image in the target image;

and determining the target positions of other pixel points in the current image except the first pixel point in the target image based on the target position of the first pixel point in the target image to obtain the target image.

In a specific implementation, for any current image, a pixel point located in the first row and the first column is a first pixel point in the current image. In the example shown in fig. 2, the first pixel points are the first pixel points in 4 pieces of image data to be processed, as shown by a, b, c, and d in fig. 2.

Firstly, any image data to be processed is taken as a current image, and the target position of the first pixel point of the current image in a target image is determined.

In specific implementation, for example, the target position of the first pixel point of the current feature map in the target image may be determined based on the following method:

determining a target pixel matrix based on the number of rows and columns of the multiplier-adder array and the number of rows and columns of the image data to be processed; determining the target position of the first pixel point of each image data to be processed in the target image according to the matrix element value of the target pixel matrix; and the target position of the first pixel point of the current image in the target image is any one of the determined target positions.

Exemplary, column number A of the multiplier-adder array_xNumber of rows A of 14_y14, the number of columns F of image data to be processed_xIs 7, the number of rows F_yTo 7, it is finally determined that the required number of image data to be processed is

As shown in fig. 4, the calculation formula 2 × 2 is 4 corresponding to the amount of the image data to be processed, the target pixel matrix is a matrix with two rows and two columns, and the position of the first pixel point of the target pixel matrix is the target position of the first pixel point in the first image data to be processed in the target image; based on the position of the first pixel point of the target pixel matrix, the positions of other three pixel points in the target pixel matrix, that is, the target positions of the first pixel points in the other three pieces of image data to be processed, for example, the position arrangement number of the target image is

If the first pixel point of the first image data to be processed is at the position 0, the corresponding position arrangement numbers of the target pixel matrixes in two rows and two columns in the target image, which are determined based on the pixel point of the first image data to be processed at the position 0, are as

Here pixel number 14 is in the second row and first column and pixel number 15 is in the second row and second column.

For example, each target position can be determined according to the formula corresponding to the target position in the matrix shown in FIG. 5The target position of the first pixel point of the image data to be processed in the target image, that is, each matrix element of the matrix shown in fig. 5 represents the target position of the first pixel point of one image to be processed in the target image. Wherein A is_xFor the number of columns of the multiplier-adder array, A_yFor the number of rows of the multiplier-adder array, F_xFor the number of columns of image data to be processed, F_yFor the number of lines of image data to be processed, floor () is a rounding down operation.

It should be noted that the target position of the first pixel point in the other three pieces of to-be-processed image data only needs to be ensured to form a two-row and two-column target pixel matrix with the target position of the first pixel point of the first piece of to-be-processed image data, and does not need to be arranged strictly according to the pattern shown in fig. 4, for example, the pixel point of the third piece of to-be-processed image data may be located next to the pixel point of the first piece of to-be-processed image data in the target pixel matrix, the pixel point of the second piece of to-be-processed image data is located below the pixel point of the third piece of to-be-processed image data, and the pixel point of the fourth piece; other arrangements are similar to those described above and are not repeated here.

After the target position of the first pixel point of each piece of image data to be processed in the target image is determined, for example, the positions of other pixel points in the same piece of image data to be processed, except for the first pixel point, in the target image may be determined based on the following method in steps one to three:

the method comprises the following steps: determining a first position relation of each pixel point in each row of the current image except for a first pixel point in the row and an adjacent previous pixel point of the pixel point in the target image based on a target position of the first pixel point of the current image in the target image, the number of columns of the multiplier-adder array and the number of columns of the current image;

the first position relationship between each pixel point in each line of the current image, except for the first pixel point in the line, and the adjacent previous pixel point of the pixel point in the target image is, for example:

wherein A is_xFor the number of columns of the multiplier-adder array, F_xFor the number of columns of image data to be processed, floor () is a round-down operation.

As shown in FIG. 6, for example, the position arrangement number of the object image is

For example, if the first image data to be processed is a current image, the first pixel point of the first image data to be processed is at position 0, and the position of the pixel point a in the first image data to be processed, which is located in the same row as the first pixel point and adjacent to the first pixel point, in the target image is:

similarly, the position of the next pixel point B adjacent to the pixel point a in the first feature graph to be processed in the target image is: and 2+ floor (14/7) ═ 4, and so on, pixel points in the same row in the same image data to be processed also belong to the same row in the target image.

Step two: determining a second position relation between each pixel point in each column of the current image except for the first pixel point in the column and an adjacent previous pixel point of the pixel point in the target image based on the target position of the first pixel point of the current image in the target image, the number of rows of the multiplier-adder array and the number of rows of the current image;

wherein each of the current images is determinedThe second position relationship between each pixel point in the row except for the first pixel point in the row and the adjacent previous pixel point of the pixel point in the target image is, for example:

wherein A is_yFor the number of rows, F, of the multiplier-adder array_yAs the number of lines of the image data to be processed,

when the multiplier-adder array processes a target image, the utilization rate of the multiplier-adder array is not one hundred percent in some cases, and the number of columns of the multiplier-adder array used actually is as follows:

the number of rows of the multiplier-adder array that is actually utilized is:

For example, if the first image point of the first image data to be processed is at position 0, the pixel point C in the first image data to be processed, which is in the same column as the first pixel point and adjacent to the first pixel point, is at the targetThe positions in the image are:

similarly, the position of the next pixel point D adjacent to the pixel point C in the first feature graph to be processed in the target image is: 28+ floor (14/7) × 7 × 2 ═ 56, and so on, the same column of pixels belonging to the same image data to be processed also belongs to the same column in the target image.

Step three: and determining the target positions of other pixel points in the current image except the first pixel point in the target image based on the first position relation and/or the second position relation.

Illustratively, as shown in FIG. 6, the multiplier-adder array has a column number A_xNumber of rows A of 14_y14, the number of columns F of image data to be processed_xIs 7, the number of rows F_yTo 7, it is finally determined that the required number of image data to be processed is

As shown in fig. 6, four pieces of image data to be processed shown in fig. 2 are: an exemplary diagram of a target image composed of first image data to be processed, second image data to be processed, third image data to be processed and fourth image data to be processed, for example, the position arrangement number of the target image is

When the first pixel point of the first image data to be processed is at the position 0, and the position of the first pixel point of each row or each column of each image data to be processed in the target image is calculated, the target positions of other pixel points except the first pixel point in the current image in the target image can be calculated by referring to the formula for calculating the first pixel point or the second pixel point. For example, if the position of the second pixel point a in the first row in the target image, which belongs to the first image data to be processed, is 2, the position of the next pixel point E in the target image, which belongs to the same column as the pixel point at the position 2 and belongs to the first image data to be processed, is:

or for example, if the position of the pixel point C in the first column of the target image that belongs to the first image data to be processed is 28, the position of the next pixel point E in the target image that belongs to the same row as the pixel point at the position 28 and belongs to the first image data to be processed is:

illustratively, as shown in fig. 4, the embodiment of the present disclosure provides a specific example of a target feature map obtained by performing pixel fusion on four feature maps. In this example, the target image is composed of four pieces of image data to be processed shown in fig. 2, the number of pixels between two adjacent pixels of the same image data to be processed in the same row of the target image is the same and is not zero, and the number of pixels between two adjacent pixels of the same image data to be processed in the same column of the target image is the same and is not zero.

For the above S102, when the multiplier-adder array is used to process the target image, the target image is stored in the register array based on the correspondence between the multiplier-adder array and the register array; wherein, the adjacent relation of each pixel point of the target image stored in the register array is unchanged; and in each data processing period in a plurality of data processing periods, reading pixel information corresponding to each data processing period from the register array by using the multiplier-adder array, and processing the read pixel information to obtain a data processing result corresponding to each processing period.

For example, in the first data processing period of processing the target image, each multiplier-adder with target image pixel information stored in the corresponding register in the multiplier-adder array is controlled, and the pixel value of the pixel point corresponding to the first data processing period of each multiplier-adder is read from the register connected with each multiplier-adder to be used as the first operand; determining the weight value corresponding to each multiplier-adder in the first data processing period as a second operand; determining a product of a first operand and a second operand of each multiplier-adder in the first data processing cycle;

for example, the pixel values of the corresponding pixel points of the first image data to be processed shown in fig. 2 are a0 to a 48; the pixel value of the pixel point corresponding to the image data II to be processed is B0-B48; the pixel value of a pixel point corresponding to the image data III to be processed is C0-C48; the pixel value of the pixel point corresponding to the image data four to be processed is D0-D48; the placement of the target image in the corresponding register array is shown in fig. 7, and in the first data processing cycle, PE0 in the multiplier-adder array shown in fig. 3 reads a pixel value a0, PE1 reads a pixel value B0, PE2 reads a pixel value a1 … …, and so on, and the multiplier-adders PE3 to PE195 in the multiplier-adder array read pixel values of pixel points in corresponding registers.

For each non-first data processing period of processing the target image, controlling each pixel of the target image to move by a preset step length in the register array according to a preset data moving mode corresponding to the data processing period; controlling each multiplier-adder, and reading the pixel value of the pixel point corresponding to each multiplier-adder in the data processing period from the register connected with each multiplier-adder as a first operand; determining the weight value of each multiplier-adder in the data processing period as a second operand; the product of the first operand and the second operand of each multiplier-adder in the data processing cycle is determined.

The method comprises the steps of determining a preset step length based on the distance between two adjacent pixel points belonging to the same image data to be processed in a target image; for example, in the target image corresponding to the register array shown in fig. 7, if the distance between two adjacent pixel points belonging to the same image data to be processed is 2, the preset step size is 2.

Illustratively, the pixel values in the register array shown in fig. 8 are the result of shifting the pixel values in the register array shown in fig. 7 two steps to the left in its entirety, the target register array is the register array shown in fig. 7, and the multiplier-adder array shown in fig. 3 reads the corresponding pixel values from the corresponding positions in the target register array shown in fig. 8.

In addition, the weight value corresponding to each period is the data corresponding to each data processing period in the convolution kernel corresponding to the image data to be processed; taking one of the image data to be processed as an example, the convolution kernel is

The weight data of the first data processing cycle, i.e. the second operand, is W₀In the data processing cycle corresponding to the register array shown in fig. 7, the multiplier-adder PE0 in the multiplier-adder array shown in fig. 3 performs the following calculation in the data processing cycle: w₀A 0; the weight data of the second data processing cycle, i.e. the second operand is W₁In this case, the multiplier-adder PE0 performs the following calculation in the second data processing cycle: w₁A1, if the convolution operation is performed on the image data to be processed, the convolution kernel includes four weight data, and performs a total of four data processing cycles, and the pixel value moves up two steps based on the position shown in fig. 8 in the register array in the third data processing cycle, at this time, the first operation data of the multiplier-adder PE0 in the third data processing cycle is A8, and the second operation data is W₂The multiplier-adder PE0 performs a calculation of W in the third data processing cycle₂A8, in the fourth data processing cycle, the pixel values are shifted to the right by two steps in the register array based on the shift completed in the third data processing cycle, where the first operation data of the multiplier-adder PE0 in the fourth data processing cycle is a7, and the second operation data is W₃The calculation performed by the multiplier-adder PE0 in the fourth data processing cycle is W₃A 7; the calculation process of the other multipliers is similar to that of the multiplier-adder PE0 and will not be described again. And performing all data processing cycles until all data in the convolution kernel is distributed as weight data.

It can be seen that in each data processing cycle, the PE storing the pixel values of different images to be processed completes the calculation of the corresponding data processing cycle, that is, the different images to be processed complete the calculation in the corresponding data processing cycle in parallel in each data processing cycle, and after all the data processing cycles, the different images to be processed complete the final calculation at the same time, thereby saving system resources.

Here, for different image data to be processed, the corresponding convolution kernels may be different or the same. For example, if two pieces of image data to be processed are respectively different feature subgraphs of the same feature graph, convolution kernels corresponding to the two pieces of image data to be processed are different. And if the two pieces of image data to be processed are image data at different positions of the same characteristic subgraph, the convolution kernels corresponding to the two pieces of images to be processed are the same.

For the above S103, after determining the products of the first operand and the second operand of each multiplier-adder in all data processing cycles, adding the products obtained by the effective multiplier-adder in each data processing cycle to obtain a sum for each effective multiplier-adder in each multiplier-adder; determining effective multiplier-adder corresponding to each image data to be processed from each effective multiplier-adder; and determining the processing result of each image data to be processed based on the sum value respectively obtained by the effective multiplier-adder corresponding to each image data to be processed.

The effective multiplier-adder corresponding to each piece of image data to be processed comprises the following steps: the register connected with each data processing period is provided with a multiplier-adder for inputting the effective pixel value of the image data to be processed; for example, multiplier-adder PEs 0 to PE195 in the multiplier-adder array shown in fig. 3 corresponding to the register array shown in fig. 7 are all effective multiplier-adders.

For example, when the multiplier-adder array shown in fig. 3 processes pixel values in the register array shown in fig. 7, after all data processing cycles have been calculated, each multiplier-adder adds and sums the calculation results respectively calculated in all data processing cycles, for example, when the multiplier-adder PE0 performs four data processing cycles, the calculation results are A, B, C, D, and the PE0 performs the following calculation: a + B + C + D.

Here, if the feature graph subjected to convolution includes 16 channels, the feature subgraphs corresponding to 4 channels are processed each time, that is, the feature subgraphs corresponding to 16 channels are divided into 4 groups, and a group of feature subgraphs are processed each time. If the 4 groups of characteristic subgraphs are respectively: when the group a, the group b, the group c and the group d are used, after 4 characteristic subgraphs included in the group a are processed, 4 results corresponding to the group a output by the multiplier-adder are accumulated; after the 4 characteristic subgraphs included in the group b are processed, accumulating 4 results corresponding to the group b, and accumulating the accumulated result corresponding to the group a and the accumulated result corresponding to the group b; after the 4 characteristic subgraphs included in the group c are processed, accumulating 4 results corresponding to the group c, and accumulating the accumulated results of the group a and the group b and the accumulated result corresponding to the group c; after the 4 characteristic subgraphs included in the group d are processed, the 4 results corresponding to the group d are accumulated, the accumulated results of the group a, the group b and the group c and the accumulated result corresponding to the group d are accumulated, and finally, the accumulated sum of the convolution results corresponding to the 16 channels is obtained.

After the 4 feature subgraphs included in the group a are processed, the obtained 4 output results corresponding to the group a are respectively: a1, a2, a3 and a 4. After the 4 feature subgraphs included in the group b are processed, the obtained 4 output results corresponding to the group b are respectively: b1, b2, b3 and b 4. At this time, a1+ b1 ═ O1, a2+ b2 ═ O2, a3+ b3 ═ O3, and a4+ b4 ═ O4 are performed. After the 4 feature subgraphs included in the group c are processed, the obtained 4 output results corresponding to the group c are respectively: c1, c2, c3 and c4, and further performing: o1+ c1, O2+ c2, O3+ c3, O4+ c 4; by analogy, a1+ b1+ c1+ d1, a2+ b2+ c2+ d2, a3+ b3+ c3+ d3 and a4+ b4+ c4+ d4 are finally obtained, and then the four results are accumulated together to obtain the accumulated sum of convolution results corresponding to 16 channels respectively.

The result of the multiplier-adder in fig. 3 corresponding to the register of the same color in fig. 7 is the processing result of the image data to be processed of the color in fig. 2, for example, multiplier-adder PE0, multiplier-adder PE2, multiplier-adder PE4, multiplier-adder PE6, multiplier-adder PE8, multiplier-adder PE10, multiplier-adder PE12, multiplier-adder PE28, multiplier-adder PE30, multiplier-adder PE32, multiplier-adder PE34, multiplier-adder PE36, multiplier-adder PE38, multiplier-adder PE40, multiplier-adder PE56, multiplier-adder PE58, multiplier-adder PE142, multiplier-adder PE58, multiplier-adder PE 36142, multiplier-adder PE 36144, multiplier-adder 58, multiplier-adder PE 36142, multiplier-adder 36144, multiplier-adder PE58, multiplier-adder, EP 36144, multiplier-adder PE146, adder PE 36144, adder PE 36142, adder, The calculation results of the multiplier-adder PE152, the multiplier-adder PE168, the multiplier-adder PE170, the multiplier-adder PE172, the multiplier-adder PE174, the multiplier-adder PE176, the multiplier-adder PE178 and the multiplier-adder PE180 are the processing results of the to-be-processed image data one.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Based on the same inventive concept, an image processing apparatus corresponding to the image processing method is also provided in the embodiments of the present disclosure, and since the principle of the apparatus in the embodiments of the present disclosure for solving the problem is similar to the image processing method described above in the embodiments of the present disclosure, the implementation of the apparatus may refer to the implementation of the method, and repeated details are not described again.

Referring to fig. 9, a schematic diagram of an image processing apparatus provided in an embodiment of the present disclosure is shown, where the apparatus includes: a controller 901; the controller 901 is configured to:

acquiring a plurality of pieces of image data to be processed;

performing fusion processing on the multiple pieces of image data to be processed to obtain a target image;

and processing the target image by utilizing a multiplier-adder array to obtain a processing result of each piece of to-be-processed image data in the plurality of pieces of to-be-processed image data.

In a possible implementation, the controller 901 is further configured to: determining the number of the image data to be processed based on the size information of the multiplier-adder array and the size information of the image data to be processed.

In one possible embodiment, the size information includes a number of rows and a number of columns; when determining the number of the image data to be processed based on the size information of the multiplier-adder array and the size information of the image data to be processed, the controller 901 is specifically configured to determine a first value based on the number of rows of the multiplier-adder array and the number of rows of the image data to be processed, and determine a second value based on the number of columns of the multiplier-adder array and the number of columns of the image data to be processed; and determining the quantity of the image data to be processed based on the first value and the second value.

In a possible implementation manner, when performing pixel combination processing on a plurality of pieces of image data to be processed to obtain a target image, the controller 901 is specifically configured to sequentially use each piece of image data to be processed in the plurality of pieces of image data to be processed as a current image, and determine a target position of a first pixel point of the current image in the target image; and determining the target positions of other pixel points in the current image except the first pixel point in the target image based on the target position of the first pixel point in the target image to obtain the target image.

In a possible implementation manner, when determining, based on the target position of the first pixel point in the target image, the target positions of other pixel points in the target image except the first pixel point in the current image, the controller 901 is specifically configured to determine, based on the target position of the first pixel point in the target image in the current image, the number of columns of the multiplier-adder array, and the number of columns of the current image, a first positional relationship between each pixel point in each row of the current image except the first pixel point in the row and an adjacent previous pixel point of the pixel point in the target image; determining a second position relation of each pixel point in each column of the current image except for the first pixel point in the column and an adjacent previous pixel point of the pixel point in the target image based on the target position of the first pixel point of the current image in the target image, the row number of the multiplier-adder array and the row number of the current image; and determining the target positions of other pixel points in the current image except the first pixel point in the target image based on the first position relation and/or the second position relation.

In a possible implementation manner, when determining a target position of a first pixel point of the current image in the target image, the controller 901 is specifically configured to determine a target pixel matrix based on the number of rows and columns of the multiplier-adder array and the number of rows and columns of the image data to be processed; determining the target position of the first pixel point of each image data to be processed in the target image according to the matrix element value of the target pixel matrix; and the target position of the first pixel point of the current image in the target image is any one of the determined target positions.

In a possible implementation manner, when the target image is processed by using a multiplier-adder array, the controller 901 is specifically configured to store the target image into a register array based on a correspondence relationship between the multiplier-adder array and the register array; wherein the adjacent relation of each pixel point of the target image stored in the register array is unchanged; and in each data processing period of a plurality of data processing periods, reading pixel information corresponding to each data processing period from the register array by using the multiplier-adder array, and processing the read pixel information to obtain a data processing result corresponding to each processing period.

In one possible embodiment, when, in each data processing cycle of a plurality of data processing cycles, the multiplier-adder array reads pixel information corresponding to the data processing cycle from the register array, and processes the read pixel information to obtain a data processing result corresponding to the each processing cycle, the controller 901 is specifically configured to control each multiplier-adder having the target image pixel information stored in a corresponding register in the multiplier-adder array in a first data processing cycle in which the target image is processed, and read a pixel value of a pixel point corresponding to each multiplier-adder in the first data processing cycle from a register connected to the multiplier-adder as a first operand; determining the weight value corresponding to each multiplier-adder in the first data processing period as a second operand; determining the product of the first operand and the second operand of each multiplier-adder in the first data processing cycle respectively; for each non-first data processing period of processing the target image, controlling each pixel of the target image to move by a preset step length in the register array according to a preset data movement mode corresponding to the data processing period; controlling each multiplier-adder, and reading the pixel value of the pixel point corresponding to each multiplier-adder in the data processing period from a register connected with each multiplier-adder as a first operand; determining the weight value corresponding to each multiplier-adder in the data processing period as a second operand; the product of the first operand and the second operand of each multiplier-adder in the data processing period is respectively determined.

In a possible embodiment, when determining the processing result of each to-be-processed image data based on the processing results obtained by the multipliers and adders corresponding to the to-be-processed image data in the multiplier and adder array, the controller 901 is specifically configured to add the products obtained by each effective multiplier and adder in each data processing cycle to obtain a sum for each effective multiplier and adder in each multiplier and adder; determining effective multiplier-adders corresponding to each piece of image data to be processed from each effective multiplier-adder; and determining the processing result of each image data to be processed based on the sum value respectively obtained by the effective multiplier-adder corresponding to each image data to be processed.

In a possible implementation manner, the controller 901 is further configured to determine the preset step length based on a distance between two adjacent pixel points in the target image that belong to the same image data to be processed.

In a possible implementation manner, when acquiring a plurality of pieces of image data to be processed, the controller 901 is specifically configured to acquire a plurality of original images; taking a first original image in the plurality of original images as the image data to be processed; and/or performing data filling processing on a second original image in the plurality of original images to obtain to-be-processed image data corresponding to the second original image.

The image processing device provided by the embodiment of the disclosure may include a chip, an AI chip, and the like.

The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.

An embodiment of the present disclosure further provides a computer device, as shown in fig. 10, which is a schematic structural diagram of the computer device provided in the embodiment of the present disclosure, and the computer device includes:

a controller 101 and a memory 102; the memory 102 stores machine-readable instructions executable by the controller 101, and the controller 101 is configured to execute the machine-readable instructions stored in the memory 102, when the machine-readable instructions are executed by the controller 101, the controller 101 performs the following steps:

carrying out pixel combination processing on the multiple pieces of image data to be processed to obtain a target image;

processing the target image by utilizing a multiplier-adder array;

and determining the processing result of each piece of image data to be processed based on the processing results obtained by the multipliers and adders respectively corresponding to the plurality of pieces of image data to be processed in the multiplier and adder array.

The storage 102 includes a memory 1021 and an external storage 1022; here, the memory 1021 is also referred to as an internal memory, and is used to temporarily store arithmetic data in the controller 101 and data exchanged with the external memory 1022 such as a hard disk, and the controller 101 exchanges data with the external memory 1022 through the memory 1021.

The computer device provided by the embodiment of the present disclosure may include an intelligent terminal such as a mobile phone, or may also be other devices, servers, and the like that have a camera and can perform image processing, and is not limited herein.

For the specific execution process of the instruction, reference may be made to the steps of the image processing method described in the embodiments of the present disclosure, and details are not described here.

The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to perform the steps of the image processing method described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

The embodiments of the present disclosure also provide a computer program product, where the computer program product carries a program code, and instructions included in the program code may be used to execute the steps of the image processing method in the foregoing method embodiments, which may be referred to specifically in the foregoing method embodiments, and are not described herein again.

The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. An image processing method, comprising:

carrying out pixel combination processing on a plurality of pieces of image data to be processed to obtain a target image;

processing the target image by utilizing a multiplier-adder array;

2. The method according to claim 1, wherein the number of pixels between two adjacent pixels of the same image data to be processed in the same row of the target image is the same and is not zero, and the number of pixels between two adjacent pixels of the same image data to be processed in the same column of the target image is the same and is not zero.

3. The image processing method according to claim 1 or 2, further comprising: determining the number of the image data to be processed based on the size information of the multiplier-adder array and the size information of the image data to be processed.

4. The image processing method according to claim 3, wherein the size information includes a number of rows and a number of columns;

the determining the number of the image data to be processed based on the size information of the multiplier-adder array and the size information of the image data to be processed includes:

determining a first value based on the number of rows of the multiplier-adder array and the number of rows of the image data to be processed, and determining a second value based on the number of columns of the multiplier-adder array and the number of columns of the image data to be processed;

and determining the quantity of the image data to be processed based on the first value and the second value.

5. The image processing method according to any one of claims 1 to 4, wherein the performing pixel combination processing on the plurality of pieces of image data to be processed to obtain the target image comprises:

6. The image processing method according to claim 5, wherein the determining the target position of the other pixel points in the current image except the first pixel point in the target image based on the target position of the first pixel point in the target image comprises:

determining a first position relation of each pixel point in each row of the current image except for a first pixel point in the row and an adjacent previous pixel point of the pixel point in the target image based on a target position of the first pixel point of the current image in the target image, the number of columns of the multiplier-adder array and the number of columns of the current image;

determining a second position relation of each pixel point in each column of the current image except for the first pixel point in the column and an adjacent previous pixel point of the pixel point in the target image based on the target position of the first pixel point of the current image in the target image, the row number of the multiplier-adder array and the row number of the current image;

and determining the target positions of other pixel points in the current image except the first pixel point in the target image based on the first position relation and/or the second position relation.

7. The image processing method according to claim 5 or 6, wherein the determining the target position of the first pixel point of the current image in the target image comprises:

determining a target pixel matrix based on the number of rows and columns of the multiplier-adder array and the number of rows and columns of the image data to be processed;

determining the target position of the first pixel point of each image data to be processed in the target image according to the matrix element value of the target pixel matrix;

and the target position of the first pixel point of the current image in the target image is any one of the determined target positions.

8. The image processing method according to any one of claims 1 to 7, wherein the processing the target image using a multiplier-adder array includes:

storing the target image into a register array based on the corresponding relation between the multiplier-adder array and the register array; wherein the adjacent relation of each pixel point of the target image stored in the register array is unchanged;

and in each data processing period of a plurality of data processing periods, reading pixel information corresponding to each data processing period from the register array by using the multiplier-adder array, and processing the read pixel information to obtain a data processing result corresponding to each processing period.

9. The image processing method according to claim 8, wherein the reading, with the multiplier-adder array, pixel information corresponding to each of the plurality of data processing cycles from the register array and processing the read pixel information to obtain a data processing result corresponding to each of the plurality of data processing cycles, comprises:

in the first data processing period of processing the target image, controlling each multiplier-adder with the target image pixel information stored in the corresponding register in the multiplier-adder array, and reading the pixel value of the pixel point corresponding to the first data processing period of each multiplier-adder from the register connected with each multiplier-adder as a first operand; determining the weight value corresponding to each multiplier-adder in the first data processing period as a second operand; determining the product of the first operand and the second operand of each multiplier-adder in the first data processing cycle respectively;

for each non-first data processing period of processing the target image, controlling each pixel of the target image to move by a preset step length in the register array according to a preset data movement mode corresponding to the data processing period; controlling each multiplier-adder, and reading the pixel value of the pixel point corresponding to each multiplier-adder in the data processing period from a register connected with each multiplier-adder as a first operand; determining the weight value corresponding to each multiplier-adder in the data processing period as a second operand; the product of the first operand and the second operand of each multiplier-adder in the data processing period is respectively determined.

10. The image processing method according to claim 8, wherein determining the processing result of each piece of image data to be processed based on the processing results obtained by the multipliers and adders corresponding to the plurality of pieces of image data to be processed in the multiplier and adder array, respectively, comprises:

for each effective multiplier-adder in each multiplier-adder, adding products obtained by the effective multiplier-adder in each data processing period to obtain a sum;

determining effective multiplier-adders corresponding to each piece of image data to be processed from each effective multiplier-adder;

and determining the processing result of each image data to be processed based on the sum value respectively obtained by the effective multiplier-adder corresponding to each image data to be processed.

11. The image processing method of claim 10, wherein the valid multiplier-adder for each piece of image data to be processed comprises: the register connected with each data processing cycle is provided with a multiplier-adder for inputting the effective pixel value of the piece of image data to be processed.

12. The image processing method according to any one of claims 9 to 11, characterized in that the image processing method further comprises: and determining the preset step length based on the distance between two adjacent pixel points belonging to the same image data to be processed in the target image.

13. The image processing method according to any one of claims 1 to 12, wherein the acquiring the plurality of pieces of image data to be processed includes:

acquiring a plurality of original images;

taking a first original image in the plurality of original images as the image data to be processed; and/or the presence of a gas in the gas,

and performing data filling processing on a second original image in the plurality of original images to obtain to-be-processed image data corresponding to the second original image.

14. An image processing apparatus characterized by comprising: a controller; the controller is configured to:

acquiring a plurality of pieces of image data to be processed;

15. A computer device, comprising: a controller, a memory storing machine-readable instructions executable by the controller, the controller to execute the machine-readable instructions stored in the memory, the machine-readable instructions, when executed by the controller, the controller to perform the steps of the image processing method of any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when executed by a computer device, performs the steps of the image processing method according to any one of claims 1 to 13.