WO2022160704A1 - 一种图像处理方法、装置、计算机设备及存储介质 - Google Patents

一种图像处理方法、装置、计算机设备及存储介质 Download PDF

Info

Publication number
WO2022160704A1
WO2022160704A1 PCT/CN2021/115745 CN2021115745W WO2022160704A1 WO 2022160704 A1 WO2022160704 A1 WO 2022160704A1 CN 2021115745 W CN2021115745 W CN 2021115745W WO 2022160704 A1 WO2022160704 A1 WO 2022160704A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
processed
multiplier
pixel
adder
Prior art date
Application number
PCT/CN2021/115745
Other languages
English (en)
French (fr)
Inventor
周军
周亮
常亮
吴飞
Original Assignee
成都商汤科技有限公司
电子科技大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 成都商汤科技有限公司, 电子科技大学 filed Critical 成都商汤科技有限公司
Publication of WO2022160704A1 publication Critical patent/WO2022160704A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • the present disclosure relates to the field of computer technology, and in particular, to an image processing method, an apparatus, a computer device, and a storage medium.
  • the convolutional neural network mainly relies on the multiplier-adder array for convolution processing.
  • the multiplier-adder array stores the data to be processed in the data matrix in the corresponding register array, and the data to be processed moves in the register array in different data processing cycles.
  • the current data processing method has the problems of low utilization rate of the multiplier-adder array and waste of computing resources.
  • Embodiments of the present disclosure provide at least an image processing method, apparatus, computer device, and storage medium.
  • an embodiment of the present disclosure provides an image processing method, including: performing pixel combination processing on multiple images to be processed to obtain a target image; processing the target image by using a multiplier-adder array; The processing results obtained by the respective multiplier-adders corresponding to the images to be processed in the multiplier-adder array determine the processing result of each image to be processed.
  • the size of the obtained target image is larger than the size of each image to be processed in the multiple images to be processed, and the multiplier-adder array is used in the process of processing the target image.
  • the multiplier-adder array processes multiple images to be processed at the same time, and more multiplier-adders in the multiplier-adder array can be used, which improves the utilization rate of the multiplier-adder array, reduces the waste of computing resources, and improves the processing efficiency. image processing efficiency.
  • the number of pixels between adjacent two pixels of the same image to be processed is the same and not zero, and in the same column of the target image, the same image to be processed has the same number of pixels.
  • the number of pixels between adjacent two pixels is the same and not zero.
  • the multiplier-adder array can obtain the correct processing result corresponding to each image to be processed when the target image is processed, and improve the accuracy of the image to be processed. processing efficiency.
  • the method further includes: determining the number of the to-be-processed images based on the size information of the multiplier-adder array and the size information of the to-be-processed images.
  • the number of images to be processed that can be processed by the multiplier-adder array at the same time can be determined, which improves the utilization efficiency of the multiplier-adder array and ensures the multiplication and addition.
  • the accuracy of the result of the processing of the image to be processed by the array of detectors can be determined, which improves the utilization efficiency of the multiplier-adder array and ensures the multiplication and addition.
  • performing pixel combination processing on the plurality of images to be processed to obtain the target image includes: sequentially using each to-be-processed image in the plurality of images to be processed as the current image, and determining the target image.
  • the target position of the first pixel of the current image in the target image includes: sequentially using each to-be-processed image in the plurality of images to be processed as the current image, and determining the target image.
  • the target position of the first pixel of the current image in the target image includes: sequentially using each to-be-processed image in the plurality of images to be processed as the current image, and determining the target image.
  • the target position of the first pixel of the current image in the target image includes: sequentially using each to-be-processed image in the plurality of images to be processed as the current image, and determining the target image.
  • the target position of the first pixel of the current image in the target image includes: sequentially using each to-be-processed image in the plurality of images to be processed
  • each image to be processed can be determined based on the target position of the first pixel of each image to be processed in the target image
  • the target position of other pixels in the target image except the first pixel in the target image improves the efficiency of obtaining the target image.
  • the target position in the image includes: based on the number of columns of the multiplier-adder array and the number of columns of the pixel data array of the image to be processed, determining the number of pixels in each row of the current image divided by the first pixel of the row.
  • the first positional relationship between each pixel outside the pixel and the adjacent previous pixel of the pixel in the target image; based on the number of rows and columns of the multiplier-adder array and the pixels of the image to be processed The number of rows and columns of the data array, to determine the relationship between each pixel point in each column of the current image except the first pixel point of the column and the adjacent previous pixel point of the pixel point in the target image.
  • the second positional relationship based on the target position of the first pixel of the current image in the target image, the first positional relationship and the second positional relationship, determine that the current image is not the first pixel in the current image except the first pixel.
  • the target position of other pixels in the target image based on the target position of the first pixel of the current image in the target image, the first positional relationship and the second positional relationship.
  • the determining the target position of the first pixel of the current image in the target image includes: based on the number of rows and columns of the multiplier-adder array, the Process the number of rows and columns of the pixel data array of the image to determine the target pixel matrix; according to the matrix element value of the target pixel matrix, determine the target position of the first pixel of each image to be processed in the target image; wherein , the target position of the first pixel of the current image in the target image is any target position in the determined target positions.
  • the processing the target image by using the multiplier-adder array includes: storing the target image in the target image based on the correspondence between the multiplier-adder array and the register array. In the register array; wherein, the adjacent relationship of each pixel of the target image stored in the register array is unchanged; in each data processing cycle in a plurality of data processing cycles, the multiplier-adder array is used Pixel information corresponding to each data processing period is read from the register array, and the read pixel information is processed to obtain a data processing result corresponding to each data processing period.
  • the multiplier-adder array ensures the validity of the processing result of each image to be processed by reading the corresponding pixel information in different data processing cycles.
  • the multiplier-adder array is used to read the corresponding data processing cycle from the register array from the register array. pixel information, and process the read pixel information to obtain the data processing result corresponding to each data processing cycle, including: controlling the multiplier-adder in the first data processing cycle of processing the target image
  • the corresponding registers in the array store each multiplier-adder of the target image pixel information, and from the register connected to each multiplier-adder, read the pixel corresponding to each multiplier-adder in the first data processing cycle
  • the pixel value of the point is used as the first operand; and the weight value corresponding to each multiplier-adder in the first data processing cycle is determined as the second operand;
  • the product of the first operand and the second operand of the first data processing cycle for each non-first data processing cycle that processes the target image, control the pixel values of the target image according to the The preset data movement mode corresponding to the data processing cycle moves the preset step size in the
  • the pixel information of the target image is shifted in an orderly manner with the transformation of the data processing cycle in the register array, so as to ensure that the corresponding multiplier-adder in the multiplier-adder array can obtain Valid data to ensure the validity of the processing results of the target image.
  • determining the processing result of each image to be processed based on the processing results obtained by the multiplier-adders corresponding to the multiple images to be processed in the multiplier-adder array including : for each effective multiplier-adder in the multiplier-adders, add the products obtained by the effective multiplier-adders in each data processing cycle to obtain a sum value; from each effective multiplier-adder, determine the The effective multiplier-adders corresponding to the images to be processed respectively; and the processing result of each image to be processed is determined based on the sum values obtained by the effective multiplier-adders corresponding to each image to be processed.
  • the processing result of each to-be-processed image can be obtained based on the sum values obtained by the effective multiplier-adders corresponding to each of the to-be-processed images, thereby improving the processing efficiency of the to-be-processed images.
  • the effective multiplier-adder corresponding to each image to be processed includes: a register connected to each data processing cycle has a multiplier-adder inputting the effective pixel value of the image to be processed .
  • the image processing method further includes: determining the preset step size based on the distance between two adjacent pixels in the target image belonging to the same image to be processed.
  • the validity of the multiplier-adder array for processing results of each image to be processed included in the target image can be ensured based on the setting of the preset step size, and the multiplier-adder array is improved.
  • the flexibility of the array to process the target image is improved.
  • the acquiring multiple images to be processed includes: acquiring multiple original images; using a first original image in the multiple original images as the image to be processed; and/or, Data filling processing is performed on a second original image in the plurality of original images to obtain an image to be processed corresponding to the second original image.
  • a to-be-processed image of a suitable size is obtained based on data filling of the second original image, so that the sources of the to-be-processed image are more abundant, and the image processing performed on the to-be-processed image is more flexible.
  • an embodiment of the present disclosure further provides an image processing apparatus, including: a controller; the controller is configured to: acquire multiple images to be processed; perform pixel combination processing on the multiple images to be processed to obtain a target image ; Use the multiplier-adder array to process the target image, and determine the processing of each to-be-processed image based on the processing results obtained by the corresponding multiplier-adders in the multiplier-adder array of the plurality of images to be processed result.
  • the number of pixels between adjacent two pixels of the same image to be processed is the same and not zero, and in the same column of the target image, the same image to be processed has the same number of pixels.
  • the number of pixels between adjacent two pixels is the same and not zero.
  • the controller is further configured to: determine the number of the to-be-processed images based on the size information of the multiplier-adder array and the size information of the to-be-processed images.
  • the size information includes the number of rows and columns; when determining the number of the images to be processed based on the size information of the multiplier-adder array and the size information of the images to be processed, the The controller is specifically configured to determine the first value based on the number of rows of the multiplier-adder array and the number of rows of the pixel data array of the image to be processed, and based on the number of columns of the multiplier-adder array and all The number of columns of the pixel data array of the image to be processed is determined, and a second value is determined; the number of the image to be processed is determined based on the first value and the second value.
  • the controller when pixel combination processing is performed on multiple images to be processed to obtain a target image, the controller is specifically configured to sequentially combine each image to be processed in the multiple images to be processed As the current image, determine the target position of the first pixel of the current image in the target image; determine the target position of the first pixel in the target image based on the target position of the first pixel The target image is obtained from the target positions of other pixels except the first pixel in the target image.
  • the controller is specifically configured to determine, based on the number of columns of the multiplier-adder array and the number of columns of the pixel data array of the image to be processed, to divide the row in each row of the current image The first positional relationship between each pixel point other than the first pixel point and the adjacent previous pixel point of the pixel point in the target image; based on the number of rows and columns of the multiplier-adder array and all The number of rows and columns of the pixel data array of the image to be processed, it is determined that each pixel point in each column of the current image except the first pixel point of the column and the adjacent previous pixel point of the pixel point are in The second positional relationship in the target image; based on the target position of the first pixel of the current image in the target image, the first positional relationship and the second positional relationship
  • the controller when determining the target position of the first pixel of the current image in the target image, is specifically configured to be based on the number of rows and the sum of the multiplier arrays.
  • the number of columns, the number of rows and columns of the pixel data array of the image to be processed determine the target pixel matrix; according to the matrix element value of the target pixel matrix, it is determined that the first pixel of each image to be processed is in the target image.
  • the target position in the target image; wherein, the target position of the first pixel of the current image in the target image is any target position in the determined target positions.
  • the controller when the target image is processed by the multiplier-adder array, the controller is specifically configured to, based on the corresponding relationship between the multiplier-adder array and the register array, The target image is stored in the register array; wherein, the adjacent relationship of each pixel point of the target image stored in the register array is unchanged; in each data processing cycle in a plurality of data processing cycles, use The multiplier-adder array reads pixel information corresponding to each data processing cycle from the register array, and processes the read pixel information to obtain a data processing result corresponding to each data processing cycle .
  • the multiplier-adder array in each data processing cycle in a plurality of data processing cycles, is used to read pixel information corresponding to each data processing cycle from the register array , and process the read pixel information to obtain the data processing result corresponding to each data processing cycle, the controller is specifically configured to control the target image in the first data processing cycle of processing the target image.
  • the corresponding registers in the multiplier-adder array store each multiplier-adder of the pixel information of the target image, and from the register connected to the each multiplier-adder, read the first data of the each multiplier-adder.
  • the pixel value of the pixel corresponding to the processing cycle is used as the first operand; and the weight value corresponding to each multiplier-adder in the first data processing cycle is determined as the second operand;
  • the pixel value of the pixel corresponding to each multiplier-adder in the data processing period is used as the first operand; and the weight value corresponding to each multiplier-adder in the data processing period is determined as the second operand;
  • the product of the first operand and the second operand of each multiplier-accumulator in the data processing cycle is determined respectively.
  • the The controller when determining the processing result of each image to be processed based on the processing results obtained by the respective multiplier-adders corresponding to the multiple images to be processed in the multiplier-adder array, the The controller is specifically configured to, for each effective multiplier-adder in the various multiplier-adders, add products obtained by the effective multiplier-adders in each data processing cycle to obtain a sum value; In the process, the effective multiplier-adder corresponding to each image to be processed is determined; the processing result of each image to be processed is determined based on the sum value obtained by the effective multiplier-adder corresponding to each image to be processed.
  • the effective multiplier-adder corresponding to each image to be processed includes: a register connected to each data processing cycle has a multiplier-adder inputting the effective pixel value of the image to be processed .
  • the controller is further configured to determine the preset step size based on the distance between two adjacent pixels in the target image that belong to the same image to be processed.
  • the controller when acquiring multiple images to be processed, is specifically configured to acquire multiple original images; and use the first original image in the multiple original images as the to-be-processed image and/or, performing data filling processing on a second original image in the plurality of original images to obtain an image to be processed corresponding to the second original image.
  • an optional implementation manner of the present disclosure further provides a computer device, a controller, and a memory, where the memory stores machine-readable instructions executable by the controller, and the controller is configured to execute the instructions stored in the memory.
  • machine-readable instructions when the machine-readable instructions are executed by the controller, the machine-readable instructions are executed by the controller to execute the first aspect above, or any possible implementation of the first aspect steps in the method.
  • an optional implementation manner of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program executes the first aspect, or any of the first aspect, when the computer program is run. steps in one possible implementation.
  • FIG. 1 shows a flowchart of an image processing method provided by an embodiment of the present disclosure
  • FIG. 2 shows an example diagram of a pixel data array of an image to be processed provided by an embodiment of the present disclosure
  • FIG. 3 shows an example diagram of a multiplier-adder array provided by an embodiment of the present disclosure
  • FIG. 4 shows an example diagram of a pixel data array of a target image provided by an embodiment of the present disclosure
  • FIG. 5 shows an example diagram of a matrix for determining the target position of the first pixel in each image to be processed in the target image provided by an embodiment of the present disclosure
  • FIG. 6 shows an example diagram of a target image provided by an embodiment of the present disclosure
  • FIG. 7 shows an example diagram of a register array provided by an embodiment of the present disclosure
  • FIG. 8 shows an example diagram of a register array provided by an embodiment of the present disclosure after the pixel value is moved by a preset step size
  • FIG. 9 shows a schematic diagram of an image processing apparatus provided by an embodiment of the present disclosure.
  • FIG. 10 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.
  • convolutional neural networks mainly rely on multiplier-adder arrays for convolution processing.
  • the data to be processed will be stored in the register array connected to the multiplier-adder array; the data to be processed stored in the register array will be moved in the register array in different data processing cycles; the multiplier-adder array
  • the operands of the data processing cycle are read from the registers (belonging to the register array) connected to the multiplier-adder, and multiplication, and/or addition operations are performed.
  • the effective multiplier-accumulator in the multiplier-adder array outputs a partial result after the data to be processed is convolved.
  • Multiplier-adder arrays are usually of fixed size. When the size of the data matrix corresponding to the data to be processed is smaller than the size of the multiplier-adder array, only some multiplier-adders in the multiplier-adder array need to be used when the multiplier-adder array performs the convolution calculation on the data matrix, that is, Only some of the multiplier-adders in the multiplier-adder array are effective multiplier-adders, so there are problems of low utilization rate of the multiplier-adder array and waste of computing resources.
  • the present disclosure provides an image processing method, device, computer equipment and storage medium.
  • a target image is obtained by pixel combination of a plurality of images to be processed, and the target image is processed by a multiplier-adder array, so that When the size of the image to be processed is smaller than the size of the multiplier-adder array, the same multiplier-adder array can be used to process multiple images to be processed at the same time.
  • the processing result of each image to be processed can be determined, thereby improving the utilization rate of the multiplier-adder array, reducing the waste of computing resources, and improving the efficiency of data processing.
  • the device includes, for example, a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), Handheld devices, computing devices, in-vehicle devices, wearable devices, etc.
  • the image processing method may be implemented by the processor calling computer-readable instructions stored in the memory.
  • FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure, the method includes steps S101-S103, wherein:
  • S101 Perform pixel combination processing on a plurality of images to be processed to obtain a target image
  • S103 Determine the processing result of each image to be processed based on the processing results obtained by the respective multiplier-adders corresponding to the multiple images to be processed in the multiplier-adder array.
  • the present disclosure obtains a target image by performing pixel combination on multiple images to be processed, and uses the multiplier-adder array to process the target image, so that when the size of the pixel data of the to-be-processed image is smaller than the size of the multiplier-adder array, the multiplier-adder array can pass Process the target image composed of multiple images to be processed, and then process multiple images to be processed at the same time; based on the processing results obtained by the respective multiplier-adders corresponding to the multiple images to be processed in the multiplier-adder array, determine The processing result of each image to be processed; the utilization rate of the multiplier-adder array is improved, the waste of computing resources is reduced, and the processing efficiency of the image to be processed is improved.
  • the image to be processed includes, for example, at least one of the following: an original image to be processed; a sub-image corresponding to any color channel in the original image to be processed; a feature map obtained after feature extraction is performed on the original image; The feature submap corresponding to at least one channel in the feature map obtained after feature extraction of the image; the image obtained after data filling processing is performed on the submap corresponding to at least one color channel in the original image; the image corresponding to at least one channel in the feature map is obtained.
  • the image obtained after the feature submap is filled with data.
  • the image to be processed is obtained after data filling processing is performed on the second original image
  • the desired pixel data of the to-be-processed image is The size of the array is 7 rows and 8 columns
  • the second original image is processed by 0-filling to obtain an image data array with 7 rows and 8 columns.
  • the pixel values of the 8th column of the image data array are all 0.
  • the image data array is used as the pixel data array of the image to be processed.
  • the number of images to be processed to be processed by pixel combination processing may be determined first.
  • the number of images to be processed may be determined in the following manner: the number of images to be processed is determined based on the size information of the multiplier-adder array and the size information of the images to be processed.
  • the size information of the image to be processed includes the number of rows and columns of the pixel array corresponding to each image to be processed; the size information of the multiple images to be processed is consistent, that is, the size of the pixel array corresponding to each image to be processed is the same.
  • the number of rows and columns are the same; wherein, the pixel array may be referred to as a pixel data array, or an image data array; correspondingly, the pixel array corresponding to the image to be processed may be referred to as the pixel data array of the image to be processed, or simply referred to as Array of image data to be processed.
  • the size information of the multiplier-adder array includes the row number and the column number of the multiplier-adder array included in the multiplier-adder array; exemplarily, as shown in FIG.
  • the multiplier-adder array contains 14 rows and 14 columns of multiplier-adders.
  • the number of images to be processed may be determined based on the following method: based on the number of rows of the multiplier-adder array and the number of rows of the image data array to be processed, the first value Ny is determined, and the column of the multiplier-adder array is determined based on and the number of columns of the image data array to be processed, determine the second value Nx; determine the number of images to be processed based on the first value Ny and the second value Nx.
  • the number of columns A x of the multiplier-adder array is 14, the number of rows A y is 14, the number of columns F x of the image data array to be processed is 7, and the number of rows F y is 7, using the rows of the multiplier-adder array. and the number of rows of the image data array to be processed, determine the first value Ny, that is, the quotient of the row number of the multiplier-adder array and the row number of the image data array to be processed is rounded down
  • the second value Nx is determined, that is, the quotient of the number of columns of the multiplier-adder array and the number of columns of the image data array to be processed is rounded down
  • the number N of images to be processed is determined based on the first value Ny and the second value Nx: namely
  • the number N of images to be processed is 4. That is, the result of rounding down the quotient of the number of columns of the multiplier-adder array and the number of columns of the image data array to be processed is rounded down to the quotient of the number of rows of the multiplier-adder array and the number of rows of the image data array to be processed.
  • the product between the results is determined as the number of images to be processed that can be processed.
  • N images to be processed can be acquired.
  • pixel combination processing is performed on the acquired N images to be processed to obtain a target image.
  • An embodiment of the present disclosure provides a specific method for performing pixel combination processing on multiple images to be processed, including: sequentially taking each image to be processed in the multiple images to be processed as a current image, and determining the The target position of the first pixel in the target image; based on the target position of the first pixel in the target image, determine that other pixels in the current image except the first pixel are in the target image. The target position in the target image is obtained to obtain the target image.
  • the pixel located in the first row and the first column is the first pixel in the current image.
  • the first pixel points are shown as a, b, c, and d in FIG. 2 , which are the first pixel points in the four images to be processed.
  • the target position of the first pixel of the current image in the target image can be determined based on the following method: based on the number of rows and columns of the multiplier-adder array, the number of rows and columns of the image data array to be processed, Determine the target pixel matrix; according to the matrix element value of the target pixel matrix, determine the target position of the first pixel of each image to be processed in the target image; wherein, the target position of the first pixel of the current image in the target image is determined any of the outgoing target positions.
  • the number of columns A x of the multiplier-adder array is 14, the number of rows A y is 14, the number of columns F x of the image data array to be processed is 7, and the number of rows F y is 7, and finally the required image to be processed is determined.
  • the first pixel point of the first image to be processed is at position 0, then the target pixel matrix with two rows and two columns determined based on the pixel point a of the first image to be processed at position 0 is numbered at the corresponding position in the target image as the following numbered array:
  • the pixel numbered 1 is located in the first row and second column in the target image, which can correspond to the first pixel b in the second to-be-processed image;
  • the pixel numbered 14 is located in the second row and first column in the target image, It can correspond to the first pixel c in the third image to be processed;
  • the pixel number 15 is located in the second row and second column of the target image, and can correspond to the first pixel d of the fourth image to be processed.
  • the target position of the first pixel of each image to be processed in the target image can also be determined according to the formula corresponding to the target position in the matrix shown in FIG. 5 , that is, the matrix shown in FIG. 5 .
  • Each matrix element represents the target position of the first pixel of an image to be processed in the target image.
  • a x is the column number of the multiplier-adder array
  • a y is the row number of the multiplier-adder array
  • F x is the column number of the image data array to be processed
  • F y is the row number of the image data array to be processed
  • floor( ) is a round-down operation.
  • the target position of the first pixel in the other three image data arrays to be processed only needs to be guaranteed to form a target with two rows and two columns with the target position of the first pixel in the first image data array to be processed.
  • the pixel matrix is sufficient, and it does not need to be arranged in strict accordance with the style shown in Figure 4.
  • the right side of the pixel point of the first image to be processed in the target pixel matrix can be the pixel point of the third image to be processed, and the third image to be processed Below the pixels of the second image to be processed are the pixels of the second to-be-processed image, and below the pixels of the first to-be-processed image are the pixels of the fourth to-be-processed image; other arrangements are similar to the above examples, and the examples are not repeated here.
  • Step 1 Based on the number of columns of the multiplier-adder array and the number of columns of the image data array to be processed, determine the neighbors of each pixel in each row of the current image except the first pixel of the row and the pixel.
  • the first positional relationship of the previous pixel in the target image that is, to determine the positional relationship between other pixels in the current image except the first pixel and the previous pixel in the same line and adjacent to the target image .
  • the position number SN(Pi) of each pixel point Pi in the target image except for the first pixel point of this row in each row of the current image is the same as the previous pixel point Pi that is in the same row and adjacent to the pixel point Pi.
  • the first positional relationship between the position number SN(Pi-1) of -1 in the target image is, for example:
  • the number of columns A x of the multiplier-adder array is 14, the number of rows A y is 14, the number of columns F x of the image data array to be processed is 7, and the number of rows F y is 7, and finally the required image to be processed is determined.
  • the position number SN(B) of the next pixel point B in the same line and adjacent to the pixel point A in the target image is:
  • pixels belonging to the same row in the same image to be processed also belong to the same row in the target image.
  • Step 2 Based on the number of rows and columns of the multiplier-adder array and the number of rows and columns of the image data array to be processed, determine the relationship between each pixel point in each column of the current image except the first pixel point of the column.
  • the second positional relationship of the adjacent previous pixel of the pixel in the target image that is, it is determined that other pixels except the first pixel in the current image and the adjacent previous pixel in the same column are in the target image. positional relationship in the image.
  • the utilization rate of the multiplier-adder array is not 100%, and the actual number of columns of the multiplier-adder array is:
  • the actual number of rows of the multiplier-adder array used is:
  • the number of columns A x of the multiplier-adder array is 14, the number of rows A y is 14, the number of columns F x of the image data array to be processed is 7, and the number of rows F y is 7, and finally the first value Ny is determined.
  • the positional arrangement number of the target image is as shown in the number array SN above, for example, the first image to be processed is the current image, then the first pixel of the first image to be processed is At the position number 0 of the target image, the position number SN(C) of the pixel point C that is located in the same column as the first pixel point in the first to-be-processed image and is adjacent to the first pixel point in the target image is:
  • the position number SN(D) of the next pixel point D in the target image that is in the same column and adjacent to the pixel point C in the first image to be processed is:
  • pixels belonging to the same column in the same image to be processed also belong to the same column in the target image.
  • Step 3 Based on the target position of the first pixel of the current image in the target image, the first positional relationship and the second positional relationship, determine the target positions of other pixels in the current image except the first pixel in the target image.
  • the positional arrangement number of the target image is as shown in the number array SN above, and the first pixel of the first image to be processed is in the target image.
  • the position number 0 in the image, the first pixel of the second image to be processed is at position number 1 in the target image, the first pixel of the third image to be processed is at position number 14 in the target image, and the fourth pixel is at position number 14 in the target image.
  • the first pixel of the image to be processed is at position number 15 in the target image.
  • the position of the first pixel of each image to be processed in the target image After calculating the position of the first pixel of each image to be processed in the target image, other pixels in the image to be processed except the first pixel can be calculated with reference to the above-mentioned first positional relationship and second positional relationship The target position in the target image. Specifically, after calculating the target position of the first pixel of each row and the first pixel of each column in the target image in the current image, other pixels in the current image can be calculated based on the first positional relationship or the second positional relationship The target position in the target image.
  • the second position of the first row in the target image belongs to the pixel A of the first image to be processed, and its corresponding position number SN(A) is 2, then the first image to be processed belongs to the same column as the pixel A And the position number SN(E) of the next adjacent pixel E in the target image is:
  • the first image to be processed belongs to the same pixel point C as the pixel point C.
  • the position number SN(E) of the next adjacent pixel E in the target image is:
  • an embodiment of the present disclosure provides a specific example of a target image obtained by pixel combination of four feature maps.
  • the target image is composed of the four images to be processed as shown in Figure 2.
  • the number of pixels between adjacent two pixels of the same image to be processed is the same and not zero
  • the same column of the target image the number of pixels between adjacent two pixels of the same image to be processed is the same and not zero.
  • the target image when the target image is processed by the multiplier-adder array, the target image is stored in the register array based on the corresponding relationship between the multiplier-adder array and the register array; wherein, the target images stored in the register array are each The adjacent relationship of pixel points remains unchanged; in each data processing cycle of multiple data processing cycles, the multiplier-adder array is used to read pixel information corresponding to each data processing cycle from the register array, and the read data is processed.
  • the pixel information is processed to obtain the data processing result corresponding to each data processing cycle.
  • the pixel information may include pixel values.
  • control each multiplier-adder whose corresponding register in the multiplier-adder array stores the pixel information of the target image and read each multiplier-adder from the register connected to each multiplier-adder.
  • the pixel value of the pixel corresponding to the multiplier-adder in the first data processing cycle is taken as the first operand; and the weight value corresponding to each multiplier-adder in the first data processing cycle is determined as the second operand;
  • the pixel values of the corresponding pixels of the first to-be-processed image are A0 to A48; the pixel values of the corresponding pixels of the second to-be-processed image are B0 to B48; the third to-be-processed image corresponds to the pixels
  • the pixel values of the points are C0 to C48; the pixel values of the pixels corresponding to the fourth to-be-processed image are D0 to D48; the placement of these four to-be-processed images in the register array corresponding to the target image is shown in Figure 7.
  • PE0 in the multiplier-adder array as shown in Figure 3 reads the pixel value A0
  • PE1 reads the pixel value B0
  • PE2 reads the pixel value A1...
  • the multiplier-adder array in the The multipliers PE3-PE195 read the pixel value of the pixel point in the corresponding register.
  • each pixel value of the target image For each non-first data processing cycle of processing the target image, control each pixel value of the target image to move a preset step size in the register array according to the preset data movement mode corresponding to the data processing cycle; and control each multiplication and addition
  • the device reads the pixel value of the pixel corresponding to each multiplier-adder in the data processing cycle as the first operand from the register connected to each multiplier-adder; and determines that each multiplier-adder corresponds to the data processing cycle
  • the weight value of , as the second operand determines the product of the first operand and the second operand of each multiplier-adder in this data processing cycle.
  • the preset step size is determined based on the distance between two adjacent pixels belonging to the same image to be processed in the target image; for example, in the target image corresponding to the register array shown in FIG. If the distance between two adjacent pixels of the image is 2, the preset step size is 2.
  • the pixel value in the register array shown in FIG. 8 is the result of moving the pixel value in the register array shown in FIG. 7 to the left by two steps as a whole
  • the target register array is the result shown in FIG. 7
  • the multiplier-adder array shown in FIG. 3 reads the corresponding pixel value from the corresponding position in the target register array shown in FIG. 8 .
  • the weight values corresponding to each data processing cycle are multiple weight values of the convolution kernel corresponding to the image to be processed, and the multiple weight values correspond to the respective data processing cycles; taking one of the images to be processed as an example, its convolution The core is for example Then the weight value of the first data processing cycle, that is, the second operand is W 0 .
  • the multiplier-adder array shown in Figure 3 The calculation performed by the multiplier-adder PE0 in this data processing cycle is: W 0 *A0; the weight value of the second data processing cycle, that is, the second operand is W 1 , and the multiplier-adder PE0 at this time is in the The calculation performed in the second data processing cycle is: W 1 *A1. If the image to be processed is subjected to convolution operation, the convolution kernel contains four weight values, and a total of four data processing cycles are performed. In the third data processing cycle , the storage position of the pixel value of the target image in the register array is moved up by two steps on the basis of the position shown in Figure 8.
  • the first operand of the multiplier-adder PE0 in the third data processing cycle is A8
  • the second operand is W 2
  • the calculation performed by the multiplier-adder PE0 in the third data processing cycle is W 2 *A8
  • the pixel value of the target image is in the third data processing cycle.
  • the entire register array is shifted to the right by two steps.
  • the first operand of the multiplier-adder PE0 in the fourth data processing cycle is A7
  • the second operand is W 3 .
  • the calculation performed by the multiplier-adder PE0 in the fourth data processing cycle is W 3 *A7; the calculation process of other multiplier-accumulators is similar to that of the multiplier-adder PE0, and will not be repeated here. Until all the data in the convolution kernel is assigned as a weight value, all data processing cycles are completed.
  • the PE that obtains the pixel values of the corresponding pixels in different images to be processed has completed the calculation of the corresponding data processing cycle, that is to say, the calculation of different images to be processed in each data processing cycle is parallelized Completed, after all data processing cycles, the final calculations of different images to be processed are completed simultaneously, saving system resources.
  • the corresponding convolution kernels may be different or the same. For example, if the two images to be processed are different feature submaps of the same feature map, the convolution kernels corresponding to the two images to be processed are different. If the two images to be processed are image data of different positions of the same feature submap, the convolution kernels corresponding to the two images to be processed are the same.
  • a MIMD Multiple Instruction Multiple Data, multiple instruction stream, multiple data stream
  • Weights In each data processing cycle, a weight value corresponding to the data processing cycle can be assigned to multiple multipliers and adders in a SIMD (Single Instruction Multiple Data, single instruction stream, multiple data stream) manner.
  • the effective multiplier-adder is The products obtained by the adder in each data processing cycle are added to obtain a sum value; from each effective multiplier-adder, the effective multiplier-adder corresponding to each image to be processed is determined; based on the effective multiplication and addition corresponding to each image to be processed The sum value obtained by the processor respectively determines the processing result of each image to be processed.
  • the effective multiplier-adder corresponding to each image to be processed includes: in each data processing cycle, the connected register has a multiplier-adder for inputting the effective pixel value of the image to be processed; for example, as shown in Figure 3
  • the multiplier-adders PE0 to PE195 in the multiplier-adder array of are all valid multiplier-adders.
  • each multiplier-adder when the multiplier-adder array shown in FIG. 3 processes the pixel values in the register array shown in FIG. 7 , after all data processing cycles are calculated, each multiplier-adder will The calculation results of all data processing cycles are added and summed.
  • the multiplier-accumulator PE0 performs the calculation of four data processing cycles, and the calculation results are A, B, C, and D respectively. Then PE0 performs the following calculation: A+B+ C+D.
  • the feature sub-maps corresponding to 4 channels are processed each time, that is, the feature sub-maps corresponding to the 16 channels need to be divided into 4 groups, and one group is performed each time. Processing of feature subgraphs. If the four groups of feature sub-maps are: group a, group b, group c and group d, after performing image processing on the four feature sub-maps included in group a, 4 results are accumulated; after image processing is performed on the 4 feature submaps included in group b, the 4 results corresponding to group b are accumulated, and the accumulated results corresponding to group a and the corresponding accumulation results of group b are accumulated.
  • the results are accumulated; after image processing is performed on the 4 feature submaps included in group c, the 4 results corresponding to group c are accumulated, and the accumulated results of group a, group b, and group c corresponding to Accumulation results are accumulated; after image processing is performed on the 4 feature submaps included in group d, the 4 results corresponding to group d are accumulated, and the accumulated results of group a, group b, group c, and The accumulated results corresponding to group d are accumulated, and finally, the accumulated sum of the convolution results corresponding to the 16 channels is obtained.
  • the obtained 4 output results corresponding to the group a are: a1, a2, a3 and a4.
  • the obtained 4 output results corresponding to group b are: b1, b2, b3 and b4 respectively.
  • a1+b1 O1
  • a2+b2 O2
  • a3+b3 O3
  • the obtained 4 output results corresponding to group c are: c1, c2, c3 and c4, and then execute: O1+c1, O2+c2, O3 +c3, O4+c4; and so on, finally get a1+b1+c1+d1, a2+b2+c2+d2, a3+b3+c3+d3, a4+b4+c4+d4, and then four
  • the results are accumulated together to obtain the accumulated sum of the convolution results corresponding to the 16 channels respectively.
  • the processing result of the image to be processed can be obtained from the multiplier-adder of the register corresponding to a specific color in the multiplier-adder array shown in FIG. Manipulate the color of the image.
  • the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.
  • the embodiment of the present disclosure also provides an image processing apparatus corresponding to the image processing method. Reference may be made to the implementation of the method, and repeated descriptions will not be repeated.
  • the apparatus includes a controller 901; the controller 901 is configured to: acquire multiple images to be processed; Perform pixel combination processing to obtain a target image; use a multiplier-adder array to process the target image, and based on the processing results obtained by the corresponding multiplier-adders in the multiplier-adder array of the plurality of images to be processed, Determine the processing result for each image to be processed.
  • the number of pixels between adjacent two pixels of the same image to be processed is the same and not zero, and in the same column of the target image, the same image to be processed has the same number of pixels.
  • the number of pixels between adjacent two pixels is the same and not zero.
  • the controller 901 is further configured to: determine the number of the to-be-processed images based on the size information of the multiplier-adder array and the size information of the to-be-processed images.
  • the size information includes the number of rows and columns; when determining the number of the images to be processed based on the size information of the multiplier-adder array and the size information of the images to be processed, the The controller 901 is specifically configured to determine the first value based on the number of rows of the multiplier-adder array and the number of rows of the pixel data array of the image to be processed, and based on the number of columns of the multiplier-adder array and The number of columns of the pixel data array of the image to be processed determines a second value; the number of images to be processed is determined based on the first value and the second value.
  • the controller 901 when pixel combination processing is performed on a plurality of images to be processed to obtain a target image, the controller 901 is specifically configured to sequentially combine each of the plurality of images to be processed The image is used as the current image, and the target position of the first pixel of the current image in the target image is determined; based on the target position of the first pixel in the target image, the target position of the first pixel in the current image is determined The target position of other pixels except the first pixel in the target image is obtained to obtain the target image.
  • the target position of the first pixel in the target image it is determined that other pixels in the current image except the first pixel are in the target image when the target position in The first positional relationship between each pixel point except the first pixel point of the row and the adjacent previous pixel point of the pixel point in the target image; based on the number of rows, columns and The number of rows and columns of the pixel data array of the image to be processed, to determine each pixel point in each column of the current image except the first pixel point of the column and the adjacent previous pixel point of the pixel point.
  • the second positional relationship in the target image based on the target position of the first pixel of the current image in the target image, the first positional relationship and the second positional relationship, determine the other than the current image.
  • the target positions of other pixels except the first pixel in the target image based on the target position of the first pixel in the target image.
  • the controller 901 when determining the target position of the first pixel of the current image in the target image, is specifically configured to be based on the number of rows of the multiplier-adder array and the number of columns, the number of rows and columns of the pixel data array of the image to be processed, determine the target pixel matrix; according to the matrix element value of the target pixel matrix, determine that the first pixel of each image to be processed is in the target pixel matrix The target position in the image; wherein, the target position of the first pixel of the current image in the target image is any target position in the determined target positions.
  • the controller 901 when using the multiplier-adder array to process the target image, is specifically configured to, based on the corresponding relationship between the multiplier-adder array and the register array, The target image is stored in the register array; wherein, the adjacent relationship of each pixel of the target image stored in the register array is unchanged; in each data processing cycle in the multiple data processing cycles, Use the multiplier-adder array to read pixel information corresponding to each data processing cycle from the register array, and process the read pixel information to obtain data processing corresponding to each data processing cycle result.
  • the multiplier-adder array is used to read pixel information corresponding to each data processing cycle from the register array , and process the read pixel information to obtain the data processing result corresponding to each data processing cycle
  • the controller 901 is specifically used for processing the target image in the first data processing cycle, Controlling each multiplier-adder whose corresponding register in the multiplier-adder array stores the pixel information of the target image, and reading the multiplier-adder in the first
  • the pixel value of the pixel corresponding to the data processing cycle is used as the first operand; and the weight value corresponding to each multiplier-adder in the first data processing cycle is determined as the second operand;
  • the product of the first operand and the second operand of the multiplier-adder in the first data processing cycle for each non-first data processing cycle that processes the target image, control the Each pixel value moves a preset step size in the register array according to a preset data movement mode corresponding to the data processing cycle;
  • the The controller 901 is specifically configured to, for each effective multiplier-adder in the various multiplier-adders, add products obtained by the effective multiplier-adders in each data processing cycle to obtain a sum value; In the device, determine the effective multiplier-adder corresponding to each image to be processed; determine the processing result of each image to be processed based on the sum value obtained by the effective multiplier-adder corresponding to each image to be processed .
  • the effective multiplier-adder corresponding to each image to be processed includes: a register connected to each data processing cycle has a multiplier-adder inputting the effective pixel value of the image to be processed .
  • the controller 901 is further configured to determine the preset step size based on the distance between two adjacent pixel points belonging to the same image to be processed in the target image.
  • the controller 901 when acquiring multiple images to be processed, is specifically configured to acquire multiple original images; the first original image in the multiple original images is used as the to-be-processed image. processing an image; and/or, performing data filling processing on a second original image in the plurality of original images to obtain an image to be processed corresponding to the second original image.
  • the image processing apparatus may include a chip, an AI chip, and the like.
  • An embodiment of the present disclosure further provides a computer device.
  • a schematic structural diagram of the computer device provided by the embodiment of the present disclosure includes a controller 1010 and a memory 1020; Machine-readable instructions, the controller 1010 is configured to execute the machine-readable instructions stored in the memory 1020, when the machine-readable instructions are executed by the controller 1010, the controller 1010 performs the following steps: Perform pixel combination processing to obtain a target image; use a multiplier-adder array to process the target image; based on the processing results obtained by the corresponding multiplier-adders in the multiplier-adder array of the plurality of images to be processed, determine The processing result of each image to be processed.
  • the above-mentioned memory 1020 includes a memory 1021 and an external memory 1022; the memory 1021 here is also called internal memory, which is used to temporarily store the operation data in the controller 1010 and the data exchanged with the external memory 1022 such as the hard disk.
  • the external memory 1022 performs data exchange.
  • the computer device provided by the embodiment of the present disclosure may include a smart terminal such as a mobile phone, or may also be other devices, servers, etc. that have a camera and can perform image processing, which is not limited here.
  • Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the image processing method described in the foregoing method embodiments are executed.
  • the storage medium may be a volatile or non-volatile computer-readable storage medium.
  • Embodiments of the present disclosure further provide a computer program product, where the computer program product carries program codes, and the instructions included in the program codes can be used to execute the steps of the image processing methods described in the foregoing method embodiments.
  • the above-mentioned computer program product can be specifically implemented by means of hardware, software or a combination thereof.
  • the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.
  • the units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.
  • the functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium.
  • the computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

一种图像处理方法、装置、计算机设备及存储介质，其中，该方法包括：对多张待处理图像进行像素组合处理，得到目标图像（S101）；利用乘加器阵列对所述目标图像进行处理（S102）；基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器得到的处理结果，确定每张待处理图像的处理结果（S103）。上述方法能用同一个乘加器阵列同时处理多张待处理图像，基于多张待处理图像在乘加器阵列中分别对应的乘加器得到的处理结果，就可以确定每张待处理图像的处理结果，提高了乘加器阵列的利用率、减少对计算资源的浪费，并且提高了对待处理图像的处理效率。

Description

一种图像处理方法、装置、计算机设备及存储介质
相关申请的交叉引用
本公开要求于2021年1月31日提交的、申请号为202110132579.7的中国专利公开的优先权，该中国专利公开的全部内容以引用的方式并入本文中。
技术领域
本公开涉及计算机技术领域，具体而言，涉及一种图像处理方法、装置、计算机设备及存储介质。
背景技术
目前，卷积神经网络主要依赖乘加器阵列来进行卷积处理，乘加器阵列将数据矩阵中的待处理数据存储在对应的寄存器阵列中，待处理数据在不同数据处理周期在寄存器阵列中移动，但是当前的数据处理方法存在对乘加器阵列利用率低、计算资源浪费的问题。
发明内容
本公开实施例至少提供一种图像处理方法、装置、计算机设备及存储介质。
第一方面，本公开实施例提供了一种图像处理方法，包括：对多张待处理图像进行像素组合处理，得到目标图像；利用乘加器阵列对所述目标图像进行处理；基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器得到的处理结果，确定每张待处理图像的处理结果。
这样，基于将多张待处理图像组合成一张目标图像，得到的目标图像的尺寸大于多张待处理图像中的每张待处理图像的尺寸，利用乘加器阵列对目标图像进行处理的过程中实际上乘加器阵列同时处理了多张待处理图像，能利用到乘加器阵列中更多的乘加器，提高了乘加器阵列的利用率、减少计算资源的浪费、并提高了对待处理图像的处理效率。
在一种可能的实施方式中，所述目标图像的同一行中，同一待处理图像的相邻两像素间隔像素数量相同且不为零，以及所述目标图像的同一列中，同一待处理图像的相邻两像素间隔像素数量相同且不为零。
这样，基于目标图像中多张待处理图像的像素点的排布规则能保证乘加器阵列在对目标图像进行处理时，得到每张待处理图像对应的正确地处理结果，提高对待处理图像的处理效率。
在一种可能的实施方式中，还包括：基于所述乘加器阵列的尺寸信息、以及所述待处理图像的尺寸信息确定所述待处理图像的数量。
这样，基于乘加器阵列的尺寸信息、以及每张待处理图像的尺寸信息可以确定出乘加器阵列能同时处理的待处理图像的数量，提高乘加器阵列利用效率的同时保证了乘加器阵列对待处理图像的处理结果的准确性。
在一种可能的实施方式中，尺寸信息包括行数以及列数；所述基于所述乘加器阵列的尺寸信息、以及所述待处理图像的尺寸信息确定所述待处理图像的数量，包括：基于所述乘加器阵列的行数以及所述待处理图像的像素数据阵列的行数，确定第一取值，并基于所述乘加器阵列的列数以及所述待处理图像的像素数据阵列的列数，确定第二取值；基于所述第一取值和所述第二取值确定所述待处理图像的数量。
在一种可能的实施方式中，所述对多张待处理图像进行像素组合处理，得到目标图像，包括：依次将所述多张待处理图像中的每张待处理图像作为当前图像，确定所述当 前图像的首个像素点在所述目标图像中的目标位置；基于所述首个像素点在所述目标图像中的目标位置，确定所述当前图像中除所述首个像素点外的其他像素点在所述目标图像中的目标位置，得到所述目标图像。
这样，将每张待处理图像的首个像素点在目标图像中的目标位置确定后，基于每张待处理图像的首个像素点在目标图像中的目标位置就可以确定出每张待处理图像中除首个像素点外的其他像素点在目标图像中的目标位置，提高了得到目标图像的效率。
在一种可能的实施方式中，所述基于所述首个像素点在所述目标图像中的目标位置，确定所述当前图像中除所述首个像素点外的其他像素点在所述目标图像中的目标位置，包括：基于所述乘加器阵列的列数、以及所述待处理图像的像素数据阵列的列数，确定所述当前图像每一行中除该行第一个像素点之外的每个像素点与该像素点的相邻前一像素点在所述目标图像中的第一位置关系；基于所述乘加器阵列的行数、列数以及所述待处理图像的像素数据阵列的行数、列数，确定所述当前图像每一列中除该列第一个像素点之外的每个像素点与该像素点的相邻前一像素点在所述目标图像中的第二位置关系；基于所述当前图像首个像素点在所述目标图像中的目标位置、所述第一位置关系和所述第二位置关系，确定所述当前图像中除首个像素点外的其他像素点在所述目标图像中的目标位置。
在一种可能的实施方式中，所述确定所述当前图像的首个像素点在所述目标图像中的目标位置，包括：基于所述乘加器阵列的行数和列数、所述待处理图像的像素数据阵列的行数和列数，确定目标像素矩阵；根据所述目标像素矩阵的矩阵元素值，确定各待处理图像的首个像素点在所述目标图像中的目标位置；其中，所述当前图像的首个像素点在所述目标图像中的目标位置为确定的目标位置中的任一目标位置。
在一种可能的实施方式中，所述利用乘加器阵列对所述目标图像进行处理，包括：基于所述乘加器阵列与寄存器阵列之间的对应关系，将所述目标图像存储至所述寄存器阵列中；其中，存储至所述寄存器阵列中的所述目标图像各个像素点的相邻关系不变；在多个数据处理周期中的每个数据处理周期，利用所述乘加器阵列从所述寄存器阵列中读取与所述每个数据处理周期对应的像素信息，并对读取的像素信息进行处理，得到所述每个数据处理周期对应的数据处理结果。
这样，乘加器阵列通过在不同的数据处理周期读取对应的像素信息保证了对每张待处理图像的处理结果的有效性。
在一种可能的实施方式中，所述在多个数据处理周期中的每个数据处理周期，利用所述乘加器阵列从所述寄存器阵列中读取与所述每个数据处理周期对应的像素信息，并对读取的像素信息进行处理，得到所述每个数据处理周期对应的数据处理结果，包括：在对所述目标图像进行处理的首个数据处理周期，控制所述乘加器阵列中对应寄存器存储有所述目标图像像素信息的各乘加器，从与所述各乘加器连接的寄存器中，读取所述各乘加器在所述首个数据处理周期对应的像素点的像素值作为第一个操作数；并确定所述各乘加器在所述首个数据处理周期对应的权重值，作为第二个操作数；分别确定所述各乘加器在所述首个数据处理周期的第一个操作数和第二个操作数的乘积；针对对所述目标图像进行处理的每个非首个数据处理周期，控制所述目标图像的各像素值按照与该数据处理周期对应的预设数据移动方式在所述寄存器阵列中移动预设步长；并控制所述各乘加器，从与所述各乘加器连接的寄存器中，读取所述各乘加器在该数据处理周期对应的像素点的像素值作为第一个操作数；并确定所述各乘加器在该数据处理周期对应的权重值，作为第二个操作数；分别确定所述各乘加器在该数据处理周期的第一个操作数和第二个操作数的乘积。
这样，基于预设步长、以及预设数据移动方式使得目标图像的像素信息在寄存器阵 列中随数据处理周期的变换作出有序的位移，确保乘加器阵列中对应的乘加器能获取到有效数据，保证对目标图像的处理结果的有效性。
在一种可能的实施方式中，所述基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器所得到的处理结果，确定每张待处理图像的处理结果，包括：针对所述各乘加器中的每个有效乘加器，将该有效乘加器在各个数据处理周期中得到的乘积相加得到和值；从各有效乘加器中，确定所述每张待处理图像分别对应的有效乘加器；基于所述每张待处理图像对应的有效乘加器分别得到的和值，确定所述每张待处理图像的处理结果。
这样，基于每张待处理图像分别对应的有效乘加器分别得到的和值就可以得到每张待处理图像的处理结果，提高了对待处理图像的处理效率。
在一种可能的实施方式中，所述每张待处理图像对应的有效乘加器，包括：每一个数据处理周期所连接的寄存器都有该张待处理图像的有效像素值输入的乘加器。
在一种可能的实施方式中，所述图像处理方法还包括：基于所述目标图像中属于同一待处理图像的相邻两个像素点之间的距离，确定所述预设步长。
这样，能在目标图像包含不同数量的待处理图像时基于对预设步长的设定保证乘加器阵列对目标图像中包含的每张待处理图像处理结果的有效性，提高了乘加器阵列对目标图像进行处理的灵活性。
在一种可能的实施方式中，所述获取多张待处理图像，包括：获取多张原始图像；将所述多张原始图像中的第一原始图像作为所述待处理图像；和/或，对所述多张原始图像中的第二原始图像进行数据填充处理，得到与所述第二原始图像对应的待处理图像。
这样，基于对第二原始图像进行数据填充得到适用尺寸的待处理图像，使得待处理图像的来源更丰富，对待处理图像进行的图像处理更加灵活。
第二方面，本公开实施例还提供一种图像处理装置，包括：控制器；所述控制器用于：获取多张待处理图像；对所述多张待处理图像进行像素组合处理，得到目标图像；利用乘加器阵列对所述目标图像进行处理，并基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器得到的处理结果，确定每张待处理图像的处理结果。
在一种可能的实施方式中，所述目标图像的同一行中，同一待处理图像的相邻两像素间隔像素数量相同且不为零，以及所述目标图像的同一列中，同一待处理图像的相邻两像素间隔像素数量相同且不为零。
在一种可能的实施方式中，所述控制器还用于：基于所述乘加器阵列的尺寸信息、以及所述待处理图像的尺寸信息确定所述待处理图像的数量。
在一种可能的实施方式中，尺寸信息包括行数以及列数；在基于所述乘加器阵列的尺寸信息、以及所述待处理图像的尺寸信息确定所述待处理图像的数量时，所述控制器，具体用于基于所述乘加器阵列的行数以及所述待处理图像的像素数据阵列的行数，确定第一取值，并基于所述乘加器阵列的列数以及所述待处理图像的像素数据阵列的列数，确定第二取值；基于所述第一取值和所述第二取值确定所述待处理图像的数量。
在一种可能的实施方式中，在对多张待处理图像进行像素组合处理，得到目标图像时，所述控制器，具体用于依次将所述多张待处理图像中的每张待处理图像作为当前图像，确定所述当前图像的首个像素点在所述目标图像中的目标位置；基于所述首个像素点在所述目标图像中的目标位置，确定所述当前图像中除所述首个像素点外的其他像素点在所述目标图像中的目标位置，得到所述目标图像。
在一种可能的实施方式中，在基于所述首个像素点在所述目标图像中的目标位置，确定所述当前图像中除所述首个像素点外的其他像素点在所述目标图像中的目标位置时，所述控制器，具体用于基于所述乘加器阵列的列数、以及所述待处理图像的像素数据阵列的列数，确定所述当前图像每一行中除该行第一个像素点之外的每个像素点与该像素点的相邻前一像素点在所述目标图像中的第一位置关系；基于所述乘加器阵列的行数、列数以及所述待处理图像的像素数据阵列的行数、列数，确定所述当前图像每一列中除该列第一个像素点之外的每个像素点与该像素点的相邻前一像素点在所述目标图像中的第二位置关系；基于当前图像首个像素点在所述目标图像中的目标位置、所述第一位置关系和所述第二位置关系，确定所述当前图像中除首个像素点外的其他像素点在所述目标图像中的目标位置。
在一种可能的实施方式中，在确定所述当前图像的首个像素点在所述目标图像中的目标位置时，所述控制器，具体用于基于所述乘加器阵列的行数和列数、所述待处理图像的像素数据阵列的行数和列数，确定目标像素矩阵；根据所述目标像素矩阵的矩阵元素值，确定各待处理图像的首个像素点在所述目标图像中的目标位置；其中，所述当前图像的首个像素点在所述目标图像中的目标位置为确定的目标位置中的任一目标位置。
在一种可能的实施方式中，在利用乘加器阵列对所述目标图像进行处理时，所述控制器，具体用于基于所述乘加器阵列与寄存器阵列之间的对应关系，将所述目标图像存储至所述寄存器阵列中；其中，存储至所述寄存器阵列中的所述目标图像各个像素点的相邻关系不变；在多个数据处理周期中的每个数据处理周期，利用所述乘加器阵列从所述寄存器阵列中读取与所述每个数据处理周期对应的像素信息，并对读取的像素信息进行处理，得到所述每个数据处理周期对应的数据处理结果。
在一种可能的实施方式中，在多个数据处理周期中的每个数据处理周期，利用所述乘加器阵列从所述寄存器阵列中读取与所述每个数据处理周期对应的像素信息，并对读取的像素信息进行处理，得到所述每个数据处理周期对应的数据处理结果时，所述控制器，具体用于在对所述目标图像进行处理的首个数据处理周期，控制所述乘加器阵列中对应寄存器存储有所述目标图像像素信息的各乘加器，从与所述各乘加器连接的寄存器中，读取所述各乘加器在所述首个数据处理周期对应的像素点的像素值作为第一个操作数；并确定所述各乘加器在所述首个数据处理周期对应的权重值，作为第二个操作数；分别确定所述各乘加器在所述首个数据处理周期的第一个操作数和第二个操作数的乘积；针对对所述目标图像进行处理的每个非首个数据处理周期，控制所述目标图像的各像素值按照与该数据处理周期对应的预设数据移动方式在所述寄存器阵列中移动预设步长；并控制所述各乘加器，从与所述各乘加器连接的寄存器中，读取所述各乘加器在该数据处理周期对应的像素点的像素值作为第一个操作数；并确定所述各乘加器在该数据处理周期对应的权重值，作为第二个操作数；分别确定所述各乘加器在该数据处理周期的第一个操作数和第二个操作数的乘积。
在一种可能的实施方式中，在基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器所得到的处理结果，确定每张待处理图像的处理结果时，所述控制器，具体用于针对所述各乘加器中的每个有效乘加器，将该有效乘加器在各个数据处理周期中得到的乘积相加得到和值；从各有效乘加器中，确定所述每张待处理图像分别对应的有效乘加器；基于所述每张待处理图像对应的有效乘加器分别得到的和值，确定所述每张待处理图像的处理结果。
在一种可能的实施方式中，所述每张待处理图像对应的有效乘加器，包括：每一个数据处理周期所连接的寄存器都有该张待处理图像的有效像素值输入的乘加器。
在一种可能的实施方式中，所述控制器，还用于基于所述目标图像中属于同一待处 理图像的相邻两个像素点之间的距离，确定所述预设步长。
在一种可能的实施方式中，在获取多张待处理图像时，所述控制器，具体用于获取多张原始图像；将所述多张原始图像中的第一原始图像作为所述待处理图像；和/或，对所述多张原始图像中的第二原始图像进行数据填充处理，得到与所述第二原始图像对应的待处理图像。
第三方面，本公开可选实现方式还提供一种计算机设备，控制器、存储器，所述存储器存储有所述控制器可执行的机器可读指令，所述控制器用于执行所述存储器中存储的机器可读指令，所述机器可读指令被所述控制器执行时，所述机器可读指令被所述控制器执行时执行上述第一方面，或第一方面中任一种可能的实施方式中的步骤。
第四方面，本公开可选实现方式还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被运行时执行上述第一方面，或第一方面中任一种可能的实施方式中的步骤。
关于上述图像处理装置、计算机设备、及计算机可读存储介质的效果描述参见上述图像处理方法的说明，这里不再赘述。
为使本公开的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍。这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。应当理解，以下附图仅示出了本公开的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1示出了本公开实施例所提供的一种图像处理方法的流程图；
图2示出了本公开实施例所提供的待处理图像的像素数据阵列的示例图；
图3示出了本公开实施例所提供的一种乘加器阵列的示例图；
图4示出了本公开实施例所提供的一种目标图像的像素数据阵列的示例图；
图5示出了本公开实施例所提供的一种确定每张待处理图像中首个像素点在目标图像中的目标位置的矩阵示例图；
图6示出了本公开实施例所提供的一种目标图像的示例图；
图7示出了本公开实施例所提供的一种寄存器阵列的示例图；
图8示出了本公开实施例所提供的像素值移动预设步长后寄存器阵列的示例图；
图9示出了本公开实施例所提供的一种图像处理装置的示意图；
图10示出了本公开实施例所提供的一种计算机设备的示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。通常在此处描述和示出的本公开实施例的组件可以以各种不同的配置来布置和设计。因此，以下对本公开的实施例的详细描述并非旨在限制要求保护的本公开的范围，而是仅仅表示本公开的选定实施例。基于本 公开的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本公开保护的范围。
经研究发现，卷积神经网络主要依赖乘加器阵列来进行卷积处理。在进行卷积处理时，待处理数据会被存储至与乘加器阵列连接的寄存器阵列中；存储在寄存器阵列中的待处理数据会在不同的数据处理周期在寄存器阵列中移动；乘加器阵列在每个数据处理周期，从与乘加器连接的寄存器(属于寄存器阵列)中读取该数据处理周期的操作数，并执行乘法、和/或加法运算。在经过多个数据处理周期的处理后，乘加器阵列中的有效乘加器输出对待处理数据进行卷积处理后的部分结果。将多个有效乘加器输出的结果结合，即得到对待处理数据进行卷积处理的最终结果。乘加器阵列的尺寸通常都是固定的。在待处理数据所对应的数据矩阵的尺寸小于乘加器阵列的尺寸时，乘加器阵列对数据矩阵进行卷积计算时只需要利用到乘加器阵列中的部分乘加器，也即，乘加器阵列中的乘加器只有部分为有效乘加器，因此存在对乘加器阵列利用率低、计算资源浪费的问题。
基于上述研究，本公开提供了一种图像处理方法、装置、计算机设备及存储介质，通过将多张待处理图像进行像素组合得到一张目标图像，利用乘加器阵列对目标图像进行处理，使得在待处理图像的尺寸小于乘加器阵列的尺寸时，能用同一个乘加器阵列同时处理多张待处理图像，基于多张待处理图像在乘加器阵列中分别对应的乘加器得到的处理结果，就可以确定每张待处理图像的处理结果，从而提高了乘加器阵列的利用率，减少计算资源的浪费，且提高了对数据处理的效率。
以上方案所存在的缺陷，均是发明人在经过实践并仔细研究后得出的，因此，上述问题的发现过程以及下文中本公开针对上述问题所提出的解决方案，都应该是发明人在本公开过程中对本公开做出的贡献。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步定义和解释。
为便于对本实施例进行理解，首先对本公开实施例所公开的一种图像处理方法进行详细介绍，本公开实施例所提供的图像处理方法的执行主体一般为具有一定计算能力的计算机设备，该计算机设备例如包括终端设备或服务器或其它处理设备，终端设备可以为用户设备(User Equipment，UE)、移动设备、用户终端、终端、蜂窝电话、无绳电话、个人数字助理(Personal Digital Assistant，PDA)、手持设备、计算设备、车载设备、可穿戴设备等。在一些可能的实现方式中，该图像处理方法可以通过处理器调用存储器中存储的计算机可读指令的方式来实现。
下面对本公开实施例提供的图像处理方法加以说明。
参见图1所示，为本公开实施例提供的图像处理方法的流程图，所述方法包括步骤S101～S103，其中：
S101：对多张待处理图像进行像素组合处理，得到目标图像；
S102：利用乘加器阵列对目标图像进行处理；
S103：基于多张待处理图像在乘加器阵列中分别对应的乘加器得到的处理结果，确定每张待处理图像的处理结果。
本公开通过对多张待处理图像进行像素组合得到目标图像，利用乘加器阵列处理目标图像，使得在待处理图像的像素数据的尺寸小于乘加器阵列的尺寸时，乘加器阵列能通过对由多张待处理图像组合成的目标图像进行处理，进而实现同时处理多张待处理图像；基于该多张待处理图像在乘加器阵列中分别对应的乘加器得到的处理结果，确定每张待处理图像的处理结果；提高了乘加器阵列的利用率、减少了计算资源的浪费，并且 提高了对待处理图像的处理效率。
下面对上述S101～S103加以详细说明。
针对上述S101，待处理图像例如包括下述至少一种：要处理的原始图像；要处理的原始图像中任一颜色通道对应的子图；对原始图像进行特征提取后得到的特征图；对原始图像进行特征提取后得到的特征图中至少一个通道对应的特征子图；对原始图像中的至少一个颜色通道对应的子图进行数据填充处理后得到的图像；对特征图中至少一个通道对应的特征子图进行数据填充处理后得到的图像。
示例性的，在对第二原始图像进行数据填充处理后得到待处理图像的情况下，如果原始图像中的第二原始图像的尺寸信息为7行7列，想要的待处理图像的像素数据阵列的尺寸大小为7行8列，则对第二原始图像进行补0处理，得到一张7行8列的图像数据阵列，该图像数据阵列的第8列的像素值都为0，将该图像数据阵列作为待处理图像的像素数据阵列。
在对多张待处理图像进行像素组合处理之前，首先可以确定要进行像素组合处理的待处理图像的数量。
例如可以利用下述方式确定待处理图像的数量：基于乘加器阵列的尺寸信息、以及待处理图像的尺寸信息确定待处理图像的数量。
此处，待处理图像的尺寸信息包括每张待处理图像对应的像素点阵列的行数、以及列数；多张待处理图像的尺寸信息一致，即每张待处理图像对应的像素点阵列的行数、以及列数相同；其中，像素点阵列可称为像素数据阵列、或图像数据阵列；相应地，待处理图像对应的像素点阵列可称为待处理图像的像素数据阵列、或简称为待处理图像数据阵列。示例性的，如图2所示为本公开提供的一种待处理图像的像素数据阵列的示例图，包含四张尺寸信息一致，为7行7列的待处理图像数据阵列。乘加器阵列的尺寸信息包括乘加器阵列中包含的乘加器的行数、以及列数；示例性的，如图3所示为本公开提供的一种乘加器阵列的示例图，该乘加器阵列包含14行14列乘加器。
具体实施时，例如可以基于下述方法确定待处理图像的数量：基于乘加器阵列的行数以及待处理图像数据阵列的行数，确定第一取值Ny，并基于乘加器阵列的列数以及所述待处理图像数据阵列的列数，确定第二取值Nx；基于第一取值Ny和第二取值Nx确定待处理图像的数量。
示例性的，乘加器阵列的列数A x为14，行数A y为14，待处理图像数据阵列的列数F x为7，行数F y为7，利用乘加器阵列的行数、以及待处理图像数据阵列的行数，确定第一取值Ny，即将乘加器阵列的行数与待处理图像数据阵列的行数的商向下取整
Figure PCTCN2021115745-appb-000001
利用乘加器阵列的列数以及待处理图像数据阵列的列数，确定第二取值Nx，即将乘加器阵列的列数以及待处理图像数据阵列的列数的商向下取整
Figure PCTCN2021115745-appb-000002
基于第一取值Ny和第二取值Nx确定待处理图像的数量N：即
Figure PCTCN2021115745-appb-000003
所以待处理图像的数量N是4。即乘加器阵列的列数与待处理图像数据阵列的列数的商向下取整的结果，与乘加器阵列的行数与待处理图像数据阵列的行数的商向下取整 的结果之间的乘积确定为可处理的待处理图像的数量。
在确定了待处理图像的数量N后，即可以获取N个待处理图像。
在获取了N个待处理图像后，将获取的该N个待处理图像进行像素组合处理，得到目标图像。
本公开实施例提供了一种将多张待处理图像进行像素组合处理的具体方法，包括：依次将所述多张待处理图像中的每张待处理图像作为当前图像，确定所述当前图像的首个像素点在所述目标图像中的目标位置；基于所述首个像素点在所述目标图像中的目标位置，确定所述当前图像中除所述首个像素点外的其他像素点在所述目标图像中的目标位置，得到所述目标图像。
在具体实施中，针对任一张当前图像，位于第一行第一列的像素点，即为该当前图像中的首个像素点。如图2所示的示例中，首个像素点如图2中的a、b、c以及d所示，分别为4张待处理图像中的首个像素点。
首先将其中任一待处理图像作为当前图像，确定该当前图像的首个像素点在目标图像中的目标位置。
具体实施时，例如可以基于下述方法确定当前图像的首个像素点在目标图像中的目标位置：基于乘加器阵列的行数和列数、待处理图像数据阵列的行数和列数，确定目标像素矩阵；根据目标像素矩阵的矩阵元素值，确定各待处理图像的首个像素点在目标图像中的目标位置；其中，当前图像的首个像素点在目标图像中的目标位置为确定出的目标位置中的任一目标位置。
示例性的，乘加器阵列的列数A x为14，行数A y为14，待处理图像数据阵列的列数F x为7，行数F y为7，最终确定需要的待处理图像的数量为2*2＝4；如图4所示，对应于待处理图像的数量计算公式2*2＝4，则目标像素矩阵为一个两行两列的矩阵，该目标像素矩阵的第一个像素点的位置就是首张待处理图像中的首个像素点在目标图像中的目标位置；基于目标像素矩阵的第一个像素点的位置，就可以确定该目标像素矩阵中其他三个像素点的位置，也即其他三张待处理图像中的首个像素点的目标位置，例如目标图像的位置排布编号为如下的编号阵列SN所示：
Figure PCTCN2021115745-appb-000004
第一待处理图像的首个像素点在位置0处，则基于位置0处的第一待处理图像的像素点a确定的两行两列的目标像素矩阵在目标图像中对应的位置排布编号为如下的编号阵列所示：
Figure PCTCN2021115745-appb-000005
这里编号1的像素点位于目标图像中的第一行第二列，可对应第二待处理图像中的首个像素点b；编号14的像素点位于目标图像中的第二行第一列，可对应第三待处理图像中的首个像素点c；编号15的像素点位于目标图像中的第二行第二列，可对应第四待处理图像的首个像素点d。
示例性的，还可以按照图5所示的矩阵中目标位置对应的公式来确定每张待处理图像的首个像素点在目标图像中的目标位置，也就是说，图5中所示矩阵的每个矩阵元素分别表征一个待处理图像的首个像素点在目标图像中的目标位置。其中，A x为乘加器阵列的列数，A y为乘加器阵列的行数，F x为待处理图像数据阵列的列数，F y为待处理图像 数据阵列的行数，floor()为向下取整运算。
需要说明的是，其他三个待处理图像数据阵列中的首个像素点所在的目标位置只需要保证和首个待处理图像数据阵列的首个像素点的目标位置构成一个两行两列的目标像素矩阵即可，无需严格按照图4所示的样式排布，例如，目标像素矩阵中的第一待处理图像的像素点的右边可以是第三待处理图像的像素点，第三待处理图像的像素点下边是第二待处理图像的像素点，第一待处理图像的像素点下边是第四待处理图像的像素点；其他的排布方式与上述举例类似，此处不再重复举例。
在确定了每张待处理图像的首个像素点在目标图像中的目标位置后，例如可以基于下述步骤一～步骤三所述的方法确定同一张待处理图像中除了首个像素点外的其他像素点在目标图像中的位置：
步骤一：基于乘加器阵列的列数、以及待处理图像数据阵列的列数，确定当前图像每一行中除该行第一个像素点之外的每个像素点与该像素点的相邻前一像素点在目标图像中的第一位置关系，也即确定当前图像中除首个像素点之外的其他像素点与同行且相邻的前一像素点之间在目标图像中的位置关系。
其中，当前图像每一行中除该行第一个像素点之外的每个像素点Pi在目标图像中的位置编号SN(Pi)与该像素点Pi的同行且相邻的前一像素点Pi-1在目标图像中的位置编号SN(Pi-1)之间的第一位置关系例如为：
SN(Pi)＝SN(Pi-1)+Nx。
示例性的，乘加器阵列的列数A x为14，行数A y为14，待处理图像数据阵列的列数F x为7，行数F y为7，最终确定需要的待处理图像的数量为2*2＝4；如图6所示，例如目标图像的位置排布编号为如上编号阵列SN所示，例如第一待处理图像为当前图像，则第一待处理图像的首个像素点在目标图像的位置编号0处，则第一待处理图像中与首个像素点位于同一行并与该首个像素点相邻的像素点A在目标图像中的位置编号SN(A)为：
SN(A)＝0+Nx＝2。
同理，第一待处理图像中与像素点A同行且相邻的下一像素点B在目标图像中的位置编号SN(B)为：
SN(B)＝SN(A)+Nx＝2+Nx＝4。
以此类推，属于同一张待处理图像中的同一行像素点在目标图像中也属于同一行。
步骤二：基于乘加器阵列的行数、列数以及待处理图像数据阵列的行数、列数，确定当前图像每一列中除该列第一个像素点之外的每个像素点与该像素点的相邻前一像素点在目标图像中的第二位置关系，也即确定当前图像中除首个像素点之外的其他像素点与同列且相邻的前一像素点之间在目标图像中的位置关系。
其中，确定当前图像每一列中除该列第一个像素点之外的每个像素点Pj在目标图像中的位置编号SN(Pj)与该像素点Pj的同列且相邻的前一像素点Pj-1在目标图像中的位置编号SN(Pj-1)之间的第二位置关系例如为：
SN(Pj)＝SN(Pj-1)+Ny*A′ x＝SN(Pj-1)+Ny*F x*Nx。
其中，乘加器阵列在对目标图像进行处理时，有些情况下乘加器阵列的利用率不是百分之百，实际用的乘加器阵列的列数为：
Figure PCTCN2021115745-appb-000006
实际利用到的乘加器阵列的行数为：
Figure PCTCN2021115745-appb-000007
示例性的，乘加器阵列的列数A x为14，行数A y为14，待处理图像数据阵列的列数F x为7，行数F y为7，最终确定第一取值Ny＝2，第二取值Nx＝2，需要的待处理图像的数量N为4，实际利用到的乘加器阵列的列数A′ x等于14，实际利用到的乘加器阵列的行数A′ y等于14；如图6所示，例如目标图像的位置排布编号为如上的编号阵列SN所示，例如第一待处理图像为当前图像，则第一待处理图像的首个像素点在目标图像的位置编号0处，则第一待处理图像中与首个像素点位于同一列并与该首个像素点相邻的像素点C在目标图像中的位置编号SN(C)为：
SN(C)＝0+Ny*F x*Nx＝0+2*7*2＝28。
同理第一待处理图像中与像素点C同列且相邻的下一像素点D在目标图像中的位置编号SN(D)为：
SN(D)＝SN(C)+Ny*F x*Nx＝28+2*7*2＝56。
以此类推，属于同一张待处理图像中的同一列像素点在目标图像中也属于同一列。
步骤三：基于当前图像首个像素点在目标图像中的目标位置、第一位置关系和第二位置关系，确定当前图像中除首个像素点外的其他像素点在目标图像中的目标位置。
示例性的，如图6所示，乘加器阵列的列数A x为14，行数A y为14，待处理图像数据阵列的列数F x为7，行数F y为7，最终确定需要的待处理图像的数量为Nx*Ny＝2*2＝4；如图6所示为由图2所示的四张待处理图像：第一待处理图像、第二待处理图像、第三待处理图像、第四待处理图像，组成的一种目标图像的示例图，例如目标图像的位置排布编号为如上的编号阵列SN所示，第一待处理图像的首个像素点在目标图像中的位置编号0处，第二待处理图像的首个像素点在目标图像中的位置编号1处，第三待处理图像的首个像素点在目标图像中的位置编号14处，第四待处理图像的首个像素点在目标图像中的位置编号15处。
在计算出每张待处理图像的首个像素点在目标图像中的位置之后，就可以参照上述第一位置关系和第二位置关系计算该待处理图像中除首个像素点外的其他像素点在目标图像中的目标位置。具体的，在计算得到当前图像中每行首个像素点、每列首个像素点在目标图像中的目标位置后，可以基于第一位置关系或第二位置关系计算出当前图像中其他像素点在目标图像中的目标位置。例如，目标图像中第一行的第二个位置属于第一待处理图像的像素点A，其对应的位置编号SN(A)为2，则第一待处理图像中与像素点A属于同一列并相邻的下一个像素点E在目标图像中的位置编号SN(E)为：
SN(E)＝2+Ny*F x*Nx＝2+2*7*2＝30。
或者例如，目标图像中第一列的第二个位置属于第一待处理图像的像素点C，其对应的位置编号SN(C)为28，则第一待处理图像中与像素点C属于同一行并相邻的下一个像素点E在目标图像中的位置编号SN(E)为：
SN(E)＝28+Nx＝28+2＝30。
示例性的，如图4所示，本公开实施例提供一种将四张特征图进行像素组合，得到的目标图像的具体示例。该示例中，目标图像由图2所示的四张待处理图像组成，目标图像的同一行中，同一待处理图像的相邻两像素间隔像素数量相同且不为零，以及 目标图像的同一列中，同一待处理图像的相邻两像素间隔像素数量相同且不为零。
针对上述S102，利用乘加器阵列对目标图像进行处理时，基于乘加器阵列与寄存器阵列之间的对应关系，将目标图像存储至寄存器阵列中；其中，存储至寄存器阵列中的目标图像各个像素点的相邻关系不变；在多个数据处理周期中的每个数据处理周期，利用乘加器阵列从寄存器阵列中读取与每个数据处理周期对应的像素信息，并对读取的像素信息进行处理，得到每个数据处理周期对应的数据处理结果。本公开实施例中，像素信息可以包括像素值。
例如，在对目标图像进行处理的首个数据处理周期，控制乘加器阵列中对应寄存器存储有目标图像的像素信息的各乘加器，从与各乘加器连接的寄存器中，读取各乘加器在首个数据处理周期对应的像素点的像素值作为第一个操作数；并确定各乘加器在首个数据处理周期对应的权重值，作为第二个操作数；分别确定各乘加器在所述首个数据处理周期的第一个操作数和第二个操作数的乘积。
示例性的，如图2所示的第一待处理图像的对应像素点的像素值为A0～A48；第二待处理图像对应像素点的像素值为B0～B48；第三待处理图像对应像素点的像素值为C0～C48；第四待处理图像对应像素点的像素值为D0～D48；这四个待处理图像在目标图像对应的寄存器阵列中的摆放如图7所示，在第一个数据处理周期，如图3所示的乘加器阵列中的PE0读取像素值A0，PE1读取像素值B0，PE2读取像素值A1……以此类推，乘加器阵列中的乘加器PE3～PE195读取对应的寄存器中的像素点的像素值。
针对对目标图像进行处理的每个非首个数据处理周期，控制目标图像的各像素值按照与该数据处理周期对应的预设数据移动方式在寄存器阵列中移动预设步长；并控制各乘加器，从与各乘加器连接的寄存器中，读取各乘加器在该数据处理周期对应的像素点的像素值作为第一个操作数；并确定各乘加器在该数据处理周期对应的权重值，作为第二个操作数；确定各乘加器在该数据处理周期的第一个操作数和第二个操作数的乘积。
其中，基于目标图像中属于同一待处理图像的相邻两个像素点之间的距离，确定预设步长；例如，如图7所示的寄存器阵列对应的目标图像中，属于同一张待处理图像的相邻两个像素点之间的距离为2，则预设步长就为2。
示例性的，如图8所示的寄存器阵列中的像素值为图7所示的寄存器阵列中的像素值整体向左移动两个步长后的结果，目标寄存器阵列为图7中所示的寄存器阵列，图3所示的乘加器阵列从如图8所示的目标寄存器阵列中对应的位置读取对应的像素值。
另外，各个数据处理周期对应的权重值为待处理图像对应的卷积核的多个权重值，多个权重值分别与各个数据处理周期对应；以其中一张待处理图像为例，其卷积核例如为
Figure PCTCN2021115745-appb-000008
则第一个数据处理周期的权重值也即第二个操作数为W 0，如图7所示的寄存器阵列存储的像素值对应的数据处理周期中，如图3所示的乘加器阵列中的乘加器PE0在该数据处理周期进行的计算为：W 0*A0；第二个数据处理周期的权重值也即第二个操作数为W 1，此时的乘加器PE0在该第二个数据处理周期进行的计算为：W 1*A1，如果对待处理图像进行卷积运算，卷积核包含四个权重值，一共进行四个数据处理周期，在第三个数据处理周期中，目标图像的像素值在寄存器阵列中的存储位置在图8所示位置的基础上向上移动两个步长，此时乘加器PE0在第三个数据处理周期的第一个操作数为A8，第二个操作数为W 2，乘加器PE0在第三个数据处理周期进行的计算为W 2*A8，在第四个数据处理周期，目标图像的像素值在第三个数据处理周期完成的移动的基础上在寄存器阵列中整体向右移动两个步长，此时乘加器PE0在第四个数据处理周期的第一个操作数为A7，第二个操作数为W 3，乘加器PE0在第四个数据处理周期进行的计算为 W 3*A7；其他乘加器的计算过程与乘加器PE0类似，不再赘述。直至卷积核中的所有数据都作为权重值分配完之后，为进行完所有的数据处理周期。
可见在每个数据处理周期，获取不同待处理图像中对应像素点的像素值的PE都完成了对应数据处理周期的计算，也就是说不同待处理图像在每个数据处理周期中的计算被并行完成了，在所有数据处理周期之后，不同待处理图像最终的计算被同时完成，节省了系统资源。
此处，针对不同的待处理图像，对应的卷积核可以不同，也可以相同。例如，若两张待处理图像分别为同一特征图的不同特征子图，则两张待处理图像对应的卷积核不同。若两张待处理图像为同一个特征子图的不同位置的图像数据，则两张待处理图像对应的卷积核相同。示例性的，对于对应于同一待处理图像的多个乘加器，针对不同数据处理周期，可以以MIMD(Multiple Instruction Multiple Data，多指令流多数据流)的方式为每个数据处理周期分配一个权重值。在每个数据处理周期中，可以以SIMD(Single Instruction Multiple Data，单指令流多数据流)的方式为多个乘加器分配对应于该数据处理周期的权重值。
针对上述S103，在确定各乘加器在所有的数据处理周期的第一个操作数和第二个操作数的乘积之后，针对各乘加器中的每个有效乘加器，将该有效乘加器在各个数据处理周期中得到的乘积相加得到和值；从各有效乘加器中，确定每张待处理图像分别对应的有效乘加器；基于每张待处理图像对应的有效乘加器分别得到的和值，确定每张待处理图像的处理结果。
其中，每张待处理图像对应的有效乘加器包括：每一个数据处理周期中，所连接的寄存器都有该张待处理图像的有效像素值输入的乘加器；例如，如图3所示的乘加器阵列中的乘加器PE0～PE195都为有效乘加器。
示例性的，如图3所示的乘加器阵列在对如图7所示的寄存器阵列中的像素值进行处理，在经过所有的数据处理周期的计算之后，每个乘加器将各自在所有数据处理周期计算的计算结果相加求和，例如乘加器PE0进行了四个数据处理周期的计算，计算结果分别为A、B、C、D，则PE0进行如下计算：A+B+C+D。
这里，若进行卷积的特征图包括16个通道，每次处理4个通道分别对应的特征子图，也即需要将16个通道分别对应的特征子图划分为4组，每次进行一组特征子图的处理。若4组特征子图分别为：组a、组b、组c和组d，在对组a中包括的4张特征子图执行了图像处理后，先将乘加器输出的组a对应的4个结果进行累加；在对组b中包括的4张特征子图执行了图像处理后，再将组b对应的4个结果进行累加，并将组a对应的累加结果和组b对应的累加结果进行累加；在对组c中包括的4张特征子图执行了图像处理后，再将组c对应的4个结果进行累加，并将组a、组b的累加结果、和组c对应的累加结果进行累加；在对组d中包括的4张特征子图执行了图像处理后，再将组d对应的4个结果进行累加，并将组a、组b、组c的累加结果、和组d对应的累加结果进行累加，最终，得到16个通道分别对应的卷积结果的累加和。
在对组a中包括的4张特征子图执行了处理后，得到的组a对应的4个输出结果分别为：a1、a2、a3和a4。在对组b中包括的4张特征子图执行了处理后，得到的组b对应的4个输出结果分别为：b1、b2、b3和b4。此时，执行a1+b1＝O1，a2+b2＝O2，a3+b3＝O3，a4+b4＝O4。在对组c中包括的4张特征子图执行了处理后，得到的组c对应的4个输出结果分别为：c1、c2、c3和c4，再执行：O1+c1，O2+c2，O3+c3，O4+c4；以此类推，最终得到a1+b1+c1+d1，a2+b2+c2+d2，a3+b3+c3+d3，a4+b4+c4+d4，然后再将四个结果累加到一起，得到16个通道分别对应的卷积结果的累加和。
对于某一待处理图像，该待处理图像的处理结果可以从图3所示的乘加器阵列中，对应于特定颜色的寄存器的乘加器中获取，该特定颜色为图2中表示该待处理图像的颜色。例如，乘加器PE0、乘加器PE2、乘加器PE4、乘加器PE6、乘加器PE8、乘加器PE10、乘加器乘加器PE12、乘加器PE28、乘加器PE30、乘加器PE32、乘加器PE34、乘加器PE36、乘加器PE38、乘加器PE40、乘加器PE56、乘加器PE58、乘加器PE60、乘加器PE62、乘加器PE64、乘加器PE66、乘加器PE68、乘加器PE84、乘加器PE86、乘加器PE88、乘加器PE90、乘加器PE92、乘加器PE94、乘加器PE96、乘加器PE112、乘加器PE114、乘加器PE116、乘加器EP118、乘加器PE120、乘加器PE122、乘加器PE124、乘加器PE140、乘加器PE142、乘加器PE144、乘加器PE146、乘加器PE148、乘加器PE150、乘加器PE152、乘加器PE168、乘加器PE170、乘加器PE172、乘加器PE174、乘加器PE176、乘加器PE178、乘加器PE180的计算结果为第一待处理图像的处理结果。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
基于同一发明构思，本公开实施例中还提供了与图像处理方法对应的图像处理装置，由于本公开实施例中的装置解决问题的原理与本公开实施例上述图像处理方法相似，因此装置的实施可以参见方法的实施，重复之处不再赘述。
参照图9所示，为本公开实施例提供的一种图像处理装置的示意图，所述装置包括控制器901；控制器901用于：获取多张待处理图像；对所述多张待处理图像进行像素组合处理，得到目标图像；利用乘加器阵列对所述目标图像进行处理，并基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器得到的处理结果，确定每张待处理图像的处理结果。
在一种可能的实施方式中，所述目标图像的同一行中，同一待处理图像的相邻两像素间隔像素数量相同且不为零，以及所述目标图像的同一列中，同一待处理图像的相邻两像素间隔像素数量相同且不为零。
在一种可能的实施方式中，所述控制器901还用于：基于所述乘加器阵列的尺寸信息、以及所述待处理图像的尺寸信息确定所述待处理图像的数量。
在一种可能的实施方式中，尺寸信息包括行数以及列数；在基于所述乘加器阵列的尺寸信息、以及所述待处理图像的尺寸信息确定所述待处理图像的数量时，所述控制器901，具体用于基于所述乘加器阵列的行数以及所述待处理图像的像素数据阵列的行数，确定第一取值，并基于所述乘加器阵列的列数以及所述待处理图像的像素数据阵列的列数，确定第二取值；基于所述第一取值和所述第二取值确定所述待处理图像的数量。
在一种可能的实施方式中，在对多张待处理图像进行像素组合处理，得到目标图像时，所述控制器901，具体用于依次将所述多张待处理图像中的每张待处理图像作为当前图像，确定所述当前图像的首个像素点在所述目标图像中的目标位置；基于所述首个像素点在所述目标图像中的目标位置，确定所述当前图像中除所述首个像素点外的其他像素点在所述目标图像中的目标位置，得到所述目标图像。
在一种可能的实施方式中，在基于所述首个像素点在所述目标图像中的目标位置，确定所述当前图像中除所述首个像素点外的其他像素点在所述目标图像中的目标位置时，所述控制器901，具体用于基于所述乘加器阵列的列数、以及所述待处理图像的像素数据阵列的列数，确定所述当前图像每一行中除该行第一个像素点之外的每个像素 点与该像素点的相邻前一像素点在所述目标图像中的第一位置关系；基于所述乘加器阵列的行数、列数以及所述待处理图像的像素数据阵列的行数、列数，确定所述当前图像每一列中除该列第一个像素点之外的每个像素点与该像素点的相邻前一像素点在所述目标图像中的第二位置关系；基于当前图像首个像素点在所述目标图像中的目标位置、所述第一位置关系和所述第二位置关系，确定所述当前图像中除首个像素点外的其他像素点在所述目标图像中的目标位置。
在一种可能的实施方式中，在确定所述当前图像的首个像素点在所述目标图像中的目标位置时，所述控制器901，具体用于基于所述乘加器阵列的行数和列数、所述待处理图像的像素数据阵列的行数和列数，确定目标像素矩阵；根据所述目标像素矩阵的矩阵元素值，确定各待处理图像的首个像素点在所述目标图像中的目标位置；其中，所述当前图像的首个像素点在所述目标图像中的目标位置为确定的目标位置中的任一目标位置。
在一种可能的实施方式中，在利用乘加器阵列对所述目标图像进行处理时，所述控制器901，具体用于基于所述乘加器阵列与寄存器阵列之间的对应关系，将所述目标图像存储至所述寄存器阵列中；其中，存储至所述寄存器阵列中的所述目标图像各个像素点的相邻关系不变；在多个数据处理周期中的每个数据处理周期，利用所述乘加器阵列从所述寄存器阵列中读取与所述每个数据处理周期对应的像素信息，并对读取的像素信息进行处理，得到所述每个数据处理周期对应的数据处理结果。
在一种可能的实施方式中，在多个数据处理周期中的每个数据处理周期，利用所述乘加器阵列从所述寄存器阵列中读取与所述每个数据处理周期对应的像素信息，并对读取的像素信息进行处理，得到所述每个数据处理周期对应的数据处理结果时，所述控制器901，具体用于在对所述目标图像进行处理的首个数据处理周期，控制所述乘加器阵列中对应寄存器存储有所述目标图像像素信息的各乘加器，从与所述各乘加器连接的寄存器中，读取所述各乘加器在所述首个数据处理周期对应的像素点的像素值作为第一个操作数；并确定所述各乘加器在所述首个数据处理周期对应的权重值，作为第二个操作数；分别确定所述各乘加器在所述首个数据处理周期的第一个操作数和第二个操作数的乘积；针对对所述目标图像进行处理的每个非首个数据处理周期，控制所述目标图像的各像素值按照与该数据处理周期对应的预设数据移动方式在所述寄存器阵列中移动预设步长；并控制所述各乘加器，从与所述各乘加器连接的寄存器中，读取所述各乘加器在该数据处理周期对应的像素点的像素值作为第一个操作数；并确定所述各乘加器在该数据处理周期对应的权重值，作为第二个操作数；分别确定所述各乘加器在该数据处理周期的第一个操作数和第二个操作数的乘积。
在一种可能的实施方式中，在基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器所得到的处理结果，确定每张待处理图像的处理结果时，所述控制器901，具体用于针对所述各乘加器中的每个有效乘加器，将该有效乘加器在各个数据处理周期中得到的乘积相加得到和值；从各有效乘加器中，确定所述每张待处理图像分别对应的有效乘加器；基于所述每张待处理图像对应的有效乘加器分别得到的和值，确定所述每张待处理图像的处理结果。
在一种可能的实施方式中，所述每张待处理图像对应的有效乘加器，包括：每一个数据处理周期所连接的寄存器都有该张待处理图像的有效像素值输入的乘加器。
在一种可能的实施方式中，所述控制器901，还用于基于所述目标图像中属于同一待处理图像的相邻两个像素点之间的距离，确定所述预设步长。
在一种可能的实施方式中，在获取多张待处理图像时，所述控制器901，具体用于获取多张原始图像；将所述多张原始图像中的第一原始图像作为所述待处理图像；和 /或，对所述多张原始图像中的第二原始图像进行数据填充处理，得到与所述第二原始图像对应的待处理图像。
本公开实施例提供的图像处理装置可以包括芯片、AI芯片等。
关于装置中的各模块的处理流程、以及各模块之间的交互流程的描述可以参照上述方法实施例中的相关说明，这里不再详述。
本公开实施例还提供了一种计算机设备，如图10所示，为本公开实施例提供的计算机设备结构示意图，包括控制器1010和存储器1020；所述存储器1020存储有控制器1010可执行的机器可读指令，控制器1010用于执行存储器1020中存储的机器可读指令，所述机器可读指令被控制器1010执行时，控制器1010执行下述步骤：对所述多张待处理图像进行像素组合处理，得到目标图像；利用乘加器阵列对所述目标图像进行处理；基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器得到的处理结果，确定每张待处理图像的处理结果。
上述存储器1020包括内存1021和外部存储器1022；这里的内存1021也称内存储器，用于暂时存放控制器1010中的运算数据，以及与硬盘等外部存储器1022交换的数据，控制器1010通过内存1021与外部存储器1022进行数据交换。
本公开实施例提供的计算机设备可以包括手机等智能终端，或者也可以是具有摄像头并可以进行图像处理的其他设备、服务器等，这里并不限制。
上述指令的具体执行过程可以参考本公开实施例中所述的图像处理方法的步骤，此处不再赘述。
本公开实施例还提供一种计算机可读存储介质，该计算机可读存储介质上存储有计算机程序，该计算机程序被处理器运行时执行上述方法实施例中所述的图像处理方法的步骤。其中，该存储介质可以是易失性或非易失的计算机可读取存储介质。
本公开实施例还提供一种计算机程序产品，该计算机程序产品承载有程序代码，所述程序代码包括的指令可用于执行上述方法实施例中所述的图像处理方法的步骤，具体可参见上述方法实施例，在此不再赘述。其中，上述计算机程序产品可以具体通过硬件、软件或其结合的方式实现。在一个可选实施例中，所述计算机程序产品具体体现为计算机存储介质，在另一个可选实施例中，计算机程序产品具体体现为软件产品，例如软件开发包(Software Development Kit，SDK)等等。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统和装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。在本公开所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，又例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些通信接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个处理器可执行的非易失的计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
最后应说明的是：以上所述实施例，仅为本公开的具体实施方式，用以说明本公开的技术方案，而非对其限制，本公开的保护范围并不局限于此，尽管参照前述实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，其依然可以对前述实施例所记载的技术方案进行修改或可轻易想到变化，或者对其中部分技术特征进行等同替换；而这些修改、变化或者替换，并不使相应技术方案的本质脱离本公开实施例技术方案的精神和范围，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应所述以权利要求的保护范围为准。

Claims (16)

  1. 一种图像处理方法，包括：
    对多张待处理图像进行像素组合处理，得到目标图像；
    利用乘加器阵列对所述目标图像进行处理；
    基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器得到的处理结果，确定每张待处理图像的处理结果。
  2. 根据权利要求1所述的图像处理方法，其特征在于，
    所述目标图像的同一行中，同一待处理图像的相邻两像素间隔像素数量相同且不为零，以及
    所述目标图像的同一列中，同一待处理图像的相邻两像素间隔像素数量相同且不为零。
  3. 根据权利要求1或2所述的图像处理方法，其特征在于，还包括：
    基于所述乘加器阵列的尺寸信息、以及所述待处理图像的尺寸信息确定所述待处理图像的数量。
  4. 根据权利要求3所述的图像处理方法，其特征在于，所述基于所述乘加器阵列的尺寸信息、以及所述待处理图像的像素数据阵列的尺寸信息确定所述待处理图像的数量，包括：
    基于所述乘加器阵列的行数以及所述待处理图像的像素数据阵列的行数，确定第一取值，并
    基于所述乘加器阵列的列数以及所述待处理图像的像素数据阵列的列数，确定第二取值；
    基于所述第一取值和所述第二取值确定所述待处理图像的数量。
  5. 根据权利要求1至4任一项所述的图像处理方法，其特征在于，所述对多张待处理图像进行像素组合处理，得到目标图像，包括：
    依次将所述多张待处理图像中的每张待处理图像作为当前图像，
    确定所述当前图像的首个像素点在所述目标图像中的目标位置；
    基于所述首个像素点在所述目标图像中的目标位置，确定所述当前图像中除所述首个像素点外的其他像素点在所述目标图像中的目标位置。
  6. 根据权利要求5所述的图像处理方法，其特征在于，所述基于所述首个像素点在所述目标图像中的目标位置，确定所述当前图像中除所述首个像素点外的其他像素点在所述目标图像中的目标位置，包括：
    针对当前图像中除所述首个像素点外的每个像素点，
    基于所述乘加器阵列的列数、以及所述待处理图像的像素数据阵列的列数，确定该像素点与该像素点的同行且相邻前一像素点在所述目标图像中的第一位置关系；
    基于所述乘加器阵列的行数、列数以及所述待处理图像的像素数据阵列的行数、列数，确定该像素点与该像素点的同列且相邻前一像素点在所述目标图像中的第二位置关系；
    基于所述当前图像首个像素点在所述目标图像中的目标位置、所述第一位置关系和所述第二位置关系，确定该像素点在所述目标图像中的目标位置。
  7. 根据权利要求5或6所述的图像处理方法，其特征在于，所述确定所述当前图像的首个像素点在所述目标图像中的目标位置，包括：
    基于所述乘加器阵列的行数和列数、所述待处理图像的像素数据阵列的行数和列数，确定目标像素矩阵；
    根据所述目标像素矩阵的矩阵元素值，确定各所述待处理图像的首个像素点在所述目标图像中的目标位置；
    其中，所述当前图像的首个像素点在所述目标图像中的目标位置为确定出的目标位置中的任一目标位置。
  8. 根据权利要求1至7任一项所述的图像处理方法，其特征在于，所述利用乘加器阵列对所述目标图像进行处理，包括：
    基于所述乘加器阵列与寄存器阵列之间的对应关系，将所述目标图像存储至所述寄存器阵列中；其中，存储至所述寄存器阵列中的所述目标图像各个像素点的相邻关系不变；
    在多个数据处理周期中的每个数据处理周期，利用所述乘加器阵列从所述寄存器阵列中读取与所述每个数据处理周期对应的像素信息，并对读取的像素信息进行处理，得到所述每个数据处理周期对应的数据处理结果。
  9. 根据权利要求8所述的图像处理方法，其特征在于，所述在多个数据处理周期中的每个数据处理周期，利用所述乘加器阵列从所述寄存器阵列中读取与所述每个数据处理周期对应的像素信息，并对读取的像素信息进行处理，得到所述每个数据处理周期对应的数据处理结果，包括：
    在对所述目标图像进行处理的首个数据处理周期，
    控制所述乘加器阵列中的各乘加器，从与所述各乘加器连接的寄存器中，读取所述各乘加器在所述首个数据处理周期对应的像素点的像素值作为第一个操作数；
    确定所述各乘加器在所述首个数据处理周期对应的权重值，作为第二个操作数；
    确定所述各乘加器在所述首个数据处理周期的第一个操作数和第二个操作数的乘积；
    针对对所述目标图像进行处理的每个非首个数据处理周期，
    控制所述目标图像的各像素值按照与该数据处理周期对应的预设数据移动方式在所述寄存器阵列中移动预设步长；
    控制所述各乘加器，从与所述各乘加器连接的寄存器中，读取所述各乘加器在该数据处理周期对应的像素点的像素值作为第一个操作数；
    确定所述各乘加器在该数据处理周期对应的权重值，作为第二个操作数；
    确定所述各乘加器在该数据处理周期的第一个操作数和第二个操作数的乘积。
  10. 根据权利要求8或9所述的图像处理方法，其特征在于，所述基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器所得到的处理结果，确定每张待处理图像的处理结果，包括：
    针对所述乘加器阵列中的每个有效乘加器，将该有效乘加器在各个数据处理周期中得到的乘积相加得到和值；
    从各有效乘加器中，确定所述每张待处理图像分别对应的有效乘加器；
    基于所述每张待处理图像对应的有效乘加器分别得到的和值，确定所述每张待处理图像的处理结果。
  11. 根据权利要求10所述的图像处理方法，其特征在于，所述每张待处理图像对应的有效乘加器，包括：每一个数据处理周期所连接的寄存器都有该张待处理图像的有效像素值输入的乘加器。
  12. 根据权利要求9至11任一项所述的图像处理方法，其特征在于，所述图像处理方法还包括：
    基于所述目标图像中属于同一待处理图像的相邻两个像素点之间的距离，确定所述预设步长。
  13. 根据权利要求1至12任一项所述的图像处理方法，其特征在于，所述待处理图像的获取方式，包括以下任意一项或多项：
    获取多张原始图像；
    将所述多张原始图像中的第一原始图像作为所述待处理图像；
    对所述多张原始图像中的第二原始图像进行数据填充处理，得到与所述第二原始图像对应的待处理图像。
  14. 一种图像处理装置，其特征在于，包括：控制器；所述控制器用于：
    获取多张待处理图像；
    对所述多张待处理图像进行像素组合处理，得到目标图像；
    利用乘加器阵列对所述目标图像进行处理，并基于所述多张待处理图像在所述乘加器阵列中分别对应的乘加器得到的处理结果，确定每张待处理图像的处理结果。
  15. 一种计算机设备，其特征在于，包括：控制器、存储器，所述存储器存储有所述控制器可执行的机器可读指令，所述控制器用于执行所述存储器中存储的机器可读指令，所述机器可读指令被所述控制器执行时，所述控制器执行如权利要求1至13任一项所述的图像处理方法的步骤。
  16. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被计算机设备运行时，所述计算机设备执行如权利要求1至13任一项所述的图像处理方法的步骤。
PCT/CN2021/115745 2021-01-31 2021-08-31 一种图像处理方法、装置、计算机设备及存储介质 WO2022160704A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110132579.7 2021-01-31
CN202110132579.7A CN112967211A (zh) 2021-01-31 2021-01-31 一种图像处理方法、装置、计算机设备及存储介质

Publications (1)

Publication Number Publication Date
WO2022160704A1 true WO2022160704A1 (zh) 2022-08-04

Family

ID=76272538

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/115745 WO2022160704A1 (zh) 2021-01-31 2021-08-31 一种图像处理方法、装置、计算机设备及存储介质

Country Status (2)

Country Link
CN (1) CN112967211A (zh)
WO (1) WO2022160704A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112967211A (zh) * 2021-01-31 2021-06-15 成都商汤科技有限公司 一种图像处理方法、装置、计算机设备及存储介质
CN113469041A (zh) * 2021-06-30 2021-10-01 北京市商汤科技开发有限公司 一种图像处理方法、装置、计算机设备和存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105260773A (zh) * 2015-09-18 2016-01-20 华为技术有限公司 一种图像处理装置以及图像处理方法
US20180005074A1 (en) * 2016-07-01 2018-01-04 Google Inc. Convolutional Neural Network On Programmable Two Dimensional Image Processor
CN108288252A (zh) * 2018-02-13 2018-07-17 北京旷视科技有限公司 图像批处理方法、装置及电子设备
CN111597029A (zh) * 2020-05-20 2020-08-28 上海商汤智能科技有限公司 数据处理方法及装置、电子设备和存储介质
CN111814983A (zh) * 2020-03-04 2020-10-23 深圳芯英科技有限公司 数据处理方法、装置、芯片以及计算机可读存储介质
CN112967211A (zh) * 2021-01-31 2021-06-15 成都商汤科技有限公司 一种图像处理方法、装置、计算机设备及存储介质

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111897579B (zh) * 2020-08-18 2024-01-30 腾讯科技（深圳）有限公司 图像数据处理方法、装置、计算机设备和存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105260773A (zh) * 2015-09-18 2016-01-20 华为技术有限公司 一种图像处理装置以及图像处理方法
US20180005074A1 (en) * 2016-07-01 2018-01-04 Google Inc. Convolutional Neural Network On Programmable Two Dimensional Image Processor
CN108288252A (zh) * 2018-02-13 2018-07-17 北京旷视科技有限公司 图像批处理方法、装置及电子设备
CN111814983A (zh) * 2020-03-04 2020-10-23 深圳芯英科技有限公司 数据处理方法、装置、芯片以及计算机可读存储介质
CN111597029A (zh) * 2020-05-20 2020-08-28 上海商汤智能科技有限公司 数据处理方法及装置、电子设备和存储介质
CN112967211A (zh) * 2021-01-31 2021-06-15 成都商汤科技有限公司 一种图像处理方法、装置、计算机设备及存储介质

Also Published As

Publication number Publication date
CN112967211A (zh) 2021-06-15

Similar Documents

Publication Publication Date Title
US10943167B1 (en) Restructuring a multi-dimensional array
US20230325348A1 (en) Performing concurrent operations in a processing element
JP6764954B2 (ja) プログラマブル二次元画像プロセッサ上の畳み込みニューラルネットワーク
CN110050267B (zh) 用于数据管理的系统和方法
WO2022160704A1 (zh) 一种图像处理方法、装置、计算机设备及存储介质
US20230306249A1 (en) Transposed convolution using systolic array
EP3093757B1 (en) Multi-dimensional sliding window operation for a vector processor
JP2022171926A (ja) 畳み込みニューラルネットワークのための効率的データレイアウト
US10642622B2 (en) Arithmetic processing device and control method of the arithmetic processing device
CN109844738A (zh) 运算处理电路和识别系统
TW201824096A (zh) 執行卷積計算的系統及方法
KR102126857B1 (ko) 행 단위 연산 뉴럴 프로세서 및 이를 이용한 데이터 처리 방법
CN110188869B (zh) 一种基于卷积神经网络算法的集成电路加速计算的方法及系统
CN110796236B (zh) 多样本多通道卷积神经网络池化的向量化实现方法
US20210019606A1 (en) Cellular neural network integrated circuit having multiple convolution layers of duplicate weights
EP3938950A1 (en) Spatially sparse convolutional neural networks for inking applications
US11164032B2 (en) Method of performing data processing operation
JP2020204894A (ja) フィルタ処理装置及びその制御方法
WO2022160706A1 (zh) 一种数据处理方法、装置、计算机设备及存储介质
CN109416743B (zh) 一种用于识别人为动作的三维卷积装置
CN114677286A (zh) 一种图像处理方法、装置、存储介质及终端设备
CN113657587B (zh) 基于fpga的可变形卷积加速方法及装置
Geng et al. MacSim: a MAC-enabled high-performance low-power SIMD architecture
CN110163793B (zh) 卷积计算加速方法和装置
KR20220083820A (ko) 뉴럴 네트워크 프로세서에서의 3차원 콘볼루션

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21922299

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21922299

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 18.01.2024)