WO2021018199A1 - 一种基于卷积神经网络的图像处理方法及装置 - Google Patents

一种基于卷积神经网络的图像处理方法及装置 Download PDF

Info

Publication number
WO2021018199A1
WO2021018199A1 PCT/CN2020/105537 CN2020105537W WO2021018199A1 WO 2021018199 A1 WO2021018199 A1 WO 2021018199A1 CN 2020105537 W CN2020105537 W CN 2020105537W WO 2021018199 A1 WO2021018199 A1 WO 2021018199A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
sub
convolution
information
convolution operation
Prior art date
Application number
PCT/CN2020/105537
Other languages
English (en)
French (fr)
Inventor
李锐
张磊
李敏丽
贺秋丽
邓禹丹
李国亮
杨勤富
Original Assignee
瀚博半导体(上海)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 瀚博半导体(上海)有限公司 filed Critical 瀚博半导体(上海)有限公司
Publication of WO2021018199A1 publication Critical patent/WO2021018199A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • This application relates to the field of image processing, and in particular to a technology of image processing based on convolutional neural networks.
  • Convolutional Neural Network As a kind of artificial neural network, has become a research hotspot in the field of image processing.
  • Convolutional neural network is a multi-layer neural network, each layer is composed of multiple three-dimensional planes, and each plane is convolved by a different convolution kernel.
  • Each neural network layer of a convolutional neural network generates a large amount of data. The output of the upper layer becomes the input of the next layer. Since a neural network can have multiple layers, the amount of convolution operations of a convolutional neural network is very large, and each layer of the network requires a convolution operation, and an image processing requires a multi-layer convolution kernel and a multi-plane convolution.
  • the data generated between each neural network layer needs to be cached by the external memory of the convolutional neural network processor (such as GDDR or DDR or HBM memory); that is, in the process of performing convolutional layer operations on the image to be recognized,
  • the data between each convolutional layer needs to be output to an external memory, and then read into the processor.
  • Each neural network layer generates a large amount of data, and the bandwidth requirements for the processor's external memory are very high. Even if the bandwidth of the processor's external memory is large enough, the performance of the convolutional neural network system may be affected by too much data flow of convolution operation.
  • One purpose of this application is to provide an image processing method and device based on a convolutional neural network to solve the problems of reducing the bandwidth of the external memory of the convolutional neural network processor and optimizing the performance of the convolutional neural network processor during image processing. .
  • an image processing method based on a convolutional neural network including:
  • convolution operation is performed on each sub-image after cutting independently and sequentially;
  • the convolution result does not need to be cut again according to the preset image processing strategy information, perform the remaining convolution operation, and obtain the final convolution operation result;
  • the final convolution operation result is output to an external storage unit.
  • an image processing device based on a convolutional neural network including:
  • the image cutting unit is used for cutting the read image to be processed into multiple sub-images according to the number of sub-images corresponding to this cutting in the preset image processing strategy and the size-related information;
  • the sub-image convolution unit is configured to independently and sequentially perform convolution operations on each sub-image after cutting according to the convolution operation strategy corresponding to the sub-image in the preset image processing strategy;
  • the sub-image aggregation unit is configured to aggregate the convolution operation results of the sub-images at the corresponding aggregation point according to the sub-image aggregation point information in the preset image processing strategy;
  • the remaining convolution operation execution unit is configured to perform the remaining convolution operation and obtain the final convolution operation result when the convolution result does not need to be cut again according to the preset image processing strategy information;
  • the image processing result output unit outputs the final convolution operation result to the external storage unit.
  • the method and device described in this application can change the number of sub-images corresponding to this cut in the preset image processing strategy and the related information about their size, and the convolution operation strategy corresponding to the sub-images.
  • the read image to be processed is cut into multiple sub-images, and each sub-image after the cut is independently and sequentially convolved; further according to the sub-image convergence point information in the preset image processing strategy, the corresponding convergence point is collected
  • the convolution operation result of each sub-image here, it is necessary to determine whether the convolution operation result after converging needs to continue cutting according to the preset image processing strategy information, and if necessary, the convolution operation result is output to the external storage unit.
  • Fig. 1 is a flowchart of an image processing method based on a convolutional neural network according to an embodiment of the present application
  • Fig. 2 is a schematic diagram of determining boundary information of a to-be-processed image according to an embodiment of the present application
  • Fig. 3 is a schematic diagram of a convolutional network structure according to an embodiment of the present application.
  • Fig. 4 shows a schematic diagram of identifying the edge pixel value of a sub-image of an image to be processed according to an embodiment of the present application
  • Fig. 5 shows a schematic block diagram of an image processing device based on a convolutional neural network according to an embodiment of the present application.
  • the image processing technology based on convolutional neural network includes but is not limited to image processing methods such as image recognition, image segmentation, image classification, image optimization, and image enhancement.
  • image processing methods such as image recognition, image segmentation, image classification, image optimization, and image enhancement.
  • the image processed by the convolutional neural network is relatively large. Take the 1920x1080 pixel value image as the input of the convolutional neural network and the output channel size is 32 as an example, then the output data size of each convolutional layer can be obtained It is 1920x1080x32 or 132,710,400 bytes (2 bytes per pixel).
  • the storage processing space of 132,710,400 bytes (or 132M bytes) is very expensive to implement for the internal storage unit of the image processing apparatus. This application solves this problem by image cutting. Specifically, as shown in FIG.
  • the image to be processed is cut into 4 sub-images, so that the size of the intermediate data of image processing can be suitable for the internal memory of the image processing device.
  • the image to be processed is cut into 4 sub-images, so that the size of the intermediate data of image processing can be suitable for the internal memory of the image processing device.
  • the image size of 1920x1080x32 if the image is not cut, 132M bytes are required; in the example shown in Figure 2, only a quarter or 33M bytes (including adjacent sub- Corresponding data content of overlapping parts between images).
  • the deeper the network layer of the convolutional neural network the smaller the output data volume. This is because the image size is greatly reduced in the subsequent layers of the machine learning network, and even if the output channel size increases, in the subsequent layers, the total amount of data generated by the layer and therefore the buffer requirements are greatly reduced.
  • the image processing method and device based on the convolutional network provided by this application can ensure that the convolution is in the middle of the convolution operation by cutting, convolution operation processing, aggregation, and residual convolution operation processing on the image to be processed.
  • the data can be stored in the internal storage unit of the image processing device.
  • the convolution operation result needs to be output to the external storage unit, and then read into the device to execute the corresponding image Cutting, converging, etc.; thus, it can effectively reduce the bandwidth requirements of the external storage unit of the image processing device, and significantly improve the overall performance of the image processing device and the system.
  • a method for image processing based on a convolutional neural network includes the following steps:
  • S1 cuts the read image to be processed into multiple sub-images according to the number of sub-images corresponding to this cutting in the preset image processing strategy and the size-related information.
  • the image processing strategy includes all strategy information in the process of performing convolutional neural network processing for a certain image to be processed, for example, information about the convolutional neural network that needs to be executed for the image to be processed, and the number of times the image to be processed is cut , Each cut into the size and quantity information of the sub-image, the convolutional layer operation information that the sub-image needs to be executed after each cut, the convolutional layer operation information of the sub-image convolution result, and the convolutional layer operation information that the aggregated image needs to perform and many more.
  • the image to be processed may be image information input after preprocessing after collection by related equipment, or image information directly read from an external buffer.
  • the read image to be processed is cut into multiple sub-images, and the image to be processed should be cut in units of pixels.
  • the boundaries of the sub-images include the boundaries of the image to be processed and the boundaries adjacent to other sub-images.
  • the boundaries between the sub-images are specifically based on the sub-images generated during this cutting in the image processing strategy.
  • the number and size of the images are determined. More specifically, it can be determined by first determining the column boundary and then determining the row boundary, but the specific determination method is not limited.
  • the image to be processed is divided into multiple sub-images.
  • the image processing strategy includes all strategy information in the process of performing convolutional neural network operations for a certain image to be processed, for example, information about the convolutional neural network that needs to be executed for the image to be processed , The number of times the image to be processed is cut, the size and quantity of the sub-images are cut each time, the convolutional layer calculation information that the sub-images need to perform after each cut, the location information of the sub-image convolution results, and the aggregated images need to be executed Convolutional layer operation information and so on.
  • the convolution operations are performed on each sub-image after cutting independently and sequentially.
  • all convolutional layer operation information that needs to be performed on the sub-image after the cutting is determined, and then the above-mentioned operation is performed independently on each sub-image after the cutting.
  • All convolution layers are operated, and the convolution operation result corresponding to each sub-image is obtained.
  • the image to be processed is cut into four sub-images: sub-image 310, sub-image 320, sub-image 330, and sub-image 340.
  • the convolution operations that need to be performed are shown in Fig. 3 for convolution layers 1 to Convolutional layer 3 (including convolutional layers 3.1/3.2/3.3 three parallel convolutional layers), at this time, sub-image 310 needs to be executed from convolutional layer 1 to convolutional layer 3, and sub-image 320 is performed from convolution Layer 1 to convolutional layer 3, sub-image 330 execution from convolutional layer 1 to convolutional layer 3, and sub-image 340 execution from convolutional layer 1 to convolutional layer 3, respectively and independently executed in sequence.
  • the S2 includes:
  • the convolutional layer operation information that the sub-image should execute independently according to preset convolution operation strategy information. For example, if the total number of convolution operation layers is 30, one cut, the convergence point is the convolution operation. After the convolution operation is performed at the 26th level, the convolution operation that needs to be performed on the sub-image after the image is cut is the first level. To the 26th floor.
  • S22 independently and sequentially executes the convolutional layer operation on each sub-image after cutting, and obtains the convolution operation result of each sub-image.
  • the cut sub-images are independent of each other when performing the corresponding convolution operation. According to the above example, if the image is cut into two sub-images a and b, then the cut sub-image a performs the first layer to After the 26th convolution operation, the sub-image b performs the convolution operation from the first layer to the 26th layer again, and the sub-images are executed independently and sequentially.
  • the preset image processing strategy information includes the convergence point information of the sub-images.
  • the convolutional layer operation results of the sub-images are gathered and spliced together.
  • the convolution result can be directly spliced to obtain the image processing result after the convolution operation in S2.
  • the sub-image convolution operation result is performed after all sub-images are independently executed from convolution layer 1 to convolution layer 3 (including convolution layer 3.1/3.2/3.3 three parallel convolution layers) Converge.
  • the image processing strategy information includes information about the number of times the image processing needs to be cut. If the intermediate convolution result needs to be cut again, the convolution operation result must be output to the external storage unit of the image processing device. If the intermediate convolution result does not need to be cut again, at this time, only the remaining convolution operation needs to be performed on the basis of the convolution operation result to obtain the final convolution operation result, that is, the image processing result.
  • the S4 includes:
  • S41 determines, according to preset image processing strategy information, remaining convolution operation information that needs to be executed for the converged convolution result.
  • the remaining convolution operation information here refers to the situation where the convolution result after the aggregation does not need to be cut again, and only the corresponding convolution operation is required to complete the image processing.
  • the image processing preset information includes the The image needs to be cut once, and a total of 50 layers of convolutional layers need to be executed. The convergence point is where the execution of the 39th layer is completed, then the remaining convolutional layers can be deduced from the 40th layer in the convolution operation to the last 11 of the 50th layer. Floor.
  • the image processing preset information includes that the image needs to be cut twice, a total of 60 layers of convolutional layers need to be executed, and the convergence point is where the execution of the 39th layer is completed and the execution of the 59th layer is completed, then the remaining volumes can be pushed out in turn
  • the build-up layer is the 60th layer and 1 layer in the convolution operation.
  • S42 (not shown) performs the remaining convolution operation on the aggregated convolution result, and obtains the final convolution operation result.
  • the convolution operation result here is the final result of image processing, and the result is output to the external storage unit of the image processing device.
  • the method before S1, the method further includes:
  • the method of acquiring the image to be processed is not limited.
  • the image collected by the image acquisition device can be read into the image processing system of this application; it can also be directly read the existing image information in the storage space, etc.
  • the storage space here is also not limited, for example, it can be a common ROM/RAM type memory, it can also be a cloud disk, communication software network storage space, etc.
  • After acquiring the image to be processed it needs to be preprocessed, such as , Image size, brightness, etc. processing.
  • the method further includes S6 (not shown):
  • the image processing strategy is preset according to the image information to be processed, the size of the internal storage space, and the convolution operation strategy information that the image to be processed needs to perform; wherein the image processing strategy information includes image cutting strategy information and convergence point information.
  • the aforementioned cutting strategy for image processing includes all strategy information in the process of performing convolutional neural network operations on the image to be processed, so when setting, the known information includes the size of the image to be processed and the requirements for the image to be processed.
  • All executed convolutional neural network layers, image processing processor memory size, etc. also need to meet the memory size of the image processing processor must meet the independent processing space of the sub-image; determine on the basis of the above known information and requirements The number of times the image to be processed is cut, the size and number of sub-images each cut, the number of convolutional layers that need to be executed for each sub-image cut, and so on.
  • the convergence point of the image processing strategy is preset, it can be considered whether the size of the image after convergence is suitable Factors such as the size of the internal storage unit of the image processing device.
  • the S5 when the convergence point information is known, the S5 includes:
  • S52 compares the estimation result with the size of the internal storage space of the image processing device.
  • the internal storage space of the image processing device is known. Just compare the two directly.
  • the estimated result here should not be too close to the internal storage space, because the result is estimated, not the actual size Value, you can generally consider leaving some extra space to ensure that the actual execution process can proceed smoothly.
  • S53 determines whether the convolution result needs to be cut again according to the above comparison result. Specifically, if the estimated result is less than the internal storage space of the image processing device, it means that the convolution operation can be continued without cutting again based on the result. Otherwise, the image cutting strategy needs to be set to increase the number of image cutting times.
  • the method further includes:
  • S7 outputs the convolution operation result of the aggregated sub-images to the external storage unit.
  • the aggregated convolution far calculation result is relatively large, it is not suitable for completing the remaining convolution operation in the internal storage unit of the image processing device, and it needs to be cut again to reduce the data size of each convolution operation.
  • the aggregated convolution operation result needs to be cut again, it must be output to the external storage unit of the image processing device first, and then read in from the external storage unit before executing subsequent image processing strategies.
  • the method further includes after S7:
  • S8 reads the convolution operation result of each sub-image output to the external storage unit.
  • this embodiment is mainly aimed at the situation where the convolution operation performed on the sub-images of the image to be processed has not been completed.
  • the preset strategy image needs to execute all convolutional layers. At this time, it needs to be output to the external storage unit of the image processing device.
  • the puzzle information is re-read into the image processing processor, and the image convolution processing is continued.
  • the S1 includes:
  • the number of sub-images corresponding to the cut and the size-related information, the read convolution operation result is cut into multiple sub-images again.
  • this embodiment is mainly aimed at the situation where the image processing strategy of the image to be processed includes multiple image cuts.
  • the puzzle information read from the external memory needs to be used as the image to be processed, and the corresponding image processing strategy is related
  • the information continues to be divided into multiple sub-images and corresponding convolution operations are performed, and the information is aggregated according to the strategy information.
  • the image cutting processing strategy includes cutting the image to be processed into the number of sub-images and related information about their size, and information related to the convolution operation performed on the sub-images after the cutting, and the S1 includes:
  • S11 (not shown) determining the first boundary information of each sub-image according to the number and size information of the divided image to be processed into sub-images;
  • S12 (not shown) Determine the second boundary of each sub-image in the image to be processed according to the size of the convolution kernel in the convolution operation information performed on the cut sub-images and the first boundary information of each sub-image information;
  • S13 sequentially cuts the image according to the second boundary information of the sub-images in the image to be processed.
  • the first boundary information is actual boundary information between the sub-image and other sub-images, that is, boundary information that effectively identifies the content of the sub-image.
  • the actual boundary between the sub-image 310 and the sub-image 320 is the solid line 400 in the figure
  • the actual boundary between the sub-image 310 and the sub-image 330 is the solid line 600 in the figure.
  • step S12 since the convolution operation with a convolution kernel size of NxN requires adjacent NxN data and corresponding weights to be multiplied and the results are accumulated together, it is necessary to reserve N-1 between the two sub-images The overlapping part of the rows/columns in order to fully identify the image information of each sub-image. As shown in Fig.
  • the first boundary between the sub-image 310 and the sub-image 320 is the solid line 400, and the size of the convolution kernel to be executed for the sub-image is 5x5; at this time, to ensure that the part corresponding to the most edge pixel 311 in the sub-image 310 is effectively identified, it is necessary to reserve 4 columns of overlap between the sub-image 310 and the sub-image 320 to implement the actual convolution operation process
  • the sub-image 310 is fully recognized.
  • the extended boundary of the first boundary solid line 400 is the second boundary dashed line 500, which is the cutting boundary of the sub-image 310. Only by cutting this boundary can the sub-image 310 The pixel 311 is effectively recognized.
  • step S13 referring to FIG. 2 in conjunction with FIG. 4, the image to be processed is cut into 4 sub-images.
  • the actual boundary between the sub-images, the solid line 400 and the solid line 600 are both next to the two sub-images.
  • the cutting boundary of the sub-image that is, the second boundary
  • the second boundary 500 of the sub-image 310 has been extended into the sub-image 320, so the cutting
  • each cutting should be performed on the complete image to be recognized.
  • the S12 includes:
  • the convolutional neural network layer required for the factor image is very complex, including parallel convolution kernels and cascaded convolution kernels. To ensure that the sub-images can be fully recognized, all convolutions need to be considered.
  • the size information of the core as shown in Figure 3.
  • the operation result of the previous convolutional layer (convolutional layer 1) is the input of the operation of the next convolutional layer (convolutional layer 2).
  • the parallel convolutional kernels can include multiple Convolutional layers (Convolutional layer 3.1, Convolutional layer 3.2, Convolutional layer 3.3).
  • the edge extension pixel value of the sub-image needs to satisfy the convolution kernel sizes of all cascaded convolutional layers and all parallel convolution kernel sizes at the same time, that is, the sum of the two can satisfy the complete recognition of the sub-image.
  • the size of the extended pixel value is:
  • the kernel sizes of the convolutional layers 3.1, 3.2, and 3.3 are 3 ⁇ 3, 1 ⁇ 1, and 5 ⁇ 5, respectively.
  • convolution size 1x1 does not need to be adjacent Pixels.
  • the sub-images of the image to be recognized perform all the convolutional layers 1, 2 and 3.1/3.2/3.3 respectively and converge after the convolutional layer 3.1/3.2/3.3.
  • the extended pixel value of each sub-image is the extension required for convolutional layer 1 (2 pixels), convolutional layer 2 (2 pixels), and convolutional layer 3.1, 3.2, and 3.3 (4 pixels).
  • the sum of the pixels in the adjacent sub-image range, where the pixel value to be extended is 8.
  • S122 determines the second boundary information of each sub-image according to the first boundary information of each sub-image and the corresponding extended pixel value information. Specifically, based on the first boundary of each sub-image, the corresponding second boundary information can be determined by extending the corresponding pixel value to the range of the next sub-image.
  • the method before S3, the method further includes:
  • the operation result of the non-maximum size convolution kernel in the parallel convolution kernel with respect to the pixels in the extension area is filtered out by the provided filter circuit module.
  • the extended pixels of the sub-image shown are not the actual image content of the sub-image, when performing convolutional layer operations on the sub-image, it does not care whether the image content of the extended pixels needs to be recognized, or whether it is not. Need to be recognized, because the part of the content will be effectively recognized in other sub-images, so as not to repeat the need to filter it out.
  • the extended pixel value is determined for the parallel convolution kernel part, only the largest size of the convolution kernel is considered, that is, the pixel identified by the largest size convolution kernel operation must belong to the effective recognition range of the sub-image, so The convolution result of the largest size in the parallel convolution kernel needs to be retained, and the convolution operation results of other sizes in the parallel convolution kernel for pixels in the extended area should be filtered out.
  • the image processing method based on convolutional neural network described in this application can be based on the number of sub-images corresponding to this cut in the preset image processing strategy and related information about their sizes, as well as the corresponding information of the sub-images.
  • Convolution operation strategy the read image to be processed is cut into multiple sub-images, and each sub-image after the cutting is independently and sequentially convolved; further according to the sub-image convergence point information in the preset image processing strategy, the corresponding The converging point of, converges the convolution operation results of the sub-images; here, it is necessary to determine whether the convolution operation results after converging need to be continuously cut according to the preset image processing strategy information, and if necessary, the convolution operation results After outputting to an external storage unit, it is read into the device, and the steps of image cutting, sub-image convolution operation, and aggregation in the above image processing method are executed, until the convolution operation result after aggregation does not need to be cut again; The remaining convolution operation is performed, and the final convolution operation result is obtained; the final convolution operation result is output to the external storage unit.
  • the present application also provides an image processing device based on a convolutional neural network.
  • the device in this application includes, but is not limited to, the image processing chip based on the convolutional neural network in this application.
  • the image cutting unit 110 is configured to cut the read image to be processed into multiple sub-images according to the number of sub-images corresponding to this cutting in the preset image processing strategy and the size-related information.
  • the image processing strategy includes all strategy information in the process of performing convolutional neural network operations for a certain to-be-processed image, for example, the convolutional neural network information that the to-be-processed image needs to be executed, and the to-be-processed image cutting Number of times, each cut into the size and quantity information of the sub-image, the convolutional layer operation information that the sub-image needs to perform after each cut, the convolutional layer operation information of the sub-image convolution result, and the convolutional layer operation that the converged image needs to perform Information and so on.
  • the image to be processed may be image information input after preprocessing after collection by related equipment, or image information directly read from an external buffer.
  • the read image to be processed is cut into multiple sub-images, and the image to be processed should be cut in units of pixels.
  • the boundaries of the sub-images include the boundaries of the image to be processed and the boundaries adjacent to other sub-images.
  • the boundaries between the sub-images are specifically based on the sub-images generated during this cutting in the image processing strategy.
  • the number and size of the images are determined. More specifically, it can be determined by first determining the column boundary and then determining the row boundary, but the specific determination method is not limited.
  • the image to be processed is divided into multiple sub-images.
  • the sub-image convolution unit 120 is configured to independently and sequentially perform convolution operations on each sub-image after cutting according to information related to the convolution operation strategy corresponding to the sub-image in the preset image processing strategy.
  • the image processing strategy includes all strategy information in the process of performing a convolutional neural network operation for a certain image to be processed, for example, information about the convolutional neural network that needs to be executed for the image to be processed , The number of times the image to be processed is cut, the size and quantity of the sub-images are cut each time, the convolutional layer operation information that the sub-images need to perform after each cut, the location information of the sub-image convolution results, and the aggregated images need to be executed Convolutional layer operation information and so on.
  • the convolution operations are performed on each sub-image after cutting independently and sequentially.
  • all convolutional layer operation information that needs to be performed on the sub-image after the cutting is determined, and then the above-mentioned operation is performed independently on each sub-image after the cutting.
  • All convolution layers are operated, and the convolution operation result corresponding to each sub-image is obtained.
  • the image to be processed is cut into four sub-images: sub-image 310, sub-image 320, sub-image 330, and sub-image 340.
  • the convolution operations that need to be performed are shown in Fig. 3 for convolution layers 1 to Convolutional layer 3 (including convolutional layers 3.1/3.2/3.3 three parallel convolutional layers), at this time, sub-image 310 needs to be executed from convolutional layer 1 to convolutional layer 3, and sub-image 320 is performed from convolution Layer 1 to convolutional layer 3, sub-image 330 execution from convolutional layer 1 to convolutional layer 3, and sub-image 340 execution from convolutional layer 1 to convolutional layer 3, respectively and independently executed in sequence.
  • the sub-image convolution unit 120 includes:
  • the sub-image execution convolutional layer operation determining module 121 (not shown) is configured to determine the convolutional layer operation information performed by the sub-image after cutting according to the convolution operation strategy information in the image cutting processing strategy.
  • the convolutional layer operation information that the sub-image should execute independently according to preset convolution operation strategy information. For example, if the total number of convolution operation layers is 30, one cut, the convergence point is the convolution operation. After the convolution operation is performed at the 26th level, the convolution operation that needs to be performed on the sub-image after the image is cut is the first level. To the 26th floor.
  • the sub-image convolution layer operation execution module 122 (not shown) is configured to independently and sequentially execute the convolution layer operation on each sub-image after cutting, and obtain the convolution operation result of each sub-image.
  • the cut sub-images are independent of each other when performing the corresponding convolution operation. According to the above example, if the image is cut into two sub-images a and b, then the cut sub-image a performs the first layer to After the 26th convolution operation, the sub-image b performs the convolution operation from the first layer to the 26th layer again, and the sub-images are executed independently and sequentially.
  • the sub-image aggregation unit 130 is configured to aggregate the convolution operation results of the sub-images at the corresponding convergence point according to the sub-image convergence point information in the preset image processing strategy.
  • the preset image processing strategy information includes the convergence point information of the sub-images.
  • the convolutional layer operation results of the sub-images are aggregated and spliced together.
  • the convolution result can be directly spliced to obtain the image processing result after the convolution operation in S2.
  • the sub-image convolution operation result is performed after all sub-images are independently executed from convolution layer 1 to convolution layer 3 (including convolution layer 3.1/3.2/3.3 three parallel convolution layers) Converge.
  • the remaining convolution operation execution unit 140 is configured to perform the remaining convolution operation and obtain the final convolution operation result under the condition that the convolution result does not need to be cut again according to the preset image processing strategy information.
  • the image processing strategy information includes information about the number of times that the image processing needs to be cut. If the intermediate convolution result needs to be cut again, the convolution operation result must be output to the external storage unit of the image processing device If the intermediate convolution result does not need to be cut again, at this time, only the remaining convolution operation needs to be performed on the basis of the convolution operation result to obtain the final convolution operation result, that is, the image processing result.
  • the remaining convolution operation execution unit 140 includes:
  • the remaining convolution operation determining module 141 (not shown) is configured to determine, according to preset image processing strategy information, the remaining convolution operation information that needs to be executed for the aggregated convolution result.
  • the remaining convolution operation information here refers to the situation where the convolution result after the aggregation does not need to be cut again, and only the corresponding convolution operation is required to complete the image processing.
  • the image processing preset information includes the The image needs to be cut once, and a total of 50 layers of convolutional layers need to be executed. The convergence point is where the execution of the 39th layer is completed, then the remaining convolutional layers can be deduced from the 40th layer in the convolution operation to the last 11 of the 50th layer. Floor.
  • the image processing preset information includes that the image needs to be cut twice, a total of 60 layers of convolutional layers need to be executed, and the convergence point is where the execution of the 39th layer is completed and the execution of the 59th layer is completed, then the remaining volumes can be pushed out in turn
  • the build-up layer is the 60th layer and 1 layer in the convolution operation.
  • the remaining convolution execution module 142 (not shown) is configured to execute the remaining convolution operation on the aggregated convolution result, and obtain the final convolution operation result.
  • the image processing result output unit 150 is configured to output the final convolution operation result to an external storage unit.
  • the convolution operation result here is the final result of image processing, and the result is output to the external storage unit of the image processing device.
  • the device further includes:
  • the to-be-processed image acquisition unit is used to acquire the to-be-processed image information and preprocess the to-be-processed image.
  • the method of acquiring the image to be processed is not limited.
  • the image collected by the image acquisition device can be read into the image processing system of this application; it can also be directly read the existing image information in the storage space, etc.
  • the storage space here is also not limited, for example, it can be a common ROM/RAM type memory, it can also be a cloud disk, communication software network storage space, etc.
  • After acquiring the image to be processed it needs to be preprocessed, such as , Image size, brightness, etc. processing.
  • the device further includes:
  • the image processing strategy presetting unit is used to preset the image processing strategy according to the image information to be processed, the size of the internal storage space, and the convolution operation strategy information to be performed on the image to be processed; wherein the image processing strategy information includes the image cutting strategy Information, convergence point information.
  • the aforementioned cutting strategy for image processing includes all strategy information in the process of performing convolutional neural network operations on the image to be processed, so when setting, the known information includes the size of the image to be processed and the requirements for the image to be processed.
  • All the executed convolutional neural network layers, image processing processor memory size, etc. also need to meet the memory size of the image processing device, which must meet the independent processing space of the sub-image; determine the waiting room based on the above known information and the requirements that need to be met. Process the number of times the image is cut, the size and number of sub-images each cut, the number of convolutional layers that need to be executed for each sub-image cut, etc.
  • the convergence point of the image processing strategy is preset, it can be considered whether the size of the image after convergence is suitable Factors such as the size of the internal storage unit of the image processing device.
  • the image processing strategy presetting unit when the convergence point information is known, includes:
  • the convergence point convolution calculation result estimation module is used to estimate the size of the convolution calculation result gathered at the convergence point.
  • the estimated value and memory size comparison module is used to compare the estimated result with the size of the internal storage space of the image processing device.
  • the internal storage space of the image processing device is known. Just compare the two directly.
  • the estimated result here should not be too close to the internal storage space, because the result is estimated, not the actual size Value, you can generally consider leaving some extra space to ensure that the actual execution process can proceed smoothly.
  • the convolution result cut again determining module is used to determine whether the convolution result needs to be cut again according to the above comparison result. Specifically, if the estimated result is less than the internal storage space of the image processing device, it means that based on the result, the convolution operation can be continued without cutting again; otherwise, it is necessary to set the image cutting strategy to increase the number of image cutting times.
  • the device further includes:
  • the intermediate convolution result output unit is used to output the convolution operation result of each sub-image of the aggregation to an external storage unit.
  • the aggregated convolution far calculation result is relatively large, it is not suitable for completing the remaining convolution operation in the internal storage unit of the image processing device, and it needs to be cut again to reduce the data size of each convolution operation. .
  • the aggregated convolution operation result needs to be cut again, it must be output to the external storage unit of the image processing device first, and then read in from the external storage unit before executing subsequent image processing strategies.
  • the device further includes:
  • the intermediate convolution result reading unit is used to read the convolution operation result of each sub-image output to the external storage unit.
  • this embodiment is mainly aimed at the situation where the convolution operation performed on the sub-images of the image to be processed has not been completed.
  • the preset strategy image needs to execute all convolutional layers. At this time, it needs to be output to the external storage unit of the image processing device.
  • the puzzle information is re-read into the image processing processor, and the image convolution processing is continued.
  • the image cutting unit is used for:
  • the number of sub-images corresponding to the cut and the size-related information, the read convolution operation result is cut into multiple sub-images again.
  • this embodiment is mainly aimed at the situation where the image processing strategy of the image to be processed includes multiple image cuts.
  • the puzzle information read from the external memory needs to be used as the image to be processed, and the corresponding image processing strategy is related
  • the information continues to be divided into multiple sub-images and corresponding convolution operations are performed, and the information is aggregated according to the strategy information.
  • the image cutting processing strategy includes cutting the image to be processed into the number of sub-images and information related to their size, and information related to the convolution operation performed on the sub-images after the cutting, and the image cutting unit includes :
  • a sub-image first boundary determination module configured to determine the first boundary information of each sub-image according to the number and size information of the divided image into the sub-image;
  • the sub-image second boundary determination module is used to determine each sub-image in the image to be processed according to the size of the convolution kernel in the convolution operation information performed on the cut sub-image and the first boundary information of each sub-image The second boundary information;
  • the sub-image cutting module is used to sequentially cut the image according to the second boundary information of the sub-image in the image to be processed.
  • the first boundary information is actual boundary information between a sub-image and other sub-images, that is, boundary information that actually effectively identifies the content of the sub-image.
  • the actual boundary between the sub-image 310 and the sub-image 320 is the solid line 400 in the figure
  • the actual boundary between the sub-image 310 and the sub-image 330 is the solid line 600 in the figure.
  • the convolution operation with a convolution kernel size of NxN requires adjacent NxN data and corresponding weights to be multiplied and the results are accumulated together, it is necessary to reserve N- between the two sub-images.
  • the first boundary of the sub-image 310 and the sub-image 320 is a solid line 400, the size of the convolution kernel that the sub-image needs to be executed 5x5; at this time, if you want to ensure the effective recognition of the part corresponding to the most edge pixel 311 in the sub-image 310, you need to reserve 4 columns of overlap between the sub-image 310 and the sub-image 320 to implement the actual convolution operation In the process, the sub-image 310 is fully recognized.
  • the boundary after the first boundary solid line 400 is extended is the second boundary dashed line 500, which is the cutting boundary of the sub-image 310. Only by cutting this boundary
  • the image to be processed is cut into 4 sub-images.
  • the actual boundary between the sub-images, the solid line 400 and the solid line 600 are both next to the two sub-images.
  • the cutting boundary of the sub-image that is, the second boundary, needs to be extended according to the size of the convolution kernel performed by the sub-image.
  • the second boundary 500 of the sub-image 310 has been extended into the sub-image 320, so here
  • each cutting should be performed on the complete image to be recognized.
  • Each time a sub-image is cut out from the complete image to be processed according to the second boundary information of the sub-image, and then cut in sequence Multiple corresponding sub-images.
  • the sub-image second boundary determination module includes:
  • the extended pixel value determination sub-module is used to determine the size of each sub-image in the to-be-processed image according to the size information of the cascaded convolution kernel and the parallel convolution kernel in the convolution operation related information performed on the cut sub-image Based on the first boundary, the pixel value information that needs to be extended to its edge.
  • the convolutional neural network layer required for the factor image is very complex, including parallel convolution kernels and cascaded convolution kernels. To ensure that the sub-images can be fully recognized, all convolutions need to be considered.
  • the size information of the core as shown in Figure 3.
  • the operation result of the previous convolutional layer (convolutional layer 1) is the input of the operation of the next convolutional layer (convolutional layer 2).
  • the parallel convolutional kernels can include multiple Convolutional layers (Convolutional layer 3.1, Convolutional layer 3.2, Convolutional layer 3.3).
  • the edge extension pixel value of the sub-image needs to satisfy the convolution kernel sizes of all cascaded convolutional layers and all parallel convolution kernel sizes at the same time, that is, the sum of the two can satisfy the complete recognition of the sub-image.
  • the size of the extended pixel value is:
  • the kernel sizes of the convolutional layers 3.1, 3.2, and 3.3 are 3 ⁇ 3, 1 ⁇ 1, and 5 ⁇ 5, respectively.
  • convolution size 1x1 does not need to be adjacent Pixels.
  • the sub-images of the image to be recognized perform all the convolutional layers 1, 2 and 3.1/3.2/3.3 respectively and converge after the convolutional layer 3.1/3.2/3.3.
  • the extended pixel value of each sub-image is the extension required for convolutional layer 1 (2 pixels), convolutional layer 2 (2 pixels), and convolutional layer 3.1, 3.2, and 3.3 (4 pixels).
  • the sum of the pixels in the adjacent sub-image range, where the pixel value to be extended is 8.
  • the second boundary information determining sub-module is configured to determine the second boundary information of each sub-image according to the first boundary information of each sub-image and its corresponding extended pixel value information. Specifically, based on the first boundary of each sub-image, the corresponding second boundary information can be determined by extending the corresponding pixel value to the range of the next sub-image.
  • the device further includes:
  • the filter circuit unit is configured to filter out the calculation result of the non-maximum size convolution core in the parallel convolution core for the pixels in the extended area through the provided filter circuit module.
  • the extended pixels of the sub-image shown are not the actual image content of the sub-image, when performing convolutional layer operations on the sub-image, it does not care whether the image content of the extended pixels needs to be recognized, or whether it is not. Need to be recognized, because this part of the content will be effectively recognized in other sub-images, so as not to repeat the need to filter it out.
  • the extended pixel value is determined for the parallel convolution kernel part, only the largest size of the convolution kernel is considered, that is, the pixel identified by the largest size convolution kernel operation must belong to the effective recognition range of the sub-image, so The convolution result of the largest size in the parallel convolution kernel needs to be retained, and the convolution operation results of other sizes in the parallel convolution kernel for pixels in the extended area should be filtered out.
  • the image processing device based on the convolutional neural network described in this application can according to the preset image processing strategy, the number of sub-images corresponding to this cut and their size-related information, and the corresponding sub-images Convolution operation strategy, the read image to be processed is cut into multiple sub-images, and each sub-image after the cutting is independently and sequentially convolved; further according to the sub-image convergence point information in the preset image processing strategy, the corresponding The converging point of, converges the convolution operation results of the sub-images; here, it is necessary to determine whether the convolution operation results after converging need to be continuously cut according to the preset image processing strategy information, and if necessary, the convolution operation results After outputting to an external storage unit, it is read into the device, and the steps of image cutting, sub-image convolution operation, and aggregation in the above image processing method are executed, until the convolution operation result after aggregation does not need to be cut again; The remaining convolution operation is performed,
  • the disclosed technical content can be implemented in other ways.
  • the device embodiments described above are merely illustrative.
  • the division of the units may be a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or may be Integrate into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of units or modules, and may be in electrical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • each unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium.
  • the technical solution of this application essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , Including several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application.
  • the aforementioned storage media include: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本申请提供一种基于卷积神经网络的图像处理方法及装置，该技术可根据预设的图像处理策略中切割处理信息及对应子图像的卷积运算策略，将待处理图像执行一次或多次切割，每次切割为多个子图像，对切割后的各子图像独立依次进行卷积运算；在预设的子图像汇聚点汇聚所述各子图像的卷积运算结果；执行剩余卷积运算，获得最终卷积运算结果；将所述结果输出至外部存储单元。仅需要在中间的卷积运算结果再次切割时，输出至外部存储单元，并再次读取执行前述图像切割、卷积运算、汇聚等步骤，不再需要将每个卷积层运算后的中间数据输出至外部存储单元，再读入作为下一个卷积层的输入；降低了处理器外部存储单元的带宽，显著提高图像处理系统的整体性能。

Description

一种基于卷积神经网络的图像处理方法及装置 技术领域
本申请涉及图像处理领域，尤其涉及一种基于卷积神经网络的图像处理的技术。
背景技术
卷积神经网络(Convolutional Neural Network，CNN)作为人工神经网络的一种，已成为当前图像处理领域的研究热点。卷积神经网络是一个多层的神经网络，每层由多个三维平面构成，而每个平面都是由不同的卷积核卷积而成。卷积神经网络的每个神经网络层都会生成大量数据。上层的输出成为下一层的输入。由于神经网络可以具有多个层，因此卷积神经网络的卷积运算量非常大，而且每层网络都需要卷积运算，做一次图像处理需要多层卷积核及多平面卷积。
现有技术中，每个神经网络层之间生成的数据需要卷积神经网络处理器的外部存储器(如GDDR或DDR或HBM存储器)进行缓存；也即对待识别图像执行卷积层运算过程中，每个卷积层之间的数据均需要输出至外部存储器，然后再读取至处理器内部。每个神经网络层都会产生大量数据，对处理器外部存储器的带宽要求就非常高。即使处理器外部存储器的带宽足够大，也可能因为卷积运算数据流量太大影响卷积神经网络系统的性能。
针对上述的问题，目前尚未提出有效的解决方案。
发明内容
本申请的一个目的是提供一种基于卷积神经网络的图像处理方法及装置，以解决图像处理过程中降低卷积神经网络处理器外部存储器的带宽，以及卷积神经网络处理器性能优化等问题。
根据本申请的一个方面，提供了一种基于卷积神经网络的图像处理方法，该方法包括：
根据预设的图像处理策略中该次切割对应子图像数量及其大小相关 信息，将读取的待处理图像切割为多个子图像；
根据预设的图像处理策略中所述子图像对应的卷积运算策略，对切割后的各子图像独立依次进行卷积运算；
根据预设的图像处理策略中的子图像汇聚点信息，在对应的汇聚点汇聚所述各子图像的卷积运算结果；
在根据预设的图像处理策略信息，所述卷积结果不需要进行再次切割的情况下，执行剩余卷积运算，并获得最终卷积运算结果；
将所述最终卷积运算结果输出至外部存储单元。
根据本申请的另一个方面，提供一种基于卷积神经网络的图像处理装置，所述装置包括：
图像切割单元，用于根据预设的图像处理策略中该次切割对应子图像数量及其大小相关信息，将读取的待处理图像切割为多个子图像；
子图像卷积单元，用于根据预设的图像处理策略中所述子图像对应的卷积运算策略，对切割后的各子图像独立依次进行卷积运算；
子图像汇聚单元，用于根据预设的图像处理策略中的子图像汇聚点信息，在对应的汇聚点汇聚所述各子图像的卷积运算结果；
剩余卷积运算执行单元，用于在根据预设的图像处理策略信息，所述卷积结果不需要进行再次切割的情况下，执行剩余卷积运算，并获得最终卷积运算结果；
图像处理结果输出单元，将所述最终卷积运算结果输出至外部存储单元。
与现有技术相比，本申请所述的方法及装置可以根据预设的图像处理策略中该次切割对应子图像数量及其大小相关信息，以及所述子图像对应的卷积运算策略，将读取的待处理图像切割为多个子图像，对切割后的各子图像独立依次进行卷积运算；进一步根据预设的图像处理策略中的子图像汇聚点信息，在对应的汇聚点汇聚所述各子图像的卷积运算结果；此处，需要根据预设的图像处理策略信息判断汇聚后的卷积运算结果是否需要继续切割，若需要则将所述卷积运算结果输出至外部存储单元后，再读入所述装置内，并执行上述图像处理方法中的图像切割、子图像卷积运算、汇聚等步骤，直至汇聚后的卷积运算结果不需要再次切割；执行剩余卷积运算，并获得最终卷积运算结果；将所述最终卷积运算结果输出至外部存 储单元。从而使图像处理执行卷积层运算过程中仅需要在中间的卷积运算结果进行再次切割的情况下，输出至所述装置的外部存储单元，并再次读入所述装置执行图像切割、卷积运算、汇聚等处理步骤，不再需要将每个卷积层运算后均需要将中间数据输出至图像处理处理器外部存储单元进行缓存，然后再读入处理器作为下一个卷积层的输入以执行下一个卷积层运算；大大降低了所述装置外部存储单元的带宽，显著提高图像处理系统的整体性能。
附图说明
通过阅读参照以下附图所作的对非限制性实施例所作的详细描述，本申请的其它特征、目的和优点将会变得更明显：
图1是根据本申请一个实施例的一种基于卷积神经网络的图像处理方法流程图；
图2是根据本申请一个实施例的一种待处理图像边界信息确定示意图；
图3是根据本申请一个实施例的一种卷积网络结构示意图；
图4示出根据本申请一个实施例的一种待处理图像的子图像边缘像素值识别示意图；
图5示出本申请的一个实施例的一种基于卷积神经网络的图像处理装置的原理框图。
附图中相同或相似的附图标记代表相同或相似的部件。
具体实施方式
为了使本技术领域的人员更好地理解本申请的技术方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分的实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本申请保护的范围。
需要说明的是，本申请的说明书和权利要求书及上述附图中的术语 “第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
本申请提供的基于卷积神经网络的图像处理技术，其中的图像处理包括但不限于图像识别、图像分割、图像归类、图像优化、图像增强等图像处理方式。一般地通过卷积神经网络进行处理的图像本身就比较大，以1920x1080像素值的图像作为卷积神经网络的输入，输出通道大小是32为例，那么可得出每个卷积层输出数据大小为1920x1080x32或132,710,400字节(每像素2个字节)。132,710,400个字节(或132M字节)的存储处理空间对于图像处理装置内部存储单元实现起来非常昂贵。本申请通过图像切割的方式解决该问题，具体地，作为举例参图2所示，这里将待处理图像切割为4个子图像，使得图像处理的中间数据大小可以适合于图像处理装置的内部存储器。例如，前述的输入图像大小为1920x1080x32示例中，在图像没有切割的情况下，需要132M字节；在图2的所示的例子中，仅需要四分之一或33M字节(含相邻子图像之间重叠部分的对应数据内容)。
通常，卷积神经网络的网络层越深，输出数据量就越小。这是因为在机器学习网络的后续层中图像尺寸大大减小，并且即使输出通道尺寸增加，在后续层中，由层生成的数据总量以及因此缓冲要求也大大减少。本申请提供的基于卷积网络的图像处理方法及装置，通过将待处理图像进行切割、卷积运算处理、汇聚、剩余卷积运算处理的方式，可确保卷积运算执行过程中的卷积中间数据可以保存在图像处理装置的内部存储单元，仅在待处理图像处理过程中需要再次切割时，需要将卷积运算结果输出至外部存储单元，再将其读入所述装置内执行相应的图像切割、汇聚等；从而可以非常有效的降低对图像处理装置外部存储单元的带宽要求，显著提高图像处 理装置及系统的整体性能。
根据本申请的一个方面，提供了一种基于卷积神经网络的图像处理的方法。参考图1，该方法包括以下步骤：
S1根据预设的图像处理策略中该次切割对应子图像数量及其大小相关信息，将读取的待处理图像切割为多个子图像。
在该步骤中，图像处理策略包括针对某确定的待处理图像在执行卷积神经网络运算处理过程中的所有策略信息，例如，待处理图像需要执行的卷积神经网络信息，待处理图像切割次数，每次切割为子图像的大小、数量信息，每次切割后的子图像需要执行的卷积层运算信息，子图像卷积结果汇聚位置信息，汇聚后的图像需要执行的卷积层运算信息等等。待处理图像可以为相关设备采集后预处理后输入的图像信息，也可以为直接从外部缓存中直接读取的图像信息。
具体地，将读取的待处理图像切割为多个子图像，应以像素为单位对待处理图像进行切割。首先，要确定切割为子图像的边界，子图像的边界包括待处理图像边界及与其他子图像相邻的边界，其中子图像间的边界具体需依据图像处理策略中本次切割对应生成的子图像的数量及大小确定。更具体地，可以通过先确定列的边界，再确定行边界的方式确定，但是具体的确定方式不做限制。其次，按照确定的子图像间的边界信息将待处理图像分割为多个子图像，进一步地，对图像切割的过程中可以选择在同一张图像上直接切割出对应的子图像，也可以在多张完整待处理图像上分别切割出不同的子图像，也可以每次均在完整的待处理图像上切割其中一个子图像，那么切割时便需要准备对应子图像数量的完整待切割待处理图像，以确保图像切割的准确性。
S2根据预设的图像处理策略中所述子图像对应的卷积运算策略相关信息，对切割后的各子图像独立依次进行卷积运算。
在所述S2中，如前所述，图像处理策略包括针对某确定的待处理图像在执行卷积神经网络运算处理过程中的所有策略信息，例如，待处理图像需要执行的卷积神经网络信息，待处理图像切割次数，每次切割为子图像的大小、数量信息，每次切割后的子图像需要执行的卷积层运算信息， 子图像卷积结果汇聚位置信息，汇聚后的图像需要执行的卷积层运算信息等等。
具体地，对切割后的各子图像独立依次进行卷积运算，首先确定该次切割后的子图像需要执行的所有卷积层运算信息，然后，分别对切割后的每个子图像独立执行所述所有卷积层运算，并获得每个子图像对应的卷积运算结果。
作为举例，参图2所示，待处理图像被切割为子图像310、子图像320、子图像330及子图像340四个子图像，需要执行的卷积运算如图3所示卷积层1至卷积层3(含卷积层3.1/3.2/3.3三个并联卷积层)，此时，需要分别将子图像310执行从卷积层1至卷积层3、子图像320执行从卷积层1至卷积层3、子图像330执行从卷积层1至卷积层3、子图像340执行从卷积层1至卷积层3，分别独立依次执行。
在一些实施例中，所述S2包括：
S21(未示出)根据图像切割处理策略中的卷积运算策略信息，确定切割后子图像执行的卷积层运算信息。
具体地，将待处理图像切割为子图像后，需根据预设的卷积运算策略信息确定所述子图像应当独立执行的卷积层运算信息。例如，若卷积运算总层数为30层，1次切割，汇聚点为卷积运算执行卷积运算第26层完毕后，那么图像切割后的子图像需要执行的卷积运算为第一层至第26层。
S22(未示出)对切割后的每个子图像独立、依次执行所述卷积层运算，并获得每个子图像的卷积运算结果。具体地，切割后的子图像在执行对应的卷积运算时，是相互独立的，按照上述举例，若将图像切割为a、b两个子图像，那么切割后的子图像a执行第1层至第26卷积运算后，子图像b再执行从第1层至第26层的卷积运算，子图像之间为独立依次执行的。
S3根据预设的图像处理策略中的子图像汇聚点信息，在对应的汇聚点汇聚所述各子图像的卷积运算结果。
在该步骤中，如前所述，预设的图像处理策略信息中包括子图像的汇聚点信息，在汇聚点处，将上述子图像的卷积层运算结果进行汇聚，拼接 在一起，此时可以直接将卷积结果进行拼接，获得上述S2中卷积运算后的图像处理结果。作为举例，参图3所示，子图像卷积运算结果在所有子图像分别独立执行卷积层1至卷积层3(含卷积层3.1/3.2/3.3三个并联卷积层)后进行汇聚。
S4在根据预设的图像处理策略信息，所述卷积结果不需要进行再次切割的情况下，执行剩余卷积运算，并获得最终卷积运算结果。
在该步骤中，图像处理策略信息包括图像处理过程中需要切割的次数信息，若中间卷积结果需要重新进行切割的，必须将所述卷积运算结果输出至图像处理装置的外部存储单元中，若中间卷积结果不需要再次进行切割，此时，只需要在所述卷积运算结果的基础上执行剩余的卷积运算即可获得最终卷积运算结果，也即图像处理结果。
在其中一个实施例中，所述S4包括：
S41(未示出)根据预设的图像处理策略信息，确定所述汇聚的卷积结果需要执行的剩余卷积运算信息。
具体地，此处的剩余卷积运算信息为所述汇聚后的卷积结果不需要再次执行切割，仅需要执行相应卷积运算即完成图像处理的情况，例如，若图像处理预设信息包括该图像需要切割1次，共需要执行卷积层50层，汇聚点为第39层执行完毕处，那么依次就可推出剩余卷积层为卷积运算中的第40层至第50层的最后11层。再例如，若图像处理预设信息包括该图像需要切割2次，共需要执行卷积层60层，汇聚点为第39层执行完毕处及第59层执行完毕处，那么依次就可推出剩余卷积层为卷积运算中的第60层1层。
S42(未示出)对所述汇聚的卷积结果执行所述剩余的卷积运算，并获得最终卷积运算结果。
S5将所述最终卷积运算结果输出至外部存储单元。
具体地，此处的卷积运算结果为图像处理最终结果，将该结果输出至图像处理装置的外部存储单元中。
在一些实施例中，所述方法在S1之前还包括：
获取待处理图像信息，并对待处理图像进行预处理。
具体地，此处获取待处理图像的方式不做限制，可以为图像采集设备 采集的图像读入至本申请的图像处理系统中；也可以是直接读取存储空间中现有的图像信息等等，此处的存储空间也不做限制，例如，可以是普通的ROM/RAM类型的存储器，也可以是云盘、通信软件网络存储空间等，获取待处理图像后需要对其进行预处理，例如，图像大小、亮度等的处理。
在一些实施例中，所述方法还包括S6(未示出)：
根据待处理图像信息、内部存储空间大小及待处理图像需要执行的卷积运算策略信息，预设图像处理策略；其中，所述图像处理策略信息包括图像切割策略信息、汇聚点信息。具体地，如前述待图像处理的切割策略包括针对该待处理图像执行卷积神经网络运算处理过程中的所有策略信息，故在设置时，已知信息包括待处理图像大小、针对待处理图像需要执行的所有卷积神经网络层、图像处理处理器内存大小等，还需满足图像处理处理器的内存大小必须满足子图像的独立处理空间；在上述已知信息及需要满足条件的基础上确定出待处理图像被切割的次数、每次切割的子图像大小及数量、每次切割的子图像需要执行的卷积层数等等。
更具体地，因图像处理过程中对应卷积神经网络的网络层越深，输出数据量就越小，可依此在预设图像处理策略的汇聚点时，可考虑图像汇聚后的大小是否适合图像处理装置的内部存储单元大小等因素。
在其中一个实施例中，在已知汇聚点信息的情况下，所述S5包括：
S51(未示出)预估汇聚点处汇聚的卷积运算结果大小。
S52(未示出)将所述预估结果与图像处理装置的内部存储空间大小进行比较。图像处理装置的内部存储空间是已知的，直接将二者进行比较即可，此处要求预估的结果不能太接近内部存储空间，因该结果是预估的，并不是实际执行是的大小值，一般可以考虑留出一些多余空间，以确保实际执行过程中可以顺利进行。
S53(未示出)根据上述比较结果确定所述卷积结果是否需要再次切割。具体地，若所述预估结果小于图像处理装置的内部存储空间，则说明基于该结果可以继续执行卷积运算而不需要再次切割，否则，需要设置图像切割策略中增加一次图像切割次数。
继续参图1所示，在一些实施例中，根据预设的图像处理策略信息， 在所述卷积结果需要进行再次切割的情况下，所述方法还包括：
S7将所述汇聚的各子图像的卷积运算结果输出至外部存储单元。
具体地，在该步骤中，因汇聚的卷积远算结果比较大，不适合在图像处理装置内部存储单元中完成剩余的卷积运算，需要再次切割，降低每次卷积运算的数据大小。在所述汇聚的卷积运算结果需要再次切割的情况下，必须先输出至图像处理装置的外部存储单元中，从外部存储单元读入后再执行后续的图像处理策略。
继续参图1所示，在一些实施例中，所述方法在S7后还包括：
S8读取所述输出至外部存储单元的各子图像的卷积运算结果。具体地，该实施例主要针对待处理图像的子图像所执行的卷积运算未完成预设的待策略图像需要执行所有卷积层的情况，此时需要将输出至图像处理装置外部存储单元的拼图信息重新读入图像处理处理器内，继续进行图像卷积运算处理。
其中，所述S1包括：
根据预设的图像处理策略中对应该次切割的子图像数量及其大小相关信息，将读取的卷积运算结果再次切割为多个子图像。
具体地，该实施例主要针对待处理图像的图像处理策略中包括多次图像切割的情况，此时需要将从所述外部存储器读入的拼图信息作为待处理图像，根据对应的图像处理策略相关信息继续切割为多个子图像并执行对应卷积运算，并根据所述策略信息进行汇聚。
在一些实施例中，所述图像切割处理策略包括将待处理图像切割为子图像的数量及其大小相关信息，以及对切割后的子图像执行的卷积运算相关信息，所述S1包括：
S11(未示出)根据所述将待处理图像分割为子图像的数量及其大小信息，确定每个子图像的第一边界信息；
S12(未示出)根据所述对切割后的子图像执行的卷积运算信息中卷积核尺寸及每个子图像的第一边界信息，确定所述待处理图像中每个子图像的第二边界信息；
S13(未示出)根据所述待处理图像中子图像的第二边界信息对图像进行依次切割。
具体地，在步骤S11中，第一边界信息为子图像与其他子图像之间的实际边界信息，也即实际上有效识别子图像内容的边界信息。参图2所示，子图像310与子图像320之间的实际边界为图中实线400，子图像310与子图像330之间的实际边界为图中实线600。
具体地，在步骤S12中，由于卷积核尺寸为NxN的卷积运算需要相邻的NxN数据及相应的权重相乘并将结果累加在一起，故需要在两个子图像之间保留N-1行/列的重叠部分，以便完整识别出每个子图像的图像信息，参图3所示，子图像310与子图像320的第一边界为实线400，子图像需要执行的卷积核尺寸为5x5；此时若要确保对子图像310中最边缘像素311对应的部分进行有效识别，则需要在子图像310与子图像320之间保留4列的重叠部分，以实现实际执行卷积运算过程中将子图像310完整识别，此时，第一边界实线400延展后的边界即为第二边界虚线500，也即子图像310的切割边界，以此边界进行切割才可以确保子图像310中像素311被有效识别。
具体地，在步骤S13中，参图2并结合图4所示，将待处理图像切割为4个子图像中子图像间的实际边界实线400及实线600均是紧挨着两个子图像的，而子图像的切割边界也即第二边界是需要根据子图像执行的卷积核尺寸进行延展的，例如，子图像310的第二边界500已经延展到子图像320中，故，此处切割待处理图像为多个子图像时，每次切割均应在完整的待识别图像上进行，每次从完整的待处理图像上根据该子图像的第二边界信息切割出一个子图像，依次切割出对应的多个子图像。在一些实施例中，所述S12(未示出)包括：
S121(未示出)根据所述对切割后的子图像执行的卷积运算相关信息中级联卷积核及并联卷积核尺寸信息，确定所述待处理图像中每个子图像在第一边界的基础上需要向其边缘延展的像素值信息。
具体地，参图3所示，因子图像需要的运行的卷积神经网络层非常复杂，其中包括并联卷积核和级联卷积核，为确保子图像能够被完整识别， 需要考虑所有卷积核的尺寸信息，参图3所示，上一层卷积层(卷积层1)运算结果为下一层卷积层(卷积层2)运算的输入，并联的卷积核可包括多个卷积层(卷积层3.1、卷积层3.2、卷积层3.3)。其中所述子图像边缘延展像素值需要同时满足所有级联卷积层的卷积核尺寸以及所有并联卷积核尺寸，也即二者相加，方可满足所述子图像的完整识别。
在一些实施例中，所述延展像素值大小为：
将所述子图像对应的所有卷积核中级联卷积核及并联卷积核需要延展的像素值相加，其中级联卷积核部分对应需要延展的像素值为：每个级联卷积核尺寸减1后累加，其中并联卷积核部分对应需要延展的像素值为：并联卷积核中最大尺寸的卷积核的尺寸减1。
具体地，参图3所示，若卷积层1和2的尺寸大小均为3×3，卷积层3.1、3.2和3.3的内核大小分别为3x3、1x1和5x5。卷积核尺寸为3x3的需要2个相邻像素(N-1＝2)，卷积核尺寸为5x5的需要4个相邻像素(N-1＝4)，卷积大小1x1不需要相邻像素。
在该实施例中，待识别图像的子图像分别执行所有卷积层1、2和3.1/3.2/3.3并且在卷积层层3.1/3.2/3.3之后汇聚。那么每个子图像的延展像素值为卷积层层1(2个像素)，卷积层2(2个像素)和卷积层层3.1、3.2和3.3(4个像素)所需的延展至相邻子图像范围的像素的总和，此处需延展的像素值为8。
S122(未示出)根据所述每个子图像第一边界信息及其对应的延展像素值信息确定所述每个子图像的第二边界信息。具体地，在每个子图像的第一边界基础上，向旁边的子图像范围上延展对应的像素值便可确定对应的第二边界信息。
在一些实施例中，所述方法在S3之前还包括：
通过设置的滤波电路模块将所述并联卷积核中非最大尺寸的卷积核针对所述延展区域的像素的运算结果过滤掉。
具体地，因所示子图像的延展像素并非子图像的实际图像内容，在对所述子图像进行卷积层运算时，并不关心所述延展像素的图像内容是否需要识别，或者说是不需要识别的，因该部分内容在其他子图像中会被有效 识别的，以免重复需要将其过滤掉。因延展像素值确定时针对并联卷积核部分，仅考虑其中最大尺寸的卷积核大小，也即该最大尺寸卷积核运算所识别的像素肯定属于所述子图像的有效识别范围内，故需保留并联卷积核中最大尺寸的卷积结果，并联卷积核中其他尺寸的针对延展区域像素的卷积运算结果均应过滤掉。
与现有技术相比，本申请所述的基于卷积神经网络的图像处理方法可以根据预设的图像处理策略中该次切割对应子图像数量及其大小相关信息，以及所述子图像对应的卷积运算策略，将读取的待处理图像切割为多个子图像，对切割后的各子图像独立依次进行卷积运算；进一步根据预设的图像处理策略中的子图像汇聚点信息，在对应的汇聚点汇聚所述各子图像的卷积运算结果；此处，需要根据预设的图像处理策略信息判断汇聚后的卷积运算结果是否需要继续切割，若需要则将所述卷积运算结果输出至外部存储单元后，再读入所述装置内，并执行上述图像处理方法中的图像切割、子图像卷积运算、汇聚等步骤，直至汇聚后的卷积运算结果不需要再次切割；执行剩余卷积运算，并获得最终卷积运算结果；将所述最终卷积运算结果输出至外部存储单元。从而使图像处理执行卷积层运算过程中仅需要在中间的卷积运算结果进行再次切割的情况下，输出至所述装置的外部存储单元，并再次读入所述装置执行图像切割、卷积运算、汇聚等处理步骤，不再需要将每个卷积层运算后均需要将中间数据输出至图像处理处理器外部存储单元进行缓存，然后再读入处理器作为下一个卷积层的输入以执行下一个卷积层运算；大大降低了处理器外部存储单元的带宽，显著提高图像处理系统的整体性能。
参图5所示，根据本申请的另一个方面，本申请还提供一种基于卷积神经网络的图像处理装置，本申请所述装置包括但不限于本申请中基于卷积神经网络图像处理芯片、处理器，其中所述图像处理包括但不限于图像的识别、分割、归类、优化、增强等，所述装置100包括：
图像切割单元110，用于根据预设的图像处理策略中该次切割对应子图像数量及其大小相关信息，将读取的待处理图像切割为多个子图像。
在该实施例中，图像处理策略包括针对某确定的待处理图像在执行卷 积神经网络运算处理过程中的所有策略信息，例如，待处理图像需要执行的卷积神经网络信息，待处理图像切割次数，每次切割为子图像的大小、数量信息，每次切割后的子图像需要执行的卷积层运算信息，子图像卷积结果汇聚位置信息，汇聚后的图像需要执行的卷积层运算信息等等。待处理图像可以为相关设备采集后预处理后输入的图像信息，也可以为直接从外部缓存中直接读取的图像信息。
具体地，将读取的待处理图像切割为多个子图像，应以像素为单位对待处理图像进行切割。首先，要确定切割为子图像的边界，子图像的边界包括待处理图像边界及与其他子图像相邻的边界，其中子图像间的边界具体需依据图像处理策略中本次切割对应生成的子图像的数量及大小确定。更具体地，可以通过先确定列的边界，再确定行边界的方式确定，但是具体的确定方式不做限制。其次，按照确定的子图像间的边界信息将待处理图像分割为多个子图像，进一步地，对图像切割的过程中可以选择在同一张图像上直接切割出对应的子图像，也可以在多张完整待处理图像上分别切割出不同的子图像，也可以每次均在完整的待处理图像上切割其中一个子图像，那么切割时便需要准备对应子图像数量的完整待切割待处理图像，以确保图像切割的准确性。
子图像卷积单元120，用于根据预设的图像处理策略中所述子图像对应的卷积运算策略相关信息，对切割后的各子图像独立依次进行卷积运算。
在该实施例中，如前所述，图像处理策略包括针对某确定的待处理图像在执行卷积神经网络运算处理过程中的所有策略信息，例如，待处理图像需要执行的卷积神经网络信息，待处理图像切割次数，每次切割为子图像的大小、数量信息，每次切割后的子图像需要执行的卷积层运算信息，子图像卷积结果汇聚位置信息，汇聚后的图像需要执行的卷积层运算信息等等。
具体地，对切割后的各子图像独立依次进行卷积运算，首先确定该次切割后的子图像需要执行的所有卷积层运算信息，然后，分别对切割后的每个子图像独立执行所述所有卷积层运算，并获得每个子图像对应的卷积运算结果。
作为举例，参图2所示，待处理图像被切割为子图像310、子图像320、子图像330及子图像340四个子图像，需要执行的卷积运算如图3所示卷积层1至卷积层3(含卷积层3.1/3.2/3.3三个并联卷积层)，此时，需要分别将子图像310执行从卷积层1至卷积层3、子图像320执行从卷积层1至卷积层3、子图像330执行从卷积层1至卷积层3、子图像340执行从卷积层1至卷积层3，分别独立依次执行。
在一些实施例中，所述子图像卷积单元120包括：
子图像执行卷积层运算确定模块121(未示出)，用于根据图像切割处理策略中的卷积运算策略信息，确定切割后子图像执行的卷积层运算信息。
具体地，将待处理图像切割为子图像后，需根据预设的卷积运算策略信息确定所述子图像应当独立执行的卷积层运算信息。例如，若卷积运算总层数为30层，1次切割，汇聚点为卷积运算执行卷积运算第26层完毕后，那么图像切割后的子图像需要执行的卷积运算为第一层至第26层。
子图像卷积层运算执行模块122(未示出)，用于对切割后的每个子图像独立、依次执行所述卷积层运算，并获得每个子图像的卷积运算结果。具体地，切割后的子图像在执行对应的卷积运算时，是相互独立的，按照上述举例，若将图像切割为a、b两个子图像，那么切割后的子图像a执行第1层至第26卷积运算后，子图像b再执行从第1层至第26层的卷积运算，子图像之间为独立依次执行的。
子图像汇聚单元130，用于根据预设的图像处理策略中的子图像汇聚点信息，在对应的汇聚点汇聚所述各子图像的卷积运算结果。
在该实施例中，如前所述，预设的图像处理策略信息中包括子图像的汇聚点信息，在汇聚点处，将上述子图像的卷积层运算结果进行汇聚，拼接在一起，此时可以直接将卷积结果进行拼接，获得上述S2中卷积运算后的图像处理结果。作为举例，参图3所示，子图像卷积运算结果在所有子图像分别独立执行卷积层1至卷积层3(含卷积层3.1/3.2/3.3三个并联卷积层)后进行汇聚。
剩余卷积运算执行单元140，用于在根据预设的图像处理策略信息，所述卷积结果不需要进行再次切割的情况下，执行剩余卷积运算，并获得最终卷积运算结果。
在该实施例中，图像处理策略信息包括图像处理过程中需要切割的次数信息，若中间卷积结果需要重新进行切割的，必须将所述卷积运算结果输出至图像处理装置的外部存储单元中，若中间卷积结果不需要再次进行切割，此时，只需要在所述卷积运算结果的基础上执行剩余的卷积运算即可获得最终卷积运算结果，也即图像处理结果。
在其中一个实施例中，所述剩余卷积运算执行单元140包括：
剩余卷积运算确定模块141(未示出)，用于根据预设的图像处理策略信息，确定所述汇聚的卷积结果需要执行的剩余卷积运算信息。
具体地，此处的剩余卷积运算信息为所述汇聚后的卷积结果不需要再次执行切割，仅需要执行相应卷积运算即完成图像处理的情况，例如，若图像处理预设信息包括该图像需要切割1次，共需要执行卷积层50层，汇聚点为第39层执行完毕处，那么依次就可推出剩余卷积层为卷积运算中的第40层至第50层的最后11层。再例如，若图像处理预设信息包括该图像需要切割2次，共需要执行卷积层60层，汇聚点为第39层执行完毕处及第59层执行完毕处，那么依次就可推出剩余卷积层为卷积运算中的第60层1层。
剩余卷积执行模块142(未示出)，用于对所述汇聚的卷积结果执行所述剩余的卷积运算，并获得最终卷积运算结果。
图像处理结果输出单元150，用于将所述最终卷积运算结果输出至外部存储单元。
具体地，此处的卷积运算结果为图像处理最终结果，将该结果输出至图像处理装置的外部存储单元中。
在一些实施例中，所述装置还包括：
待处理图像获取单元，用于获取待处理图像信息，并对待处理图像进行预处理。
具体地，此处获取待处理图像的方式不做限制，可以为图像采集设备采集的图像读入至本申请的图像处理系统中；也可以是直接读取存储空间 中现有的图像信息等等，此处的存储空间也不做限制，例如，可以是普通的ROM/RAM类型的存储器，也可以是云盘、通信软件网络存储空间等，获取待处理图像后需要对其进行预处理，例如，图像大小、亮度等的处理。
在一些实施例中，所述装置还包括：
图像处理策略预设单元，用于根据待处理图像信息、内部存储空间大小及待处理图像需要执行的卷积运算策略信息，预设图像处理策略；其中，所述图像处理策略信息包括图像切割策略信息、汇聚点信息。具体地，如前述待图像处理的切割策略包括针对该待处理图像执行卷积神经网络运算处理过程中的所有策略信息，故在设置时，已知信息包括待处理图像大小、针对待处理图像需要执行的所有卷积神经网络层、图像处理处理器内存大小等，还需满足图像处理装置的内存大小必须满足子图像的独立处理空间；在上述已知信息及需要满足条件的基础上确定出待处理图像被切割的次数、每次切割的子图像大小及数量、每次切割的子图像需要执行的卷积层数等等。
更具体地，因图像处理过程中对应卷积神经网络的网络层越深，输出数据量就越小，可依此在预设图像处理策略的汇聚点时，可考虑图像汇聚后的大小是否适合图像处理装置的内部存储单元大小等因素。
在其中一个实施例中，在已知汇聚点信息的情况下，所述图像处理策略预设单元包括：
汇聚点卷积运算结果估算模块，用于预估汇聚点处汇聚的卷积运算结果大小。
预估值与内存大小比较模块，用于将所述预估结果与图像处理装置的内部存储空间大小进行比较。图像处理装置的内部存储空间是已知的，直接将二者进行比较即可，此处要求预估的结果不能太接近内部存储空间，因该结果是预估的，并不是实际执行是的大小值，一般可以考虑留出一些多余空间，以确保实际执行过程中可以顺利进行。
卷积结果再次切割确定模块，用于根据上述比较结果确定所述卷积结果是否需要再次切割。具体地，若所述预估结果小于图像处理装置的内部存储空间，则说明基于该结果可以继续执行卷积运算而不需要再次切割， 否则，需要设置图像切割策略中增加一次图像切割次数。
在一些实施例中，根据预设的图像处理策略信息，在所述卷积结果需要进行再次切割的情况下，所述装置还包括：
中间卷积结果输出单元，用于将所述汇聚的各子图像的卷积运算结果输出至外部存储单元。
具体地，在该实施例中，因汇聚的卷积远算结果比较大，不适合在图像处理装置内部存储单元中完成剩余的卷积运算，需要再次切割，降低每次卷积运算的数据大小。在所述汇聚的卷积运算结果需要再次切割的情况下，必须先输出至图像处理装置的外部存储单元中，从外部存储单元读入后再执行后续的图像处理策略。
在一些实施例中，所述装置还包括：
中间卷积结果读取单元，用于读取所述输出至外部存储单元的各子图像的卷积运算结果。具体地，该实施例主要针对待处理图像的子图像所执行的卷积运算未完成预设的待策略图像需要执行所有卷积层的情况，此时需要将输出至图像处理装置外部存储单元的拼图信息重新读入图像处理处理器内，继续进行图像卷积运算处理。
其中，所述图像切割单元用于：
根据预设的图像处理策略中对应该次切割的子图像数量及其大小相关信息，将读取的卷积运算结果再次切割为多个子图像。
具体地，该实施例主要针对待处理图像的图像处理策略中包括多次图像切割的情况，此时需要将从所述外部存储器读入的拼图信息作为待处理图像，根据对应的图像处理策略相关信息继续切割为多个子图像并执行对应卷积运算，并根据所述策略信息进行汇聚。
在一些实施例中，所述图像切割处理策略包括将待处理图像切割为子图像的数量及其大小相关信息，以及对切割后的子图像执行的卷积运算相关信息，所述图像切单元包括：
子图像第一边界确定模块，用于根据所述将待处理图像分割为子图像的数量及其大小信息，确定每个子图像的第一边界信息；
子图像第二边界确定模块，用于根据所述对切割后的子图像执行的卷积运算信息中卷积核尺寸及每个子图像的第一边界信息，确定所述待处理图像中每个子图像的第二边界信息；
子图像切割模块，用于根据所述待处理图像中子图像的第二边界信息对图像进行依次切割。
具体地，在该实施例中，第一边界信息为子图像与其他子图像之间的实际边界信息，也即实际上有效识别子图像内容的边界信息。参图2所示，子图像310与子图像320之间的实际边界为图中实线400，子图像310与子图像330之间的实际边界为图中实线600。
具体地，在该实施例中，由于卷积核尺寸为NxN的卷积运算需要相邻的NxN数据及相应的权重相乘并将结果累加在一起，故需要在两个子图像之间保留N-1行/列的重叠部分，以便完整识别出每个子图像的图像信息，参图3所示，子图像310与子图像320的第一边界为实线400，子图像需要执行的卷积核尺寸为5x5；此时若要确保对子图像310中最边缘像素311对应的部分进行有效识别，则需要在子图像310与子图像320之间保留4列的重叠部分，以实现实际执行卷积运算过程中将子图像310完整识别，此时，第一边界实线400延展后的边界即为第二边界虚线500，也即子图像310的切割边界，以此边界进行切割才可以确保子图像310中像素311被有效识别。
具体地，在该实施例中，参图2并结合图4所示，将待处理图像切割为4个子图像中子图像间的实际边界实线400及实线600均是紧挨着两个子图像的，而子图像的切割边界也即第二边界是需要根据子图像执行的卷积核尺寸进行延展的，例如，子图像310的第二边界500已经延展到子图像320中，故，此处切割待处理图像为多个子图像时，每次切割均应在完整的待识别图像上进行，每次从完整的待处理图像上根据该子图像的第二边界信息切割出一个子图像，依次切割出对应的多个子图像。
在一些实施例中，所述子图像第二边界确定模块包括：
延展像素值确定子模块，用于根据所述对切割后的子图像执行的卷积运算相关信息中级联卷积核及并联卷积核尺寸信息，确定所述待处理图像中每个子图像在第一边界的基础上需要向其边缘延展的像素值信息。
具体地，参图3所示，因子图像需要的运行的卷积神经网络层非常复杂，其中包括并联卷积核和级联卷积核，为确保子图像能够被完整识别，需要考虑所有卷积核的尺寸信息，参图3所示，上一层卷积层(卷积层1)运算结果为下一层卷积层(卷积层2)运算的输入，并联的卷积核可包括多个卷积层(卷积层3.1、卷积层3.2、卷积层3.3)。其中所述子图像边缘延展像素值需要同时满足所有级联卷积层的卷积核尺寸以及所有并联卷积核尺寸，也即二者相加，方可满足所述子图像的完整识别。
在一些实施例中，所述延展像素值大小为：
将所述子图像对应的所有卷积核中级联卷积核及并联卷积核需要延展的像素值相加，其中级联卷积核部分对应需要延展的像素值为：每个级联卷积核尺寸减1后累加，其中并联卷积核部分对应需要延展的像素值为：并联卷积核中最大尺寸的卷积核的尺寸减1。
具体地，参图3所示，若卷积层1和2的尺寸大小均为3×3，卷积层3.1、3.2和3.3的内核大小分别为3x3、1x1和5x5。卷积核尺寸为3x3的需要2个相邻像素(N-1＝2)，卷积核尺寸为5x5的需要4个相邻像素(N-1＝4)，卷积大小1x1不需要相邻像素。
在该实施例中，待识别图像的子图像分别执行所有卷积层1、2和3.1/3.2/3.3并且在卷积层层3.1/3.2/3.3之后汇聚。那么每个子图像的延展像素值为卷积层层1(2个像素)，卷积层2(2个像素)和卷积层层3.1、3.2和3.3(4个像素)所需的延展至相邻子图像范围的像素的总和，此处需延展的像素值为8。
第二边界信息确定子模块，用于根据所述每个子图像第一边界信息及其对应的延展像素值信息确定所述每个子图像的第二边界信息。具体地，在每个子图像的第一边界基础上，向旁边的子图像范围上延展对应的像素值便可确定对应的第二边界信息。
在一些实施例中，所述装置还包括：
滤波电路单元，用于通过设置的滤波电路模块将所述并联卷积核中非最大尺寸的卷积核针对所述延展区域的像素的运算结果过滤掉。
具体地，因所示子图像的延展像素并非子图像的实际图像内容，在对所述子图像进行卷积层运算时，并不关心所述延展像素的图像内容是否需要识别，或者说是不需要识别的，因该部分内容在其他子图像中会被有效识别的，以免重复需要将其过滤掉。因延展像素值确定时针对并联卷积核部分，仅考虑其中最大尺寸的卷积核大小，也即该最大尺寸卷积核运算所识别的像素肯定属于所述子图像的有效识别范围内，故需保留并联卷积核中最大尺寸的卷积结果，并联卷积核中其他尺寸的针对延展区域像素的卷积运算结果均应过滤掉。
与现有技术相比，本申请所述的基于卷积神经网络的图像处理装置可以根据预设的图像处理策略中该次切割对应子图像数量及其大小相关信息，以及所述子图像对应的卷积运算策略，将读取的待处理图像切割为多个子图像，对切割后的各子图像独立依次进行卷积运算；进一步根据预设的图像处理策略中的子图像汇聚点信息，在对应的汇聚点汇聚所述各子图像的卷积运算结果；此处，需要根据预设的图像处理策略信息判断汇聚后的卷积运算结果是否需要继续切割，若需要则将所述卷积运算结果输出至外部存储单元后，再读入所述装置内，并执行上述图像处理方法中的图像切割、子图像卷积运算、汇聚等步骤，直至汇聚后的卷积运算结果不需要再次切割；执行剩余卷积运算，并获得最终卷积运算结果；将所述最终卷积运算结果输出至外部存储单元。从而使图像处理执行卷积层运算过程中仅需要在中间的卷积运算结果进行再次切割的情况下，输出至所述装置的外部存储单元，并再次读入所述装置执行图像切割、卷积运算、汇聚等处理步骤，不再需要将每个卷积层运算后均需要将中间数据输出至图像处理处理器外部存储单元进行缓存，然后再读入处理器作为下一个卷积层的输入以执行下一个卷积层运算；大大降低了处理器外部存储单元的带宽，显著提高图像处理系统的整体性能。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
在本申请的上述实施例中，对各个实施例的描述都各有侧重，某个实 施例中没有详述的部分，可以参见其他实施例的相关描述。
在本申请所提供的几个实施例中，应该理解到，所揭露的技术内容，可通过其它的方式实现。其中，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，可以为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述仅是本申请的优选实施方式，应当指出，对于本技术领域的普通技术人员而言，显然本申请不限于上述示范性实施例的细节，而且在不背离本申请的精神或基本特征的情况下，能够以其他的具体形式实现本申请。因此，无论从哪一点来看，均应将实施例看作是示范性的，而且是 非限制性的，本申请的范围由所附权利要求而不是上述说明限定，因此旨在将落在权利要求的等同要件的含义和范围内的所有变化涵括在本申请内。不应将权利要求中的任何附图标记视为限制所涉及的权利要求。此外，显然“包括”一词不排除其他单元或步骤，单数不排除复数。装置权利要求中陈述的多个单元或装置也可以由一个单元或装置通过软件或者硬件来实现。第一，第二等词语用来表示名称，而并不表示任何特定的顺序。

Claims (24)

  1. 一种基于卷积神经网络的图像处理方法，其中，所述方法包括：
    根据预设的图像处理策略中该次切割对应子图像数量及其大小相关信息，将读取的待处理图像切割为多个子图像；
    根据预设的图像处理策略中所述子图像对应的卷积运算策略，对切割后的各子图像独立依次进行卷积运算；
    根据预设的图像处理策略中的子图像汇聚点信息，在对应的汇聚点汇聚所述各子图像的卷积运算结果；
    在根据预设的图像处理策略信息，所述卷积结果不需要进行再次切割的情况下，执行剩余卷积运算，并获得最终卷积运算结果；
    将所述最终卷积运算结果输出至外部存储单元。
  2. 根据权利要求1所述的方法，其中，所述方法还包括：
    获取待处理图像，并对待处理图像进行预处理。
  3. 根据权利要求1所述的方法，其中，所述方法还包括：
    根据待处理图像信息、内部存储空间大小及待处理图像需要执行的卷积运算策略信息，预设图像处理策略；其中，所述图像处理策略信息包括图像切割策略信息、汇聚点信息。
  4. 根据权利要求3所述的方法，其中，在已知汇聚点信息的情况下，所述根据待处理图像信息、内部存储空间大小及待处理图像需要执行的卷积运算策略信息，预设图像处理策略包括：
    预估汇聚点处汇聚的卷积运算结果大小；
    将所述预估结果与图像处理装置的内部存储空间大小进行比较；
    根据上述比较结果确定所述卷积结果是否需要再次切割。
  5. 根据权利要求1所述的方法，其中，根据预设的图像处理策略信息，在所述卷积结果需要进行再次切割的情况下，所述方法还包括：
    将所述汇聚的各子图像的卷积运算结果输出至外部存储单元。
  6. 根据权利要求5所述的方法，其中，所述方法还包括：
    读取所述输出至外部存储单元的各子图像的卷积运算结果；
    其中，所述根据预设的图像处理策略中该次切割对应子图像数量及其大小相关信息，将读取的待处理图像切割为多个子图像包括：
    根据预设的图像处理策略中对应该次切割的子图像数量及其大小相关信息，将读取的卷积运算结果再次切割为多个子图像。
  7. 根据权利要求1所述的方法，其中，所述在根据预设的图像处理策略信息，所述卷积结果不需要进行再次切割的情况下，执行剩余卷积运算，并获得最终卷积运算结果包括：
    根据预设的图像处理策略信息，确定所述汇聚的卷积结果需要执行的剩余卷积运算信息；
    对所述汇聚的卷积结果执行所述剩余的卷积运算，并获得最终卷积运算结果。
  8. 根据权利要求1所述的方法，其中，所述图像切割处理策略包括将待处理图像切割为子图像的数量及其大小相关信息，以及对切割后的子图像执行的卷积运算相关信息，所述根据预设的图像处理策略中该次切割对应子图像数量及其大小相关信息，将读取的待处理图像切割为多个子图像包括：
    根据所述将待处理图像分割为子图像的数量及其大小信息，确定每个子图像的第一边界信息；
    根据所述对切割后的子图像执行的卷积运算信息中卷积核尺寸及每个子图像的第一边界信息，确定所述待处理图像中每个子图像的第二边界信息；
    根据所述待处理图像中子图像的第二边界信息对图像进行依次切割。
  9. 根据权利要求8所述的方法，其中，所述根据所述对切割后的子图像执行的卷积运算信息中卷积核尺寸及每个子图像的第一边界信息，确定所述待处理图像中每个子图像的第二边界信息包括：
    根据所述对切割后的子图像执行的卷积运算相关信息中级联卷积核及并联卷积核尺寸信息，确定所述待处理图像中每个子图像在第一边界的基础上需要向其边缘延展的像素值信息；
    根据所述每个子图像第一边界信息及其对应的延展像素值信息确定所述每个子图像的第二边界信息。
  10. 根据权利要求9所述的方法，其中，所述延展像素值大小为：
    将所述子图像对应的所有卷积核中级联卷积核及并联卷积核需要延展的像素值相加，其中级联卷积核部分对应需要延展的像素值为：每个级联卷积核尺寸减1后累加，其中并联卷积核部分对应需要延展的像素值为：并联卷积核中最大尺寸的卷积核的尺寸减1。
  11. 根据权利要求10所述的方法，其中，所述方法还包括：
    通过设置的滤波电路模块将所述并联卷积核中非最大尺寸的卷积核针对所述延展区域的像素值的运算结果过滤掉。
  12. 根据权利要求1至11中任一项所述的方法，其中，所述根据预设的图像处理策略中所述子图像对应的卷积运算策略，对切割后的各子图像独立依次进行卷积运算包括：
    根据图像切割处理策略中的卷积运算策略信息，确定切割后子图像执行的卷积层运算信息；
    对切割后的每个子图像独立、依次执行所述卷积层运算，并获得每个子图像的卷积运算结果。
  13. 一种基于卷积神经网络的图像处理装置，其中，所述装置包括：
    图像切割单元，用于根据预设的图像处理策略中该次切割对应子图像数量及其大小相关信息，将读取的待处理图像切割为多个子图像；
    子图像卷积单元，用于根据预设的图像处理策略中所述子图像对应的卷积运算策略，对切割后的各子图像独立依次进行卷积运算；
    子图像汇聚单元，用于根据预设的图像处理策略中的子图像汇聚点信息，在对应的汇聚点汇聚所述各子图像的卷积运算结果；
    剩余卷积运算执行单元，用于在根据预设的图像处理策略信息，所述卷积结果不需要进行再次切割的情况下，执行剩余卷积运算，并获得最终卷积运算结果；
    图像处理结果输出单元，将所述最终卷积运算结果输出至外部存储单元。
  14. 根据权利要求13所述的装置，其中，所述装置还包括：
    待处理图像获取单元，用于获取待处理图像，并对待处理图像进行预处理。
  15. 根据权利要求13所述的装置，其中，所述装置还包括：
    图像处理策略预设单元，用于根据待处理图像信息、内部存储空间大小及待处理图像需要执行的卷积运算策略信息，预设图像处理策略；其中，所述图像处理策略信息包括图像切割策略信息、汇聚点信息。
  16. 根据权利要求15所述的装置，其中，在已知汇聚点信息的情况下，所述图像处理策略预设单元包括：
    汇聚点卷积运算结果估算模块，用于预估汇聚点处汇聚的卷积运算结果大小；
    预估值与内存大小比较模块，用于将所述预估结果与图像处理装置的内部存储空间大小进行比较；
    卷积结果再次切割确定模块，用于根据上述比较结果确定所述卷积结果是否需要再次切割。
  17. 根据权利要求13所述的装置，其中，根据预设的图像处理策略信息，在所述卷积结果需要进行再次切割的情况下，所述装置还包括：
    中间卷积结果输出单元，用于将所述汇聚的各子图像的卷积运算结果输出至外部存储单元。
  18. 根据权利要求17所述的装置，其中，所述装置还包括：
    中间卷积结果读取单元，用于读取所述输出至外部存储单元的各子图像的卷积运算结果；
    其中，所述图像切割单元用于：
    根据预设的图像处理策略中对应该次切割的子图像数量及其大小相关信息，将读取的卷积运算结果再次切割为多个子图像。
  19. 根据权利要求13所述的装置，其中，所述剩余卷积运算执行单元包括：
    剩余卷积运算确定模块，用于根据预设的图像处理策略信息，确定所述汇聚的卷积结果需要执行的剩余卷积运算信息；
    剩余卷积执行模块，用于对所述汇聚的卷积结果执行所述剩余的卷积运算，并获得最终卷积运算结果。
  20. 根据权利要求13所述的装置，其中，所述图像切割处理策略包括 将待处理图像切割为子图像的数量及其大小相关信息，以及对切割后的子图像执行的卷积运算相关信息，所述图像切割单元包括：
    子图像第一边界确定模块，用于根据所述将待处理图像分割为子图像的数量及其大小信息，确定每个子图像的第一边界信息；
    子图像第二边界确定模块，用于根据所述对切割后的子图像执行的卷积运算信息中卷积核尺寸及每个子图像的第一边界信息，确定所述待处理图像中每个子图像的第二边界信息；
    子图像切割模块，用于根据所述待处理图像中子图像的第二边界信息对图像进行依次切割。
  21. 根据权利要求20所述的装置，其中，所述子图像第二边界确定模块包括：
    延展像素值确定子模块，用于根据所述对切割后的子图像执行的卷积运算相关信息中级联卷积核及并联卷积核尺寸信息，确定所述待处理图像中每个子图像在第一边界的基础上需要向其边缘延展的像素值信息；
    第二边界信息确定子模块，用于根据所述每个子图像第一边界信息及其对应的延展像素值信息确定所述每个子图像的第二边界信息。
  22. 根据权利要求21所述的装置，其中，所述延展像素值大小为：
    将所述子图像对应的所有卷积核中级联卷积核及并联卷积核需要延展的像素值相加，其中级联卷积核部分对应需要延展的像素值为：每个级联卷积核尺寸减1后累加，其中并联卷积核部分对应需要延展的像素值为：并联卷积核中最大尺寸的卷积核的尺寸减1。
  23. 根据权利要求21所述的装置，其中，所述装置还包括：
    滤波电路单元，用于将所述并联卷积核中非最大尺寸的卷积核针对所述延展区域的像素S3的运算结果过滤掉。
  24. 根据权利要求13至23中任一项所述的装置，其中，所述子图像卷积单元包括：
    子图像执行卷积层运算确定模块，用于根据图像切割处理策略中的卷积运算策略信息，确定切割后子图像执行的卷积层运算信息；
    子图像卷积层运算执行模块，用于对切割后的每个子图像独立、依次 执行所述卷积层运算，并获得每个子图像的卷积运算结果。
PCT/CN2020/105537 2019-07-31 2020-07-29 一种基于卷积神经网络的图像处理方法及装置 WO2021018199A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910698659.1 2019-07-31
CN201910698659.1A CN110555847B (zh) 2019-07-31 2019-07-31 一种基于卷积神经网络的图像处理方法及装置

Publications (1)

Publication Number Publication Date
WO2021018199A1 true WO2021018199A1 (zh) 2021-02-04

Family

ID=68736725

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/105537 WO2021018199A1 (zh) 2019-07-31 2020-07-29 一种基于卷积神经网络的图像处理方法及装置

Country Status (2)

Country Link
CN (1) CN110555847B (zh)
WO (1) WO2021018199A1 (zh)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110555847B (zh) * 2019-07-31 2021-04-02 瀚博半导体(上海)有限公司 一种基于卷积神经网络的图像处理方法及装置
CN111179149B (zh) * 2019-12-17 2022-03-08 Tcl华星光电技术有限公司 图像处理方法、装置、电子设备和计算机可读存储介质
CN113052292B (zh) * 2019-12-27 2024-06-04 北京硅升科技有限公司 卷积神经网络技术方法、装置及计算机可读存储介质
CN111311599B (zh) * 2020-01-17 2024-03-26 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备和存储介质
CN111008982A (zh) * 2020-03-10 2020-04-14 光子算数(北京)科技有限责任公司 基于光子神经网络芯片的数据处理方法及相关装置或设备
CN111738276A (zh) * 2020-06-19 2020-10-02 浪潮(北京)电子信息产业有限公司 基于多核卷积神经网络的图像处理方法、装置及设备
CN112231627A (zh) * 2020-10-14 2021-01-15 南京风兴科技有限公司 边界卷积计算方法、装置、计算机设备及可读存储介质
CN113642589B (zh) * 2021-08-11 2023-06-06 南方科技大学 图像特征提取方法及装置、计算机设备和可读存储介质
CN113570612B (zh) * 2021-09-23 2021-12-17 苏州浪潮智能科技有限公司 一种图像处理方法、装置及设备
CN114118395B (zh) * 2021-11-29 2023-03-28 锐宸微(上海)科技有限公司 智能处理器装置与降低存储器带宽的方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107729905A (zh) * 2017-10-19 2018-02-23 珠海格力电器股份有限公司 图像信息处理方法及装置
CN107766932A (zh) * 2017-10-19 2018-03-06 珠海格力电器股份有限公司 基于神经网络的图像处理方法及装置
CN107862316A (zh) * 2017-10-19 2018-03-30 珠海格力电器股份有限公司 卷积运算方法及装置
CN108074211A (zh) * 2017-12-26 2018-05-25 浙江大华技术股份有限公司 一种图像处理装置及方法
CN108268931A (zh) * 2016-12-30 2018-07-10 华为技术有限公司 数据处理的方法、装置和系统
US10262235B1 (en) * 2018-02-26 2019-04-16 Capital One Services, Llc Dual stage neural network pipeline systems and methods
CN110555847A (zh) * 2019-07-31 2019-12-10 瀚博半导体(上海)有限公司 一种基于卷积神经网络的图像处理方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105023023B (zh) * 2015-07-15 2018-08-17 福州大学 一种用于计算机辅助诊断的乳腺b超图像特征自学习提取方法
CN109447937B (zh) * 2017-08-29 2021-07-06 中国移动通信有限公司研究院 一种图像处理模型的确定方法及其相关装置
US10410350B2 (en) * 2017-10-30 2019-09-10 Rakuten, Inc. Skip architecture neural network machine and method for improved semantic segmentation
CN109410307B (zh) * 2018-10-16 2022-09-20 大连理工大学 一种场景点云语义分割方法
CN109584244B (zh) * 2018-11-30 2023-05-23 无锡本希奥智能技术有限公司 一种基于序列学习的海马体分割方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108268931A (zh) * 2016-12-30 2018-07-10 华为技术有限公司 数据处理的方法、装置和系统
CN107729905A (zh) * 2017-10-19 2018-02-23 珠海格力电器股份有限公司 图像信息处理方法及装置
CN107766932A (zh) * 2017-10-19 2018-03-06 珠海格力电器股份有限公司 基于神经网络的图像处理方法及装置
CN107862316A (zh) * 2017-10-19 2018-03-30 珠海格力电器股份有限公司 卷积运算方法及装置
CN108074211A (zh) * 2017-12-26 2018-05-25 浙江大华技术股份有限公司 一种图像处理装置及方法
US10262235B1 (en) * 2018-02-26 2019-04-16 Capital One Services, Llc Dual stage neural network pipeline systems and methods
CN110555847A (zh) * 2019-07-31 2019-12-10 瀚博半导体(上海)有限公司 一种基于卷积神经网络的图像处理方法及装置

Also Published As

Publication number Publication date
CN110555847A (zh) 2019-12-10
CN110555847B (zh) 2021-04-02

Similar Documents

Publication Publication Date Title
WO2021018199A1 (zh) 一种基于卷积神经网络的图像处理方法及装置
TWI748151B (zh) 神經網絡計算加速器及其執行的方法
US20210398252A1 (en) Image denoising method and apparatus
CN109871510B (zh) 二维卷积运算处理方法、系统、设备及计算机存储介质
JP7179695B2 (ja) 車線追跡方法及び装置
CN108573305B (zh) 一种数据处理方法、设备及装置
CN108664981A (zh) 显著图像提取方法及装置
JP7376720B2 (ja) 画像データ検出方法及び装置並びにコンピュータ装置及びプログラム
WO2019085709A1 (zh) 一种应用于卷积神经网络的池化处理的方法及系统
WO2021164269A1 (zh) 基于注意力机制的视差图获取方法和装置
WO2020220797A1 (zh) 特征图放大的方法、装置、设备和计算机可读存储介质
WO2019008951A1 (ja) 畳み込みニューラルネットワーク
WO2021082819A1 (zh) 一种图像生成方法、装置及电子设备
WO2019084788A1 (zh) 用于神经网络的运算装置、电路及相关方法
JP2023523047A (ja) 画像処理方法、装置、コンピュータ機器及び記憶媒体
WO2021147276A1 (zh) 数据处理方法、装置及芯片、电子设备、存储介质
US11669943B2 (en) Dual-stage system for computational photography, and technique for training same
EP3663938A1 (en) Signal processing method and apparatus
CN110009644B (zh) 一种特征图行像素分段的方法和装置
US20200327638A1 (en) Connected component detection method, circuit, device and computer-readable storage medium
CN114677286A (zh) 一种图像处理方法、装置、存储介质及终端设备
CN112861803A (zh) 一种图像识别方法、装置、服务器以及计算机可读存储介质
WO2021000495A1 (zh) 一种图像处理方法以及装置
CN111914988A (zh) 神经网络设备、计算系统和处理特征图的方法
TWI794822B (zh) 適用於物件偵測的快速非最大抑制演算法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20846973

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20846973

Country of ref document: EP

Kind code of ref document: A1