WO2021037042A1

WO2021037042A1 - Pooling processing method and apparatus, and storage medium

Info

Publication number: WO2021037042A1
Application number: PCT/CN2020/111277
Authority: WO
Inventors: 蒋燚
Original assignee: Oppo广东移动通信有限公司
Priority date: 2019-08-27
Filing date: 2020-08-26
Publication date: 2021-03-04
Also published as: CN110516793A; CN110516793B

Abstract

A pooling processing method and apparatus, and a storage medium. The method comprises: acquiring a plurality of memory addresses from a target channel of a picture to be pooled, wherein the number of the plurality of memory addresses is the same as the edge length of a pooling window (S101); successively performing interleaving access on the plurality of memory addresses, determining a plurality of groups of memory addresses corresponding to the plurality of memory addresses, and acquiring a plurality of groups of data from the plurality of groups of memory addresses, wherein one memory address from among the plurality of memory addresses corresponds to one group of memory addresses, and the number of one group of data from among the plurality of groups of data is determined by the bit width of a storage unit (S102); dividing the plurality of groups of data into a plurality of groups of data to be pooled corresponding to a plurality of channels, wherein the plurality of channels comprise the target channel, and one channel from among the plurality of channels corresponds to one group of data to be pooled (S103); and according to the pooling window, performing corresponding pooling processing on a group of data to be pooled corresponding to any channel from among the plurality of channels (S104).

Description

Pool processing method, device and storage medium

Cross-references to related applications

This application is filed based on a Chinese patent application with an application number of 201910797622.4 and an application date of August 27, 2019, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated into this application by way of introduction.

Technical field

This application relates to the field of image processing, and in particular to a pooling processing method, device, and storage medium.

Background technique

In recent years, as a neural network with self-organization, self-learning and association functions, it has been successfully applied to many aspects of image processing, such as image compression, image segmentation, edge detection, image enhancement, image recognition, etc., and pooling is neural An indispensable operation in the network, because the image has the "static" attribute, that is, the features in one image area are very likely to be equally applicable in another area, so it can be pooled by aggregation statistics of the features in different locations A way to describe an image. The pooling method can include calculating the average or maximum value of a feature in an image area. Using this pooling method can retain useful information and remove redundant information, which is conducive to subsequent extraction of effective information .

At present, there are two main types of memory layouts in neural networks: NCHW and NHWC. For the NCHW format, the exemplary memory layout is (RRRR GGGG BBBB). When the NCHW is pooled, due to its continuous memory layout, the "sliding window method" is often used, while the exemplary memory layout of the NHWC format is (RGB RGB RGB RGB), it can be seen that the multi-channel memory is not continuous, and the "sliding window method" cannot be used. It is necessary to calculate the memory address for each value to be compared. Therefore, the interval is required when pooling the NHWC. Acquiring memory, which in turn causes the hit rate of the memory cache to decrease, and the pooling speed decreases.

Summary of the invention

The embodiments of the present application provide a pooling processing method, device, and storage medium, which can increase the pooling speed.

The technical solution of this application is realized as follows:

The embodiment of the present application provides a pooling processing method, the method includes:

Acquire multiple memory addresses in the target channel of the picture to be pooled, and the number of the multiple memory addresses is the same as the side length of the pooling window, where the picture to be pooled performs memory layout according to the NHWC layout type, N Is the number of pictures, C is the number of channels, H is the height of the picture, and W is the width of the picture;

The multiple memory addresses are sequentially interleaved to determine multiple sets of memory addresses corresponding to the multiple memory addresses, and multiple sets of data are obtained from the multiple sets of memory addresses. One memory address corresponds to a set of memory addresses, and the number of one set of data in the multiple sets of data is determined by the bit width of the storage unit;

Dividing the multiple sets of data into multiple sets of data to be pooled corresponding to multiple channels, the multiple channels include the target channel, and one channel of the multiple channels corresponds to a set of data to be pooled;

According to the pooling window, corresponding pooling processing is performed on a group of data to be pooled corresponding to any one of the multiple channels.

In the above method, the step of sequentially interleaving the multiple memory addresses to determine multiple groups of memory addresses corresponding to the multiple memory addresses includes:

According to the bit width of the storage unit and the bit width of the data in the memory address, sequentially determining the first number of a group of memory addresses corresponding to the first memory address among the plurality of memory addresses;

Use the single instruction multiple data stream instruction NEON instruction to interleave the first memory address to obtain a set of memory addresses of the first quantity;

The multiple sets of memory addresses are determined according to multiple sets of first quantity data corresponding to the multiple memory addresses.

In the above method, after dividing the multiple sets of data into multiple sets of data to be pooled corresponding to multiple channels, the method further includes:

The multiple sets of data to be pooled corresponding to the multiple channels are respectively stored in multiple storage units, and one storage unit of the multiple storage units stores a set of data to be pooled corresponding to one channel.

In the above method, the performing corresponding pooling processing on a group of data to be pooled corresponding to any one of the multiple channels according to the pooling window includes:

Determining target pooling data from the set of data to be pooled according to the size of the pooling window;

Perform corresponding pooling processing on the target pooling data.

In the above method, the size of the pooling window is smaller than or equal to the storage capacity of the memory address of the storage unit.

In the above method, the storage unit is a register, and according to the bit width of the storage unit and the bit width of the data in the memory address, a group of memory corresponding to the first memory address among the plurality of memory addresses is sequentially determined. The first number of addresses, including:

According to the bit width of the register and the bit width of the data in the memory address, determine the first amount of data stored in the memory address at a time by the register;

Store a group of data in a group of memory addresses of a channel cross read according to the first memory address into a register.

In the above method, the interleaved access to the first memory address using the single instruction, multiple data stream SIMD extended structure NEON instruction to obtain a set of memory addresses of the first quantity includes:

Use NEON instruction interleaving vld3q_f32 interleaving to read a group of memory addresses of the first memory address in multiple channels;

Correspondingly, the method further includes:

Get a set of data from a set of memory addresses.

In the above method, the storage unit is a register; the register stores a memory address or stores specific data in the memory address.

In the above method, the pooling processing includes: maximum pooling processing and average pooling processing; and performing corresponding pooling processing on the target pooling data includes:

Maximum pooling or average pooling is performed on the target pooled data, and data other than the target pooled data in a set of to-be-pooled data is eliminated.

An embodiment of the application provides a pooling processing device, and the device includes:

The acquiring part is configured to acquire multiple memory addresses in the target channel of the picture to be pooled, and the number of the multiple memory addresses is the same as the side length of the pooling window, wherein the picture to be pooled is in accordance with the NHWC layout type Perform memory layout, N is the number of pictures, C is the number of channels, H is the height of the picture, and W is the width of the picture;

The interleaving part is configured to sequentially interleave the multiple memory addresses, determine multiple sets of memory addresses corresponding to the multiple memory addresses, and obtain multiple sets of data from the multiple sets of memory addresses, so One memory address in the multiple memory addresses corresponds to a group of memory addresses, and the number of one group of data in the multiple groups of data is determined by the bit width of the storage unit;

The dividing part is configured to divide the multiple sets of data into multiple sets of data to be pooled corresponding to multiple channels, the multiple channels include the target channel, and one channel of the multiple channels corresponds to a group of to be pooled化 data;

The pooling part is configured to perform corresponding pooling processing on a group of data to be pooled corresponding to any one of the multiple channels according to the pooling window.

In the above device, the device further includes: a determining part;

The determining part is configured to sequentially determine the first number of a group of memory addresses corresponding to the first memory address among the plurality of memory addresses according to the bit width of the storage unit and the bit width of the data in the memory address; according to Multiple sets of data of the first quantity corresponding to the multiple memory addresses determine the multiple sets of memory addresses;

The interleaving part is also configured to use a single instruction multiple data stream instruction NEON instruction to perform interleaving access to the first memory address to obtain a set of memory addresses of the first quantity.

In the above device, the device further includes: a storage part;

The storage part is configured to store the multiple sets of data to be pooled corresponding to the multiple channels in multiple storage units, and one storage unit of the multiple storage units stores a set of to-be-pooled data corresponding to one channel. Pooled data.

In the above device, the determining part is further configured to determine the target pooling data from the set of data to be pooled according to the size of the pooling window;

The pooling part is further configured to perform corresponding pooling processing on the target pooling data.

In the above device, the size of the pooling window is smaller than or equal to the storage capacity of the memory address of the storage unit.

In the above device, the storage unit is a register,

The determining part is further configured to determine the first amount of data stored in the memory address in the register at one time according to the bit width of the register and the bit width of the data in the memory address;

The storage part is also configured to store a group of data in a group of memory addresses of a channel that is cross-read according to the first memory address into a register.

In the above-mentioned device, the interleaving part is further configured to interleave a group of memory addresses in a plurality of channels of the first memory address by using the interleaving vld3q_f32 of the NEON instruction;

The acquiring part is also configured to acquire a group of data from a group of memory addresses.

In the above device, the storage unit is a register; the register stores a memory address or stores specific data in the memory address.

In the above device, the pooling processing includes: maximum pooling processing and average pooling processing,

The pooling part is also configured to perform maximum pooling or average pooling on the target pooled data, and remove data from a group of to-be-pooled data except for the target pooled data.

An embodiment of the present application provides a pooling processing device. The pooling processing device includes a processor, a memory, and a communication bus; when the processor executes an operating program stored in the memory, the method according to any one of the above is implemented.

The embodiment of the present application provides a storage medium on which a computer program is stored, which is applied to a pooling processing device, and when the computer program is executed by a processor, the method as described in any one of the above is implemented.

The embodiments of the present application provide a pooling processing method and device, and a storage medium. The method includes: acquiring multiple memory addresses in a target channel of a picture to be pooled, the number of multiple memory addresses, and the margins of the pooling window. The length is the same, where the pictures to be pooled are arranged in memory according to the NHWC layout type, N is the number of pictures, C is the number of channels, H is the picture height, and W is the picture width; multiple memory addresses are sequentially interleaved to determine Multiple sets of memory addresses correspond to multiple sets of memory addresses, and multiple sets of data are obtained from multiple sets of memory addresses. One memory address of multiple sets of memory addresses corresponds to one set of memory addresses. The number of sets of data in multiple sets of data is determined by The bit width of the storage unit is determined; multiple sets of data are divided into multiple sets of data to be pooled corresponding to multiple channels, multiple channels include the target channel, and one channel of the multiple channels corresponds to a set of data to be pooled; according to pooling Window, a group of data to be pooled corresponding to any one of the multiple channels is pooled accordingly. Using the above implementation scheme, multiple memory addresses are read by interleaving to obtain multiple sets of memory addresses. After that, the multiple data in the multiple sets of memory addresses are divided into multiple sets of data to be pooled corresponding to multiple channels , One-time interleaving can obtain the data to be pooled for pooling, thereby improving the hit rate and pooling speed of the memory cache.

Description of the drawings

FIG. 1 is a first flowchart of a pooling processing method provided by an embodiment of the application;

FIG. 2 is an exemplary schematic diagram of obtaining three memory addresses in the first column of the R channel according to an embodiment of the application;

FIG. 3 is an exemplary schematic diagram of interleaving A1 according to an embodiment of the application;

FIG. 4 is an exemplary schematic diagram of interleaving A1, A2, and A3 provided by an embodiment of the application;

FIG. 5 is an exemplary schematic diagram of performing maximum processing on 12 values of 3×4 of the R channel according to an embodiment of the application; FIG.

FIG. 6 is an exemplary schematic diagram of setting the value of the fourth digit to the minimum value provided by an embodiment of the application; FIG.

FIG. 7 is a first structural diagram of a pooling processing device provided by an embodiment of the application;

FIG. 8 is a second structural diagram of a pooling processing device provided by an embodiment of the application.

detailed description

It should be understood that the specific embodiments described here are only used to explain the present application. It is not used to limit this application.

Taking the pooling window size of 3×3 and the color channel of the three color channels of RGB as an example, the process of pooling the existing NHWC picture with the memory layout is: determine the first from the first column of the R channel The memory addresses of row A1, the second row A2 and the third row A3, after that, the memory address of A1 is increased by 3, and the memory address of A1 is increased by 6 to determine the two memory addresses in the same line as A1 in the R channel; Add 3 to the memory address, add 6 to the memory address of A2 to determine the two memory addresses in the R channel that are in line with A2; add 3 to the memory address of A3, and add 6 to the memory address of A3 to determine the R channel that is in line with A3 Two memory addresses. At this time, a 3×3 memory address window is obtained on the R channel, and the 3×3 data of the corresponding area is obtained from the 3×3 memory address window, and the 3×3 data is processed Maximum pooling or average pooling operation, the above process is also applicable to G channel and B channel. The existing process of pooling a color channel involves calculating the address of each data, which results in a very slow pooling speed. Therefore, the solution of the present application is proposed, and the solution of the present application is described in detail below.

Example one

The embodiment of the present application provides a pooling processing method. As shown in FIG. 1, the method may include:

S101. Obtain multiple memory addresses in the target channel of the picture to be pooled, and the number of the multiple memory addresses is the same as the side length of the pooling window, where the picture to be pooled is arranged in memory according to the NHWC layout type, and N is the picture Quantity, C is the number of channels, H is the height of the picture, and W is the width of the picture.

The pooling processing method provided in the embodiments of the present application is applicable to a scenario where a pooling processing device performs pooling processing on pictures.

In the embodiment of the present application, the pictures to be pooled are stored according to the memory layout of the NHWC dimension. When the pictures to be pooled need to be pooled, the pooling processing device determines the amount to be acquired according to the side length of the pooling window. The number of memory addresses.

Exemplarily, when the preset window is 3×3, the pooling processing device determines to acquire 3 memory addresses in the target channel of the picture to be pooled.

In the embodiment of this application, the target channel includes R channel, G channel, B channel or other color channels, which are specifically selected according to actual conditions. The embodiment of this application does not make specific limitations, and is the R channel in actual applications.

Exemplarily, as shown in Fig. 2, the picture to be pooled includes data of three color channels of R channel, G channel and B channel, and the pooling processing device needs to use the pooling window to pool the picture to be pooled to 3×3. Operation, at this time, the pooling processing device is on the R color channel, starting from the upper right corner to frame the data of the pooling window size, and obtaining the three memory addresses A1, A2, and A3 in the first column of the pooling window.

In the embodiment of the present application, it is assumed that a 3×3 pooling window is used to perform the pooling operation, and the multiple memory addresses are A1, A2, and A3. The pooling processing device uses formula (1) to obtain the memory address of A1 and uses the formula ( 2) Obtain the memory address of A2 and use formula (3) to obtain the memory address of A3.

A1=T+2*channelNums (1)

A2=T+(kernelWidth+2)*channelNums (2)

A3=T+(kernelWidth+kernelWidth+2)*channelNums (3)

Among them, kernelWidth is the width of the pooling window, with a value of 3; channelNums is the number of channels, with a value of 3; T is the starting address.

S102. Interleave multiple memory addresses in sequence, determine multiple sets of memory addresses corresponding to the multiple memory addresses, and obtain multiple sets of data from the multiple sets of memory addresses, and one memory address of the multiple memory addresses corresponds to one set The memory address, the number of a group of data in multiple groups of data is determined by the bit width of the storage unit.

After the pooling processing device obtains multiple memory addresses in the target channel of the picture to be pooled, the pooling processing device sequentially interleaves the multiple memory addresses to determine multiple sets of memory addresses corresponding to the multiple memory addresses , And obtain multiple sets of data from multiple sets of memory addresses.

In the embodiment of the present application, the pooling processing device sequentially determines the first number of a group of memory addresses corresponding to the first memory address among the multiple memory addresses according to the bit width of the storage unit and the bit width of the data in the memory address; The instruction multi-data stream SIMD extension structure NEON instruction interleaves the first memory address to obtain a group of the first number of memory addresses; multiple groups are determined according to the multiple groups of the first number of data corresponding to the multiple memory addresses Memory address.

In the embodiment of the present application, the storage unit is a register.

In the embodiment of the present application, the pooling processing device determines the first amount of data stored in the memory address at a time according to the bit width of the register and the bit width of the data in the memory address, and the pooling processing device will interleave according to the first memory address. A set of data in a set of memory addresses of a channel read is stored in a register. Therefore, corresponding to the three channels of RGB, three registers are required to store the three cross-reads according to the first memory address. Three sets of data for each channel.

Exemplarily, when a 128-bit register is used, it is determined that the data involved in the operation is a 32-bit floating point number, and the bit width of the register is divided by the bit width of the data in the memory address, that is, 128 is divided by 32, and the register stores 4 at a time The data in the memory address.

In the embodiment of the present application, the pooling processing device uses the interleaving vld3q_f32 of the NEON instruction to interleave a group of memory addresses of the first memory address in multiple channels, and obtain a group of data from the group of memory addresses.

S103. Divide the multiple sets of data into multiple sets of data to be pooled corresponding to multiple channels, the multiple channels include the target channel, and one channel of the multiple channels corresponds to a set of data to be pooled.

After the pooling processing device determines multiple sets of memory addresses corresponding to the multiple memory addresses, and obtains multiple sets of data from the multiple sets of memory addresses, the pooling processing device divides the multiple sets of data into multiple sets of waiting pools corresponding to multiple channels化数据。 Data.

In the embodiment of this application, since the multiple sets of data obtained by the pooling processing device interleaving multiple memory addresses are data in multiple channels, the pooling processing device sequentially divides the data in the multiple sets of data according to the multiple channels. A group of data is divided into a group of data to be pooled.

Exemplarily, as shown in FIG. 3, when the pooling processing device determines the memory address of A1, the pooling processing device uses the interleaving vld3q_f32 of the NEON instruction to read the four data from left to right in the first row of the RGB channel. , And store the four data in the R channel in the R1 register, namely R1(1,1,1,1), and store the four data in the G channel in the R2 register, namely R2(2,2,2, 2), store the four data in the B channel in the R3 register, namely R3(3,3,3,3); among them, when the memory address of A1 is 0, the 12 values read across in the NHWC format The address arrangement in the memory is {0,1,2,3,4,5,6,7,8,9,10,11}, and the arrangement order of the corresponding 12 values is {1,2,3 ,1,2,3,1,2,3,1,2,3}. Therefore, the addresses of the four values in the R1 register are: {0,3,6,9}, the four values in the R2 register are {1,4,7,10}, and the four values in the R3 register are {2,5,8, 11}.

Further, after the pooling processing device divides the multiple sets of data into multiple sets of data to be pooled corresponding to the multiple channels, the pooling processing device stores the multiple sets of data to be pooled corresponding to the multiple channels into multiple storage units, respectively , One storage unit among the multiple storage units stores a group of data to be pooled corresponding to one channel.

It should be noted that, for the register, a memory address or specific data in a memory address can be stored, and the specific selection is made according to actual conditions, and the embodiment of the present application does not make specific limitations.

Exemplarily, as shown in FIG. 4, the three memory addresses of A1, A2, and A3 are respectively interleaved to obtain 12*3 values corresponding to the RGB channels.

S104: According to the pooling window, perform corresponding pooling processing on a group of data to be pooled corresponding to any one of the multiple channels.

After the pooling processing device divides multiple sets of data into multiple sets of data to be pooled corresponding to multiple channels, the pooling processing device performs a set of data to be pooled corresponding to any one of the multiple channels according to the pooling window. Carry out the corresponding pooling treatment.

In the embodiment of the present application, the pooling processing device determines the target pooling data from a set of data to be pooled according to the size of the pooling window; after that, the pooling processing device performs corresponding pooling processing on the target pooling data, Among them, pooling processing includes maximum pooling processing and average pooling processing.

In the embodiment of the present application, the pooling processing device defines the target pooling data according to the size of the pooling window from a group of memory addresses to be pooled, and performs maximum pooling or average pooling on the target pooling data, and then Group the data to be pooled except the target pooled data for elimination processing.

Optionally, when the maximum pooling process is performed on a group of to-be-pooled memory addresses, the data in a group of to-be-pooled data except the target pooled data is set to the minimum value. In this case, filter when calculating the maximum value. A set of data to be pooled except the target pooled data is dropped.

Exemplarily, as shown in FIG. 5, for the 12 values of 3×4 acquired by the R channel, the pooling window size for the maximum pooling operation is 3×3, and the pooling processing device sets each row of The 4-bit value is set to the minimum value, -max, as shown in Figure 6, and then the maximum value vmaxq_f32 is calculated for the 3×3 value, and the result is 5.

In the embodiment of the present application, a register stores data in a memory address of a channel, and the pooling processing device performs pooling processing on the data corresponding to the memory address in each register to obtain a pooling result corresponding to a channel.

In the embodiment of the present application, the size of the pooling window is less than or equal to the storage capacity of the storage memory address of the storage unit, so that a pooled data greater than or equal to the pooling window can be obtained, and the corresponding pooling operation can be completed.

It is understandable that multiple memory addresses are read by interleaving to obtain multiple sets of memory addresses, and then multiple sets of data in multiple sets of memory addresses are divided into multiple sets of data to be pooled corresponding to multiple channels , One-time interleaving can obtain the data to be pooled for pooling, thereby improving the hit rate and pooling speed of the memory cache.

Example two

The embodiment of the present application provides a pooling processing device 1. As shown in FIG. 7, the pooling processing device 1 may include:

The acquiring part 10 is configured to acquire multiple memory addresses in the target channel of the picture to be pooled, the number of the multiple memory addresses is the same as the side length of the pooling window, wherein the picture to be pooled is laid out according to NHWC Type for memory layout, N is the number of pictures, C is the number of channels, H is the height of the picture, and W is the width of the picture;

The interleaving part 11 is configured to sequentially interleave the multiple memory addresses, determine multiple sets of memory addresses corresponding to the multiple memory addresses, and obtain multiple sets of data from the multiple sets of memory addresses, One memory address in the multiple memory addresses corresponds to a group of memory addresses, and the number of a group of data in the multiple groups of data is determined by the bit width of the storage unit;

The dividing part 12 is configured to divide the multiple sets of data into multiple sets of data to be pooled corresponding to multiple channels, the multiple channels include the target channel, and one channel of the multiple channels corresponds to a set of data to be pooled. Pooled data;

The pooling part 13 is configured to perform corresponding pooling processing on a group of data to be pooled corresponding to any one of the multiple channels according to the pooling window.

Optionally, the device further includes: a determining part 14;

The determining part 14 is configured to sequentially determine the first number of a group of memory addresses corresponding to the first memory address among the plurality of memory addresses according to the bit width of the storage unit and the bit width of the data in the memory address; Determine the multiple sets of memory addresses according to multiple sets of first quantity data corresponding to the multiple memory addresses;

The interleaving part 11 is also configured to use a single instruction multiple data stream instruction NEON instruction to perform interleaving access to the first memory address to obtain a set of memory addresses of the first quantity.

Optionally, the device further includes: a storage part 15;

The storage part 15 is configured to store the plurality of groups of data to be pooled corresponding to the plurality of channels into a plurality of storage units, and one storage unit of the plurality of storage units stores a group corresponding to one channel Data to be pooled.

Optionally, the determining part 14 is further configured to determine target pooling data from the set of data to be pooled according to the size of the pooling window;

The pooling part 13 is further configured to perform corresponding pooling processing on the target pooling data.

Optionally, the size of the pooling window is smaller than or equal to the storage capacity of the memory address of the storage unit. The storage unit is a register,

Optionally, the determining part 14 is further configured to determine the first amount of data stored in the memory address in the register at one time according to the bit width of the register and the bit width of the data in the memory address;

The storage part 15 is also configured to store a group of data in a group of memory addresses of a channel that is cross-read according to the first memory address into a register.

Optionally, the interleaving part 11 is further configured to interleave a group of memory addresses of the first memory address in multiple channels by using the interleaving vld3q_f32 of the NEON instruction;

The acquiring part 10 is also configured to acquire a group of data from a group of memory addresses.

Optionally, the storage unit is a register; the register stores a memory address or stores specific data in the memory address.

Optionally, the pooling processing includes: maximum pooling processing and average pooling processing,

The pooling part 13 is also configured to perform maximum pooling or average pooling on the target pooled data, and remove data from a group of to-be-pooled data except for the target pooled data.

An embodiment of the present application provides a pooling processing device that acquires multiple memory addresses in a target channel of a picture to be pooled, and the number of the multiple memory addresses is the same as the side length of the pooling window, where the picture to be pooled The memory layout is carried out according to the NHWC layout type, N is the number of pictures, C is the number of channels, H is the picture height, and W is the picture width; multiple memory addresses are sequentially interleaved to determine multiple groups of memory corresponding to multiple memory addresses Address, and obtain multiple sets of data from multiple sets of memory addresses. One memory address in the multiple memory addresses corresponds to a set of memory addresses. The number of a set of data in the multiple sets of data is determined by the bit width of the storage unit; The group data is divided into multiple groups of data to be pooled corresponding to multiple channels. The multiple channels include the target channel, and one channel of the multiple channels corresponds to a group of data to be pooled; according to the pooling window, any of the multiple channels is A group of data to be pooled corresponding to a channel undergoes corresponding pooling processing. It can be seen that the pooling processing device proposed in this embodiment uses interleaving to read multiple memory addresses to obtain multiple sets of memory addresses, and then divide the multiple data in the multiple sets of memory addresses into multiple For multiple sets of data to be pooled corresponding to the channel, one interleaved access can obtain the data to be pooled required for pooling, thereby improving the hit rate and pooling speed of the memory cache.

FIG. 8 is a second schematic diagram of the composition structure of a pooling processing device 1 provided by an embodiment of the application. In practical applications, based on the same disclosure concept of the above-mentioned embodiment, as shown in FIG. 8, the pooling processing of this embodiment is The device 1 includes a processor 16, a memory 17 and a communication bus 18.

In the process of a specific embodiment, the acquisition unit 10, the interleaving unit 11, the dividing unit 12, the pooling unit 13, and the determining unit 14 may be implemented by the processor 16 located on the pooling processing device 1, and the storage unit 15 may be The memory 17 located on the pooling processing device 1 is implemented. The above-mentioned processor 16 may be an Application Specific Integrated Circuit (ASIC), a digital signal processor (DSP, Digital Signal Processor), or a digital signal processing pooling processing device. (DSPD, Digital Signal Processing Device), Programmable Logic Pooling Processing Device (PLD, Programmable Logic Device), Field Programmable Gate Array (FPGA, Field Programmable Gate Array), CPU, controller, microcontroller, microprocessor At least one of them. It can be understood that, for different devices, the electronic devices used to implement the above-mentioned processor functions may also be other, which is not specifically limited in this embodiment.

In the embodiment of the present application, the above-mentioned communication bus 18 is used to realize the connection and communication between the processor 16 and the memory 17; when the above-mentioned processor 16 executes the operating program stored in the memory 17, the pooling process as described in the first embodiment is implemented. method.

The embodiments of the present application provide a storage medium on which a computer program is stored, and the above-mentioned computer-readable storage medium stores one or more programs, and the above-mentioned one or more programs can be executed by one or more processors and applied to the pool. In the chemical processing device, the computer program implements the pool processing method as described in the first embodiment.

The above are only preferred embodiments of the present application, and are not used to limit the protection scope of the present application.

Industrial applicability

Claims

A pooling processing method, the method includes:

Acquire multiple memory addresses in the target channel of the picture to be pooled, and the number of the multiple memory addresses is the same as the side length of the pooling window, where the picture to be pooled performs memory layout according to the NHWC layout type, N Is the number of pictures, C is the number of channels, H is the height of the picture, and W is the width of the picture;

The multiple memory addresses are sequentially interleaved to determine multiple sets of memory addresses corresponding to the multiple memory addresses, and multiple sets of data are obtained from the multiple sets of memory addresses. One memory address corresponds to a set of memory addresses, and the number of one set of data in the multiple sets of data is determined by the bit width of the storage unit;

Dividing the multiple sets of data into multiple sets of data to be pooled corresponding to multiple channels, the multiple channels include the target channel, and one channel of the multiple channels corresponds to a set of data to be pooled;

According to the pooling window, corresponding pooling processing is performed on a group of data to be pooled corresponding to any one of the multiple channels.
The method according to claim 1, wherein the sequentially interleaving the multiple memory addresses to determine multiple groups of memory addresses corresponding to the multiple memory addresses comprises:

According to the bit width of the storage unit and the bit width of the data in the memory address, sequentially determining the first number of a group of memory addresses corresponding to the first memory address among the plurality of memory addresses;

Use single instruction, multiple data stream SIMD extended structure NEON instruction to interleave the first memory address to obtain a set of memory addresses of the first quantity;

The multiple sets of memory addresses are determined according to multiple sets of first quantity data corresponding to the multiple memory addresses.
The method according to claim 1 or 2, wherein after the dividing the multiple sets of data into multiple sets of data to be pooled corresponding to multiple channels, the method further comprises:

The multiple sets of data to be pooled corresponding to the multiple channels are respectively stored in multiple storage units, and one storage unit of the multiple storage units stores a set of data to be pooled corresponding to one channel.
The method according to claim 1, wherein, according to the pooling window, performing corresponding pooling processing on a group of data to be pooled corresponding to any one of the plurality of channels comprises:

Determining target pooling data from the set of data to be pooled according to the size of the pooling window;

Perform corresponding pooling processing on the target pooling data.
The method according to claim 1 or 4, wherein the size of the pooling window is smaller than or equal to the storage capacity of the memory address of the storage unit.
The method according to claim 2, wherein the storage unit is a register, and the first memory address among the plurality of memory addresses is sequentially determined according to the bit width of the storage unit and the bit width of the data in the memory address. The first number of a group of memory addresses corresponding to the address, including:

According to the bit width of the register and the bit width of the data in the memory address, determine the first amount of data stored in the memory address at a time by the register;

Store a group of data in a group of memory addresses of a channel cross-read according to the first memory address into a register.
The method according to claim 2, wherein said using a single instruction, multiple data stream SIMD extended structure NEON instruction to interleave the first memory address to obtain a set of memory addresses of the first quantity comprises:

Use NEON instruction interleaving vld3q_f32 interleaving to read a group of memory addresses of the first memory address in multiple channels;

Correspondingly, the method further includes:

Get a set of data from a set of memory addresses.
The method according to claim 3, wherein the storage unit is a register; the register stores a memory address or stores specific data in the memory address.
The method according to claim 4, wherein the pooling processing includes: maximum pooling processing and average pooling processing; and performing corresponding pooling processing on the target pooling data includes:

Maximum pooling or average pooling is performed on the target pooled data, and data other than the target pooled data in a set of to-be-pooled data is eliminated.
A pooling treatment device, the device comprising:

The acquiring part is configured to acquire multiple memory addresses in the target channel of the picture to be pooled, and the number of the multiple memory addresses is the same as the side length of the pooling window, wherein the picture to be pooled is in accordance with the NHWC layout type Perform memory layout, N is the number of pictures, C is the number of channels, H is the height of the picture, and W is the width of the picture;

The interleaving part is configured to sequentially interleave the multiple memory addresses, determine multiple sets of memory addresses corresponding to the multiple memory addresses, and obtain multiple sets of data from the multiple sets of memory addresses, so One memory address in the multiple memory addresses corresponds to a group of memory addresses, and the number of one group of data in the multiple groups of data is determined by the bit width of the storage unit;

The dividing part is configured to divide the multiple sets of data into multiple sets of data to be pooled corresponding to multiple channels, the multiple channels include the target channel, and one channel of the multiple channels corresponds to a group of to be pooled化 data;

The pooling part is configured to perform corresponding pooling processing on a group of data to be pooled corresponding to any one of the multiple channels according to the pooling window.
The device according to claim 10, wherein the device further comprises: a determining part;

The determining part is configured to sequentially determine the first number of a group of memory addresses corresponding to the first memory address among the plurality of memory addresses according to the bit width of the storage unit and the bit width of the data in the memory address; according to Multiple sets of data of the first quantity corresponding to the multiple memory addresses determine the multiple sets of memory addresses;

The interleaving part is also configured to use the NEON instruction to interleave the first memory address to obtain a set of memory addresses of the first number.
The device according to claim 10 or 11, wherein the device further comprises: a storage part;

The storage part is configured to store the multiple sets of data to be pooled corresponding to the multiple channels in multiple storage units, and one storage unit of the multiple storage units stores a set of to-be-pooled data corresponding to one channel. Pooled data.
The device according to claim 10, wherein:

The determining part is further configured to determine target pooling data from the group of data to be pooled according to the size of the pooling window;

The pooling part is further configured to perform corresponding pooling processing on the target pooling data.
The device according to claim 10 or 13, wherein the size of the pooling window is smaller than or equal to the storage capacity of the memory address of the storage unit.
The device according to claim 10, wherein the storage unit is a register,

The determining part is further configured to determine the first amount of data stored in the memory address in the register at one time according to the bit width of the register and the bit width of the data in the memory address;

The storage part is also configured to store a group of data in a group of memory addresses of a channel that is cross-read according to the first memory address into a register.
The device according to claim 11, wherein:

The interleaving part is also configured to use the interleaving vld3q_f32 of the NEON instruction to interleavingly read a group of memory addresses of the first memory address in multiple channels;

The acquiring part is also configured to acquire a group of data from a group of memory addresses.
The device according to claim 12, wherein the storage unit is a register; the register stores a memory address or stores specific data in the memory address.
The device according to claim 13, wherein the pooling processing comprises: maximum pooling processing and average pooling processing,

The pooling part is also configured to perform maximum pooling or average pooling on the target pooled data, and remove data from a group of to-be-pooled data except for the target pooled data.
A pooling processing device, the pooling processing device comprising: a processor, a memory, and a communication bus; when the processor executes an operating program stored in the memory, the method according to any one of claims 1-9 is implemented.
A storage medium on which a computer program is stored, and should be configured as a pooling processing device. When the computer program is executed by a processor, the method according to any one of claims 1-9 is realized.