WO2024051298A1 - Image processing method, apparatus, and computing device - Google Patents

Abstract

An image processing method, comprising: acquiring a first image; for any pixel point in the first image, determining an image block corresponding to any pixel point by using a plurality of lookup tables associated with any pixel point, the image block comprising a single pixel point or a plurality of pixel points, the index area of each lookup table comprising any pixel point, and the index areas of different lookup tables being different and at least partially the same; according to the image block corresponding to each pixel point in the first image, obtaining a second image, the resolution of the second image being higher than or equal to the resolution of the first image; and outputting the second image. Therefore, during the image processing process, for each pixel point, a plurality of lookup tables are used for cooperative work, so that the receptive field is expanded while maintaining linear growth of computing amount and storage amount, thereby improving the image processing performance.

Description

An image processing method, device and computing equipment

This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 9, 2022, with application number 202211102553.9 and the application title "An image processing method, device and computing device", the entire content of which is incorporated by reference. incorporated in this application.

Technical field

This application relates to the technical field of artificial intelligence (AI), and in particular to an image processing method, device and computing equipment.

Background technique

Image processing is the main research direction of underlying computer vision. In recent years, methods based on convolutional neural networks (CNN) have achieved impressive results in image processing performance. Generally, more complex neural networks can achieve better results. However, due to limitations of operating speed and power consumption, the network for image processing on terminal devices with weak computing capabilities such as mobile phones generally cannot be too complex. Although this constraint can be alleviated through a carefully designed network structure or a dedicated computing engine (such as a graphics processor), the corresponding hardware cost and power consumption still increase the difficulty of deploying existing CNN methods on terminal devices. On the other hand, on terminal devices such as smart cars, application scenarios such as lane changing during high-speed driving and handling emergencies have higher requirements on the running delay of image processing methods. Therefore, how to design an efficient image processing method that meets the requirements of low power consumption and low latency when deployed in terminal equipment, while having good image processing performance, is an urgent technical problem that needs to be solved.

Contents of the invention

This application provides an image processing method, device, computing equipment, computing equipment cluster, computer storage medium and computer product, which can meet the requirements of low power consumption and low delay when deployed in terminal equipment, and at the same time have better image processing performance.

In a first aspect, this application provides an image processing method, which method includes: acquiring a first image; for any pixel point in the first image, using N (N≥2) searches associated with any one pixel point. table to determine the image block corresponding to any pixel point. The image block includes single or multiple pixel points, where the index area of each lookup table contains the any one pixel point, and the index areas of different lookup tables Different and at least partially the same; obtain a second image based on the image blocks corresponding to each pixel point in the first image, wherein the resolution of the second image is higher than or equal to the resolution of the first image; and output the second image.

In this way, during the image processing process, multiple lookup tables are used to work together for each pixel, thereby expanding the receptive field and improving image processing performance while maintaining linear growth in the amount of calculation and storage. In contrast, existing lookup table technology that utilizes spatial domain information only uses a single lookup table. The expansion of the receptive field will increase the storage capacity exponentially and will be limited by memory scale and hardware cost.

In a possible implementation, N lookup tables associated with any one pixel are used to determine the image block corresponding to the any one pixel, which specifically includes: converting the index area of each lookup table in the N lookup tables Input it into the corresponding lookup table to obtain N first sub-image blocks. Each first sub-image block includes single or multiple pixels; the pixels at the same position in the N first sub-image blocks are evenly phased. Add and average to obtain the image block corresponding to any pixel. Since the index area corresponding to the output result of the lookup table increases, the receptive field of the lookup table will also increase accordingly. At the same time, the dimensions of the lookup table have not changed, so the receptive field of the lookup table can be increased without increasing the dimensions of the lookup table.

In one possible implementation, the index area of each lookup table in the N lookup tables is input into the corresponding lookup table to obtain N first sub-image blocks, which specifically includes: targeting any one of the N lookup tables For the index area of the lookup table, center the index area of any lookup table on any pixel and rotate it M times along the first direction to obtain M index areas, M≥1; based on each of the M index areas Lookup tables corresponding to the index areas determine the second sub-image blocks corresponding to each of the M index areas, so as to obtain M second sub-image blocks, each second sub-image block including single or multiple pixels; Based on the rotation angles corresponding to each of the M index areas, rotate the second sub-image blocks corresponding to each of the M index areas by the same angle in the direction opposite to the first direction to obtain M third sub-image blocks; Add and average the pixels at the same position in the first sub-image block and the M third sub-image blocks corresponding to the index area of any look-up table, and use the calculation result as the index of any look-up table The first sub-image block corresponding to the area. Thus, by rotating the index area, the receptive field of the lookup table is further increased without increasing the dimension of the lookup table.

In a possible implementation, the output of the i-th lookup table among the N lookup tables is the input of the (i+1)-th lookup table, and the input of the 1st lookup table among the N lookup tables is the 1st The index area of the lookup table, the output of the Nth lookup table among the N lookup tables is any image The image block corresponding to the prime point. That is to say, N lookup tables can be cascaded, thereby increasing the receptive field of the lookup tables.

In a possible implementation, N lookup tables are composed of lookup tables included in M lookup table sets, and each lookup table set includes multiple lookup tables. The inputs of each lookup table in the jth lookup table set are respectively the index areas of the corresponding lookup tables in the jth lookup table set determined based on the output of the (j-1)th lookup table set. The input of each lookup table in the first lookup table set is the index area of the corresponding lookup table in the first lookup table set determined based on any of the aforementioned pixels. The output of the M-th lookup table set is the image block corresponding to any of the aforementioned pixels. In this way, the image blocks corresponding to each pixel are determined through the complementary and cascaded manner of multiple lookup tables, further improving image processing performance.

In a second aspect, this application provides an image processing device, which includes: an acquisition module and a processing module. Wherein, the acquisition module is used to acquire the first image. The processing module is configured to use N (N≥2) lookup tables associated with any pixel point in the first image to determine the image block corresponding to the any pixel point, and the image block includes Single or multiple pixels, wherein the index area of each lookup table contains any one pixel, and the index areas of different lookup tables are different and at least partially the same. The processing module is also configured to obtain a second image based on the image blocks corresponding to each pixel point in the first image, where the resolution of the second image is higher than or equal to the resolution of the first image; and output the second image.

In a possible implementation, when the processing module uses N lookup tables associated with any one pixel point to determine the image block corresponding to the any one pixel point, it is specifically used to: convert each of the N lookup tables The index area of the lookup table is input into the corresponding lookup table, and N first sub-image blocks are obtained. Each first sub-image block includes single or multiple pixels; the same position in the N first sub-image blocks is obtained. The pixels are added together and averaged to obtain the image block corresponding to any pixel.

In a possible implementation, when the processing module inputs the index area of each lookup table in the N lookup tables into the corresponding lookup table to obtain N first sub-image blocks, it is specifically used to: The index area of any lookup table in the lookup table is centered on any pixel and rotated M times along the first direction to obtain M index areas, M≥1; based on M The lookup table corresponding to each index area in the index areas determines the second sub-image block corresponding to each of the M index areas, so as to obtain M second sub-image blocks. Each second sub-image block includes multiple Pixel points; based on the rotation angles corresponding to each of the M index areas, rotate the second sub-image blocks corresponding to each of the M index areas by the same angle in the direction opposite to the first direction to obtain M third sub-image blocks. Image block; add and average the pixel points at the same position in the first sub-image block corresponding to the index area of any lookup table and the M third sub-image blocks, and use the calculation result as the any one Look up the first sub-image block corresponding to the index area of the table.

In a possible implementation, the output of the i-th lookup table among the N lookup tables is the input of the (i+1)-th lookup table, and the input of the 1st lookup table among the N lookup tables is the 1st The index area of the lookup table. The output of the Nth lookup table among the N lookup tables is the image block corresponding to any pixel.

In a possible implementation, N lookup tables are composed of lookup tables included in M lookup table sets, and each lookup table set includes multiple lookup tables. The inputs of each lookup table in the jth lookup table set are respectively the index areas of the corresponding lookup tables in the jth lookup table set determined based on the output of the (j-1)th lookup table set. The input of each lookup table in the first lookup table set is the index area of the corresponding lookup table in the first lookup table set determined based on any of the aforementioned pixels. The output of the M-th lookup table set is the image block corresponding to any of the aforementioned pixels.

In a third aspect, the present application provides a computing device, including: at least one memory for storing a program; at least one processor for executing the program stored in the memory; wherein, when the program stored in the memory is executed, the processor is configured to Perform the method described in the first aspect or any possible implementation of the first aspect.

In a fourth aspect, the present application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory. The processor of at least one computing device is configured to execute instructions stored in the memory of at least one computing device, so that the cluster of computing devices executes the method described in the first aspect or any possible implementation of the first aspect.

In a fifth aspect, the application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is run on a processor, it causes the processor to execute the first aspect or any one of the first aspects. Possible implementations of the methods described.

In a sixth aspect, the present application provides a computer program product. When the computer program product is run on a processor, it causes the processor to execute the method described in the first aspect or any possible implementation of the first aspect.

It can be understood that the beneficial effects of the above-mentioned second to sixth aspects can be referred to the relevant descriptions in the above-mentioned first aspect, and will not be described again here.

Description of the drawings

The drawings required to be used in the description of the embodiments or prior art will be briefly introduced below.

Figure 1 is a comparative schematic diagram of image processing by CNN networks of different complexity provided by the embodiment of the present application;

Figure 2 is a schematic diagram of a process for obtaining a lookup table using a neural network according to an embodiment of the present application;

Figure 3 is a schematic architectural diagram of an image processing system provided by an embodiment of the present application;

Figure 4 is a schematic diagram of an index area and an equivalent index area provided by an embodiment of the present application;

Figure 5 is a schematic diagram of rotating an index area provided by an embodiment of the present application;

Figure 6 is a schematic diagram of rotating the output result of the lookup table provided by the embodiment of the present application;

Figure 7 is a schematic diagram of a hierarchical table value re-indexing provided by an embodiment of the present application;

Figure 8 is a schematic diagram of a quantitatively perceptible hierarchical neural network training provided by an embodiment of the present application;

Figure 9 is a schematic diagram of a multi-lookup table collaborative work provided by an embodiment of the present application;

Figure 10 is a schematic diagram of the adaptation process of a convolutional network to the Y-shaped index mode provided by the embodiment of the present application;

Figure 11 is a schematic diagram of a process for processing Bayer pixel images provided by an embodiment of the present application;

Figure 12 is a schematic flowchart of an image processing method provided by an embodiment of the present application;

Figure 13 is a schematic structural diagram of an image processing device provided by an embodiment of the present application;

Figure 14 is a schematic structural diagram of a computing device provided by an embodiment of the present application;

Figure 15 is a schematic structural diagram of a computing device cluster provided by an embodiment of the present application.

Detailed ways

The term "and/or" in this article is an association relationship that describes related objects, indicating that there can be three relationships. For example, A and/or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. these three situations. The symbol "/" in this article indicates that the associated object is or, for example, A/B means A or B.

The terms "first", "second", etc. in the description and claims herein are used to distinguish different objects, rather than to describe a specific order of objects. For example, the first response message and the second response message are used to distinguish different response messages, but are not used to describe a specific sequence of response messages.

In the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "such as" in the embodiments of the present application is not to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the words "exemplary" or "such as" is intended to present the concept in a concrete manner.

In the description of the embodiments of this application, unless otherwise stated, the meaning of “multiple” refers to two or more, for example, multiple processing units refers to two or more processing units, etc.; multiple Component refers to two or more components, etc.

For example, since the receptive field (RF) of a complex neural network is large, the image processing performance of the neural network can be improved. For example, as shown in Figure 1, (A) in Figure 1 is the input of the neural network; (B) in Figure 1 is the output result of the neural network when the neural network is a simple CNN network and its receptive field is 17x17 ; Figure 1 (C) is the result of the neural network output when the neural network is a complex CNN network and its receptive field is 45x45; Figure 1 (D) is the true value, that is, the original image. Comparing (B), (C) and (D) in Figure 1, we can see that complex neural networks have larger receptive fields, so they can produce results that are closer to the true value, that is, their image processing performance is better. .

Generally, the neural network can be converted into a look-up table (LUT), and the look-up table is applied in image processing to meet the deployment on the terminal device. Specifically, you can first use the trained neural network to obtain the mapping relationship between input data and output data, and store it in the lookup table in the form of key-value. Then, use this lookup table for image processing. For example, after obtaining an input data, you can use the input data to find the corresponding numerical result in the lookup table and output the result. This completes image processing.

For example, as shown in Figure 2 (a), a neural network for image processing can be trained first. The neural network can be composed of several convolutional layers, and the input and output are single-channel images; unlike general images The neural network processed is different. The receptive field of the network is strictly limited to the input range of length and width 2x2, that is, a set of output results only depends on the corresponding 4-dimensional input I ₀ , I ₁ , I ₂ , I ₃ . This neural network can use conventional backward gradient propagation to achieve parameter optimization.

After training is complete, the conversion from neural network to lookup table can be performed. As shown in (b) of Figure 2, after completing the training of network parameters, the input of the neural network is constructed by traversing the sampled numerical combination of 4-dimensional pixels, and the corresponding output result is obtained after operation. Then the traversal results are stored in the lookup table in the form of key-value, resulting in a 4-dimensional lookup table, namely 4DLUT.

Finally, the transformed lookup table can be utilized for image processing. As shown in (c) of Figure 2, given a set of inputs I ₀ , I ₁ , I ₂ , and I ₃ to be processed with I ₀ as the center point, find the numerical results stored in the table. If the results corresponding to sampling points I ₀ , I ₁ , I ₂ and I ₃ are not directly stored in 4DLUT, then First obtain the key value of the sampling point adjacent to the input, and then calculate the output result through interpolation operation. Thus, the image processing is completed. For example, continue to refer to (c) of Figure 2. When it is necessary to process the pixel point I ₀ in an image, the input can be I ₀ , I ₁ , I ₂ , I ₃ , and then the output pixel point can be obtained as That is, the image block corresponding to I ₀ is obtained; after obtaining the image block corresponding to each pixel point in the image, these image blocks are placed at the positions of their respective corresponding pixel points, and the processed image can be obtained. In some embodiments, an image block may include one pixel or multiple pixels, depending on the actual situation.

Although this method can be deployed on terminal devices, the storage size required for the lookup table often increases exponentially with the linear increase of the receptive field of the neural network. Taking 4DLUT as an example, for a 4-bit sampled image, 4DLUT requires a total of (2 ⁴ +1) ⁴ =1.274MB of space to store the image processing results. When the LUT dimension is expanded by 1 dimension, 5DLUT requires (2 ⁴ +1) ⁵ = 21.665MB of storage space. Therefore, when the dimension of the lookup table increases, the requirements for the storage space of the terminal device will also increase, making it difficult to deploy on a terminal device with a small memory size. In addition, the size of the receptive field (i.e., index range) of a single lookup table is limited by the memory scale and hardware cost, so only a lookup table with a smaller index range can be constructed, which limits the performance of image processing.

In view of this, embodiments of the present application provide an image processing method that can increase the size of the receptive field of the lookup table without increasing the dimension of the lookup table during the image processing process, significantly alleviating the memory size and cost of the terminal device. Constraints on hardware costs, thereby greatly improving its performance on a variety of image processing tasks while maintaining the high efficiency and low power consumption of the lookup table method.

By way of example, Figure 3 shows a schematic architectural diagram of an image processing system. As shown in FIG. 3 , the image processing system 300 may include: a multi-table collaborative processing unit 310 based on spatial complementary indexing and/or a multi-level processing unit 320 based on table value re-indexing.

Among them, the multi-table collaborative processing unit 310 based on the spatial complementary index is mainly used to define the center pixel point of the area as I ₀ for any local area in the input image, and construct multiple tables around the center point I ₀ Complementary index regions, wherein each index region includes the center point I ₀ . And, based on the constructed multiple complementary index areas, determine the lookup table corresponding to each index area. For example, each constructed index area can be input into a pre-trained neural network model to obtain a lookup table corresponding to each index area. For example, the complementarity described in this embodiment is different from the strict mathematical definition. The complementarity described in this embodiment means that each index area may have overlapping parts. The complementation described in this embodiment only requires that each lookup table has an area that cannot be covered by the indexes of other lookup tables. In addition, the multi-table cooperative processing unit 310 based on the spatial complementary index can also determine the image block corresponding to each pixel point based on the lookup table corresponding to each index area associated with each pixel point to complete image processing.

For constructing multiple complementary index areas around the center point I ₀ , taking the construction of an index area as an example, the number of pixels contained in the index area can be determined by the dimensions of the preset lookup table. For example, when the dimension of the lookup table is 4, the index area contains 4 pixels, and these 4 pixels include the center point I ₀ .

For the process of determining the lookup table, a 4-dimensional lookup table is determined. Taking an area in the input image as an example, as shown in Figure 4, the center pixel of the area can be defined as I ₀ . Then, multiple complementary indexes are constructed around the center point I ₀ , namely, the LUT ⁽¹⁾ index area, the LUT ⁽²⁾ index area, and the LUT ⁽³⁾ index area. Among them, the LUT ⁽¹⁾ index area is [I ₀ ][I ₁ ][I ₃ ][I ₄ ], the LUT ⁽²⁾ index area is [I ₀ ][I ₂ ][I ₆ ][I ₈ ], The LUT ⁽³⁾ index area is [I ₀ ][I ₄ ][I ₅ ][I ₇ ]. In Figure 4, only the LUT ⁽¹⁾ index area contains {I ₁ , I ₃ }, only the LUT ⁽²⁾ index area contains {I ₂ , I ₆ , I ₈ }, and only the LUT ⁽³⁾ index area contains Contains {I ₅ , I ₇ }, therefore, the index areas corresponding to each lookup table can complement each other to expand the receptive field. Of course, other complementary index lookup tables can also be used. Different lookup tables can be complementary to each other, and there is no limitation here. After multiple complementary index areas are determined, these index areas can be input into a pre-trained neural network model respectively, so that the neural network model outputs a lookup table corresponding to each index area.

After determining the lookup tables of multiple complementary index areas, the output results corresponding to the center point I ₀ in each lookup table can be averaged to obtain the required result, that is, the image corresponding to the center point I ₀ is obtained. piece. For example, continuing to refer to Figure 4, the output result V ₀ corresponding to the center point I ₀ = (LUT ⁽¹⁾ [I ₀ ][I ₁ ][I ₃ ][I ₄ ]+LUT ⁽²⁾ [I ₀ ][I ₂ ][I ₆ ][I ₈ ]+LUT ⁽³⁾ [I ₀ ][I ₄ ][I ₅ ][I ₇ ])/3. When the input is [I ₀ ][I ₁ ][I ₃ ][I ₄ ], the index area corresponding to the output result can be equivalent to [I ₀ ][I ₁ ][I ₃ ][I ₄ ][ I ₅ ][I ₆ ][I ₇ ][I ₈ ]. Since the index area corresponding to the output result of the lookup table increases, the receptive field of the lookup table will also increase accordingly. At the same time, the dimensions of the lookup table have not changed, so the receptive field of the lookup table can be increased without increasing the dimensions of the lookup table. Among them, when the input data is [I ₀ ][I ₁ ][I ₃ ][I ₄ ], the output result can be found as V ₀ through the lookup table. For lookup tables in other dimensions, you can refer to the method of increasing the receptive field of a 4-dimensional lookup table, which will not be described again here.

In some embodiments, in order to further increase the receptive field of the lookup table without increasing the dimension of the lookup table, the index area corresponding to each constructed lookup table can also be rotated, and based on the center point I ₀ The output result of the lookup table corresponding to the index area before rotation and the output result of the lookup table corresponding to the index area after rotation determine the final output result of the lookup table corresponding to the center point I ₀ . Specifically, for any lookup table in Figure 4, taking LUT ⁽¹⁾ as an example, as shown in (A) of Figure 5, the index area can first be [I ₀ ][I ₁ ][I ₃ ][I ₄ ] lookup table to determine the output V _j (1) corresponding to the center point I ₀ in the index area. Then, as shown in (B) of Fig. 5, (A) of Fig. 5 The index area in is rotated 90 degrees counterclockwise. At this time, the index area becomes [I ₀ ][I ₁ ][I ₉ ][I ₁₀ ], and then based on the index area it can be [I ₀ ][I ₁ ][I ₉ ][I ₁₀ ] lookup table to determine the output V _j (2) corresponding to the center point I ₀ in the index area. Next, as shown in Figure 5(C), the index area in Figure 5(B) is rotated 90 degrees counterclockwise. The index area at this time becomes [I ₀ ][I ₉ ][I ₁₁ ][I ₁₂ ], and then based on the lookup table with the index area [I ₀ ][I ₉ ][I ₁₁ ][I ₁₂ ], the output V _j corresponding to the center point I ₀ under the index area can be determined (3). Next, as shown in Figure 5 (D), the index area in Figure 5 (C) is rotated 90 degrees counterclockwise. The index area at this time becomes [I ₀ ][I ₃ ][I ₁₂ ][I ₁₃ ], and then based on the lookup table with the index area [I ₀ ][I ₃ ][I ₁₂ ][I ₁₃ ], the output V _j corresponding to the center point I ₀ under the index area can be determined (4). Since V _j (2) and V _j (1) are not in the same calculation area, for unification, V _j (2) can be rotated 90 degrees clockwise to obtain V _j (2)′. Likewise, V _j (3) can be rotated 180 degrees clockwise to obtain V _j (3)', and V _j (4) can be rotated 270 degrees clockwise to obtain V _j (4)'. For example, as shown in Figure 6 (A), assuming that V _j (3) is [1][2][3][4], rotate it 90 degrees clockwise, you can get the figure shown in Figure 6 (B) The result shown in Figure 6(B); by rotating 90 degrees clockwise on the basis of Figure 6(B), the result shown in Figure 6(C) can be obtained, where the result shown in Figure 6(C) is the result The desired result is V _j (3)′. Finally, V _j (1), V _j (2)′, V _j (3)′, and V _j (4)′ can be added and averaged, and the calculated result is: When the input is [I ₀ ][I ₁ ][I ₃ ][I ₄ ] The output result obtained through LUT ⁽¹⁾ , that is, V=(V _j (1)+V _j (2)′+V _j (3)′+V _j (4)′)/4. The method of increasing the receptive fields of other lookup tables can be processed in the manner described in Figure 5, and will not be described again here. In this implementation, the angle of rotation can be selected according to the actual situation, and is not limited here. For example, you can only choose to rotate 90 degrees, you can only choose to rotate 180 degrees or 270 degrees, or you can choose to rotate 90 degrees and 180 degrees. , and you can choose to rotate 180 degrees and 270 degrees, etc. Similarly, after determining the output results corresponding to the center point I ₀ in each lookup table, these results can be added and averaged, and the calculated results can be used as the final required results, and the input can be established in the lookup table and the relationship between the output, and then obtain the latest lookup table. For example, continuing to refer to FIG. 4 , when each lookup table is rotated by 90 degrees, 180 degrees, and 270 degrees, the final equivalent index area may be the equivalent index area as shown in FIG. 4 . In this way, the index area corresponding to the output result corresponding to the lookup table is further increased, thereby further increasing the receptive field of the lookup table.

The multi-level processing unit 320 based on table value re-indexing is mainly used to perform hierarchical processing on multiple lookup tables related to a certain center point I ₀ , where the input of the k-th level lookup table is the (k-1)-th level The output of the lookup table. This expands the equivalent receptive field by cascading the inputs and outputs between multiple lookup tables. Illustratively, as shown in Figure 7, taking the cascade process of two-level lookup tables as an example, LUT ⁽¹⁾ represents the first-level lookup table, and LUT ⁽²⁾ represents the secondary lookup table. In Figure 7, the index value result in LUT ⁽¹⁾ can be used as the index of the secondary LUT ⁽²⁾ . This hierarchical indexing process can be expressed as:
V＝LUT ⁽²⁾ [LUT ⁽¹⁾ [I _* ]][LUT ⁽¹⁾ [I _* ]][LUT ⁽¹⁾ [I _* ]][LUT ⁽¹⁾ [I _* ]]

When a low-quality image is input into the multi-level processing unit 320 based on table value re-indexing, a high-quality image can be output through the hierarchical processing of the multi-level processing unit 320 based on table value re-indexing.

From the perspective of receptive fields, this cascade framework is similar in principle to cascading multiple convolutional layers in a deep convolutional neural network. However, unlike the cascade of convolutional layers in CNN, due to the limitation of LUT size, the index of image data is usually stored in the format of integer numbers. Before the conversion step from a neural network to a lookup table described in Figure 2, training the neural network requires gradients of the Float-Point data type. In order to solve the problems caused by data types, a quantification-aware re-indexing mechanism is designed in this embodiment to introduce the hierarchical indexing pattern into the training process of the neural network. As shown in Figure 8, the output value of LUT ⁽¹⁾ is quantized to an integer in the forward pass (i.e., the direction pointed by the dashed line pointing to Integer), while its gradient is retained as a floating point value in the backward pass (i.e. points in the direction of the dashed line pointing to the Float-Point). In this way, numerical quantization operations can be perceived and modeled during the hierarchical training process of neural networks, thus making up for the performance loss caused by numerical quantization.

In some embodiments, the multi-table collaborative processing unit 310 based on spatial complementary index and the multi-level processing unit 320 based on table value re-indexing can be used alone or in combination. By way of example, Figure 9 shows a process in which the two are combined together. As shown in Figure 9, multiple lookup tables (Multiple LUT) are abbreviated as MuLUT, and MuLUT-S/D/Y respectively represent three LUTs with different pixel index modes. Block 1x1 means that the resolution of the output image remains the same as that of the input image, and Block 2x2 means that the resolution of the output image is 2 times that of the input image. The neural networks corresponding to the three LUTs are all implemented by stacking and connecting several convolutional layers. The first layer of the convolutional network corresponding to the S-type LUT uses a standard convolution with a convolution kernel size of 2x2, which is recorded as Conv2x2. The first layer of the convolutional network corresponding to the D-type LUT uses a dilated convolution with a convolution kernel size of 2x2, which is recorded as Conv2x2-D2. The first layer of the convolutional network corresponding to the Y-type LUT is implemented using a standard convolution with a convolution kernel size of 1x4, which is recorded as Conv1x4. Since the Y-shaped index mode does not belong to the conventional grid index, the existing function library cannot be directly called for operation. Therefore, the operation of the corresponding convolutional network requires special adaptation. As shown in Figure 10, first, the input image is split into image blocks of 3x3 size in a sliding window, which is the unfold operation. Then, in each obtained image block, select the pixel coordinates corresponding to the Y-type index pattern, and rearrange the values at the corresponding coordinates into 1x4 image blocks. The split and rearranged image can be adapted to a convolutional network with a Conv1x4 first layer.

Continue to refer to Figure 9. In Figure 9, first select three index ranges corresponding to multiple lookup tables related to the center point I ₀ , and then input the data in each index range into their corresponding convolutional networks. The output results of the corresponding convolutional networks are all 1-dimensional data. Then, the output results of each convolutional network are combined into one, such as averaging. Then, using the result as the center point, determining the index ranges of multiple lookup tables related to the result, and inputting the data in each index range into their corresponding convolutional networks, their respective corresponding volumes The output results of the product network are all data with the same dimensions as the required lookup table dimensions. Finally, these data can be merged into one and associated with an index range containing the center point I ₀ to establish a new lookup table. The receptive field of this lookup table is larger than that of the previous lookup table, and the dimensions remain unchanged.

In some embodiments, when the pixels of the input image are arranged in a Bayer pixel pattern, the input can be LUT queried within a 2x2 window by sliding the window with a step size of 2 times, thereby avoiding the Bayer pixel pattern. Influence. Then, the aforementioned image processing system 300 is used to perform multi-lookup table collaborative processing, and/or the value of the first-stage lookup table is used as the input of the second stage to implement hierarchical processing. For example, as shown in Figure 11, you can first extract the pixels of the red channel, the pixels of the green channel, and the pixels of the blue channel with a step size of 2 times, and then use the previously described image. The processing system 300 processes the extracted pixels respectively.

Next, based on the foregoing description, the image processing method provided in the embodiment of the present application is introduced.

By way of example, Figure 12 shows the flow of an image processing method. It can be understood that this method can be executed by any device, device, platform, or device cluster with computing and processing capabilities. As shown in Figure 12, the image processing method may include the following steps:

S1201. Obtain the first image to be processed.

In this embodiment, after the user inputs the image that needs to be processed, the first image to be processed can be obtained. In addition, the first image to be processed can also be automatically obtained from a storage area set by the user. The details may be determined according to the actual situation.

S1202. For any pixel in the first image, use N (N≥2) lookup tables associated with the pixel to determine the image block corresponding to the pixel. The image block includes Single or multiple pixels, wherein the index area of each lookup table contains any one pixel, and the index areas of different lookup tables are different and at least partially the same.

In this embodiment, for any pixel in the first image, multiple lookup tables associated with the pixel can be used to determine the image block corresponding to the pixel. The image block includes single or multiple pixels. In addition, the index range of each lookup table includes any one pixel, and the index ranges of different first lookup tables are different and at least partially the same, that is, the index ranges of different lookup tables are complementary. For example, continuing to refer to FIG. 4 , any pixel point can be the center point I ₀ , that is, LUT ⁽¹⁾ , LUT ⁽²⁾ and LUT ⁽³⁾ can be lookup tables associated with the center point I ₀ . For a method of using multiple lookup tables associated with any one pixel point to determine the image block corresponding to any one pixel point, please refer to the aforementioned "Multi-table collaborative processing unit 310 based on spatial complementary index to obtain the center point I ₀ corresponding to The process of "image block" will not be described again here.

As a possible implementation, the index area of each lookup table in the N lookup tables can be input into the corresponding lookup table to obtain N first sub-image blocks. Each first sub-image block includes single or multiple pixels. Then, the pixel points at the same position in the N first sub-image blocks are added and averaged to obtain the image block corresponding to any one pixel point. Since the index area corresponding to the output result of the lookup table increases, the receptive field of the lookup table will also increase accordingly. At the same time, the dimensions of the lookup table have not changed, so the receptive field of the lookup table can be increased without increasing the dimensions of the lookup table. For details, please refer to the aforementioned description of Figure 4 and will not be described again here.

Further, for the index area of any one of the N look-up tables, the index area of any one look-up table can be centered on any pixel and rotated M times along the first direction to obtain M index areas, M≥1. Then, based on the lookup table corresponding to each index area in the M index areas, the second sub-image blocks corresponding to each of the M index areas are determined to obtain M second sub-image blocks. Each second sub-image block includes single or multiple pixels. Then, based on the rotation angles corresponding to each of the M index regions, the second sub-image blocks corresponding to each of the M index regions can be rotated by the same angle in the direction opposite to the first direction to obtain M third sub-image blocks. Image blocks. Finally, the pixel points at the same position in the first sub-image block and the M third sub-image blocks corresponding to the index area of any look-up table are added and averaged, and the calculation result is used as the result of any look-up table. The first sub-image block corresponding to the index area. Thus, by rotating the index area, the receptive field of the lookup table is further increased without increasing the dimension of the lookup table. For details, please refer to the aforementioned descriptions of Figures 5 and 6 and will not be described again here.

As another possible implementation, the output of the i-th lookup table among the N lookup tables is the input of the (i+1)-th lookup table, and the input of the 1st lookup table among the N lookup tables is the 1st The index area of the lookup table. The output of the Nth lookup table among the N lookup tables is the image block corresponding to any pixel. That is to say, N lookup tables can be cascaded, thereby increasing the receptive field of the lookup tables.

As another possible implementation, N lookup tables may be composed of lookup tables included in M lookup table sets, and each lookup table set includes multiple lookup tables. The inputs of each lookup table in the jth lookup table set are respectively the index areas of the corresponding lookup tables in the jth lookup table set determined based on the output of the (j-1)th lookup table set. The input of each lookup table in the first lookup table set is the index area of the corresponding lookup table in the first lookup table set determined based on any of the aforementioned pixels. The output of the M-th lookup table set is the image block corresponding to any of the aforementioned pixels. For example, this processing process can be understood as the aforementioned processing process in FIG. 9. For details, please refer to the aforementioned description and will not be described again here.

S1203. Obtain a second image based on the image blocks corresponding to each pixel point in the first image. The resolution of the second image is higher than or equal to the resolution of the first image.

In this embodiment, after obtaining the image blocks corresponding to each pixel point in the first image, these image blocks can be spliced together to obtain the second image. Wherein, the resolution of the second image is higher than or equal to the resolution of the first image. In some embodiments, when splicing each image block, each image block may be placed at the position of its corresponding pixel point. For example, if pixel point 1 in the first image corresponds to image block 1, then image block 1 can be placed at the position of pixel point 1.

S1204. Output the second image.

In this embodiment, after the second image is acquired, the second image can be output. For example, displayed on the device's display, etc.

Therefore, during the image processing process, multiple lookup tables are used to work together for each pixel, thereby expanding the receptive field and improving image processing performance while maintaining linear growth in the amount of calculation and storage. In contrast, existing lookup table technology that utilizes spatial domain information only uses a single lookup table. The expansion of the receptive field will increase the storage capacity exponentially and will be limited by memory scale and hardware cost.

Based on the methods in the above embodiments, embodiments of the present application also provide an image processing device.

By way of example, FIG. 13 shows an image processing device. As shown in Figure 13, the image processing device 1300 may include an acquisition module 1301 and a processing module 1302. Among them, the acquisition module 1301 is used to acquire the first image. The processing module 1302 is configured to use N (N≥2) lookup tables associated with any pixel point in the first image to determine the image block corresponding to the arbitrary pixel point. Including single or multiple pixels, wherein the index area of each lookup table contains any one pixel, and the index areas of different lookup tables are different and at least partially the same. The processing module 1302 is also configured to obtain a second image based on the image blocks corresponding to each pixel point in the first image, wherein the resolution of the second image is higher than or equal to the resolution of the first image; and, output the second image .

In some embodiments, when using N lookup tables associated with any one pixel point to determine the image block corresponding to any one pixel point, the processing module 1302 is specifically used to: The index area is input into the corresponding lookup table to obtain N first sub-image blocks. Each first sub-image block includes multiple pixels; the pixels at the same position in the N first sub-image blocks are evenly matched. Add and average to obtain the image block corresponding to any pixel.

In some embodiments, when the processing module 1302 inputs the index area of each lookup table in the N lookup tables into the corresponding lookup table to obtain N first sub-image blocks, it is specifically used to: for the N lookup tables The index area of any lookup table is centered on any pixel and rotated M times along the first direction to obtain M index areas, M≥1; based on M indexes The lookup table corresponding to each index area in the area determines the second sub-image blocks corresponding to each of the M index areas to obtain M second sub-image blocks. Each second sub-image block includes a single or multiple Pixel points; based on the rotation angles corresponding to each of the M index areas, rotate the second sub-image blocks corresponding to each of the M index areas by the same angle in the direction opposite to the first direction to obtain M third sub-image blocks. Image block; add and average the pixel points at the same position in the first sub-image block corresponding to the index area of any lookup table and the M third sub-image blocks, and use the calculation result as the any one Look up the first sub-image block corresponding to the index area of the table.

In some embodiments, the output of the i-th lookup table among the N lookup tables is the input of the (i+1)th lookup table, and the input of the 1st lookup table among the N lookup tables is the input of the 1st lookup table. In the index area, the output of the Nth lookup table among the N lookup tables is the image block corresponding to any pixel.

In some embodiments, the N lookup tables are composed of lookup tables contained in M lookup table sets, each lookup table set including multiple lookup tables. The inputs of each lookup table in the jth lookup table set are respectively the index areas of the corresponding lookup tables in the jth lookup table set determined based on the output of the (j-1)th lookup table set. The input of each lookup table in the first lookup table set is the index area of the corresponding lookup table in the first lookup table set determined based on any pixel point. The output of the M-th lookup table set is the image block corresponding to any pixel.

It should be understood that the above device is used to execute the method in the above embodiment. The corresponding program module in the device has implementation principles and technical effects similar to those described in the above method. For the working process of the device, please refer to the corresponding program module in the above method. The process will not be described again here.

In some embodiments, both the acquisition module 1301 and the processing module 1302 shown in Figure 13 can be implemented through software, or can be implemented through Implemented through hardware. Illustratively, the following takes the acquisition module 1301 shown in Figure 13 as an example to introduce the implementation of the acquisition module 1301. Similarly, the implementation of the processing module 1302 shown in FIG. 13 may refer to the implementation of the acquisition module 1301 shown in FIG. 13 .

Module As an example of a software functional unit, the processing module 1302 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the above computing instance may be one or more. For example, processing module 1302 may include code running on multiple hosts/virtual machines/containers. It should be noted that multiple hosts/virtual machines/containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, multiple hosts/virtual machines/containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs. Each AZ includes one data center or multiple AZs. geographically close data centers. Among them, usually a region can include multiple AZs.

Likewise, the multiple hosts/VMs/containers used to run the code can be distributed in the same virtual private cloud (VPC), or across multiple VPCs. Among them, usually a VPC is set up in a region. Cross-region communication between two VPCs in the same region and between VPCs in different regions requires a communication gateway in each VPC, and the interconnection between VPCs is realized through the communication gateway. .

Module As an example of a hardware functional unit, the processing module 1302 may include at least one computing device, such as a server. Alternatively, the processing module 1302 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). Among them, the above-mentioned PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (field-programmable gate array, FPGA), a general array logic (generic array logic, GAL), or any combination thereof.

Multiple computing devices included in the processing module 1302 may be distributed in the same region or in different regions. Multiple computing devices included in the processing module 1302 may be distributed in the same AZ or in different AZs. Similarly, multiple computing devices included in the processing module 1302 may be distributed in the same VPC or in multiple VPCs. The plurality of computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

It should be noted that in other embodiments, the processing module 1302 can be used to perform any steps in the method shown in Figure 12, and the processing module 1302 shown in Figure 13 can be used to perform the method provided in Figure 12. any step in . The steps that the acquisition module 1301 and the processing module 1302 shown in Figure 13 are responsible for implementing can be specified as needed. The acquisition module 1301 and the processing module 1302 respectively implement different steps in the method provided in Figure 12 to implement the image processing device 1300. Full functionality.

The present application also provides a computing device 1400. As shown in Figure 14, computing device 1400 includes: bus 1402, processor 1404, memory 1406, and communication interface 1408. The processor 1404, the memory 1406 and the communication interface 1408 communicate through a bus 1402. Computing device 1400 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 1400.

The bus 1402 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one line is used in Figure 14, but it does not mean that there is only one bus or one type of bus. Bus 1404 may include a path that carries information between various components of computing device 1400 (eg, memory 1406, processor 1404, communications interface 1408).

The processor 1404 may include a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP). any one or more of them.

Memory 1406 may include volatile memory, such as random access memory (RAM). The processor 144 may also include non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, hard disk drive (HDD) or solid state drive (solid state drive). drive, SSD).

The memory 1406 stores executable program code, and the processor 1404 executes the executable program code to respectively realize the functions of the acquisition module 1301 and the processing module 1302 shown in Figure 13, thereby realizing all the methods in the above embodiments. or partial steps. That is, the memory 146 stores instructions for executing all or part of the steps in the methods of the above embodiments.

Alternatively, executable code is stored in the memory 1406, and the processor 1404 executes the executable code to respectively implement the functions of the aforementioned image processing device 1300, thereby implementing all or part of the steps in the method of the above embodiment. That is, the memory 1406 stores instructions for executing all or part of the steps in the methods of the above embodiments.

The communication interface 1403 uses transceiver modules such as but not limited to network interface cards and transceivers to implement the communication between the computing device 140 and other Communication between devices or communication networks.

An embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

As shown in Figure 15, the computing device cluster includes at least one computing device 1400. The memory 1406 in one or more computing devices 1400 in the computing device cluster may store the same instructions for executing all or part of the steps in the above embodiment method.

In some possible implementations, the memory 1406 of one or more computing devices 1400 in the computing device cluster may also store partial instructions for executing all or part of the steps in the above embodiment methods. In other words, a combination of one or more computing devices 1400 may jointly execute instructions for performing all or part of the steps in the above embodiment methods.

It should be noted that the memory 1406 in different computing devices 1400 in the computing device cluster can store different instructions, respectively used to execute part of the functions of the image processing apparatus 1300. That is, the instructions stored in the memory 1406 in different computing devices 1400 can implement the functions of one or more modules in the acquisition module 814 and the processing module 820 shown in FIG. 13 .

In some possible implementations, one or more computing devices in a cluster of computing devices may be connected through a network. Wherein, the network may be a wide area network or a local area network, etc.

Based on the methods in the above embodiments, embodiments of the present application provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is run on a processor, it causes the processor to execute the steps in the above embodiments. Methods.

Based on the methods in the above embodiments, embodiments of the present application provide a computer program product, which when the computer program product is run on a processor, causes the processor to execute the methods in the above embodiments.

It can be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), or other general-purpose processor, digital signal processor (DSP), or application-specific integrated circuit (application specific integrated circuit, ASIC), field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. Software instructions can be composed of corresponding software modules, which can be stored in random access memory (random access memory, RAM), flash memory, read-only memory (read-only memory, ROM), programmable read-only memory (programmable rom) , PROM), erasable programmable read-only memory (erasable PROM, EPROM), electrically erasable programmable read-only memory (electrically EPROM, EEPROM), register, hard disk, mobile hard disk, CD-ROM or other well-known in the art any other form of storage media. An exemplary storage medium is coupled to the processor such that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage media may be located in an ASIC.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted over a computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server or data center to another website through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. , computer, server or data center for transmission. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state disk (SSD)), etc.

It can be understood that the various numerical numbers involved in the embodiments of the present application are only for convenience of description and are not used to limit the scope of the embodiments of the present application.

Claims (14)

1. An image processing method, characterized in that the method includes:

   Get the first image;

   For any pixel point in the first image, use N (N≥2) lookup tables associated with the any pixel point to determine the image block corresponding to the any pixel point, and the image block includes single or multiple pixels, wherein the index area of each look-up table contains any one of the pixels, and the index areas of different look-up tables are different and at least partially the same;

   Obtain a second image based on the image blocks corresponding to each pixel point in the first image, wherein the resolution of the second image is higher than or equal to the resolution of the first image;

   Output the second image.
2. The method according to claim 1, characterized in that the image block corresponding to any one pixel is determined by using N (N≥2) lookup tables associated with any one pixel, specifically including: :

   Input the index area of each lookup table in the N lookup tables into the corresponding lookup table to obtain N first sub-image blocks. Each of the first sub-image blocks includes single or multiple pixels. ;

   The pixel points at the same position in the N first sub-image blocks are added and averaged to obtain the image block corresponding to any one of the pixel points.
3. The method according to claim 2, characterized in that said inputting the index area of each lookup table in the N lookup tables into the corresponding lookup table to obtain N first sub-image blocks specifically includes:

   For the index area of any one of the N look-up tables, center the index area of any one of the look-up tables on any one pixel and rotate it M times along the first direction to obtain M indexes. Area, M≥1;

   Based on the lookup table corresponding to each index area in the M index areas, the second sub-image blocks corresponding to each of the M index areas are determined to obtain M second sub-image blocks, each of the second sub-image blocks Each sub-image block includes single or multiple pixels;

   Based on the rotation angles corresponding to each of the M index areas, the second sub-image blocks corresponding to each of the M index areas are rotated by the same angle in the direction opposite to the first direction to obtain M The third sub-image block;

   Add and average the pixel points at the same position in the first sub-image block corresponding to the index area of any one of the lookup tables and the M third sub-image blocks, and use the calculation result as the arbitrary The first sub-image block corresponding to the index area of a lookup table.
4. The method according to claim 1, characterized in that the output of the i-th lookup table among the N lookup tables is the input of the (i+1)th lookup table, and the 1st among the N lookup tables The input of the lookup table is the index area of the first lookup table, and the output of the Nth lookup table among the N lookup tables is the image block corresponding to any one of the pixels.
5. The method according to claim 1, characterized in that the N look-up tables are composed of look-up tables included in M look-up table sets, and each of the look-up table sets includes multiple look-up tables;

   Wherein, the input of each lookup table in the jth lookup table set is respectively the index area of the corresponding lookup table in the jth lookup table set determined based on the output of the (j-1)th lookup table set. ;

   The input of each lookup table in the first lookup table set is the index area of the corresponding lookup table in the first lookup table set determined based on any one pixel point;

   The output of the M-th lookup table set is the image block corresponding to any one of the pixels.
6. An image processing device, characterized in that the device includes:

   Acquisition module, used to acquire the first image;

   A processing module configured to use N (N≥2) lookup tables associated with any pixel point in the first image to determine the image block corresponding to the any pixel point. , the image block includes single or multiple pixels, wherein the index area of each look-up table contains any one of the pixels, and the index areas of different look-up tables are different and at least partially the same. ;

   The processing module is further configured to obtain a second image based on the image blocks corresponding to each pixel point in the first image, wherein the resolution of the second image is higher than or equal to the resolution of the first image. rate; and, output the second image.
7. The device according to claim 6, wherein the processing module uses N (N≥2) lookup tables associated with any one of the pixels to determine the image block corresponding to the any one of the pixels. When, it is specifically used for:

   Input the index area of each lookup table in the N lookup tables into the corresponding lookup table to obtain N first sub-image blocks. Each of the first sub-image blocks includes single or multiple pixels. ;

   Add the pixels at the same position in the N first sub-image blocks and average them to obtain the pixel corresponding to any one pixel. Image block.
8. The device according to claim 7, wherein the processing module inputs the index area of each lookup table in the N lookup tables into the corresponding lookup table to obtain N first sub-image blocks. , specifically used for:

   For the index area of any one of the N look-up tables, center the index area of any one of the look-up tables on any one pixel and rotate it M times along the first direction to obtain M indexes. Area, M≥1;

   Based on the lookup table corresponding to each index area in the M index areas, the second sub-image blocks corresponding to each of the M index areas are determined to obtain M second sub-image blocks, each of the second sub-image blocks Each sub-image block includes single or multiple pixels;

   Based on the rotation angles corresponding to each of the M index areas, the second sub-image blocks corresponding to each of the M index areas are rotated by the same angle in the direction opposite to the first direction to obtain M The third sub-image block;

   Add and average the pixel points at the same position in the first sub-image block corresponding to the index area of any one of the lookup tables and the M third sub-image blocks, and use the calculation result as the arbitrary The first sub-image block corresponding to the index area of a lookup table.
9. The device according to claim 6, characterized in that the output of the i-th lookup table among the N lookup tables is the input of the (i+1)th lookup table, and the 1st among the N lookup tables The input of the lookup table is the index area of the first lookup table, and the output of the Nth lookup table among the N lookup tables is the image block corresponding to any one of the pixels.
10. The method according to claim 6, characterized in that the N look-up tables are composed of look-up tables included in M look-up table sets, and each of the look-up table sets includes multiple look-up tables;

    Wherein, the input of each lookup table in the jth lookup table set is respectively the index area of the corresponding lookup table in the jth lookup table set determined based on the output of the (j-1)th lookup table set. ;

    The input of each lookup table in the first lookup table set is the index area of the corresponding lookup table in the first lookup table set determined based on any one pixel point;

    The output of the M-th lookup table set is the image block corresponding to any one of the pixels.
11. A computing device, characterized by including:

    At least one memory for storing programs;

    At least one processor for executing programs stored in the memory;

    Wherein, when the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1-5.
12. A computing device cluster, characterized by including at least one computing device, each computing device including a processor and a memory;

    The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the cluster of computing devices performs the method according to any one of claims 1-5.
13. A computer-readable storage medium that stores a computer program. When the computer program is run on a processor, it causes the processor to execute the method as described in any one of claims 1-5. .
14. A computer program product, characterized in that, when the computer program product is run on a processor, it causes the processor to execute the method according to any one of claims 1-5.