CN115578272A

CN115578272A - Image processing method and device and computing equipment

Info

Publication number: CN115578272A
Application number: CN202211102553.9A
Authority: CN
Inventors: 陈畅; 彭竞阳; 宋风龙; 李家丞; 熊志伟
Original assignee: University of Science and Technology of China USTC; Huawei Technologies Co Ltd
Current assignee: University of Science and Technology of China USTC; Huawei Technologies Co Ltd
Priority date: 2022-09-09
Filing date: 2022-09-09
Publication date: 2023-01-06
Also published as: WO2024051298A1

Abstract

An image processing method, comprising: acquiring a first image; for any pixel point in the first image, determining an image block corresponding to the any pixel point by using a plurality of lookup tables associated with the any pixel point, wherein the image block comprises a single or a plurality of pixel points, the index area of each lookup table comprises the any pixel point, and the index areas of different lookup tables are different and at least partially the same; obtaining a second image according to the image block corresponding to each pixel point in the first image, wherein the resolution of the second image is higher than or equal to that of the first image; and outputting the second image. Therefore, in the image processing process, a plurality of lookup tables are used for each pixel point to work cooperatively, so that the receptive field is expanded under the condition of maintaining the linear increase of the calculated amount and the memory space, and the image processing performance is improved.

Description

Image processing method and device and computing equipment

Technical Field

The present application relates to the technical field of Artificial Intelligence (AI), and in particular, to an image processing method, an image processing apparatus, and a computing device.

Background

Image processing is the main direction of research in underlying computer vision. In recent years, a method based on a Convolutional Neural Network (CNN) has a remarkable effect in terms of image processing performance. Generally, a more complex neural network can achieve better results. However, due to the limitations of operating speed and power consumption, networks for image processing on mobile phones and other terminal devices with weak computing power cannot be generally too complex. Although this constraint can be alleviated by an elaborate network architecture or a dedicated computational engine (such as a graphics processor), the corresponding hardware cost and power consumption still increase the difficulty of deploying existing CNN methods on end devices. On the other hand, on terminal equipment such as intelligent vehicles, application scenes such as lane changing and sudden dangerous case processing during high-speed driving have higher requirements on the running time delay of the image processing method. Therefore, how to design an efficient image processing method, which meets the requirements of low power consumption and low time delay when deployed in a terminal device, and has better image processing performance is a technical problem that needs to be solved at present.

Disclosure of Invention

The application provides an image processing method, an image processing device, a computing device cluster, a computer storage medium and a computer product, which can meet the requirements of low power consumption and low time delay when deployed in a terminal device and have better image processing performance.

In a first aspect, the present application provides an image processing method, including: acquiring a first image; for any pixel point in the first image, determining an image block corresponding to the any pixel point by using N (N is more than or equal to 2) lookup tables associated with the any pixel point, wherein the image block comprises a single or a plurality of pixel points, the index area of each lookup table comprises the any pixel point, and the index areas of different lookup tables are different and at least partially the same; obtaining a second image according to image blocks corresponding to all pixel points in the first image, wherein the resolution of the second image is higher than or equal to that of the first image; and outputting the second image.

Therefore, in the image processing process, a plurality of lookup tables are used for each pixel point to work cooperatively, so that the receptive field is expanded under the condition of maintaining the linear increase of the calculated amount and the memory space, and the image processing performance is improved. In contrast, the existing lookup table technology using spatial domain information only uses a single lookup table, and the expansion of the receptive field causes exponential increase of the storage capacity, which is limited by the size of the memory and the cost of the hardware.

In a possible implementation manner, determining an image block corresponding to any one pixel point by using N lookup tables associated with the any one pixel point specifically includes: inputting the index area of each lookup table in the N lookup tables into the corresponding lookup table to obtain N first sub-image blocks, wherein each first sub-image block comprises a single or multiple pixel points; and adding and averaging the pixel points at the same position in the N first sub-image blocks to obtain an image block corresponding to any one pixel point. Because the index area corresponding to the output result corresponding to the lookup table is increased, the receptive field of the lookup table is also correspondingly increased. Meanwhile, the dimensionality of the lookup table is not changed, so that the receptive field of the lookup table is increased on the premise of not increasing the dimensionality of the lookup table.

In a possible implementation manner, inputting the index area of each of the N lookup tables into a corresponding lookup table to obtain N first sub image blocks specifically includes: aiming at the index area of any one lookup table in the N lookup tables, rotating the index area of the any one lookup table for M times along a first direction by taking any pixel point as a center to obtain M index areas, wherein M is more than or equal to 1; determining second sub image blocks corresponding to the M index areas respectively based on a lookup table corresponding to each of the M index areas to obtain M second sub image blocks, wherein each second sub image block comprises a single or multiple pixel points; respectively rotating the second sub image blocks corresponding to the M index areas by the same angle along the direction opposite to the first direction based on the rotation angles corresponding to the M index areas respectively to obtain M third sub image blocks; and adding and averaging pixel points at the same position in the first sub-image block corresponding to the index area of any one lookup table and the M third sub-image blocks, and taking the calculation result as the first sub-image block corresponding to the index area of any one lookup table. Therefore, the receptive field of the lookup table is increased on the premise of not increasing the dimensionality of the lookup table by rotating the index area.

In a possible implementation manner, the output of the ith lookup table in the N lookup tables is the input of the (i + 1) th lookup table, the input of the 1 st lookup table in the N lookup tables is the index region of the 1 st lookup table, and the output of the nth lookup table in the N lookup tables is the image block corresponding to the arbitrary pixel point. That is, N look-up tables may be cascaded to thereby achieve an increased receptive field of the look-up tables.

In one possible implementation, the N lookup tables are formed by lookup tables included in M lookup table sets, and each lookup table set includes a plurality of lookup tables. And the input of each lookup table in the jth lookup table set is the index area of the corresponding lookup table in the jth lookup table set determined by taking the output of the (j-1) th lookup table set as the reference. The input of each lookup table in the 1 st lookup table set is the index region of the corresponding lookup table in the 1 st lookup table set determined by taking any one of the pixel points as the reference. The output of the mth lookup table set is the image block corresponding to any one of the pixel points. Therefore, the image blocks corresponding to the pixels are determined in a mode of complementing and cascading the plurality of lookup tables, and the image processing performance is further improved.

In a second aspect, the present application provides an image processing apparatus comprising: the device comprises an acquisition module and a processing module. The acquisition module is used for acquiring a first image. The processing module is used for determining an image block corresponding to any pixel point in the first image by using N (N is more than or equal to 2) lookup tables associated with the any pixel point, wherein the image block comprises a single or a plurality of pixel points, the index area of each lookup table comprises the any pixel point, and the index areas of different lookup tables are different and at least partially the same. The processing module is further used for obtaining a second image according to the image block corresponding to each pixel point in the first image, wherein the resolution of the second image is higher than or equal to that of the first image; and outputting the second image.

In a possible implementation manner, when determining an image block corresponding to any one pixel point by using N lookup tables associated with the any one pixel point, the processing module is specifically configured to: inputting the index area of each lookup table in the N lookup tables into a corresponding lookup table to obtain N first sub-image blocks, wherein each first sub-image block comprises a single or a plurality of pixel points; and adding and averaging the pixel points at the same position in the N first sub-image blocks to obtain an image block corresponding to any one pixel point.

In a possible implementation manner, when the processing module inputs the index area of each of the N lookup tables into a corresponding lookup table to obtain N first sub image blocks, the processing module is specifically configured to: aiming at the index area of any one lookup table in the N lookup tables, rotating the index area of the any one lookup table for M times along a first direction by taking any pixel point as a center to obtain M index areas, wherein M is more than or equal to 1; determining second sub image blocks corresponding to the M index areas respectively based on a lookup table corresponding to each index area in the M index areas to obtain M second sub image blocks, wherein each second sub image block comprises a plurality of pixel points; respectively rotating the second sub image blocks corresponding to the M index areas by the same angle along the direction opposite to the first direction based on the rotation angles corresponding to the M index areas respectively to obtain M third sub image blocks; and adding and averaging pixel points at the same position in the first sub-image block corresponding to the index area of any one lookup table and the M third sub-image blocks, and taking the calculation result as the first sub-image block corresponding to the index area of any one lookup table.

In a possible implementation manner, the output of the ith lookup table in the N lookup tables is the input of the (i + 1) th lookup table, the input of the 1 st lookup table in the N lookup tables is the index region of the 1 st lookup table, and the output of the nth lookup table in the N lookup tables is the image block corresponding to the arbitrary pixel point.

In one possible implementation, the N lookup tables are formed by lookup tables included in M lookup table sets, and each lookup table set includes a plurality of lookup tables. And the input of each lookup table in the jth lookup table set is the index area of the corresponding lookup table in the jth lookup table set determined by taking the output of the (j-1) th lookup table set as the reference. The input of each lookup table in the 1 st lookup table set is the index area of the corresponding lookup table in the 1 st lookup table set determined based on any one of the pixel points. The output of the mth lookup table set is the image block corresponding to any one of the pixel points.

In a third aspect, the present application provides a computing device comprising: at least one memory for storing a program; at least one processor for executing programs stored by the memory; wherein the processor is adapted to perform the method described in the first aspect or any one of the possible implementations of the first aspect when the memory stores a program that is executed.

In a fourth aspect, the present application provides a cluster of computing devices comprising at least one computing device, each computing device comprising a processor and a memory. The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method described in the first aspect or any one of the possible implementations of the first aspect.

In a fifth aspect, the present application provides a computer-readable storage medium storing a computer program which, when run on a processor, causes the processor to perform the method as described in the first aspect or any one of the possible implementations of the first aspect.

In a sixth aspect, the present application provides a computer program product for causing a processor to perform the method as described in the first aspect or any one of the possible implementations of the first aspect, when the computer program product is run on the processor.

It is understood that the beneficial effects of the second to sixth aspects can be seen from the description of the first aspect, and are not described herein again.

Drawings

Fig. 1 is a schematic diagram illustrating a comparison between CNN networks with different complexity levels for processing images according to an embodiment of the present application;

FIG. 2 is a schematic diagram of a process for obtaining a lookup table by using a neural network according to an embodiment of the present application;

FIG. 3 is a block diagram of an image processing system according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of an index area and an equivalent index area provided in an embodiment of the present application;

FIG. 5 is a schematic diagram illustrating rotation of an index region according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram illustrating a rotation of an output result of a lookup table according to an embodiment of the present application;

FIG. 7 is a hierarchical table-valued re-index diagram provided by an embodiment of the present application;

FIG. 8 is a diagram illustrating a quantized perceptive hierarchical neural network training scheme provided by an embodiment of the present application;

FIG. 9 is a diagram illustrating a multi-lookup table cooperative work according to an embodiment of the present application;

fig. 10 is a schematic diagram illustrating an adaptation process of a convolutional network to a Y-index pattern according to an embodiment of the present application;

FIG. 11 is a schematic diagram of a process for processing an image of Bayer pixels according to an embodiment of the present disclosure;

fig. 12 is a schematic flowchart of an image processing method according to an embodiment of the present application;

fig. 13 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application;

FIG. 14 is a schematic structural diagram of a computing device according to an embodiment of the present application;

fig. 15 is a schematic structural diagram of a computing device cluster according to an embodiment of the present application.

Detailed Description

The term "and/or" herein is an association relationship describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. The symbol "/" herein denotes an association of or, e.g., a/B denotes a or B.

The terms "first" and "second," and the like, in the description and in the claims herein are used for distinguishing between different objects and not for describing a particular order of the objects. For example, the first response message and the second response message, etc. are for distinguishing different response messages, not for describing a specific order of the response messages.

In the embodiments of the present application, the words "exemplary" or "such as" are used herein to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

In the description of the embodiments of the present application, unless otherwise specified, "a plurality" means two or more, for example, a plurality of processing units means two or more processing units, or the like; plural elements means two or more elements, and the like.

Illustratively, since the Receptive Field (RF) of a complex neural network is large, the image processing performance of the neural network can be improved. For example, as shown in fig. 1, (a) of fig. 1 is an input of a neural network; fig. 1 (B) is a result output by the neural network when the neural network is a simple CNN network and its receptive field is 17 × 17; fig. 1 (C) is a result output by the neural network when the neural network is a complex CNN network and its receptive field is 45 × 45; fig. 1 (D) is a true value, i.e., an original image. As can be seen from comparison of (B), (C), and (D) in fig. 1, the complex neural network has a larger receptive field, and therefore can produce a result closer to the true value, that is, the image processing performance is better.

Generally, the neural network may be converted into a look-up table (LUT) and the LUT may be applied in image processing to meet the deployment on the terminal device. Specifically, the trained neural network may be used to obtain the mapping relationship between the input data and the output data, and store the mapping relationship in the lookup table in the form of a key-value. Then, image processing is performed using the lookup table. For example, after an input data is obtained, the input data may be used to look up a corresponding numerical result in a look-up table, and output the result. Thereby completing the image processing.

For example, as shown in fig. 2 (a), a neural network for image processing may be trained, the neural network may be composed of several convolutional layers, and the input and output are single-channel images; unlike a general image processing neural network, the receptive field of the network is strictly limited to an input range with a length and width of 2x2, i.e. a set of output results depends only on the corresponding 4-dimensional input I ₀ ,I ₁ ,I ₂ ,I ₃ . The neural network can adopt the conventional inverse gradient propagation to realize parameter optimization.

After training is complete, a conversion from the neural network to a look-up table may be performed. As shown in fig. 2 (b), after the training of the network parameters is completed, the input of the neural network is constructed by traversing the sampled numerical combinations of the 4-dimensional pixels, and a corresponding output result is obtained through operation. And then, storing the result obtained by traversing in a lookup table in a key-value mode to obtain a 4-dimensional lookup table, namely 4DLUT.

Finally, the converted look-up table may be utilized for image processing. As shown in (c) of FIG. 2, given a set of data to be processed and represented by I ₀ Input I as center point ₀ ,I ₁ ,I ₂ ,I ₃ The numerical result already stored in the table is looked up. If the 4DLUT does not directly store the sampling point I ₀ ,I ₁ ,I ₂ ,I ₃ And obtaining key values of adjacent input sampling points according to the corresponding result, and calculating an output result through interpolation operation. Thereby, the image processing is completed. Illustratively, with continuing reference to FIG. 2 (c), when it is desired to identify a pixel point I in an image ₀ When processed, the input may be I ₀ ,I ₁ ,I ₂ ,I ₃ Then, the output pixel point can be obtained as

Namely to obtain I ₀ The corresponding image block; after image blocks corresponding to all pixel points in the image are obtained, the image blocks are respectively arranged at the positions of the pixel points corresponding to the image blocks, and the processed image can be obtained. In some embodiments, an image block may include one pixel point or may include a plurality of pixel points, which may be determined according to actual situations.

Although this approach can be deployed on terminal devices, the size of the memory required for the look-up table often increases exponentially as the receptive field of the neural network increases linearly. Taking 4DLUT as an example, 4DLUT is commonly required for 4-bit (bit) sampled images (2) ⁴ +1) ⁴ Space of =1.274MB stores the image processing result. While the LUT dimension is expanded to 1 dimension, 5DLUT is required (2) ⁴ +1) ⁵ Memory space of 21.665 MB. Therefore, as the dimension of the lookup table increases, the requirement for the storage space of the terminal device also increases, making it difficult to deploy on a terminal device with a smaller memory size. In addition, the size of the receptive field (i.e., the index range) of a single lookup table is storedDue to the limitations of the size of the device and the cost of hardware, only a lookup table with a small index range can be constructed, which results in limited performance of image processing.

In view of this, an embodiment of the present application provides an image processing method, which can increase the size of the receptive field of a lookup table on the premise that the dimension of the lookup table is not increased in the image processing process, and significantly alleviate the constraints of the scale of a memory on a terminal device and the hardware cost, so that the performance of the lookup table on various image processing tasks is greatly improved while the characteristics of high efficiency and low power consumption of the lookup table method are maintained.

By way of example, fig. 3 shows an architectural diagram of an image processing system. As shown in fig. 3, the image processing system 300 may include: a spatial domain complementary index based multi-table co-processing unit 310 and/or a table value re-indexing based multi-level processing unit 320.

The multi-table cooperative processing unit 310 based on the spatial domain complementary index is mainly used for defining a central pixel point of any local area in an input image as I ₀ And around a central point I ₀ Constructing a plurality of complementary indexing regions, wherein each indexing region includes a center point I therein ₀ . And determining a lookup table corresponding to each index region based on the constructed multiple complementary index regions. For example, each constructed index region may be input into a pre-trained neural network model to obtain a lookup table corresponding to each index region. For example, the complementarity described in this embodiment is different from the strict definition in the mathematical definition, and the complementarity described in this embodiment is a portion where the index regions may overlap. The complementation described in this embodiment only requires that each lookup table has an area that cannot be covered by the remaining lookup table indices. In addition, the multi-table cooperative processing unit 310 based on the airspace complementary index may further determine an image block corresponding to each pixel point based on a lookup table corresponding to each index area associated with each pixel point, so as to complete image processing.

For surrounding the center point I ₀ Constructing a plurality of complementary indexing regions to construct an indexing regionFor example, the number of pixels included in the index area may be determined by the dimension of the preset lookup table. For example, when the dimension of the lookup table is 4 dimensions, the index region includes 4 pixels, and the 4 pixels include the center point I ₀ 。

For the process of determining the lookup table, taking the 4-dimensional lookup table as an example, and determining a region in the input image, as shown in fig. 4, a central pixel point of the region may be defined as I ₀ . Then, around the center point I ₀ Constructing a plurality of complementary indices, i.e. LUTs ⁽¹⁾ Index region, LUT ⁽²⁾ Index region and LUT ⁽³⁾ An index area. Wherein, LUT ⁽¹⁾ The index region is [ I ₀ ][I ₁ ][I ₃ ][I ₄ ]，LUT ⁽²⁾ Index region of [ I ₀ ][I ₂ ][I ₆ ][I ₈ ]，LUT ⁽³⁾ Index region of [ I ₀ ][I ₄ ][I ₅ ][I ₇ ]. In FIG. 4, there is only a LUT ⁽¹⁾ The index region contains { I ₁ ,I ₃ LUT only ⁽²⁾ The index region contains { I ₂ ,I ₆ ,I ₈ LUT only ⁽³⁾ The index region contains { I ₅ ,I ₇ Therefore, the index areas corresponding to the lookup tables can be complemented with each other to enlarge the receptive field. Of course, other lookup tables with complementary indexes may be used, and different lookup tables may be complementary to each other, which is not limited herein. After a plurality of complementary index regions are determined, the index regions may be respectively input into a pre-trained neural network model, so that the neural network model outputs a lookup table corresponding to each index region.

After determining the lookup tables of the plurality of complementary index regions, the center point I may be determined ₀ The average calculation is carried out on the corresponding output results in each lookup table, namely the required result is obtained, namely the central point I is obtained ₀ The corresponding image block. For example, with continued reference to FIG. 4, center point I ₀ Corresponding output result V ₀ ＝(LUT ⁽¹⁾ [I ₀ ][I ₁ ][I ₃ ][I ₄ ]+LUT ⁽²⁾ [I ₀ ][I ₂ ][I ₆ ][I ₈ ]+LUT ⁽³⁾ [I ₀ ][I ₄ ][I ₅ ][I ₇ ])/3. When the input is [ I ] ₀ ][I ₁ ][I ₃ ][I ₄ ]When the index region corresponding to the output result is equivalent to the index region represented by [ I ] ₀ ][I ₁ ][I ₃ ][I ₄ ][I ₅ ][I ₆ ][I ₇ ][I ₈ ]And (4) forming. Because the index area corresponding to the output result corresponding to the lookup table is increased, the receptive field of the lookup table is also increased correspondingly. Meanwhile, the dimensionality of the lookup table is not changed, so that the receptive field of the lookup table is increased on the premise of not increasing the dimensionality of the lookup table. Wherein, when the input data is [ I ] ₀ ][I ₁ ][I ₃ ][I ₄ ]Then, the output result V can be obtained by searching through the lookup table ₀ . For the lookup tables with other dimensions, the way of increasing the receptive field of the 4-dimensional lookup table may be referred to, and details are not repeated here.

In some embodiments, in order to further increase the receptive field of the lookup table without increasing the dimensionality of the lookup table, the index regions corresponding to the constructed lookup tables may be further rotated and based on the central point I ₀ Determining the central point I according to the output results of the lookup table corresponding to the index area before rotation and the lookup table corresponding to the index area after rotation ₀ And finally outputting the result in the corresponding lookup table. Specifically, for any one of the look-up tables in fig. 4, with LUT ⁽¹⁾ For example, as shown in FIG. 5 (A), the index area may be first set to [ I ] ₀ ][I ₁ ][I ₃ ][I ₄ ]Determining the center point I ₀ Corresponding output V under the index region _j (1). Then, as shown in fig. 5 (B), the index area in fig. 5 (a) is rotated 90 degrees counterclockwise, and the index area at this time becomes [ I ₀ ][I ₁ ][I ₉ ][I ₁₀ ]Then may be [ I ] based on the index region ₀ ][I ₁ ][I ₉ ][I ₁₀ ]Determining the center point I ₀ Corresponding output V under the index region _j (2). Followed byAs shown in FIG. 5C, the index area in FIG. 5B is rotated 90 degrees counterclockwise, and the index area at this time becomes [ I ] ₀ ][I ₉ ][I ₁₁ ][I ₁₂ ]Then may be [ I ] based on the index region ₀ ][I ₉ ][I ₁₁ ][I ₁₂ ]Determining the center point I ₀ Corresponding output V under the index region _j (3). Subsequently, as shown in fig. 5 (D), the index area in fig. 5 (C) is rotated 90 degrees counterclockwise, and the index area at this time becomes [ I ₀ ][I ₃ ][I ₁₂ ][I ₁₃ ]Then may be [ I ] based on the index region ₀ ][I ₃ ][I ₁₂ ][I ₁₃ ]Determining the center point I ₀ Corresponding output V under the index region _j (4). Due to V _j (2) And V _j (1) Not in the same calculation region, V may be set for uniformity _j (2) Rotated 90 degrees clockwise to obtain V _j (2)'. Likewise, V may be _j (3) Rotate clockwise by 180 degrees to obtain V _j (3) ', and, will V _j (4) Rotate clockwise 270 degrees to get V _j (4)'. For example, as shown in (A) of FIG. 6, assume V _j (3) Is [1 ]][2][3][4]When the rotation angle is 90 degrees clockwise, the result shown in fig. 6 (B) can be obtained; further, the clockwise rotation is performed by 90 degrees based on fig. 6 (B), and the result shown in fig. 6 (C) can be obtained, wherein the result shown in fig. 6 (C) is the desired result V _j (3)'. Finally, V can be adjusted _j (1)、V _j (2)′、V _j (3) ' and V _j (4) ' add and average, and take the result of the calculation as: when the input is [ I ] ₀ ][I ₁ ][I ₃ ][I ₄ ]Time-through LUT ⁽¹⁾ The obtained output result, i.e., V = (V) _j (1)+V _j (2)′+V _j (3)′+V _j (4)')/4. For the way of adding the receptive fields of other lookup tables, the processing can be performed in the way described in fig. 5, and will not be described herein again. In this embodiment, the rotation angle can be selected according to actual conditions, and is not limited herein, and for example, only 90 degrees of rotation may be selected, or only 90 degrees of rotation may be selectedThe rotation is 180 degrees or 270 degrees, or alternatively 90 degrees and 180 degrees, or alternatively 180 degrees and 270 degrees, and so on. Similarly, the center point I is determined ₀ After the corresponding output results in each lookup table, the results can be added and averaged, and the calculation result is used as the final required result, and the association relationship between the input and the output can be established in the lookup table, so as to obtain the latest lookup table. Illustratively, with continued reference to FIG. 4, when the selection is rotated 90 degrees, 180 degrees, and 270 degrees for each look-up table, the resulting equivalent index region may be the equivalent index region shown in FIG. 4. By the method, the index area corresponding to the output result corresponding to the lookup table is further increased, so that the receptive field of the lookup table is further increased.

The multi-level processing unit 320 based on table value re-indexing is mainly used for processing a data object with a certain center point I ₀ The associated plurality of look-up tables are hierarchically processed, wherein the input of the k-th level look-up table is the output of the (k-1) -th level look-up table. Thereby expanding the equivalent field of view by cascading inputs and outputs between multiple look-up tables. Illustratively, as shown in FIG. 7, the LUT is a cascaded process of two levels of look-up tables ⁽¹⁾ Representing first-level look-up tables, LUTs ⁽²⁾ Representing a secondary look-up table. In FIG. 7, the LUT may be used ⁽¹⁾ Using the index value result in the secondary LUT ⁽²⁾ The hierarchical indexing process can be expressed as:

V＝LUT ⁽²⁾ [LUT ⁽¹⁾ [I _* ]][LUT ⁽¹⁾ [I _* ]][LUT ⁽¹⁾ [I _* ]][LUT ⁽¹⁾ [I _* ]]

when an image of low quality is input to the multi-level processing unit 320 based on table-valued re-indexing, an image of high quality can be output through the level processing by the multi-level processing unit 320 based on table-valued re-indexing.

From a receptive field perspective, this cascading framework is similar in principle to cascading multiple convolutional layers in a deep convolutional neural network. However, unlike convolutional layer cascading in CNN, indexing of image data is general due to the size limitation of LUTOften in the format of an integer number. Whereas the training of the neural network requires a gradient of the floating-Point data type before the aforementioned step of converting from the neural network to the look-up table described in fig. 2. In order to solve the problem caused by the data type, a quantization perceptible re-indexing mechanism is designed in the embodiment, and a hierarchical indexing mode is introduced into the training process of the neural network. As shown in fig. 8, LUT ⁽¹⁾ The output value of (b) is quantized to an Integer value in the forward pass (i.e., the direction indicated by the dashed line pointing to Integer), while its gradient is retained as a floating Point value in the reverse pass (i.e., the direction indicated by the dashed line pointing to Float-Point). In this way, the numerical quantification operation can be perceived and modeled during the hierarchical training process of the neural network, thereby compensating for the performance loss caused by the numerical quantification.

In some embodiments, the spatial domain complementary index based multi-table co-processing unit 310 and the table value re-index based multi-level processing unit 320 may be used separately or in combination. Illustratively, FIG. 9 shows a process in which the two are used in combination. As shown in fig. 9, a Multiple lookup table (Multiple LUT) is abbreviated as a MuLUT, and the MuLUT-S/D/Y respectively indicates three LUTs having different pixel index patterns. Block 1x1 indicates that the resolution of the output image remains the same as the input image, and Block 2x2 indicates that the resolution of the output image is 2 times that of the input image. The neural networks corresponding to the three LUTs are all realized by stacking and connecting a plurality of convolution layers, wherein the first layer of the convolution network corresponding to the S-shaped LUT adopts standard convolution with a convolution kernel size of 2x2 and is marked as Conv2x2. The first layer of the convolution network corresponding to the D-type LUT adopts hole convolution (scaled convolution) with a convolution kernel size of 2x2, and is marked as Conv2x2-D2. The first layer of the convolutional network corresponding to the Y-type LUT is implemented by a standard convolution with a convolution kernel size of 1x4, which is denoted as Conv1x4. Since the Y-index mode does not belong to the conventional grid index, the conventional function library cannot be directly called for operation. The operation of the corresponding convolutional network therefore requires special adaptation. As shown in fig. 10, first, the input image is split into 3 × 3 image blocks in a sliding window manner, i.e., an underfold operation. And then selecting a pixel coordinate corresponding to the Y-type index mode from the obtained image blocks, and rearranging the numerical values under the corresponding coordinates to 1x4 image blocks. The image after splitting and rearranging can be adapted to the convolution network with the Conv1x4 layer at the first layer.

With continued reference to FIG. 9, in FIG. 9, first, 3 and the center point I are selected ₀ And respectively inputting the data in each index range into the convolution network corresponding to each index range, wherein the output results of the convolution networks corresponding to each index range are 1-dimensional data. Then, the output results of the convolution networks are combined into one, for example, an averaging method. Then, taking the result as a central point, determining index ranges of a plurality of lookup tables related to the result, and inputting the data in each index range into each corresponding convolution network, wherein the results output by the convolution networks corresponding to the index ranges are all data with the same dimension as the dimension of the required lookup table. Finally, the data can be combined into one and established with a certain contained center point I ₀ The index range of the index table is obtained, and a new lookup table is built. The reception field of the lookup table is larger than that of the previous lookup table, and the dimension is unchanged.

In some embodiments, when the pixels of the input image are in bayer pixel arrangement mode, the LUT query may be performed on the input within a 2 × 2 window by sliding the window in 2 times steps, thereby avoiding the effect of bayer pixel pattern. Then, the image processing system 300 described above is used to perform multi-lookup table cooperative processing, and/or the values of the lookup table in the first stage are used as the input in the second stage, so as to implement hierarchical processing. For example, as shown in fig. 11, the pixel points of the red channel, the pixel points of the green channel, and the pixel points of the blue channel may be extracted in 2-time steps, and then the extracted pixel points are processed by using the image processing system 300 described above.

Next, based on the foregoing description, an image processing method provided in the embodiment of the present application will be described.

Illustratively, fig. 12 shows a flow of an image processing method. It is to be appreciated that the method can be performed by any computing, processing capable apparatus, device, platform, cluster of devices. As illustrated in fig. 12, the image processing method may include the steps of:

s1201, acquiring a first image to be processed.

In this embodiment, after the user inputs an image to be processed, the first image to be processed may be acquired. In addition, the first image to be processed may also be automatically acquired from a storage area set by the user. The concrete can be determined according to actual conditions.

S1202, for any pixel point in the first image, determining an image block corresponding to the any pixel point by using N lookup tables (N is more than or equal to 2) associated with the any pixel point, wherein the image block comprises a single pixel point or a plurality of pixel points, an index area of each lookup table comprises the any pixel point, and the index areas of different lookup tables are different and at least partially the same.

In this embodiment, for any one pixel point in the first image, the image block corresponding to the any one pixel point may be determined by using a plurality of lookup tables associated with the any one pixel point. The image block comprises a single or a plurality of pixel points. In addition, the index range of each lookup table includes the arbitrary pixel, and the index ranges of different first lookup tables are different and at least partially the same, that is, the index ranges of different lookup tables are complementary. For example, with continued reference to FIG. 4, the arbitrary pixel point can be the center point I ₀ I.e. LUT ⁽¹⁾ 、LUT ⁽²⁾ And LUT ⁽³⁾ May be the center point I ₀ An associated look-up table. The mode of determining the image block corresponding to any pixel point by using a plurality of lookup tables associated with the any pixel point can be referred to the aforementioned "spatial domain complementary index-based multi-table cooperative processing unit 310 to obtain the central point I ₀ The corresponding process of the image block "is not described herein again.

As a possible implementation manner, the index area of each of the N lookup tables may be input into the corresponding lookup table, so as to obtain N first sub image blocks. Each first sub image block comprises a single or a plurality of pixel points. Then, the pixel points at the same position in the N first sub-image blocks are added and averaged to obtain an image block corresponding to any one pixel point. Because the index area corresponding to the output result corresponding to the lookup table is increased, the receptive field of the lookup table is also increased correspondingly. Meanwhile, the dimensionality of the lookup table is not changed, so that the receptive field of the lookup table is increased on the premise of not increasing the dimensionality of the lookup table. See the foregoing description of fig. 4 for details, which are not repeated herein.

Further, for the index area of any one lookup table in the N lookup tables, the index area of any one lookup table may be centered on any one pixel point and rotated M times along the first direction to obtain M index areas, where M is greater than or equal to 1. Then, second sub image blocks corresponding to the M index areas are determined based on the lookup table corresponding to each of the M index areas, so as to obtain M second sub image blocks. And each second sub image block comprises a single or a plurality of pixel points. Then, the second sub image blocks corresponding to the M index areas may be rotated by the same angle in a direction opposite to the first direction based on the rotation angles corresponding to the M index areas, respectively, to obtain M third sub image blocks. And finally, adding and averaging pixel points at the same position in the first sub-image block corresponding to the index area of any one lookup table and the M third sub-image blocks, and taking the calculation result as the first sub-image block corresponding to the index area of any one lookup table. Therefore, the receptive field of the lookup table is increased on the premise of not increasing the dimensionality of the lookup table by rotating the index area. See the foregoing description regarding fig. 5 and 6 for details, which are not repeated herein.

As another possible implementation manner, the output of the ith lookup table in the N lookup tables is the input of the (i + 1) th lookup table, the input of the 1 st lookup table in the N lookup tables is the index area of the 1 st lookup table, and the output of the nth lookup table in the N lookup tables is the image block corresponding to any pixel point. That is, N look-up tables may be cascaded to thereby achieve an increased receptive field of the look-up tables.

As another possible implementation manner, the N lookup tables may be formed by lookup tables included in M lookup table sets, and each lookup table set includes a plurality of lookup tables. And the input of each lookup table in the jth lookup table set is the index area of the corresponding lookup table in the jth lookup table set determined by taking the output of the (j-1) th lookup table set as the reference. The input of each lookup table in the 1 st lookup table set is the index region of the corresponding lookup table in the 1 st lookup table set determined by taking any one of the pixel points as the reference. The output of the mth lookup table set is the image block corresponding to the aforementioned any one pixel point. For example, the processing procedure may be understood as the processing procedure in fig. 9, and is described in detail in the foregoing description, which is not described again here.

And S1203, obtaining a second image according to the image blocks corresponding to the pixel points in the first image, wherein the resolution of the second image is higher than or equal to that of the first image.

In this embodiment, after obtaining image blocks corresponding to each pixel point in the first image, the image blocks may be spliced together to obtain the second image. Wherein the resolution of the second image is higher than or equal to the resolution of the first image. In some embodiments, when the image blocks are spliced, the image blocks may be placed at the positions of their respective corresponding pixel points. For example, if pixel 1 in the first image corresponds to image block 1, then image block 1 may be placed at the position of pixel 1.

And S1204, outputting a second image.

In this embodiment, after the second image is acquired, the second image may be output. Such as on a display screen of the device, etc.

Therefore, in the image processing process, a plurality of lookup tables are used for each pixel point to work cooperatively, so that the receptive field is expanded under the condition of maintaining the linear increase of the calculated amount and the memory amount, and the image processing performance is improved. In contrast, existing lookup table techniques that utilize spatial domain information use only a single lookup table, and the expansion of the receptive field may result in an exponential increase in memory, which may be limited by memory size and hardware cost.

Based on the method in the embodiment, the embodiment of the application also provides an image processing device.

Exemplarily, fig. 13 shows an image processing apparatus. As shown in fig. 13, the image processing apparatus 1300 may include an acquisition module 1301 and a processing module 1302. The obtaining module 1301 is configured to obtain a first image. The processing module 1302 is configured to determine, for any pixel in the first image, an image block corresponding to the any pixel by using N (N is greater than or equal to 2) lookup tables associated with the any pixel, where the image block includes a single or multiple pixels, an index area of each lookup table includes the any pixel, and index areas of different lookup tables are different and at least partially the same. The processing module 1302 is further configured to obtain a second image according to an image block corresponding to each pixel point in the first image, where a resolution of the second image is higher than or equal to a resolution of the first image; and outputting the second image.

In some embodiments, when the processing module 1302 determines, by using N lookup tables associated with any one pixel, an image block corresponding to any one pixel, the processing module is specifically configured to: inputting the index area of each lookup table in the N lookup tables into the corresponding lookup table to obtain N first sub-image blocks, wherein each first sub-image block comprises a plurality of pixel points; and adding and averaging the pixel points at the same position in the N first sub-image blocks to obtain an image block corresponding to any one pixel point.

In some embodiments, when the processing module 1302 inputs the index area of each of the N lookup tables into a corresponding lookup table to obtain N first sub image blocks, the processing module is specifically configured to: aiming at the index area of any one lookup table in the N lookup tables, rotating the index area of the any lookup table for M times along a first direction by taking any pixel point as a center to obtain M index areas, wherein M is more than or equal to 1; determining second sub image blocks corresponding to the M index areas respectively based on a lookup table corresponding to each index area in the M index areas to obtain M second sub image blocks, wherein each second sub image block comprises a single or multiple pixel points; respectively rotating the second sub image blocks corresponding to the M index areas by the same angle along the direction opposite to the first direction based on the rotation angles corresponding to the M index areas respectively to obtain M third sub image blocks; and adding and averaging pixel points at the same position in the first sub-image block corresponding to the index area of any one lookup table and the M third sub-image blocks, and taking the calculation result as the first sub-image block corresponding to the index area of any one lookup table.

In some embodiments, the output of the ith lookup table in the N lookup tables is the input of the (i + 1) th lookup table, the input of the 1 st lookup table in the N lookup tables is the index region of the 1 st lookup table, and the output of the nth lookup table in the N lookup tables is the image block corresponding to any one pixel point.

In some embodiments, the N lookup tables are comprised of lookup tables included in M sets of lookup tables, each set of lookup tables including a plurality of lookup tables. And the input of each lookup table in the jth lookup table set is the index area of the corresponding lookup table in the jth lookup table set determined by taking the output of the (j-1) th lookup table set as the reference. The input of each lookup table in the 1 st lookup table set is the index area of the corresponding lookup table in the 1 st lookup table set determined by taking any pixel point as a reference. The output of the Mth lookup table set is an image block corresponding to any pixel point.

It should be understood that, the above-mentioned apparatus is used for executing the method in the above-mentioned embodiments, and the implementation principle and technical effect of the corresponding program module in the apparatus are similar to those described in the above-mentioned method, and the working process of the apparatus may refer to the corresponding process in the above-mentioned method, and is not described herein again.

In some embodiments, the acquisition module 1301 and the processing module 1302 shown in fig. 13 may both be implemented by software, or may be implemented by hardware. Illustratively, taking the obtaining module 1301 shown in fig. 13 as an example, an implementation manner of the obtaining module 1301 is described next. Similarly, the implementation of the processing module 1302 shown in fig. 13 may refer to the implementation of the obtaining module 1301 shown in fig. 13.

Module as an example of a software functional unit, the processing module 1302 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container, among others. Further, the above calculation examples may be one or more. For example, the processing module 1302 may include code running on multiple hosts/virtual machines/containers. It should be noted that multiple hosts/virtual machines/containers for running the code may be distributed in the same region (region) or may be distributed in different regions. Further, multiple hosts/virtual machines/containers for running the code may be distributed in the same Available Zone (AZ) or may be distributed in different azis, each AZ including one data center or multiple data centers with close geographic locations. Wherein, in general, one region may comprise a plurality of AZ.

Likewise, multiple hosts/virtual machines/containers for running the code may be distributed in the same Virtual Private Cloud (VPC), or may be distributed in multiple VPCs. In general, one VPC is arranged in one region, cross-region communication between two VPCs in the same region and between VPCs of different regions needs to arrange a communication gateway in each VPC, and interconnection between VPCs is realized through the communication gateway.

Modules as an example of hardware functional units, the processing module 1302 may include at least one computing device, such as a server or the like. Alternatively, the processing module 1302 may be a device implemented by an application-specific integrated circuit (ASIC), a Programmable Logic Device (PLD), or the like. The PLD may be a Complex Programmable Logic Device (CPLD), a field-programmable gate array (FPGA), a General Array Logic (GAL), or any combination thereof.

The processing module 1302 may include multiple computing devices distributed in the same region or distributed in different regions. The processing module 1302 may include multiple computing devices distributed in the same AZ or distributed in different azis. Likewise, the processing module 1302 can include multiple computing devices distributed in the same VPC or distributed across multiple VPCs. Wherein the plurality of computing devices may be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

It should be noted that in other embodiments, the processing module 1302 may be configured to perform any step of the method shown in fig. 12, and the processing module 1302 shown in fig. 13 may be configured to perform any step of the method provided in fig. 12. The steps of the acquisition module 1301 and the processing module 1302 shown in fig. 13 can be designated as required, and the acquisition module 1301 and the processing module 1302 respectively implement different steps of the method provided in fig. 12 to implement all functions of the image processing apparatus 1300.

The present application also provides a computing device 1400. As shown in fig. 14, computing device 1400 includes: a bus 1402, a processor 1404, a memory 1406, and a communication interface 1408. Communication between the processor 1404, the memory 1406, and the communication interface 1408 is via a bus 1402. Computing device 1400 may be a server or a terminal device. It should be understood that the present application does not limit the number of processors, memories in the computing device 1400.

The bus 1402 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one line is shown in FIG. 14, but it is not intended that there be only one bus or one type of bus. Bus 1404 may include a pathway to transfer information between various components of computing device 1400 (e.g., memory 1406, processor 1404, communication interface 1408).

The processor 1404 may include any one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), a Micro Processor (MP), or a Digital Signal Processor (DSP).

Memory 1406 may include volatile memory (volatile memory), such as Random Access Memory (RAM). The processor 144 may also include non-volatile memory (non-volatile memory), such as read-only memory (ROM), flash memory, a Hard Disk Drive (HDD), or a Solid State Drive (SSD).

The memory 1406 stores executable program code, and the processor 1404 executes the executable program code to implement the functions of the acquisition module 1301 and the processing module 1302 shown in fig. 13, respectively, so as to implement all or part of the steps of the method in the above-described embodiments. That is, the memory 146 stores instructions for performing all or some of the steps of the method of the above-described embodiments.

Alternatively, the memory 1406 stores executable codes, and the processor 1404 executes the executable codes to realize the functions of the image processing apparatus 1300, respectively, so as to realize all or part of the steps in the method of the above embodiment. That is, the memory 1406 has stored thereon instructions for carrying out all or a portion of the steps of the method of the above-described embodiments.

The communication interface 1403 enables communication between the computing device 140 and other devices or communication networks using transceiver modules such as, but not limited to, network interface cards, transceivers, and the like.

The embodiment of the application also provides a computing device cluster. The cluster of computing devices includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a notebook computer, or a smart phone.

As shown in fig. 15, the cluster of computing devices includes at least one computing device 1400. Memory 1406 in one or more computing devices 1400 in the cluster of computing devices may have stored therein the same instructions for performing all or a portion of the steps of the method of the embodiments described above.

In some possible implementations, the memory 1406 of one or more computing devices 1400 in the computing device cluster may also store a portion of the instructions for performing all or part of the steps of the method of the above embodiments. In other words, a combination of one or more computing devices 1400 can collectively execute instructions for performing all or a portion of the steps of the above-described embodiment methods.

It should be noted that the memory 1406 in different computing devices 1400 in the computing device cluster can store different instructions for respectively executing partial functions of the image processing apparatus 1300. That is, instructions stored by memory 1406 in the different computing device 1400 may implement the functionality of one or more of the aforementioned acquisition module 814 and processing module 820 shown in fig. 13.

In some possible implementations, one or more computing devices in a cluster of computing devices may be connected over a network. Wherein the network may be a wide area network or a local area network, etc.

Based on the methods in the foregoing embodiments, embodiments of the present application provide a computer-readable storage medium, which stores a computer program, and when the computer program runs on a processor, causes the processor to execute the methods in the foregoing embodiments.

Based on the methods in the foregoing embodiments, the present application provides a computer program product, which when run on a processor, causes the processor to execute the methods in the foregoing embodiments.

It is understood that the processor in the embodiments of the present application may be a Central Processing Unit (CPU), other general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The general purpose processor may be a microprocessor, but may be any conventional processor.

The method steps in the embodiments of the present application may be implemented by hardware, or may be implemented by software instructions executed by a processor. The software instructions may consist of corresponding software modules that may be stored in Random Access Memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically Erasable PROM (EEPROM), registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an ASIC.

In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in or transmitted over a computer-readable storage medium. The computer instructions may be transmitted from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.

It is to be understood that the various numerical references referred to in the embodiments of the present application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of the present application.

Claims

1. An image processing method, characterized in that the method comprises:

acquiring a first image;

for any pixel point in the first image, determining an image block corresponding to the any pixel point by using N lookup tables (N is more than or equal to 2) associated with the any pixel point, wherein the image block comprises a single or a plurality of pixel points, the index region of each lookup table comprises the any pixel point, and the index regions of different lookup tables are different and at least partially the same;

obtaining a second image according to the image blocks corresponding to the pixel points in the first image, wherein the resolution of the second image is higher than or equal to that of the first image;

and outputting the second image.

2. The method according to claim 1, wherein the determining the image block corresponding to the any one pixel point by using N (N ≧ 2) lookup tables associated with the any one pixel point specifically includes:

inputting the index area of each lookup table in the N lookup tables into a corresponding lookup table to obtain N first sub-image blocks, wherein each first sub-image block comprises a single or multiple pixel points;

and adding and averaging the pixel points at the same position in the N first sub-image blocks to obtain an image block corresponding to any one pixel point.

3. The method according to claim 2, wherein the inputting the index area of each of the N lookup tables into a corresponding lookup table to obtain N first sub image blocks specifically includes:

aiming at the index area of any one lookup table in the N lookup tables, taking the index area of the any lookup table as the center and rotating for M times along a first direction to obtain M index areas, wherein M is more than or equal to 1;

determining second sub image blocks corresponding to the M index areas respectively based on a lookup table corresponding to each index area in the M index areas to obtain M second sub image blocks, wherein each second sub image block comprises a single or multiple pixel points;

respectively rotating second sub image blocks corresponding to the M index areas by the same angle along a direction opposite to the first direction based on the rotation angles corresponding to the M index areas respectively to obtain M third sub image blocks;

and adding and averaging the pixels at the same position in the first sub-image block corresponding to the index area of any one lookup table and the pixels at the same position in the M third sub-image blocks, and taking the calculation result as the first sub-image block corresponding to the index area of any one lookup table.

4. The method according to claim 1, wherein an output of an ith lookup table in the N lookup tables is an input of an (i + 1) th lookup table, an input of a 1 st lookup table in the N lookup tables is an index region of the 1 st lookup table, and an output of an nth lookup table in the N lookup tables is an image block corresponding to the arbitrary pixel point.

5. The method according to claim 1, wherein the N look-up tables are comprised of look-up tables included in M sets of look-up tables, each set of look-up tables including a plurality of look-up tables;

the input of each lookup table in the jth lookup table set is an index area of a corresponding lookup table in the jth lookup table set determined by taking the output of the (j-1) th lookup table set as a reference;

the input of each lookup table in the 1 st lookup table set is an index area of a corresponding lookup table in the 1 st lookup table set determined by taking any pixel point as a reference;

and the output of the Mth lookup table set is the image block corresponding to the any pixel point.

6. An image processing apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring a first image;

the processing module is used for determining an image block corresponding to any pixel point in the first image by using N (N is more than or equal to 2) lookup tables associated with the any pixel point, wherein the image block comprises a single pixel point or a plurality of pixel points, the index area of each lookup table comprises the any pixel point, and the index areas of different lookup tables are different and at least partially the same;

the processing module is further configured to obtain a second image according to the image block corresponding to each pixel point in the first image, where the resolution of the second image is higher than or equal to the resolution of the first image; and outputting the second image.

7. The apparatus according to claim 6, wherein the processing module, when determining the image block corresponding to the any one pixel point by using N (N ≧ 2) lookup tables associated with the any one pixel point, is specifically configured to:

8. The apparatus according to claim 7, wherein the processing module, when inputting the index area of each of the N lookup tables into the corresponding lookup table to obtain N first sub image blocks, is specifically configured to:

aiming at the index area of any one lookup table in the N lookup tables, rotating the index area of any one lookup table for M times along a first direction by taking any pixel point as a center to obtain M index areas, wherein M is more than or equal to 1;

9. The apparatus according to claim 6, wherein an output of an ith lookup table of the N lookup tables is an input of an (i + 1) th lookup table, an input of a 1 st lookup table of the N lookup tables is an index region of the 1 st lookup table, and an output of an nth lookup table of the N lookup tables is an image block corresponding to the arbitrary pixel point.

10. The method of claim 6, wherein the N lookup tables are comprised of lookup tables included in M sets of lookup tables, each set of lookup tables including a plurality of lookup tables;

and the output of the Mth lookup table set is the image block corresponding to any pixel point.

11. A computing device, comprising:

at least one memory for storing a program;

at least one processor for executing programs stored by the memory;

wherein the processor is configured to perform the method of any of claims 1-5 when the program stored in the memory is executed.

12. A cluster of computing devices comprising at least one computing device, each computing device comprising a processor and a memory;

the processor of the at least one computing device is to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of any of claims 1-5.

13. A computer-readable storage medium, having stored thereon a computer program which, when run on a processor, causes the processor to carry out the method according to any one of claims 1-5.

14. A computer program product, characterized in that, when run on a processor, causes the processor to execute the method according to any of claims 1-5.