WO2022016927A1

WO2022016927A1 - Image processing method and apparatus, terminal device, and storage medium

Info

Publication number: WO2022016927A1
Application number: PCT/CN2021/088428
Authority: WO
Inventors: 蒋文
Original assignee: 深圳云天励飞技术股份有限公司
Priority date: 2020-07-22
Filing date: 2021-04-20
Publication date: 2022-01-27
Also published as: CN111915673A; CN111915673B

Abstract

An image processing method and apparatus, a terminal device, and a storage medium, which are applied in the technical field of computers. The method comprises: determining at least two adjacent target source pixel points corresponding to a pixel point to be generated in an image to be generated, the target source pixel points being source pixel points in a source image (S101); according to location information of each target source pixel point in the source image, respectively acquiring pixel data of each target source pixel point from different preset memories, each preset memory storing a source pixel point of a location type, and the location type being determined according to location information of the source pixel point in the source image (S102); and performing an interpolation operation according to the pixel data of each target source pixel point, so as to determine target pixel data of the pixel point to be generated (S103). In the foregoing method, image processing efficiency can be improved.

Description

Image processing method, device, terminal device and storage medium

This application claims the priority of the Chinese patent application filed on June 4, 2020 with the application number 202010500223.X and the invention titled "Image Processing Method, Apparatus, Terminal Equipment and Storage Medium", the entire contents of which are approved by Reference is incorporated in this application.

technical field

The present application belongs to the field of computer technology, and in particular, relates to an image processing method, apparatus, terminal device and storage medium.

Background technique

In the existing image processing technologies such as image alignment, image transformation, image correction, etc., there is often a processing process that needs to determine the pixel data of the target image according to the pixel data of the source image.

In this processing process, the pixel data of a plurality of source pixel points need to be read multiple times in order to determine the pixel data of one pixel point of the target image. However, this way of reading pixel data is inefficient, thereby reducing the efficiency of image processing.

SUMMARY OF THE INVENTION

In view of this, the embodiments of the present application provide an image processing method, an apparatus, a terminal device, and a storage medium, so as to solve the problem of how to improve the image processing efficiency in the prior art.

A first aspect of the embodiments of the present application provides an image processing method, including:

Determine at least two adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image, where the target source pixel points are the source pixel points in the source image;

According to the position information of each target source pixel in the source image, the pixel data of each target source pixel is obtained from different preset memories, wherein each of the preset memories stores a The source pixel point of the position type, the position type is determined according to the position information of the source pixel point in the source image;

An interpolation operation is performed according to the pixel data of each of the target source pixel points, and the target pixel data of the to-be-generated pixel point is determined.

A second aspect of the embodiments of the present application provides an image processing apparatus, including:

a target source pixel point determination unit, configured to determine at least two adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image, where the target source pixel points are the source pixel points in the source image;

A pixel data acquisition unit, configured to acquire pixel data of each of the target source pixels from different preset memories according to the position information of each of the target source pixels in the source image, wherein each of the target source pixels The preset memories respectively store source pixels of a position type, and the position type is determined according to the position information of the source pixels in the source image;

The target pixel data determination unit is configured to perform interpolation operation according to the pixel data of each of the target source pixel points, and determine the target pixel data of the to-be-generated pixel point.

A third aspect of the embodiments of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program At the time, the terminal device is made to implement the steps of the image processing method.

A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, enables a terminal device to perform image processing as described above steps of the method.

A fifth aspect of the embodiments of the present application provides a computer program product that, when the computer program product runs on a terminal device, causes the terminal device to execute the steps of the image processing method described in the first aspect.

The beneficial effects of the embodiments of the present application compared with the prior art are: in the embodiments of the present application, when the target pixel data of the pixels to be generated needs to be determined, the corresponding data can be obtained from different preset memories at one time. The pixel data of multiple target source pixel points corresponding to the current pixel to be generated, perform interpolation operation according to the read pixel data of the target source pixel to obtain the target pixel data of the pixel to be generated, that is, it can be read at one time. The pixel data of multiple target source pixel points required for the interpolation operation, without waiting to read the pixel data multiple times, so the reading efficiency of the pixel data can be improved, and the target pixel data of the pixel to be generated can be determined efficiently, Further, the image processing efficiency based on interpolation operation is improved.

Description of drawings

FIG. 1 is a schematic diagram of an implementation flowchart of a first image processing method provided by an embodiment of the present application;

2 is a schematic diagram of a to-be-generated pixel point and two corresponding adjacent target source pixel points provided by an embodiment of the present application;

3 is a schematic diagram of a to-be-generated pixel point and corresponding four adjacent target source pixel points provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of an implementation flowchart of a second image processing method provided by an embodiment of the present application;

5 is a schematic diagram of a storage process flow of pixel data of a source pixel point provided by an embodiment of the present application;

6 is a hardware architecture diagram of a bus control interface module provided by an embodiment of the present application;

7 is a schematic diagram of a bus data alignment provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of a storage flow of pixel data of another source pixel point provided by an embodiment of the present application;

9 is a schematic diagram of a hardware architecture of an image processing apparatus provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of an implementation flowchart of a third image processing method provided by an embodiment of the present application;

11 is a schematic diagram of the architecture of an STN provided by an embodiment of the present application;

12 is a schematic diagram of an image processing apparatus provided by an embodiment of the present application;

FIG. 13 is a schematic diagram of a terminal device provided by an embodiment of the present application.

detailed description

In order to illustrate the technical solutions described in the present application, the following specific embodiments are used for description.

Example 1:

The image processing methods in the embodiments of the present application are applied to application scenarios such as image transformation (such as image zoom-in transformation, image translation transformation, and image rotation transformation), image correction (that is, obtaining a target image according to distorted image correction). In these application scenarios, it is necessary to generate target images (images after transformation and correction) according to the information of the source images (images before transformation and correction). In this process, the target image to be generated (image to be generated) The determined image processing method of the pixel data of each pixel to be generated is the key link. The image processing method in the embodiment of the present application determines at least two adjacent source pixels corresponding to the pixels to be generated in the image to be generated, and then simultaneously obtains each source pixel from different preset memories at one time After the pixel data is generated, interpolation operation is performed to determine the target pixel data of the pixel points to be generated, which improves the image processing efficiency.

FIG. 1 shows a schematic flowchart of a first image processing method provided by an embodiment of the present application, and details are as follows:

In S101, at least two adjacent target source pixels corresponding to to-be-generated pixels in the to-be-generated image are determined, where the target source pixels are source pixels in the source image.

In the embodiment of the present application, the pixels of the source image are called source pixels; the image to be generated has a certain mapping relationship with the source image, and according to the mapping relationship, the source corresponding to the pixels to be generated can be determined from the source image. Pixel point, the source pixel point corresponding to the pixel point to be generated is called the target source pixel point. In the image to be generated in the embodiment of the present application, the pixel data of the pixel to be generated is specifically determined according to the pixel data of at least two adjacent target source pixels in the source image. Specifically, according to the coordinates of the pixel point to be generated and the preset mapping formula, the corresponding mapping coordinates of the pixel point to be generated in the source image are calculated; according to the mapping coordinates, two or more near the mapping coordinates are determined. The adjacent source pixel points of are used as the target source pixel points corresponding to the pixel points to be generated.

Exemplarily, as shown in FIG. 2 , according to the mapping relationship, two adjacent target source pixel points, a target source pixel point A and a target source pixel point B corresponding to the pixel point A' to be generated, are determined. In this embodiment of the present application, the number of target source pixels corresponding to the to-be-generated pixels may be more than two. As shown in FIG. 3 , the target source pixels corresponding to the to-be-generated pixels A′ include the source pixel A , B, C, D, the four target source pixels.

In S102, according to the position information of each of the target source pixels in the source image, respectively acquire pixel data of each of the target source pixels from different preset memories, wherein each of the preset memories The source pixel points of one location type are respectively stored, and the location type is determined according to the location information of the source pixel points in the source image.

In this embodiment of the present application, the pixel data may specifically be data used to express image content, such as grayscale values, RGB values, etc., and the pixel data of each source pixel of the source image is pre-stored in its corresponding preset memory. Specifically, each source pixel in the source image is divided into different position types, and each preset memory stores source pixels of one position type respectively. Specifically, the parity of the row position in which the position type is located and/or the parity of the column position in which the position type is located is determined.

According to the position information of at least two adjacent target source pixels in the source image determined in step S101, determine the position type corresponding to each target source pixel corresponding to the pixel to be generated respectively; then, according to each target source pixel Determine the preset memory corresponding to each target source pixel; then, obtain the pixel data of each target source pixel from the corresponding preset memory according to the position information of each target source pixel. Since the pixel data of each target source pixel is stored in different preset memories, it can be acquired at one time and at the same time in the same clock cycle. That is, since the pixel data of the target source pixel is not stored in the same memory, there is no need to wait for the pixel data to be read one by one, but each preset memory can be read at one time to quickly acquire each target pixel data.

Exemplarily, as shown in FIG. 2 , suppose that each column in the source image is marked with four column numbers of 0, 1, 2, and 3 in turn, and the source pixels in the source image are based on the column number of the column where they are located. Parity is divided into two position types, an even column position type and an odd column position type. Correspondingly, the preset memory includes an even-numbered column memory and an odd-numbered column memory, the even-numbered column memory is specially used to store the pixel data of the source pixel point of the even-numbered column type, and the odd-numbered column memory is specially used to store the pixel data of the source pixel point of the odd-numbered column type. . After determining the target source pixel point A and the target source pixel point B corresponding to the pixel point A' to be generated, according to the position information of the two target source pixel points, it is determined that the position type corresponding to the target source pixel point A is an odd column position Type, the position type corresponding to the target source pixel point B is the even-numbered column position type. At this time, the pixel data of the target source pixel point A is obtained from the odd-numbered column memory and the pixel of the target source pixel point B is obtained from the even-numbered column memory at the same time. data.

In S103, an interpolation operation is performed according to the pixel data of each of the target source pixel points, and the target pixel data of the to-be-generated pixel point is determined.

An interpolation operation is performed according to the pixel data of each target source pixel obtained at one time in step S102 to determine the target pixel data of the pixel to be generated. Exemplarily, as shown in Figure 2, if the source pixel corresponding to the pixel A' to be generated includes the target source pixel A and the target source pixel B, then the target source After the pixel data of the pixel point A and the target source pixel point B, a one-dimensional interpolation operation (such as a unilinear interpolation operation) is performed to obtain the pixel data of the pixel point A' to be generated. Exemplarily, as shown in FIG. 3 , if the target source pixel corresponding to the pixel point A' to be generated includes four target source pixels A, B, C, and D, then specifically when the target source pixel is obtained. After the pixel data of the four target source pixel points, a two-dimensional interpolation operation (for example, a bilinear interpolation operation) is performed to obtain the pixel data of the pixel point A' to be generated.

Optionally, after determining the current target pixel data of the pixel to be generated, the target pixel data can be filled into the corresponding position in the image to be generated or buffered in the buffer, and then the current pixel to be generated can be used as the current target pixel data. The adjacent next pixel (for example, a pixel located to the right of the pixel A' to be generated) is used as the next pixel to be generated, and returns to execute the steps S101 to S103 until each pixel in the image to be generated The corresponding pixel data has been determined for each point, that is, the data of the to-be-generated image has been generated, and a complete target image has been obtained.

Optionally, before the step S101, it further includes:

Acquiring a source image, where each source pixel in the source image contains respective pixel data;

According to the position type corresponding to each source pixel in the source image, the pixel data of each source pixel in the source image is stored in a preset memory corresponding to the position type.

In the embodiment of the present application, before step S101, it also includes a preprocessing step of acquiring a source image to store the pixel data of each source pixel in the source image in a preset memory, which is described in detail as follows:

Obtain a source image, each source pixel in the source image contains its own pixel data, that is, the source image contains complete image information, which can be used to express specific image content (such as portraits, landscapes, etc.) ). The source image can be an image captured by a camera, or an image created by drawing software.

According to the position type of each source pixel in the source image, the pixel data of each source pixel is stored in each preset memory. Specifically, the position type of each source pixel is determined according to the position information of each source pixel, specifically determined according to the parity of the row position and/or the column position of the source pixel in the source image. Exemplarily, the position type includes an even row position type and an odd row position type, then the corresponding preset memory includes an even row storage and an odd row storage; if the position type corresponding to the source pixel point is an even row position type (that is, the source pixel If the row number corresponding to the row position of the point is an even number), the pixel data of the source pixel point is stored in the even row memory; if the position type corresponding to the source pixel point is an odd row position type (that is, the source pixel point is located in The row sequence number corresponding to the row position is an odd number), the pixel data of the source pixel point is stored in the odd-numbered row memory. Exemplarily, the position type includes an even column position type and an odd column position type, and the corresponding preset memory includes an even column memory and an odd column memory; if the position type corresponding to the source pixel point is an even column position type (that is, the source pixel If the column number corresponding to the column position of the point is an even number), the pixel data of the source pixel point is stored in the even column memory; if the position type corresponding to the source pixel point is an odd column position type (that is, the source pixel point is located in The column serial number corresponding to the column position is an odd number), then the pixel data of the source pixel point is stored in the odd-numbered column memory.

In the embodiment of the present application, before determining the target pixel data of the pixels to be generated in the image to be generated, the source image is obtained first, and the pixel data of each source pixel is accurately stored in the location type of each source pixel. In each preset memory, the pixel data of each target source pixel point corresponding to the to-be-generated pixel point can be acquired from each preset memory at one time and accurately, thereby ensuring the accuracy of the target pixel data.

In the embodiment of the present application, when the target pixel data of the pixel to be generated needs to be determined, the pixel data of multiple target source pixels corresponding to the current pixel to be generated can be obtained from different preset memories at one time. , perform interpolation operation according to the read pixel data of the target source pixel to obtain the target pixel data of the pixel to be generated, that is, the pixel data of multiple target source pixels required for the interpolation operation can be read at one time , without waiting for reading pixel data multiple times, so the reading efficiency of pixel data can be improved, the target pixel data of the pixel point to be generated can be efficiently determined, and the image processing efficiency based on interpolation operation can be improved.

FIG. 4 shows a schematic flowchart of a second image processing method provided by an embodiment of the present application, and details are as follows:

In S401, four adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image are determined.

In the embodiment of the present application, the image data of the pixels to be generated in the image to be generated is specifically determined according to the four target source pixels in the source image. Since the corresponding mapping coordinates of the pixels to be generated in the source image are generally non-integer, such as (1.6, 1.4), and in the source image, the source pixel whose coordinates are integers is the smallest unit, so the non-integer mapping coordinates Four surrounding source pixels (up, down, left, and right directions adjacent to each other) are determined as four adjacent target source pixels corresponding to the pixels to be generated. Exemplarily, if the mapping coordinates corresponding to the pixels to be generated are (1.6, 1.4), then the corresponding four adjacent target source pixels have coordinates (1, 1) and (1, 2) in the source image, respectively. ), (2,1), (2,2) of the four source pixels, such as the four target source pixels of A, B, C, and D shown in FIG. 3 .

In S402, according to the position information of the four adjacent target source pixels in the source image, the first preset memory, the second preset memory, the third preset memory and the fourth preset The memory acquires the pixel data of each of the target source pixel points; wherein, the first preset memory is used to store the pixel data of the source pixel points whose row position serial numbers are even and column position serial numbers are even in the source image, so The second preset memory is used to store the pixel data of the source pixels whose row position serial numbers are even and the column position serial numbers are odd in the source image, and the third preset memory is used to store the row positions in the source image. The pixel data of the source pixels whose serial numbers are odd and the column position serial numbers are even, the fourth preset memory is used to store the pixel data of the source pixels whose row position serial numbers are odd and the column position serial numbers are odd in the source image .

The parity of the row position and the column position of the four adjacent target source pixels corresponding to the pixels to be generated are alternate, and each of the four adjacent target source pixels corresponds to a position. Type: the position type where the row position number and the column position number are even numbers (such as the target source pixel D shown in Figure 3), the position type where the row position number is even and the column position number is odd (such as the one shown in Figure 3) The target source pixel point C), the position type whose row position number is odd and the column position number is even (for example, the target source pixel point B shown in Figure 3), and the position type whose row position number and column position number are odd (such as The target source pixel point A) shown in Figure 3. Correspondingly, the preset memory includes four memories: the first preset memory, the second preset memory, the third preset memory and the fourth preset memory, according to the position information of the four target source pixels in the source image. , the pixel data of the four target source pixels can be read from the four memories at one time. Wherein, the first preset memory is used to store pixel data of source pixels whose row position serial numbers are even and column position serial numbers are even in the source image, and the second preset memory is used to store the even row position serial numbers and column position serial numbers in the source image. The pixel data of the source pixel whose position number is odd, the third preset memory is used to store the pixel data of the source pixel whose row position number is odd and the column position number is even in the source image, and the fourth preset memory is used to store The pixel data of the source pixel points with odd row position numbers and odd column position numbers in the source image.

In S403, interpolation operation is performed according to the pixel data of each of the target source pixel points to determine the target pixel data of the to-be-generated pixel points

After obtaining the pixel data of the four target source pixel points at one time, perform two-dimensional interpolation operation (interpolation calculation in the row direction and column direction) according to the four target source pixel points to obtain the target pixel data of the pixel to be generated.

Optionally, before step S401, it also includes:

S4001: Acquire a source image, where each source pixel in the source image includes respective pixel data;

S4002: According to the parity of the row position serial numbers corresponding to each source pixel in the source image, correspondingly divide the pixel data of each source pixel in the source image into even row pixel data and odd row pixel data;

S4003: According to the parity of the column position serial numbers corresponding to each source pixel point in the even-numbered row pixel data, correspondingly select and store the even-numbered row pixel data in the first preset memory or the second preset memory;

S4004: According to the parity of the column position serial numbers corresponding to each source pixel point in the odd-numbered row pixel data, correspondingly select and store the odd-numbered row pixel data in the third preset memory or the fourth preset memory.

Specifically, as shown in FIG. 5 , a source image is obtained, the pixel data of the source pixels in the source image are read line by line, and parity line selection is performed first (that is, according to the parity of the row position serial numbers corresponding to each source pixel point) Storage selection), divide the pixel data of the source pixel point into even-numbered row pixel data (that is, the pixel data whose row position serial number is an even number, such as row0 and row2 in Figure 5) and odd-numbered row pixel data (that is, the pixel data whose row position serial number is an odd number) , such as row1, row3 in Figure 5). Afterwards, the even-numbered rows of pixel data are selected for parity columns (that is, the storage selection is performed according to the parity of the column position serial numbers corresponding to each source pixel point), and the even-numbered columns of pixel data in the even-numbered rows are obtained. Correspondingly stored in the first preset memory and the second preset memory; odd-numbered rows of pixel data are selected for odd-even columns (that is, storage selection is performed according to the parity of the column position serial numbers corresponding to each source pixel point), and the odd-numbered rows are obtained. The pixel data of the even-numbered columns and the pixel data of the odd-numbered columns in the odd-numbered rows are correspondingly stored in the third preset memory and the fourth preset memory.

Further, in the embodiment of the present application, steps S4001-S4004 are performed through a bus control interface module, and the architecture of the bus control interface module is shown in FIG. 6 , and the details are as follows:

In S4001, the bus control interface module obtains the source image from the external memory, the pixel data of the source pixel of the source image is inputted row by row, and is transmitted from the bus into the bus control interface module in the form of serial bus data. Among them, the data amount of the bus data transmitted in one clock cycle is determined according to the bus bit width. For example, if the bit width of the bus is 128 bits (binary digit, bit), that is, 16 bytes, then 16 words are transmitted in one clock cycle. section bus data. Specifically, in the embodiment of the present application, the length of pixel data of each source pixel point is one byte. Optionally, the bus data passed in from the external memory may be unaligned. Therefore, in this embodiment of the present application, after the bus data is passed in, the bus data register and the splicing unit are used to realize the splicing and alignment of the pixel data. Among them, the misalignment of bus data means that the pixel data of the source pixel input from the external memory does not start with the 0th bit (0x00) of the bus data, resulting in that the bus data read in one clock cycle is not necessarily the source Pixel data for the pixel. For example, as shown in Figure 6, set the bus bit width to 128bit (16 bytes, each small cell in Figure 6 represents one byte of data), the read source pixel data starts at 0x02, and 0x00 , 0x01, these two bytes of data may be other data information (such as identification bit information), then the bus data (16 bytes of data in the previous clock cycle, that is, the label in Figure 6) cached in the bus data register is converted by the splicing unit. The source pixel data (that is, the 14-byte data labeled 2 to 15 in Figure 6) in the 16-byte data of 0 to 15) and the current bus data (specifically, the labels of this clock cycle are 16 and 17). The two bytes of data) are spliced to obtain aligned 16-byte bus data.

In S4002, the aligned bus data starts from the start address 0x00, and each byte is the source pixel data of a source pixel. At this time, the flag bit of the parity line identifier register is switched according to the line width information of the source image acquired in advance, so as to judge the parity of the line position serial number of the source pixel point corresponding to each pixel data transmitted in sequence, so as to convert each source pixel The pixel data of the point is divided into two lines of pixel data of even-numbered lines and pixel data of odd-numbered lines and transmitted backwards. Exemplarily, let the line width information in the source image be 32 pixels (that is, each line in the source image includes 32 source pixels), and the line position serial number of the initial line in the source image is an even number (for example, the serial number of the initial line is 0). ), then the odd-even row identifier register is initially an even row identification bit, and the bus data output from the splicing unit is then determined to be the even row pixel data transmitted backward through the first line in turn; when the odd row identifier register detects that the transmission After the bus data reaches 32 bytes, the identification bit of the odd-even row data register is switched to the odd row identification bit, and then the bus data output from the splicing unit is sequentially determined as odd-numbered row pixel data through the second line to transmit backward through the second line; By analogy, by cyclically switching the identification bits of the odd-even row identification register, the pixel data of each source pixel is correspondingly divided into even row pixel data or odd row pixel data, and is transmitted backward through two different lines, so that the The pixel data of the source pixels whose row position serial numbers are even and those whose row position serial numbers are odd in the source image are processed separately.

In S4003, the even-numbered rows of pixel data are transmitted backwards, and the identification bits of the odd-even column identification registers are switched alternately, so that the even-numbered rows of pixel data whose column position sequence numbers are even and whose column position sequence numbers are odd are transferred to the buffer by branching in the first even-numbered column register, the first odd-numbered column register, and finally stored in the first preset memory and the second preset memory. Wherein, the even-numbered row data registers and the splicing unit in FIG. 6 are used for splicing the even-numbered row pixel data to a length equal to the bit width of the preset memory and then transmitting backwards. Exemplarily, as shown in FIG. 8 , it is assumed that row 0 in the source image contains a total of 32 bytes of pixel data with column position numbers ranging from 0 to 31 (each byte corresponds to the pixel data of one source pixel) , assuming that the bit width of the bus is 16 bytes, and the bit width of each preset memory is 8 bytes, then in the first clock cycle, the 16 pixel data whose column position serial numbers are 0 to 15 are sorted by column The parity of the position serial number is cached in the first even-numbered column register and the first odd-numbered column register, respectively, and then transferred to the address 0 of the first preset memory and the address 0 of the second preset memory for storage; in the second In each clock cycle, the 16 pixel data whose column position serial numbers are 16 to 31 are respectively cached in the first even-numbered column register and the first odd-numbered column register according to the parity of the column position serial numbers, and then transferred to the first preset register respectively. It is stored in the address 1 of the memory and the address 1 of the second preset memory; and so on, until all the even-numbered rows of pixel data have been stored in the first preset memory and the second preset memory, respectively.

In S4004, the odd-numbered row pixel data is transmitted backwards, and the identification bits of the odd-even column identification register are alternately switched, so that the odd-numbered row pixel data whose column position serial number is an even number and whose column position serial number is an odd-numbered pixel data is transmitted and buffered in the second even-numbered column register, the second odd-numbered column register, and finally stored in the third preset memory and the fourth preset memory respectively. Its specific execution logic is similar to that of step S4003, which is not repeated here.

In this embodiment of the present application, before step S401, the pixel data of each source pixel in the source image is accurately stored in the first preset memory, the second preset memory, and the third preset memory through steps S4001 to S4004, respectively. In the memory and the fourth preset memory, in step S402, the pixel data of the four adjacent source pixel points can be accurately obtained from the four preset memories, thereby ensuring the accuracy of the target pixel data determined later. accuracy.

Exemplarily, FIG. 9 shows a schematic diagram of a hardware architecture of an image processing apparatus provided by an embodiment of the present application, where the hardware architecture includes a system control module, an external memory, a bus control interface module, a calculation unit (calculated by a coordinate calculation unit, an address The calculation unit and the pixel calculation unit are composed of), a register, a first preset memory, a second preset memory, a third preset memory and a fourth preset memory. Specifically, the data transmission process of the hardware architecture is as follows: (1) The bus control interface module obtains the pixel data of the source image from the external memory through the bus data transmission in advance, and stores them in the first preset memory and the second preset memory respectively. , the third preset memory and the fourth preset memory; (2) the system control module instructs the calculation unit and the bus control interface module to cooperate with each other, and the coordinate calculation unit in the calculation unit determines the current four corresponding pixels to be generated. For the adjacent target source pixels, the pixel data of the four adjacent target source pixels are determined by the address calculation unit to correspond to the address information in the preset memory, and then the calculation unit transmits the address information to the bus control interface module; (3) The bus control interface module obtains the four adjacent preset memories from the first preset memory, the second preset memory, the third preset memory and the fourth preset memory respectively within the same clock cycle according to the address information The pixel data of the target source pixel point; (4) the bus control interface module transmits the acquired pixel data of the four adjacent target source pixel points to the calculation unit, and calculates the interpolation operation through the pixel calculation unit therein to obtain the current The target pixel data of the pixel to be generated; (5) transfer the target pixel data to the buffer for buffering; (6) return to perform steps (2)-(5) to perform the next target pixel data of the pixel to be generated is determined until the buffer is full, and the data in the buffer is transmitted to the external memory through the bus control interface module.

In the embodiment of the present application, considering that the mapping coordinates corresponding to the pixels to be generated in the source image are usually coordinates whose horizontal and vertical coordinates are non-integers, the corresponding pixels to be generated are corresponding to the source pixels in the source image. Four adjacent source pixels; since these four source pixels correspond to four types of positions, the corresponding four pixel data can be obtained from the four preset memories at one time, that is, within one clock cycle The required pixel data of the four source pixel points is read, the reading efficiency of the pixel data is improved, the target pixel data of the pixel point to be generated is efficiently determined, and the image processing efficiency based on the interpolation operation is improved.

Optionally, according to the position information of each target source pixel in the source image, acquiring the pixel data of each target source pixel from different preset memories, including:

A1: Determine the preset memory and offset corresponding to each of the target source pixels according to the position information of each of the target source pixels in the source image;

A2: According to each of the offsets, respectively determine the address information of the pixel data of each of the target source pixel points in the corresponding preset memory;

A3: According to each of the address information, acquire pixel data of each of the corresponding target source pixels from each of the preset memories.

In A1, the position type corresponding to each target source pixel is determined according to the position information of each target source pixel in the source image, and the corresponding preset memory is determined according to the position type. After that, the offset of the target source pixel in the corresponding preset memory is determined according to the position information of each target source pixel. Exemplarily, as shown in FIG. 2 , according to the location information of the target source pixel A in the source image, it is determined that its corresponding location type is an odd-numbered column location type, then the corresponding preset memory is an odd-numbered column memory, and According to the position information of the target source pixel A, it is determined that it is the source pixel of the third position type odd-numbered column position type in the source image, and the offset is 3.

In A2, each target source pixel is based on its corresponding offset and the bit width of the preset memory (in units of bytes) to obtain the pixel data of the target source pixel in its corresponding preset memory. The address information includes the storage address of the pixel data in the preset memory and the byte offset information in the storage address. Specifically, offset is used to represent the offset corresponding to the target source pixel, W represents the bit width of the preset memory, Address represents the storage address, and byte_offset represents the byte offset information, then:

Address=offset>>(log ₂ W)

byte_offset=offset%W

Among them, ">>" represents the right shift operator, and "%" represents the remainder operator. Specifically, taking the obtained logarithm of the bit width W of the preset memory with the base 2 as the target right shift number, the offset represented by binary is shifted to the right by the target right shift number to obtain the storage address Address ; Specifically, the remainder obtained by dividing the offset offset by the bit width W of the preset memory is used as the byte offset information byte_offset.

For example, when the bit width of the preset memory is 8 bytes, then:

Address=offset>>3

byte_offset=offset%8

In A3, according to each address information obtained in A2, the pixel data of the corresponding target source pixel is obtained from each preset storage respectively. Exemplarily, if the storage address obtained by the target source pixel point A in step A2 as shown in FIG. 2 is address 0 and the byte offset is 3, then the corresponding memory address in the memory with the odd-numbered column memory address number 0 is 0. The third byte sums the acquired pixel data of the target source pixel point A; at the same time, the pixel data of the target source pixel point B is also acquired in the even-numbered column memory according to its corresponding address information.

Since the embodiment of the present application can accurately determine the preset memory, offset and address information corresponding to each target source pixel according to the position information of each target source pixel in the source image, it can ensure that the pixel of the target source pixel is The accuracy of data reading, thereby ensuring the accuracy of target pixel data calculation.

Optionally, if the position information includes the coordinate information of each of the target source pixels, then determining each of the target sources according to the position information of each of the target source pixels in the source image. The preset memory and offset corresponding to the pixel points, including:

B1: Determine the position type corresponding to each of the target source pixels according to the coordinate information of each of the target source pixels in the source image;

B2: Determine the preset memory corresponding to each of the target source pixels and the corresponding first offset calculation formula according to the position type of each of the target source pixels in the source image;

B3: Determine the offset of each of the target source pixels in the corresponding preset memory according to the coordinate information of each of the target source pixels and the first offset calculation formula.

In B1, each target source pixel is determined according to its coordinate information in the source image to determine the position type corresponding to the target source pixel. Exemplarily, as shown in FIG. 3 , according to the coordinates (1, 1) of the target source pixel A, it is determined to be an odd row and odd column position type; according to the coordinates (2, 1) of the target source pixel B, it is determined It is the position type of odd-numbered rows and even-numbered columns; according to the coordinates (1, 2) of the target source pixel point C, it is determined to be the position type of even-numbered rows and odd-numbered columns; according to the coordinates (2, 2) of the target source pixel point D, it is determined to be an even number Row even column position type.

In B2, a preset memory corresponding to each target source pixel and a corresponding first offset calculation formula for calculating the offset are determined according to the position type of each target source pixel in the source image. Exemplarily, in FIG. 3 , the preset memory corresponding to the target source pixel point A is the above-mentioned fourth preset memory, the preset memory corresponding to the target source pixel point B is the above-mentioned third preset memory, and the target source pixel point B is the above-mentioned third preset memory. The preset memory corresponding to point C is the aforementioned second preset memory, and the preset memory corresponding to the target source pixel point D is the aforementioned first preset memory. Optionally, in the target source pixel point, the target source pixel point of the odd-numbered row and odd-numbered column position type and the target source pixel of the even-numbered row and odd-numbered column type (that is, the source pixel point whose x-axis coordinate is an odd number, such as A and C as shown in Figure 3) corresponds to The first offset calculation formula is:

offset=(width>>1)*(y>>1)+(x>>1)

The first offset corresponding to the position type of odd-numbered rows and even-numbered columns in the source pixel point and the target source pixel point of the even-numbered row and even-numbered column type (that is, the source pixel point whose x-axis coordinate is an even number, as shown in B and D in Figure 3) The calculation formula is:

offset=((width>>1)+width[0])*(y>>1)+(x>>1)

Among them, offset represents the offset, width represents the width of the source image, >> represents the right shift operator, * represents the multiplication sign, x represents the abscissa (ie column coordinate) of the target source pixel, and y represents the target source pixel. Vertical coordinate (ie row coordinate), width[0] is 0 when width is odd, and 1 when width is even.

In B3, the offset of each target source pixel in the corresponding preset memory is determined according to the coordinate information of each target source pixel and the determined first offset calculation formula. Exemplarily, the target source pixel point A(1,1) and the target source pixel point C(1,2) as shown in FIG. 3 are respectively substituted into the corresponding first offset calculation determined in step B2. The formula offset=(width>>1)*(y>>1)+(x>>1), respectively obtain the offset corresponding to the target source pixel A in the fourth preset memory and the target source pixel C Corresponding to the offset in the second preset memory; substitute the target source pixel point B(2,1) and the target source pixel point D(2,2) into the corresponding first offset determined in step B2 respectively. The offset calculation formula offset=((width>>1)+width[0])*(y>>1)+(x>>1), respectively obtain the target source pixel point B corresponding to the third preset memory and the target source pixel point D corresponds to the offset in the first preset memory.

Optionally, if the at least two adjacent target source pixels include a first target source pixel and a second target source pixel other than the first target source pixel, the position information includes the The coordinate information of the first target source pixel point and the relative position information of the second target source pixel point relative to the first target source pixel point, then according to each target source pixel point in the source image The position information of , determine the preset memory and offset corresponding to each of the target source pixels, including:

C1: Determine the position type corresponding to each target source pixel according to the coordinate information of the first target source pixel and the relative position information of the second target source pixel relative to the first target source pixel and the corresponding second offset calculation formula;

C2: Determine the offset of each target source pixel in the corresponding preset memory according to the coordinate information of the first target source pixel and each of the second offset calculation formulas.

In the embodiment of the present application, the source pixel with a smaller coordinate value may be selected as the first target source pixel among the at least two adjacent target source pixels determined, and the other target source pixels as the second target source pixel point. Optionally, the at least two adjacent target source pixels specifically include four adjacent target source pixels, then the source pixel located in the upper left corner is specifically used as the first target source pixel, and the other three target The source pixel is used as the second target source pixel; according to the coordinate information of the first target source pixel and the relative position of the second target source pixel and the first target source pixel, the corresponding position type and the second offset are determined. The displacement calculation formula; then, according to the coordinates of the first target source pixel point and the determined four second offset calculation formulas corresponding to the four target source pixel points, it is determined that the four target source pixel points correspond to the respective The offset in the preset memory for . Optionally, the second offset calculation formula is determined as follows:

The formula for calculating the second offset of the target source pixel in the upper left corner:

When sx is odd, offset=(width>>1)*(sy>>1)+(sx>>1)

When sx is an even number, offset=((width>>1)+width[0])*(sy>>1)+(sx>>1)

The formula for calculating the second offset of the target source pixel in the upper right corner:

When sx is odd, offset=((width>>1)+width[0])*(sy>>1)+(sx>>1)+1

When sx is an even number, offset=((width>>1)+width[0])*(sy>>1)+(sx>>1)

The formula for calculating the second offset of the target source pixel in the lower left corner:

When sx is even and sy is even:

offset=((width>>1)+width[0])*(sy>>1)+(sx>>1)

When sx is odd and sy is even:

offset=(width>>1)*(sy>>1)+(sx>>1)

When sx is even and sy is odd:

offset=((width>>1)+width[0])*((sy>>1)+1)+(sx>>1)

When sx is odd and sy is odd:

offset=(width>>1)*((sy>>1)+1)+(sx>>1)

The formula for calculating the second offset of the target source pixel in the lower right corner:

When sx is even and sy is even:

offset=(width>>1)*(sy>>1)+(sx>>1)

When sx is odd and sy is even:

offset=((width>>1)+width[0])*(sy>>1)+(sx>>1)+1

When sx is odd and sy is odd:

offset=(width>>1)*((sy>>1)+1)+(sx>>1)

When sx is even and sy is odd:

offset=((width>>1)+width[0])*((sy>>1)+1)+(sx>>1)+1

Among them, offset represents the offset, width represents the width of the source image, >> represents the right shift operator, * represents the multiplication sign, sx represents the abscissa (ie the column coordinate) of the first target source pixel, and sy represents the first target The ordinate (ie row coordinate) of the source pixel, width[0] is 0 when width is odd, and 1 when width is even.

Exemplarily, as shown in FIG. 3 , the target source pixel A is the first target source pixel, and its coordinates are (1, 1), and the target source pixels B, C, and D are the second target source pixels. In C1, according to the first target source pixel point A(1,1), it is determined that sx=1, sy=1, and both sx and sy are odd numbers; it is determined that the position type corresponding to the target source pixel point A is an odd-numbered row and an odd-numbered column position type and the corresponding second offset calculation formula is: offset=(width>>1)*(sy>>1)+(sx>>1); determine that the position type of the target source pixel B is odd-numbered rows and even-numbered columns The calculation formula of the position type and the corresponding second offset is: offset=((width>>1)+width[0])*(sy>>1)+(sx>>1)+1; determine the target source pixel The position type of point C is the position type of even row and odd column and the corresponding second offset calculation formula is: offset=(width>>1)*((sy>>1)+1)+(sx>>1) ; Determine that the position type of the target source pixel point D is the position type of the even-numbered row and even-numbered column and the corresponding second offset calculation formula is: offset=(width>>1)*((sy>>1)+1)+( sx>>1). After that, in C2, the coordinates sx=1 and sy=1 of the target source pixel A are substituted into the four determined second offset calculation formulas to obtain the corresponding target source pixels A, B, C, and D respectively. Offset.

In the embodiment of the present application, the offset corresponding to each target source pixel is specifically determined by the coordinate information of each target source pixel and the first offset calculation formula, or, specifically by the determined target source pixel in the The coordinate information of the first target source pixel, the relative positions of other target source pixels and the first target source pixel, and the determined second offset calculation formula to determine the offset corresponding to each target source pixel , to ensure the accuracy of reading the pixel data of the source pixel point, and then to ensure the accuracy of the calculation of the target pixel data.

FIG. 10 shows a schematic flowchart of a third image processing method provided by an embodiment of the present application. The image processing method is applied to a Spatial Transformer Network (STN), and an architecture diagram of the STN is shown in FIG. 11 . The source data in Figure 11 is the source image data, including the pixel data of each source pixel in the source image, and the target data is the image data obtained by generating the target pixel data one by one in the image to be generated, which is called target image data. Each pixel in the target image data is filled with a corresponding target pixel data. The image processing method shown in Figure 10 is detailed as follows:

In S1001, the target parameters of the STN are determined through parameter prediction.

In the STN network, the coordinates (x, y) of the image to be generated and their corresponding mapping coordinates (x', y') in the source image have the following coordinate mapping relationship:

Among them, a, b, c, d, e, and f are six target parameters, which can reflect the transformation relationship between the image to be generated and the source image, such as translation, scaling, and rotation. Specifically, parameter prediction can be achieved by training several consecutive neural network layers (including convolutional layers and connection layers) in the STN through training samples, and the target parameters a, b, c, d, e, and f of the STN can be obtained.

In S1002, according to the target parameter, the mapping coordinates of the source image corresponding to the to-be-generated pixel points in the to-be-generated image are determined.

According to the determined target parameters a, b, c, d, e, f and the coordinate mapping relationship shown above, determine the mapping coordinates (x', y) of the pixel to be generated (x, y) corresponding to the source image ').

In S1003, the four adjacent target source pixels are determined according to the mapping coordinates.

In the determined mapping coordinates (x', y'), x' and y' are decimals, and in the source image, there is only the pixel data of the source pixels whose horizontal and vertical coordinates are integers. Map the four adjacent target source pixels with the closest coordinates. Exemplarily, as shown in FIG. 3 , if the mapping coordinates corresponding to the pixel point A' to be generated in the source image are (1.6, 1.4), then the corresponding four adjacent target source pixel points in the source image are A. (1,1), B(2,1), C(1,2), D(2,2).

In S1004, according to the position information of the four adjacent target source pixels in the source image, the first preset memory, the second preset memory, the third preset memory and the fourth preset memory are respectively obtained. The memory acquires the pixel data of each of the target source pixel points; wherein, the first preset memory is used to store the pixel data of the source pixel points whose row position serial numbers are even and column position serial numbers are even in the source image, so The second preset memory is used to store the pixel data of the source pixels whose row position serial numbers are even and the column position serial numbers are odd in the source image, and the third preset memory is used to store the row positions in the source image. The pixel data of the source pixels whose serial numbers are odd and the column position serial numbers are even, the fourth preset memory is used to store the pixel data of the source pixels whose row position serial numbers are odd and the column position serial numbers are odd in the source image .

Step S1004 in this embodiment of the present application is the same as the above-mentioned step S402. For details, please refer to the above-mentioned related description of the step S402, which will not be repeated here.

In S1005, a bilinear interpolation operation is performed according to the pixel data of the four adjacent target source pixel points, and the target pixel data of the to-be-generated pixel point is determined.

Exemplarily, as shown in FIG. 3, the pixel point A' to be generated corresponds to the four adjacent target source pixel points in the source image as A(1,1), B(2,1), C(1, 2), D(2,2), suppose its corresponding pixel data are represented by dataA, dataB, dataC, dataD respectively, then the calculation formula of the target pixel data data_A' of the pixel point A' to be generated is as follows:

data_A'=(1-0.6)×(1-0.4)×dataA

+(1-0.6)×(1-0.6)×dataB

+(1-0.4)×(1-0.4)×dataC

+(1-0.4)×(1-0.6)×dataD

In the embodiment of the present application, the image processing method is specifically applied to the STN. The amount of image data to be processed by the STN is generally relatively large, and the target pixel data of each pixel to be generated needs to be based on four adjacent target source pixels. The pixel data is determined by interpolation. Because through the method of the embodiment of the present application, the pixel data of the four adjacent source pixel points corresponding to the pixel points to be generated can be simultaneously acquired from different preset memories at one time, so that the target pixel data can be quickly determined, so it is possible to Greatly improve the processing efficiency of STN.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Embodiment 2:

FIG. 12 shows a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application. For the convenience of description, only parts related to the embodiment of the present application are shown:

The image processing apparatus includes: a target source pixel point determination unit 121 , a pixel data acquisition unit 122 , and a target pixel data determination unit 123 . in:

The target source pixel point determination unit 121 is configured to determine at least two adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image, where the target source pixel points are the source pixel points in the source image.

The pixel data obtaining unit 122 is configured to obtain the pixel data of each target source pixel from different preset memories according to the position information of each target source pixel in the source image, wherein each target source pixel is The preset memory respectively stores source pixels of one type of location, and the location type is determined according to the location information of the source pixels in the source image.

The target pixel data determination unit 123 is configured to perform an interpolation operation according to the pixel data of each of the target source pixel points, and determine the target pixel data of the to-be-generated pixel point.

Optionally, the image processing apparatus further includes:

a first acquiring unit, configured to acquire a source image, where each source pixel in the source image contains respective pixel data;

a first storage unit, configured to store the pixel data of each source pixel in the source image in a preset memory corresponding to the location type according to the location type corresponding to each source pixel in the source image .

Optionally, the source pixel point determination unit 121 is specifically configured to determine four adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image;

Correspondingly, the pixel data acquisition unit 122 is specifically configured to obtain information from the first preset memory, the second preset memory, the The third preset memory and the fourth preset memory acquire the pixel data of each of the target source pixels; wherein, the first preset memory is used for storing the row position serial number is even and the column position serial number is even in the source image The pixel data of the even-numbered source pixels, the second preset memory is used to store the pixel data of the source pixels whose row position serial numbers are even and the column position serial numbers are odd in the source image, and the third preset memory for storing pixel data of source pixels whose row position serial numbers are odd and column position serial numbers are even in the source image, and the fourth preset memory is used for storing odd row position serial numbers and column positions in the source image The pixel data of the odd-numbered source pixel.

Optionally, the image processing apparatus further includes:

a second acquiring unit, configured to acquire a source image, where each source pixel in the source image contains respective pixel data;

The first classification unit is used to correspondingly divide the pixel data of each source pixel in the source image into even-numbered row pixel data and odd-numbered pixel data according to the parity of the row position serial numbers corresponding to each source pixel in the source image. row pixel data;

The second storage unit, according to the parity of the column position serial number corresponding to each source pixel point in the even-numbered row pixel data, correspondingly selects and stores the even-numbered row pixel data in the first preset memory or the second preset memory;

The third storage unit selects and stores the odd row pixel data in the third preset memory or the fourth preset memory according to the parity of the column position serial numbers corresponding to each source pixel in the odd row pixel data.

Optionally, if the image processing apparatus is specifically applied to a spatial transformation network, the image processing apparatus further includes:

a target parameter determination unit, configured to determine the target parameter of the spatial transformation network through parameter prediction;

Correspondingly, the source pixel point determination unit 121 includes a mapping coordinate determination module and a source pixel point determination module:

a mapping coordinate determination module, configured to determine, according to the target parameter, the mapping coordinates of the source image corresponding to the to-be-generated pixels in the to-be-generated image;

a source pixel point determination module, configured to determine the four adjacent target source pixel points according to the mapping coordinates;

Correspondingly, the target pixel data determining unit 123 is specifically configured to perform a bilinear interpolation operation according to the pixel data of the four adjacent target source pixel points, and determine the target pixel data of the to-be-generated pixel point.

Optionally, the pixel data acquisition unit 122 includes an offset determination module, an address information determination module and a pixel data acquisition module:

an offset determination module, configured to determine a preset memory and an offset corresponding to each of the source pixels according to the position information of each of the target source pixels in the source image;

The address information determination module is configured to determine, according to each of the offsets, the address information of the pixel data of each of the target source pixel points in the respective corresponding preset memories;

A pixel data acquisition module, configured to acquire pixel data of each of the corresponding target source pixels from each of the preset memories according to each of the address information.

Optionally, if the position information includes coordinate information of each of the target source pixels, the offset determination module is specifically configured to, according to the coordinate information of each of the target source pixels in the source image, , determine the position type corresponding to each target source pixel point; according to the position type of each target source pixel point in the source image, determine the preset memory corresponding to each target source pixel point and the corresponding first an offset calculation formula; according to the coordinate information of each of the target source pixels and the first offset calculation formula, determine the corresponding position of each target source pixel in the corresponding preset memory. Offset;

Optionally, if the at least two adjacent target source pixels include a first target source pixel and a second target source pixel other than the first target source pixel, the position information includes the The coordinate information of the first target source pixel point and the relative position information of the second target source pixel point with respect to the first target source pixel point, the offset determination module is specifically used for determining the offset according to the first target source pixel point. The coordinate information of the target source pixel point and the relative position information of the second target source pixel point relative to the first target source pixel point, determine the position type corresponding to each target source pixel point and the corresponding second offset according to the coordinate information of the first target source pixel and each of the second offset calculation formulas, determine the offset of each of the target source pixels in the corresponding preset memory .

It should be noted that the information exchange, execution process and other contents between the above-mentioned devices/units are based on the same concept as the method embodiments of the present application. For specific functions and technical effects, please refer to the method embodiments section. It is not repeated here.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated to different functional units, Module completion, that is, dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated in one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit, and the above-mentioned integrated units may adopt hardware. It can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present application. For the specific working processes of the units and modules in the above-mentioned system, reference may be made to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

Embodiment three:

FIG. 13 is a schematic diagram of a terminal device provided by an embodiment of the present application. As shown in FIG. 13 , the terminal device 13 in this embodiment includes: a processor 130 , a memory 131 , and a computer program 132 stored in the memory 131 and executable on the processor 130 , such as an image processing program. When the processor 130 executes the computer program 132 , the steps in each of the above image processing method embodiments are implemented, for example, steps S101 to S103 shown in FIG. 1 . Alternatively, when the processor 130 executes the computer program 132, the functions of the modules/units in each of the foregoing apparatus embodiments, for example, the functions of the units 121 to 123 shown in FIG. 12, are implemented.

Exemplarily, the computer program 132 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 131 and executed by the processor 130 to complete the this application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 132 in the terminal device 13 . For example, the computer program 132 can be divided into a target source pixel point determination unit, a pixel data acquisition unit, and a target pixel data determination unit. The specific functions of each unit are as follows:

The terminal device 13 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The terminal device may include, but is not limited to, the processor 130 and the memory 131 . Those skilled in the art can understand that FIG. 13 is only an example of the terminal device 13, and does not constitute a limitation on the terminal device 13, and may include more or less components than the one shown, or combine some components, or different components For example, the terminal device may further include an input and output device, a network access device, a bus, and the like.

The so-called processor 130 may be a central processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 131 may be an internal storage unit of the terminal device 13 , such as a hard disk or a memory of the terminal device 13 . The memory 131 can also be an external storage device of the terminal device 13, such as a plug-in hard disk equipped on the terminal device 13, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, Flash Card, etc. Further, the memory 131 may also include both an internal storage unit of the terminal device 13 and an external storage device. The memory 131 is used to store the computer program and other programs and data required by the terminal device. The memory 131 may also be used to temporarily store data that has been output or will be output.

In the foregoing embodiments, the description of each embodiment has its own emphasis. For parts that are not described or described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

Claims

An image processing method, comprising:

Determine at least two adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image, where the target source pixel points are the source pixel points in the source image;

According to the position information of each target source pixel in the source image, the pixel data of each target source pixel is obtained from different preset memories, wherein each of the preset memories stores a The source pixel point of the position type, the position type is determined according to the position information of the source pixel point in the source image;

An interpolation operation is performed according to the pixel data of each of the target source pixel points, and the target pixel data of the to-be-generated pixel point is determined.
The image processing method according to claim 1, wherein before the determining at least two adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image, the method further comprises:

Acquiring a source image, where each source pixel in the source image contains respective pixel data;

According to the position type corresponding to each source pixel in the source image, the pixel data of each source pixel in the source image is stored in a preset memory corresponding to the position type.
The image processing method according to claim 1, wherein the determining at least two adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image comprises:

Determine four adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image;

The obtaining pixel data of each target source pixel from different preset memories according to the position information of each target source pixel in the source image, including:

According to the position information of the four adjacent target source pixel points in the source image, obtain each The pixel data of the target source pixel; wherein, the first preset memory is used to store the pixel data of the source pixel whose row position sequence number is even and the column position sequence number is even in the source image, and the second preset memory It is assumed that the memory is used to store the pixel data of the source pixel points whose row position serial numbers are even and the column position serial numbers are odd in the source image, and the third preset memory is used for storing the row position serial numbers in the source image are odd and odd. The pixel data of the source pixels whose column position numbers are even, and the fourth preset memory is used to store the pixel data of the source pixels whose row position numbers are odd and the column position numbers are odd in the source image.
The image processing method according to claim 3, characterized in that before said determining the four adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image, the method further comprises:

Acquiring a source image, where each source pixel in the source image contains respective pixel data;

According to the parity of the row position serial number corresponding to each source pixel in the source image, correspondingly divide the pixel data of each source pixel in the source image into even-numbered row pixel data and odd-numbered row pixel data;

According to the parity of the column position serial number corresponding to each source pixel point in the even-numbered row pixel data, the even-numbered row pixel data is correspondingly selected and stored in the first preset memory or the second preset memory;

According to the parity of the column position serial numbers corresponding to each source pixel point in the odd-numbered row pixel data, the odd-numbered row pixel data is correspondingly selected and stored in the third preset memory or the fourth preset memory.
The image processing method according to claim 3, wherein, if the image processing method is applied to a spatial transformation network, in the determining of four adjacent objects corresponding to the pixels to be generated in the image to be generated Before the source pixel point, it also includes:

Determine the target parameters of the spatial transformation network through parameter prediction;

The determining of four adjacent target source pixels corresponding to the pixels to be generated in the image to be generated includes:

According to the target parameter, determine the mapping coordinates of the source image corresponding to the to-be-generated pixel points in the to-be-generated image;

According to the mapping coordinates, determine the four adjacent target source pixels;

Correspondingly, the performing interpolation operation according to the pixel data of each of the target source pixels to determine the target pixel data of the pixels to be generated includes:

A bilinear interpolation operation is performed according to the pixel data of each target source pixel among the four adjacent source pixels, and the target pixel data of the to-be-generated pixel is determined.
The image processing method according to any one of claim 1, wherein, according to the position information of each target source pixel in the source image, each of the target source pixels is obtained from different preset memories respectively. The pixel data of the target source pixel, including:

According to the position information of each target source pixel in the source image, determine the preset memory and offset corresponding to each target source pixel respectively;

According to each of the offsets, respectively determine the address information of the pixel data of each of the target source pixel points in the corresponding preset memory;

According to each of the address information, the pixel data of each of the corresponding target source pixel points is obtained from each of the preset memories, respectively.
6. The image processing method according to claim 6, wherein, if the position information includes coordinate information of each of the target source pixels, the method according to the coordinate information of each of the target source pixels in the source image The position information determines the preset memory and offset corresponding to each of the target source pixels, including:

According to the coordinate information of each target source pixel in the source image, determine the position type corresponding to each target source pixel;

According to the position type of each target source pixel in the source image, determine the preset memory corresponding to each target source pixel and the corresponding first offset calculation formula;

According to the coordinate information of each of the target source pixels and the first offset calculation formula, determine the offset of each of the target source pixels in the corresponding preset memory;

Or, if the at least two adjacent target source pixels include a first target source pixel and a second target source pixel other than the first target source pixel, the position information includes the first target source pixel The coordinate information of the target source pixel point and the relative position information of the second target source pixel point with respect to the first target source pixel point, then according to all the target source pixel points in the source image The position information is determined, and the preset memory and offset corresponding to each of the target source pixels are determined, including:

According to the coordinate information of the first target source pixel point and the relative position information of the second target source pixel point relative to the first target source pixel point, determine the position type and corresponding position of each target source pixel point The second offset calculation formula of ;

According to the coordinate information of the first target source pixel point and each of the second offset calculation formulas, the offset of each of the target source pixel points in the corresponding preset memory is determined.
An image processing device, comprising:

a target source pixel point determination unit, configured to determine at least two adjacent target source pixel points corresponding to the to-be-generated pixel points in the to-be-generated image, where the target source pixel points are the source pixel points in the source image;

A pixel data acquisition unit, configured to acquire pixel data of each of the target source pixels from different preset memories according to the position information of each of the target source pixels in the source image, wherein each of the target source pixels The preset memories respectively store source pixels of a position type, and the position type is determined according to the position information of the source pixels in the source image;

The target pixel data determination unit is configured to perform interpolation operation according to the pixel data of each of the target source pixel points, and determine the target pixel data of the to-be-generated pixel point.
A terminal device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that when the processor executes the computer program, the terminal device realizes the The steps of the method as claimed in any one of claims 1 to 7.
A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the terminal device is made to realize the implementation of any one of claims 1 to 7 steps of the method.