CN115311146A

CN115311146A - Image data processing method, image data processing device, computer equipment and storage medium

Info

Publication number: CN115311146A
Application number: CN202211036940.7A
Authority: CN
Inventors: 黄斌; 李永杰; 赵多
Original assignee: Shenzhen Zhouming Technology Co Ltd
Current assignee: Shenzhen Zhouming Technology Co Ltd
Priority date: 2022-08-25
Filing date: 2022-08-25
Publication date: 2022-11-08

Abstract

The application relates to an image data processing method, an apparatus, a device and a storage medium. The method comprises the following steps: receiving image data to be processed, wherein the image data to be processed comprises a pixel set which is input at a current time point and extracted from an image to be processed according to a selected data input mode, the pixel set is composed of a plurality of continuous pixels positioned in a first pixel arrangement direction, parallel interpolation processing is carried out on the pixel set according to the first pixel arrangement direction to obtain a current interpolation result corresponding to the pixel set, when a previous interpolation result of the current interpolation result exists, parallel interpolation processing is carried out on the previous interpolation result and the current interpolation result according to a second pixel arrangement direction perpendicular to the first pixel arrangement direction to obtain interpolation results corresponding to adjacent pixel sets, and the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed are spliced to obtain the processed image data. The method can realize the effects of less hardware resource occupation and high image processing real-time performance.

Description

Image data processing method, image data processing device, computer equipment and storage medium

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image data processing method, an image data processing apparatus, a computer device, a storage medium, and a computer program product.

Background

At present, image scaling processing is widely applied to the field of image or video display, and commonly used image scaling algorithms include nearest neighbor interpolation algorithms, bilinear interpolation algorithms, bicubic interpolation algorithms, high-order interpolation algorithms and other image scaling algorithms.

Because the LED (light emitting diode) display screen is a tiled display screen, the resolution of the LED display screen is not necessarily exactly the same as the image resolution, and therefore, image scaling processing is often required when an image or a video is displayed on the LED display screen, and a scaling function is also used when a plurality of video sources are required to be displayed in some special scenes. For example, when a video source with a resolution of 4K or 8K and a frame rate of 60 frames is input, under the condition of image magnification, the operating frequency of a zoom core on common commercial hardware is about 150M-200M, one clock processes one display pixel, 5-7 zoom cores are needed for screen display of 8k @30fps, when the 8K video source is applied to an LED control system, the 8K video source is usually cut to 8 or 16 net ports for output by a transmission card, at this time, 40-112 zoom cores are needed, and obviously, the hardware resource consumption of the number of the cores is unreasonable. According to the scheme used by the existing multi-core zooming function of the LED control system, after an image is completely cached, the cached image is divided into a plurality of image sub-blocks, the image sub-blocks are simultaneously input into a plurality of zooming cores to be subjected to multi-core parallel processing, the image zooming real-time performance is reduced because the image can be subjected to parallel processing only by caching, and more resources are consumed by the high-resolution image cache.

Therefore, it is desirable to provide a scheme for achieving higher real-time image processing using less hardware resources.

Disclosure of Invention

In view of the foregoing, it is desirable to provide an image data processing method, an image data processing apparatus, a computer device, a computer readable storage medium, and a computer program product, which occupy less hardware resources and have high image processing real-time performance.

In a first aspect, the present application provides an image data processing method. The method comprises the following steps:

receiving image data to be processed, wherein the image data to be processed comprises a pixel set which is input at the current time point and extracted from an image to be processed according to a selected data input mode, and the pixel set consists of a plurality of continuous pixels positioned in the first pixel arrangement direction;

according to the first pixel arrangement direction, carrying out parallel interpolation processing on the pixel set to obtain a current interpolation result corresponding to the pixel set, wherein the element number of the current interpolation result is linearly related to the element number of the pixel set;

when a previous interpolation result of the current interpolation result exists, performing parallel interpolation processing on the previous interpolation result and the current interpolation result according to a second pixel arrangement direction to obtain an interpolation result corresponding to an adjacent pixel set, wherein the previous interpolation result is obtained by performing parallel interpolation processing on a pixel set received at a previous time point, and the first pixel arrangement direction is vertical to the second pixel arrangement direction;

and collecting the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed, and splicing and merging the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed to obtain processed image data.

In one embodiment, the parallel interpolation processing of the pixel set includes:

acquiring position information of each pixel point in a pixel set;

correspondingly acquiring a plurality of first-type interpolation coefficient arrays from a preset first-type interpolation coefficient table according to the position information of each pixel point;

performing parallel interpolation processing on the pixel set according to the plurality of first type interpolation coefficient arrays;

the first type interpolation coefficient table comprises a mapping relation between position information of a pixel to be interpolated and a first type interpolation coefficient array, and the number of interpolation coefficients in the first type interpolation coefficient array is determined based on the position information of each pixel point in the pixel set.

In one embodiment, the parallel interpolation processing of the pixel set according to the plurality of arrays of the first type interpolation coefficients includes:

taking each pixel point in the pixel set as a start, taking adjacent pixel points to obtain adjacent pixel groups, sequentially inputting each adjacent pixel group to each built-in first-type interpolation processing core, and respectively inputting the first-type interpolation coefficient arrays to the corresponding first-type interpolation processing cores according to the position information of the first pixel point in the adjacent pixel groups, so that the first-type interpolation processing cores perform interpolation processing on each input adjacent pixel group according to the input first-type interpolation coefficient arrays to obtain the interpolation arrays corresponding to each adjacent pixel group.

In one embodiment, the parallel interpolation processing on the previous interpolation result and the current interpolation result includes:

acquiring the initial row and column positions of the pixel set and the row and column numbers participating in interpolation processing;

correspondingly acquiring a plurality of second-type interpolation coefficient arrays from a preset second-type interpolation coefficient table according to the initial row and column positions and the row and column numbers participating in interpolation;

according to the plurality of second type interpolation coefficient arrays, performing parallel interpolation processing on the previous interpolation result and the current interpolation result;

the second-type interpolation coefficient table contains the mapping relation between the position information of the pixel to be interpolated and a second-type interpolation coefficient array, and the number of interpolation coefficients in the second-type interpolation coefficient array is determined based on the position information of each pixel point in the pixel set.

In one embodiment, the previous interpolation result and the current interpolation result include a plurality of interpolation arrays;

according to the plurality of second-type interpolation coefficient arrays, the parallel interpolation processing of the previous interpolation result and the current interpolation result comprises the following steps:

sequentially and correspondingly extracting a first interpolation array of a previous interpolation result and a second interpolation array in a current interpolation result, and combining the first interpolation array and the second interpolation array to obtain a plurality of target interpolation arrays;

and respectively inputting the target interpolation array and the second type interpolation coefficient array into each built-in second type interpolation processing core, so that each second type interpolation processing core carries out interpolation processing on the input target interpolation array according to the input second type interpolation coefficient array.

In one embodiment, before performing parallel interpolation processing on the pixel set, the method further includes:

screening out effective pixel points to be interpolated in the pixel set to obtain an effective pixel set;

the parallel interpolation processing of the pixel set comprises the following steps:

and carrying out parallel interpolation processing on the effective pixel set.

screening out effective pixel points to be interpolated and invalid pixel points which do not need to be interpolated in the pixel set;

and stopping the built-in first type interpolation processing core which has a corresponding relation with the invalid pixel points so as to enable the remaining first type interpolation processing cores to perform parallel interpolation processing on the valid pixel points.

In one embodiment, the data input mode comprises a point-by-point and line-by-line input mode, a point-by-point and column-by-column input mode, a multiple row point-by-point input mode, or a multiple column point-by-point input mode;

if the data input mode is a point-by-point line-by-line input mode, the pixel set consists of a plurality of continuous pixels positioned in the transverse pixel arrangement direction;

if the data input mode is a point-by-point and column-by-column input mode, the pixel set consists of a plurality of continuous pixels positioned in the longitudinal pixel arrangement direction;

if the data input mode is a multi-row point-by-point input mode, the pixel set consists of a plurality of pixels with the same column number and continuous row numbers;

if the data input mode is a multi-column point-by-point input mode, the pixel set is composed of a plurality of pixels with the same row number and continuous column numbers.

In one embodiment, the data input mode comprises a point-by-point and line-by-line input mode, a point-by-point and column-by-column input mode, a multi-row and point-by-point input mode or a multi-column and point-by-point input mode, and the image data to be processed comprises an image scaling ratio and row position information and/or column position information of each pixel in the pixel set;

screening out effective pixel points to be interpolated in a pixel set comprises the following steps:

if the data input mode is a point-by-point and line-by-line input mode, screening effective pixel points to be interpolated in a pixel set according to the image scaling and the line position information of each pixel in the pixel set;

if the data input mode is a point-by-point and column-by-column input mode, screening out effective pixel points to be interpolated in the pixel set according to the image scaling and the column position information of each pixel in the pixel set;

and if the data input mode is a multi-row point-by-point input mode or a multi-column point-by-point input mode, screening out effective pixel points to be interpolated in the pixel set according to the image scaling, the row position information and the column position information.

In a second aspect, the present application further provides an image data processing apparatus. The device comprises:

the data receiving module is used for receiving image data to be processed, the image data to be processed comprises a pixel set which is input at the current time point and extracted from an image to be processed according to the selected data input mode, and the pixel set consists of a plurality of continuous pixels positioned in the first pixel arrangement direction;

the first parallel interpolation processing module is used for performing parallel interpolation processing on the pixel set according to the first pixel arrangement direction to obtain a current interpolation result corresponding to the pixel set and caching the current interpolation result, wherein the element number of the current interpolation result is linearly related to the element number of the pixel set;

the second parallel interpolation processing module is used for performing parallel interpolation processing on the previous interpolation result and the current interpolation result according to a second pixel arrangement direction to obtain an interpolation result corresponding to an adjacent pixel set when the previous interpolation result of the current interpolation result exists in the cache data, the previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point, and the first pixel arrangement direction is vertical to the second pixel arrangement direction;

and the data merging module is used for collecting the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed, and splicing and merging the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed to obtain processed image data.

In a third aspect, the present application further provides an image data processing apparatus. The device comprises a data screening module, a first type interpolation module, a second type interpolation module, a data cache management module and a data merging module;

the data screening module receives a pixel set which is input at a current time point and extracted from an image to be processed according to a selected data input mode, the pixel set is input into the first type interpolation module, the first type interpolation module performs parallel interpolation processing on the received pixel set according to a first pixel arrangement direction to obtain a current interpolation result corresponding to the pixel set, the current interpolation result is sent to the data cache management module for caching, when the data cache management module detects that a last interpolation result of the current interpolation result exists in cache data, the current interpolation result and the last interpolation result of the current interpolation result are sent to the second type interpolation module, the second type interpolation module performs parallel interpolation processing on the last interpolation result and the current interpolation result according to a second pixel arrangement direction to obtain an interpolation result corresponding to an adjacent pixel set, the interpolation results corresponding to the adjacent pixel sets are output to the data merging module, the data merging module merges the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed, and outputs image data after the image data are merged;

the pixel set consists of a plurality of continuous pixels positioned in a first pixel arrangement direction, the number of elements of a current interpolation result is linearly related to the number of elements of the pixel set, a previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at a previous time point, and the first pixel arrangement direction is vertical to a second pixel arrangement direction.

In a fourth aspect, the present application further provides a computer device. The computer device comprises a memory and a processor, the memory stores a computer program, and the processor realizes the following steps when executing the computer program:

In a fifth aspect, the present application further provides a computer-readable storage medium. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of:

In a sixth aspect, the present application further provides a computer program product. Computer program product comprising a computer program which, when executed by a processor, performs the steps of:

On one hand, the method, the device, the equipment, the computer equipment, the storage medium and the computer program product take continuous pixels extracted from the image to be processed as input, and carry out parallel interpolation processing on the input pixels, namely, the image does not need to be cached and segmented, and an input-processing-as-processed 'pipeline' real-time image processing mode is realized, so that the real-time performance of image data processing is higher; on the other hand, because the input pixel sets have a time sequence and are not input simultaneously, the parallel interpolation processing of the pixel sets input at different time points in the first pixel arrangement direction can share one interpolation processing core for interpolation processing, so that the number of the interpolation processing cores can be reduced, and the consumption of hardware resources is saved. In conclusion, the adoption of the scheme can realize the effects of less hardware resource occupation and high image processing real-time performance.

Drawings

FIG. 1 is a diagram of an exemplary embodiment of a method for processing image data;

FIG. 2 is a flowchart illustrating a method for processing image data according to one embodiment;

FIG. 3 is a flowchart illustrating a method for processing image data according to another embodiment;

FIG. 4 is a detailed flowchart of an image data processing method according to another embodiment;

FIG. 5 is a block diagram showing the configuration of an image data processing apparatus according to an embodiment;

FIG. 6 is a block diagram showing a configuration of an image data processing apparatus according to another embodiment;

FIG. 7 is a block diagram showing a configuration of an image data processing apparatus according to an embodiment;

FIG. 8 is a block diagram showing a configuration of an image data processing apparatus according to an embodiment;

FIG. 9 is a signal flow diagram of an image data processing apparatus according to an embodiment;

FIG. 10 is a block diagram of the internal components of the data screening module in one embodiment;

FIG. 11 is an internal block diagram of a scaling factor calculation module in one embodiment;

FIG. 12 is a block diagram of the internal components of the lateral interpolation module in one embodiment;

FIG. 13 is a diagram of the internal structure of a pixel interpolation sub-module in one embodiment;

FIG. 14 is a diagram illustrating the internal structure of a data cache management module in accordance with one embodiment;

FIG. 15 is a diagram illustrating the internal structure of a cache submodule in one embodiment;

FIG. 16 is a block diagram of the internal components of the vertical interpolation module in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The image data processing method provided by the embodiment of the application can be applied to the application environment shown in fig. 1. Wherein, the upper computer 102 and the image processing terminal 104 communicate with each other, and take image zooming as an example for explanation, a worker can upload an image to be processed to the upper computer 102, select a corresponding data input mode, input corresponding image processing data such as size information before and after image zooming, send an image processing instruction to the upper computer 102, the upper computer 102 receives the image processing data, sequentially extract a certain number of continuous pixel points from the image to be processed according to the selected data input mode to obtain image data to be processed, then sequentially input the extracted pixel sets and the image processing data to the image processing terminal 104, the image processing terminal 104 receives the image data to be processed, the image data to be processed includes the pixel sets extracted from the image to be processed according to the selected data input mode input at the current time point, the pixel set is composed of a plurality of continuous pixels positioned in a first pixel arrangement direction, then, according to the first pixel arrangement direction, the pixel set is subjected to parallel interpolation processing to obtain a current interpolation result corresponding to the pixel set, the element number of the current interpolation result is linearly related to the element number of the pixel set, when a last interpolation result of the current interpolation result exists, according to a second pixel arrangement direction, the last interpolation result and the current interpolation result are subjected to parallel interpolation processing to obtain an interpolation result corresponding to an adjacent pixel set, the last interpolation result is obtained by carrying out parallel interpolation processing on the pixel set received at a last time point, the first pixel arrangement direction is vertical to the second pixel arrangement direction, the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed are collected, and the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed are spliced and processed, and outputting the processed image data to an external data receiving end such as a receiving card. The upper computer 102 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices and portable wearable devices, and the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle-mounted devices, and the like. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, and the like. The image processing terminal 104 includes, but is not limited to, a transmitting card of an LED control system.

In one embodiment, as shown in fig. 2, an image data processing method is provided, which is described by taking the method as an example applied to the image processing end in fig. 1, and includes the following steps:

step 100, receiving image data to be processed, wherein the image data to be processed comprises a pixel set which is input at the current time point and extracted from an image to be processed according to a selected data input mode, and the pixel set is composed of a plurality of continuous pixels in the first pixel arrangement direction.

In this embodiment, an example of scaling an image will be described. The data input mode refers to an input mode of pixels of an image, and includes a line-by-line pixel-by-line (may be referred to as a point-by-line input mode), and the point-by-line input mode refers to that a pixel point in a first line is taken as an input pixel of a current time point, and the pixel points in the first line are sequentially input. The image data to be processed includes image size information and a scaling, and a set of pixels extracted from the image to be processed according to the selected data input mode inputted at a current time point. The pixel set is composed of a plurality of continuous pixels located in the first pixel arrangement direction, and specifically, may be composed of a plurality of memory continuous pixels located in the first pixel arrangement direction. If the data input mode is a point-by-point and line-by-line input mode, the pixel set is composed of a plurality of continuous pixels located in the horizontal pixel arrangement direction. It is understood that the data input mode may also be a point-by-point and column-by-column input mode, and other input modes, which may be determined according to practical situations and are not limited herein. In specific implementation, pixels of an image to be processed are continuously input in a data stream mode according to a selected data input mode, namely, an input pixel set is sequentially processed according to the sequence of data input, and then the input and processing of 'pipeline' type operation is achieved.

And 200, performing parallel interpolation processing on the pixel set according to the first pixel arrangement direction to obtain a current interpolation result corresponding to the pixel set, wherein the number of elements of the current interpolation result is linearly related to the number of elements of the pixel set.

In this embodiment, bilinear interpolation is taken as an example for explanation. And after receiving the pixel set to be processed at the current time, performing parallel interpolation processing according to the inherent first pixel arrangement direction of each pixel in the pixel set. If the pixel set consists of continuous pixel points positioned in the arrangement direction of the transverse pixels, parallel transverse interpolation processing is carried out on the pixel set, and if the pixel set consists of continuous pixel points positioned in the arrangement direction of the longitudinal pixels, parallel longitudinal interpolation processing is carried out on the pixel set. Specifically, parallel interpolation processing may be performed simultaneously by a plurality of built-in interpolation processing cores to obtain a current interpolation result corresponding to the pixel set. The current interpolation result comprises a plurality of interpolation arrays, the number of elements of each interpolation array is linearly related to the number of elements of the pixel set, and the specific linear relationship is determined by the scaling. If the number of the pixel points in the pixel set is 4, and the scaling ratio is 1. In specific implementation, the current interpolation result corresponding to the pixel set is obtained, and the current interpolation result is cached. And then, receiving the pixel set input at the next time point, and then performing parallel interpolation processing on the pixel set according to the first pixel arrangement direction, wherein the input of the pixel set has a time sequence, so that the same interpolation processing core can be adopted to perform interpolation processing on the pixel set after receiving the pixel set every time, namely the pixel sets input at different time points can share one interpolation processing core to perform interpolation processing. Specifically, taking a data input mode as an example of point-by-point and line-by-line input, taking a first pixel arrangement direction as a horizontal direction, when parallel horizontal interpolation processing is performed on a pixel set to obtain a horizontal interpolation result, the horizontal interpolation result number and an interpolation result array can be cached, each horizontal interpolation processing core has an interpolation result cache region, each interpolation result cache region at least has a size space for caching two lines of terminal display width pixel numbers, although the processing mode only needs to cache one line, in order to avoid that a second line of current interpolation is a first line of next line interpolation, the line interpolation needs to cache two lines of interpolation results, and interpolation results of odd lines and even lines are written into a fixed position during caching, such processing ensures that when pixels are input, other lines except the first line can always obtain interpolation results of a previous line.

And 300, when a previous interpolation result of the current interpolation results exists, performing parallel interpolation processing on the previous interpolation result and the current interpolation result according to a second pixel arrangement direction to obtain an interpolation result corresponding to an adjacent pixel set, wherein the previous interpolation result is obtained by performing parallel interpolation processing on a pixel set received at a previous time point, and the first pixel arrangement direction is vertical to the second pixel arrangement direction.

With the above embodiment, each current interpolation result corresponding to a pixel set is buffered, so that when a previous interpolation result of the current interpolation result exists in the buffered data, it indicates that parallel interpolation processing in the second pixel arrangement direction can be performed at this time. For example, if a pixel set composed of pixel points in a first row of an image to be processed is input from a time point t0, parallel horizontal interpolation processing is performed on the pixel set input from the t0, a horizontal interpolation result corresponding to the t0 is obtained, and the cache is stored, at this time, because only a horizontal interpolation result of one row of pixels exists in cache data, parallel vertical interpolation processing cannot be performed, therefore, a pixel set composed of pixel points in a second row of the image to be processed and input from the time point t1 is continuously received, horizontal interpolation processing is performed on the received pixel set, when an interpolation result corresponding to a pixel set input from the time point t1 is obtained, at this time, two groups of continuous interpolation results corresponding to time points t0 and t1 exist in cache data, that is, an interpolation result of a current row of pixels and an interpolation result of a previous row of pixels exist, vertical interpolation processing may be performed, that two groups of continuous interpolation results corresponding to time points t0 and t1 are parallel interpolated, an interpolation result corresponding to an adjacent pixel set, that is a coming time point t2, that is, when a time point t2 is obtained, that is, a pixel result corresponding to a third row of a pixel, and a vertical interpolation result corresponding to an adjacent pixel set may be obtained. Thus, according to the above manner, the interpolation result corresponding to each group of adjacent pixel sets of the image to be processed can be obtained through processing.

And 400, collecting interpolation results corresponding to each group of adjacent pixel sets of the image to be processed, and splicing and merging the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed to obtain processed image data.

After the interpolation result corresponding to each group of adjacent pixel sets of the image to be processed is obtained, the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed can be collected, and the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed are spliced and merged according to the pixel positions to obtain complete zoomed image information so as to normally display image data.

On one hand, on the one hand, continuous pixels extracted from an image to be processed are used as input, and parallel interpolation processing is carried out on the input pixels, namely, caching and image segmentation operations are not needed to be carried out on the image, and a pipeline real-time image processing mode of input and processing is realized, so that the real-time performance of image data processing is higher; on the other hand, because the input pixel sets have time sequence and are not input simultaneously, the parallel interpolation processing of the pixel sets input at different time points in the first pixel arrangement direction can share one interpolation processing core for interpolation processing, so that the number of the interpolation processing cores can be reduced, and the hardware resource consumption is saved. In conclusion, the adoption of the scheme can realize the effects of less hardware resource occupation and high image processing real-time performance.

In one embodiment, the data input mode comprises a point-by-point and row-by-row input mode, a point-by-column input mode, a multiple row-by-point input mode, or a multiple column-by-point input mode;

if the data input mode is a point-by-point and line-by-line input mode, the pixel set consists of a plurality of continuous pixels positioned in the transverse pixel arrangement direction;

In this embodiment, the data input mode includes a dot-by-dot and row-by-row input mode, a dot-by-row and column-by-column input mode, a multi-row and dot-by-dot input mode, or a multi-column and dot-by-dot input mode. Specifically, if the data input mode is a point-by-point and line-by-line input mode, the pixel set is composed of a plurality of continuous pixels located in the horizontal pixel arrangement direction, in a popular way, the pixel set is composed of continuous pixels located in the same line, and all the pixel points located in the first line are sequentially input from the first line, or a plurality of the pixel points located in the first line are input at one time. If the data input mode is a point-by-point and column-by-column input mode, the pixel set is composed of a plurality of continuous pixels located in the longitudinal pixel arrangement direction, in a colloquial manner, that is, composed of continuous pixels located in the same column, and all the pixel points located in the first column are sequentially input from the first column, or a plurality of pixel points located in the first column are input at a time. If the data input mode is a multi-row point-by-point input mode, the pixel set is composed of a plurality of pixels with the same column number and continuous row numbers, that is, part of the pixels in the continuous rows of pixels are input, the first pixel of the first row, the second row, the third row and the fourth row can be input at one time, the second pixel of the first row, the second row, the third row and the fourth row is input at the next time point, and so on until all the pixels of the image to be processed are input. If the data input mode is a multi-column point-by-point input mode, the pixel set is composed of a plurality of pixels with the same row number and continuous column numbers, that is, part of pixels in the continuous multi-column pixels are input, the first pixels of the first column, the second column, the third column and the fourth column can be input at one time, the second pixels of the first column, the second column, the third column and the fourth column are input at the next time point, and so on until all the pixels of the image to be processed are input. Specifically, the data input mode may be determined according to actual situations, and is not limited herein, and it should be understood that the data input mode includes, but is not limited to, the four data input modes described above, and may also be other input modes, as long as the data input mode that can ensure that the input pixel is a plurality of consecutive pixels should be within the scope of the present embodiment. In this embodiment, by providing a plurality of data input modes, the selectivity is higher and the flexibility is higher.

In one embodiment, before performing parallel interpolation processing on the pixel set, the method further includes: screening out effective pixel points to be interpolated in the pixel set to obtain an effective pixel set; step 200 comprises: and step 220, performing parallel interpolation processing on the effective pixel set according to the first pixel arrangement direction to obtain a current interpolation result corresponding to the effective pixel set, wherein the element number of the current interpolation result is linearly related to the element number of the pixel set.

In practical application, because all the input pixels do not need to be subjected to interpolation calculation, before parallel interpolation processing is performed, a pixel set can be screened, effective pixel points to be interpolated are screened out, an effective pixel set is obtained, and then parallel interpolation processing is performed on the effective pixel set. In the embodiment, the pixel sets are screened, unnecessary interpolation processing can be saved, and the data processing speed is increased.

As shown in fig. 3, in an embodiment, before performing parallel interpolation processing on a pixel set, the method further includes: step 120, screening out effective pixel points to be interpolated and invalid pixel points which do not need to be interpolated in the pixel set; step 200 comprises: step 220, the first-type interpolation processing cores corresponding to the invalid pixel points are closed, so that the remaining first-type interpolation processing cores perform parallel interpolation processing on the effective pixel set according to the first pixel arrangement direction to obtain a current interpolation result corresponding to the effective pixel set, wherein the number of elements of the current interpolation result is linearly related to the number of elements of the effective pixel set.

In practical application, the input pixels may be subjected to parallel interpolation processing by a plurality of first-type interpolation processing cores, and the input pixels may be subjected to secondary parallel interpolation processing by a plurality of second-type interpolation processing cores, and the number of the first-type interpolation processing cores is equal to the number of the second-type interpolation processing cores, and the number of the first-type interpolation processing cores is determined by the maximum magnification factor supportable by the image processing terminal, and if the supportable maximum magnification factor is 16 times, the number of the 16 first-type interpolation processing cores and the number of the second-type interpolation processing cores are correspondingly set to support the parallel interpolation processing of the pixels. In this embodiment, for convenience of understanding, the first-type interpolation processing core is taken as a horizontal interpolation processing core, and the second-type interpolation processing core is taken as a vertical interpolation processing core. Before parallel interpolation processing is carried out on pixel points, effective pixel points to be interpolated and invalid pixel points which do not need to be interpolated can still be screened out, because the number of built-in transverse interpolation processing kernels and the number of longitudinal interpolation processing kernels are preset, if the invalid pixel points which do not need to be interpolated are detected, the transverse interpolation processing kernels which are distributed before closing and need to be processed on the invalid pixel points can be enabled through enabling signals, the transverse interpolation processing kernels are enabled to stop processing or close the output validity of the transverse interpolation processing kernels, so that the parallel interpolation processing is carried out on the effective pixel points, because the transverse interpolation processing kernels and the longitudinal interpolation processing kernels are arranged in a one-to-one correspondence mode, if the transverse interpolation processing kernels do not need to work, the longitudinal interpolation processing which is arranged in a corresponding mode can be correspondingly enabled through enabling signals. Thus, the corresponding interpolation processing core can be saved.

step 120 includes:

step 122, if the data input mode is a point-by-point and line-by-line input mode, screening effective pixel points to be interpolated in a pixel set according to the image scaling and the line position information of each pixel in the pixel set;

in step 124, if the data input mode is a point-by-point and column-by-column input mode, the effective pixel points to be interpolated in the pixel set are screened out according to the image scaling and the column position information of each pixel in the pixel set.

And step 126, screening effective pixel points to be interpolated in the pixel set according to the image scaling, the line position information and the column position information if the data input mode is a multi-line point-by-point input mode or a multi-column point-by-point input mode.

In particular, the selection of the pixel to be interpolated is dependent on the selected data input mode. Specifically, it may be: if the data input mode is a point-by-point and line-by-line input mode, effective pixel points to be interpolated in the pixel set are screened out according to the image scaling and the line position information (which can be regarded as a line number) of each pixel in the pixel set, if the data input mode is a point-by-point and line-by-line input mode, the effective pixel points to be interpolated in the pixel set are screened out according to the image scaling and the line position information of each pixel in the pixel set, and if the data input mode is a multi-line and point-by-point input mode or a multi-line and point-by-point input mode, the effective pixel points to be interpolated in the pixel set are screened out according to the image scaling, the line position information and the line position information. Specifically, whether the pixel needs to be interpolated may be determined according to whether the product of the image scaling and the line number of the pixel increases by an integer. For example, if the data input mode is input point-by-point and line-by-line, if the line number of the pixel 1 is 9, the line number of the pixel 2 is 10, and the image scaling is 1/5, the product of the pixel 1 and the image scaling is 9/5, and the product of the pixel 2 and the image scaling is 2, the product crosses from 9/5 to 2, and the crossing transition of the integer occurs when the product crosses the pixel 2, so the pixel 1 is an invalid pixel and the pixel 2 is an valid pixel. Similarly, for other data input modes, the effective pixel points can be screened according to the above mode. In this embodiment, according to the difference of data input mode, the corresponding screening mode that is provided with different effective pixel sets can select effective pixel fast, and is more nimble convenient.

As shown in FIG. 4, in one embodiment, step 200 includes: step 222, obtaining position information of each pixel point in the pixel set, correspondingly obtaining a plurality of first-type interpolation coefficient arrays from a preset first-type interpolation coefficient table according to the position information of each pixel point, and performing parallel interpolation processing on the pixel set according to the plurality of first-type interpolation coefficient arrays and the arrangement direction of the first pixels to obtain the result, wherein the first-type interpolation coefficient table contains the mapping relation between the position information of the pixel to be interpolated and the first-type interpolation coefficient arrays, and the number of the interpolation coefficients in the first-type interpolation coefficient arrays is determined based on the position information of each pixel point in the pixel set.

Since parallel interpolation processing is performed on an input pixel set, it is necessary to extract a plurality of interpolation coefficients at a time to perform interpolation. In this embodiment, the obtaining of the interpolation coefficient may be by using a mapping table calculated in advance, and specifically, the calculation and storage of the interpolation coefficient table may be completed according to the size information of the input image (including the width and height of the input image and the width and height of the zoomed image), and the mapping relationship is to map the interpolation coefficient by the position of the input interpolation pixel. And binding an interpolation coefficient array by using the first pixel position of the two input pixel points participating in interpolation, namely caching a plurality of interpolation coefficients by using one position. Because the interpolation processing of the pixels generally takes two continuous pixel points as a unit, and the position of the first pixel in the two input pixel points participating in the interpolation is bound with the interpolation coefficient array, the first type of interpolation coefficient array can be obtained according to the position of the first pixel in the two input pixel points participating in the interpolation. When the interpolation coefficient table is cached, the coefficient numbers and a plurality of coefficients are cached together, mapping and searching are facilitated, a horizontal row and a vertical column are taken for cache calculation, the cached data amount at each position is consistent, and the maximum magnification factor (the maximum magnification factor is an integer, otherwise, the integer is upwards taken) +1 can be supported for determining and storing the coefficients and the coefficient numbers. Scaling factors determine that two interpolation positions of input can be directly inserted with a few pixels, part of the positions may not participate in interpolation during scaling, the number of buffer coefficients and coefficients are both 0 at the moment, and the scaling factor k is not necessarily an integer during scaling, and the number of coefficients between two input pixel positions may vary between k rounding down and k rounding up. And, the number of interpolation coefficients for each pixel to be interpolated is determined based on the position of the pixel, i.e. the number of interpolation coefficients for the pixel is calculated from the position of the input pixel, unlike the conventional calculation of the number of interpolation coefficients for the pixel based on the position of the output pixel. In this embodiment, the interpolation coefficient is pre-calculated and cached in the form of the interpolation coefficient table, so that the interpolation coefficient can be accurately and quickly taken out, and the calculation of interpolation processing is facilitated.

In one embodiment, the parallel interpolation processing of the pixel set according to the plurality of arrays of the first type interpolation coefficients includes: taking each pixel point in the pixel set as a start, taking adjacent pixel points to obtain adjacent pixel groups, sequentially inputting each adjacent pixel group to each built-in first-type interpolation processing core, and respectively inputting the first-type interpolation coefficient arrays to the corresponding first-type interpolation processing cores according to the position information of the first pixel point in the adjacent pixel groups, so that the first-type interpolation processing cores perform interpolation processing on each input adjacent pixel group according to the input first-type interpolation coefficient arrays to obtain the interpolation arrays corresponding to each adjacent pixel group.

In this embodiment, the first-type interpolation processing core is also taken as an example of the horizontal interpolation processing core. The method includes the steps of taking adjacent pixel points from each pixel point in a pixel set as a start point to obtain adjacent pixel groups, sequentially inputting the adjacent pixel groups to each built-in first-type interpolation processing core, and respectively inputting first-type interpolation coefficient arrays to the corresponding first-type interpolation processing cores according to position information of a first pixel point in the adjacent pixel groups, so that each first-type interpolation processing core performs interpolation processing on the input adjacent pixel groups according to the received first-type interpolation coefficient arrays to obtain interpolation arrays corresponding to the adjacent pixel groups. For example, for convenience of illustration, taking 4 consecutive pixel inputs as an example, if P5, P6, P7, and P8 represent four pixels connected in one line, the last pixel point P4 and P5 input at the last time point may be input to a first horizontal interpolation processing core, P5 and P6 may be input to a second horizontal interpolation processing core, P6 and P7 may be input to a third horizontal interpolation processing core, P7 and P8 may be input to a fourth horizontal interpolation processing core, and then the interpolation coefficient array corresponding to the pixel point may be input to the corresponding horizontal interpolation processing core, and after each horizontal interpolation processing core receives the horizontal interpolation coefficient array, the interpolation calculation may be performed according to the horizontal position of the pixel point received by the horizontal interpolation processing core, such as a line number. The last pixel in the pixel set input at the last time point and the first pixel input currently are used as adjacent pixel groups to be input, so that the continuity of interpolation processing can be ensured. It can be understood that, if the input of the first round of pixels is, the last pixel at the previous time point is empty, and at this time, the first horizontal interpolation processing core does not need to perform interpolation processing. In this embodiment, by respectively inputting the adjacent pixel groups and the corresponding horizontal interpolation coefficient arrays into the corresponding horizontal interpolation processing cores, parallel calculation of horizontal interpolation can be achieved, and image processing real-time performance is improved.

As shown in FIG. 4, in one embodiment step 300 comprises: and 320, acquiring the initial row and column positions of the pixel set and the row and column numbers participating in interpolation processing, correspondingly acquiring a plurality of second-type interpolation coefficient arrays from a preset second-type interpolation coefficient table according to the initial row and column positions and the row and column numbers participating in interpolation, and performing parallel interpolation processing on the previous interpolation result and the current interpolation result according to the plurality of second-type interpolation coefficient arrays and the second pixel arrangement direction, wherein the second-type interpolation coefficient table comprises the mapping relation between the position information of the pixel to be interpolated and the second-type interpolation coefficient arrays, and the number of interpolation coefficients in the second-type interpolation coefficient arrays is determined based on the position information of each pixel point in the pixel set.

In this embodiment, the second-type interpolation coefficient table is taken as an example of a vertical interpolation coefficient table, and the second-type interpolation processing core is taken as an example of a vertical interpolation processing core. The acquisition mode of the longitudinal interpolation coefficient array may be: the method comprises the steps of obtaining the position of a starting row and a starting column in an input pixel and the number of rows participating in interpolation processing, obtaining the position of the starting row of the input pixel and the number of rows participating in interpolation if the data input mode is a point-by-point and line-by-line input mode or a multi-row and point-by-point input mode, continuously obtaining longitudinal interpolation coefficient arrays of the corresponding number of rows from the position of the starting row according to the number of rows participating in interpolation, and then carrying out parallel longitudinal interpolation processing on a previous interpolation result and a current interpolation result according to a plurality of longitudinal interpolation coefficient arrays. Similarly, if the data input mode is a point-by-point and column-by-column or multi-column and point-by-point input mode, the initial row and column positions in the input pixels and the row and column numbers participating in the interpolation processing are acquired, and if the data input mode is a point-by-point and row-by-row input mode or a multi-row and point-by-point input mode, the initial row and column positions in the input pixels and the column numbers participating in the interpolation are acquired, the longitudinal interpolation coefficient arrays of the corresponding number of columns are continuously acquired from the initial row positions according to the column numbers participating in the interpolation, and then the previous interpolation result and the current interpolation result are subjected to the column-by-column interpolation processing according to a plurality of second type interpolation coefficient arrays. In this embodiment, the interpolation coefficient is pre-calculated and cached in the form of the interpolation coefficient table, so that the interpolation coefficient can be accurately and quickly taken out, and the calculation of interpolation processing is facilitated.

In one embodiment, the previous interpolation result and the current interpolation result include a plurality of interpolation arrays; according to the plurality of second type interpolation coefficient arrays, the parallel interpolation processing of the previous interpolation result and the current interpolation result comprises the following steps: and sequentially and correspondingly extracting a first interpolation array of a previous interpolation result and a second interpolation array in a current interpolation result, combining the first interpolation array and the second interpolation array to obtain a plurality of target interpolation arrays, and respectively inputting the target interpolation arrays and the second type interpolation coefficient arrays to each built-in second type interpolation processing core so that each second type interpolation processing core can carry out interpolation processing on the input target interpolation arrays according to the input second type interpolation coefficient arrays.

After the multiple longitudinal interpolation coefficient arrays are obtained, the first interpolation array of the previous interpolation result and the second interpolation array of the current interpolation result may be sequentially taken from the beginning in a manner of taking numbers one by one, and the first interpolation array and the second interpolation array are combined to obtain multiple target interpolation arrays. Specifically, if the data input mode is point-by-point and line-by-line input or point-by-point and column-by-column input, because the input pixels have a time sequence, one core can be shared for interpolation processing, and the cache of the horizontal interpolation result is based on the cache of two continuous lines or two continuous columns of interpolation results as a unit, so that the horizontal interpolation results of the input pixels at the current time point and the last time point can be continuously stored in one cache area, that is, the interpolation arrays can be sequentially taken from the same cache area directly and combined to obtain the target interpolation array. If the data input mode is multi-row point-by-point input or multi-column point-by-point input, the horizontal interpolation results of the input pixels at the current time point and the previous time point are not necessarily stored in a buffer area, and at the moment, the current interpolation result and the previous interpolation result are taken from the corresponding buffer areas. Then, the target interpolation array and the longitudinal interpolation coefficient array are respectively input to each built-in longitudinal interpolation processing core, it should be noted that the longitudinal interpolation coefficients received by each longitudinal interpolation processing core are the same, each longitudinal interpolation processing core may correspondingly select a corresponding longitudinal interpolation coefficient array from the received longitudinal interpolation coefficient array according to a relative line number or a relative column number, where the relative line number and the relative column number are determined by a difference between the line number and the column number of the pixel received by the longitudinal interpolation processing core and the minimum line number or the minimum column number of the pixels in the pixel set, and if the line number of the input pixel is the 5 th line, the 6 th line, the 7 th line, and the 8 th line, respectively, and the line number of the pixel received by the second longitudinal interpolation processing core is the 6 th line, the relative line number of the second longitudinal interpolation processing core is 2, the longitudinal interpolation coefficient array arranged at the 2 nd position in the received longitudinal interpolation coefficient array is correspondingly selected. If the input is the pixels of the same line or the same column, the relative line number of the longitudinal interpolation kernel is 0, and at this time, the longitudinal interpolation coefficient array is selected directly according to the line number or the column number of the received interpolation result to perform longitudinal interpolation calculation. In specific implementation, the longitudinal interpolation processing core and the transverse interpolation processing core are both interpolation processing cores in nature, and are only distinguished in naming according to functions. Each longitudinal interpolation processing core also comprises m interpolation processing cores, m is the longitudinal maximum magnification, the longitudinal interpolation processing cores split and input the input transverse interpolation result arrays into each interpolation processing core for interpolation processing, and finally output of all the longitudinal interpolation processing cores is merged and output. In this embodiment, the target interpolation array and the longitudinal interpolation coefficient array are respectively input into the corresponding longitudinal interpolation processing cores, so that parallel computation of longitudinal interpolation can be realized, and the image processing real-time performance is improved.

For clearly explaining the image data processing method provided by the present application, the following description is made with reference to a specific embodiment, and in this embodiment, a data input mode is described point-by-point and line-by-line, which specifically includes the following contents:

a worker selects a data input mode as a point-by-point line-by-line input mode in an upper computer, extracts line pixels of an image to be processed line by line to obtain a pixel set, sends parameters such as size information and scaling of the image to be processed to an image processing end, and sequentially transmits the extracted pixel set to the image processing end according to time points.

The image processing end receives a pixel set consisting of a first line of pixel points of an image to be processed and transmitted by a host computer at the current time, and parameters such as size information, scaling multiple and the like, then, each pixel point in the pixel set is taken as a start, adjacent pixel points are obtained, adjacent pixel groups are obtained, the adjacent pixel groups are sequentially input into each built-in transverse interpolation processing core, a transverse interpolation coefficient array is obtained according to the position of a first pixel in two input pixel points participating in interpolation, the transverse interpolation coefficient array is respectively input into each transverse interpolation processing core, then, according to the image scaling ratio and the line position information (which can be regarded as a line number) of each pixel in the pixel set, an effective pixel point to be interpolated and an ineffective pixel point which does not need interpolation in the pixel set are correspondingly closed, so that the rest transverse interpolation processing cores are screened out, and the received effective pixel points are subjected to parallel transverse interpolation processing according to the received transverse interpolation coefficient array, and a transverse interpolation processing result corresponding to the effective pixel set (comprising a plurality of adjacent pixel groups) is obtained and cached. And when the next time point arrives, performing parallel transverse interpolation processing on the second row of pixel points according to the mode to obtain a transverse interpolation processing result corresponding to the second row of pixel points, and caching. In this case, if the result of the horizontal interpolation process for the previous line of pixels to the current line of pixels exists in the buffer data, it means that the vertical interpolation process needs to be performed.

The processing of the vertical interpolation processing may be to acquire a starting line position of the input pixel and the number of lines participating in the interpolation, continuously acquire the vertical interpolation coefficient arrays of the corresponding number of lines from the starting line position according to the number of lines participating in the interpolation, and then perform parallel vertical interpolation processing on the previous interpolation result and the current interpolation result according to the plurality of vertical interpolation coefficient arrays. The specific parallel vertical processing may be: the method comprises the steps of sequentially and correspondingly extracting a first interpolation array of a previous interpolation result and a second interpolation array in a current interpolation result, combining the first interpolation array and the second interpolation array to obtain a plurality of target interpolation arrays, respectively inputting the target interpolation arrays and the longitudinal interpolation coefficient arrays to built-in longitudinal interpolation processing cores, and selecting corresponding longitudinal interpolation coefficient arrays from the received longitudinal interpolation coefficient arrays correspondingly by each longitudinal interpolation processing core according to a relative line number or a relative column number, and then performing longitudinal interpolation processing on the received target interpolation arrays according to the selected longitudinal interpolation coefficients to obtain interpolation results corresponding to adjacent pixel sets. And finally, collecting the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed, and splicing and merging the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed according to the pixel positions to obtain complete zoomed image information.

It should be understood that, although the steps in the flowcharts related to the embodiments as described above are sequentially displayed as indicated by arrows, the steps are not necessarily performed sequentially as indicated by the arrows. The steps are not performed in the exact order shown and described, unless explicitly stated otherwise, and may be performed in other orders. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the present application further provides an image data processing apparatus for implementing the image data processing method. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the method, so specific limitations in one or more embodiments of the image data processing apparatus provided below may refer to the limitations on the image data processing method in the foregoing, and details are not described here again.

In one embodiment, as shown in fig. 5, there is provided an image data processing apparatus including: a data receiving module 510, a first parallel interpolation processing module 520, a second parallel interpolation processing module 530, and a data stitching and merging module 540, wherein:

a data receiving module 510, configured to receive to-be-processed image data, where the to-be-processed image data includes a pixel set extracted from a to-be-processed image according to a selected data input mode and input at a current time point, and the pixel set is composed of a plurality of consecutive pixels located in a first pixel arrangement direction.

The first parallel interpolation processing module 520 is configured to perform parallel interpolation processing on the pixel set according to the first pixel arrangement direction, obtain a current interpolation result corresponding to the pixel set, and cache the current interpolation result, where the number of elements of the current interpolation result is linearly related to the number of elements of the pixel set.

A second parallel interpolation processing module 530, configured to, when a previous interpolation result of the current interpolation result exists in the cache data, perform parallel interpolation processing on the previous interpolation result and the current interpolation result according to a second pixel arrangement direction to obtain an interpolation result corresponding to an adjacent pixel set, where the previous interpolation result is obtained by performing parallel interpolation processing on a pixel set received at a previous time point, and a first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

And the data stitching and merging module 540 is configured to collect interpolation results corresponding to each group of adjacent pixel sets of the image to be processed, and perform stitching and merging processing on the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed to obtain processed image data.

On one hand, the image data processing device takes continuous pixels extracted from an image to be processed as input, and performs parallel interpolation processing on the input pixels, namely, the image does not need to be cached and segmented, and an input-processing pipeline real-time image processing mode is realized, so that the real-time performance of image data processing is higher; on the other hand, because the input pixel sets have a time sequence and are not input simultaneously, the parallel interpolation processing of the pixel sets input at different time points in the first pixel arrangement direction can share one interpolation processing core for interpolation processing, so that the number of the interpolation processing cores can be reduced, and the consumption of hardware resources is saved. In conclusion, the device can achieve the effects of less hardware resource occupation and high image processing real-time performance.

In one embodiment, the first parallel interpolation processing module 520 is further configured to obtain position information of each pixel point in the pixel set; according to the position information of each pixel point, correspondingly acquiring a plurality of first-type interpolation coefficient arrays from a preset first-type interpolation coefficient table; performing parallel interpolation processing on the pixel set according to the plurality of first type interpolation coefficient arrays; the first-type interpolation coefficient table comprises a mapping relation between the position information of the pixel to be interpolated and a first-type interpolation coefficient array, and the number of interpolation coefficients in the first-type interpolation coefficient array is determined based on the position information of each pixel point in the pixel set.

In an embodiment, the first parallel interpolation processing module 520 is further configured to start with each pixel point in the pixel set, obtain neighboring pixel groups, sequentially input each neighboring pixel group to each built-in first-type interpolation processing core, and respectively input the first-type interpolation coefficient arrays to the corresponding first-type interpolation processing cores according to the position information of the first pixel point in the neighboring pixel groups, so that the first-type interpolation processing cores perform interpolation processing on each input neighboring pixel group according to the input first-type interpolation coefficient array, and obtain interpolation arrays corresponding to each neighboring pixel group.

In one embodiment, the second parallel interpolation processing module 530 is further configured to obtain a starting row and column position of the pixel set and a number of rows and columns participating in the interpolation processing; correspondingly acquiring a plurality of second-type interpolation coefficient arrays from a preset second-type interpolation coefficient table according to the initial row and column positions and the row and column numbers participating in interpolation; according to the plurality of second type interpolation coefficient arrays, performing parallel interpolation processing on the previous interpolation result and the current interpolation result; the second-type interpolation coefficient table comprises a mapping relation between the position information of the pixel to be interpolated and a second-type interpolation coefficient array, and the number of interpolation coefficients in the second-type interpolation coefficient array is determined based on the position information of each pixel point in the pixel set.

In one embodiment, the previous and current interpolation results include a plurality of interpolation arrays;

the second parallel interpolation processing module 530 is further configured to sequentially and correspondingly extract the first interpolation array of the previous interpolation result and the second interpolation array of the current interpolation result, and combine the first interpolation array and the second interpolation array to obtain a plurality of target interpolation arrays; and respectively inputting the target interpolation array and the second type interpolation coefficient array into each built-in second type interpolation processing core, so that each second type interpolation processing core performs interpolation processing on the input target interpolation array according to the input second type interpolation coefficient array.

As shown in fig. 6, in an embodiment, the apparatus further includes a pixel filtering module 550, configured to filter out effective pixels to be interpolated in the pixel set, so as to obtain an effective pixel set; the first parallel interpolation module 520 is further configured to perform parallel interpolation processing on the effective pixel set.

In one embodiment, the pixel filtering module 550 is further configured to filter out valid pixel points to be interpolated and invalid pixel points that do not need to be interpolated in the pixel set; the first parallel interpolation module 520 is further configured to deactivate the built-in first type interpolation processing core having a corresponding relationship with the invalid pixel point, so that the remaining first type interpolation processing cores perform parallel interpolation processing on the valid pixel points.

In one embodiment, the data input mode comprises a point-by-point and line-by-line input mode, a point-by-point and column-by-column input mode, a multi-row and point-by-point input mode or a multi-column and point-by-point input mode, and the image data to be processed comprises an image scaling ratio and row position information and/or column position information of each pixel in a pixel set;

the pixel screening module 550 is further configured to screen out effective pixel points to be interpolated in the pixel set according to the image scaling and the row position information of each pixel in the pixel set if the data input mode is a point-by-point and line-by-line input mode; if the data input mode is a point-by-point and column-by-column input mode, screening out effective pixel points to be interpolated in the pixel set according to the image scaling and the column position information of each pixel in the pixel set; and if the data input mode is a multi-row point-by-point input mode or a multi-column point-by-point input mode, screening effective pixel points to be interpolated in the pixel set according to the image scaling, the row position information and the column position information.

The respective modules in the image data processing apparatus described above may be wholly or partially implemented by software, hardware, and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, as shown in fig. 7, the present application further provides an image data processing apparatus including a data filtering module 710, a first type interpolation module 720, a second type interpolation module 730, a data cache management module 740, and a data merging module 750.

The data screening module 710 receives a pixel set extracted from an image to be processed according to a selected data input mode and input at a current time point, the pixel set is input to the first type interpolation module 720, the first type interpolation module 720 performs parallel interpolation processing on the received pixel set according to a first pixel arrangement direction to obtain a current interpolation result corresponding to the pixel set, and sends the current interpolation result to the data cache management module 740 for caching, when the data cache management module detects that a previous interpolation result of the current interpolation result exists in cache data, the current interpolation result and a previous interpolation result of the current interpolation result are sent to the second type interpolation module 730, the second type interpolation module 730 performs parallel interpolation processing on the previous interpolation result and the current interpolation result according to a second pixel arrangement direction to obtain an interpolation result corresponding to an adjacent pixel set, outputs the interpolation result corresponding to the adjacent pixel set to the data merging module 750, the data merging module 750 merges the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed, and outputs merged image data after the image data merging processing is performed;

the pixel set is composed of a plurality of continuous pixels located in a first pixel arrangement direction, the number of elements of a current interpolation result is linearly related to the number of elements of the pixel set, a previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at a previous time point, and the first pixel arrangement direction is perpendicular to a second pixel arrangement direction.

In specific implementation, the data filtering module 710 is configured to filter out valid pixel points to be interpolated and invalid pixel points that do not need to be interpolated in the pixel set according to the scaling size information of the image.

On one hand, the image data processing equipment takes continuous pixels extracted from an image to be processed as input, and performs parallel interpolation processing on the input pixels, namely, the image does not need to be cached or segmented, and a pipeline real-time image processing mode of input and processing is realized, so that the real-time performance of image data processing is higher; on the other hand, because the input pixel sets have time sequence and are not input simultaneously, the parallel interpolation processing of the pixel sets input at different time points in the first pixel arrangement direction can share one interpolation processing core for interpolation processing, so that the number of the interpolation processing cores can be reduced, and the hardware resource consumption is saved. In conclusion, the device can achieve the effects of less hardware resource occupation and high image processing real-time performance.

As shown in fig. 8, in an embodiment, the apparatus further includes a scaling factor calculation module 760, configured to calculate a horizontal interpolation factor and a vertical interpolation factor, obtain a horizontal interpolation factor table and a vertical interpolation factor table, and provide functions such as information query of the interpolation factor and the scaling size.

In this embodiment, the scaling coefficient calculation module calculates in advance to obtain the horizontal interpolation coefficient table and the vertical interpolation coefficient table, so that the interpolation coefficient can be quickly obtained in the interpolation process, and the processing speed of interpolation calculation is increased.

In order to make a more clear description of the image data processing apparatus provided in the present application, the following description is made with reference to fig. 9:

the Input signal of the data screening module 710 includes Input mode, clk, src _ w, src _ h, dst _ w, dst _ h, P1, P2, P3, and P4, where Input mode is a mode of inputting data, and supports a point-by-point and row-by-column Input mode, a multi-row and point-by-point Input mode, and a multi-column and point-by-point Input mode, taking an Input pixel as 4 pixels as an example, where the point-by-point and row-by-line Input mode and the point-by-column Input mode refer to inputting four continuous pixels on a memory located in the same pixel arrangement direction (including a horizontal direction or a vertical direction), and the multi-row and point-by-point Input mode or the multi-column and point-by-point Input mode represents synchronous point-by-point pixels located in the same pixel arrangement direction of multiple continuous rows or continuous columns of an image. For convenience of description, the data input mode in the following text exemplarily takes a point-by-point and line-by-line input mode and a multi-line point-by-point input mode as an example, then P1, P2, P3, and P4 are four continuous pixels located on the memory of the horizontal pixel arrangement, or are synchronous point-by-point pixels on multiple continuous lines of the image; clk is a synchronous signal, src _ w and Src _ h respectively represent the width and height of an input image, dst _ w and Dst _ h respectively represent the width and height of a zoomed image, and P1, P2, P3 and P4 are continuous 4 pixel values of the image in the horizontal direction in space; the output signal includes vaild1, vaild2, vaild3, vaild4, K0, K1, K2, K3, P1, P2, P3, P4, and Row _ flag, where vaild1, vaild2, vaild3, vaild4 indicate whether four input pixels need interpolation calculation results for controlling the following four horizontal interpolation modules, K0, K1, K2, K3, P1, P2, P3, P4 indicate 4 pairs of pixels to be interpolated, and Row _ flag indicates a Row flag, which records the current interpolation Row number and the Row number of the four interpolation kernels relative to the current interpolation Row (i.e. the relative Row number mentioned above).

Specifically, the specific structure of the data filtering module is as shown in fig. 10, and the data filtering module includes four sub-modules, which are a counting sub-module 810, a row and column position calculating sub-module 820, an interpolation filtering sub-module 830, and a data defining sub-module 840, respectively. The counting submodule 810 is configured to record and output the pixel input times, and also has a reset function, and resets when a frame is changed; the row-column position calculation submodule 820 is used for performing position calculation according to src _ w and count and outputting row-column positions of currently input four pixels, performing differentiated processing according to a data input mode during calculation, when the data input mode is row-by-row and point-by-point, sequentially increasing the four positions by adding 4 each time, and when the data input mode is 4 rows and point-by-point, adding 1 each time when the four positions are the same, and having a reset function; the function of the data definition sub-module 840 discriminates processing according to the data input mode, and when the data input mode is row-by-row, K0= P4 at the previous time, K1= P1, K2= P2, K3= P3, and when the data input mode is 4 row-by-row, K0= P1 at the previous time, K1= P2 at the previous time, K2= P3 at the previous time, K3= P4 at the previous time, and finally, P1, P2, P3, P4, K0, K1, K2, K3 are output together; the function of the interpolation screening submodule 830 distinguishes and processes according to the data input mode, when the data input mode is row-by-row and point-by-point, it is calculated and detected whether the line position of the currently input four pixels needs interpolation through the scaled size information, when the data input mode is 4 rows and point-by-point, it is directly judged whether the line position needs interpolation, at the same time, the line number of the currently interpolated line and the relative line numbers of the four longitudinal interpolation kernels are calculated according to the line, and finally, the following interpolation module is controlled through the four output signals of vaild1, vaild2, vaild3 and vaild 4.

Specifically, the input signal of the scaling factor calculation module 760 includes Src _ w, src _ h, dst _ w, dst _ h, row, col _ pos1, col _ pos2, col _ pos3, col _ pos4, where Src _ w and Src _ h respectively represent the width and height of the input image, dst _ w and Dst _ h respectively represent the width and height of the scaled image, row represents the current interpolation line number, col _ pos1, col _ pos2, col _ pos3, col _ pos4 respectively represent the positions of the four horizontal interpolation kernel input pixels; the output signals include Coef _ arr1, coef _ arr2, coef _ arr3, coef _ arr4, coef _ cnt1, coef _ cnt2, coef _ cnt3, coef _ cnt4, row _ Coef _ cnt and Row _ Coef _ arr, wherein Coef _ arr1, coef _ arr2, coef _ arr3 and Coef _ arr4 respectively represent interpolation coefficient arrays required by four transverse interpolation cores, coef _ cnt1, coef _ cnt2, coef _ cnt3 and Coef _ cnt4 respectively represent effective lengths of the four transverse interpolation coefficient arrays, and Row _ Coef _ cnt and Row _ Coef _ arr respectively represent effective numbers of the longitudinal interpolation coefficient array and the longitudinal interpolation coefficient array.

Specifically, the specific internal structure of the scaling coefficient calculation module is as shown in fig. 11, and the scaling coefficient calculation module includes an interpolation coefficient calculation sub-module 762, and a horizontal interpolation coefficient cache sub-module 764 specifically includes a horizontal interpolation coefficient cache sub-module 1, a horizontal interpolation coefficient cache sub-module 2, a horizontal interpolation coefficient cache sub-module 3, a horizontal interpolation coefficient cache sub-module 4, a vertical interpolation coefficient cache sub-module 766, and an interpolation coefficient output sub-module 768. The interpolation coefficient calculation sub-module 762 is configured to calculate a horizontal interpolation coefficient and a vertical interpolation coefficient, and perform caching; the transverse interpolation coefficient cache sub-module 764 is used for caching the transverse interpolation coefficient, and is matched with the four transverse interpolation modules for reading and writing; the vertical interpolation coefficient buffer sub-module 766 is used for buffering the vertical interpolation coefficient. The interpolation coefficient output sub-module 768 is configured to read and output a horizontal interpolation coefficient according to the horizontal position of the pixel, read two lines of valid vertical interpolation coefficient sets each time according to the vertical position of the pixel, and read and output a vertical interpolation coefficient set of the first line in the second line in the last line.

The first interpolation module 720 and the second interpolation module 730 can be a horizontal interpolation module or a vertical interpolation module, if the first interpolation module is a horizontal interpolation module, the second interpolation module is a vertical interpolation module, and the vertical interpolation module and the horizontal interpolation processing module can be interpolation kernels in nature. The following description will use the first interpolation module 720 as a horizontal interpolation module and the second interpolation module 730 as a vertical interpolation module. The horizontal interpolation module is used for carrying out horizontal interpolation on the received pixels to obtain a current interpolation result, and the longitudinal interpolation module is used for carrying out longitudinal interpolation on the received current interpolation result and a previous interpolation result to obtain an interpolation result corresponding to an adjacent pixel set. Specifically, in this embodiment, four pixel inputs are adopted, so that four transverse interpolation modules are correspondingly processed in parallel, and input signals of the whole module include vaild1, vaild2, vaild3, vaild4, col1, col2, col3, col4, P0, P1, P2, P3, P4, coef _ arr1, coef _ arr2, coef _ arr3, coef _ arr4, coef _ cnt1, coef _ cnt2, coef _ cnt3, and Coef _ cnt4, where vaild1, vaild2, vaild3, and vaild4 represent enable control signals of the four transverse interpolation modules; col1, col2, col3 and col4 represent the transverse positions of input pixels of the four transverse interpolation modules, and the transverse positions of the pixels are used for acquiring an interpolation coefficient array; p0, P1, P2, P3 and P4 represent pixels to be interpolated after screening; eight signals, i.e., signals expressed in the scaling coefficient calculation module, coef _ arr1, coef _ arr2, coef _ arr3, coef _ arr4, coef _ cnt1, coef _ cnt2, coef _ cnt3, and Coef _ cnt4 are used for the lateral interpolation calculation.

The specific internal structure of a single horizontal interpolation module is shown in fig. 12, and includes a horizontal interpolation coefficient splitting sub-module 722, a processing core validity control sub-module 724, a synchronous processing sub-module 726, n pixel interpolation sub-modules 728, and a data merging sub-module 729. The synchronous processing submodule 726 is configured to synchronously process data, synchronize the input vaild with the output P _ out _ arr and P _ out _ count signals, and transmit the col signal to the scaling coefficient calculation module to obtain a transverse interpolation coefficient array; the transverse interpolation coefficient splitting submodule 722 is used for splitting a transverse interpolation coefficient; the processing core validity control submodule 724 is used for controlling the enabling of the interpolation core according to the number of valid interpolation coefficients; the data merging sub-module 729 is configured to merge all pixels of the lateral interpolation; the pixel interpolation sub-module 728 is a pixel interpolation Core, n is the number of the pixel interpolation cores, and is determined by the lateral magnification supported by the system of the designed device (n = lateral magnification rounded up), and the structure of the pixel interpolation sub-module is as shown in fig. 13, and is designed by using an RGB (Red, green, blue, three primary colors) color model, P1 and P2 represent RGB color data of two pixels participating in interpolation, P1 obtains R1, G1, and B1 respectively after channel splitting, P2 obtains R2, G2, and B2 respectively after channel splitting, core _ coef represents an interpolation coefficient group of the processing Core, two coefficients coef1 and coef2 of linear interpolation are obtained by splitting, three groups of linear interpolation processing obtain RO, GO, BO, and Core _ vacuum represents an interpolation Core enable control signal, and the control channel combining processing and the control channel combining signal are transmitted and output to the vacuum signal, which includes a channel splitting processing sub-unit 820, an interpolation coefficient splitting processing sub-unit 822, a linear interpolation Core 824, and a channel combining processing module 826.

The buffer management module 740 is configured to buffer the current interpolation result and the interpolation result corresponding to the adjacent pixel set, the horizontal interpolation coefficient table, the vertical interpolation coefficient table, and other data sent by the first interpolation module and the second interpolation module. And, provide the data output function, for example when needing the longitudinal interpolation to calculate, output the interpolation result that two consecutive time points correspond to. Specifically, the input signals include PO _ arr1, PO _ arr2, PO _ arr3, PO _ arr4, PO _ cnt1, PO _ cnt2, PO _ cnt3, PO _ cnt4, vaild _ d1, vaild _ d2, vaild _ d3, vaild _ d4, and Row _ flag; wherein PO _ arr1, PO _ arr2, PO _ arr3 and PO _ arr4 represent pixel arrays of four transverse interpolation results, namely transverse interpolation arrays, PO _ cnt1, PO _ cnt2, PO _ cnt3 and PO _ cnt4 represent effective pixel numbers of the pixel arrays of the four transverse interpolation results, vaild _ d1, vaild _ d2, vaild _ d3 and vaild _ d4 represent the effectiveness of the transverse interpolation results, and Row _ flag represents a Row flag, which contains information of a Row feed position and a Row number; the output signals include 2P _arr1, 2P _arr2, 2P _arr3, 2P _uarr4, 2P _cnt1, 2P _cnt2, 2P _cnt3, 2P _cnt4, row1, row2, row3, row4, vaild _ d21, vaild _ d22, vaild _ d23, vaild _ d24, row. 2p _arr1, 2p _arr2, 2p _arr3and 2p _arr4represent two rows of pixel arrays generated by four horizontal interpolation modules, 2p _cnt1, 2p _cnt2, 2p _cnt3 and 2p _cnt4 represent the effective numbers of the four pixel arrays respectively, row1, row2, row3 and Row4 represent Row numbers of the relative current Row, namely relative Row numbers as described in the above embodiment, and are obtained by Row _ flag analysis and used for selecting a corresponding vertical interpolation coefficient from the vertical interpolation coefficient array, ilvad _ d21, vaild _ d22, vaild _ d23 and vaild _ d24 represent enable signals and Row number information of the four vertical interpolation modules respectively, row number represents the current interpolation start Row number, num represents the Row number participating in interpolation and is used for obtaining a vertical interpolation coefficient, and the effective vertical interpolation coefficient of m of each Row is obtained from the Row.

The internal structure of the data cache management module 740 is shown in fig. 14, and includes a line mark parsing sub-module 742, four interpolation result cache sub-modules 744, and four interpolation array combination sub-modules 746. The Row flag parsing sub-module 742 is configured to parse the Row _ flag signal to obtain four relative Row numbers and a current interpolation Row number, set the relative Row numbers of the four cache management modules according to the four relative Row numbers, and calculate the relative Row numbers by the previous module to obtain a longitudinal interpolation coefficient for selection; the interpolation result buffer sub-module 744 is configured to buffer two lines of interpolation results and read a previous line of interpolation results, the internal structure is as shown in fig. 15, vaild represents an effective enable control signal, cnt and 2p \ucnt represent the number of interpolation results, arr and P _ arr represent a horizontal interpolation result array, W _ addr is a write address, R _ addr is a read address, and Prew row data represents previous line of interpolation result data corresponding to a current position, and includes a synchronization processing sub-module 726, a buffer sub-module 750, a buffer write sub-module 752, a ping-pong processing sub-module 754 where vertical interpolation data and horizontal line interpolation results are written, and a buffer read sub-module 756, where the synchronization processing sub-module 726 is configured to synchronize an effective signal vaild; the cache submodule 750 is configured to cache two lines of horizontal interpolation results in a cache area, and the ping-pong processing control module that writes in the vertical interpolation data and the horizontal line interpolation results is configured to operate one memory when reading data, and write in the other memory, and finally output the horizontal interpolation results of the current position pixel and the previous row position pixel; the interpolation array combination submodule 746 is configured to combine 2 horizontal interpolation arrays from three horizontal interpolation arrays according to the data input mode to obtain a target interpolation array for output, when the data input mode is point-by-point and line-by-line input, combine the interpolation array P _ arr (n) of the current line and the interpolation array Prev _ arr (n) of the line pixel input at the previous time, when the data input mode is 4 line-by-point and line-by-line input, combine the interpolation array P _ arr (n-1) of the previous line and the interpolation array P _ arr (n) of the current line, and because the horizontal interpolation result may be buffered in different Inter caches, the first vertical interpolation core needs to perform vertical interpolation processing in combination with the interpolation result obtained at the previous time.

The vertical interpolation module 730 is used for performing vertical interpolation calculation on the interpolation array obtained by the previous horizontal interpolation, the input signals include 2P _arr1, 2P _arr2, 2P _arr3, 2P _arr4, 2P _cnt1, 2P _cnt2, 2P _cnt3, 2P _cnt4, vaild _ d21, vaild _ d22, vaild _ d23, vaild _ d24, the meanings of which are given in the above description of the cache management module, and will not be described in detail herein. In addition, two inputs, namely Row _ coef _ arr and Row _ coef _ cnt, represent the number of longitudinal interpolation coefficient arrays and arrays of the current interpolation line and the next interpolation line, and Row1, row2, row3 and Row4 represent the interpolation ordinate positions of four longitudinal interpolation modules; the output signal includes: 2p _, u _, arr1, 2p _, u arr2, 2p _, u _, arr3, 2p _, u _, arr4, 2p _, u _, cnt1, 2p _, u _, cnt2, 2p _, u _, cnt3, 2p _, u _, cnt4, vaild _ d31, vaild _ d32, vaild _ d33, vaild _ d34, where 2p _, u _, arr2, 2p _, u _, arr3, 2p _, u _, arr4 represent the interpolation result arrays corresponding to adjacent pixel sets output by the four vertical interpolation modules, respectively, and 2p _, u _, cnt1, 2p _, cnt2, 2p _, 2p, cnt3, and 2p _, u _, and alut _, and 4 represent the effective interpolation result arrays corresponding to the adjacent pixel sets of the four sets, respectively, and the effective interpolation result arrays of the adjacent pixel sets 32, and the effective interpolation signal arrays.

As shown in fig. 16, the internal structure of the vertical interpolation module includes an interpolation core control sub-module 732, configured to control enabling of m linear interpolation cores 824 (e.g., vertical interpolation cores) according to the number P _ out _ count of input start cores, where, of the m linear interpolation cores, only the first P _ out _ count cores are in an effective working state; a longitudinal interpolation coefficient number parameter splitting sub-module 734, configured to split the two rows of longitudinal interpolation coefficient number parameters; the longitudinal interpolation coefficient array splitting submodule 736 is configured to split two rows of longitudinal interpolation coefficient arrays, and select and distribute the split two rows of interpolation coefficient arrays and the number of interpolation coefficients to m linear interpolation cores according to an input row signal, where a selection mode is that a first row is selected when row =0, and a second row is selected when row = 1; an interpolation array partitioning submodule 738 for partitioning the interpolation array into m pairs of interpolation pixels; the functions of the vertical interpolation module are as described above, the data merging sub-module 729 is configured to merge all valid pixels output by the valid linear interpolation cores for output, and the synchronization processing sub-module 726 is configured to perform synchronization processing, so as to match the valid enable valid signal with the output P _ out _ arr and P _ out _ count signals.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the image data processing method described above when executing the computer program.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, realizes the steps in the above-mentioned image data processing method.

In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, carries out the steps in the image data processing method described above.

It should be noted that, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), magnetic Random Access Memory (MRAM), ferroelectric Random Access Memory (FRAM), phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), for example. The databases referred to in various embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method of processing image data, the method comprising:

receiving image data to be processed, wherein the image data to be processed comprises a pixel set which is input at the current time point and extracted from an image to be processed according to a selected data input mode, and the pixel set consists of a plurality of continuous pixels in a first pixel arrangement direction;

according to a first pixel arrangement direction, carrying out parallel interpolation processing on the pixel set to obtain a current interpolation result corresponding to the pixel set, wherein the element number of the current interpolation result is linearly related to the element number of the pixel set;

2. The image data processing method according to claim 1, wherein the parallel interpolation processing on the pixel set includes:

acquiring position information of each pixel point in the pixel set;

according to the position information of each pixel point, correspondingly acquiring a plurality of first-type interpolation coefficient arrays from a preset first-type interpolation coefficient table;

according to a plurality of first type interpolation coefficient arrays, carrying out parallel interpolation processing on the pixel set;

the first-type interpolation coefficient table comprises a mapping relation between position information of a pixel to be interpolated and a first-type interpolation coefficient array, and the number of interpolation coefficients in the first-type interpolation coefficient array is determined based on the position information of each pixel point in the pixel set.

3. The image data processing method according to claim 2, wherein said performing parallel interpolation processing on the pixel set according to a plurality of first-type interpolation coefficient arrays includes:

4. The image data processing method according to claim 1, wherein said parallel interpolation processing on the last interpolation result and the current interpolation result includes:

according to a plurality of second type interpolation coefficient arrays, performing parallel interpolation processing on the previous interpolation result and the current interpolation result;

the second-type interpolation coefficient table comprises a mapping relation between position information of pixels to be interpolated and a second-type interpolation coefficient array, and the number of interpolation coefficients in the second-type interpolation coefficient array is determined based on the position information of each pixel point in the pixel set.

5. The image data processing method according to claim 4, wherein the previous interpolation result and the current interpolation result include a plurality of interpolation arrays;

the parallel interpolation processing of the previous interpolation result and the current interpolation result according to a plurality of second-type interpolation coefficient arrays comprises:

sequentially and correspondingly extracting a first interpolation array of the previous interpolation result and a second interpolation array of the current interpolation result, and combining the first interpolation array and the second interpolation array to obtain a plurality of target interpolation arrays;

6. The image data processing method according to any one of claims 1 to 5, wherein before the parallel interpolation processing is performed on the pixel set, the method further includes:

the parallel interpolation processing of the pixel set comprises:

and carrying out parallel interpolation processing on the effective pixel set.

7. The image data processing method according to any one of claims 1 to 5, wherein before the parallel interpolation processing on the pixel set, the method further includes:

the parallel interpolation processing of the pixel set comprises:

and stopping the built-in first type interpolation processing core which has a corresponding relation with the invalid pixel point so as to enable the remaining first type interpolation processing cores to perform parallel interpolation processing on the valid pixel point.

8. The image data processing method according to any one of claims 1 to 5, wherein the data input mode includes a point-by-point and line-by-line input mode, a point-by-point and column-by-column input mode, a multi-row and point-by-point input mode, or a multi-column and point-by-point input mode;

if the data input mode is a multi-column point-by-point input mode, the pixel set consists of a plurality of pixels with the same row number and continuous column numbers.

9. The image data processing method according to claim 6, wherein the data input mode includes a point-by-point and row-by-row input mode, a point-by-point and column-by-column input mode, a multi-row and point-by-point input mode, or a multi-column and point-by-point input mode, and the image data to be processed includes an image scaling ratio, and row position information and/or column position information of each pixel in the pixel set;

screening out effective pixel points to be interpolated in the pixel set comprises the following steps:

if the data input mode is a point-by-point and line-by-line input mode, screening effective pixel points to be interpolated in the pixel set according to the image scaling and the line position information of each pixel in the pixel set;

and if the data input mode is a multi-row point-by-point input mode or a multi-column point-by-point input mode, screening effective pixel points to be interpolated in the pixel set according to the image scaling, the row position information and the column position information.

10. An image data processing apparatus, characterized in that the apparatus comprises:

the data receiving module is used for receiving image data to be processed, wherein the image data to be processed comprises a pixel set which is input at the current time point and extracted from an image to be processed according to a selected data input mode, and the pixel set consists of a plurality of continuous pixels positioned in the first pixel arrangement direction;

the first parallel interpolation processing module is used for performing parallel interpolation processing on the pixel set according to a first pixel arrangement direction to obtain a current interpolation result corresponding to the pixel set and caching the current interpolation result, wherein the element number of the current interpolation result is linearly related to the element number of the pixel set;

a second parallel interpolation processing module, configured to perform parallel interpolation processing on a previous interpolation result of the current interpolation result and the current interpolation result according to a second pixel arrangement direction when the previous interpolation result of the current interpolation result exists in the cache data, to obtain an interpolation result corresponding to an adjacent pixel set, where the previous interpolation result is obtained by performing parallel interpolation processing on a pixel set received at a previous time point, and the first pixel arrangement direction is perpendicular to the second pixel arrangement direction;

11. An image data processing device is characterized by comprising a data screening module, a first type interpolation module, a second type interpolation module, a data cache management module and a data merging module;

the data screening module receives a pixel set which is input at a current time point and extracted from an image to be processed according to a selected data input mode, the pixel set is input to the first type interpolation module, the first type interpolation module performs parallel interpolation processing on the received pixel set according to a first pixel arrangement direction to obtain a current interpolation result corresponding to the pixel set, the current interpolation result is sent to the data cache management module for caching, when the data cache management module detects that a last interpolation result of the current interpolation result exists in cache data, the current interpolation result and the last interpolation result of the current interpolation result are sent to the second type interpolation module, the second type interpolation module performs parallel interpolation processing on the last interpolation result and the current interpolation result according to a second pixel arrangement direction to obtain an interpolation result corresponding to an adjacent pixel set, the interpolation results corresponding to the adjacent pixel set are output to the data merging module, the data merging module merges the interpolation results corresponding to each group of adjacent pixel sets of the image to be processed, and outputs merged data of the image sets after the interpolation processing corresponding to each group of the adjacent pixel sets of the image to be processed are merged, and the image data merging processing data is output;

the pixel set is composed of a plurality of continuous pixels located in a first pixel arrangement direction, the number of elements of the current interpolation result is linearly related to the number of elements of the pixel set, the previous interpolation result is obtained by performing parallel interpolation processing on the pixel set received at the previous time point, and the first pixel arrangement direction is perpendicular to the second pixel arrangement direction.

12. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any one of claims 1 to 9 when executing the computer program.

13. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 9.

14. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 9 when executed by a processor.