CN113096002A - Video frame scale transformation heterogeneous system and method and storage medium - Google Patents
Video frame scale transformation heterogeneous system and method and storage medium Download PDFInfo
- Publication number
- CN113096002A CN113096002A CN202110362204.XA CN202110362204A CN113096002A CN 113096002 A CN113096002 A CN 113096002A CN 202110362204 A CN202110362204 A CN 202110362204A CN 113096002 A CN113096002 A CN 113096002A
- Authority
- CN
- China
- Prior art keywords
- list
- weight
- window size
- scale
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000009466 transformation Effects 0.000 title claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 101
- 238000004364 calculation method Methods 0.000 claims abstract description 43
- 238000006243 chemical reaction Methods 0.000 claims description 17
- 230000008569 process Effects 0.000 abstract description 24
- 238000004422 calculation algorithm Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 10
- 238000011161 development Methods 0.000 description 8
- 238000009825 accumulation Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013178 mathematical model Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- ATHVAWFAEPLPPQ-VRDBWYNSSA-N 1-stearoyl-2-oleoyl-sn-glycero-3-phosphocholine Chemical compound CCCCCCCCCCCCCCCCCC(=O)OC[C@H](COP([O-])(=O)OCC[N+](C)(C)C)OC(=O)CCCCCCC\C=C/CCCCCCCC ATHVAWFAEPLPPQ-VRDBWYNSSA-N 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G06T3/02—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/60—Memory management
-
- G06T3/04—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Abstract
The invention relates to the technical field of image processing, in particular to a video frame scale transformation heterogeneous system and method and a storage medium. Wherein the system includes: the system comprises a processor unit and a plurality of logic processing units connected with the processor unit, wherein the logic processing units are connected in parallel; the processor unit is used for calculating the weight and the window size during transformation according to the scale of the source image and the target scale so as to obtain a weight list and a window size list; and the plurality of logic processing units are used for carrying out size transformation on the source image according to the weight list and the window size list so as to obtain a target image with a target scale. Because the weight calculation and the window size comprise a large amount of floating point data or fixed point data, the part of the processing process is put in a processor unit with rich calculation capacity, and because the number of pixel points is large, the pixel processing process is put in a plurality of logic processing units which are arranged in parallel, the overall processing efficiency of the system is improved, and the hardware cost is low.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a video frame scale transformation heterogeneous system and method and a storage medium.
Background
The scaling of image and video frame data is a basic problem in multimedia applications, generally used is that independent processing is divided into pure software processing, the pure software processing mode is mainly established on a CPU and an arm machine of an x86 or other systems, and the scaling function of a video image is realized by installing open source software opencv, ffmpeg and the like, so that a simple and many schemes based on the same development are developed, but the performance processing and the flexibility cannot meet the requirements, for example, under the condition that one-way multiplexing or multi-way multiplexing is processed in real time by the video image, the resources of the CPU are greatly consumed to reduce the stability of the whole machine, so the processing efficiency is low, and if the requirement of the processing efficiency is met by replacing the CPU, the product cost is increased.
Disclosure of Invention
The invention mainly solves the technical problem that the efficiency cannot be ensured and the cost cannot be reduced simultaneously when the existing system adopting pure software is used for converting the video frame scale.
A video frame scale transform heterogeneous system, comprising: the system comprises a processor unit and a plurality of logic processing units connected with the processor unit, wherein the logic processing units are connected in parallel;
the processor unit is used for calculating the weight and the window size during transformation according to the scale of the source image and the target scale to obtain a weight list and a window size list;
and the plurality of logic processing units are used for carrying out size transformation on the source image according to the weight list and the window size list so as to obtain a target image with a target scale.
Wherein the processor unit comprises a weight calculation module and a window size calculation module;
the weight calculation module is used for calculating the weight corresponding to each pixel in the target image during transformation according to the scale of the source image and the target scale to obtain the weight list; and the window size calculation module is used for calculating the window size corresponding to each pixel during transformation according to the scale of the source image and the target scale to obtain the window size list.
The plurality of logic processing units simultaneously convert a plurality of pixels in the source image according to the weight list and the window size list to obtain a plurality of target pixels;
the video frame scale conversion heterogeneous system further comprises a cache module, and the cache module is used for caching the target pixels to obtain a target image.
And the logic processing unit comprises a scale conversion module which is used for converting the pixels in the source image according to the weight list and the window size list to obtain the target pixels in the target image.
Further, the processor unit includes a storage module, and the storage module is configured to store the weight list and the window size list, and send a storage address to the logic processing unit.
Furthermore, the logic processing unit further comprises a reading judgment module; the reading judgment module is used for reading the weight list and the window size list from the storage module, judging whether the weight list and the window size list matched with the current target image exist in the storage module, and if so, selecting the weight list and the window size list matched with the current target image;
if not, the scale of the source image and the target scale are sent to the processor unit, the processor unit calculates according to the scale of the source image and the target scale to obtain a weight list and a window size list, then the weight list and the window size list are stored in the storage module, and storage addresses of the weight list and the window size list are returned to the reading judgment module.
A method of video frame scaling, comprising:
calculating the weight and the window size during transformation by adopting a processor unit according to the scale of the source image and the target scale to obtain a weight list and a window size list;
and carrying out size transformation on the source image by adopting a plurality of logic processing units arranged in parallel according to the weight list and the window size list to obtain a target image with a target scale.
Wherein the processor unit comprises a weight calculation module and a window size calculation module;
the calculating the weight and the window size during transformation according to the scale of the source image and the target scale to obtain a weight list and a window size list comprises: adopting a weight calculation module to calculate the weight corresponding to each pixel in the target image during transformation according to the scale of the source image and the target scale to obtain a weight list; and calculating the window size corresponding to each pixel during transformation according to the scale of the source image and the target scale by adopting a window size calculation module to obtain the window size list.
The multiple parallel logic processing units are adopted to simultaneously transform multiple pixels in a source image according to the weight list and the window size list to obtain multiple target pixels;
and caching the target pixels to obtain a target image.
A computer-readable storage medium comprising a program executable by a processor to implement the video frame scaling method as described above.
The video frame scaling heterogeneous system according to the above embodiment includes: the system comprises a processor unit and a plurality of logic processing units connected with the processor unit, wherein the logic processing units are connected in parallel; the processor unit is used for calculating the weight and the window size during transformation according to the scale of the source image and the target scale to obtain a weight list and a window size list; and the plurality of logic processing units are used for carrying out size transformation on the source image according to the weight list and the window size list so as to obtain a target image with a target scale. Because the weight calculation and the window size include a large amount of floating point or fixed point data, the part of the processing process is placed in a processor unit with rich computing capability, such as a CPU or an arm, and because the pixel processing process is mostly multiplication and summation operation, but the number of pixel points is large, the pixel processing process is placed in a plurality of logic processing units which are arranged in parallel, such as an FPGA (field programmable gate array) for parallel processing, so that a plurality of pixels can be processed simultaneously, the overall processing efficiency of the system is improved, and the hardware cost is low.
Drawings
FIG. 1 is a diagram of a video frame scaling system according to the prior art;
fig. 2 is a schematic diagram of an overall structure of a video frame scaling heterogeneous system according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of a video frame scaling heterogeneous system according to an embodiment of the present application;
FIG. 4 is a general flowchart of a video frame scaling method according to an embodiment of the present application;
fig. 5 is a detailed flowchart of a video frame scaling method according to an embodiment of the present application.
Detailed Description
The present invention will be described in further detail with reference to the following detailed description and accompanying drawings. Wherein like elements in different embodiments are numbered with like associated elements. In the following description, numerous details are set forth in order to provide a better understanding of the present application. However, those skilled in the art will readily recognize that some of the features may be omitted or replaced with other elements, materials, methods in different instances. In some instances, certain operations related to the present application have not been shown or described in detail in order to avoid obscuring the core of the present application from excessive description, and it is not necessary for those skilled in the art to describe these operations in detail, so that they may be fully understood from the description in the specification and the general knowledge in the art.
Furthermore, the features, operations, or characteristics described in the specification may be combined in any suitable manner to form various embodiments. Also, the various steps or actions in the method descriptions may be transposed or transposed in order, as will be apparent to one of ordinary skill in the art. Thus, the various sequences in the specification and drawings are for the purpose of describing certain embodiments only and are not intended to imply a required sequence unless otherwise indicated where such sequence must be followed.
Referring to fig. 1, fig. 1 shows that a conventional pure software method is used to convert a source image from one path to multiple paths, for example, 1080P is converted into 8 paths of 720P, 8 paths of 540P, and 8 paths of 360P target images, if all processing processes are performed in the existing software, the main frequency of the CPU needs to be increased, but the main frequency of the CPU has already reached a bottleneck, so that the efficiency is slow when pure software is used for processing, and even when the CPU with the strongest processing capability is used at present, the requirement for image processing cannot be met. If the pure hardware mode is adopted, the function of the scale conversion is directly processed by adopting an asic (special application integrated circuit) or an fpga (logic processing circuit), although the pure hardware design can easily meet the performance requirement and flexibly solve different schemes, the development is complex, the period is long, a large amount of time is needed to establish a verification mathematical model in the early stage, and then the method can be realized in hardware. Particularly, if the details required by the video image are rich, a more complex conversion method is required, and more mathematical operations are often required, such as bicubic algorithm (bicubic interpolation algorithm) with an indefinite length window, if hardware needs to calculate a large number of floating point numbers in weight implementation, the development cycle is long, and more resources are required.
In the embodiment of the invention, in the video frame scale conversion process, because the weight calculation and the window size comprise a large amount of floating point or fixed point data, the part of the processing process is put in a processor unit with rich computing capability, such as a CPU or an arm, because the pixel processing process is mostly multiplication and summation operation, but the number of pixel points is large, the pixel processing process is put in a plurality of logic processing units which are arranged in parallel, such as an FPGA (field programmable gate array) for parallel processing, so that a plurality of pixels can be processed simultaneously, the overall processing efficiency of the system is improved, and the hardware cost is low.
The first embodiment is as follows:
referring to fig. 1 and fig. 2, the present embodiment provides a video frame scaling heterogeneous system, which includes: a processor unit 10 (i.e. a software processing unit) and a plurality of logical processing units 20 (i.e. hardware processing units) connected to the processor unit 10, the plurality of logical processing units 20 being connected in parallel, such that a source image (i.e. an image to be transformed) can be processed simultaneously in parallel.
The processor unit 10 is configured to calculate weights and window sizes during transformation according to the scale of the source image and the target scale to obtain a weight list and a window size list; the plurality of logical processing units 20 are configured to perform a size transformation on the source image according to the weight list and the window size list to obtain a target image having a target dimension. Because the weight calculation and the window size include a large amount of floating point or fixed point data, the part of the processing process is placed in a processor unit with rich computing capability, such as a CPU or an arm, because the pixel processing process is mostly multiplication and summation operation, but the number of pixel points is large, the pixel processing process is placed in a plurality of logic processing units which are arranged in parallel, the real-time performance of the logic processing units is good, but the computing capability is weaker than that of the processor, but the parallel computing can improve the processing efficiency, such as placing in an FPGA (logic processing circuit) for parallel processing, so that a plurality of pixels can be processed simultaneously, the overall processing efficiency of the system is improved, and the hardware cost is low. Because the pure software and the pure hardware have respective superiority and respective limitation, in order to increase the flexibility of processing, reduce the development period, reduce the development cost and improve the performance, the heterogeneous system provided by the embodiment can realize the scale conversion of the image video frame by calculation, and the design can improve the performance, reduce the development period and reduce the development cost.
The heterogeneous system of the embodiment converts the image video frames by using a software platform with strong computing capability and large task resource tension and delay as well as a hardware platform with good data parallel processing, low cost and high portability. The software platform processes complex algorithms such as floating point and fixed point in the conversion process and provides contents such as parameters of hardware platform operators; the hardware platform processes data in parallel, partial parameters provided by the software platform are matched with the hardware operator to perform parallel processing, and a plurality of hardware platforms of one software platform can be matched. The software platform is divided into a CPU, an ARM processor, a DSP, an SOPC and a MiscrOBlaze under a server of x86/ARM and the like; the hardware platform can be selected from: any one of GPU, FPGA, ASIC.
Specifically, as shown in fig. 3, the processor unit 10 includes a weight calculation module 101 and a window size calculation module 102; the window size calculation module 102 is configured to calculate a window size corresponding to each pixel during transformation according to the scale of the source image and the target scale, and obtain a window size list. Specifically, the weight calculation module 101 is configured to calculate a weight corresponding to each pixel in the target image during transformation according to the scale of the source image, the target scale, and the window size, so as to obtain a weight list.
Wherein, the plurality of logic processing units 20 transform a plurality of pixels in the source image according to the weight list and the window size list to obtain a plurality of target pixels; further, the video frame scale conversion heterogeneous system of the embodiment further includes a cache module, where the cache module is configured to cache a plurality of target pixels to obtain a target image.
Specifically, the logic processing unit 20 of this embodiment includes a scaling module, and the scaling module is configured to transform pixels in the source image according to the weight list and the window size list to obtain target pixels in the target image.
Wherein the processor unit 10 comprises a storage module 103, the storage module 103 is configured to store the corresponding weight according to the window size. A target pixel needs a window and contains a plurality of weights, the source resolution and the target resolution are determined, all pixels are of the window size of 6N with the fixed length, and the storage address is sent to the logic processing unit, so that the logic processing unit can read the weight list and the window size list from the storage address when processing the source image pixel.
In the actual processing, the size of the source target image is generally directly sent to the logic processing unit 20, and if the type of source image is processed before the logic processing unit 20, for example, 1080P is converted into 720P, and the target of this conversion is the same as the source resolution, the originally stored weight list and window size list are directly read for processing without recalculating the weight list and window size list, so that the weight is irrelevant to the feature information of the source image itself in the image scale conversion process and is only relevant to the scale.
In another embodiment, the logic processing unit further comprises a reading judgment module; the reading judgment module is used for reading all the weight lists and window size lists from the storage module 103, judging whether the storage module has a weight list and a window size list which are matched with the current target image, and if so, selecting the weight list and the window size list which are matched with the current target image; if not, the scale of the source image and the target scale are sent to the weight calculation module 101 and the window size calculation module 102 of the processor unit 10, after the weight calculation module 101 and the window size calculation module 102 of the processor unit calculate the weight list and the window size list according to the scale of the source image and the target scale, the weight list and the window size list are stored in a new storage address of the storage module, and the new storage address of the weight list and the window size list is returned to the reading judgment module. And the reading and judging module reads the new weight list and the new window size list and then processes the pixels.
In this embodiment, an ARM + FPGA platform is used, and a large number of floating point or fixed point and other mathematical operation parts are placed on an ARM (soft core or hard core) with rich computing power, and the ARM + FPGA platform can be used for parallel real-time processing and placed in the FPGA. The conversion algorithm of bicubic interpolation (bicubic interpolation) with variable length has higher image quality in detail than other conversion algorithms, so that a more complicated mathematical model is needed, a window with 6 × scaling times is needed for generating each target image point, and the window length and the required weight are not fixed due to the fact that the scaling target resolution is not fixed。Dividing the position of the pixel point of the target image by the zoom factor to be equal to the position of the pixel point of the corresponding source image, adding and subtracting the zoom factor of 3 multiplied by taking the position as the center to obtain the positions of all the pixel points of the window, and obtaining the sum of the weight of the column and the weight sum of the row as the target pixel of the point. For example, pixels (height 27, width 10) at the 27 th row and 10 th column points of the target image, a high zoom factor of 3, a window size of 18, and a wide zoom factor of 18The multiple is 2, the window size is 12, and since the data of the DDR cache is stored row by row, the logic processing units process row by row and then column by column. First, the corresponding high position of the target image is calculated to be 27/3 ═ 9, resulting in a high window of [1,2,3,4,5,6,7,8,9,, 18]10/2-5, and the size of the wide window is [ -1,0,1,2,3,4,5,6,7,8,9,10,11]Since there is no-1 location in the source, less than 0 in the target is all left unprocessed. The line processing module firstly processes the first line with effective line, the summation of all the wide data in the window and the weight obtains the value after line conversion multiplied by the weight of the line, the value is cached in the RAM according to the position of the pixel point, the sum of the weights of the corresponding windows of other target pixels of the line is continuously processed until the line processing is completed and the obtained weight sum is completely cached in the RAM, the processing of the second line is continuously carried out, and the same calculation is carried out for [ -1,0,1,2,3,4,5,6,7,8,9,10,11]The sum of the weights in the window, multiplied by the weight in row 2, is then summed with the value from row one and buffered in that location, and the sum of the weights for all other target pixels in row two is continued until the sum of the wide weight of row window 18 multiplied by the row weight and the sum of the weights from row one adds up to the pixel value in column 27, row 10 of the target image. Specifically, the method is implemented in hardware, and is implemented according to a pipeline design, 8 pixel points are read at one time, calculation of 8 windows at most is participated, weight sums are calculated in corresponding windows respectively, each pixel point obtains an initial position according to the central position of an obtained source image pixel point, 6N weights are read from the initial position, all source pixels in the window are multiplied by the weights to obtain a sum, the sum is cached in an RAM, calculation of other target pixel points is continued, the size of the line window is simultaneously counted, a value of the size of one window of the height is met, the value is directly output as a target pixel value, and the obtained target pixel value is a continuous pixel value of a target image because scale change is converted according to the position of the target image.
If pure software is adopted, the api interface function of opencv/ffmpeg can be conveniently called to realize the processing, but great requirements are made on a software processing platform during multi-path real-time processing; if the FPGA/ASIC is adopted for processing, all the scaling conditions cannot be completely stored due to the fact that the table look-up mode is adopted for weight change; and real-time calculation is adopted, namely: if the weight needs to be calculated every time the source comes, the resource consumption and the operation delay are increased, and the whole parallel processing speed is affected. The heterogeneous computing resize is adopted to combine the optimal performance of pure software and pure hardware, the weight operation is realized through the ARM, the FPGA reads the obtained weight from the DDR, and then the weight is distributed to each pixel point to perform parallel computing, so that the processing efficiency is improved.
The Bicubic algorithm divides the process into two steps of row transformation and column transformation due to small cache of the FPGA and consideration of running clocks, and defines all pixel points in the beginning and the end by using a window. The algorithm flow is accumulation of pixel multiplied by weight sum, if the first line is the line conversion value of the first line after waiting for the first line to be finished, the weight sum of two pixel points of a column is obtained by the subsequent accumulation with the weight sum of the second line, and the original cached value is covered by the cache. If not, the accumulated sum of the line pixel points is obtained, and then the accumulated sum is multiplied by the values of the cached line pixel points through respective weights to obtain the value which is covered by the accumulated sum and the cached line pixel points, so that the size of the column cache is reduced, the final result is not obtained until all window data of the column cache are met, and the whole conversion time is prolonged.
Example two:
referring to fig. 4 and fig. 5, the present embodiment provides a video frame scaling method, which includes:
step 201: and calculating the weight and the window size during transformation by adopting a processor unit according to the scale of the source image and the target scale to obtain a weight list and a window size list.
Step 202: and carrying out size transformation on the source image by adopting a plurality of logic processing units arranged in parallel according to the weight list and the window size list to obtain a target image with a target scale.
The processor unit comprises a weight calculation module and a window size calculation module. Calculating the weight and the window size during transformation according to the scale of the source image and the target scale to obtain a weight list and a window size list, wherein the weight list and the window size list comprise: adopting a weight calculation module to calculate the weight corresponding to each pixel in the target image during transformation according to the scale of the source image and the target scale to obtain a weight list; and calculating the window size corresponding to each pixel during transformation according to the scale of the source image and the target scale by adopting a window size calculation module to obtain the window size list.
A plurality of logic processing units which are arranged in parallel are adopted to simultaneously transform a plurality of pixels in a source image according to the weight list and the window size list to obtain a plurality of target pixels; and caching the target pixels to obtain a target image.
Specifically, after the source data and the parameters such as the corresponding resolution enter the system of the first embodiment for processing, the obtained source image resolution and target image resolution are transmitted to the reading judgment module. That is, once the request is valid, the resolution of the source image and the resolution of the target image are transmitted to weight matching, the reading judgment module checks whether generated weight data exist or not, if the generated weight data exist, the weight cache address is directly transmitted back to the reading judgment module, the resolution of the source image and the resolution of the target image are not transmitted to the ARM, after the ARM receives new weight calculation, the weight of each pixel point is calculated according to a weight calculation formula of a bicubic algorithm, the bicubic starting point of one target pixel is (x +0.5) N-3N +0.5, the end point (x +0.5) N +3N +0.5 is the end point of the seventh target pixel, so that 6 windows exist between the first target pixel and the seventh target pixel, the corresponding pixel is repeatedly calculated for 6 times at most, but each pixel has different weights in different windows, each original pixel has a maximum of 6 weights. After the ARM calculates the weight of the source image pixel, the source image pixel is stored in different addresses according to YUV (color coding method), operation positions in different windows and window sequences, the cached DDR address is transmitted to a weight calculation module, the weight calculation module updates a current weight table in a matching mode, and returns to an address reading judgment module cached in a weight list, and the reading judgment module is transmitted to a logic processing unit for processing.
As shown in fig. 5, a specific process of the video frame scale transformation heterogeneous method of the present embodiment includes:
301, obtaining the scale of the source image and the scale of the target image.
Step 302: and calculating the weight size and the window size according to the scale of the source image and the scale of the target image to obtain a weight list and a window size list, and storing the weight list and the window size list in the DDR. In other words, the process can be understood as: after a source image or video frame data is sent to a processing system implementing the first embodiment, firstly, cache addresses are distributed to the data, the data are cached in a DDR, meanwhile, a target resolution of the source image is sent to a reading judgment module, and after the reading judgment module receives the cached weight and the DDR source data caching is completed, a logic processing unit is started to start parallel processing.
Step 303: the weight list and the window size list are read from the memory address. The specific process is as follows: and reading the weight list from the DDR, caching the weight list into a parallel RAM, and then reading source image data from the DDR.
Step 304: and the plurality of logic processing units arranged in parallel carry out size transformation on the source image according to the weight list and the window size list. Specifically, source image data is subjected to line parallel processing by 8 bytes at 1 clock, 8 parallel logic processing units extract the starting point and the end point of a line and pixel values in a window from the source image, and pixels are subjected to weight multiplication and accumulation according to the position of the line window to obtain line transformation of a target point.
Step 305: and caching the transformed data of each line to a RAM.
Step 306: and performing row processing on the data cached in the current row and the previous row. For example, the data after the first line transformation is buffered, and the accumulated sum of the multiplication of the pixel point obtained after the line corresponding position transformation in the second line transformation and the previous line buffered point with respect to the column weight is obtained, so that the data buffered in the column only needs one line, and the column transformation is started without the data of all the column windows being prepared.
Step 307: and accumulating the data after the column processing to obtain a target image. After the data weight accumulation sum of the row window is obtained, the row weight accumulation sum can be carried out with the data of the row buffer memory until all the data of the column window are finished, so that the target image is obtained.
The method has the characteristics of integrating the advantages of software and hardware, extracting floating point and fixed point operation parts in a transformation algorithm and the like to be processed on a software platform, carrying out pipeline parallel processing on data to be processed in a hardware platform, improving the performance of the algorithm by reducing time delay brought by hardware operation amount, realizing part of functions in the software platform, reducing algorithm development period, integrating heterogeneous calculation of the software platform and the hardware platform, and realizing different requirements of flexible design of multiple pieces of hardware in one piece of software.
EXAMPLE III
The present embodiment provides a computer-readable storage medium containing a program executable by a processor to implement the video frame scaling method as provided in the second embodiment.
Those skilled in the art will appreciate that all or part of the functions of the various methods in the above embodiments may be implemented by hardware, or may be implemented by computer programs. When all or part of the functions of the above embodiments are implemented by a computer program, the program may be stored in a computer-readable storage medium, and the storage medium may include: a read only memory, a random access memory, a magnetic disk, an optical disk, a hard disk, etc., and the program is executed by a computer to realize the above functions. For example, the program may be stored in a memory of the device, and when the program in the memory is executed by the processor, all or part of the functions described above may be implemented. In addition, when all or part of the functions in the above embodiments are implemented by a computer program, the program may be stored in a storage medium such as a server, another computer, a magnetic disk, an optical disk, a flash disk, or a removable hard disk, and may be downloaded or copied to a memory of a local device, or may be version-updated in a system of the local device, and when the program in the memory is executed by a processor, all or part of the functions in the above embodiments may be implemented.
The present invention has been described in terms of specific examples, which are provided to aid understanding of the invention and are not intended to be limiting. For a person skilled in the art to which the invention pertains, several simple deductions, modifications or substitutions may be made according to the idea of the invention.
Claims (10)
1. A video frame scale heterogeneous system, comprising: the system comprises a processor unit and a plurality of logic processing units connected with the processor unit, wherein the logic processing units are connected in parallel;
the processor unit is used for calculating the weight and the window size during transformation according to the scale of the source image and the target scale to obtain a weight list and a window size list;
and the plurality of logic processing units are used for carrying out size transformation on the source image according to the weight list and the window size list so as to obtain a target image with a target scale.
2. The video frame scale conversion heterogeneous system of claim 1, wherein the processor unit comprises a weight calculation module and a window size calculation module;
the weight calculation module is used for calculating the weight corresponding to each pixel in the target image during transformation according to the scale of the source image and the target scale to obtain the weight list; and the window size calculation module is used for calculating the window size corresponding to each pixel during transformation according to the scale of the source image and the target scale to obtain the window size list.
3. The video frame scaling heterogeneous system of claim 2, wherein the plurality of logical processing units simultaneously transform the plurality of pixels in the source image according to the weight list and the window size list to obtain a plurality of target pixels;
the video frame scale conversion heterogeneous system further comprises a cache module, and the cache module is used for caching the target pixels to obtain a target image.
4. The video frame scaling heterogeneous system of claim 3, wherein the logical processing unit comprises a scaling module to transform pixels in the source image according to the weight list and the window size list to obtain a target pixel in the target image.
5. The video frame scaling heterogeneous system of claim 4, wherein the processor unit comprises a storage module to store the weight list and/or window size list and to send a storage address to the logical processing unit.
6. The video frame scaling heterogeneous system of claim 5, wherein the logical processing unit further comprises a read determination module; the reading judgment module is used for reading the weight list and the window size list from the storage module, judging whether the weight list and the window size list matched with the current target image exist in the storage module, and if so, selecting the weight list and the window size list matched with the current target image;
if not, the scale of the source image and the target scale are sent to the processor unit, the processor unit calculates according to the scale of the source image and the target scale to obtain a weight list and a window size list, then the weight list and the window size list are stored in the storage module, and storage addresses of the weight list and the window size list are returned to the reading judgment module.
7. A method for video frame scaling, comprising:
calculating the weight and the window size during transformation by adopting a processor unit according to the scale of the source image and the target scale to obtain a weight list and a window size list;
and carrying out size transformation on the source image by adopting a plurality of logic processing units arranged in parallel according to the weight list and the window size list to obtain a target image with a target scale.
8. The video frame scaling method of claim 6, wherein the processor unit comprises a weight calculation module and a window size calculation module;
the calculating the weight and the window size during transformation according to the scale of the source image and the target scale to obtain a weight list and a window size list comprises: adopting a weight calculation module to calculate the weight corresponding to each pixel in the target image during transformation according to the scale of the source image and the target scale to obtain a weight list; and calculating the window size corresponding to each pixel during transformation according to the scale of the source image and the target scale by adopting a window size calculation module to obtain the window size list.
9. The method according to claim 7, wherein the plurality of logic processing units arranged in parallel are adopted to simultaneously transform a plurality of pixels in the source image according to the weight list and the window size list to obtain a plurality of target pixels;
and caching the target pixels to obtain a target image.
10. A computer-readable storage medium, characterized in that the storage medium comprises a program which is executable by a processor to implement the method according to any one of claims 7-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110362204.XA CN113096002A (en) | 2021-04-02 | 2021-04-02 | Video frame scale transformation heterogeneous system and method and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110362204.XA CN113096002A (en) | 2021-04-02 | 2021-04-02 | Video frame scale transformation heterogeneous system and method and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113096002A true CN113096002A (en) | 2021-07-09 |
Family
ID=76673401
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110362204.XA Pending CN113096002A (en) | 2021-04-02 | 2021-04-02 | Video frame scale transformation heterogeneous system and method and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113096002A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20100009288A (en) * | 2008-07-18 | 2010-01-27 | 엘지전자 주식회사 | Method for image-scaling using variable filter-window and calculating a size of filter-window for image-scaling |
| US8081816B1 (en) * | 2008-06-06 | 2011-12-20 | Kevin Maurice Irick | Apparatus and method for hardware implementation of object recognition from an image stream using artificial neural network |
| KR20150125026A (en) * | 2014-04-29 | 2015-11-09 | 경희대학교 산학협력단 | Fast face detection system using priority address allocation and moving window technique |
| CN105260475A (en) * | 2015-10-30 | 2016-01-20 | 努比亚技术有限公司 | Data searching method, data saving method and related equipment |
| CN105894494A (en) * | 2016-02-05 | 2016-08-24 | 中国科学院西安光学精密机械研究所 | Three-line-array stereo aerial survey camera parallel spectrum band registration method based on GPU technology |
| CN108171117A (en) * | 2017-12-05 | 2018-06-15 | 南京南瑞信息通信科技有限公司 | Electric power artificial intelligence visual analysis system based on multinuclear heterogeneous Computing |
-
2021
- 2021-04-02 CN CN202110362204.XA patent/CN113096002A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8081816B1 (en) * | 2008-06-06 | 2011-12-20 | Kevin Maurice Irick | Apparatus and method for hardware implementation of object recognition from an image stream using artificial neural network |
| KR20100009288A (en) * | 2008-07-18 | 2010-01-27 | 엘지전자 주식회사 | Method for image-scaling using variable filter-window and calculating a size of filter-window for image-scaling |
| KR20150125026A (en) * | 2014-04-29 | 2015-11-09 | 경희대학교 산학협력단 | Fast face detection system using priority address allocation and moving window technique |
| CN105260475A (en) * | 2015-10-30 | 2016-01-20 | 努比亚技术有限公司 | Data searching method, data saving method and related equipment |
| CN105894494A (en) * | 2016-02-05 | 2016-08-24 | 中国科学院西安光学精密机械研究所 | Three-line-array stereo aerial survey camera parallel spectrum band registration method based on GPU technology |
| CN108171117A (en) * | 2017-12-05 | 2018-06-15 | 南京南瑞信息通信科技有限公司 | Electric power artificial intelligence visual analysis system based on multinuclear heterogeneous Computing |
Non-Patent Citations (2)
| Title |
|---|
| 姚玉川: "《知识系统》", 30 September 1988, 大连理工大学出版社, pages: 228 - 229 * |
| 王平: ""基于双线性插值的图像缩放在GPU上的实现"", 《微电子学与计算机》, vol. 36, no. 2016, pages 1 - 4 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230305808A1 (en) | Accelerated mathematical engine | |
| JP4018601B2 (en) | Digital image scaling method for embedded system | |
| US8766992B2 (en) | Methods and apparatus for image processing at pixel rate | |
| US7580567B2 (en) | Method and apparatus for two dimensional image processing | |
| WO2013087003A1 (en) | Nonlocality-based super-resolution reconstruction method and device | |
| US7920150B2 (en) | Image scaling system capable of saving memory | |
| CN110519603B (en) | Hardware circuit for real-time video zooming and zooming method thereof | |
| CN106910162A (en) | Image zoom processing method and device based on FPGA | |
| CN112188236B (en) | Video interpolation frame model training method, video interpolation frame generation method and related device | |
| US20140092116A1 (en) | Wide dynamic range display | |
| CN116483587B (en) | Video super-division parallel method, server and medium based on image segmentation | |
| Cadenas et al. | Parallel pipelined array architectures for real-time histogram computation in consumer devices | |
| CN113096002A (en) | Video frame scale transformation heterogeneous system and method and storage medium | |
| CN116227599A (en) | Inference model optimization method and device, electronic equipment and storage medium | |
| CN115546373A (en) | NeRF renderer based on FPGA and multi-resolution Hash coding algorithm and rendering method | |
| CN112001492B (en) | Mixed running water type acceleration architecture and acceleration method for binary weight DenseNet model | |
| US11503297B2 (en) | Method and device for inverse quantization and inverse transformation and non-transitory computer-readable storage medium | |
| WO2021202470A1 (en) | Super-resolution of block-compressed texture for texture mapping applications | |
| CN111428560A (en) | Video image processing method and device, electronic equipment and storage medium | |
| WO2023072072A1 (en) | Blurred image generating method and apparatus, and network model training method and apparatus | |
| US20230237613A1 (en) | Method for generating metadata, image processing method, electronic device, and program product | |
| CN101727655B (en) | Image zooming method and device thereof | |
| CN116681588A (en) | Super-resolution implementation method and system | |
| KR101652264B1 (en) | Method for implementing gpu-parallel operating the edge-directed adaptive intra-field deinterlacing | |
| CN115955584A (en) | Image processing device, method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |