WO2020107319A1 - Image processing method and device, and video processor - Google Patents

Abstract

Provided by embodiments of the present application are an image processing method and device, and provided is a flexible raw image processing structure, facilitating the extension and authentication of an image processing architecture. The method comprises: reading image data, the read image data comprising a plurality of pixels, the plurality of pixels having at least one color component, and each pixel having one color component respectively; de-interleaving the read image data so as to obtain image data corresponding to each color component; storing the image data corresponding to each color component in a storage region corresponding to each color component in a storage space; and from within the storage region, executing a read operation for each color component respectively so as to obtain an image block of each color component.

Description

Image processing method and equipment, and video processor

Copyright statement

The content disclosed in this patent document contains material protected by copyright. The copyright is owned by the copyright owner. The copyright owner has no objection to anyone copying the patent document or the patent disclosure existing in the official records and archives of the Patent and Trademark Office.

Technical field

The present application relates to the field of image processing, and more specifically, to an image processing method and device, and a video processor.

Background technique

For raw images, a pixel has a color component, and the pixels of each color component are arranged in an interleaved manner, and in subsequent applications, the pixels of each color component need to be taken out separately for Handle separately. When processing the pixels of each color component, it needs to be processed according to the image block.

At present, the specific processing flow is to cache the image data in the cache unit, and then read the corresponding image data for deinterleaving according to the requirements of subsequent applications, and then send it to the block unit for block processing.

This scheme tightly couples the block dividing unit and the deinterleaving unit, and is related to each other. Therefore, a variety of possible situations need to be considered when processing. For example, if there are 4 de-interleaving schemes supported by an algorithm, and there are 6 possible block sizes at the same time, there are 4x6=24 cases in total to be dealt with. Therefore, this structure is neither flexible nor easy to expand and verify.

Summary of the invention

Embodiments of the present application provide an image processing method, device, and video processor, which correspond to a flexible raw image processing architecture, and facilitate the expansion and verification of the image processing architecture.

In a first aspect, an image processing method is provided, including: reading image data, wherein the read image data includes a plurality of pixels, the plurality of pixels has at least one color component, and each pixel has a A color component; deinterleaving the read image data to obtain image data corresponding to each color component; storing the image data corresponding to each color component into the storage space separately for each color component In the corresponding storage area; from the storage area, for each color component, a read operation is separately performed to obtain an image block of each color component.

According to a second aspect, there is provided an image processing apparatus including: a first reading unit for reading image data, wherein the read image data includes a plurality of pixels, the plurality of pixels having at least one color Component, and each pixel has a color component; a deinterleaving unit, used to deinterleave the read image data to obtain image data corresponding to each color component; a storage unit, used to convert the The image data corresponding to each color component is respectively stored in the storage area corresponding to each color component in the storage space; the second reading unit is used to execute separately for each color component from the storage area Reading operation to obtain the image block of each color component.

In a third aspect, there is provided an image processing apparatus, including a processing circuit, the processing unit configured to perform the method in the first aspect.

In a fourth aspect, a video processor is provided, including the image processing device described in the second aspect or the third aspect.

In the embodiment of the present application, storage of image data is added between deinterleaving and segmentation, and the image data corresponding to each color component is stored in the storage area corresponding to each color component during storage, which can be implemented in When reading the image block, read from the storage area corresponding to each color component, and decoupling the deinterleaving from the image block. If you need to change the deinterleaving strategy, you don’t need to change the way of dividing. Configure the deinterleaving parameters to the deinterleaving unit. Similarly, if you need to change the deblocking strategy without changing the deinterleaving method, you only need to configure the deblocking related parameters to the deblocking unit, which not only simplifies the design The complexity increases the flexibility, and can more easily support the new deinterleaving mode or the new block mode, which is beneficial to the expansion of the system architecture. When the image processing fails, the storage area can be used as the demarcation point to narrow down the troubleshooting And adjust the position, the verification of the processing method also played a simplified and accelerated effect.

BRIEF DESCRIPTION

In order to more clearly explain the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some of the applications of the present application For the embodiment, for those of ordinary skill in the art, without paying any creative labor, other drawings may be obtained based on these drawings.

FIG. 1 is a schematic diagram of an image interlacing arrangement method according to an embodiment of the present application.

FIG. 2 is a schematic diagram of another image interlacing arrangement method according to an embodiment of the present application.

FIG. 3 is a schematic diagram of another image interlacing arrangement method according to an embodiment of the present application.

FIG. 4 is a schematic diagram of another image interlacing arrangement method according to an embodiment of the present application.

FIG. 5 is a schematic diagram of an image processing method according to an embodiment of the present application.

6 is a schematic diagram of image data reading according to an embodiment of the present application.

7 is a schematic diagram of another image data reading according to an embodiment of the present application.

FIG. 8 is a schematic diagram of image data storage according to an embodiment of the present application.

9 is a schematic diagram of another image data storage according to an embodiment of the present application.

FIG. 10 is a schematic diagram of another image processing method according to an embodiment of the present application.

11 is a schematic diagram of image data processing according to an embodiment of the present application.

12 is a schematic diagram of another image data processing according to an embodiment of the present application.

13 is a schematic diagram of another image processing method according to an embodiment of the present application.

14 is a schematic diagram of an image processing architecture according to an embodiment of the present application.

15 is a schematic diagram of an image processing apparatus according to an embodiment of the present application.

16 is a schematic diagram of another image processing device according to an embodiment of the present application.

17 is a schematic diagram of a video processor according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

Unless otherwise stated, all technical and scientific terms used in the embodiments of the present application have the same meaning as commonly understood by those skilled in the technical field of the present application. The terminology used in this application is for the purpose of describing specific embodiments only, and is not intended to limit the scope of this application.

For raw images, a pixel can have one color component, and a raw image may have multiple color components (for example, including four color components R-Gr-Gb-B), and the multiple color components may be interleaved Arranged in a manner of arrangement. Among them, the interlaced arrangement mentioned here means that the pixels of different color components in the same raw image are arranged in an interlaced manner. In the same interlaced arrangement, the color components appearing in odd rows may be different from the color components appearing in even rows, and the color components appearing in odd rows may be different from those appearing in even columns. For different interweaving arrangements, the color components appearing in the same row may be different, or the color components appearing in the same row are the same, but the order in which the color components appear is different.

The following uses the interlace arrangement shown in FIGS. 1 to 4 as an example for description. Among them, in Figures 1 to 4, each small square represents a pixel.

For example, in the interleaved arrangement shown in FIG. 1, R and Gr appear in odd rows, where R belongs to odd columns and Gr belongs to even columns, B and Gb appear in even rows, B belongs to even columns, and Gb belongs to odd columns. In the interleaved arrangement shown in Figure 2, B and Gb appear in odd rows, where B belongs to odd columns and Gb belongs to even columns, R and Gr appear in even rows, R belongs to even columns, and Gr belongs to odd columns. In the interleaved arrangement shown in Figure 3, B and Gb appear in odd rows, where Gb belongs to odd columns and B belongs to even columns, R and Gr appear in even rows, Gr belongs to even columns, and R belongs to odd columns. In the interleaved arrangement shown in FIG. 4, R and Gr appear in odd rows, where Gr belongs to odd columns and R belongs to even columns, B and Gb appear in even rows, Gb belongs to even columns, and B belongs to odd columns.

In the application of raw images (for example, compression processing), the pixels of each color component need to be processed separately, so the pixels of each color component need to be taken out separately. At this time, special logic can be used to read and split the interlace Image data together, this process can be called deinterleaving.

In addition, in image processing, it is often necessary to segment the input image, because most image processing algorithms are for an image block rather than the entire image. A certain amount of image data can be deinterleaved according to the bus bandwidth and on-chip storage resources, and then the required image block size can be cut out from it. Among them, the cutting process can be referred to as dicing.

Taking prores raw image compression algorithm as an example, discrete cosine transform (Discrete Cosine Transform, DCT) can be used to process image blocks. Each color component takes 8x8 image blocks as the minimum unit to participate in the calculation. This 8x8 block can be called 1 code block (code block, cb). Since the pixels of the 4 color components are interwoven together, a 16x16 image block of the original image will contain 1 cb for each of the 4 color components. This 16x16 original image block can be called a microblock (mb). When processing prores, the algorithm may require 1mb, 2mb, 4mb, 8mb or 16mb in the horizontal direction at a time. That is to say, the purpose of block division is to cut out image blocks of the required size from the deinterleaved image data.

In a normal processing flow, after reading the interleaved image data, the image data may be deinterleaved, and the deinterleaved image data may be divided. If the deinterleaving unit and the block unit are tightly coupled together, although it is less difficult for the deinterleaving unit to implement, the deinterleaving unit and the block unit are related to each other, so many possible situations need to be considered. That is, when deinterleaving, it is necessary to also consider how to block, and different block sizes will affect the amount of deinterleaved data. For example, if there are 4 deinterleaving schemes supported by an algorithm and 6 possible block sizes, the deinterleaving unit needs to consider 4x6=24 cases. This structure is neither flexible nor easy to expand and verify.

Therefore, the embodiments of the present application provide the following solutions, which can solve the above problems.

FIG. 5 is a schematic flowchart of an image processing method 100 according to an embodiment of the present application. The method 100 may include at least part of the following content. The method 100 may be implemented by a processing device, which may be part of an encoder or independent of the encoder.

In 110, the processing device reads the image data, wherein the read image data includes a plurality of pixels having at least one color component, and each pixel has a color component, respectively. The number of bits of the pixel value of each pixel is greater than or equal to 1, for example, 12 bits or 16 bits. Among them, the read image data may be image data belonging to the raw image.

Specifically, the processing device can read the image data of the raw image from Double Rate Synchronous Dynamic Random Access (SDRAM) (Double Data Data Rate SDRAM, DDR), where the image data can be Stored in DDR after capturing the image.

Wherein, the data amount of the image data read by the processing device per clock can optionally be determined according to the size of the storage space and/or the bus bandwidth mentioned below for storing the deinterleaved image data.

For example, the amount of image data read per clock is less than the amount of data allowed by the bus bandwidth. For another example, the amount of data of a single color component in the image data read by each clock may be the maximum amount of data that can be stored in one storage address of the storage space.

When the processing device reads the image data, the number of pixels read by each clock may be an integer, or may not be an integer. The number of pixels mentioned here is an integer number means that at least one complete pixel is read in one clock, and the bit of each pixel is only read in one clock, and will not be divided into two or two More than one clock.

For example, it is assumed that each clock can read 128-bit data (many implementations in the embodiments of the present application are described using reading 128-bit data per clock as an example, but the embodiment of the present application is not limited to this ), if the number of bits of the pixel value of a single pixel is 12 bits, an integer number of pixels cannot be read by one clock, and a cycle of 3 clocks can accommodate the pixel value of a complete 32 pixels. Among them, these 32 pixels can belong to 2 color components and are staggered. Specifically, as shown in FIG. 6, p0-p31 represent 32 pixels. Since 128 bits cannot accommodate the pixel values of an integer number of 12-bit pixels, the pixel value of one pixel may be stored separately in the data of two adjacent clocks Medium, that is, the pixel value of pixel p10 is divided into clock 1 and clock 2, where clock 1 reads 8 bits of pixel p10, and clock 2 reads 4 bits of pixel p10, and the pixel value of pixel 21 is divided To clock 2 and clock 3, where clock 2 reads 4 bits of pixel p21, and clock 3 reads 8 bits of pixel p21.

For another example, suppose that each clock can read 128-bit data. If the number of pixels of a single pixel is 16 bits, then one clock can read 8 pixels of data, where these 8 pixels can belong to 2 The color components are staggered. Specifically, as shown in FIG. 7, p0-p7 represents 8 pixels. Since 128 bits can accommodate an integer number of 16-bit pixels, the pixel values of the 8 pixels can be accommodated in one clock, namely clock 1.

As can be seen from the above, if the number of bits that can be read per clock can be divided by the number of bits of the pixel value of a single pixel, the number of pixels that can be read per clock is an integer, but it should be understood that even if each The number of clocks that can be read can be divided by the number of bits of the pixel value of a single pixel, but it is not necessary that the number of pixels read per clock is an integer.

For example, suppose that each clock can read 128 bits of data. If the number of pixels of a single pixel is 16 bits, then there can be 8 bits of a pixel read by the last clock, and the remaining 8 of the pixel The bit is read by the next clock.

In 120, the processing device deinterleaves the read image data to obtain image data corresponding to each color component.

Specifically, the processing device can deinterleave the read image data, that is, determine the color component of each pixel read, where the image data is interleaved and arranged in a certain way, it can be based on the read image The position of the data in the original image determines the color component of each pixel.

In 130, the processing device separately stores the image data corresponding to each color component into a storage area corresponding to each color component in a storage space.

Specifically, in the storage space available to the system, each color component may exist in a corresponding storage area, and the processing device may store each color component in its corresponding storage area, so that in the storage area corresponding to each color component, read each The image blocks of the color components are subjected to subsequent image block division processing to achieve decoupling between deinterleaving and image block (here and the image blocks mentioned below may be cb) division.

In addition, the image data corresponding to each color component is stored in the storage area corresponding to each color component in the storage space, and it can be stored without being affected by the interweaving arrangement of the original image data.

In the embodiment of the present application, the storage area corresponding to each color component may include one or more, each storage area may have an independent writing and reading interface, and the one storage area may be a random access memory (random access memory) Access (memory, RAM), of course, may also be other forms of memory, which are not specifically limited in the embodiments of the present application.

Optionally, in the embodiment of the present application, in order to facilitate the subsequent block operation of the image block, a storage address of each storage area may store the pixel value of a certain number of pixels (defined here as the number N), where The number N may optionally be equal to the number of pixels included in the row of the image block, or an integer multiple of the number of pixels included in the row, and one storage address stores only pixels of one color component. For one color component, one storage area may include one storage address or multiple storage addresses.

Since the number of pixels of each color component obtained by deinterleaving each clock is not fixed, and may contain incomplete pixels (for example, as shown in FIG. 6), it may appear after storing a pixel of a clock. The number of pixels stored in one storage address does not reach the aforementioned N. In this case, if the number of pixels stored at an address in the storage area does not reach the above N, when the data of the next clock arrives, the remaining space of the current address needs to be filled in, and if there is an extra part, it needs to be written Next address. However, since the mechanism of RAM cannot write two addresses in the same clock, the present invention splits each RAM into at least two slices. That is, two RAMs can be provided for each color component, and after the address of one RAM is full, the remaining data of the color component is written in the other RAM.

For example, for the scenario shown in FIG. 6, in the case where the pixel value of a single pixel occupies 12 bits, each address of the storage area can store 8 pixels of a single color component, where the 8 pixels include The total data can be 8 16-bit data, each 16-bit data can include a 12-bit pixel value of a single pixel and a 4-bit placeholder bit, or the total data of the 8 pixels can also be 8 The 12-bit data can be determined according to the data format to be stored in each storage area (the data format is distinguished by the number of bits in one pixel). Assuming that each clock can read 128-bit data, the color component corresponding to p0, p2, p4...p30 is color component 1, and the color component corresponding to p1, p3, p5...p31 is color component 2. When clock 1 reads and deinterleaves the image data for storage, the storage address 1 corresponding to color component 1 can store 8 bits of the pixel values of p0, p2, p4, p6, p8 and p10, and the color component 2 corresponds to The storage address 1 can store the pixel values of p1, p3, p5, p7, and p9. When storing the image data read by clock 2, you need to store the four bits of the pixel value of p10, and p12 and p14. To address 1, when storage address 1 has reached 8 addresses, you need to store p16, p18, and p20 to a storage address in another RAM. For color component 2, the process is similar.

Therefore, in the embodiment of the present application, each color component may have at least two storage areas respectively having independent reading and writing interfaces, and specifically may have at least two RAMs. During storage, the image data of a certain color component can be stored to a storage address in one of the storage areas, and after the one address is full, the remaining unstored data of the image data of the color component is stored to A storage address in another storage area.

Among them, when writing from the RAM, it is possible to switch between at least two pieces of RAM in an interleaved manner, for example, as shown in FIG. 8. This storage method can ensure that when the remaining space of any address of a certain RAM is insufficient, and the next address needs to be written at the same time, the address of another RAM can be written, thereby solving the uncertainty of the number of pixels per clock. The problem of writing data needs to cross the address.

Similarly, when reading data from RAM, it is also possible to switch between at least two pieces of RAM in an interleaved manner, and the order of reading image data from RAM may be equal to the order of writing image data from RAM.

Optionally, in an embodiment of the present application, when one color component corresponds to at least two storage areas, the addresses of each storage area may be continuous. For example, in storage area 1, the address may be 0-127, and in storage area 2, the storage address may also be 0-127. Although each storage area has 128 storage addresses, the actual storage addresses have reached 256.

In this case, the processing device can record the number of all addresses (not distinguishing the storage area, which are all storage areas) where the image data of the specific color component has been stored. When the image data of the color component needs to be stored currently, The value x can be obtained by adding 1 on the basis of the recorded number, and this x can be understood as the order of the current addresses to be stored in all the storage addresses corresponding to the color component. Since there are at least two storage areas, the address to be stored in the storage area may be further determined based on the value x corresponding to the address currently to be stored. For example, assuming that there are 2 storage areas, if x is even, the image data can be stored in the address x/2 of storage area 1, if x is odd, the image data can be stored in the address of storage area 1. (x-1)/2.

Optionally, in an embodiment of the present application, when a color component corresponds to at least two storage areas, even if the image data of each clock can be stored at one address, there is no need to divide it into two addresses. The two storage areas are interleaved, for example, clock 1 data is stored in one address in storage area 1, clock 2 data is stored in another address in storage area 2, and clock 3 data is stored in another storage area 1 Address etc. Of course, after all the storage addresses corresponding to the color component in one storage area are stored, they may be stored in the storage addresses corresponding to the color component in another storage area.

It should be understood that in the embodiment of the present application, for a certain color component, if the data read by multiple clocks can just reach the number of pixels required by one address, the image data of a certain clock will not appear and need to be stored in two addresses In this case, at this time, the one color component may correspond to one storage area.

For example, for the scenario shown in FIG. 7, assuming that the number of bits occupied by the pixel value of a single pixel is 16 bits and the number of bits read per clock is 128 bits, one clock can read each color of the two color components The pixel value of the 4 pixels of the component. The two clocks can store a storage address for each of the two color components, that is, pixels that do not have a color component need to be stored in two addresses at the same clock. In this case, one color component may correspond to one storage area, wherein, after the data in the storage area is read, the data of the color component continues to be stored.

According to the above analysis, if we take the interleaved arrangement of color components as shown in Figures 1 to 4 as an example, one color component requires two storage areas, and in the case of 4 color components, 8 storage areas are required. However, it can be seen from the interleaved arrangement of the color components shown in FIGS. 1 to 4 that the color components appearing in one line of image data are some of the four color components. If the image data is read by line (That is, read line by line, read the next line after reading a line, where the length of the line can be determined according to the actual situation), so the color components of the same line may appear in the same clock, you need to avoid the same line The color components are shared in the storage area, which can avoid multiple color components of the same row being stored to one address when storing, and can avoid the problem that one clock needs to store two addresses in one storage area.

For multiple color components that do not appear in the same line, because the data is read by line, and the image data of these color components does not appear in the same clock, these color components can be shared in the storage area.

For example, for the interleaved arrangement shown in Figures 1 to 4, the image data of a single clock contains at most 2 color components, so you can merge the color component storage areas that do not need to appear on the same line. 4 storage areas (that is, 4 RAMs).

For example, for the interleaved arrangement shown in Figure 1, R and Gr appear in odd rows and B and Gb appear in even rows, then the storage area of R can be merged with one storage area of B and Gb, and/or, The storage area of Gr can also be merged with another storage area of B and Gb.

For example, as shown in FIG. 9, the storage areas of B and R can be merged, that is, both B and R occupy RAM1 and RAM2, and the storage areas of Gb and Gr can be combined, that is, both Gb and Gr occupy RAM3 and RAM4.

Therefore, in the interleaved arrangement shown in FIGS. 1 to 4, only four RAMs are needed to achieve the de-interleaved data buffering function.

Optionally, in the embodiment of the present application, when multiple color components are shared in one storage area, the address allocated to each color component is different. For example, for one of the storage areas, for the color component R, then The corresponding storage address is address 0-127, and for another color component B, the corresponding storage address may be address 128-255.

Therefore, in the embodiments of the present application, a single storage area with independent writing and reading interfaces may be used to store image data of at least two color components that do not appear in the same row of the image before deinterleaving, thereby While realizing the storage area, the image data of each color component can be stored independently by address. Among them, the sharing of the storage area can reduce the number of interfaces and the number of wires and improve the wiring quality.

Of course, in the embodiment of the present application, multiple color components may not be shared in the storage area, which is not specifically limited in the embodiment of the present application.

Optionally, in the embodiment of the present application, at least one storage area group may exist in the available storage space, and each storage area group includes at least one storage area corresponding to a color component. Each storage area group can be used to cyclically store the de-interleaved image data, that is, the data can be read after it is full, and after reading, it can continue to be stored. Different storage area groups include different storage areas for the same color component. For example, for storage area group 1, for color component 1, storage areas included are storage area 1 and storage area 2, and for storage area group 2, for color component 2, storage areas included are storage areas 3 and 4.

Specifically, the above-mentioned 4 RAMs (for example, as shown in FIG. 9) may be formed into a group, and 2 to 3 RAM groups may be deployed for ping-pong operation. Each RAM group can be marked as idle, receiving and sending 3 states. All RAMs are in an idle state during initialization, and are in a receive state when data reception begins. When a certain RAM group is filled with de-interleaved data, this RAM group will be marked as sending. When the block unit detects that a RAM group is in the transmit state, it starts to read data from it. Because the deinterleaved data is stored in the buffer unit, the block dividing unit only needs to read the corresponding data according to the configured image block size, and does not need to communicate with the deinterleaving unit.

For example, as shown in FIG. 10, in 201, wait for the configuration start signal; in 202, determine whether the RAM group is idle, when not idle, continue to wait for the configuration start signal in 201, when the RAM group is idle, then Go to 203; in 203, configure the RAM group to receive mode, which means that the RAM group can store data; in 204, store the deinterleaved data in the RAM group; in 205, determine the storage Is the last data that the RAM group needs to store. If the storage address of the RAM group is full, it can be considered that the currently stored data is the last data. If it is not, in 204, continue to The deinterleaved data is stored in the RAM group. If it is the last image data, jump to 206; in 206, the RAM group can be configured as the transmission mode, that is, the image data in the RAM group can be read, And continue to execute 202, from then on can realize the cyclic storage of the RAM group.

Optionally, in the embodiments of the present application, when the image data is read, the order of returning the image data may not be returned according to the read image data (this out-of-order return method can improve bus efficiency), so The color components of the returned image data are irregular, which may result in the inability to determine the color components of the pixels. Specifically, if the deinterleaving unit is behind the storage unit (after the reading unit), then the deinterleaving unit receives sequential data (that is, the storage unit has sorted the data), which can be easily Complete pixels are stitched together in each clock. However, if the deinterleaving unit is before the storage unit, the deinterleaving unit needs to be able to process the read data returned by the reading unit out of order. Since the color components of the returned image data are irregular, it may cause the pixel color to be undetermined Component, which makes it difficult to deinterleave the image data.

In order to solve the above problems, the embodiment of the present application proposes a tag mechanism, that is, generating a tag for the read image data, the tag indicates that the image data is stored in the original image (that is, the image data before reading, for example, stored in DDR Based on the label, the processing device can determine the position of each pixel in the image data in the original image, and thus can deinterleave the read image data, that is, determine each pixel Color component.

Wherein, when the processing device sends a read request for reading image data to the DDR through the Advanced Extensible Interface (AXI), it can generate the above-mentioned label based on the identifier (ID) number of the read request and return The image data may carry the ID number, and the processing device may determine a label corresponding to the returned image data based on the ID number, and deinterleave the returned image data based on the label.

As shown in Figures 1 to 4, there are different color components for even-numbered lines and odd-numbered lines, that is, there are two color components for even-numbered lines, and there are two other color components for odd-numbered lines, which can be based on whether the image data is even or odd To determine which two color components of the returned image data; and, the image data of each line can be read by at least one clock, and the data read by each clock is the data of a specific position of the line, which can be based on the reading The clock information of the image data determines which of the two color components of the corresponding row each pixel is.

Thus, the image data is deinterleaved based on the line information and clock information of the image data indicated by the label mentioned above. Wherein, the line information can represent whether the image data is an odd line or an even line. The clock information may represent the number of clocks in the current row that the clock for reading the image data belongs to. Alternatively, a clock corresponding to a row of data may be divided into cycles, and each row of data may correspond to a cycle, and each cycle may include one or more clocks. Therefore, the position of the image data in the original image can be determined based on the number of cycles the image data belongs to and the number of clocks in the cycle, so that the image data can be deinterleaved.

Optionally, in the embodiment of the present application, the storage location of the pixel may also be determined based on the label. Specifically, the storage area corresponding to the image data and the storage address in the storage area may be determined according to the line information of the image data and the clock information.

Assume that the size of the image block to be divided is N*N (N rows and N columns), one storage area group (available storage space may include one storage area group or multiple storage area groups, and each storage area group is directed to the original image Each color component included includes at least one storage area) The data that can be stored for one row of the original data can be data read by m clocks, and the data read by m clocks can correspond to m/s=w cycles, each Each cycle includes s clocks. For a certain color component, each cycle can deinterleave pixels with p addresses. p is the number of storage areas corresponding to the color component. Each cycle is for each of the p storage areas. One storage area can be written with data at one address. When calculating the storage address, the value of i can be from 0 to N-1, and the value of j can be from 0 to w-1. If the above storage exists In the case of area sharing, for other color components, a constant can be added on the basis of j*N+i, and the storage addresses of different color components can be distinguished. For example, the storage address of one color component is 0 to wN-1, and the storage address of another color component is wN to 2wN-1.

The clock number of the clock that reads the image data in the cycle to which the image data belongs and the row information of the image data can determine which storage area is stored, and the clock number of the clock that reads the image data in the cycle that belongs to can determine the data splicing method (also That is, to determine whether to write to the address of the last clock is not full).

For example, suppose that the size of the image data that a storage area group can store is 32mb, the size of the image block is 8*8, 128 bits of data are read per clock, and the pixel value of each pixel is 12 bits. Each address can store 8 pixels, then converted to original data is 48 clocks per line, of which, every 3 clocks can be a cycle, there are 16 cycles, for a color component, each cycle can produce 16 pixels , The 16 pixels can occupy 2 addresses, the 2 addresses belong to 2 storage areas, then for a cycle, a storage area needs to add 1 address, and because the number of lines of an image block is 8, Each additional loop can add 8 addresses, so you can use j*8+i or j*8+i+256 (where j is the number of loops, i is the number of even rows or odd rows) Calculate the storage address in the storage area to which it belongs. Since one cycle can occupy two addresses, and the two addresses belong to two storage areas, you can determine the data splicing method and the specific storage area according to the number of clocks in the cycle of the clock that reads the image data. .

For another example, suppose that the size of the image data that a storage area group can store is 32mb, the size of the image block is 8*8, 128 bits of data are read per clock, and the pixel value of each pixel is 16 bits. Each address can store 8 pixels, and the original data is converted into 64 clocks per line. Among them, every 4 clocks can be a cycle, there are 16 cycles, and for a color component, each cycle can produce 16 Pixels, the 16 pixels can occupy 2 addresses, the 2 addresses belong to 2 storage areas, then for a cycle, a storage area needs to add 1 address, and because an image block has 8 lines , Each additional cycle, you can add 8 addresses, so you can use j * 8 + i or j * 8 + i + 256 (where j is the number of cycles, i is the number of even rows or odd rows) Method to calculate the storage address in the storage area to which it belongs.

It should be understood that the calculation method of the storage address given above is only an implementation manner of the embodiment of the present application, and should not be particularly limited to the present application.

It should be understood that the above description is based on the example that the image block is a square, but it should be understood that the embodiments of the present application are not limited thereto, and the size of the divided image block may not be square.

For example, assuming that the size of the image block to be divided is Q*N (Q rows and N columns), the data that one storage area group can store for one row of the original data can be data read by m clocks, and m clocks can correspond to m /s=w cycles, each cycle includes s clocks, for a single color component, each cycle can generate p addresses of data, p is the number of storage areas corresponding to the single color component, then each cycle is for one Data in an address can be written in the storage area. Therefore, in calculating the storage location, the value of i can be from 0 to N-1 according to j*Q+i, and the value of j can be from 0 to w-1.

Therefore, based on the above scheme, it is possible to deinterleave the image data based on the label and the address of each color component in the storage area. The mechanism of the label of the present application can avoid buffering the data before deinterleaving and after the data is read to sort the read data.

If the image data processed by the deinterleaving process is sequential data, it can be deinterleaved and written into the storage area when the image data of a certain clock is collected. But if the deinterleaving process is out of order data, the data return is irregular, you can deinterleave the data in the current clock, because it is meaningless to wait for the data of the next clock, you can carry out on the image data in time Processing to avoid more and more unprocessed image data, occupying storage resources.

Optionally, the storage area of the embodiment of the present application can be used to store image data in multiple formats, where the number of pixels of the image data in different formats is different.

Specifically, in order to ensure compatibility, the processing device may process image data in multiple formats, for example, processing image data of 12 and 16 bits occupied by a single pixel, respectively.

For this reason, when the number of bits of a single pixel of image data corresponding to each color component is less than the first bit number, when storing the image data corresponding to each color component, the single pixel is place-filled so that the The number of bits of a single pixel reaches the first number of bits, where the first number of bits is the maximum number of bits corresponding to a single pixel in image data in multiple formats.

Taking the number of bits occupied by a single pixel as 12-bit and 16-bit, respectively, in order to ensure that the storage area can store both 12-bit pixels and 16-bit pixels, when storing 12-bit pixels, you can target Each 12-bit pixel is filled with 4 bits. Among them, the lower bits of 12 bits can be filled. Of course, the higher bits of 12 bits can also be filled. The value of the filled bits can be 0.

In order to facilitate a clearer understanding of the present application, the following will take the number of bits of a single pixel as 12 bits and the number of bits of a single pixel as 16 bits as an example. storage.

Assuming that the number of bits of a single pixel is 12 bits, and the number of bits read per clock is 128 bits, the interleaved arrangement of the image can be as shown in Figure 1, each color component has two RAMs, and color components B and R share Storage area, the color components Gb and Gr share the storage area. At this time, three clocks can be used as one cycle, and each cycle can generate two addresses of image data for each of the two color components. The available storage space includes a total of 4 RAMs, namely RAM1, RAM2, RAM3 and RAM4. R and B share RAM1 and RAM2, and Gr and Gb share RAM3 and RAM4.

Fig. 11 shows the image data storage of 3 clocks in one cycle. There are 3 large frames in Fig. 11, each of which is a clock. Among them, the top row of each large box is a clock of 128 bits of data read, each small box in the row represents 4 bits of data, 32 small boxes of 128 bits of data, of which One pixel is arranged adjacent to every 12 bits, and each color represents a color component.

The bottom four rows of each large frame in FIG. 11 are the data formats written to 4 pieces of RAM. In order to be compatible with the 16-bit format, the lower 4 bits of each 12-bit pixel are filled with 0s, as shown by the small gray and black boxes. Each time the deinterleaving unit receives a row of interleaved data, it can be split into the single-color image data shown below and written to the corresponding RAM.

It can be seen from FIG. 11 that the image data read by clock 1 includes color components R and Gr, the color corresponding to R can be stored in address 1 in RAM1, and the color component corresponding to Gr can be stored in RAM3 The image data read by address 1 and clock 2 still include the color components R and Gr. Since neither address 1 in RAM 1 nor address 1 in RAM 3 is fully stored, a part of the pixel corresponding to R read by clock 2 can be used Data is stored in address 1 of RAM1, another part of data is stored in address 1 of RAM2, and part of the data of the pixel corresponding to Gr read by this clock 2 is stored in address 1 of RAM3, and the other part of data is stored in Address 1 in RAM4. The image data read by clock 3 still includes the color components R and Gr. Since neither address 1 in RAM2 nor address 1 in RAM4 is full, the pixels corresponding to R read by clock 3 can be stored in RAM3. Address 1 (the remaining space at this address just stores the pixel corresponding to R read by clock 3), and the pixel corresponding to the Gr read by this clock is stored in address 1 of RAM 4 (the remaining space at this address just stores clock 3 The pixel corresponding to R read).

As shown in FIG. 11, the address written to the RAM is marked in a rectangular frame on the right. Assuming that the available storage space can open up 32MB of RAM, the conversion to raw data will take 48 clocks per line. The 48 clocks are divided into 16 cycles with 3 clocks as one cycle. In order to calculate the address where the data should be written, the raw data of each clock will have a label, which is displayed in the rectangular box on the right side of the figure. Where i represents how many even or odd rows the current data is, (2*i+0) represents that this is an even row of data. 48 is the increment of the address that each line should jump based on the size of the storage space. This number will change with the size of the RAM. j represents how many cycles the current data belongs to. The last 0, 1, 2 represents that this is the data of the 0, 1, 2 clock of the current cycle. According to the information of i and j, the RAM address where the current deinterleaved data should be stored can be calculated. The data splicing method can be determined according to the data of the several clocks of the current cycle.

Assuming that the number of bits of a single pixel is 16 bits, and the number of bits read per clock is 128 bits, the interleaved arrangement of the image can be as shown in Figure 1, each color component has two RAMs, and color components B and R share Storage area, the color components Gb and Gr share the storage area. At this time, 4 clocks may be used as one cycle, and each cycle may generate image data of two addresses for each of the two color components. A total of 4 RAMs are included, namely RAM1, RAM2, RAM3 and RAM4. R and B share RAM1 and RAM2, and Gr and Gb share RAM3 and RAM4.

Fig. 12 shows the storage of image data of two clocks in one cycle. There are two large frames in Fig. 12, and each large frame contains a clock. Among them, the top row of each large box is a clock of 128 bits of data read, each small box in the row represents 4 bits of data, 32 small boxes of 128 bits of data, of which One pixel is arranged adjacent to every 16 bits, and each color represents a color component. The lower four rows in Fig. 12 are the data formats written to 2 RAMs.

As can be seen from FIG. 12, the image data read by clock 1 includes color components R and Gr, the color corresponding to R can be stored in address 1 in RAM1, and the color component corresponding to Gr can be stored in RAM3 Address 1. The image data read by clock 2 still includes the color components R and Gr. Since neither address 1 in RAM1 nor address 1 in RAM2 is full, the pixels corresponding to R read by clock 2 can be stored in RAM1. Address 1 (the remaining space at this address just stores the pixel corresponding to R read by clock 2), and the pixel corresponding to the Gr read by this clock is stored in address 1 of RAM2 (the remaining space at this address just stores clock 2 The pixel corresponding to the read Gr).

As shown in FIG. 12, the address written to the RAM is marked in a rectangular frame on the right. Assuming that the size of each ram group can be cached by 32mb, it is converted into 64 clocks per line of raw data. The 64 clocks are divided into 16 cycles with 4 clocks as one cycle. In order to calculate the address where the data should be written, the raw data of each clock will have a label, such as shown in the rectangular box on the right side of the figure. Where i represents how many even or odd rows the current data is, (2*i+0) represents that this is an even row of data. 64 is the address increment that each line should jump based on the size of the storage space. This number will change with the RAM capacity. j represents how many cycles the current data belongs to. The last 0, 1, 2, 3 represents that this is the data of the 0, 1, 2, 3 clock of the current cycle. According to the information of i and j, the RAM address where the current deinterleaved data should be stored can be calculated. The data splicing method can be determined according to the data of the several clocks of the current cycle.

In 140, from the storage area, for each color component, a read operation is separately performed to obtain an image block of each color component.

For example, as shown in FIG. 13, in 301, wait for a configuration start signal; in 302, determine whether a back-end module (for example, a module that performs image compression) requests image data, and if not, in 301, continue to wait for configuration The start signal, if it is, in 303, determine whether there is a RAM group as the transmission mode. If not, continue to execute 303, if yes, perform 304; in 304, read the image data in the RAM group; in 305, Determine whether the read data reaches the configured image block size, if not, continue to perform 304, read data, if yes, then perform 306; in 306, determine whether there is unused data in the RAM group, if No, if the RAM group is configured in idle mode, the deinterleaving data can be written subsequently. If yes, go to 302 to determine whether the back-end module requests data.

In order to understand the application more clearly, the embodiments of the application will be described below with reference to FIG. 14.

Figure 14 illustrates the technical background of prores raw video coding architecture as an example. It should be understood that the present application can be used in all similar video/image processing architectures. FIG. 14 is a general block diagram of the prores raw video coding architecture. The prores raw video coding architecture 400 is composed of three units, namely a preprocessing unit 410, a processing unit 420, and a writing unit 430. The preprocessing unit 410 here may correspond to the processing device in the embodiment of the present application.

The reading unit 411 can send a read request to the DDR500 through the axi channel, read image data from the DDR, the reading unit 411 can generate a tag based on the ID, and can receive the image data sent by the DDR through the axi channel, the reading unit 411 can Send the label and the image data to the deinterleaving unit 412; the deinterleaving unit 412 can deinterleave the image data and generate a write address, and based on the write address, send the deinterleaved image data to the buffer unit 413, which may include multiple RAM group, each RAM group can be stored cyclically; the block unit 414 can perform RAM reading, block image data, and the image block is sent to the processing unit 420, the processing unit 420 can process the image block (for example, compression Process), and send the processed image data to the write-out unit 430, where the write-out unit 430 can write the processed data into the DDR500.

In the embodiment of the present application, de-interleaving the read image data to obtain image data corresponding to each color component, and storing the image data corresponding to each color component to the storage space respectively In the storage area corresponding to the color component, from the storage area, read operations are performed for each color component separately to obtain an image block of each color component. The storage of image data is added between the blocks, and the image data corresponding to each color component is stored in the storage area corresponding to each color component during storage, which can be achieved from each color component when the image block is read Read the corresponding storage area, decoupling the deinterleaving from the image block. If you need to change the deinterleaving strategy, you don’t need to change the method of block division, just configure the relevant parameters of the deinterleaving to the solution. Interleaving unit, similarly, if you need to change the strategy of partitioning, you don’t need to change the deinterleaving method, you only need to configure the relevant parameters of the partitioning to the partitioning unit, which not only simplifies the design complexity, but also increases the flexibility. Moreover, it can more conveniently support the new deinterleaving mode or the new block mode, which is conducive to the expansion of the system architecture. When the image processing fails, the storage area can be used as the demarcation point to narrow down the troubleshooting and adjust the position, and verify the processing method. It also simplifies and accelerates.

15 is a schematic block diagram of an image processing apparatus 600 according to an embodiment of the present application. The device 600 includes a first reading unit 610, a deinterleaving unit 620, a storage unit 630, and a second reading unit 640.

The first reading unit 610 is used to read image data, wherein the read image data includes a plurality of pixels, the plurality of pixels have at least one color component, and each pixel has a color component ; Deinterleaving unit 620, used to deinterleave the read image data to obtain image data corresponding to each color component; storage unit 630, used to store the image data corresponding to each color component respectively Into a storage area corresponding to each color component in the storage space; a second reading unit 640 is configured to perform a reading operation for each color component from the storage area to obtain the Image block for each color component.

Optionally, in the embodiment of the present application, each color component corresponds to at least two storage areas, and each storage area has an independent writing and reading interface.

Optionally, in the embodiment of the present application, the at least one color component includes a first color component, and the storage area corresponding to the first color component includes a first storage area and a second storage area, and the first color component The corresponding image data is first image data, wherein the read image data is read by a single clock, and each storage area in the first storage area and the second storage includes at least one storage address, Each storage address stores the pixel value of at least one pixel;

The storage unit 630 is further used for:

Storing the first image data to a storage address of the first storage area, and after the one address is full, storing the remaining unstored data of the first image data to the second storage area A storage address.

Optionally, in the embodiment of the present application, each color component corresponds to at least one storage area, and each storage area is used to store image data of at least two color components that do not appear in the same row, and each storage area Separate write and read interfaces.

Optionally, in the embodiment of the present application, each color component corresponds to at least one storage area, and the storage areas corresponding to the color components belonging to the same row are different, and each storage area has an independent writing and reading interface.

Optionally, in the embodiment of the present application, the deinterleaving unit 620 is further used to:

Deinterleaving the image data based on the label of the image data;

The storage unit 630 is further used for:

Based on the label, determining the storage address of each color component in the corresponding storage area;

Wherein, the tag represents the position information of the image data in the original image before reading.

Optionally, in the embodiment of the present application, the label information includes line information of the image data and clock information of the image data in the line to which it belongs.

Optionally, in the embodiment of the present application, the line information indicates whether the line to which the image data belongs is an odd line or an even line, and indicates the number of odd or even lines to which the image data belongs.

Optionally, in the embodiment of the present application, the clock information indicates the number of cycles of the cycle to which the clock for reading the image data belongs, and indicates the number of clocks in the cycle to which the clock for reading the image data belongs; Wherein, the storage space includes at least one storage area group, each storage area group includes a storage area corresponding to the at least one color component, and the maximum storable line of data in each storage area group needs to be at least one cycle clock Read, each cycle includes at least one clock, and each cycle deinterleaves pixels of p addresses for each color component in the at least one color component, where p is the storage area corresponding to each color component Quantity.

Optionally, in the embodiment of the present application, the label is generated based on the identification ID number of the read request to read the image data.

Optionally, in the embodiment of the present application, the storage space can be used to store image data in multiple formats, wherein the number of pixels of the image data in different formats is different.

Optionally, in the embodiment of the present application, the storage unit 630 is further used to:

When the number of bits of a single pixel of the image data corresponding to each color component is less than the first bit number, when storing the image data corresponding to each color component, the single pixel is place-filled so that the The number of bits reaches the first number of bits, where the first number of bits is the maximum number of bits of a single pixel corresponding to image data in multiple formats.

Optionally, in the embodiment of the present application, the storage space includes a plurality of storage area groups, each storage area group includes a storage area corresponding to the at least one color component, and different storage area groups include the same color component Each storage area group is used to cyclically store deinterleaved image data.

It should be understood that the image processing device 600 may implement the method in the foregoing method embodiments, and for the sake of brevity, no further description is provided here.

16 is a schematic block diagram of an image processing apparatus 700 according to an embodiment of the present application. As shown in FIG. 16, the image processing apparatus 700 may include a processing circuit 710. Wherein, the processing circuit 710 may be used to implement the method in the above method embodiments, and for the sake of brevity, no further description is provided here.

Optionally, in the embodiment of the present application, the image processing device 700 further includes a memory, which may be used to provide a storage space for storing the deinterleaved image data mentioned in the embodiment of the present application.

17 is a schematic block diagram of a video processor 800 according to an embodiment of the present application. The video processor 800 may include an image processing device 810, which may correspond to the image processing device 600 or 700 described above.

It should be understood that the video processor 800 may further include other units, for example, a unit for compressing the image blocks read by the image processing device, etc. For the sake of brevity, details are not described here.

The image processing device or video processor in the embodiment of the present application may be applied to a drone to provide image processing for a RAM photographed by a photographing device carried on the drone.

The processing circuit in the embodiment of the present application may be an integrated circuit chip with data processing capabilities. In the implementation process, each step of the foregoing method embodiments may be completed by an integrated logic circuit of hardware in a processing circuit or an instruction in the form of software. The above-mentioned processing circuit may be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), an existing programmable gate array (Field Programmable Gate Array, FPGA), or other available Programming logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps, and logical block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in conjunction with the embodiments of the present application may be directly embodied and executed by a hardware decoding processor, or may be executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, and registers. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

Those of ordinary skill in the art may realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application of the technical solution and design constraints. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Those skilled in the art can clearly understand that for the convenience and conciseness of the description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed system, device, and method may be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a division of logical functions. In actual implementation, there may be other divisions, for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.

If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application essentially or part of the contribution to the existing technology or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to enable a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

The above is only the specific implementation of this application, but the scope of protection of this application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in this application. It should be covered by the scope of protection of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (29)

1. An image processing method, characterized in that it includes:

   Reading image data, wherein the read image data includes a plurality of pixels, the plurality of pixels has at least one color component, and each pixel has a color component;

   De-interleaving the read image data to obtain image data corresponding to each color component;

   Storing the image data corresponding to each color component to a storage area corresponding to each color component in the storage space;

   From the storage area, for each color component, a read operation is separately performed to obtain an image block of each color component.
2. The method according to claim 1, wherein each color component corresponds to at least two storage areas, and each storage area has an independent writing and reading interface.
3. The method according to claim 2, wherein the at least one color component includes a first color component, and the storage area corresponding to the first color component includes a first storage area and a second storage area, the first The image data corresponding to the color component is first image data, wherein the read image data is read by a single clock, and each of the first storage area and the second storage includes at least one storage Address, each storage address stores the pixel value of at least one pixel;

   The storing the image data corresponding to each color component to the storage area corresponding to each color component respectively includes:

   Storing the first image data to a storage address of the first storage area, and after the one address is full, storing the remaining unstored data of the first image data to the second storage area A storage address.
4. The method according to any one of claims 1 to 3, wherein each color component corresponds to at least one storage area, and each storage area is used to store at least two color components that do not appear in the same row For the image data, each storage area has an independent write and read interface.
5. The method according to any one of claims 1 to 4, wherein each color component corresponds to at least one storage area, and the color components belonging to the same row correspond to different storage areas, and each storage area has an independent Write and read interface.
6. The method according to any one of claims 1 to 5, wherein the deinterleaving the read image data includes:

   Deinterleaving the image data based on the label of the image data;

   The storing the image data corresponding to each color component to the storage area corresponding to each color component respectively includes:

   Based on the label, determining the storage address of each color component in the corresponding storage area;

   Wherein, the tag represents the position information of the image data in the original image before reading.
7. The method according to claim 6, wherein the tag information includes line information of the image data and clock information of the image data in the line to which it belongs.
8. The method according to claim 7, wherein the line information indicates whether the line to which the image data belongs is an odd line or an even line, and indicates the number of odd or even lines to which the image data belongs.
9. The method according to claim 7 or 8, wherein the clock information indicates the cycle number of the cycle to which the clock for reading the image data belongs, and indicates that the clock for reading the image data is in the corresponding cycle The number of clocks; wherein, the storage space includes at least one storage area group, each storage area group includes the storage area corresponding to the at least one color component, and the maximum storable line of data in each storage area group needs to be at least One cycle of clock reading, each cycle includes at least one clock, and each cycle deinterleaves pixels of p addresses for each color component in the at least one color component, where p is corresponding to each color component The number of storage areas.
10. The method according to any one of claims 6 to 9, wherein the tag is generated based on an identification ID number of a read request to read the image data.
11. The method according to any one of claims 1 to 10, wherein the storage space can be used to store image data in multiple formats, wherein the number of pixels of the image data in different formats is different.
12. The method according to claim 11, wherein the storing the image data corresponding to each color component to the storage area corresponding to each color component respectively comprises:

    When the number of bits of a single pixel of the image data corresponding to each color component is less than the first bit number, when storing the image data corresponding to each color component, the single pixel is place-filled so that the The number of bits reaches the first number of bits, where the first number of bits is the maximum number of bits of a single pixel corresponding to image data in multiple formats.
13. The method according to any one of claims 1 to 12, wherein the storage space includes a plurality of storage area groups, and each storage area group includes a storage area corresponding to the at least one color component, different storage The area groups include different storage areas for the same color component, and each storage area group is used to cyclically store deinterleaved image data.
14. An image processing device, characterized in that it includes:

    A first reading unit for reading image data, wherein the read image data includes a plurality of pixels, the plurality of pixels has at least one color component, and each pixel has a color component;

    A deinterleaving unit for deinterleaving the read image data to obtain image data corresponding to each color component;

    A storage unit, configured to store the image data corresponding to each color component in a storage area corresponding to each color component in a storage space;

    The second reading unit is configured to perform a reading operation for each color component from the storage area to obtain an image block of each color component.
15. The device according to claim 14, wherein each color component corresponds to at least two storage areas, and each storage area has an independent writing and reading interface.
16. The device according to claim 15, wherein the at least one color component includes a first color component, and the storage area corresponding to the first color component includes a first storage area and a second storage area, the first The image data corresponding to the color component is first image data, wherein the read image data is read by a single clock, and each of the first storage area and the second storage includes at least one storage Address, each storage address stores the pixel value of at least one pixel;

    The storage unit is further used for:

    Storing the first image data to a storage address of the first storage area, and after the one address is full, storing the remaining unstored data of the first image data to the second storage area A storage address.
17. The device according to any one of claims 14 to 16, wherein each color component corresponds to at least one storage area, and each storage area is used to store at least two color components that do not appear in the same row For the image data, each storage area has an independent write and read interface.
18. The device according to any one of claims 14 to 17, wherein each color component corresponds to at least one storage area, and color components belonging to the same row correspond to different storage areas, and each storage area has an independent Write and read interface.
19. The device according to any one of claims 14 to 18, wherein the deinterleaving unit is further used to:

    Deinterleaving the image data based on the label of the image data;

    The storage unit is further used for:

    Based on the label, determining the storage address of each color component in the corresponding storage area;

    Wherein, the tag represents the position information of the image data in the original image before reading.
20. The device according to claim 19, wherein the tag information includes line information of the image data and clock information of the image data in the line to which it belongs.
21. The apparatus according to claim 19, wherein the line information indicates whether the line to which the image data belongs is an odd line or an even line, and indicates the number of odd or even lines to which the image data belongs.
22. The apparatus according to claim 20 or 21, wherein the clock information indicates a cycle number of a cycle to which the clock for reading the image data belongs, and instructs a clock for reading the image data to belong to the cycle The number of clocks; wherein, the storage space includes at least one storage area group, each storage area group includes the storage area corresponding to the at least one color component, and the maximum storable line of data in each storage area group needs to be at least One cycle of clock reading, each cycle includes at least one clock, and each cycle deinterleaves pixels of p addresses for each color component in the at least one color component, where p is corresponding to each color component The number of storage areas.
23. The apparatus according to any one of claims 19 to 22, wherein the tag is generated based on an identification ID number of a read request to read the image data.
24. The device according to any one of claims 14 to 23, wherein the storage space can be used to store image data in multiple formats, wherein the number of pixels of the image data in different formats is different.
25. The device according to claim 24, wherein the storage unit is further used to:

    When the number of bits of a single pixel of the image data corresponding to each color component is less than the first bit number, when storing the image data corresponding to each color component, the single pixel is place-filled so that the The number of bits reaches the first number of bits, where the first number of bits is the maximum number of bits of a single pixel corresponding to image data in multiple formats.
26. The device according to any one of claims 14 to 25, wherein the storage space includes a plurality of storage area groups, and each storage area group includes a storage area corresponding to the at least one color component, different storage The area groups include different storage areas for the same color component, and each storage area group is used to cyclically store deinterleaved image data.
27. An image processing apparatus, characterized by comprising a processing circuit and a memory; wherein the processing unit is used to execute the method according to any one of claims 1 to 13.
28. The apparatus according to claim 27, wherein the image processing apparatus further comprises a memory, wherein the memory is used to provide a storage space, and the storage space is used by the processing circuit to store deinterleaved image data .
29. A video processor, characterized by comprising the image processing device according to any one of claims 14 to 28.

Images