CN112468789A - RAW image conversion circuit and RAW image conversion method - Google Patents
RAW image conversion circuit and RAW image conversion method Download PDFInfo
- Publication number
- CN112468789A CN112468789A CN202110141248.XA CN202110141248A CN112468789A CN 112468789 A CN112468789 A CN 112468789A CN 202110141248 A CN202110141248 A CN 202110141248A CN 112468789 A CN112468789 A CN 112468789A
- Authority
- CN
- China
- Prior art keywords
- pixel
- array
- color component
- unit
- interpolation calculation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/843—Demosaicing, e.g. interpolating colour pixel values
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Image Processing (AREA)
Abstract
The application discloses a RAW image conversion circuit and a RAW image conversion method, wherein the cache position of each pixel with initial color components is determined according to the distribution position of each pixel in an initial RAW frame image, and an interpolation calculation array of the corresponding pixel is output based on the cache position, and the pixel comprises at least one of a pixel to be converted and a non-conversion pixel; generating a pixel to be converted with a target color component according to the interpolation calculation array; and correspondingly outputting at least one of a non-conversion pixel with an initial color component and a pixel to be converted with a target color component according to the cache position; the method supports the stream input and stream output of pixel streams, and no interval exists between frame images, so that the conversion efficiency is improved; and the image which does not need to be converted can be directly output in a streaming mode, and the target RAW frame image in the corresponding RAW format can be output in a streaming mode after conversion according to the cache position of the corresponding pixel.
Description
Technical Field
The present application relates to the field of RAW image conversion technologies, and in particular, to a RAW image conversion circuit and a RAW image conversion method.
Background
An Image captured by a digital camera is generally not suitable for display, and an unprocessed RAW data format (RAW) is processed by an ISP (Image Signal Processor) and converted into an Image format suitable for display, such as RGB or YUV. An image in RAW format also comprises a plurality of different image formats, for example RGB-Ir4x4, RGB-Ir2x2 are two different RAW formats. Since both the digital camera technology and the processing technology of the ISP are rapidly changing, it sometimes happens that the RAW format output by the digital camera is different from the RAW format supported by the ISP. In this case, the RAW image needs to be subjected to corresponding format conversion.
Disclosure of Invention
The application provides a RAW image conversion circuit and a RAW image conversion method, which solve the problem that images in different RAW formats need to be converted.
In a first aspect, the present application provides a RAW image conversion circuit, which includes an input processing unit, a logic conversion unit, and a register output unit; the input processing unit is used for determining the cache position of each pixel with an initial color component according to the distribution position of each pixel in the initial RAW frame image, and outputting an interpolation calculation array of the corresponding pixel based on the cache position, wherein the pixel comprises at least one of a pixel to be converted and a pixel which is not converted; the logic conversion unit is connected with the input processing unit and used for generating a pixel to be converted with a target color component according to the interpolation calculation array; and the register output unit is connected with the logic conversion unit and is used for correspondingly outputting at least one of the non-conversion pixel with the initial color component and the pixel to be converted with the target color component according to the cache position.
Based on the first aspect, in a first implementation manner of the first aspect, the interpolation calculation array includes a first interpolation calculation sub-array and a second interpolation calculation sub-array; the input processing unit outputs a first interpolation calculation sub-array of the corresponding pixel based on the cache position; and the logic conversion unit obtains a corresponding second interpolation calculation sub-array according to the first interpolation calculation sub-array, and generates a pixel to be converted with a target color component according to the second interpolation calculation sub-array.
In a second implementation manner of the first aspect, based on the first implementation manner of the first aspect, the input processing unit includes a buffer unit and a first control unit; the cache unit is used for accessing and caching each pixel with the initial color component; the first control unit is connected with the cache unit and used for controlling each pixel with the initial color component to be stored to the corresponding cache position according to the distribution position and controlling the cache unit to output the first interpolation calculation sub-array of the corresponding pixel based on the cache position and the selection algorithm.
In a third implementation manner of the first aspect, based on the second implementation manner of the first aspect, the logic conversion unit includes a first register array unit and a second control unit; the first register array unit is connected with the cache unit and used for registering the first interpolation calculation sub-array; the second control unit is connected with the first register array unit and used for generating and outputting a corresponding second interpolation calculation sub-array according to the first interpolation calculation sub-array and the first interpolation algorithm.
Based on the third implementation manner of the first aspect, in a fourth implementation manner of the first aspect, the logic conversion unit further includes a second register array unit and a third control unit; the second register array unit is connected with the first register array unit and used for registering a second interpolation calculation sub-array; and the third control unit is connected with the second register array unit and used for generating the pixel to be converted with the target color component according to the second interpolation calculation sub-array and the second interpolation algorithm.
Based on the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect, the register output unit includes a 1 × 1 register array, and the 1 × 1 register array is connected to the second register array unit and is configured to register and output at least one of a pixel to be converted having a target color component and a non-converted pixel having an initial color component.
Based on the first implementation manner of the first aspect, in a sixth implementation manner of the first aspect, the first interpolation computation subarray is a 9 × 9 pixel array formed by selecting peripheral pixels with initial color components by taking corresponding pixels with initial color components as centers based on the buffer positions.
In a seventh implementation form of the first aspect, based on the sixth implementation form of the first aspect, the second interpolation computation subarray is a 5x5 pixel array generated based on the first interpolation algorithm.
In an eighth implementation manner of the first aspect, based on any of the foregoing implementation manners of the first aspect, when the distribution positions of the pixels are located at the periphery of the initial RAW frame image and are not enough to form the corresponding interpolation calculation arrays, at least one 4 × 4 pixel array is virtually complemented at the periphery of the buffer positions of the pixels with the initial color components; the 4-by-4 pixel array is a pixel array formed by the first four rows of color components and the first four columns of color components in the interpolation calculation array.
In a second aspect, the present application provides a RAW image conversion method, including: determining a cache position of each pixel with an initial color component according to the distribution position of each pixel in the initial RAW frame image, and outputting an interpolation calculation array of the corresponding pixel based on the cache position, wherein the pixel comprises at least one of a pixel to be converted and a non-converted pixel; generating a pixel to be converted with a target color component according to the interpolation calculation array; and correspondingly outputting at least one of the non-conversion pixel with the initial color component and the pixel to be converted with the target color component according to the buffer position.
According to the RAW image conversion circuit and the RAW image conversion method, the cache position of each pixel with an initial color component is determined according to the distribution position of each pixel in an initial RAW frame image, and an interpolation calculation array of the corresponding pixel is output based on the cache position, wherein the pixel comprises at least one of a pixel to be converted and a non-conversion pixel; generating a pixel to be converted with a target color component according to the interpolation calculation array; and correspondingly outputting at least one of a non-conversion pixel with an initial color component and a pixel to be converted with a target color component according to the cache position; the method supports the stream input and stream output of pixel streams, and no interval exists between frame images, so that the conversion efficiency is improved; and the image which does not need to be converted can be directly output in a streaming mode, and the target RAW frame image in the corresponding RAW format can be output in a streaming mode after conversion according to the cache position of the corresponding pixel.
Drawings
The technical solution and other advantages of the present application will become apparent from the detailed description of the embodiments of the present application with reference to the accompanying drawings.
Fig. 1 is a schematic structural diagram of a RAW image conversion circuit according to an embodiment of the present application.
Fig. 2 is a schematic structural diagram of the RAW image conversion circuit in fig. 1.
Fig. 3 is a schematic structural diagram of a first register array unit storing color components of corresponding pixels according to an embodiment of the present disclosure.
Fig. 4 is a schematic structural diagram of stitching color components of peripheral pixels according to the embodiment of the present application.
Fig. 5 is a schematic structural diagram of a RAW image in an rgbirr 4x4 format according to an embodiment of the present application.
Fig. 6 is a schematic structural diagram of a RAW image in an rgbirr 2x2 format according to an embodiment of the present application.
Fig. 7 is a schematic structural diagram of the initial pixel repeat array of fig. 5.
Fig. 8 is a schematic flowchart of obtaining a second interpolation computation subarray according to an embodiment of the present application.
Fig. 9 is a schematic flowchart of obtaining a color component of a target according to an embodiment of the present application.
Fig. 10 is a flowchart illustrating a RAW image conversion method according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1 to 4, as shown in fig. 1, the present embodiment provides a RAW image conversion circuit, which includes an input processing unit 100, a logic conversion unit 200, and a register output unit 300. The input processing unit 100 is configured to determine a buffer position of an initial color component of each pixel according to a distribution position of each pixel in the initial RAW frame image, and output an interpolation calculation array of a corresponding pixel based on the buffer position, where the pixel includes at least one of a pixel to be converted and a non-conversion pixel. The logic conversion unit 200 is connected to the input processing unit 100, and is configured to generate a target color component of a pixel to be converted according to the interpolation calculation array. The register output unit 300 is connected to the logic converting unit 200, and is configured to output the initial color component of the non-conversion pixel and the target color component of the pixel to be converted according to the buffer location.
It should be noted that, the RAW image conversion circuit in this embodiment is based on the corresponding pixel stream accessed at the distribution position and outputs the corresponding pixel stream according to the corresponding cache position, so that it supports stream input and stream output of the pixel stream, and there is no need to exist an interval between frame images, thereby improving the conversion efficiency; and the image which does not need to be converted can be directly output in a streaming mode, and the target RAW frame image in the corresponding RAW format can be output in a streaming mode after conversion according to the cache position of the corresponding pixel.
It can be understood that the RAW image conversion circuit determines a buffer position of an initial color component of each pixel according to a distribution position of each pixel in an initial RAW frame image, outputs an interpolation calculation array of a corresponding pixel based on the buffer position, generates a target color component of a pixel to be converted according to the interpolation calculation array, and outputs an initial color component of a non-conversion pixel and a target color component of the pixel to be converted according to the buffer position, so that the initial RAW frame image can be converted into a target RAW frame image in a corresponding RAW format.
As shown in fig. 2, in one embodiment, the interpolation calculation array includes a first interpolation calculation sub-array and a second interpolation calculation sub-array; the input processing unit 100 outputs a first interpolation calculation sub-array of the corresponding pixel based on the buffer position; the logic conversion unit 200 obtains a corresponding second interpolation computation sub-array according to the first interpolation computation sub-array, and generates a target color component of the pixel to be converted according to the second interpolation computation sub-array.
As shown in fig. 2, in one embodiment, the input processing unit 100 includes a buffer unit 110 and a first control unit 120; the buffer unit 110 is configured to access and buffer the initial color component of each pixel; the first control unit 120 is connected to the buffer unit 110, and configured to control the initial color component of each pixel to be stored in the corresponding buffer location according to the distribution location, and control the buffer unit 110 to output the first interpolation computation subarray of the corresponding pixel based on the buffer location and the selection algorithm.
As shown in FIG. 2, in one embodiment, the logic conversion unit 200 includes a first register array unit 210 and a second control unit 220; the first register array unit 210 is connected to the buffer unit 110, and is configured to register a first interpolation computation subarray; the second control unit 220 is connected to the first register array unit 210, and is configured to generate and output a corresponding second interpolation calculation sub-array according to the first interpolation calculation sub-array and the first interpolation algorithm.
As shown in FIG. 2, in one embodiment, the logic conversion unit 200 further includes a second register array unit 230 and a third control unit 240; the second register array unit 230 is connected to the first register array unit 210, and is used for registering a second interpolation calculation sub-array; the third control unit 240 is connected to the second register array unit 230, and is configured to generate a target color component of the pixel to be converted according to the second interpolation computation subarray and the second interpolation algorithm.
As shown in fig. 2, in one embodiment, the register output unit 300 includes a 1 × 1 register array 310, and the 1 × 1 register array 310 is connected to the second register array unit 230 for registering and outputting the target color component or the initial color component.
In one embodiment, the first interpolation sub-array is a 9 × 9 pixel array formed by selecting the initial color components of the peripheral pixels based on the buffer locations and centering on the initial color components of the corresponding pixels.
In one embodiment, the second interpolation computation subarray is a 5x5 pixel array generated based on the first interpolation algorithm.
Based on the above, the RAW image conversion circuit in the present application may specifically be as follows:
the buffer unit 110 may be a 9-line buffer composed of 9 blocks of RAM (Random Access Memory), and is configured to simultaneously and respectively buffer the initial color components of 9 rows of pixels in the initial RAW frame image. The first control unit 120 is used to control the input and output of the initial color components in the buffer unit 110.
Specifically, the cache unit 110 may cache 9 consecutive lines, for example, lines 1 to 9. When the 10 th row of pixels arrives one by one, the pixels of the 1 st row are covered one by one. The buffer unit 110 outputs 9 pixels at a time, which belong to one column of 9 different rows. The first control unit 120 adjusts the order of the 9 pixels, and outputs the 9 pixels having consecutive line numbers all the time. For example, the output of the buffer unit 110 may be the pixels in the same column of the 10 th, 2, 3 … 9 th row, and the first control unit 120 adjusts it to be the pixels in the same column of the 2 th, 3 th, 4 … 9, 10 th row. This has the advantage that the correct pixel position can be maintained when a new line arrives, without shifting all the data in the RAM.
First register array unit 210 is a 9x9 size register array. The second control unit 220 calculates a second interpolation calculation sub-array, registered in the second registration array unit 230, by using the first interpolation algorithm, with the first interpolation calculation sub-array registered in the first registration array unit 210 as a pixel array formed by the initial color components of the pixels in the initial RAW frame image, where the second registration array unit 230 is a 5 × 5 register array for storing the second interpolation calculation sub-array for each pixel, and updates the second interpolation calculation sub-array once every clock cycle. If the current pixel is a non-transformed pixel, then the corresponding interpolation calculation is not needed, that is, the corresponding second interpolation calculation sub-array is not needed, then only one color component value located at the center of the second register array unit 230 needs to be updated, and the color component value is updated to the color component value of the current pixel, in other words, the target color component of the non-transformed pixel is the same as the initial color component, and is not changed. The third control unit 240 is a combinational logic circuit that controls the second register array unit 230 to calculate a target color component or to transfer an initial color component that does not need to be calculated. The register output unit 300 is a 1 × 1 register array, and registers and outputs a target color component or an initial color component as an output of the RAW image conversion circuit.
It is understood that the output bandwidth of the RAW image conversion circuit can be kept stable due to the presence of the first control unit 120, and the transition between adjacent frames does not require a special setting of the interval time. For example, in the beginning stage of a frame, when the buffer unit 110 receives the 4 th row after receiving the 0 th, 1 st, 2 nd, and 3 rd rows, the first control unit 120 controls the buffer unit 110 to output the initial color components of the pixels in the zeroth to eighth rows, e.g., (0, n), (1, n), (2, n), (3, n), (0, n), (1, n), (2, n), (3, n), (4, n), in the nth column to the first register array unit 210. Thus, the first register array unit 210 does not need to consider whether to calculate the joint points, and all the points are calculated in a similar manner. At the next time, when the buffer unit 110 receives the nth column of the 4 th row, the register output unit 300, i.e. the 1 × 1 register array, registers and outputs the nth column of the 0 th row. When the buffer unit 110 is receiving the zeroth line at the beginning of the second frame, the first control unit 120 controls the buffer unit 110 to output to the first register array unit 210 the initial color components of (m-7, n), (m-6, n), (m-5, n), (m-4, n), (m-3, n), (m-2, n), (m-1, n), (m, n), (m-3, n) pixels, where m denotes the last line of the corresponding frame, and represents the last line of the first frame.
When the temporal buffer unit 110 receives the 0 th row and the n th column of the 2 nd frame, the 1 × 1 register array registers and outputs the color component corresponding to the n th column of the m-3 th row of the first frame, which is the initial color component or the target color component of the corresponding pixel. When the buffer unit 110 receives the 1 st row and the nth column of the second frame, the 1 × 1 register array registers and outputs the color component corresponding to the m-2 th row and the nth column of the first frame, which is the initial color component or the target color component of the corresponding pixel. When the buffer unit 110 receives the 2 nd row and the nth column of the second frame, the 1 × 1 register array registers and outputs the color component corresponding to the m-1 th row and the nth column of the first frame, which is the initial color component or the target color component of the corresponding pixel. When the buffer unit 110 receives the 3 rd row and the nth column of the second frame, the 1 × 1 register array registers and outputs the color component corresponding to the mth row and the nth column of the first frame, which is the initial color component or the target color component of the corresponding pixel. Therefore, 9 RAMs are always used, and the utilization efficiency is improved; and this is avoided: at the end of a frame, inputting an initial color component of a pixel, and outputting color components of 4 rows of pixels correspondingly; therefore, the input of the initial color component of the first pixel in the next frame needs to wait for the output time of the color components of the 4 rows of pixels. The benefits of this are: the camera can continuously output in a streaming mode, and the function of waiting in a crowded mode is not required to be designed; and the transmission bus can be designed according to the average bandwidth without designing excessive bandwidth allowance.
It should be noted that, as shown in fig. 3, since the image is input by one pixel every clock cycle and is input row by row, any calculation related to more than one pixel needs to be buffered. The output of the RAW image conversion circuit is also output one pixel line by line in each clock cycle, so that the connection mode between a camera and an ISP is not changed, and the integration is more convenient.
The buffer unit 110 receives the initial color components of the pixels line by line and buffers in the RAM, and the first control unit 120 reads out 9 pixels from different RAMs every clock cycle while controlling the reception of the initial color components of the pixels and the writing into the corresponding RAMs. When writing into the RAM, the 9 RAMs are written full and then written next, each RAM just stores the initial color component of a row of pixels, and when reading out, each RAM simultaneously outputs the initial color component of a pixel. Simultaneously output are 9 initial color components located at different rows of pixels. This design causes the buffer unit 110 to store exactly one first interpolation computation subarray corresponding to 9x9 pixels on the image, sliding a distance of one pixel to the right each clock cycle.
Since each time a target color component is calculated, it is necessary to know the color components corresponding to zero 4 pixels in 4 rows following the current pixel, the output of the RAW image conversion circuit described in this application is delayed by 4 rows of zero 4 pixels from the input.
For example, at the beginning line of the initial RAW frame image, the pixel line numbers for inputting to the buffer unit 110 are lines 1, 2, 3, 4, 0, 1, 2, 3, 4. The 0 th line represents the first line of the initial RAW frame image, that is, the position of no pixel before the 0 line is replaced by 1, 2, 3, 4 lines. At the leftmost side of each row, the pixel column numbers stored in the buffer unit 110 are columns 1, 2, 3, 4, 0, 1, 2, 3, and 4. At the rightmost side of each row, the column numbers stored in the buffer unit 110 are the N-5, N-4, N-3, N-2, N-1, N-5, N-4, N-3, and N-2 columns, where N is the number of pixel columns of the RAW frame image. This is designed because the input to the RAW image conversion circuit is one pixel per clock cycle, and thus is performed continuously without an interval between one drawing and the next. The RAW image conversion circuit needs to immediately switch from a state of calculating a color component of the last pixel of the last diagram to a state of calculating the first pixel of the next diagram. 5 of the 9 RAMs are used for storing initial color components of five rows of pixels in the previous figure, and the other 4 of the 9 RAMs are used for storing initial color components of four rows of pixels in the next figure. For example, the 5 th pixel from the last in the previous image is being computed at this time: the initial color components of 5 rows of zero 4 pixels in the 9 RAMs are of the previous figure and the initial color components of 3 rows of zero N-4 pixels are of the next figure. The initial color component of the last pixel of the last image is now being computed: the initial color component of 5 rows of zero 5 pixels in the 9 RAMs is of the previous map and the initial color component of 3 rows of zero N-5 pixels is of the next map, when the initial color component of the first pixel of the next map is being calculated.
As shown in fig. 4, in one embodiment, when the distribution positions of the pixels are located at the periphery of the initial RAW frame image and are not enough to form the corresponding interpolation calculation array, at least one 4 × 4 pixel array is virtually complemented at the periphery of the buffer position of the initial color component of the pixels, and the 4 × 4 pixel array is shown as the thick upper left-corner frame line in fig. 4; the 4-by-4 pixel array is a pixel array formed by the first four rows of color components and the first four columns of color components in the interpolation calculation array.
Referring to fig. 1 to 10, in one embodiment, the RAW image conversion circuit provided by the present application is a digital circuit, which is capable of converting a RAW image in rgbi 4x4 format into a RAW image in rgbi 2x2 format.
The RAW image conversion method provided by the application is also used for converting the RAW image in the RGBIR4x4 format into the RAW image in the RGBIR2x2 format. The specific conversion process is as follows:
as shown in fig. 5, each of the small squares shown thereon represents a pixel, and a RAW image is generally 1920 × 1080 pixels or other common size. Each pixel of the RAW image has only one color component, and after ISP processing, each pixel has a plurality of color components, and thus, can be used for displaying the image. For example, three color components of R (Red), G (Green), and B (Blue). The RAW image has only one color component per pixel, and this component may be R or G, or B or Ir (infrared light color component). The position of a pixel on the frame image determines the color components it has. In fig. 5, B indicates that the pixel at this position has a blue component, G indicates that the pixel at this position has a green component, R indicates that the pixel at this position has a red component, and Ir indicates that the pixel at this position has a gray-violet component. It can be seen that the pixel arrangement of rgbi 4x4 is cyclically repeated according to an initial pixel repetition array, which may be, but is not limited to, a bounding region as formed by the bold line frame in fig. 5, which corresponds to that shown in fig. 7. The initial pixel repeat array may also be a one-line pixel down-shift of the thick line frame shown in fig. 5, while shifting the surrounding area of one column of pixels to the right. The initial pixel repeat array is a 4x4 array of pixels.
The pixel arrangement in the RAW image in the RGBIr2x2 format is cyclically repeated according to a target pixel repetition array, which is a 2x2 pixel array, as a surrounding area formed by a thick line frame in fig. 6. Where Ir represents that the pixel at this location has an infrared light color component.
The operation of this conversion process is also repeated once for each 4x4 square grid cycle. As indicated by reference numerals in fig. 7, R components at positions B03 and B21 are calculated by interpolation, and the R components are used instead of the original B components. Then, the B component at positions of Ir10, Ir12, Ir30 and Ir32 is calculated and used instead of the original Ir component. Then, Ir components at the G11, G13, G31 and G33 positions are calculated and used to replace the original G component. The color components of the other pixels remain unchanged.
The specific interpolation calculation of the target color component corresponding to each pixel is to calculate the target color component of the current pixel according to the initial color components of the surrounding pixels. Through repeated calculation experiments, the visual effect and the calculation vision realization difficulty are chosen, and the obtained interpolation algorithm can be completed in two steps. Wherein, the first step: calculating a 5x 5G-plane from the initial color components of 81 pixel points in a 9x9 square around the current pixel point; the second step is that: and calculating the target color component of the current pixel needing to be interpolated according to the G-plane.
The method for calculating the G-plane comprises the following steps:
the method for calculating the G-plane is different according to the different positions of the color components of the current pixel in the repeating unit; the calculation method for each point on the G-plane is also different.
For example, as shown in the first diagram of the second row in fig. 8, at two points B03 and B21, the G-plane calculation method is as shown in the fourth diagram of the second row in fig. 8 (i.e., the rightmost diagram of the second row): the G-plane of the two points B03 and B21 is a square of 5x5, there are 25 pixels on the G-plane, such as the four gray pixels Ir11, Ir13, Ir31 and Ir33 represented by the second line of fig. 8, and the calculation method of the points on the four G-plane is 9-point interpolation, such as the first line of fig. 8, which is the value of itself (assumed to be 1) multiplied by 8 minus the value (-1) of the surrounding 8 points, and the values of the four points may not be added according to the setting, and then the points on the four G-plane may be obtained according to the setting, plus the value (1) of the four points nearest to the center point.
For example, at two points B03 and B21, their G-plane calculation method is shown in the fourth diagram of the second row in FIG. 8: the G-plane of the two points B03 and B21 is a square of 5x5, there are 25 pixels on the G-plane, such as 9 red and blue pixels B00, R02, B04, R20, B22, R24, B40, R42, and B44 represented by the third graph of the second row in fig. 8, and the calculation method of the points on the 9G-plane is 5-point interpolation, such as the second graph of the first row in fig. 8, the value of itself (assumed to be 1) is multiplied by 4 minus the value of the surrounding 4 points (-1), and the values of the four points closest to the center point (1) are added according to the setting, and the four point values (1) may not be added according to the setting, thus obtaining the color components corresponding to the points on the 9G-plane.
For example, at two points B03 and B21, their G-plane calculation method is shown in the fourth diagram of the second row in FIG. 8: g-planes at two points B03 and B21 are squares of 5 × 5, there are 25 pixels on the G-plane, such as 12 green pixels G10, G12, G14, G21, G23, G30, G32, G34, G41 and G43 shown in the fourth graph of the second row in fig. 8, and the 12 points on the G-plane directly copy the values on the original because the original color component at these 12 positions is G.
The size of the G-plane is 5x5, and when calculating the point on the G-plane, it is also based on the pixel square of 5x5 around its own position, so in this algorithm, the value of the pixel point in the square where one point is related to 9x9 around is calculated.
Each point needing conversion calculation has its own G-plane, and after the G-plane is obtained, the required interpolation result can be calculated according to the G-plane. The calculation method is shown in the fourth diagram of the first row in fig. 9: in the case of two points, B03 and B21, R02, Ir13, R24, Ir33, R42, Ir31, R20 and Ir11 on the G-plane are multiplied by-1, the product of the values of the four points, R02, R24, R42 and R20 and a coefficient can be added according to the setting, the product of the values of the four points can be also added according to the setting, then the product of G12, G23, G32, GB21 and-2 can be added according to the setting, and then the product of B22 and 16 can be added, so that the final calculation result of the point B03 or B21 can be obtained.
The four panels in the second row of fig. 9 show a flow chart of the final results from the G-plane for four points Ir10, Ir12, Ir30, Ir 32. Similar to the above, and will not be described herein.
The four panels in the third row of fig. 9 show a flow chart of the final results from the G-plane for four points G11, G13, G31, G33. Similar to the above, and will not be described herein.
As shown in fig. 4, for interpolation at the edge of an image, since calculating the converted value of a point requires the values of 81 points in a 9x9 square around the point, and for the point in row 0 and column 0, some points in the required 9x9 square have no values, the current algorithm uses a method of stitching the first 4 rows and 4 columns again to fill in the gap, so that the interpolation can be calculated at the point of the edge, and the image has no edge with abnormal color or texture visually. In fig. 4, the thick outline area at the upper left corner fills the vacant area, and correspondingly fills the first row and first column of the thick outline area at the lower right corner (0, 0).
As shown in fig. 10, in one embodiment, based on the above, the present application provides a RAW image conversion method, which includes the following steps:
step S10: and determining the buffer position of each pixel with the initial color component according to the distribution position of each pixel in the initial RAW frame image, and outputting an interpolation calculation array of the corresponding pixel based on the buffer position, wherein the pixel comprises at least one of a pixel to be converted and a non-converted pixel.
Step S20: and generating the pixel to be converted with the target color component according to the interpolation calculation array.
And step S30: and correspondingly outputting at least one of the non-conversion pixel with the initial color component and the pixel to be converted with the target color component according to the buffer position.
It can be understood that, the RAW image conversion method in this embodiment is based on the corresponding pixel stream accessed by the distribution location and outputs the corresponding pixel stream according to the corresponding cache location, so that it supports stream-wise input and stream-wise output of the pixel stream, and no interval is required between frame images, thereby improving the conversion efficiency; and the image which does not need to be converted can be directly output in a streaming mode, and the target RAW frame image in the corresponding RAW format can be output in a streaming mode after conversion according to the cache position of the corresponding pixel.
In the RAW image conversion circuit and the RAW image conversion method according to the present application, corresponding pixel data is processed, and the corresponding color component is only one of the pixel data.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The RAW image conversion circuit and the RAW image conversion method provided by the embodiments of the present application are described in detail above, and specific examples are applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiments is only used to help understand the technical solutions and the core ideas of the present application; those of ordinary skill in the art will understand that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications or substitutions do not depart from the spirit and scope of the present disclosure as defined by the appended claims.
Claims (10)
1. A RAW image conversion circuit, comprising:
an input processing unit, configured to determine a buffer location of each pixel having an initial color component according to a distribution location of each pixel in an initial RAW frame image, and output an interpolation calculation array of a corresponding pixel based on the buffer location, where the pixel includes at least one of a pixel to be converted and a non-conversion pixel;
the logic conversion unit is connected with the input processing unit and used for generating the pixel to be converted with the target color component according to the interpolation calculation array; and
and the register output unit is connected with the logic conversion unit and is used for correspondingly outputting at least one of the non-conversion pixel with the initial color component and the pixel to be converted with the target color component according to the cache position.
2. The RAW image conversion circuit according to claim 1, wherein the interpolation calculation array includes a first interpolation calculation sub-array and a second interpolation calculation sub-array; the input processing unit outputs the first interpolation calculation sub-array of the corresponding pixel based on the buffer position; and the logic conversion unit obtains a corresponding second interpolation calculation sub-array according to the first interpolation calculation sub-array, and generates a pixel to be converted with the target color component according to the second interpolation calculation sub-array.
3. The RAW image conversion circuit according to claim 2, wherein the input processing unit includes:
the cache unit is used for accessing and caching each pixel with the initial color component; and
and the first control unit is connected with the cache unit and used for controlling each pixel with the initial color component to be stored to the corresponding cache position according to the distribution position and controlling the cache unit to output a first interpolation calculation sub array of the corresponding pixel based on the cache position and a selection algorithm.
4. The RAW image conversion circuit according to claim 3, wherein the logic conversion unit includes:
the first register array unit is connected with the cache unit and used for registering the first interpolation calculation sub-array; and
and the second control unit is connected with the first register array unit and used for generating and outputting a corresponding second interpolation calculation sub-array according to the first interpolation calculation sub-array and the first interpolation algorithm.
5. The RAW image conversion circuit according to claim 4, wherein the logic conversion unit further includes:
a second register array unit connected to the first register array unit, for registering the second interpolation calculation sub-array; and
and the third control unit is connected with the second register array unit and used for generating the pixel to be converted with the target color component according to the second interpolation calculation sub-array and the second interpolation algorithm.
6. The RAW image conversion circuit according to claim 5, wherein the register output unit includes a 1x1 register array, and the 1x1 register array is connected to the second register array unit, and is configured to register and output at least one of the pixels to be converted having the target color component and the non-converted pixels having the initial color component.
7. The RAW image conversion circuit according to claim 2, wherein the first interpolation computation subarray is a 9x9 pixel array formed by selecting peripheral pixels having the initial color component centered on corresponding pixels having the initial color component based on the buffer locations.
8. The RAW image conversion circuit of claim 7, wherein the second interpolation computation subarray is a 5 by 5 pixel array generated based on a first interpolation algorithm.
9. The RAW image conversion circuit according to any of claims 1 to 8, wherein the distribution positions of the pixels are located at the periphery of the initial RAW frame image, and are insufficient to form the corresponding interpolation calculation arrays, at least one 4x4 pixel array is virtually complemented at the periphery of the buffer position of the pixel having the initial color component; the 4-by-4 pixel array is a pixel array formed by the first four rows of color components and the first four columns of color components in the interpolation calculation array.
10. A RAW image conversion method, comprising:
determining a cache position of each pixel with an initial color component according to the distribution position of each pixel in the initial RAW frame image, and outputting an interpolation calculation array of the corresponding pixel based on the cache position, wherein the pixel comprises at least one of a pixel to be converted and a non-converted pixel;
generating the pixel to be converted with the target color component according to the interpolation calculation array; and
and correspondingly outputting at least one of the non-conversion pixel with the initial color component and the pixel to be converted with the target color component according to the cache position.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110141248.XA CN112468789B (en) | 2021-02-02 | 2021-02-02 | RAW image conversion circuit and RAW image conversion method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110141248.XA CN112468789B (en) | 2021-02-02 | 2021-02-02 | RAW image conversion circuit and RAW image conversion method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112468789A true CN112468789A (en) | 2021-03-09 |
| CN112468789B CN112468789B (en) | 2021-04-23 |
Family
ID=74802391
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110141248.XA Active CN112468789B (en) | 2021-02-02 | 2021-02-02 | RAW image conversion circuit and RAW image conversion method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112468789B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6356276B1 (en) * | 1998-03-18 | 2002-03-12 | Intel Corporation | Median computation-based integrated color interpolation and color space conversion methodology from 8-bit bayer pattern RGB color space to 12-bit YCrCb color space |
| US20030034984A1 (en) * | 2001-08-10 | 2003-02-20 | Nobuo Murata | Apparatus and method of registration correction for video signal processor and a television camera having registration correcting function |
| US20040218073A1 (en) * | 2003-04-30 | 2004-11-04 | Nokia Corporation | Color filter array interpolation |
| CN1917616A (en) * | 2005-08-17 | 2007-02-21 | 宾得株式会社 | Photographing apparatus |
| CN101360247A (en) * | 2008-09-18 | 2009-02-04 | 北京中星微电子有限公司 | Method for image interpolation |
| CN103379343A (en) * | 2012-04-12 | 2013-10-30 | 索尼公司 | Image processing device, image processing method, and program |
| CN103765876A (en) * | 2011-08-31 | 2014-04-30 | 索尼公司 | Image processing device, image processing method, and program |
| CN106447597A (en) * | 2016-11-02 | 2017-02-22 | 上海航天控制技术研究所 | High-resolution image accelerated processing method based on parallel pipeline mechanism |
| CN108460805A (en) * | 2018-01-18 | 2018-08-28 | 西安理工大学 | A method of obtaining RGB and infrared image using traditional digital cameras |
| US10750195B2 (en) * | 2017-07-21 | 2020-08-18 | Samsung Electronics Co., Ltd. | Electronic device and method for encoding image data therein |
| CN112261391A (en) * | 2020-10-26 | 2021-01-22 | Oppo广东移动通信有限公司 | Image processing method, camera assembly and mobile terminal |
-
2021
- 2021-02-02 CN CN202110141248.XA patent/CN112468789B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6356276B1 (en) * | 1998-03-18 | 2002-03-12 | Intel Corporation | Median computation-based integrated color interpolation and color space conversion methodology from 8-bit bayer pattern RGB color space to 12-bit YCrCb color space |
| US20030034984A1 (en) * | 2001-08-10 | 2003-02-20 | Nobuo Murata | Apparatus and method of registration correction for video signal processor and a television camera having registration correcting function |
| US20040218073A1 (en) * | 2003-04-30 | 2004-11-04 | Nokia Corporation | Color filter array interpolation |
| CN1917616A (en) * | 2005-08-17 | 2007-02-21 | 宾得株式会社 | Photographing apparatus |
| CN101360247A (en) * | 2008-09-18 | 2009-02-04 | 北京中星微电子有限公司 | Method for image interpolation |
| CN103765876A (en) * | 2011-08-31 | 2014-04-30 | 索尼公司 | Image processing device, image processing method, and program |
| CN103379343A (en) * | 2012-04-12 | 2013-10-30 | 索尼公司 | Image processing device, image processing method, and program |
| CN106447597A (en) * | 2016-11-02 | 2017-02-22 | 上海航天控制技术研究所 | High-resolution image accelerated processing method based on parallel pipeline mechanism |
| US10750195B2 (en) * | 2017-07-21 | 2020-08-18 | Samsung Electronics Co., Ltd. | Electronic device and method for encoding image data therein |
| CN108460805A (en) * | 2018-01-18 | 2018-08-28 | 西安理工大学 | A method of obtaining RGB and infrared image using traditional digital cameras |
| CN112261391A (en) * | 2020-10-26 | 2021-01-22 | Oppo广东移动通信有限公司 | Image processing method, camera assembly and mobile terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112468789B (en) | 2021-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6587602B2 (en) | Resolution conversion system and method | |
| US7425933B2 (en) | Systems and methods for correcting green disparity in imager sensors | |
| JP3942569B2 (en) | Imaging apparatus and image data conversion method | |
| JP6251029B2 (en) | Control device, image processing device, control method, and program | |
| JP4317624B2 (en) | Image processing device | |
| JP6400331B2 (en) | Display panel driving apparatus and display panel driving method | |
| US8412003B2 (en) | Image processing methods, apparatus and computer program products using interdependent pixel interpolation operations | |
| CN112468789B (en) | RAW image conversion circuit and RAW image conversion method | |
| CN102739918A (en) | Image processing apparatus, method, recording medium, and program | |
| JP4621944B2 (en) | Image filter device, method and computer program | |
| US6816202B1 (en) | Picture frame generating circuit and digital television system using the same | |
| US8194150B2 (en) | Moving image processing apparatus and video camera apparatus using the same | |
| JP3272309B2 (en) | Pixel interpolation processing method and unit thereof, and digital image display device having the same | |
| JP6655685B2 (en) | Display panel driving device | |
| JP2001086355A (en) | Image processor and recording medium | |
| JP2010283588A (en) | High-definition video signal processor | |
| JP2012134626A (en) | Image processing device, image processing method, and imaging apparatus | |
| US6801671B1 (en) | Controlled image deterioration correction device with reduction/enlargement | |
| JP3444170B2 (en) | Field image interpolation method and field image interpolation device | |
| CN112788310B (en) | Image processing method and related image processing circuit | |
| KR100723046B1 (en) | Display driver and image prosessing apparatus for interpolating color and control method thereof | |
| WO2000057631A1 (en) | Image processing device and processing method | |
| TWI715269B (en) | Image processing method and associated image processing circuit | |
| JP2006154576A (en) | Gradation correction apparatus and gradation correction method | |
| US20220332126A1 (en) | Image processing apparatus, method of controlling the same, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |