CN117956300B - Image processing architecture, image processing method and image processing chip - Google Patents
Image processing architecture, image processing method and image processing chip Download PDFInfo
- Publication number
- CN117956300B CN117956300B CN202410350600.4A CN202410350600A CN117956300B CN 117956300 B CN117956300 B CN 117956300B CN 202410350600 A CN202410350600 A CN 202410350600A CN 117956300 B CN117956300 B CN 117956300B
- Authority
- CN
- China
- Prior art keywords
- pixel
- neighborhood
- type
- matrix
- pixels
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims abstract description 73
- 238000004364 calculation method Methods 0.000 claims abstract description 15
- 238000010586 diagram Methods 0.000 description 13
- 238000000034 method Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 8
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000012938 design process Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
Abstract
The disclosure provides an image processing architecture, an image processing method and an image processing chip, wherein the image processing architecture comprises a cache unit, a processing unit and a processing unit, wherein the cache unit is used for acquiring a neighborhood pixel matrix corresponding to a pixel to be processed in an image according to a preset neighborhood processing algorithm; the input processing unit is used for reading the neighborhood pixel matrix, and adjusting the pixel arrangement of the neighborhood pixel matrix to generate a neighborhood processing matrix corresponding to the pixels to be processed, wherein all green pixels in the neighborhood processing matrix are arranged in columns; the neighborhood processing unit is used for carrying out neighborhood processing calculation based on the neighborhood processing matrix according to a neighborhood processing algorithm so as to obtain a corresponding calculation result; and the output processing unit is used for recovering the neighborhood processing matrix into a neighborhood pixel matrix. Through the technical scheme provided by the disclosure, the positions of the pixels can be adjusted when the image pixels are input and output to the neighborhood window, so that the chip area is reduced, the chip circuit design is simplified, and the cost is reduced.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to an image processing architecture, an image processing method, and an image processing chip.
Background
Along with the high-speed development of image sensor technology, the resolution of an image sensor is higher and higher, the complexity of an image processing algorithm is continuously improved, so that the circuit resources required by an image processing chip in the process of realizing the image processing algorithm are also more and more, and the preparation cost of the chip is rapidly increased.
In the prior art, CMOS (Complementary Metal Oxide Semiconductor ) image sensors all support pixels in the form of bayer arrays. Specifically, fig. 1 shows a schematic diagram of a pixel array arranged in a bayer array, where G represents green pixels, R represents red pixels, and B represents blue pixels. As can be seen from fig. 1, the bayer array includes two types of rows, namely, a row in which red pixels and green pixels are arranged at intervals and a row in which blue pixels and green pixels are arranged at intervals, and the two types of rows are alternately output.
In the existing image processing algorithm, a neighborhood window of the current pixel is often needed to perform operation, wherein the neighborhood window is a pixel array with a plurality of rows and columns. By way of example, fig. 2 shows a schematic diagram of a 5 row 6 column neighborhood window. In the design process of the image processing chip, in order to realize a corresponding image processing algorithm, pixels in a neighborhood window can be pre-stored in a register, and then proper pixels are selected from the neighborhood window for operation according to the algorithm requirement. For example, when the image processing algorithm may calculate the average value of all green pixels in a neighborhood window of 5 rows and 6 columns, the image processing algorithm may need to calculate the average value of all green pixels in the neighborhood window of 5 rows and 6 columns separately: specifically, fig. 3 shows a first type of arrangement schematic diagram of a 5-row and 6-column neighborhood window, wherein a current row (i.e., a middle row in the neighborhood window, i.e., a3 rd row from top to bottom, and the same below) in the neighborhood window is an interval arrangement row of red pixels and green pixels, and a summation formula of all green pixels is as follows: gsum=g11+g13+g15+g22+g24+g26+g31+g33+g35+g42+g44+g46+g51+g53+g55, where G sum represents the sum of all green pixel values in the neighborhood window; gij represents the green pixel value at the j-th column position of the i-th row in the neighborhood window; i=1, 2,3,4,5; j=1, 2,3,4,5,6, the same applies below. Fig. 4 shows a second type of arrangement schematic diagram of a neighborhood window of 5 rows and 6 columns, in which rows are arranged at intervals between blue pixels and green pixels in the neighborhood window, and a summation formula of all green pixels is: gsum=g12+g14+g16+g21+g23+g25+g32+g34+g36+g41+g43+g45+g52+g54+g56.
It can be understood that two different calculation methods need to be designed according to the difference of the current line in the neighborhood window, two sets of circuits need to be applied in chip design, at least two adder circuit resources are needed, and the area of the adder circuit is larger, so that the area of the chip is increased, and the cost is increased.
Disclosure of Invention
Aiming at the problems in the prior art, the object of the present disclosure is to provide an image processing architecture, an image processing method and an image processing chip, which can simplify the chip circuit design, thereby reducing the chip area and the cost.
Specifically, the first aspect of the present disclosure provides an image processing architecture, which may specifically include:
the buffer memory unit is used for acquiring a neighborhood pixel matrix corresponding to the pixel to be processed in the image according to a preset neighborhood processing algorithm, and the neighborhood pixel matrix is arranged in a Bayer array;
The input processing unit is used for reading the neighborhood pixel matrix, and adjusting the pixel arrangement of the neighborhood pixel matrix to generate a neighborhood processing matrix corresponding to the pixels to be processed, wherein all green pixels in the neighborhood processing matrix are arranged in columns;
the neighborhood processing unit is used for carrying out neighborhood processing calculation based on the neighborhood processing matrix according to a neighborhood processing algorithm so as to obtain a corresponding calculation result;
and the output processing unit is used for recovering the neighborhood processing matrix into a neighborhood pixel matrix.
In a possible implementation of the first aspect, the neighborhood processing algorithm includes:
Based on a preset neighborhood window, acquiring the average value of all green pixels in a corresponding neighborhood pixel matrix; and/or
Acquiring the average value of all red pixels in a neighborhood pixel matrix; and/or
The average value of all blue pixels in the neighborhood pixel matrix is obtained.
In a possible implementation manner of the first aspect, the neighborhood pixel matrix includes a first type of pixel row in which red pixels and green pixels are arranged at intervals, and a second type of pixel row in which blue pixels and green pixels are arranged at intervals;
the input processing unit directly outputs the first type pixel row under the condition that the current input row is the first type pixel row;
the input processing unit is used for exchanging the positions of the blue pixels and the green pixels under the condition that the current input line is the second type of pixel line so as to generate a corresponding third type of pixel line and output the third type of pixel line.
In a possible implementation manner of the first aspect, the output processing unit directly outputs the first type of pixel row in the case of the current input row as the first type of pixel row;
and under the condition that the current input line is a third type pixel line, the output processing unit is used for exchanging the positions of the blue pixels and the green pixels to generate a corresponding second type pixel line and outputting the second type pixel line.
In a possible implementation manner of the first aspect, the neighborhood pixel matrix includes a first type of pixel row in which red pixels and green pixels are arranged at intervals, and a second type of pixel row in which blue pixels and green pixels are arranged at intervals;
The input processing unit is used for exchanging the positions of the red pixels and the green pixels under the condition that the current input line is a first type pixel line so as to generate a corresponding fourth type pixel line and output the fourth type pixel line;
The input processing unit directly outputs the second-type pixel row under the condition that the current input row is the second-type pixel row.
In a possible implementation manner of the first aspect, the output processing unit, in a case of a fourth type of pixel row of a current input row, pairs positions of red pixels and green pixels to generate corresponding first type of pixel rows and outputs the first type of pixel rows;
and the output processing unit directly outputs the second-type pixel row under the condition that the current input row is the second-type pixel row.
In a possible implementation of the first aspect described above, the input processing unit and the output processing unit are each implemented based on a multiplexer.
In a possible implementation manner of the first aspect, the input processing unit includes twice as many multiplexers as the number of bits of the pixel to be processed;
The output processing unit comprises twice as many multiplexers as the number of bits of the pixel to be processed.
A second aspect of the present disclosure provides an image processing method, which is applied to the image processing architecture provided in the foregoing first aspect, and specifically may include the following steps:
Obtaining a neighborhood pixel matrix corresponding to a pixel to be processed in an image according to a preset neighborhood processing algorithm, wherein the neighborhood pixel matrix is arranged in a Bayer array;
adjusting pixel arrangement of the neighborhood pixel matrix to generate a neighborhood processing matrix corresponding to the pixels to be processed, wherein all green pixels in the neighborhood processing matrix are arranged in columns;
And in the neighborhood window, carrying out neighborhood processing calculation based on the neighborhood processing matrix according to a neighborhood processing algorithm so as to obtain a corresponding calculation result.
A third aspect of the present disclosure provides an image processing chip comprising the image processing architecture provided in the foregoing first aspect.
Compared with the prior art, the method has the following beneficial effects:
According to the technical scheme, through adjusting the positions of the pixels when the image pixels are input and output to the neighborhood windows, the corresponding adder circuits are prevented from being arranged for the pixels of different neighborhood windows, the setting quantity of the adder circuits and the multiplexer circuits in chip design is reduced, and therefore the chip area is reduced, the chip circuit design is simplified, and the cost is reduced.
Drawings
Other features, objects and advantages of the present disclosure will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings.
Fig. 1 provides a schematic illustration of a pixel array arranged in a bayer array, according to the prior art.
Fig. 2 provides a schematic illustration of a 5 row 6 column neighborhood window according to the prior art.
Fig. 3 provides a schematic diagram of a first type of arrangement of a 5 row 6 column neighborhood window according to the prior art.
Fig. 4 provides a second type of arrangement schematic of a 5 row 6 column neighborhood window according to the prior art.
Fig. 5 provides a schematic structural diagram of an image processing architecture according to an embodiment of the present disclosure.
FIG. 6 provides a schematic diagram of a first type of pixel traveling through an image processing architecture, according to an embodiment of the present disclosure.
FIG. 7 provides a schematic diagram of a second type of pixel traveling through an image processing architecture, according to an embodiment of the present disclosure.
FIG. 8 provides a schematic diagram of a neighborhood processing matrix, according to an embodiment of the present disclosure.
Fig. 9 provides a flowchart of an image processing method according to an embodiment of the present disclosure.
Detailed Description
Other advantages and effects of the present disclosure will become readily apparent to those skilled in the art from the present disclosure, as the following detailed description proceeds, by way of specific examples. The disclosure may be practiced or carried out in other embodiments or applications, and details of the disclosure may be modified or changed from various points of view and applications without departing from the spirit of the disclosure. It should be noted that, without conflict, the embodiments of the present disclosure and features of the embodiments may be combined with each other.
The embodiments of the present disclosure will be described in detail below with reference to the attached drawings so that those skilled in the art to which the present disclosure pertains can easily implement the same. The present disclosure may be embodied in many different forms and is not limited to the embodiments described herein.
In the description of the present disclosure, references to the terms "one embodiment," "some embodiments," "examples," "particular examples," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present disclosure. Furthermore, the particular features, structures, materials, or characteristics may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples, as well as features of various embodiments or examples, presented in this disclosure may be combined and combined by those skilled in the art without contradiction.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the representations of the present disclosure, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.
For the purpose of clarity of the present disclosure, components that are not related to the description are omitted, and the same or similar components are given the same reference numerals throughout the specification.
Throughout the specification, when a device is said to be "connected" to another device, this includes not only the case of "direct connection" but also the case of "indirect connection" with other elements interposed therebetween. In addition, when a certain component is said to be "included" in a certain device, unless otherwise stated, other components are not excluded, but it means that other components may be included.
When a device is said to be "on" another device, this may be directly on the other device, but may also be accompanied by other devices therebetween. When a device is said to be "directly on" another device in contrast, there is no other device in between.
Although the terms first, second, etc. may be used herein to connote various elements in some instances, the elements should not be limited by the terms. These terms are only used to distinguish one element from another element. For example, a first interface, a second interface, etc. Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and/or "including" specify the presence of stated features, steps, operations, elements, components, items, categories, and/or groups, but do not preclude the presence, presence or addition of one or more other features, steps, operations, elements, components, items, categories, and/or groups. The terms "or" and/or "as used herein are to be construed as inclusive, or meaning any one or any combination. Thus, "A, B or C" or "A, B and/or C" means "any of the following: a, A is as follows; b, a step of preparing a composite material; c, performing operation; a and B; a and C; b and C; A. b and C). An exception to this definition will occur only when a combination of elements, functions, steps or operations are in some way inherently mutually exclusive.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the language clearly indicates the contrary. The meaning of "comprising" in the specification is to specify the presence of stated features, regions, integers, steps, operations, elements, and/or components, but does not preclude the presence or addition of other features, regions, integers, steps, operations, elements, and/or components.
Although not differently defined, including technical and scientific terms used herein, all terms have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The term addition defined in the commonly used dictionary is interpreted as having a meaning conforming to the contents of the related art document and the current hint, so long as no definition is made, it is not interpreted as an ideal or very formulaic meaning too much.
As can be appreciated based on the related description in the prior art, in the current process of designing an image processing chip, in order to cope with the difference of pixel arrangements in a neighborhood window, two adder circuits need to be designed, which occupies a large chip area. In order to reduce the arrangement of circuit resources and reduce the chip area as much as possible, in some embodiments of the present disclosure, two sets of adder circuits may be simplified into one set by using a multiplexer structure, specifically, referring to fig. 3 and fig. 4, the positions where the green pixels are located may be located according to the current row of red pixels and the green pixels arranged at intervals or the current row of blue pixels and the green pixels arranged at intervals, taking calculating the average value of the green pixels in the neighborhood window as an example, and specifically, may be described by referring to the natural language of the following algorithm:
If the current row is a row of red pixels and green pixels arranged at intervals, configuring G11_12 as G11; if the current row is not a row of red pixels and green pixels arranged at intervals, g11_12 is configured as G12.
Where g11_12 represents the green pixel value contained in the range from row 1, column 1 to row 1, column 2 in the neighborhood window, and the following related expressions can be similarly inferred, for example g41_42 represents the green pixel value contained in the range from row 4, column 1 to row 4, column 2 in the neighborhood window. As can be seen from comparing fig. 3 and 4, g11_12 is configured as G11 if the current row is a row of red pixels and green pixels, and g11_12 is configured as G12 if the current row is a row of blue pixels and green pixels. Accordingly, the positioning and value acquisition of the remaining green pixels can be described with reference to the natural language of the following algorithm:
If the current row is arranged at intervals of red pixels and green pixels, G13_14 is configured as G13, G15_16 is configured as G15, G21_22 is configured as G22, g23_24 is configured as G24, g25_26 is configured as G26, g31_32 is configured as G31, g33_34 is configured as G33, g35_36 is configured as G35, g41_42 is configured as G42, g43_44 is configured as G44, g45_46 is configured as G46, g51_52 is configured as G51, g53_54 is configured as G53, and g55_56=g55; if the current row is not a row with red and green pixels arranged at intervals, g13_14 is configured as G14, g15_16 is configured as G16, g21_22 is configured as G21, g23_24 is configured as G23, g25_26 is configured as G25, g31_32 is configured as G32, g33_34 is configured as G34, g35_36 is configured as G36, g41_42 is configured as G41, g43_44 is configured as G43, g45_46 is configured as G45, g51_52 is configured as G52, g53_54 is configured as G54, and g55_56 is configured as G56.
Based on the positioning and value acquisition of the green pixels, when all the green pixel corresponding values are acquired, the summation of all the green pixels can be realized by using only one set of adder circuit resources:
G_sum = G11_12 + G13_14 + G15_16 + G21_22 + G23_24 + G25_26 + G31_32 + G33_34 + G35_36 + G41_42 + G43_44 + G45_46 + G51_52 + G53_54 + G55_56.
It can be appreciated that through the above-mentioned technical scheme that this disclosure provided, adder circuit resources have been practiced thrift greatly, but need increase more multiplexers, consider that the area of multiplexer circuit is about to be less than the adder circuit far away, the chip area still can effectively be reduced to the setting of above-mentioned technical scheme, reaches the effect of simplifying circuit design and reduce cost.
In the above embodiment, it is understood that assuming that a neighborhood window of 5 rows and 6 columns is used in the image processing, and assuming that the number of bits of a single pixel is 12 bits, 180 multiplexers are required for 15 green pixels appearing in the single neighborhood window. Similarly, when calculating the average value of the red pixel and the blue pixel, the same technical scheme can be used to save adder resources, but 180 multiplexers are also added, so for a neighborhood window of 5 rows and 6 columns, 360 multiplexers are used in total, and as the neighborhood window increases, the number of required multiplexers increases.
In order to optimize the situation that the number of multiplexers is large in the technical scheme, the design of the image processing chip is further optimized, an image processing architecture is provided in some embodiments of the present disclosure, and a large amount of multiplexer resources are saved by adjusting the positions of pixels when the image pixels are input and output to the neighborhood window. Specifically, fig. 5 shows a schematic structural diagram of an image processing architecture. As shown in fig. 5, specifically, the method may include:
the buffer unit 501 is configured to obtain a neighborhood pixel matrix corresponding to a pixel to be processed in the image according to a preset neighborhood processing algorithm. Wherein, the neighborhood pixel matrix is arranged in a Bayer array. It can be understood that the technical solutions discussed in the present disclosure are all established on the input/output format in which the pixels are arranged in bayer array, and will not be described in detail below. In a specific implementation of the foregoing embodiment, the buffer unit 501 may be a memory or other configurable registers, which is not limited herein.
The input processing unit 502 is configured to read the neighborhood pixel matrix, and adjust the pixel arrangement of the neighborhood pixel matrix to generate a neighborhood processing matrix corresponding to the pixel to be processed. All green pixels in the neighborhood processing matrix are arranged in columns, and the generation and structure of the neighborhood processing matrix will be described in detail later.
The neighborhood processing unit 503 is configured to perform neighborhood processing calculation based on the neighborhood processing matrix according to a neighborhood processing algorithm, so as to obtain a corresponding calculation result. Specifically, the neighborhood processing algorithm may include obtaining an average value of all green pixels in the corresponding neighborhood pixel matrix based on a preset neighborhood window, and/or obtaining an average value of all red pixels in the corresponding neighborhood pixel matrix, and/or obtaining an average value of all blue pixels in the corresponding neighborhood pixel matrix, which are not described herein.
An output processing unit 504, configured to restore the neighborhood processing matrix to a neighborhood pixel matrix. Specifically, the output processing unit 504 and the input processing unit 502 may be implemented by multiplexers. In the above embodiment, in order to simultaneously consider the processing speed and the number of multiplexers, the input processing unit 502 and the output processing unit 504 may be set to input two pixels at a time for processing, so that the number of multiplexers required by the output processing unit 504 and the input processing unit 502 is twice the number of bits of the pixel to be processed, and also taking the number of bits of a single pixel as 12 bits as an example, 360 multiplexers are required in the above embodiment, while the total number of multiplexers required in the above embodiment is 48, so that the number of multiplexers is greatly reduced (the number of multiplexers is reduced by 86.67%); the magnitude of the reduction of the multiplexers is more pronounced if the number of pixels contained within the neighborhood window is greater, thereby achieving the goals of reduced chip area, simplified chip circuit design, and reduced cost.
Further, it can be appreciated that when the number of pixels included in the neighborhood window is greater, the processing speed can be further increased by adjusting the number of pixels processed by the input processing unit 502 and the output processing unit 504 at a time: if the number of pixels processed by the input processing unit 502 and the output processing unit 504 is four at a time, the number of multiplexers required by the output processing unit 504 and the input processing unit 502 is four times the number of bits of the pixels to be processed, and so on. Those skilled in the art may input the number of pixels processed by the processing unit 502 and the output processing unit 504 at a time according to actual needs, so as to achieve both processing efficiency and reduction of the number of multiplexers, which is not limited herein.
The specific implementation of the above-mentioned buffer unit 501 to the output processing unit 504 will be further described below:
In the above embodiment, specifically, referring to fig. 3 and 4, it can be seen that the rows of the neighborhood pixel matrix in which the red pixels and the green pixels are arranged at intervals may be set as the first type of pixel rows, and the rows of the neighborhood pixel matrix in which the blue pixels and the green pixels are arranged at intervals may be set as the second type of pixel rows. Accordingly, FIG. 6 shows a schematic diagram of a first type of pixel traveling through the image processing architecture, and FIG. 7 shows a schematic diagram of a second type of pixel traveling through the image processing architecture: as can be seen from fig. 6, the input processing unit 502 directly outputs the first type pixel row in the case of the current input row as the first type pixel row; as shown in fig. 7, it can be seen that, in the case of the second type of pixel row of the current input row, the input processing unit 502 needs to exchange the positions of the blue pixels and the green pixels to generate the corresponding third type of pixel row and output the third type of pixel row. The following will describe in detail how the pixel position pair is implemented by the multiplexer:
In the above embodiment, assuming that the input processing unit 502 is set to process two pixels at a time, the two pixels are set to be Pixel [0] and Pixel [1] according to the input sequence, and the Pixel [0] is input to the input processing unit before the Pixel [1], the Pixel transformation process as shown in fig. 6 and 7 can be described by using the following natural language:
If the current row is a row of red pixels and green pixels arranged at intervals, the green Pixel bit is configured as Pixel [0], and the red Pixel bit is configured as Pixel [1]; if the current row is not the row of the red Pixel and the green Pixel which are arranged at intervals, the blue Pixel bit is configured as Pixel [0] and the green Pixel bit is configured as Pixel [1], so that the Pixel position is exchanged.
Accordingly, for the output processing unit 504, as shown in fig. 6, in the case of the first type pixel row of the current input row, the first type pixel row may be directly output; in the case of the third type of pixel row of the current input row, as shown in fig. 7, the positions of the blue pixels and the green pixels may be exchanged to generate the corresponding second type of pixel row and output the second type of pixel row. In the case where the input processing unit 502 is set to process two pixels at a time, the output processing unit 504 processes two pixels at the same time; correspondingly, the pixel transformation process shown in fig. 6 and 7 can be described in the following natural language:
If the current row is a row of red pixels and green pixels arranged at intervals, configuring Pixel [0] as a green Pixel bit, and configuring Pixel [1] as a red Pixel bit; if the current row is not the row of the red Pixel and the green Pixel which are arranged at intervals, the Pixel [0] is configured as a blue Pixel bit, and the Pixel [1] is configured as a green Pixel bit, so that the recovery of the Pixel position is realized.
In the above embodiment, fig. 8 shows a schematic diagram of a neighborhood processing matrix, as can be seen from fig. 8, after the positions of the input processing units 502 are exchanged, all green pixels in the neighborhood processing matrix are arranged in rows at intervals, and the positions of the green pixels relative to the red pixels and the blue pixels are fixed in the processes of calculating the average value, summing, and the like, so that the sum of all pixel values can be obtained only by one adder circuit.
In the above embodiment, in another implementation, the input processing unit 502 may exchange the positions of the red pixels and the green pixels to generate a corresponding fourth type of pixel row and output the fourth type of pixel row in the case that the current input row is the first type of pixel row; and in the case of the second type pixel row of the current input row, the second type pixel row is directly output. Correspondingly, the output processing unit 504 needs to exchange the positions of the red pixels and the green pixels to generate corresponding first-type pixel rows and output the first-type pixel rows under the condition that the fourth-type pixel row is the current input row; and directly outputting the second-type pixel row under the condition that the current input row is the second-type pixel row. It will be appreciated that a neighborhood processing matrix similar to that shown in fig. 8 can be obtained with the relative positions of the green pixels with respect to the red and blue pixels fixed, whether the first type of pixel is line-position adjusted or the second type of pixel is line-position adjusted.
In some embodiments of the present disclosure, fig. 9 provides a flowchart of an image processing method, where the image processing method is applied to the image processing architecture provided in the foregoing embodiments, as shown in fig. 9, and may specifically include the following steps:
Step 901: and acquiring a neighborhood pixel matrix corresponding to the pixel to be processed in the image according to a preset neighborhood processing algorithm. Wherein, the neighborhood pixel matrix is arranged in a Bayer array.
Step 902: and adjusting the pixel arrangement of the neighborhood pixel matrix to generate a neighborhood processing matrix corresponding to the pixel to be processed. Wherein all green pixels in the neighborhood processing matrix are arranged in columns;
Step 903: and in the neighborhood window, carrying out neighborhood processing calculation based on the neighborhood processing matrix according to a neighborhood processing algorithm so as to obtain a corresponding calculation result.
In some embodiments of the present disclosure, there is also provided an image processing chip including the image processing architecture provided by the foregoing embodiments. It can be understood that the number of multiplexers and adders included in the image processing chip obtained by the image processing architecture provided in the foregoing embodiment is greatly reduced,
In summary, through the technical scheme provided by the disclosure, through adjusting the positions of the pixels when the image pixels are input and output to the neighborhood windows, the corresponding adder circuits are prevented from being arranged for the pixels of the different neighborhood windows, the number of the adder circuits and the multiplexer circuits in the chip design is reduced, and therefore the chip area is reduced, the chip circuit design is simplified, and the cost is reduced.
The foregoing is a further detailed description of the present disclosure in connection with the specific preferred embodiments, and it is not intended that the practice of the present disclosure be limited to such description. It will be apparent to those skilled in the art to which the present disclosure pertains that several simple deductions or substitutions may be made without departing from the spirit of the disclosure, all of which should be considered to fall within the scope of the present disclosure.
Claims (10)
1. An image processing architecture, comprising:
The buffer memory unit is used for acquiring a neighborhood pixel matrix corresponding to the pixel to be processed in the image according to a preset neighborhood processing algorithm, and the neighborhood pixel matrix is arranged in a Bayer array;
The input processing unit is used for reading the neighborhood pixel matrix, and adjusting the pixel arrangement of the neighborhood pixel matrix to generate a neighborhood processing matrix corresponding to the pixel to be processed, wherein all green pixels in the neighborhood processing matrix are arranged in columns;
The neighborhood processing unit is used for carrying out neighborhood processing calculation based on the neighborhood processing matrix according to the neighborhood processing algorithm so as to obtain a corresponding calculation result;
And the output processing unit is used for recovering the neighborhood processing matrix into the neighborhood pixel matrix.
2. The image processing architecture of claim 1, wherein the neighborhood processing algorithm comprises:
Based on a preset neighborhood window, acquiring the average value of all green pixels in the corresponding neighborhood pixel matrix; and/or
Acquiring the average value of all red pixels in the neighborhood pixel matrix; and/or
And obtaining the average value of all blue pixels in the neighborhood pixel matrix.
3. The image processing architecture of claim 1, wherein the neighborhood pixel matrix comprises a first type of pixel rows with red pixels and green pixels spaced apart, and a second type of pixel rows with blue pixels and green pixels spaced apart;
The input processing unit directly outputs a first type pixel row under the condition that the current input row is the first type pixel row;
And the input processing unit is used for exchanging the positions of the blue pixels and the green pixels under the condition that the current input line is a second type pixel line so as to generate a corresponding third type pixel line and outputting the third type pixel line.
4. An image processing architecture according to claim 3, wherein the output processing unit directly outputs a first type of pixel row in case of a current input row of the first type of pixel row;
And the output processing unit is used for exchanging the positions of the blue pixels and the green pixels under the condition that the current input line is a third type pixel line so as to generate the corresponding second type pixel line and output the second type pixel line.
5. The image processing architecture of claim 1, wherein the neighborhood pixel matrix comprises a first type of pixel rows with red pixels and green pixels spaced apart, and a second type of pixel rows with blue pixels and green pixels spaced apart;
The input processing unit is used for exchanging the positions of the red pixels and the green pixels under the condition that the current input line is a first type pixel line so as to generate a corresponding fourth type pixel line and output the fourth type pixel line;
And the input processing unit directly outputs the second type pixel row under the condition that the current input row is the second type pixel row.
6. The image processing architecture of claim 5, wherein the output processing unit, in the case of a fourth type of pixel row of a current input row, swaps the positions of the red pixels and the green pixels to generate the corresponding first type of pixel row and outputs the first type of pixel row;
and the output processing unit directly outputs the second type pixel row under the condition that the current input row is the second type pixel row.
7. The image processing architecture of claim 1, wherein the input processing unit and the output processing unit are each implemented based on a multiplexer.
8. The image processing architecture of claim 7, wherein the input processing unit includes twice as many multiplexers as the number of bits of the pixel to be processed;
The output processing unit comprises the multiplexer with the number twice of the bit number of the pixel to be processed.
9. An image processing method applied to the image processing architecture according to any one of claims 1 to 8, comprising the steps of:
Obtaining a neighborhood pixel matrix corresponding to a pixel to be processed in an image according to a preset neighborhood processing algorithm, wherein the neighborhood pixel matrix is arranged in a Bayer array;
adjusting pixel arrangement of the neighborhood pixel matrix to generate a neighborhood processing matrix corresponding to the pixels to be processed, wherein all green pixels in the neighborhood processing matrix are arranged in columns;
And in a neighborhood window, carrying out neighborhood processing calculation based on the neighborhood processing matrix according to the neighborhood processing algorithm so as to obtain a corresponding calculation result.
10. An image processing chip comprising an image processing architecture as claimed in any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410350600.4A CN117956300B (en) | 2024-03-25 | 2024-03-25 | Image processing architecture, image processing method and image processing chip |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410350600.4A CN117956300B (en) | 2024-03-25 | 2024-03-25 | Image processing architecture, image processing method and image processing chip |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117956300A CN117956300A (en) | 2024-04-30 |
| CN117956300B true CN117956300B (en) | 2024-05-28 |
Family
ID=90805510
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410350600.4A Active CN117956300B (en) | 2024-03-25 | 2024-03-25 | Image processing architecture, image processing method and image processing chip |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117956300B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1389876A1 (en) * | 2002-08-12 | 2004-02-18 | STMicroelectronics Limited | Colour image sensor with hexagonal shaped pixels |
| WO2013189529A1 (en) * | 2012-06-20 | 2013-12-27 | Robert Bosch Gmbh | Imaging system, process for detection of sensor impulse noise and computer program |
| CN105472388A (en) * | 2015-12-08 | 2016-04-06 | 浙江大华技术股份有限公司 | Color filter array image encoding-decoding method, device and system |
| CN109345464A (en) * | 2018-07-30 | 2019-02-15 | 深圳市艾为智能有限公司 | A kind of method and system of image procossing that realizing HDR in Bayer data field |
| CN112752009A (en) * | 2019-10-29 | 2021-05-04 | 中兴通讯股份有限公司 | Image processing method, module, readable storage medium and image sensor |
| WO2022061879A1 (en) * | 2020-09-28 | 2022-03-31 | 深圳市大疆创新科技有限公司 | Image processing method, apparatus and system, and computer-readable storage medium |
| CN114723620A (en) * | 2022-03-01 | 2022-07-08 | 北京奕斯伟计算技术有限公司 | Image processing method and device, electronic equipment and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5006067B2 (en) * | 2006-06-29 | 2012-08-22 | オリンパス株式会社 | Image processing apparatus, image processing program, and image processing method |
| US20090295934A1 (en) * | 2008-05-27 | 2009-12-03 | The Hong Kong University Of Science And Technology | Color demosaicking using direction similarity in color difference spaces |
| JP5719986B2 (en) * | 2010-10-20 | 2015-05-20 | パナソニックＩｐマネジメント株式会社 | Image processing apparatus and image processing method |
-
2024
- 2024-03-25 CN CN202410350600.4A patent/CN117956300B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1389876A1 (en) * | 2002-08-12 | 2004-02-18 | STMicroelectronics Limited | Colour image sensor with hexagonal shaped pixels |
| WO2013189529A1 (en) * | 2012-06-20 | 2013-12-27 | Robert Bosch Gmbh | Imaging system, process for detection of sensor impulse noise and computer program |
| CN105472388A (en) * | 2015-12-08 | 2016-04-06 | 浙江大华技术股份有限公司 | Color filter array image encoding-decoding method, device and system |
| CN109345464A (en) * | 2018-07-30 | 2019-02-15 | 深圳市艾为智能有限公司 | A kind of method and system of image procossing that realizing HDR in Bayer data field |
| CN112752009A (en) * | 2019-10-29 | 2021-05-04 | 中兴通讯股份有限公司 | Image processing method, module, readable storage medium and image sensor |
| WO2022061879A1 (en) * | 2020-09-28 | 2022-03-31 | 深圳市大疆创新科技有限公司 | Image processing method, apparatus and system, and computer-readable storage medium |
| CN114723620A (en) * | 2022-03-01 | 2022-07-08 | 北京奕斯伟计算技术有限公司 | Image processing method and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117956300A (en) | 2024-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP0424119A2 (en) | Recycling DCT/IDCT integrated circuit apparatus using a single multiplier/accumulator and a single random access memory | |
| EP0518462B1 (en) | Filtering system and operation method for very high-speed image processing | |
| US8154636B2 (en) | Image enhancement using hardware-based deconvolution | |
| US8189944B1 (en) | Fast edge-preserving smoothing of images | |
| CN107124609A (en) | A kind of processing system of video image, its processing method and display device | |
| WO2006050290A2 (en) | Transferring a video frame from memory into an on-chip buffer for video processing | |
| US10347220B1 (en) | Data compression and decompression method for DeMura table | |
| JP2012033007A (en) | Image processing device and image processing method | |
| KR20000047576A (en) | Signal processing distributed arithmetic architecture | |
| CN113747066B (en) | Image correction method, image correction device, electronic equipment and computer readable storage medium | |
| CN117956300B (en) | Image processing architecture, image processing method and image processing chip | |
| CN111083366B (en) | Image processing method, device and system based on data segmentation | |
| US8902474B2 (en) | Image processing apparatus, control method of the same, and program | |
| US20040263653A1 (en) | Image detection processing device | |
| US11663453B2 (en) | Information processing apparatus and memory control method | |
| JP2006201914A (en) | Unit, method, and computer program for image filtering | |
| US8666172B2 (en) | Providing multiple symmetrical filters | |
| CN108805846B (en) | Method and system for optimizing binary image processing | |
| CN115775196A (en) | Hardware implementation method of bilinear interpolation | |
| CN112330525A (en) | Image processing method, electronic device, and non-volatile computer-readable storage medium | |
| JP2016095667A (en) | Image processing device and electronics apparatus | |
| US20170132750A1 (en) | Multi-line image processing with parallel processing units | |
| Zhang et al. | Anti-Aliasing and Anti-Color-Artifact Demosaicing for High-Resolution CMOS Image Sensor | |
| CN116227507B (en) | Arithmetic device for performing bilinear interpolation processing | |
| US20090030964A1 (en) | Matrix operation device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |