CN105279765A - System and method for detecting color target based on FPGA (field programmable gate array) - Google Patents

Abstract

The invention provides a system and a method for detecting a color target based on an FPGA (field programmable gate array). The system comprises a Bayer signal receiving module, a Bayer format interpolation to RGB module, an R, G, B three-path color component addition module, an R, G, B three-path color component left shifting module, a division module, and a color judgment and binarization target module. The above modules are all realized in the FPGA. The system and the method provided by the invention can detect a target with color feature information for a high-resolution image in real time, and segment the target from the high-resolution image, thereby carrying out follow-up processing such as target tracking.

Description

A kind of color target detection system based on FPGA and method

Technical field

The invention belongs to the real-time detection and tracking field of target, be specifically related to a kind of color target detection system based on FPGA and method.

Background technology

Machine vision is the science of an Applied Computer Techniques simulation people and biological vision, utilizes robot to replace human eye to carry out measuring and judging target. wherein vision tracking is an important branch of machine vision.

In vision is followed the tracks of, primary work uses vision system to carry out target detection exactly, only can detection and tracking target on the correct basis confirming target, target detection process is exactly the process of image analysis processing, selects suitable image processing method can improve real-time and the robustness of target detection.

Existing color target real-time detection method mainly Based PC and camera system color region object detection method and based on embedded color region object detection method.The color region object detection method of Based PC and camera system is linked in PC by camera, then under the pure software environment such as MicrosoftVisualStudio, target detection is carried out, but this detection method needs to buy the capture card supporting with camera, expensive, and because be pure software environment, image processing speed is very slow, does not reach the requirement of real-time; Have based on embedded color region object detection method and realize in ARM, MCU and FPGA, employ NiosII embedded central processing unit based on the Embedded color region object detection method of FPGA, algorithm travelling speed is slower.These traditional methods, owing to being subject to the restriction of PC performance and digital signal processing chip performance, Color Image Processing ability can only complete the quasi real time process of 300,000 pixel images mostly, cannot process more high-resolution image in real time.

Summary of the invention

The object of the present invention is to provide a kind of color target detection system based on FPGA and method, target with color characteristic information can be detected in real time to high-resolution image, and be split, thus carry out the follow-up process such as target following.

In order to solve the problems of the technologies described above, the invention provides a kind of color target real-time detecting system based on FPGA, comprising Bayer signal receiving module; Bayer form interpolation turns RGB module; R, G, B tri-road color component summation module; R, G, B tri-road color component to move to left module; Dividing module; Color judges and binaryzation object module; Above-mentioned module all realizes in FPGA.

The present invention also provides a kind of color target detection method based on FPGA, comprises the following steps:

Step one, color according to target to be split in image, set each pixel in region to be split whether belong to this color R, G, B tri-road color component account for the decision threshold of color sum-rate;

Step 2, the image of Bayer form is converted into the image of rgb format;

Step 3, extract rgb format image in R, G, B tri-road color component, by the rear color total amount obtaining each pixel of R, G, B tri-road color component addition, then respectively to R, G, B tri-road color component to move to left N position, the span of N between 4 to 32, obtain by R, G, B tri-road color component expand 2 Nth power doubly after component value

Step 4: calculate component accounts for the ratio of color total amount respectively;

Step 5: decision threshold step one set expands the Nth power of 2 doubly, then judges the Nth power whether ratio that three road color components account for color total amount meets expansion 2 doubly after color threshold Rule of judgment, if met, be then 1 by this pixel assignment, if do not met, be 0 by this pixel assignment, obtain the binary image of target affiliated area to be tracked, for the process that target following is follow-up.

Compared with prior art, its remarkable advantage is in the present invention: the manageable bandwidth of (1) the present invention can reach more than Gpbs, in high resolving power high frame rate image procossing, have significant advantage; (2) the present invention departs from NiosII embedded central processing unit in FPGA, adopts pure hardware logic, and processing speed can reach more than 60fps, has the detection that very strong real-time achieves color region target; (3) ratio that each for color component accounts for color total amount is set to threshold value by the present invention, makes the condition differentiating color more abundant, thus reaches more accurate effect.

Accompanying drawing explanation

Fig. 1 is a kind of color target detection system structural representation based on FPGA of the present invention.

Fig. 2 is a kind of color target detection method process flow diagram based on FPGA of the present invention.

Fig. 3 is Bayer picture format schematic diagram.

Fig. 4 is four kinds of array mode schematic diagram that in Bayer format-pattern, R, G, B component exists.

Fig. 5 is the image as detected object in emulation experiment of the present invention.

Fig. 6 is the testing result image obtained in emulation experiment of the present invention.

Embodiment

As shown in Figure 1, a kind of color target detection system based on FPGA of the present invention, comprises Bayer signal receiving module; Bayer form interpolation turns RGB module; R, G, B tri-road color component summation module; R, G, B tri-road color component to move to left module; Dividing module; Color judges and binaryzation object module; Above-mentioned module all realizes in FPGA.The overprogram that FPGA is intrinsic, make the function of present system hardware can as software flexible and programmed configurations easily, and not only can the processing speed of elevator system, ensure the Effec-tive Function of algorithm, can also system bulk be reduced, be convenient to application and popularize.

Bayer signal receiving module, for receiving the picture signal of Bayer form, then sending into Bayer form interpolation and turning RGB module by the picture signal of Bayer form;

Bayer form interpolation turns GRB module for the image of Bayer form being converted to the image of rgb format, and extract the R road color component of each pixel in rgb format image, G road color component and B road color component, then by R, G, B tri-road color component send into respectively simultaneously R, G, B tri-road color component summation module and R, G, B color component to move to left module;

R, G, B tri-road color component summation module for calculating R, G, B tri-total amount of road color component, then color total amount is sent to dividing module;

R, G, B color component move to left module for by R, G, B tri-road color component to move to left respectively N position, the span of N is 4 to 32, obtain by R, G, B tri-road color component expand 2 Nth power doubly after three road color components, and will three road color components are sent to dividing module;

Dividing module is for calculating each pixel three road color components account for the ratio of color total amount separately, and ratio is sent to color determination module;

The ratio that color judges and the color component value of each pixel is accounted for color total amount by binaryzation object module compares with the color of object decision threshold condition preset, the pixel meeting threshold condition is the pixel meeting color of object to be tracked, also namely this pixel belongs to the pixel in target area to be tracked, is 1 by this pixel assignment; The pixel not meeting threshold condition is the pixel not meeting color of object to be tracked, also namely this pixel belongs to the pixel beyond target area to be tracked, be 0 by this pixel assignment, thus obtain the binary image of target affiliated area to be tracked, then sent to by binary image rear end to do target following process.This method solves a floating-point operation difficult problem in FPGA dexterously, makes to detect target at color region and reaches more accurate effect.

As shown in Figure 2, a kind of color target detection method based on FPGA of the present invention, comprises the following steps:

Step one, color according to target to be split in image, set each pixel in region to be split whether belong to this color R, G, B tri-road color component account for the decision threshold of color sum-rate, such as:

When the color of target to be split is red, if in image certain pixel R, G, B tri-road color component account for color sum-rate and meet following threshold decision condition, then this pixel be redness, and namely this pixel belongs to the pixel in target area to be split,

Belong to the threshold decision condition of red pixel: $\left\{\begin{array}{c}r=R/(R+G+B)>0.4\\ g=G/(R+G+B)<0.3\\ b=B/(R+G+B)<0.2\end{array}\right.;$

When the color of target to be split is green, if in image certain pixel R, G, B tri-road color component account for color sum-rate and meet following threshold decision condition, then this pixel be green, and namely this pixel belongs to the pixel in target area to be split,

Belong to the threshold decision condition of green pixel: $\left\{\begin{array}{c}r=R/(R+G+B)<0.4\\ g=G/(R+G+B)>0.4\\ b=B/(R+G+B)<0.2\end{array}\right.$

When the color of target to be split is blue, if in image certain pixel R, G, B tri-road color component account for color sum-rate and meet following threshold decision condition, then this pixel be blueness, and namely this pixel belongs to and to treat point

Belong to the threshold decision condition of green pixel: $\left\{\begin{array}{c}r=R/(R+G+B)<0.4\\ g=G/(R+G+B)<0.3\\ b=B/(R+G+B)>0.3\end{array}\right.$

Wherein, R, G, B are respectively R, G, B tri-road color component, and r, g, b are respectively the ratio that R color component, G color component or B color component account for R, G, B color total amount.

Step 2, the image of Bayer form is converted into the image of rgb format, detailed process is:

Interpolation arithmetic is used the picture signal of Bayer form to be converted into the picture signal of rgb format, Bayer picture format as shown in Figure 3, in figure 3, each grid represents a pixel, and each pixel contains the one in R, G, B tri-kinds of components, such as, the pixel of odd-numbered line is alternately made up of R, G component, and the pixel of even number line is alternately made up of G, B component, and wherein G component accounts for the half of all pixels, R component and B component account for second half of all pixels altogether, and namely G component is 2 times of R, B component.Therefore, when the image of Bayer form is converted into the image of rgb format, need to carry out interpolation to G component, using R, G, B color component of R, G, B component after interpolation arithmetic as rgb format image, thus the picture format of Bayer form is converted into the image of rgb format.As shown in Figure 4, in Bayer format-pattern, there are four kinds of array modes in R, G, B component, four kinds of array modes corresponding four kinds of interpolation calculation modes respectively.

The first situation is: the first row is from left to right R, G component, and the second row is from left to right G, B component, and what this situation was corresponding carries out the mode of interpolation as shown in formula (1) to first element in the lower right corner,

B＝B ₂₂,R＝R ₁₁,G＝(G ₁₂+G ₂₁)/2；(1)

The second situation is: the first row is from left to right B, G component, and the second row is from left to right G, R component, and what this situation was corresponding carries out the mode of interpolation as shown in formula (2) to first element in the lower right corner,

G＝(G ₁₁+G ₂₂)/2；B＝B ₁₂,R＝R ₂₁；(2)

The third situation is: the first row is from left to right G, B component, and the second row is from left to right R, G component, and what this situation was corresponding carries out the mode of interpolation as shown in formula (3) to first element in the lower right corner,

G＝(G ₁₁+G ₂₂)/2；B＝B ₁₂,R＝R ₂₁；(3)

4th kind of situation is: the first row is from left to right G, R component, and the second row is from left to right B, G component, and what this situation was corresponding carries out the mode of interpolation as shown in formula (4) to first element in the lower right corner,

G＝(G ₁₁+G ₂₂)/2,R＝R ₁₂，B＝B ₂₁；(4)

In above formula, B _ijrepresent the B component pixel of the i-th row jth row; R _ijrepresent the R component pixel of the i-th row jth row; G _ijrepresent the G component pixel of the i-th row jth row, wherein the span of i and j is 1 to 2.

Step 3, extract rgb format image in R, G, B tri-road color component, by the rear color total amount obtaining each pixel of R, G, B tri-road color component addition, then respectively to R, G, B tri-road color component to move to left N position, the span of N for being generally 4 to 32, obtain by R, G, B tri-road color component expand 2 Nth power doubly after component value

Step 4: calculate component accounts for the ratio of color total amount respectively, account form as shown in formula (5),

$\left\{\begin{array}{c}\mathrm{RS}=\stackrel{\&OverBar;}{R}/(R+G+B)\\ \mathrm{GS}=\stackrel{\&OverBar;}{G}(R+G+B)\\ \mathrm{BS}=\stackrel{\&OverBar;}{B}/(R+G+B)\end{array}\right.---\left(5\right)$

In formula (5), RS, GS, BS represent respectively R, G, B tri-road color component expand 2 Nth power doubly after three road color components account for the ratio of color total amount (R+G+B).

Step 5: decision threshold step one set expands the Nth power of 2 doubly, then judges three road color components account for the ratio of color total amount (R+G+B), namely the color threshold Rule of judgment after Nth power that whether RS, GS, BS of each pixel meet expansion 2 is doubly judged, if met, be then 1 by this pixel assignment, if do not met, be 0 by this pixel assignment, obtain the binary image of target affiliated area to be tracked, for the process that target following is follow-up.

Such as:

The pixel belonging to the region to be split of red objects should meet:

(RS>0.4*2 ^N)&(GS<0.3*2 ^N)&(BS<0.2*2 ^N)；

The pixel belonging to the region to be split of blue target should meet:

(RS<0.4*2 ^N)&(GS>0.4*2 ^N)&(BS<0.2*2 ^N)；

The pixel belonging to the region to be split of blue target should meet:

(RS<0.4*2 ^N)&(GS<0.3*2 ^N)&(BS>0.3*2 ^N)；

This step solves a floating-point operation difficult problem in FPGA dexterously, makes to detect target at color region and reaches more accurate effect.

Beneficial effect of the present invention can be further illustrated by following emulation experiment:

Experimental Hardware platform is AlterCycloneIV chip, software platform is QuartusII, detected object as shown in Figure 5, black and white gridiron pattern with red, yellow, blue, grey colouring information is put on the ground, does the detection of red objects (being placed on the red Ipad shell in the upper left corner in figure).As shown in Figure 6, in Fig. 6, white portion is the red objects region detected after binaryzation to experimental result.Illustrate that red objects can extract by the present invention exactly, there is very high degree of accuracy.To differentiate other color target, revising the threshold decision condition of corresponding color in a program, having fully demonstrated dirigibility of the present invention.The present invention has also done the experiment of number of colors target detection, and process picture frame frequency can reach 60fps, does not occur the phenomenon of time delay in experiment, and therefore real-time performance of the present invention is remarkable.

Claims (7)

1. based on a color target detection system of FPGA, it is characterized in that, comprise Bayer signal receiving module; Bayer form interpolation turns RGB module; R, G, B tri-road color component summation module; R, G, B tri-road color component to move to left module; Dividing module; Color judges and binaryzation object module; Above-mentioned module all realizes in FPGA.

2., as claimed in claim 1 based on the color target detection system of FPGA, it is characterized in that,

After Bayer signal receiving module receives the picture signal of Bayer form, the picture signal of Bayer form is sent into Bayer form interpolation and turn RGB module;

Bayer form interpolation turns GRB module and uses interpolation algorithm the image of Bayer form to be converted to the image of rgb format, and extract the R road color component of each pixel in rgb format image, G road color component and B road color component, then by R, G, B tri-road color component send into respectively simultaneously R, G, B tri-road color component summation module and R, G, B color component to move to left module;

R, G, B tri-road color component summation module calculate R, G, B tri-total amount of road color component, then color total amount is sent to dividing module;

R, G, B color component move to left module by R, G, B tri-road color component to move to left respectively N position, the span of N between 4 to 32, obtain by R, G, B tri-road color component expand 2 Nth power doubly after three road color components, and will three road color components are sent to dividing module;

Dividing module calculates each pixel three road color components account for the ratio of color total amount separately, and are sent to by ratio color to judge and binaryzation object module.

3., based on a color target detection method of FPGA, it is characterized in that, comprise the following steps:

Step one, color according to target to be split in image, set each pixel in region to be split whether belong to this color R, G, B tri-road color component account for the decision threshold of color sum-rate;

Step 2, the image of Bayer form is converted into the image of rgb format;

Step 3, extract rgb format image in R, G, B tri-road color component, by the rear color total amount obtaining each pixel of R, G, B tri-road color component addition, then respectively to R, G, B tri-road color component to move to left N position, the span of N between 4 to 32, obtain by R, G, B tri-road color component expand 2 Nth power doubly after component value

Step 4: calculate component accounts for the ratio of color total amount respectively;

Step 5: decision threshold step one set expands the Nth power of 2 doubly, then judges the Nth power whether ratio that three road color components account for color total amount meets expansion 2 doubly after color threshold Rule of judgment, if met, be then 1 by this pixel assignment, if do not met, be 0 by this pixel assignment, obtain the binary image of target affiliated area to be tracked, for the process that target following is follow-up.

4., as claimed in claim 3 based on the color target detection method of FPGA, it is characterized in that, in step one,

When the color of target to be split is red, then judge that the threshold decision bar whether image certain pixel interior belongs to red area pixel is:

When the color of target to be split is green, then judge that the threshold decision bar whether image certain pixel interior belongs to green area pixel is:

When the color of target to be split is blue, then judge that the threshold decision bar whether image certain pixel interior belongs to blue region pixel is:

Wherein, R, G, B are respectively R, G, B tri-road color component of a certain pixel, and r, g, b are respectively the ratio that R color component, G color component or B color component account for R, G, B color total amount.

5., as claimed in claim 3 based on the color target detection method of FPGA, it is characterized in that, in step 2, use interpolation arithmetic that the image of Bayer form is converted into the image of rgb format; Wherein in Bayer format-pattern, the interpolation calculation mode of R, G, B component four kinds of array modes difference correspondences is as follows:

When the first row is from left to right R, G component, the second row is from left to right G, B component, this correspondence the mode of interpolation is carried out as shown in formula (1) to first element in the lower right corner,

B＝B ₂₂,R＝R ₁₁,G＝(G ₁₂+G ₂₁)/2；(1)

When the first row is from left to right B, G component, the second row is from left to right G, R component, then correspondingly carry out the mode of interpolation as shown in formula (2) to first element in the lower right corner,

G＝(G ₁₁+G ₂₂)/2；B＝B ₁₂,R＝R ₂₁；(2)

When the first row is from left to right G, B component, the second row is from left to right R, G component, then correspondingly carry out the mode of interpolation as shown in formula (3) to first element in the lower right corner,

G＝(G ₁₁+G ₂₂)/2；B＝B ₁₂,R＝R ₂₁；(3)

When the first row is from left to right G, R component, the second row is from left to right B, G component, then correspondingly carry out the mode of interpolation as shown in formula (4) to first element in the lower right corner,

G＝(G ₁₁+G ₂₂)/2,R＝R ₁₂，B＝B ₂₁；(4)

In above formula, B _ijrepresent the B component pixel of the i-th row jth row; R _ijrepresent the R component pixel of the i-th row jth row; G _ijrepresent the G component pixel of the i-th row jth row, wherein the span of i and j is 1 to 2.

6., as claimed in claim 3 based on the color target detection method of FPGA, it is characterized in that, in step 4 component accounts for the account form of the ratio of color total amount respectively as shown in formula (5),

In formula (5), RS, GS, BS represent respectively R, G, B tri-road color component expand 2 Nth power doubly after three road color components account for the ratio of color total amount (R+G+B).

7., as claimed in claim 3 based on the color target detection method of FPGA, it is characterized in that, in step 5,

The pixel belonging to the region to be split of red objects should meet:

(RS>0.4*2 ^N)&(GS<0.3*2 ^N)&(BS<0.2*2 ^N)；

The pixel belonging to the region to be split of blue target should meet:

(RS<0.4*2 ^N)&(GS>0.4*2 ^N)&(BS<0.2*2 ^N)；

The pixel belonging to the region to be split of blue target should meet:

(RS<0.4*2 ^N)&(GS<0.3*2 ^N)&(BS>0.3*2 ^N)。