CN101398896A - Device and method for extracting color characteristic with strong discernment for image forming apparatus - Google Patents

Abstract

The invention provides a method and a device for extracting color characteristics with high identifying capacity. The method includes: dividing a target area and a non-target area in an input image into a plurality of rectangles; building the color column diagrams of a target rectangle and a non-target rectangle and dividing each color column diagram into a plurality of discrete areas; extracting the main colors of the rectangle according to the color column diagrams; calculating the smallest distance between the color column diagram interval of the target rectangle and the color column diagram interval of the non-target rectangle by aiming at the extracted main colors; calculating the weight of the color column diagram interval of the target rectangle based on the smallest distance; calculating the weight of the target rectangle based on the weight of the color column diagram interval of the target rectangle; carrying out renew weighing on the color column diagram interval of the target rectangle based on the weight of the target rectangle and the weight of the colors; generating the final color model of the target based on the renew weighing result.

Description

The extraction that is used for imaging device has the equipment and the method for the color characteristic of strong discernment

Technical field

The present invention relates to pattern-recognition, feature extraction, statistical learning and object detection technique (such as Human Detection), more particularly, the Feature Extraction with strong discernment of look looks or announcement the color distribution pattern with strong separating capacity hidden relative with other object of identical or different kind that relate to object the invention still further relates to a kind of extraction and have the equipment and the method for color characteristic of strong discernment and the imaging device that uses this equipment and method.

Background technology

The representative colors feature of object appearance is widely used in object detection and the tracking in the frame of video.When the expression color characteristic, use color histogram usually.In order to catch the space layout of color distribution, the histogram of the independent subregion of dividing from subject area is fit to.Except this method, also have other method (such as movable information, shape information, geometrical constraint etc.) to compensate color model, to obtain better testing result.

Make various trials and solved the problem of using color model to follow the tracks of coherent motion object.The example of these trials comprises: use the look looks to carry out the method for probability tracking (title by people such as P.Perez showed that this method is disclosed in the computer vision Europe meeting 2002 is in " probability based on color is followed the tracks of ", is designated hereinafter simply as the Perez method); Color histogram modeling and probability graph detection method (this method is disclosed in the 5845009A1 United States Patent (USP), is designated hereinafter simply as 009 patent); The many search window method (this method is disclosed in the 2007127775A1 United States Patent (USP), is designated hereinafter simply as 775 patents) that compares similarity based on color model; Extract the method (this method is disclosed in the 2004017938A1 United States Patent (USP), is designated hereinafter simply as 938 patents) of skin color based on the statistical color model.

The representative colors feature of object appearance is widely used in the object detection in the frame of video.When the expression color characteristic, use color histogram (referring to Perez method, 009 patent, 775 patents and 938 patents) usually.The position of given motion object and size can be calculated the color histogram of this motion object.In frame of video subsequently, calculate similarity mapping for this histogram model.Then, connected domain (blob) analytical technology can be the connected domain of indicating the position of the motion object with high probability with the pixel region gathering that has with the high similarity of histogram model.Many methods are paid close attention to the color characteristic of representing the motion object how effectively.In fact, important problem is how motion object and non-motion object (that is, other object and the background of identical or different kind) are distinguished mutually, rather than perfectly is loyal to the original color distribution of target.For the motion object, it is made up of a plurality of parts.For example, suppose motion to liking a people, then this people comprises face/head, the clothing upper body, the clothing lower part of the body.People's look looks may seem similar to other image-region, especially have under the environment of mixed and disorderly background.In this case, the color region that need select to have strong discernment from human body is as color characteristic, rather than selects the representative colors feature of whole human body.

Summary of the invention

Color characteristic with strong discernment refers between motion object itself and other image-region and has big color distinction.In order to extract color characteristic with strong discernment, the color vector that is divided in the color histogram of color vector in the some color histogram of object and other image-region is compared, reduce with the importance of the color vector of the approaching motion object of the color vector in other image-region, the importance of other color vector improves.

Therefore, the invention provides a kind of different situations and extract the method and apparatus that has the color characteristic of strong discernment and focus area is provided for the dynamic adjustments of image-generating unit for imaging and scene.

According to an aspect of the present invention, the method that provides a kind of extraction to have the color characteristic of strong discernment, this method may further comprise the steps: with subject area in the input picture and non-object area dividing is a plurality of rectangles; Use three Color Channels to set up the color histogram of object rectangle and non-object rectangle, wherein, each color histogram is divided into a plurality of discrete intervals; Extract the main color of rectangle according to color histogram; Minor increment between the color histogram interval of and non-object rectangle interval at the color histogram of the main color calculation object rectangle that extracts; Weight based on the color histogram interval of the minimum distance calculation object rectangle that calculates; Weight based on the color component of the weight calculation object rectangle in the color histogram interval of the object rectangle that calculates; Weight based on the weight calculation object rectangle in the color histogram interval of the object rectangle that calculates; Based on the weight of the object rectangle that calculates and the weight of color component weighting is again carried out in the color histogram interval of described object rectangle, thereby extracted the color characteristic with strong discernment of object; Produce the final color model of object based on the weighted results again in the color histogram interval of object rectangle.

This method also can comprise: object-based final color model, from the color connected domain of new input picture extraction object; The color connected domain of extracting is carried out the connected domain analysis, come the centre of form and the size of the object in the calculating input image based on described connected domain analysis, thereby described object is positioned and follows the tracks of.

According to a further aspect in the invention, the equipment that provides a kind of extraction to have the color characteristic of strong discernment, this equipment comprises: the area dividing unit is a plurality of rectangles with subject area in the input picture and non-object area dividing; The histogram calculation unit uses Color Channel to set up the color histogram of object rectangle and non-object rectangle, and wherein, each color histogram is divided into a plurality of discrete intervals; Main color extracting unit, the color histogram of setting up according to the histogram calculation unit extracts the main color of object rectangle; Interval minimum distance calculation unit, the minor increment between the color histogram interval of the interval and non-object rectangle of the color histogram of the main color calculation object rectangle that extracts at main color extracting unit; Interval weight calculation unit, the weight in the color histogram interval of the minimum distance calculation object rectangle that calculates based on interval minimum distance calculation unit; The color component weight calculation unit, the weight of the color component of the weight calculation object rectangle in the color histogram interval of the object rectangle that calculates based on interval weight calculation unit; The rectangle weight calculation unit, the weight of the weight calculation object rectangle in the color histogram interval of the object rectangle that calculates based on interval weight calculation unit; The interval is weighted units again, the weight of the object rectangle that calculates based on the rectangle weight calculation unit and the weight of the color component that the color component weight calculation unit is calculated are carried out weighting again to the color histogram interval of described object rectangle, thereby extract the color characteristic with strong discernment of object; Final color model generation unit produces the final color model of object based on the weighted results again in the color histogram interval of object rectangle.

This equipment also can comprise: color connected domain extraction unit, and object-based final color model is from the color connected domain of new input picture extraction object; The object positioning unit carries out the connected domain analysis to the color connected domain that color connected domain extraction unit extracts, and comes the centre of form and the size of the object in the calculating input image based on described connected domain analysis, thereby described object is positioned and follows the tracks of.

According to a further aspect in the invention, provide a kind of imaging device, this imaging device comprises: image-generating unit, the image of reference object; Extraction according to the present invention has the equipment of the color characteristic of strong discernment, receive image from image-generating unit, extract the color characteristic with strong discernment of object, produce the final color model of object, extract the color connected domain of object based on the final color model, carry out the connected domain analysis, and produce the parameter of adjusting the imaging device attitude, to obtain the better pictures quality based on described connected domain analysis; Control module, the equipment that has the color characteristic of strong discernment from described extraction receives the parameter of adjusting the imaging device attitude, adjust the attitude of imaging device, wherein, the attitude of imaging device can be the selection of the pendulum angle, angle of inclination, scaling and the focal zone that are used for Pan/Tilt/Zoom camera, or in above-mentioned some, for example, be used for the selection of the scaling and the focal zone of digital camera; Storage unit, the image of the object that storage is taken; Display unit, the image of the object that demonstration is taken.

This imaging device also can comprise: the mark unit, the subject area that the user is manually marked on image offers the equipment that described extraction has the color characteristic of strong discernment.

Control module can be according in the operation in swing, inclination, convergent-divergent and the selective focus zone of the parameter adjustment imaging device of adjusting the imaging device attitude at least a.

In the operation in selective focus zone, control module is controlled to the new zone at picture choice of equipment object place as focusing on foundation, so that described zone is focused on.

When control module is controlled to picture choice of equipment focal zone, imaging device selects the picture centre zone as the imaging and focusing zone of giving tacit consent to, perhaps dynamically the new image-region at alternative place as the imaging and focusing zone, according to the image data information of focal zone, dynamically adjust convergent-divergent multiple, focal length, swing or the tilt parameters of imaging device.

Description of drawings

In conjunction with the drawings, from the description of the following examples, the present invention these and/or others and advantage will become clear, and are easier to understand, wherein:

Fig. 1 has shown that extraction according to the present invention has the process flow diagram of method of the color characteristic of strong discernment;

Fig. 2 shows how to calculate people's rectangle;

Fig. 3 shows the calculating of each rectangular histogram;

Fig. 4 shows the calculating of interval weight and uses the effect with color extracting of strong discernment of the present invention;

Fig. 5 has shown the importance weighting function;

Fig. 6 has shown the color connected domain detection of using the color characteristic with strong discernment;

Fig. 7 shows and is used to locate people's the position and the connected domain analysis of size;

Fig. 8 has shown that extraction according to the present invention has the block diagram of equipment of the color characteristic of strong discernment;

Fig. 9 has shown the block diagram according to imaging device of the present invention.

Embodiment

To describe embodiments of the invention in detail now, its example is shown in the drawings, and wherein, identical label is represented identical parts all the time.These embodiment are described below with reference to the accompanying drawings to explain the present invention.

Hereinafter, describing the present invention with the people as the example of motion object, but the invention is not restricted to this, also can be the motion object of other type.

Fig. 1 has shown that extraction according to the present invention has the process flow diagram of method of the color characteristic of strong discernment.In step 101, the input of system receiver, video, people's position and dimension information, described people's position and dimension information are by extraction, manually mark or human body detector provide (20060147108A1 is disclosed as U.S. Pat).Multiple motion extraction method is arranged, for example, because imaging device is in the stationary state under the original state, so can use background subtracting.Manually mark can directly provide people's position and dimension information.Human body detector detects the people's who is used for the candidate the position and the input picture of size.Although the people is of different sizes in image, the ratio of width to height has fixing scope, and this can be used for safely determining other image-region (that is, the non-object zone comprises other object and the background of identical or different kind) except the motion object.Following step is based on known people's rectangular image zone and inhuman rectangular image zone and how extracts the color region with strong discernment.In step 102, coloured image is quantized, the colored pixels with 3 Color Channels is quantized N discrete interval (bin), for example, N=64.

In step 103, calculate the rectangle of head part and human body.Fig. 2 shows the calculating of the rectangle of head part and human body.For example, when given head part position and size, the several rectangles with same size will be arranged downwards, up to the lower boundary that reaches image.As existing knowledge, people's face and people's hair have other look looks of phase region, and the head rectangle is divided into several little rectangles.According to identical principle, other image-region except the zone that is occupied by the people can be estimated safely based on statistical people's the wide ratio of height (head).There is no need accurately estimated background rectangular area, that is, the estimated background rectangular area just can guarantee that the image pixel of motion object is not divided into the background rectangle by mistake roughly.

In step 104, calculate rectangular histogram.Fig. 3 shows the calculating of the color histogram of each rectangle, and described rectangle comprises the sub-rectangle of head part, human body rectangle and background rectangle (perhaps other rectangle except the people's rectangle that provides, that is, non-object zone).The histogram of these rectangles is calculated respectively, and histogram is divided into a plurality of intervals, the quantity of the pixel that adds up in each interval.

In step 105, after the color histogram that has obtained each rectangle, main color is retained, and trickle color is dropped.Because noise can influence the trickle color with small number of pixels, these pixels have described trickle color, so trickle color is unstable or insecure.

In step 106, the computation interval minor increment.

Shown in equation (1) and (2), the distance between the rectangle color histogram interval of calculating object and non-object.Below with the example of background area as the non-object zone.

$J[H_r^c\left(i\right),{\mathrm{BG}}^c\left(j\right)]=\underset{j}{\arg \min }|b_i-b_j^{\′}|---\left(1\right)$

$D_r^c\left(i\right)=|b_i-b_J^{\′}|---\left(2\right)$

Wherein, J represents the interval sign (ID) that the background color of Color Channel c distributes, and described interval J has the minor increment with the interval i of the Color Channel c of the main color histogram of people's rectangle r.b _iIt is the sign (ID) of interval i.

It is the color histogram of interval i of the Color Channel c of people's rectangle r.BG ^c(j) be the background color histogram of the interval j of Color Channel c.

Be the ID of interval j.

Be people's rectangle r Color Channel c interval i and have minor increment between the background color histogram of same color passage.

In step 107, the computation interval weight.

Shown in equation (3), the discernment (weight) in an interval among definition rectangle r and the passage c.

$w_r^c\left(i\right)=T(D_r^c\left(i\right),s)\×|H_r^c\left(i\right)-{\mathrm{BG}}^c\left(J\right)|---\left(3\right)$

Wherein, $s=\mathrm{sign}(H_r^c\left(i\right)-{\mathrm{BG}}^c\left(J\right))$

T (x, y)=e ^{X * k * y}, wherein, k is a constant

In step 108, calculate the color component weight.

In each rectangle, pixel is made up of three kinds of color components, and these three kinds of color components are red, green and blue elements.Because every kind of element is quantified as a plurality of histogram, so these elements depend on the distance of the histogram that they are related and have different discernments.

$w^c=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_r^c\left(m\right)---\left(4\right)$

$W^c=\frac{w^c}{\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{w}^k}---\left(5\right)$

In equation (4), each the interval weight among the color component c is added up.Therefore, can calculate the weight of every kind of color component according to equation (5).In equation (4) and (5), M represents the quantity in the interval in people's rectangular histogram, and K represents the quantity of Color Channel.

In step 109, calculate the rectangle weight.

The importance of people's rectangle depends on the discernment between the chromatic zones that comprises in this people's rectangle.When a rectangle covers the color region of high discernment, than other people's rectangle, this rectangle will have big weight.

$w_r=\underset{k=1}{\overset{K}{\mathrm{\Σ}}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_r^k\left(m\right)---\left(6\right)$

$W_r=\frac{w_r}{\underset{l=1}{\overset{L}{\mathrm{\Σ}}}{w}_l}---\left(7\right)$

In equation (6), calculate interval weight among people's rectangle r and.Then, can calculate the rectangle weight according to equation (7), equation (7) carries out the importance weighting to these rectangles between the chromatic zones with high discernment, and wherein, L represents the quantity of people's rectangle.

In step 110, weighted histogram interval again.

$W_r^c\left(i\right)=w_r^c\left(i\right)\×W_r\×W^c---\left(8\right)$

Each histogram has different discernments between target travel object and mixed and disorderly background or other motion object.The importance of histogram depends on the initial weight of this histogram in the histogram , the color component weights W ^cAnd rectangle weights W _rTherefore, histogrammic final importance is according to the weighting again of equation (8) quilt.

In step 111, produce the final color model.

$N_h=\{N_h^r,N_h^g,N_h^b\}$

$N_h^c\left(i\right)=\underset{r=1}{\overset{R_h}{\mathrm{\Σ}}}\underset{m=1}{\overset{N_h}{\mathrm{\Σ}}}{W}_r^c(b_m=i)---\left(9\right)$

$N_t=\{N_t^r,N_t^g,N_t^b\}$

$N_t^c\left(i\right)=\underset{r=1}{\overset{R_t}{\mathrm{\Σ}}}\underset{m=1}{\overset{N_t}{\mathrm{\Σ}}}{W}_r^c(b_m=i)---\left(10\right)$

After the histogram in everyone rectangle is carried out weighting again, can produce final color model according to equation (9) and (10), equation (9) and (10) are represented the final color model (color histogram distribution) of head part and people's trunk respectively.In equation (9) and (10), added up respectively from the weight of the histogram in the rectangle of head part and people's trunk.R is the quantity of people's rectangle, and M is the quantity in the interval of people's rectangular histogram.More particularly, R _hThe quantity of expression head part rectangle, R _tThe quantity of expression people trunk rectangle, M _hThe quantity in the interval of expression head part rectangular histogram, M _tThe quantity in the interval of expression people trunk rectangular histogram.

In order to set forth the main design of this section back, Fig. 4 shows an example.(a) among Fig. 4 is a color histogram of the color component in the rectangle of motion object.(b) among Fig. 4 is a color histogram of the color component in other zone of the image except that the motion object.In these two color histograms, two intervals are arranged all, represent by BIN-1 and BIN-2.Along with the BIN-1 of motion object goes up than the BIN-2 of this motion object more away from two intervals of background in range conversion (equation (1) and (2)), the weight of the BIN-1 of motion object increases, and the weight of the BIN-2 of motion object reduces, shown in (c) among Fig. 4.Although the initial weight of BIN-2 is greater than the initial weight of BIN-1, after the conversion that the color characteristic with strong discernment extracts, the new weight of BIN-1 is greater than the new weight of BIN-2.Fig. 5 shown importance weighting function T (x, y), this function increase has the weight of big zone distance, and reduces to have the weight of minizone distance, described zone distance calculates according to equation (1) and (2).

It should be noted that the method that shows at three Color Channels of coloured image here, in the method, use the one dimension color histogram to set up the final color model independently.But in fact there is multiple variation, such as using two-dimensional color histogram or three-dimensional color histogram.For example, there are three kinds of two-dimensional color histograms, that is, and red-green two-dimensional color histogram, red-blue two-dimensional color histogram, indigo plant-green two-dimensional color histogram.But method of the present invention also can be applicable to these histograms in a comparable manner, thereby extracts the color characteristic with strong discernment.

The final color model that produces based on extracting from the color characteristic with strong discernment can extract the color connected domain that belongs to the motion object from new input picture.For each pixel in the input picture, the similarity of itself and target travel object is formed a weight matrix, and the size of this weight matrix is big or small identical with input picture.The element of this weight matrix is from the weight that adds up in the final color histogram model, and this final color histogram model quantizes to obtain with color pixel values.

In Fig. 6, there are two people.A people is selected as tracing object according to certain mode, and for example, the people on right side is selected as tracing object.(a) among Fig. 6 is new input picture, the result that (b) among Fig. 6 is to use the connected domain of people's human trunk model to detect, the result that (c) among Fig. 6 is to use the connected domain of head part's model to detect.Because white extensively is distributed among background and another person, so the people's on right side white has very low weight, it has occupied the major part of the initial weight of color histogram.

With reference to (a) among Fig. 7, there are two head candidates from head part's detecting device with shape.Therefore, head part's probability distribution is to be the Gaussian distribution at center with given head position.(b) among Fig. 7 shows the result of the color connected domain detection of using the head color model, and (c) among Fig. 7 shows the result of the color connected domain detection of using the trunk color model.In head part's color connected domain detected, people's probability distribution was to be the Gaussian distribution at center with given head position.Because supposed people's attitude up and down, probability distribution is extended downwards, with existing of assignor's trunk.In the color connected domain of people's trunk detected, people's probability distribution also was to be the Gaussian distribution at center with given trunk position.Because the head part is positioned at the top area (this has retrained geometric relationship) of people's trunk, thus probability distribution extend upward, with existing of assignor's head.Therefore, with reference to (d) among Fig. 7, the centre of form of countable probability figure (centroid) is as the final position of tracing object.

Fig. 8 has shown that extraction according to the present invention has the block diagram of equipment of the color characteristic of strong discernment.

With reference to Fig. 8, the equipment that extraction according to the present invention has the color characteristic of strong discernment comprises area dividing unit 801, histogram calculation unit 802, main color extracting unit 803, interval minimum distance calculation unit 804, interval weight calculation unit 805, color component weight calculation unit 806, rectangle weight calculation unit 807, interval weighted units 808 and final color model generation unit 809 again.

Area dividing unit 801 is a plurality of rectangles with subject area in the input picture and non-object area dividing.

Histogram calculation unit 802 uses Color Channel to set up the color histogram of object rectangle and non-object rectangle, and wherein, described color histogram is divided into a plurality of discrete intervals.

Main color extracting unit 803 extracts the main color of object rectangle according to color histogram.

Minor increment between the color histogram interval of the interval and non-object rectangle of the color histogram of the main color calculation object rectangle that interval minimum distance calculation unit 804 extracts at main color extracting unit 803.Interval minimum distance calculation unit 804 can calculate described minor increment according to equation (1) and (2).

Interval weight calculation unit 805 is based on the weight in the color histogram interval of the minimum distance calculation object rectangle of interval minimum distance calculation unit 804 calculating.The weight that interval weight calculation unit 805 can be come the color histogram interval of calculating object rectangle according to equation (3).

Color component weight calculation unit 806 is based on the weight of the color component of the weight calculation object rectangle in the color histogram interval of the object rectangle of interval weight calculation unit 805 calculating; The weight that color component weight calculation unit 806 can be come the color component of calculating object rectangle according to equation (4) and (5).

Rectangle weight calculation unit 807 is based on the weight of the weight calculation object rectangle in the color histogram interval of the object rectangle of color component weight calculation unit 806 calculating.The weight that rectangle weight calculation unit 807 can be come the calculating object rectangle according to equation (6) and (7).

Interval weighted units 808 is again carried out weighting again based on the weight of the color component that the weight and the color component weight calculation unit 806 of the object rectangle of rectangle weight calculation unit 807 calculating are calculated to the color histogram interval of described object rectangle, thereby extracts the color characteristic with strong discernment of object.Interval weighted units 808 again can be come weighting is again carried out in the color histogram interval of described object rectangle according to equation (8).

Final color model generation unit 809 produces the final color model of object based on the weighted results again in the color histogram interval of object rectangle.Final color model generation unit 809 can produce the final color model of object according to following equation:

N＝{N ^r，N ^g，N ^b}

$N^c\left(i\right)=\underset{r=1}{\overset{R}{\mathrm{\Σ}}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{W}_r^c(b_m=i)---\left(11\right)$

Wherein, the quantity of R indicated object rectangle, the quantity in the interval of M indicated object rectangular histogram.

In addition, the equipment of extraction according to the present invention with color characteristic of strong discernment also can comprise color connected domain extraction unit 810 and object positioning unit 811.

Color connected domain extraction unit 810 object-based final color models are from the color connected domain of new input picture extraction object.Object positioning unit 811 carries out the connected domain analysis based on the color connected domain that color connected domain extraction unit 810 extracts, the centre of form of the object in the calculating input image and size, thus described object is positioned and follows the tracks of.

Fig. 9 has shown the block diagram according to imaging device of the present invention.

This imaging device comprises that image-generating unit 901, extraction according to the present invention have the equipment 902 of the color characteristic of strong discernment, control module 903, storage unit 904, display unit 905 and mark unit 906.This imaging device can be PTZ (PAN, TILT, ZOOM) any among video camera, static surveillance camera, DC (digital camera), DV (digital camera) and the PVR (personal video recorder).

Image-generating unit 901 is hardware units, and for example CCD or COMS device be used for perception and produce natural image, and the picture processing chip of image-generating unit can reach the preferable image quality.For moving object tracking, there are two kinds of methods that the position and the size in motion of objects zone are provided.First method is an automated process, uses the algorithm that embeds to extract the size and the position in the zone of interested object.Second method is a manual methods, and user or operator go up the zone of the interested object of mark at the image (for example, touch-screen) that shows.For automated process, the algorithm that use to embed (disclosed method among USP20060147108A1, the USP20050276446 for example, perhaps machine vision in 2005 and use disclosed method in the paper that disclosed name in the international conference is called " head part of the various visual angles in the rest image detects "), object can be detected automatically.Mark unit 906 offers user or operator with marking Function, so that user or operator can enough pens or finger interested subject area of manual mark on image.

The equipment 902 that extraction has the color characteristic of strong discernment can receive view data from image-generating unit 901, also can receive the user for example with the size and the positional information of the interested subject area of the form mark of rough mark.The equipment 902 that extraction has a color characteristic of strong discernment extracts the color characteristic with strong discernment of objects, produce the final color model of object, extract the color connected domain of object based on the final color model, carry out the connected domain analysis, and produce the parameter of adjusting the imaging device attitude based on described connected domain analysis.It should be noted that when using the first method that object's position and size are provided, mark unit 906 is optional.When a plurality of tracing objects were selective, when a plurality of motion object of following the tracks of was for example arranged, the user can revise the tracing object that imaging device is selected automatically among the present invention.

Control module 903 can be adjusted to the attitude of picture equipment, the attitude of imaging device is carried out self-focusing operation control by swing, inclination, convergent-divergent and the selective focus zone of Pan/Tilt/Zoom camera, is perhaps controlled by the zoom operations of static surveillance camera, DC, DV or PVR.Control module 903 has the color characteristic of strong discernment from extraction equipment 902 receives the parameter of adjusting the imaging device attitude.The equipment 902 that extraction has a color characteristic of strong discernment can provide the object's position and the dimension information of new time point or new frame data.Control module 903 is according to the attitude of described parameter adjustment imaging device, to make object placed in the middle in image by swing/tilt operation, select the zone of interested object by the operation in selective focus zone, and operate zone to focus on to interested object by convergent-divergent and automatic focus, with the details of the motion object that obtains high image quality.In the operation in selective focus zone, the new zone at control module may command imaging device alternative place is as focusing on foundation, so that described zone is focused on.In addition, when control module is controlled to picture choice of equipment focal zone, imaging device is except selecting the picture centre zone imaging and focusing zone as acquiescence, dynamically the new image-region at alternative place is as the imaging and focusing zone, image data information according to focal zone, dynamically adjust convergent-divergent multiple, automatic focus, focal length, swing or the tilt parameters of imaging device, thereby obtain better imaging effect.

For the electronic product that is held in user's hand, such as DC, DV or PVR, but its attitude of user's manual adjustment makes interested object placed in the middle in image.At this time, equipment itself can automatically carry out other operation, as changing convergent-divergent multiple, automatic focus.

Storage unit 904 can be with image or video storage in memory storage, and display unit is shown to the user with the image or the video at scene.

The present invention can be implemented as software, and this software is used to be connected to imaging device and puts embedded system with control module, to regulate the parameter of imaging device attitude.For embedded imaging device system, but its receiver, video and sends to the control module of imaging device with order as input, with the attitude of regulating imaging device, lens focus zone etc.

The present invention extracts main color histogram to come indicated object look looks, but extracts the color model with strong discernment.At the representative colors model with have between the color model of strong discernment and have difference.Sometimes when motion object look looks and non-object and background color model had very big difference, the color model with strong discernment was similar to the representative colors model.Sometimes the color model and the representative colors model dissmilarity that have strong discernment are particularly when motion object look looks and non-object and background color model have the part difference.The present invention can use the color model with strong discernment to come the localizing objects object in the mixed and disorderly background with interference color region similar to the motion object effectively.The present invention can distinguish a plurality of motion objects with the similar color of part with tracing object mutually, then locatees the position and the size of tracing object, thereby regulates imaging device to obtain good image-forming condition.Color model with strong discernment uses the principle of importance weighted strategy.Have the color model increase of strong discernment and non-object or background the very importance/weight of the color elements of big difference is arranged, reduce the importance/weight of the color elements similar to non-object or background.Use static video camera or be used for the coherent Pan/Tilt/Zoom camera of following the tracks of the target travel object of interested or suspection and come tracing object, can be used for event analysis and picture quality and strengthen.Have in the detection of color model of strong discernment in use, this detects scanning input picture, and based on the color model with strong discernment this input picture is transformed to connected domain figure.This detection also can be used for checking the people candidate from human body detector, with the refusal false alarm.

In Pan/Tilt/Zoom camera, the present invention can be implemented as the software in the embedded system, embedded system receives the video image of input, analyze input picture, after setting up the color model with strong discernment of input picture, to be used for the people is located, the attitude (swing, inclination convergent-divergent, selective focus zone) of adjusting Pan/Tilt/Zoom camera so that the people is followed the tracks of, makes the people be positioned at the middle position of image, and has good resolution and picture quality.When video camera swing and inclination, the people who moves is followed the tracks of, avoid the people to leave the visual field of video camera.When video camera carries out optical zoom, regulate optical focal length, with the people's that obtains motion the local detail or the high quality graphic of whole health.Video recording device can be kept at field data in the memory storage, and this video recording device with high image quality recorder's behavior, perhaps carries out interactive operation with machine when entering regional that the user is provided with.

It should be noted that the present invention also can be used for portable imaging device (for example, DC, DV or mobile camera).People's initial position and size can manually be marked by the user, are perhaps detected automatically by detecting device.By the people who moves being positioned and directly focusing, can focus on the people who moves automatically, to obtain high image quality.For people that will motion placed in the middle in image, but the lens direction of his imaging device held of terminal user's manual adjustments.

In static video camera, the present invention can be implemented the track set that has the people of this motion of the people's of motion coherent tracking.Track comprises the people's of motion position and size.During following the tracks of, system can analyze some incident that whether taken place, and perhaps alternatively sends some alarms or notice.User-defined incident can be dangerous situation, or user's general notice situation, for example, enters position or the size of certain prohibited area that sets in advance as the people of intruder's motion.

Employed in the present invention term " imaging device " refers to the equipment that has image-generating unit and control module at least, can be video camera, camera or have other portable set of imaging function.

The present invention also can be used for using the robot imaging device to come the robot that the people is positioned.The basic function of robot comprise evade obstacle, mutual with the people, the people followed the tracks of etc.The present invention can be used for finding whether near zone exists the people, and perhaps the position to the people positions to follow the tracks of or to carry out alternately with the people.

The color characteristic with strong discernment that the present invention also can be used as the motion object of other kind extracts and detects, the front with the people as example so that explain.For candidate's intended application, be not limited to people's modeling and detection.For example, for coming with DC the motion object is carried out imaging, the terminal user can rule on touch-screen with hand or pen, manually marks interested subject area, with denoted object position, size or zone.The present invention can set up the color model with strong discernment of object, and discloses position and the size of this object in ensuing frame of video.Therefore, DC scalable imaging parameters is to obtain the high image quality of motion object.

Though the present invention is specifically described with reference to its exemplary embodiment and is shown, but will be understood by those skilled in the art that, under the situation that does not break away from the spirit and scope of the present invention that are defined by the claims, can carry out the various changes of form and details to it.

Claims (21)

1, a kind of extraction has the method for the color characteristic of strong discernment, may further comprise the steps:

With subject area in the input picture and non-object area dividing is a plurality of rectangles;

Use Color Channel to set up the color histogram of object rectangle and non-object rectangle, wherein, each color histogram is divided into a plurality of discrete intervals;

Extract the main color of rectangle according to color histogram;

Minor increment between the color histogram interval of and non-object rectangle interval at the color histogram of the main color calculation object rectangle that extracts;

Weight based on the color histogram interval of the minimum distance calculation object rectangle that calculates;

Weight based on the color component of the weight calculation object rectangle in the color histogram interval of the object rectangle that calculates;

Weight based on the weight calculation object rectangle in the color histogram interval of the object rectangle that calculates;

Based on the weight of the object rectangle that calculates and the weight of color component weighting is again carried out in the color histogram interval of described object rectangle, thereby extracted the color characteristic with strong discernment of object;

Produce the final color model of object based on the weighted results again in the color histogram interval of object rectangle.

2, method according to claim 1, wherein, the minor increment between the color histogram interval of and non-object rectangle interval according to the color histogram of following equation calculating object rectangle:

$J[H_r^c\left(i\right),{\mathrm{BG}}^c\left(j\right)]=\underset{j}{\arg \min }|b_i-b_j^{\′}|$

$D_r^c\left(i\right)=|b_i-b_J^{\′}|$

Wherein, J represents the interval of color histogram of the non-object rectangle of Color Channel c, and described interval J has the minor increment with the interval i of the Color Channel c of the main color histogram of object rectangle r, b _iBe the sign of interval i,

Be the color histogram of interval i of the Color Channel c of object rectangle r, BG ^c(j) be the non-object color histogram of the interval j of Color Channel c, Be the sign of interval j,

Be object rectangle r Color Channel c interval i and have minor increment between the non-object rectangle color histogram interval of same color passage.

3, method according to claim 2, wherein, according to the weight in the color histogram interval of following equation calculating object rectangle

$w_r^c\left(i\right)=T(D_r^c\left(i\right),s)\×|H_r^c\left(i\right)-{\mathrm{BG}}^c\left(J\right)|$

Wherein, $s=\mathrm{sign}(H_r^c\left(i\right)-{\mathrm{BG}}^c\left(J\right)),$ T (x, y)=e ^{X * k * y}, k is a constant.

4, method according to claim 3, wherein, according to the weights W of the color component of following equation calculating object rectangle ^c:

$w^c=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_r^c\left(m\right)$

$W^c=\frac{w^c}{\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{w}^k}$

Wherein, K represents the quantity of Color Channel, and the quantity in the interval in the M indicated object rectangular histogram is at equation $w^c=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_r^c\left(m\right)$ In among the color component c of calculating object rectangle r each interval weight and, at equation $W^c=\frac{w^c}{\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{w}_r^k}$ The middle weight of calculating color component c.

5, method according to claim 4, wherein, according to the weights W of following equation calculating object rectangle _r:

$w_r=\underset{k=1}{\overset{K}{\mathrm{\Σ}}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_r^k\left(m\right)$

$W_r=\frac{w_r}{\underset{l=1}{\overset{L}{\mathrm{\Σ}}}{w}_l}$

Wherein, the quantity of L indicated object rectangle is at equation $w_r=\underset{k=1}{\overset{K}{\mathrm{\Σ}}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_r^k\left(m\right)$ In in the calculating object rectangular histogram the interval weight and, at equation $W_r=\frac{w_r}{\underset{l=1}{\overset{L}{\mathrm{\Σ}}}{w}_l}$ The weight of middle calculating object rectangle r.

6, method according to claim 5, wherein, according to equation $W_r^c\left(i\right)=w_r^c\left(i\right)\×W_r\×W^c$ Weighting is again carried out in color histogram interval to described object rectangle, obtains the weight in color histogram interval of the object rectangle of weighting again

7, method according to claim 6, wherein, according to equation N={N ^r, N ^g, N ^bProduce the final color model of object,

Wherein, for Color Channel c, $N^c\left(i\right)=\underset{r=1}{\overset{R}{\mathrm{\Σ}}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{W}_r^c(b_m=i),$ The quantity of R indicated object rectangle, the quantity in the interval of M indicated object rectangular histogram.

8, method according to claim 7 also comprises:

Object-based final color model is from the color connected domain of new input picture extraction object;

The color connected domain of extracting is carried out the connected domain analysis, come the centre of form and the size of the object in the calculating input image based on described connected domain analysis, thereby described object is positioned and follows the tracks of.

9, a kind of extraction has the equipment of the color characteristic of strong discernment, comprising:

The area dividing unit is a plurality of rectangles with subject area in the input picture and non-object area dividing;

The histogram calculation unit uses Color Channel to set up the color histogram of object rectangle and non-object rectangle, and wherein, each color histogram is divided into a plurality of discrete intervals;

Main color extracting unit, the color histogram of setting up according to the histogram calculation unit extracts the main color of object rectangle;

Interval minimum distance calculation unit, the minor increment between the color histogram interval of the interval and non-object rectangle of the color histogram of the main color calculation object rectangle that extracts at main color extracting unit;

Interval weight calculation unit, the weight in the color histogram interval of the minimum distance calculation object rectangle that calculates based on interval minimum distance calculation unit;

The color component weight calculation unit, the weight of the color component of the weight calculation object rectangle in the color histogram interval of the object rectangle that calculates based on interval weight calculation unit;

The rectangle weight calculation unit, the weight of the weight calculation object rectangle in the color histogram interval of the object rectangle that calculates based on interval weight calculation unit;

The interval is weighted units again, the weight of the object rectangle that calculates based on the rectangle weight calculation unit and the weight of the color component that the color component weight calculation unit is calculated are carried out weighting again to the color histogram interval of described object rectangle, thereby extract the color characteristic with strong discernment of object;

Final color model generation unit produces the final color model of object based on the weighted results again in the color histogram interval of object rectangle.

10, equipment according to claim 9, wherein, the minor increment between the color histogram interval of minimum distance calculation unit and non-object rectangle interval according to the color histogram of following equation calculating object rectangle:

$J[H_r^c\left(i\right),{\mathrm{BG}}^c\left(j\right)]=\underset{j}{\arg \min }|b_i-b_j^{\′}|$

$D_r^c\left(i\right)=|b_i-b_J^{\′}|$

Wherein, J represents the interval of color histogram of the non-object rectangle of Color Channel c, and described interval J has the minor increment with the interval i of the Color Channel c of the main color histogram of object rectangle r, b _iBe the sign of interval i,

Be the color histogram of interval i of the Color Channel c of object rectangle r, BG ^c(j) be the non-object color histogram of the interval j of Color Channel c,

Be the sign of interval j,

Be object rectangle r Color Channel c interval i and have minor increment between the non-object rectangle color histogram interval of same color passage.

11, equipment according to claim 10, wherein, interval weight calculation unit is according to the weight in the color histogram interval of following equation calculating object rectangle

$w_r^c\left(i\right)=T(D_r^c\left(i\right),s)\×|H_r^c\left(i\right)-{\mathrm{BG}}^c\left(J\right)|$

Wherein, $s=\mathrm{sign}(H_r^c\left(i\right)-{\mathrm{BG}}^c\left(J\right)),$ T (x, y)=e ^{X * k * y}, k is a constant.

12, equipment according to claim 11, wherein, the color component weight calculation unit is according to the weights W of the color component of following equation calculating object rectangle ^c:

$w^c=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_r^c\left(m\right)$

$W^c=\frac{w^c}{\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{w}^k}$

Wherein, K represents the quantity of Color Channel, and the quantity in the interval in the M indicated object rectangular histogram is at equation $w^c=\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_r^c\left(m\right)$ In among the color component c of calculating object rectangle r each interval weight and, at equation $W^c=\frac{w^c}{\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{w}_r^k}$ The middle weight of calculating color component c.

13, equipment according to claim 12, wherein, object rectangle weight calculation unit is according to the weights W of following equation calculating object rectangle _r:

$w_r=\underset{k=1}{\overset{K}{\mathrm{\Σ}}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_r^k\left(m\right)$

$W_r=\frac{w_r}{\underset{l=1}{\overset{L}{\mathrm{\Σ}}}{w}_l}$

Wherein, the quantity of L indicated object rectangle is at equation $w_r=\underset{k=1}{\overset{K}{\mathrm{\Σ}}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{w}_r^k\left(m\right)$ In in the calculating object rectangular histogram the interval weight and, at equation $W_r=\frac{w_r}{\underset{l=1}{\overset{L}{\mathrm{\Σ}}}{w}_l}$ The weight of middle calculating object rectangle r.

14, equipment according to claim 13, wherein, the interval again weighted units according to equation $W_r^c\left(i\right)=w_r^c\left(i\right)\×W_r\×W^c$ Weighting is again carried out in color histogram interval to described object rectangle, obtains the weight in color histogram interval of the object rectangle of weighting again

15, equipment according to claim 14, wherein, final color model generation unit is according to equation N={N ^r, N ^g, N ^bProduce the final color model of object,

Wherein, for Color Channel c, $N^c\left(i\right)=\underset{r=1}{\overset{R}{\mathrm{\Σ}}}\underset{m=1}{\overset{M}{\mathrm{\Σ}}}{W}_r^c(b_m=i),$ The quantity of R indicated object rectangle, the quantity in the interval of M indicated object rectangular histogram.

16, equipment according to claim 15 also comprises:

Color connected domain extraction unit, object-based final color model is from the color connected domain of new input picture extraction object;

The object positioning unit carries out the connected domain analysis based on the color connected domain that color connected domain extraction unit extracts, and comes the centre of form and the size of the object in the calculating input image based on described connected domain analysis, thereby described object is positioned and follows the tracks of.

17, a kind of imaging device comprises:

Image-generating unit, the image of reference object;

Extraction as claimed in claim 16 has the equipment of the color characteristic of strong discernment, receive image from image-generating unit, extract the color characteristic with strong discernment of object, produce the final color model of object, extract the color connected domain of object based on the final color model, carry out the connected domain analysis, and produce the parameter of adjusting the imaging device attitude based on described connected domain analysis;

Control module, the equipment that has the color characteristic of strong discernment from described extraction receives the parameter of adjusting the imaging device attitude, adjusts the attitude of imaging device;

Storage unit, the image of the object that storage is taken;

Display unit, the image of the object that demonstration is taken.

18, imaging device according to claim 17 also comprises:

The mark unit, the subject area that the user is manually marked on image offers the equipment that described extraction has the color characteristic of strong discernment.

19, imaging device according to claim 18, wherein, at least a according in the operation in swing, inclination, convergent-divergent and the selective focus zone of the parameter adjustment imaging device of adjusting the imaging device attitude of control module.

20, imaging device according to claim 19, wherein, in the operation in selective focus zone, control module is controlled to the new zone at picture choice of equipment object place as focusing on foundation, so that described zone is focused on.

21, imaging device according to claim 20, wherein, when control module is controlled to picture choice of equipment focal zone, imaging device selects the picture centre zone as the imaging and focusing zone of giving tacit consent to, perhaps dynamically the new image-region at alternative place as the imaging and focusing zone, according to the image data information of focal zone, dynamically adjust convergent-divergent multiple, focal length, swing or the tilt parameters of imaging device.

Images