CN101169020A - Automatic door based on visual technique - Google Patents

Abstract

The invention discloses an automatic door using the computer vision technology. The opening, the closing, the duration of the opening, and the breadth of the automatic door can be effectively controlled by a vision sensor. The vision sensor can identify the movement of the human body, and can control the automatic door through the procedures of image inputting, background eliminating, noise removing, masking network establishing, human body identifying, movement model of human body establishing, pedestrian flow identifying, and the like. The automatic door of the invention is intelligent, can change the opening time and the opening breath according to different pedestrian flows, and can identify the movement direction of the human body; thereby the wrong opening of the automatic door can be avoided, and the energy is saved.

Description

Automatic door based on vision technique

Technical field

The present invention relates to a kind of automatic door, exactly, relate to a kind of automatic door that utilizes the vision technique sensor to control.

Background technology

In China, especially in the metropolis, automatic door is seen everywhere, and along with the promotion of construction of accessibilities for the people with disabilities, the development of automatic door will become inexorable trend.Present automatic door many with microwave remote sensor, infrared sensor as inductor, though some basic demands that can satisfy automatic door are still in 21 century, because the promotion of construction of accessibilities for the people with disabilities, the development of automatic door will be stepped into the new stage, and present simple automatic door has just proved definitely inferior.

Now very high in the automatic door frequency of utilization of some public places, in special time period, to concentrate by a large amount of personnel, another section is then very few in the period, so use automatic door to take all factors into consideration.When visitors flowrate is very big, can in one or two minute, reach hundreds of person-time,, increase suddenly unavoidably and can form confusion though can save energy visitors flowrate if door one heads straight for very for a short time.And if once head straight for very greatly, there be very long there is no need the time, summer, the often indoor and outdoor temperature difference was big still more, if it is big to open energy waste greatly always.Present automatic door many with microwave remote sensor, infrared sensor as inductor, it can't change the opening time and the width of door according to the difference of visitors flowrate, wasted the energy, and when people just from front of the door by but not when entering, opening of automatic door just brought awkward and bigger waste.

Summary of the invention

Improvement to existing automatic door is very necessary, and emphasis just is reselecting sensor.And to make automatic door have certain artificial intelligence, optimal sensor is no more than vision sensor.Computer vision is a very active field in the current computer science.In the human sensory information, great majority are to come from vision.Realizing artificial intelligence, is very important aspect to the Computer Processing of vision.At artificial intelligence field, the physical symbol system hypothesis of on record M.A.Simon is arranged, this hypothesis think intelligence essence calculate exactly.Corresponding therewith, the basic assumption of this subject of computer vision is: the vision mechanism that can come simulating human with calculation mode.And realize that so intelligentized automatic door just becomes possibility.

So propose to utilize vision sensor to replace the automatic door of infrared sensor.And computed programming makes automatic door add the gate opens wide under the big situation of the stream of people, increases to close the door time delay.And the stream of people is hour, reduces opens wide, shorten lockup every, thereby save the energy.Utilize vision technique to analyze human motion, just can provide portable to a greater extent, and,, make door have the effect of the identification direction of motion, flow-control, energy control action as wheelchair, go-cart etc. for special object provides service.And can integrate other more can play safety guard as recognition of face, Gait Recognition technology effect.

The core of vision control automatic door is computer vision.And the identification of computer vision program first step object also is a most basic part.Identification for object has many methods, is also more common a kind of of comparison basis based on the object identification of color characteristic.The image that is obtained by CCD camera at first enters in the computer and (passes through USB interface), the first step is selected the image color pattern, second step was carried out background subtraction, the 3rd step was removed noise, and the 4th step was set up the masking-out network, the 5th step identification human body, the 6th step, the result according to identification was single body or many human bodies, set up modelling of human body motion, the 7th step identification flow, thus automatic door is controlled.As shown in Figure 1.

The first step is determined color mode, can choose in computer.

Second step, background subtraction.Comprise initialization background model, subduction background and shade elimination.

At first initialization background model before nobody enters environment, is at first extracted i.e. initialization background model to background.Only need the image continuous acquisition n width of cloth (value of n depends on the circumstances) then by this n width of cloth image, can be set up the statistical model of an initial background.In this model, each the some i in the background, definition μ i is the expectation of the color value of this point, and following formula is arranged:

${\mathrm{\μ}}_i=\frac{1}{n}\underset{t=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\μ}}_{\mathrm{it}}$

Wherein μ it is the color value of i in t width of cloth image.

Reduce background then, promptly after initial background is set up,, just can carry out the extraction of foreground area for the new present image of gathering of each width of cloth.If the color value of present image mid point i is yi, by following formula with image binaryzation:

$\mathrm{Di}=\left\{\begin{array}{cc}1& \mathrm{if}(\mathrm{yi}-\mathrm{\μi}>3\mathrm{\σi})\\ 0& \mathrm{else}\end{array}\right.$

Wherein, all are masked as 1 some formation foreground area, are that 0 point constitutes the background area.

Next be to eliminate shade.What the method for front obtained is the human region that has shade.Because utilize the dynamic background model can eliminate the dynamic factor of background itself, still, by the shadow region that the people moves and brought in environment, top method can't distinguish itself and human body itself.Yet the shadow region will be handled follow-up tracking and be brought very big error, so we must get rid of the shadow region separately.

We have used this fact in remove the shadow region: shade is compared with background, the brightness of pixel (intensity) deepening, but colourity (chromaticity) has certain consistency.Here, for a some i (b), we can represent colourity with the ratio of three components of rgb for r, g:

rc＝r/(r+g+b)

gc＝g/(r+g+b)

In actual motion detects,, when setting up initial back-ground model, set up a colourity model (μ rc, μ gc) for each point in the background.Then, for the moving region that each two field picture obtains, calculate wherein the colourity of every bit yi (yrc, ygc), and judge by following formula whether it belongs to the shadow region:

$\mathrm{Di}=\left\{\begin{array}{cc}1,& \mathrm{If}((\mathrm{yrc}-\mathrm{\μrc})>\mathrm{thresholdor}(\mathrm{ygc}-\mathrm{\μgc})>\mathrm{threshold})\\ 0& \mathrm{else}\end{array}\right.$

The point of middle mark 0 belongs to shade, and the point of mark 1 belongs to human body.Like this, just shade can have been removed from foreground area.

We carry out post-processed with the mathematical morphology algorithm to the foreground area that obtains at last

In the 3rd step, remove noise.

Impurity and the last accurate area of shade influence tiny on some image borders calculate, and its elimination can accurately must be expressed the feature of different people with the size that guarantees the color value that we are sensed.We carry out post-processed with the mathematical morphology algorithm to the foreground area that obtains, but are to need consideration for the selection of different templates.

Filter with 3 * 3 corrosion template earlier,, then filter, recover original correct foreground area with onesize expansion template to remove scattered noise dotted line.

Corrosion template (Erosion),

Definition: the color of object is white, and background is black.Definition corrosion template is

1 1 1

1 1 1

1 1 1

Template and image are carried out add operation, if having, then the result is 1, otherwise is 0.The effect of template is equivalent to remove the single pixel at object boundary place

Expansion template (Dilation):

Expansion is the inverse operation of corrosion

Template file is

0 0 0

0 0 0

0 0 0

Its effect is equivalent to add single pixel on the border of object

Through so a series of processing, appear near the noise in human body edge in the image, the noise behind the background subtraction, the inside of human body noise all is compared effectively and removes, and makes image become level and smooth

In the 4th step, set up the masking-out network.

Put the difference of angle according to video camera, with ground square net is standard, in the image of gathering, set up grid, the size of grid is decided according to the size of human body under this environment, make the display area of human body can be full of a grid and be as the criterion, and this masking-out network is defined as the masking-out matrix D.Each element in this matrix D is represented human body area ratio/occupancy ratio in the grid in the masking-out network.As shown in Figure 4.

$D=\begin{array}{c}\end{array}\left[\begin{array}{cccccc}a51& a52& a53& a54& a55& a56\\ a41& a42& a43& a44& a45& a46\\ a31& a32& a33& a34& a35& a36\\ a21& a22& a23& a24& a25& a26\\ a11& a12& a13& a14& a15& a16\end{array}\right]$

Defining i among each element aij is this ordinate for the abscissa j of this point.

Because the grid in the image is narrower in edge, so wherein a21, a26, a11, a16 are empty.

The 5th step, the identification of human body.

1. single body identification: a certain element value in the masking-out matrix D is greater than 25%, the position that is the representative place is around this zone, as the intersection point place of human body center at network then is described about 25%, about 50% then human body be in approximately on the sideline of grid, between 50% to 100%, then be in substantially in this grid.

Carry out point by point scanning earlier, begin to detect to 16 points with 11, again from 21 o'clock to 26 o'clock, analogize in proper order with this.When the value that scans continues scanning less than 25%, must be around this element greater than 25% human body, so detect the value of 8 elements on every side of this element, in case numerical value is greater than 5% then the people importantly (may be image noise less than 5% at this point, even making the component of human body also can ignore), then carry out the weighted average of numerical value, and draw the point coordinates (im of a non-integer type, jm), the concrete coordinate point that this point is decided to be this human body.

2. the identification of many human bodies: its basic principle is identical with single body, and difference is the difficulty in the identification of human body.

Since can not be identical on everyone individuality, so the area that he occupied under video camera also is not quite similar,, realize the identification of many human bodies therefore according to this point.Principle is to calculate the shared several P1 ~ Pn of its area altogether when image a plurality of human body occurs simultaneously respectively, when extracting image, calculate the gross area of each human body next time and count Q1 ~ Qn, obtain P1, P2 respectively ... ..Pn poor with Q1 ~ Qn, get absolute to minimum be the correspondence position of this human body.

The automatic door switch situation under many human bodies of effective recognition situation comparatively by this method.

The 6th step, the human motion modeling.

The realization of " goalkeeper's algorithm "

In the process of human motion modeling,,, and created " goalkeeper's algorithm " so associated the goalkeeper in the football because its last target is to detect the people whether in the access door.And well-known, the division that big forbidden zone, goal area are arranged in the football is the big forbidden zone of automatic door and also defined a12, a22, a23, a24, a25, a15 in the masking-out network in this algorithm, and a13, a14 are the goal area.

We have set up rectangular co-ordinate before this.At first we judge the people earlier whether in the goal area, if must open the door (in any case would finding that as the goalkeeper ball all will be puted out a fire to save life and property in order to avoid unexpected in the goal area), if not in the goal area then enter the main body of whole algorithm.

We bring the human body coordinate of twice collection image into linear equation first, calculate it and whether will enter the goal area, as not being the movement locus that then continues to calculate him, judge then in this way whether it has been in big forbidden zone, then open the door in this way and (find that as the goalkeeper ball is to door in big forbidden zone, so will put out a fire to save life and property), as be not then not open the door (ball might be broken to walk or the like) and its movement locus of continuation calculating.

In whole algorithm, work as the people and be among the big forbidden zone, can just not open the door regardless of any situation as common automatic door, the 2nd, people's direction of advance has been judged in realization.And, also just avoided the embarrassment of Men Bukai owing to people's direction of motion sudden change because it must open the door to the human body that is in the goal area.

The frequency that automatic door in many public places uses is very high, and wherein many in special time period, concentrating by a large amount of personnel, another section is then very few in the period, so use automatic door to take all factors into consideration.When visitors flowrate is very big, can in one or two minute, reach hundreds of person-time,, increase suddenly unavoidably and can form confusion though can save energy visitors flowrate if door one heads straight for very for a short time.And if once head straight for very greatly, there be very long there is no need the time, summer, the often indoor and outdoor temperature difference was big still more, if it is big to open energy waste greatly always.So just need this automatic door that have certain intelligence can on-demand, this is that general microwave, infrared induction automatic door institute can not reach.

The present invention can realize the different situations according to the people, changes the opening time and the width of automatic door at any time, caters to different needs.This being controlled in the present automatic door embodied seldom, but is necessary.

Can set up the 2D or the 3D model of human motion, cross calculate and to make certain anticipation, the object that the direction of the direction of motion and door is level is considered as obstructed moving into one's husband's household upon marriage, and has avoided the automatic door fault open, has so just realized intellectuality greatly.

Figure of description

Fig. 1 is the principle schematic of vision technique control automatic door

Fig. 2 is for removing before the noise and afterwards image by the corrosion template

Fig. 3 is for removing by the expansion template before the noise and afterwards image

Fig. 4 gathers grid that ground square is divided in the image and the masking-out matrix D of setting up according to this grid for gamma camera

Fig. 5 is the concrete Coordinate Calculation method of a human body schematic diagram

Fig. 6 is for calculating the schematic diagram that modelling of human body motion is opened with the control automatic door

The specific embodiment

How further specify the present invention in conjunction with specific embodiments realizes:

Embodiment

The first step, the image that is obtained by a CCD camera at first enters in the computer and definite color mode by USB interface.

Second step was at first extracted i.e. initialization background model to background.Nobody enters before the environment, with the image continuous acquisition n width of cloth, and in this model, each the some i in the background, definition μ i is the expectation of the color value of this point, calculates the color value μ it of an i in t width of cloth image according to following formula:

${\mathrm{\μ}}_i=\frac{1}{n}\underset{t=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\μ}}_{\mathrm{it}}$

Subduction background after initial background is set up for the new present image of gathering of each width of cloth, just can have been carried out the extraction of foreground area.If the color value of present image mid point i is yi, by following formula with image binaryzation:

$\mathrm{Di}=\left\{\begin{array}{cc}1& \mathrm{if}(\mathrm{yi}-\mathrm{\μi}>3\mathrm{\σi})\\ 0& \mathrm{else}\end{array}\right.$

Wherein, all are masked as 1 some formation foreground area, are that 0 point constitutes the background area, and what obtain is the human region that has shade.。

Next be to eliminate shade.

Shade is compared with background, the brightness of pixel (intensity) deepening, but colourity (chromaticity) has certain consistency.Here, for a some i (b), we can represent colourity with the ratio of three components of rgb for r, g:

rc＝r/(r+g+b)

gc＝g/(r+g+b)

In actual motion detects,, when setting up initial back-ground model, set up a colourity model (μ rc, μ gc) for each point in the background.Then, for the moving region that each two field picture obtains, calculate wherein the colourity of every bit yi (yrc, ygc), and judge by following formula whether it belongs to the shadow region:

$\mathrm{Di}=\left\{\begin{array}{cc}1,& \mathrm{If}((\mathrm{yrc}-\mathrm{\μrc})>\mathrm{thresholdor}(\mathrm{ygc}-\mathrm{\μgc})>\mathrm{threshold})\\ 0& \mathrm{else}\end{array}\right.$

The point of mark 0 belongs to shade, and the point of mark 1 belongs to human body.Like this, just shade can have been removed from foreground area.

In the 3rd step, remove noise.

Filter with 3 * 3 corrosion template earlier, to remove scattered noise dotted line, the effect of template is equivalent to remove the single pixel at object boundary place; Then filter with onesize expansion template, recover original correct foreground area, its effect is equivalent to add single pixel on the border of object.As shown in Figures 2 and 3.

Corrosion template (Erosion):

Definition: the color of object is white, and background is black.Definition corrosion template is

1 1 1

1 1 1

1 1 1

Template and image are carried out add operation, if having, then the result is 1, otherwise is 0.

Expansion template (Dilation):

Expansion is the inverse operation of corrosion

Template file is

0 0 0

0 0 0

0 0 0

Through so a series of processing, appear near the noise in human body edge in the image, the noise behind the background subtraction, the inside of human body noise all is compared effectively and removes, and makes image become level and smooth.

In the 4th step, set up the masking-out network.

Put the difference of angle according to video camera, according to the size of human body under this environment, with ground square net is standard, in the image of gathering, set up grid, the size of grid be so that the display area of human body can be full of a grid is as the criterion, and this masking-out network is defined as the masking-out matrix D.Each element in this matrix D is represented human body area ratio/occupancy ratio in the grid in the masking-out network.As shown in Figure 4.

$D=\begin{array}{c}\end{array}\left[\begin{array}{cccccc}a51& a52& a53& a54& a55& a56\\ a41& a42& a43& a44& a45& a46\\ a31& a32& a33& a34& a35& a36\\ a21& a22& a23& a24& a25& a26\\ a11& a12& a13& a14& a15& a16\end{array}\right]$

Defining i among each element aij is this ordinate for the abscissa j of this point.

Because the grid in the image is narrower in edge, so wherein a21, a26, a11, a16 are empty.

The 5th step, the identification of human body.

For single human body, a certain element value in the masking-out matrix D is greater than 25%, the position that is the representative place is around this zone, as the intersection point place of human body center at network then is described about 25%, about 50% then human body be in approximately on the sideline of grid, between 50% to 100%, then be in substantially in this grid.

Carry out point by point scanning earlier, begin to detect to 16 points with 11, again from 21 o'clock to 26 o'clock, analogize in proper order with this.When the value that scans continues scanning less than 25%, must be around this element greater than 25% human body, so detect the value of 8 elements on every side of this element, in case numerical value is greater than 5% then the people importantly (may be image noise less than 5% at this point, even making the component of human body also can ignore), then carry out the weighted average of numerical value, and draw the point coordinates (im of a non-integer type, jm), the concrete coordinate point that this point is decided to be this human body.It calculates schematic diagram and sees Fig. 5.

Under the situation of many human bodies, because can not be identical on everyone individuality, so the area that he occupied under video camera also is not quite similar.So, realize the identification of many human bodies according to this point.Principle is to calculate the shared several P1 ~ Pn of its area altogether when image a plurality of human body occurs simultaneously respectively, when extracting image, calculate the gross area of each human body next time and count Q1 ~ Qn, obtain P1, P2 respectively ... ..Pn poor with Q1 ~ Qn, get absolute to minimum be the correspondence position of this human body.

The 6th step, human motion modeling, the realization of " goalkeeper's algorithm "

In the process of human motion modeling,, utilize " goalkeeper's algorithm " to calculate because its last target is to detect the people whether in the access door.According to the division that big forbidden zone, goal area are arranged in the football, having defined a12, a22, a23, a24, a25, a15 in the masking-out network is the big forbidden zone of automatic door, and a13, a14 are the goal area.

We have set up rectangular co-ordinate before this.At first we judge the people earlier whether in the goal area, if must open the door, if not in the goal area then enter the main body of whole algorithm.

We bring the human body coordinate of twice collection image into linear equation first, calculate it and whether will enter the goal area, as not being the movement locus that then continues to calculate him, judge then in this way whether it has been in big forbidden zone, then open the door in this way, as be not then not open the door and continue to calculate its movement locus.As shown in Figure 6.

Claims (2)

1. an automatic door is characterized in that, is controlled time and the width of whether opening, opening by vision sensor.

2. the automatic door of claim 1 is characterized in that, the step that described vision sensor is controlled automatic door comprises background subtraction, removes noise, sets up the masking-out network, human body identification and modelling of human body motion are set up.

Images