CN108230351A - Sales counter evaluation method and system based on binocular stereo vision pedestrian detection - Google Patents
Sales counter evaluation method and system based on binocular stereo vision pedestrian detection Download PDFInfo
- Publication number
- CN108230351A CN108230351A CN201611157648.5A CN201611157648A CN108230351A CN 108230351 A CN108230351 A CN 108230351A CN 201611157648 A CN201611157648 A CN 201611157648A CN 108230351 A CN108230351 A CN 108230351A
- Authority
- CN
- China
- Prior art keywords
- target
- pedestrian
- value
- sales counter
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Abstract
The invention discloses the sales counter evaluation method and system of a kind of view-based access control model pedestrian detection, for judging attraction and commercial value of the sales counter to pedestrian.System includes the modules such as binocular parallel vidicon, embeded processor, communication.System tilts installation by sales counter side, the pedestrian of dealing is detected in real time, method and step is：Left and right cameras is demarcated, Accurate Calibration principal point is poor；Synchronous acquisition left images detect foreground moving object by mixed Gaussian background modeling；The anaglyph of foreground target is obtained according to adaptive window solid matching method；Generate the two-dimensional projection image of foreground target on the ground；Pedestrian is detected, is tracked, is counted and average speed calculates；Carry out the analysis and evaluation of sales counter attraction and commercial value.The present invention is using binocular camera and principle of stereoscopic vision detection pedestrian target, and algorithm robustness is high, strong interference immunity, and accuracy of detection is high, and evaluation index is with practical value.
Description
Technical field
The present invention relates to computer vision technique application fields more particularly to one kind to be based on binocular stereo vision pedestrian detection
Sales counter evaluation method and system.
Background technology
As the improvement of people's living standards, it goes shopping, doing shopping has become a kind of common free life mode.For quotient
For family, then more concerned with the sales counter in market to the attraction of pedestrian and the commercial value of sales counter.Pedestrian's average speed is displacement
With the ratio of time, interest level of the pedestrian to sales counter can be weighed to a certain extent.By the speed of pedestrian before sales counter
Degree is slower, and the attraction for reflecting the sales counter is bigger；And the commercial value of sales counter is with passing through the pedestrian's quantity and its speed before sales counter
Spend related, quantity is more, the speed the slow, is worth bigger.It, can by the quantity and speed of pedestrian before intelligent measurement sales counter
Effectively to analyze sales counter to the attraction of customer and the commercial value of sales counter.
Computer intelligence video analysis is an emerging developing direction in artificial intelligence field and the class that is widely noticed
Topic, core is computer vision technique.Computer vision obtains image or image sequence by imaging sensor, and then passes through
Computer understands these images using the means analysis that the technologies such as image procossing, pattern-recognition, artificial intelligence are combined, to three-dimensional
The world is described and explains that final goal is human brain can be replaced to complete the understanding to real world and understanding.
Binocular Stereo Vision System is a kind of advanced computer vision system, and the solid that it simulates mankind's eyes perceived
Journey obtains plan view using binocular camera from two different positions, can analyze to obtain the three-dimensional depth information of object, energy
It is enough effectively to avoid the variation of scene light, shade, perspective effect and the influence blocked.Skill is detected compared to common monocular vision
Art, using technique of binocular stereoscopic vision can the different application scenarios of flexible adaptation, obtain better pedestrian detection precision, Jin Erke
More accurately to assess the attraction of sales counter and commercial value.
Invention content
In view of presently, there are above-mentioned deficiency, the present invention provides a kind of sales counter based on binocular stereo vision pedestrian detection and comments
Valency method and system can detect and analyze in market pedestrian's speed, the system of quantity before sales counter, and and then provide the suction of sales counter
Gravitation and commercial value.
In order to achieve the above objectives, the embodiment of the present invention adopts the following technical scheme that：
A kind of pedestrian detecting system based on binocular stereo vision, including binocular parallel vidicon, dsp processor, communication
Module and upper host, the binocular parallel vidicon include left video camera A1, right video camera A2, respectively with dsp processor phase
Even, for acquiring left images；The dsp processor is connected with upper host, is passed for data communication and real-time digital video
It is defeated.
A kind of sales counter evaluation method based on binocular stereo vision pedestrian detection, is detected and is commented using above system
Valency the described method comprises the following steps：
Left and right cameras is demarcated, Accurate Calibration principal point is poor；
Synchronous acquisition left images detect foreground moving object by mixed Gaussian background modeling；
The anaglyph of foreground target is obtained according to adaptive window solid matching method；
Generate the two-dimensional projection image of foreground target on the ground；
Pedestrian is detected, is tracked, is counted and average speed calculates；
Carry out the analysis and evaluation of sales counter attraction and commercial value.
According to one aspect of the present invention, the step demarcates left and right cameras, and Accurate Calibration principal point difference includes：Calibration is left
Right video camera obtains camera interior and exterior parameter, then carries out principal point difference calibration to left and right cameras by the way of human-computer interaction, obtains
Accurate principal point difference parameter is obtained, determines the detection range of system.
According to one aspect of the present invention, the calibration left and right cameras obtains camera interior and exterior parameter, then using man-machine
Interactive mode carries out principal point difference calibration to left and right cameras, obtains accurate principal point difference parameter, determines the detection range of system,
Its is specific as follows：
Left and right cameras acquires several pictures respectively, is demarcated, is can obtain in video camera according to Zhang Shi calibration algorithms
Outer parameter；
Binocular camera is lain in into certain altitude, shoots ground, the height of manual measurement video camera from the ground vertically downward
Degree；
According to parallel stereopsis range measurement principle, theoretical parallax value is calculated, formula is：
Wherein, lr-llFor the theoretical parallax value of same position pixel horizontal direction, f, B and H are respectively the coke of video camera
Away from parallax range and video camera terrain clearance between, left and right cameras；
Any feature point (x is chosen in right viewr,yr), for ease of matching, general fetch bit is in picture centre and texture
The pixel in apparent region；By Region Matching method, characteristic point (x is searched in left viewr,yr) corresponding pixel points (xl,
yl), obtain practical parallax value xl-xrAnd yl-yr；
Theoretical parallax value and practical parallax value are done into mathematic interpolation, obtained accurate on two coordinate directions of binocular camera
Principal point difference value；
Left and right detection boundary a, b in image manually delimited on acquisition image, determines pedestrian detection region.
According to one aspect of the present invention, the step synchronous acquisition left images are examined by mixed Gaussian background modeling
Foreground moving object is surveyed to include：Synchronous acquisition left images carry out the detection of foreground moving object using background subtraction and carry
Take including：Synchronous acquisition left images are built image by mixed Gauss model the background image of scene, utilize background difference
Method carries out the detection and extraction of foreground moving object, specific as follows,
Utilize mixed Gauss model structure image background frame Bk, for sometime t, the mixed Gaussian of any background pixel
Probability distribution is represented by：
ωit=(1- α) ωi(t-1)+αMNt (3)
Wherein N represents the number of mixed Gauss model, ωitIt is the weights of i-th of Gauss model of t moment, η is Gaussian Profile
Function, learning rate are α.Model Matching then MNt=1, otherwise MNt=0.When pixel and i-th of Model Matching, Gaussian Profile
In mean value and variance more new formula it is as follows：
μit=(1- βi)μi(t-1)+βixt (4)
Using background subtraction, by the present frame f of video imagekWith background image BkSubtract each other, if the two pixel value difference DkGreatly
In a certain threshold value T, you can it is the pixel on moving target to judge this pixel, is otherwise background pixel point.After detecting prospect, mistake
Background is filtered, is partitioned into prospect gk(x,y).It is formulated as follows：
Dk(x, y)=| fk(x,y)-Bk(x,y)| (6)
According to one aspect of the present invention, the step obtains foreground target according to adaptive window solid matching method
Anaglyph include：To having extracted the foreground target region of acquisition, stood according to adaptive window solid matching method
Body matches, and obtains the anaglyph of foreground target, specific as follows：
Using Region Matching method, matched on corresponding horizontal scanning line according to calibration principal point difference.Home window is big
Small is 15*3；
The statistical pixel gray value variance in home window if less than presetting threshold value, then illustrates home window institute
Comprising texture information it is insufficient, then by match window width expansion 50%, then proceed to calculate gray value variance and be compared
Compared with 1/5 until reaching picture traverse；
Measure for Image Similarity is mutual using NCC (Normalized Cross-Correlation) normalization in match window
Correlation function.Type interpolation twice is carried out to estimating NCC (x, y, d) value after Pixel-level Likelihood Computation, obtains NCC (x, y, d) maximums
The corresponding offset d of value is the subpixel accuracy parallax of gained.
According to one aspect of the present invention, the two-dimensional projection image of the step generation foreground target on the ground includes：
It is specific as follows using the two-dimensional projection image of calibration result and foreground target parallax generation foreground target on the ground：
According to disparity computation depth information.Using parallel binocular range measurement principle, the camera coordinates of each foreground pixel are acquired
System (Xc, Yc, Zc) in Zc be：
Wherein d is pixel parallax value, and f, B are respectively parallax range between the focal length of video camera, left and right cameras；
The Xc of camera coordinate system, Yc are converted into from pixel coordinate system (u, v).Calculation formula is：
Wherein, u0, v0 are video camera principal points；Dx, dy are physical size of the pixel in x-axis and y-axis direction；
(u, v) pixel can be obtained by conversion relational expression of the world coordinate system (Xw, Yw, Zw) between camera coordinate system
The value of the corresponding world coordinate system of point, conversion formula are：
Formula China and foreign countries parameter R, T are obtained by calibration.(Xw, Yw) represents the pixel in the position of world coordinate system, Zw tables
Show the height of target；
Generate two-dimentional ground projected image of the foreground target on world coordinate system Ow-XwYw.Target is on projection images
Gray value represented with height Zw.
According to one aspect of the present invention, the step is existed using calibration result and foreground target parallax generation foreground target
Two-dimensional projection image on ground includes：Can shade flase drop region be filtered according to the value range of pedestrian level.
According to one aspect of the present invention, the step is detected pedestrian, tracks, counting and average speed calculates packet
It includes：Pedestrian is detected and tracked in projected image, calculate the average speed of pedestrian and carries out twocouese pedestrian counting, is had
Body is as follows：
Pedestrian target detects.Moving target block in perspective view is marked, removal noise and interference Small object, with each
The barycenter of object block represents the position of target, while records its average gray；
Pedestrian target matched jamming.On adjacent two frame of video, when best match target is found from multiple targets, definition
For weighing the similarity of two matching targets, cost function is defined as follows a kind of cost function：
In formula, S (x, yn) is the absolute value of the difference of the projection gray level average value of target x and target yn, Dx (x, yn) for x and
The central point of yn image row direction distance, Dy (x, yn) be the central point of x and yn in the distance of image column direction, a, b's
Value represents above-mentioned two measurement standard shared weight in cost function respectively, can be adjusted according to actual conditions.F (x, yn) letter
Numerical value is smaller, illustrates that two target similarities are bigger, therefore if target yn makes the value of function F (x, yn) reach minimum, then under
The best match target that target yn on one frame image is previous frame target x；
Pedestrian's average speed calculates and walking direction.It is true according to its direction of motion when target crosses left bounding lines a, b
Surely it is newly into target or to leave detection zone target.To newly into target, recording its entry time and position；To leaving region
Its time departure of target record and position calculate its average speed Jing Guo detection zone, and to the counting of respective direction accordingly
Quantity adds one.
According to one aspect of the present invention, the step carries out the analysis of sales counter attraction and commercial value and evaluation packet
It includes：According to the average speed and quantity of pedestrian before sales counter, statistical analysis sales counter is to the attraction and commercial value of pedestrian, specifically such as
Under：
Record the quantity and everyone average speed of the pedestrian of all processes in a period of time；
Sales counter attraction is described with three indexs, first index is：
W=P (v0.4)+0.8P(v0.8)+0.6P(v1.2)+0.4P(v1.6)+0.2P(v1.6) (12)
P (v in formula0.4)、P(v0.8)、P(v1.2)、P(v1.6)、P(v 1.6) represent that passing number is in speed in a period of time respectively
0~0.4m/s, 0.4~0.8m/s, 0.8~1.2m/s, 1.2~1.6m/s are spent, the probability on >=1.6m/s sections；
W value ranges are [0.2,1], divide four sections：0.2~0.4,0.4~0.6,0.6~0.8,0.8~1, point
It does not represent four attraction degree incrementally, it is bigger to be worth bigger attraction；
Average value of second index for a period of time all pedestrian's average speed inverses：
In formula n represent a period of time by total number of persons, vi represent i-th of people by average speed.V is bigger, sales counter
It is bigger to pedestrian's attraction, on the contrary it is smaller；
Third index is that average speed is less than 1.2m/s (size can be according to practical feelings in the total number of persons passed through before sales counter
Condition adjust) pedestrian's accounting, ratio it is higher reflection sales counter attraction it is bigger；
Reflect sales counter commercial value with the total number of persons, the mean value two indices of crowd's average speed that pass through in a period of time,
Interested crowd's quantity is bigger, average speed mean value is smaller, then sales counter commercial value is bigger, on the contrary then smaller.
The advantages of present invention is implemented：The present invention, using principle of stereoscopic vision, obtains pedestrian target using binocular camera
Depth information, solve monocular vision technique generally existing it is sensitive to light variation and complex background, easily by shadow interference
Problem；The projected image of two-dimentional ground level is mapped to using target, realizes the tracking to pedestrian and the calculating of average speed.This hair
The sales counter attraction based on binocular stereo vision pedestrian detection of bright proposition and value analysis system, can be effectively to taking
Pedestrian on the preceding passageway of market sales counter is detected, and algorithm robustness is high, strong interference immunity, accurately can carry out both sides to pedestrian
To the calculating counted with average speed.
Description of the drawings
It to describe the technical solutions in the embodiments of the present invention more clearly, below will be to needed in the embodiment
Attached drawing is briefly described, it should be apparent that, the accompanying drawings in the following description is only some embodiments of the present invention, for ability
For the those of ordinary skill of domain, without creative efforts, it can also be obtained according to these attached drawings other attached
Figure.
Fig. 1 is that a kind of hardware system of pedestrian detecting system based on binocular stereo vision of the present invention forms signal
Figure；
Fig. 2 is a kind of pedestrian detection method schematic diagram of binocular stereo vision of the present invention；
Fig. 3 is a kind of sales counter evaluation method schematic diagram based on binocular stereo vision pedestrian detection of the present invention.
Specific embodiment
Below in conjunction with the attached drawing in the embodiment of the present invention, the technical solution in the embodiment of the present invention is carried out clear, complete
Site preparation describes, it is clear that described embodiment is only part of the embodiment of the present invention, instead of all the embodiments.It is based on
Embodiment in the present invention, those of ordinary skill in the art are obtained every other without making creative work
Embodiment shall fall within the protection scope of the present invention.
As shown in Figure 1, a kind of pedestrian detecting system based on binocular stereo vision, at binocular parallel vidicon, DSP
Manage device, communication module and upper host, the binocular parallel vidicon includes left video camera A1, right video camera A2, respectively with DSP
Processor is connected, for acquiring left images；The dsp processor is connected by communication module with upper host, for data
Communication and real-time digital video transmission.The communication module can be network interface, WIFI communication modules etc..
As shown in Fig. 2, the application scenarios of this system are the detections carried out on the passageway in market using pedestrian as target.According to
The actual conditions of market arrangement, this system camera apparatus are designed to install by sales counter side, tilt passageway video image；
In order to which the view that left and right cameras acquires is made to be easy to Stereo matching, two video cameras need to be in the base between sustained height, two video cameras
Line is parallel to passageway direction.
As shown in figure 3, a kind of sales counter evaluation method based on binocular stereo vision pedestrian detection, is carried out using above system
Detection and evaluation, the described method comprises the following steps：
Step S1：Left and right cameras is demarcated, Accurate Calibration principal point is poor；
The step demarcates left and right cameras, and Accurate Calibration principal point difference includes：Left and right cameras is demarcated to obtain in video camera
Outer parameter, then principal point difference calibration is carried out to left and right cameras by the way of human-computer interaction, accurate principal point difference parameter is obtained, really
Determine the detection range of system.Its specific embodiment is：
Left and right cameras acquires several pictures respectively, is demarcated, is can obtain in video camera according to Zhang Shi calibration algorithms
Outer parameter；
Binocular camera is lain in into certain altitude, shoots ground, the height of manual measurement video camera from the ground vertically downward
Degree；
According to parallel stereopsis range measurement principle, theoretical parallax value is calculated, formula is：
Wherein, lr-llFor the theoretical parallax value of same position pixel horizontal direction, f, B and H are respectively the coke of video camera
Away from parallax range and video camera terrain clearance between, left and right cameras；
Any feature point (x is chosen in right viewr,yr), for ease of matching, general fetch bit is in picture centre and texture
The pixel in apparent region；By Region Matching method, characteristic point (x is searched in left viewr,yr) corresponding pixel points (xl,
yl), obtain practical parallax value xl-xrAnd yl-yr；
Theoretical parallax value and practical parallax value are done into mathematic interpolation, obtained accurate on two coordinate directions of binocular camera
Principal point difference value；
Left and right detection boundary a, b in image manually delimited on acquisition image, determines pedestrian detection region.
Step S2：Synchronous acquisition left images detect foreground moving object by mixed Gaussian background modeling；
The step synchronous acquisition left images detect foreground moving object by mixed Gaussian background modeling and include：Together
Step acquisition left images, carry out the detection of foreground moving object using background subtraction and extraction include：Synchronous acquisition or so figure
Picture builds image by mixed Gauss model the background image of scene, and foreground moving object is carried out using background subtraction
Detection and extraction, it is specific as follows,
Utilize mixed Gauss model structure image background frame Bk, for sometime t, the mixed Gaussian of any background pixel
Probability distribution is represented by：
ωit=(1- α) ωi(t-1)+αMNt (3)
Wherein N represents the number of mixed Gauss model, ωitIt is the weights of i-th of Gauss model of t moment, η is Gaussian Profile
Function, learning rate are α.Model Matching then MNt=1, otherwise MNt=0.When pixel and i-th of Model Matching, Gaussian Profile
In mean value and variance more new formula it is as follows：
μit=(1- βi)μi(t-1)+βixt (4)
Using background subtraction, by the present frame f of video imagekWith background image BkSubtract each other, if the two pixel value difference DkGreatly
In a certain threshold value T, you can it is the pixel on moving target to judge this pixel, is otherwise background pixel point.After detecting prospect, mistake
Background is filtered, is partitioned into prospect gk(x,y).It is formulated as follows：
Dk(x, y)=| fk(x,y)-Bk(x,y)| (6)
Step S3：The anaglyph of foreground target is obtained according to adaptive window solid matching method；
According to one aspect of the present invention, the step obtains foreground target according to adaptive window solid matching method
Anaglyph include：To having extracted the foreground target region of acquisition, stood according to adaptive window solid matching method
Body matches, and obtains the anaglyph of foreground target, specific as follows：
Using Region Matching method, matched on corresponding horizontal scanning line according to calibration principal point difference.Home window is big
Small is 15*3；
The statistical pixel gray value variance in home window if less than presetting threshold value, then illustrates home window institute
Comprising texture information it is insufficient, then by match window width expansion 50%, then proceed to calculate gray value variance and be compared
Compared with 1/5 until reaching picture traverse；
Measure for Image Similarity is mutual using NCC (NormalizedCross-Correlation) normalization in match window
Correlation function.Type interpolation twice is carried out to estimating NCC (x, y, d) value after Pixel-level Likelihood Computation, obtains NCC (x, y, d) maximums
The corresponding offset d of value is the subpixel accuracy parallax of gained.
Step S4：Generate the two-dimensional projection image of foreground target on the ground；
The two-dimensional projection image of the step generation foreground target on the ground includes：Utilize calibration result and foreground target
The two-dimensional projection image of parallax generation foreground target on the ground, it is specific as follows：
According to disparity computation depth information.Using parallel binocular range measurement principle, the camera coordinates of each foreground pixel are acquired
System (Xc, Yc, Zc) in Zc be：
Wherein d is pixel parallax value, and f, B are respectively parallax range between the focal length of video camera, left and right cameras；
The Xc of camera coordinate system, Yc are converted into from pixel coordinate system (u, v).Calculation formula is：
Wherein, u0, v0 are video camera principal points；Dx, dy are physical size of the pixel in x-axis and y-axis direction；
(u, v) pixel can be obtained by conversion relational expression of the world coordinate system (Xw, Yw, Zw) between camera coordinate system
The value of the corresponding world coordinate system of point, conversion formula are：
Formula China and foreign countries parameter R, T are obtained by calibration.(Xw, Yw) represents the pixel in the position of world coordinate system, Zw tables
Show the height of target；
Generate two-dimentional ground projected image of the foreground target on world coordinate system Ow-XwYw.Target is on projection images
Gray value represented with height Zw.
In practical application, can the flase drops regions such as shade be filtered according to the value range of pedestrian level.
Step S5：Pedestrian is detected, is tracked, is counted and average speed calculates；
The step is detected pedestrian, tracks, counting and average speed calculating includes：To pedestrian in projected image
It is detected and tracks, calculate the average speed of pedestrian and carry out twocouese pedestrian counting, it is specific as follows：
Pedestrian target detects.Moving target block in perspective view is marked, removal noise and interference Small object, with each
The barycenter of object block represents the position of target, while records its average gray；
Pedestrian target matched jamming.On adjacent two frame of video, when best match target is found from multiple targets, definition
For weighing the similarity of two matching targets, cost function is defined as follows a kind of cost function：
In formula, S (x, yn) is the absolute value of the difference of the projection gray level average value of target x and target yn, Dx (x, yn) for x and
The central point of yn image row direction distance, Dy (x, yn) be the central point of x and yn in the distance of image column direction, a, b's
Value represents above-mentioned two measurement standard shared weight in cost function respectively, can be adjusted according to actual conditions.F (x, yn) letter
Numerical value is smaller, illustrates that two target similarities are bigger, therefore if target yn makes the value of function F (x, yn) reach minimum, then under
The best match target that target yn on one frame image is previous frame target x；
Pedestrian's average speed calculates and walking direction.It is true according to its direction of motion when target crosses left bounding lines a, b
Surely it is newly into target or to leave detection zone target.To newly into target, recording its entry time and position；To leaving region
Its time departure of target record and position calculate its average speed Jing Guo detection zone, and to the counting of respective direction accordingly
Quantity adds one.
Step S6：Carry out the analysis and evaluation of sales counter attraction and commercial value.
The step carries out sales counter attraction and the analysis and evaluation of commercial value include：It is averaged according to pedestrian before sales counter
Speed and quantity, statistical analysis sales counter are specific as follows to the attraction and commercial value of pedestrian：
Record the quantity and everyone average speed of the pedestrian of all processes in a period of time；
Sales counter attraction is described with three indexs, first index is：
W=P (v0.4)+0.8P(v0.8)+0.6P(v1.2)+0.4P(v1.6)+0.2P(v1.6) (12)
P (v in formula0.4)、P(v0.8)、P(v1.2)、P(v1.6)、P(v 1.6) represent that passing number is in speed in a period of time respectively
0~0.4m/s, 0.4~0.8m/s, 0.8~1.2m/s, 1.2~1.6m/s are spent, the probability on >=1.6m/s sections；
W value ranges are [0.2,1], divide four sections：0.2~0.4,0.4~0.6,0.6~0.8,0.8~1, point
It does not represent four attraction degree incrementally, it is bigger to be worth bigger attraction；
Average value of second index for a period of time all pedestrian's average speed inverses：
In formula n represent a period of time by total number of persons, vi represent i-th of people by average speed.V is bigger, sales counter
It is bigger to pedestrian's attraction, on the contrary it is smaller；
Third index is that average speed is less than 1.2m/s (size can be according to practical feelings in the total number of persons passed through before sales counter
Condition adjust) pedestrian's accounting, ratio it is higher reflection sales counter attraction it is bigger；
Reflect sales counter commercial value with the total number of persons, the mean value two indices of crowd's average speed that pass through in a period of time,
Interested crowd's quantity is bigger, average speed mean value is smaller, then sales counter commercial value is bigger, on the contrary then smaller.
The advantages of present invention is implemented：The present invention, using principle of stereoscopic vision, obtains pedestrian target using binocular camera
Depth information, solve monocular vision technique generally existing it is sensitive to light variation and complex background, easily by shadow interference
Problem；The projected image of two-dimentional ground level is mapped to using target, realizes the tracking to pedestrian and the calculating of average speed.This hair
The sales counter attraction based on binocular stereo vision pedestrian detection of bright proposition and value analysis system, can be effectively to taking
Pedestrian on the preceding passageway of market sales counter is detected, and algorithm robustness is high, strong interference immunity, accurately can carry out both sides to pedestrian
To the calculating counted with average speed.
The above description is merely a specific embodiment, but protection scope of the present invention is not limited thereto, any
Those skilled in the art is in technical scope disclosed by the invention, the change or replacement that can readily occur in, all should
It is included within the scope of the present invention.Therefore, protection scope of the present invention should using the scope of the claims as
It is accurate.
Claims (10)
1. a kind of pedestrian detecting system based on binocular stereo vision, which is characterized in that at binocular parallel vidicon, DSP
Manage device, communication module and upper host, the binocular parallel vidicon includes left video camera A1, right video camera A2, respectively with DSP
Processor is connected, for acquiring left images；The dsp processor is connected with upper host, counts for data communication and in real time
Word transmission of video.
2. a kind of sales counter evaluation method based on binocular stereo vision pedestrian detection, is examined using system described in claim 1
It surveys and evaluates, which is characterized in that the described method comprises the following steps：
Left and right cameras is demarcated, Accurate Calibration principal point is poor；
Synchronous acquisition left images detect foreground moving object by mixed Gaussian background modeling；
The anaglyph of foreground target is obtained according to adaptive window solid matching method；
Generate the two-dimensional projection image of foreground target on the ground；
Pedestrian is detected, is tracked, is counted and average speed calculates；
Carry out the analysis and evaluation of sales counter attraction and commercial value.
3. the sales counter evaluation method according to claim 2 based on binocular stereo vision pedestrian detection, which is characterized in that institute
Step calibration left and right cameras is stated, Accurate Calibration principal point difference includes：It demarcates left and right cameras and obtains camera interior and exterior parameter, then adopt
Principal point difference calibration is carried out to left and right cameras with the mode of human-computer interaction, accurate principal point difference parameter is obtained, determines the inspection of system
Survey range.
4. the sales counter evaluation method according to claim 3 based on binocular stereo vision pedestrian detection, which is characterized in that institute
State calibration left and right cameras obtain camera interior and exterior parameter, then by the way of human-computer interaction to left and right cameras carry out principal point it is poor
Calibration, obtains accurate principal point difference parameter, determines the detection range of system, specific as follows：
Left and right cameras acquires several pictures respectively, is demarcated according to Zhang Shi calibration algorithms, can obtain the inside and outside ginseng of video camera
Number；
Binocular camera is lain in into certain altitude, shoots ground, the height of manual measurement video camera from the ground vertically downward；
According to parallel stereopsis range measurement principle, theoretical parallax value is calculated, formula is：
Wherein, lr-llFor the theoretical parallax value of same position pixel horizontal direction, f, B and H are respectively the focal length of video camera, a left side
Parallax range and video camera terrain clearance between right video camera；
Any feature point (x is chosen in right viewr,yr), for ease of matching, general fetch bit in picture centre and texture it is apparent
The pixel in region；By Region Matching method, characteristic point (x is searched in left viewr,yr) corresponding pixel points (xl,yl),
Obtain practical parallax value xl-xrAnd yl-yr；
Theoretical parallax value and practical parallax value are done into mathematic interpolation, obtain the accurate master on two coordinate directions of binocular camera
Point difference value；
Left and right detection boundary a, b in image manually delimited on acquisition image, determines pedestrian detection region.
5. the sales counter evaluation method according to claim 2 based on binocular stereo vision pedestrian detection, which is characterized in that institute
Step synchronous acquisition left images are stated, detecting foreground moving object by mixed Gaussian background modeling includes：Synchronous acquisition or so
Image builds image by mixed Gauss model the background image of scene, and foreground moving object is carried out using background subtraction
Detection and extraction, it is specific as follows,
Utilize mixed Gauss model structure image background frame Bk, for sometime t, the mixed Gaussian probability of any background pixel
Distribution is represented by：
ωit=(1- α) ωi(t-1)+αMNt (3)
Wherein N represents the number of mixed Gauss model, ωitIt is the weights of i-th of Gauss model of t moment, η is Gaussian Profile letter
Number, learning rate is α.Model Matching then MNt=1, otherwise MNt=0.When pixel and i-th of Model Matching, in Gaussian Profile
Mean value and variance more new formula it is as follows：
μit=(1- βi)μi(t-1)+βixt (4)
Using background subtraction, by the present frame f of video imagekWith background image BkSubtract each other, if the two pixel value difference DkBig Mr. Yu
One threshold value T, you can it is the pixel on moving target to judge this pixel, is otherwise background pixel point.After detecting prospect, the filtering back of the body
Scape is partitioned into prospect gk(x,y).It is formulated as follows：
Dk(x, y)=| fk(x,y)-Bk(x,y)| (6)
6. the sales counter evaluation method according to claim 2 based on binocular stereo vision pedestrian detection, which is characterized in that institute
It states step the anaglyph of foreground target is obtained according to adaptive window solid matching method and include：Before having extracted acquisition
Scape target area carries out Stereo matching according to adaptive window solid matching method, obtains the anaglyph of foreground target, has
Body is as follows：
Using Region Matching method, matched on corresponding horizontal scanning line according to calibration principal point difference.Initial window size is
15*3；
The statistical pixel gray value variance in home window if less than presetting threshold value, then illustrates that home window is included
Texture information it is insufficient, then by match window width expansion 50%, then proceed to calculate gray value variance and be compared, directly
Reach the 1/5 of picture traverse；
Measure for Image Similarity uses NCC (Normalized Cross-Correlation) normalized crosscorrelation in match window
Function.Type interpolation twice is carried out to estimating NCC (x, y, d) value after Pixel-level Likelihood Computation, obtains NCC (x, y, d) maximum value institute
Corresponding offset d is the subpixel accuracy parallax of gained.
7. the sales counter evaluation method according to claim 2 based on binocular stereo vision pedestrian detection, which is characterized in that institute
The two-dimensional projection image of step generation foreground target on the ground is stated to include：Before calibration result and the generation of foreground target parallax
The two-dimensional projection image of scape target on the ground, it is specific as follows：
According to disparity computation depth information.Using parallel binocular range measurement principle, the camera coordinate system of each foreground pixel is acquired
Zc in (Xc, Yc, Zc) is：
Wherein d is pixel parallax value, and f, B are respectively parallax range between the focal length of video camera, left and right cameras；
The Xc of camera coordinate system, Yc are converted into from pixel coordinate system (u, v).Calculation formula is：
Wherein, u0, v0 are video camera principal points；Dx, dy are physical size of the pixel in x-axis and y-axis direction；
(u, v) pixel pair can be obtained by conversion relational expression of the world coordinate system (Xw, Yw, Zw) between camera coordinate system
The value for the world coordinate system answered, conversion formula are：
Formula China and foreign countries parameter R, T are obtained by calibration.(Xw, Yw) represents the pixel in the position of world coordinate system, and Zw represents mesh
Target height；
Generate two-dimentional ground projected image of the foreground target on world coordinate system Ow-XwYw.The ash of target on projection images
Angle value is represented with height Zw.
8. the sales counter evaluation method according to claim 7 based on binocular stereo vision pedestrian detection, which is characterized in that institute
Step is stated using the two-dimensional projection image of calibration result and foreground target parallax generation foreground target on the ground to include：It can basis
The value range of pedestrian level, filtering shade flase drop region.
9. the sales counter evaluation method according to claim 2 based on binocular stereo vision pedestrian detection, which is characterized in that institute
State step pedestrian is detected, is tracked, is counted and average speed calculating include：Pedestrian is detected in projected image with
Tracking, calculates the average speed of pedestrian and carries out twocouese pedestrian counting, specific as follows：
Pedestrian target detects.Moving target block in perspective view is marked, removal noise and interference Small object, with each target
The barycenter of block represents the position of target, while records its average gray；
Pedestrian target matched jamming.On adjacent two frame of video, when best match target is found from multiple targets, definition is a kind of
For weighing the similarity of two matching targets, cost function is defined as follows cost function：
In formula, S (x, yn) is the absolute value of the difference of the projection gray level average value of target x and target yn, and Dx (x, yn) is x's and yn
Central point image row direction distance, Dy (x, yn) be x and yn central point in the distance of image column direction, the value of a, b
Above-mentioned two measurement standard shared weight in cost function is represented respectively, can be adjusted according to actual conditions.F (x, yn) functional value
It is smaller, illustrate that two target similarities are bigger, therefore if target yn makes the value of function F (x, yn) reach minimum, then next frame
The best match target that target yn on image is previous frame target x；
Pedestrian's average speed calculates and walking direction.When target crosses left bounding lines a, b, determined according to its direction of motion be
Detection zone target is newly still left into target.To newly into target, recording its entry time and position；To leaving the target in region
Its time departure and position are recorded, calculates its average speed Jing Guo detection zone accordingly, and to the count number of respective direction
Add one.
10. the sales counter evaluation method according to claim 2 based on binocular stereo vision pedestrian detection, which is characterized in that
The step carries out sales counter attraction and the analysis and evaluation of commercial value include：According to the average speed and number of pedestrian before sales counter
Amount, statistical analysis sales counter are specific as follows to the attraction and commercial value of pedestrian：
Record the quantity and everyone average speed of the pedestrian of all processes in a period of time；
Sales counter attraction is described with three indexs, first index is：
W=P (v0.4)+0.8P(v0.8)+0.6P(v1.2)+0.4P(v1.6)+0.2P(v1.6) (12)
P (v in formula0.4)、P(v0.8)、P(v1.2)、P(v1.6)、P(v1.6) respectively represent a period of time in passing number speed 0~
0.4m/s, 0.4~0.8m/s, 0.8~1.2m/s, 1.2~1.6m/s, the probability on >=1.6m/s sections；
W value ranges are [0.2,1], divide four sections：0.2~0.4,0.4~0.6,0.6~0.8,0.8~1, it passs respectively
Increase and represent four attraction degree, it is bigger to be worth bigger attraction；
Average value of second index for a period of time all pedestrian's average speed inverses：
In formula n represent a period of time by total number of persons, vi represent i-th of people by average speed.V is bigger, and sales counter is to row
People's attraction is bigger, otherwise smaller；
Third index is that average speed is less than 1.2m/s (size can be according to actual conditions tune in the total number of persons passed through before sales counter
It is whole) pedestrian's accounting, ratio it is higher reflection sales counter attraction it is bigger；
Reflect sales counter commercial value with the total number of persons, the mean value two indices of crowd's average speed that pass through in a period of time, sense is emerging
Interesting crowd's quantity is bigger, average speed mean value is smaller, then sales counter commercial value is bigger, on the contrary then smaller.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611157648.5A CN108230351A (en) | 2016-12-15 | 2016-12-15 | Sales counter evaluation method and system based on binocular stereo vision pedestrian detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611157648.5A CN108230351A (en) | 2016-12-15 | 2016-12-15 | Sales counter evaluation method and system based on binocular stereo vision pedestrian detection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108230351A true CN108230351A (en) | 2018-06-29 |
Family
ID=62650324
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611157648.5A Pending CN108230351A (en) | 2016-12-15 | 2016-12-15 | Sales counter evaluation method and system based on binocular stereo vision pedestrian detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108230351A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740537A (en) * | 2019-01-03 | 2019-05-10 | 广州广电银通金融电子科技有限公司 | The accurate mask method and system of pedestrian image attribute in crowd's video image |
| CN110766740A (en) * | 2019-10-31 | 2020-02-07 | 西安咏圣达电子科技有限公司 | Real-time high-precision binocular ranging system and method based on pedestrian tracking |
| CN111553252A (en) * | 2020-04-24 | 2020-08-18 | 福建农林大学 | Road pedestrian automatic identification and positioning method based on deep learning and U-V parallax algorithm |
| CN112734260A (en) * | 2021-01-18 | 2021-04-30 | 中国民航信息网络股份有限公司 | Counter management method and device, electronic equipment and computer storage medium |
| CN112767442A (en) * | 2021-01-18 | 2021-05-07 | 中山大学 | Pedestrian three-dimensional detection tracking method and system based on top view angle |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408746A (en) * | 2014-11-21 | 2015-03-11 | 安徽清新互联信息科技有限公司 | Passenger flow statistical system based on depth information |
| CN104504688A (en) * | 2014-12-10 | 2015-04-08 | 上海大学 | Method and system based on binocular stereoscopic vision for passenger flow density estimation |
| CN104902258A (en) * | 2015-06-09 | 2015-09-09 | 公安部第三研究所 | Multi-scene pedestrian volume counting method and system based on stereoscopic vision and binocular camera |
| CN105225482A (en) * | 2015-09-02 | 2016-01-06 | 上海大学 | Based on vehicle detecting system and the method for binocular stereo vision |
-
2016
- 2016-12-15 CN CN201611157648.5A patent/CN108230351A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104408746A (en) * | 2014-11-21 | 2015-03-11 | 安徽清新互联信息科技有限公司 | Passenger flow statistical system based on depth information |
| CN104504688A (en) * | 2014-12-10 | 2015-04-08 | 上海大学 | Method and system based on binocular stereoscopic vision for passenger flow density estimation |
| CN104902258A (en) * | 2015-06-09 | 2015-09-09 | 公安部第三研究所 | Multi-scene pedestrian volume counting method and system based on stereoscopic vision and binocular camera |
| CN105225482A (en) * | 2015-09-02 | 2016-01-06 | 上海大学 | Based on vehicle detecting system and the method for binocular stereo vision |
Non-Patent Citations (2)
| Title |
|---|
| 王文娟: "基于双目立体视觉的客流统计算法研究", 《中国优秀硕士学位论文全文数据库（电子期刊）信息科技辑》 * |
| 郭秀杰: "基于Kinect的人流量统计系统研究", 《中国优秀硕士学位论文全文数据库（电子期刊）信息科技辑》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109740537A (en) * | 2019-01-03 | 2019-05-10 | 广州广电银通金融电子科技有限公司 | The accurate mask method and system of pedestrian image attribute in crowd's video image |
| CN109740537B (en) * | 2019-01-03 | 2020-09-15 | 广州广电银通金融电子科技有限公司 | Method and system for accurately marking attributes of pedestrian images in crowd video images |
| CN110766740A (en) * | 2019-10-31 | 2020-02-07 | 西安咏圣达电子科技有限公司 | Real-time high-precision binocular ranging system and method based on pedestrian tracking |
| CN110766740B (en) * | 2019-10-31 | 2023-09-01 | 西安咏圣达电子科技有限公司 | Real-time high-precision binocular range finding system and method based on pedestrian tracking |
| CN111553252A (en) * | 2020-04-24 | 2020-08-18 | 福建农林大学 | Road pedestrian automatic identification and positioning method based on deep learning and U-V parallax algorithm |
| CN111553252B (en) * | 2020-04-24 | 2022-06-07 | 福建农林大学 | Road pedestrian automatic identification and positioning method based on deep learning and U-V parallax algorithm |
| CN112734260A (en) * | 2021-01-18 | 2021-04-30 | 中国民航信息网络股份有限公司 | Counter management method and device, electronic equipment and computer storage medium |
| CN112767442A (en) * | 2021-01-18 | 2021-05-07 | 中山大学 | Pedestrian three-dimensional detection tracking method and system based on top view angle |
| CN112767442B (en) * | 2021-01-18 | 2023-07-21 | 中山大学 | Pedestrian three-dimensional detection tracking method and system based on top view angle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110285793B (en) | Intelligent vehicle track measuring method based on binocular stereo vision system | |
| JP6549797B2 (en) | Method and system for identifying head of passerby | |
| CN108230351A (en) | Sales counter evaluation method and system based on binocular stereo vision pedestrian detection | |
| CN102697508B (en) | Method for performing gait recognition by adopting three-dimensional reconstruction of monocular vision | |
| CN103971386B (en) | A kind of foreground detection method under dynamic background scene | |
| CN108230392A (en) | A kind of dysopia analyte detection false-alarm elimination method based on IMU | |
| CN106485735A (en) | Human body target recognition and tracking method based on stereovision technique | |
| CN104517095B (en) | A kind of number of people dividing method based on depth image | |
| CN105225482A (en) | Based on vehicle detecting system and the method for binocular stereo vision | |
| CN103530599A (en) | Method and system for distinguishing real face and picture face | |
| CN103735269B (en) | A kind of height measurement method followed the tracks of based on video multi-target | |
| CN102243765A (en) | Multi-camera-based multi-objective positioning tracking method and system | |
| CN106503605A (en) | Human body target recognition methods based on stereovision technique | |
| CN106033614B (en) | A kind of mobile camera motion object detection method under strong parallax | |
| CN104331901A (en) | TLD-based multi-view target tracking device and method | |
| CN105160649A (en) | Multi-target tracking method and system based on kernel function unsupervised clustering | |
| WO2022127181A1 (en) | Passenger flow monitoring method and apparatus, and electronic device and storage medium | |
| CN109035307B (en) | Set area target tracking method and system based on natural light binocular vision | |
| CN105096292A (en) | Object quantity estimation method and device | |
| CN106295657A (en) | A kind of method extracting human height's feature during video data structure | |
| CN111652900A (en) | Scene flow-based passenger flow counting method, system, equipment and storage device | |
| CN106709432B (en) | Human head detection counting method based on binocular stereo vision | |
| CN105957060B (en) | A kind of TVS event cluster-dividing method based on optical flow analysis | |
| Wu et al. | Geometry-aware instance segmentation with disparity maps | |
| CN111862511B (en) | Target intrusion detection device and method based on binocular stereo vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180629 |
|
| RJ01 | Rejection of invention patent application after publication |