CN109167956A - The full-bridge face traveling load spatial distribution merged based on dynamic weighing and more video informations monitors system - Google Patents
The full-bridge face traveling load spatial distribution merged based on dynamic weighing and more video informations monitors system Download PDFInfo
- Publication number
- CN109167956A CN109167956A CN201810487110.3A CN201810487110A CN109167956A CN 109167956 A CN109167956 A CN 109167956A CN 201810487110 A CN201810487110 A CN 201810487110A CN 109167956 A CN109167956 A CN 109167956A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- dynamic weighing
- visual field
- bridge
- full
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000005303 weighing Methods 0.000 title claims abstract description 53
- 238000009826 distribution Methods 0.000 title claims abstract description 29
- 230000000007 visual effect Effects 0.000 claims abstract description 74
- 238000012544 monitoring process Methods 0.000 claims abstract description 49
- 238000000034 method Methods 0.000 claims abstract description 38
- 238000005516 engineering process Methods 0.000 claims abstract description 7
- 230000010365 information processing Effects 0.000 claims abstract description 6
- 238000004891 communication Methods 0.000 claims abstract description 4
- 230000008569 process Effects 0.000 claims description 12
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 7
- 238000012546 transfer Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 230000005484 gravity Effects 0.000 claims description 4
- 238000005457 optimization Methods 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 description 21
- 230000008859 change Effects 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 230000009466 transformation Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000004927 fusion Effects 0.000 description 4
- 238000005259 measurement Methods 0.000 description 4
- 238000012805 post-processing Methods 0.000 description 4
- 230000007704 transition Effects 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000005304 joining Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 241000135164 Timea Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005315 distribution function Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/144—Movement detection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/14—Picture signal circuitry for video frequency region
- H04N5/144—Movement detection
- H04N5/145—Movement estimation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The present invention discloses a kind of full-bridge face traveling load spatial distribution monitoring system merged based on dynamic weighing and more video informations, the car weight information that dynamic weighing system obtains traveling load is laid in bridge floor initial position, arrange that multiple cameras obtain the traffic flow video information in covering full-bridge face along bridge, there are overlapping regions in the visual field of two neighboring camera, site layout project information collection and network communication system, and turn-key system is analyzed in deployment information processing beyond the clouds, track of vehicle in visual field and position are tracked in conjunction with Kalman Filter Technology and visual field boundary collimation method, and dynamic weighing is merged with more video informations are monitored according to the time of vehicle across wire casing where dynamic weighing system piezoelectric transducer, with realizing the size and location to the traveling load within the scope of full-bridge face accurate identification in real time.It is an advantage of the invention that calculating speed is fast, accuracy of identification is high, and all standing suitable for all kinds of bridge floors or road surface move vehicle load in specified region monitors.
Description
Technical field
It is the present invention relates to bridge health monitoring field, in particular to a kind of to be merged based on dynamic weighing and more video informations
Full-bridge face traveling load spatial distribution monitors system and its monitoring method.
Background technique
Vehicular load is the main variable load that highway bridge is born.With the fast development of Traffic conflicts, in traffic
Overloaded vehicle quantity also obviously increases while flow rapid growth.The repeated action of overload of vehicle and traffic loading has become shadow
The main reason for ringing bridge operational safety and shortening bridge service life.Therefore, accurately identify bridge floor traveling load size and
Position becomes extremely important.
Identification for move vehicle load, at present there are two types of main means.The first is obtained using dynamic weighing system
The data such as car weight, axis weight and the speed of pick-up carry out Direct Recognition, and this method is merely capable of obtaining the single section of bridge
Load assignment；Another kind passes through the first analytic method (IMI), the second analytic method (IMII), time domain method using beam as basic mechanical model
(TDM) and the indirect means identification such as frequency time domain method (FTDM), these methods are in the ideal case, simple enough in structure in other words
In the case of can effectively identify the size of load.However, these two kinds of methods cannot achieve load in the knowledge of bridge floor spatial distribution
Not.
Currently, dynamic weighing system and video monitoring system are widely used in the traffic administration and security monitoring of bridge,
Especially for Longspan Bridge, along bridge to usually arranging that multiple video cameras are monitored.Due to the monitoring model of single camera
It is with limit, so must be provided with multiple video cameras to complete the continuous monitoring in full-bridge face.However, current multiple-camera monitors hand
Section still falls within the combination of multiple single camera monitoring, does not establish effective connection between video camera.
Currently, there has been no a kind of methods for merging dynamic weighing system and multiple-camera monitoring information, based on the above
Background needs to invent a kind of monitoring system for merging dynamic weighing system and multi-cam monitoring information, realizes full-bridge face vehicle
The effective monitoring of load real-time distribution.
Summary of the invention
It is high based on dynamic weighing and more videos the technical problem to be solved by the present invention is to provide a kind of accuracy of identification
The full-bridge face traveling load spatial distribution of information fusion monitors system.
For the technical problem more than solving, merged the present invention provides a kind of based on dynamic weighing and more video informations
Full-bridge face traveling load spatial distribution monitors system, lays the vehicle that dynamic weighing system obtains traveling load in bridge floor initial position
Weight information arranges that multiple cameras obtain the traffic flow video information in covering full-bridge face, the visual field of two neighboring camera along bridge
There are overlapping region, site layout project information collection and network communication system, and deployment information processing analysis turn-key system beyond the clouds,
Track of vehicle in visual field and position are tracked in conjunction with Kalman Filter Technology and visual field boundary collimation method, and crossed over according to vehicle
The time of wire casing where dynamic weighing system piezoelectric transducer merges dynamic weighing with more video informations are monitored, realization pair
Accurately identify in real time to the size and location of traveling load within the scope of full-bridge face.
The present invention also provides a kind of full-bridge face traveling load spaces point merged based on dynamic weighing and more video informations
Cloth monitors the monitoring method of system, comprises the following steps that
Step 1: according to vehicle by time of dynamic weighing system by first camera monitor video and dynamic weighing system
Metrical information is merged；
Step 2: establishing image coordinate and bridge floor coordinate is converted；
Step 3: the detection of the mobile target of single camera；
Step 4: the tracking of single camera target is realized according to Kalman Filter Technology；
Step 5: the matching of same target in the different visuals field is realized according to visual field line of demarcation；
Step 6: calculating visual field line of demarcation.
The information process analysis turn-key system includes initialization module, single camera position recognition and tracking module, more
Camera information is merged to connection module and post-processing module.
In the initialization module, the car weight information that dynamic weighing system WIM is obtained is used for the vehicle tracking of full-bridge, each
Car weight information is forced to depend on the mobile target trajectory detected in first camera monitoring video, and this relations of dependence are bases
It is established in the time of vehicle across the specified horizontal line of bridge floor, it is ensured that the correctness of matching relationship, steps are as follows:
Step 1: for each vehicle passed through, car weight information is obtained using dynamic weighing system WIM, such as net weight, axis
Weight and passes through the time at speed；
Step 2: in synchronization, whether pass through bridge floor dynamic weighing system WIM with the same vehicle of first camera detection
The inbuilt trench line of piezoelectric transducer；If so, first camera head monitor video is obtained with dynamic weighing system using time cue
The car weight information association arrived.
In the single camera position recognition and tracking module, movement that subsidiary car weight information and former frame identify
Target position information is passed in the identification of current image frame, by Kalman filter, obtains the movement of current image frame
Vehicle location, and according to image object matching relationship, each target location coordinate is correctly matched, concatenation forms the movement rail of vehicle
Mark.This transmitting movement not only carries out in first camera, but also can successively be passed to the identification of remaining camera
In obtained track, until vehicle removes bridge floor monitoring range, to form moving load identification and the tracking of full-bridge.Step is such as
Under:
Step 1: extract video camera obtain current frame image, by be no less than four serial reference point image coordinate and
Bridge floor coordinate estimated projection transformation matrix；
Step 2: the vehicle location in current image frame is identified by Kalman filter；
Step 3: resulting vehicle location will be tracked using projective transformation matrix and is converted into bridge floor coordinate.
In the step, the current frame image that video camera obtains is extracted, chooses and is no less than four image coordinates, utilize pin hole
Model foundation coordinate transfer equation, establishes coefficient Z in transfer equationciFunction expression and by respectively joining in optimization algorithm solution formula
Number.
The multi-cam information fusion is to connection module, other than carrying out each frame target position identification and being formed with track, also
Need the synchronous target handoff work for carrying out the picture frame that adjacent camera obtains.The transition of moving target is limited at finger
In fixed range --- the coincidence cross-connecting area of adjacent camera visual field.Steps are as follows:
Step 1: the image coordinate according to vehicle in video and its visibility processing visual field line of demarcation in each visual field；
Step 2: target corresponding relationship is determined by visual field boundary collimation method；
Step 3: using newly into vehicle image coordinate and visibility more New view line of demarcation.
In above-mentioned steps, each pixel in video, variation of the value in image sequence, which is regarded as, constantly generates picture
The random process of element value, for each newly generated pixel, by its pixel value compared with having multiple Gauss models, judgement should
Whether pixel belongs to prospect.
In above-mentioned steps, with state vector xkMotor behavior of the vehicle at the k moment is described, with the center of gravity (u of move vehiclek,
vk) instead of the position of load, utilize the vehicle location of Kalman filter estimation next frame picture；With i-th in kth frame image
Vehicle measures matching degree at a distance from the jth vehicle mass center in+1 frame image of kth, the smaller then consecutive frame two cars of distance value
Matching degree is higher；When finding the minimum value of distance function, Kalman state mould is updated with the vehicle movement feature of+1 frame of kth
Type, and next frame is used for as input；By above step more new model, until vehicle disappears.
In above-mentioned steps, when vehicle pass through the visual field line of demarcation when, or value change, at this time calculate adjacent fields in
All vehicles are ranked up all distance values to the distance in this visual field line of demarcation, apart from the smallest as corresponding vehicle；Limit
The handover that sets the goal only is carried out in visual field overlapping region.
When thering are more vehicles to cross field of view lines in above-mentioned steps, in the visual field, if vehicleIn l frame
Ci+1In it is invisible,Enter the visual field in C in l+1 framei+1As it can be seen thatThink vehicle
It is located at visual field line of demarcation at the midpoint of l frame and the mass center line of l+1 frameOn, visual field cut-off rule is by multiple such points
Fitting obtains.
The post-processing module is to differentiate whether monitoring objective removes bridge floor monitoring boundary.If out-of-bounds are deleted in monitoring
Mobile target, it is ensured that memory is not spilt over, and keeps the continuity of operation.
Superior effect of the invention is:
1) present invention solves the problems, such as dynamic weighing system and the fusion of single video monitoring system information；Solve multiple-camera monitoring
The target handoff problem of same vehicle in video；Solve the problems, such as that full-bridge face traveling load is distributed real time monitoring；
2) calculating speed of the present invention is fast, and accuracy of identification is high, is suitable for all kinds of bridge floors or road surface move vehicle load exists
The all standing monitoring in specified region.
Detailed description of the invention
The accompanying drawings constituting a part of this application is used to provide further understanding of the present invention, and of the invention shows
Examples and descriptions thereof are used to explain the present invention for meaning property, does not constitute improper limitations of the present invention.In the accompanying drawings:
Fig. 1 is the structural schematic diagram that full-bridge face traveling load spatial distribution of the present invention monitors system；
Fig. 2 is that full-bridge face traveling load spatial distribution of the present invention monitors system information treatment process schematic diagram；
Fig. 3 is that full-bridge face traveling load spatial distribution of the present invention monitors system information Processing Algorithm flow chart；
Fig. 4 is that the present invention is applied to Shanghai City Tongji University road and bridge measured data calculated result；
Fig. 5 is animation simulation lateral coordinates identification error of the present invention (by taking four vehicles as an example)；
Fig. 6 is animation simulation longitudinal coordinate identification error of the present invention (by taking four vehicles as an example).
Specific embodiment
The embodiment of the present invention is described in detail below in conjunction with attached drawing.
Fig. 1 shows the structural schematic diagram of the full-bridge face traveling load spatial distribution monitoring system of the embodiment of the present invention；Fig. 2
Show full-bridge face traveling load spatial distribution monitoring system information treatment process schematic diagram of the present invention；Fig. 3 shows the present invention
Full-bridge face traveling load spatial distribution monitors system information Processing Algorithm flow chart.As shown in Figure 1, the present invention provides a kind of bases
System is monitored in the full-bridge face traveling load spatial distribution that dynamic weighing and more video informations merge, is laid in bridge floor initial position
Dynamic weighing system obtains the car weight information of traveling load, arranges that multiple cameras obtain the traffic flow view in covering full-bridge face along bridge
Frequency information, there are overlapping region, site layout project information collection and 4G network communication apparatus in the visual field of two neighboring camera, and
Cloud deployment information processing analysis turn-key system, in conjunction with Kalman Filter Technology and visual field boundary collimation method to track of vehicle in visual field
Tracked with position, and according to vehicle across wire casing where dynamic weighing system piezoelectric transducer time by dynamic weighing with
More video informations are monitored to be merged, it is with realizing the size and location to the traveling load within the scope of full-bridge face accurate to know in real time
Not.
As shown in Figures 2 and 3, the present invention also provides a kind of full-bridge faces merged based on dynamic weighing and more video informations
Traveling load spatial distribution monitors the monitoring method of system, comprises the following steps that
Step 1: according to vehicle by time of dynamic weighing system by first camera monitor video and dynamic weighing system
Metrical information is merged；
Step 2: establishing image coordinate and bridge floor coordinate is converted；
Wherein, coordinate transfer equation is established:
It is image coordinate by the vehicle coordinate that video flowing is calculated, determines vehicle in bridge floor according to image coordinate
Position, it is necessary to establish the projective transformation matrix between image coordinate system and bridge floor coordinate system.Enabling vehicle target, coordinate is in the picture
(u, v) is (X in the coordinate of bridge floorw,Yw).Then according to the basic principle of photogrammetry, there is following relationship between the two:
Wherein M is 3 × 4 matrixes, and f is camera focus, and dx, dy are that single pixel is wide and high (mm), (u0,v0) it is camera light
The image coordinate of axis and plane of delineation intersection point, (Xc,Yc,Zc) it is coordinate of the target in camera coordinate system.When the parameter of camera
It, need to be according to the image coordinate and world coordinates calculating matrix M of reference point when unknown.(1) formula is rewritten are as follows:
Wherein, (Xwi,Ywi,Zwi) be i-th point of space coordinate, middle finger bridge floor coordinate of the present invention；(ui,vi) it is i-th
Point image coordinate；mijFor matrix M the i-th row jth column element.By the expansion of (2) formula and abbreviation can obtain
For n reference point, 2n linear equation can be obtained by (3) formula, independent known variables number is 11 in equation
It is a, therefore at least 6 pairs of reference point coordinates can determine transition matrix M.For bridge floor, the height of bridge floor within the scope of camera coverage
Significant change does not occur for journey, therefore might as well consider ZwiFor constant, and Zwi=0.At least 4 pairs of reference point coordinates are needed at this time.
However, the conversion of image coordinate to bridge floor coordinate cannot be directly realized by after acquiring Metzler matrix, coefficient Z when coordinate transformationc
It can not ignore.The present invention is that known image coordinate is gone to bridge floor coordinate, and wherein plane where bridge floor is uniquely true by reference point
It is fixed.Therefore, the point on image and vehicle position correspond, Ji Keling
According to the matrix M and reference point coordinate acquired, it is not difficult to determine the corresponding Z of each reference pointci.Then, with minimum
Each parameter in square law Optimization Solution formula (4).It is not most accurate to optimize calculated result, if increasing the number of reference point, can obtain
To more accurately as a result, reference point number is chosen according to the actual situation.
Step 3: the detection of the mobile target of single camera；
Vehicle detection is the basis of vehicle tracking.For each pixel in video, change of the value in image sequence
Change the random process regarded as and constantly generate pixel value.T at any time, for specific pixel (x0,y0), its observation
Data set is denoted as:
{G1,G2,...,Gt}={ I (x0,y0,i)|1≤i≤t} (5)
Wherein, GtPixel is indicated in the pixel value in R, G, B channels, I refers to image sequence；
For each stochastic variable G in observation data settObey Gaussian mixtures density function:
Wherein J is distribution pattern quantity (generally taking 3 to 5), ωi,tFor the weight of i-th of Gaussian Profile of t moment, μi,tFor
Its mean vector, τi,tFor its covariance matrix, η is gaussian probability distribution function:
In addition, in order to reduce matrix inversion operation, it is assumed that pixel is mutually indepedent in tri- chrominance channel R, G, B and has identical
Variance, covariance matrix expression are as follows:
Wherein, σiFor the standard difference vector of i-th of Gaussian Profile, E is three-dimensional unit matrix；
For each new pixel value Gt, it is verified as the following formula in existing J distribution, until finding matching
Distribution.If verifying, which matches the pixel, belongs to background, otherwise belong to prospect.
|Gt-μi,t-1|≤2.5σi,t-1 (9)
Step 4: the tracking of single camera target is realized according to Kalman Filter Technology；
In Kalman filtering, the present invention considers the tracking of vehicle, that is, uses state vector xkVehicle is described at the k moment
Motor behavior, point or less the description of three steps realize vehicle tracking using Kalman kalman filtering:
1) state estimation model
Assuming that the process model and discrete model of linear discrete system are as follows:
Wherein A is process matrix (transition matrix), and H is calculation matrix (measurement matrix)；
gk-1For process noise, wkTo measure noise, they all meet Gaussian Profile, and probability density is denoted as:
Wherein, Qk,RkFor covariance matrix；
In (10) formula, due to ckIt cannot directly measure, it is therefore necessary to according to the resulting z of measurementkInformation determines.Priori
State estimationAnd error of covarianceIt can be determined by following formula:
Posteriority state estimationIt can be estimated by prior stateWith new measuring state zkLinear combination obtains.Such as following table
Up to shown in formula:
The K calculated by above formulakFor kalman gain；
Emphasis of the invention is the distribution for obtaining traveling load, and the real time position of vehicle need to be only obtained by vehicle tracking.
Consider center of gravity (such as trace image coordinate u with move vehiclekAnd vk) instead of the position of load, Kalman's tracker is at the k moment
State vector and measurement vector can be expressed as:
In the present invention, process matrix and calculation matrix are as follows:
After the state equation and measurement equation of vehicle movement determine, Kalman filter can be used to estimate that next frame is drawn
The position of face vehicle, and can further obtain the track of vehicle movement.
2) characteristic matching
In video streaming, the movement of each vehicle is all described by the image coordinate of its mass center, remembers i-th vehicle in kth frame figure
As in center-of-mass coordinate beI-th vehicle in kth frame image and the jth vehicle mass center in+1 frame image of kth away from
From function is defined as:
The value of distance function is smaller, illustrates that the matching degree of consecutive frame two cars is higher.
3) more new model
When finding the minimum value of distance function, Kalman state model is updated with the vehicle movement feature of+1 frame of kth, and
Next frame is used for as input.Above step more new model is repeated, until vehicle disappears in the visual field.
Step 5: the matching of same target in the different visuals field is realized according to visual field line of demarcation；
Step 6: calculating visual field line of demarcation.
The field range of single camera monitoring is limited, is unable to complete all standing monitoring of bridge floor, multiple cameras are arranged
This deficiency can effectively be overcome.The present invention divides collimation method by the visual field and realizes target handoff, it is assumed that all taking the photograph of bridge floor setting
As the timestamp of head is by calibration, and there is overlapping in the monitoring visual field region of adjacent camera.
The image (visual field) for remembering the shooting of i-th of camera is Ci, bridge floor is elongate strips, remembers i-th of camera along bridge side
To two horizontal boundaries beWithBecause there are overlapping, the visual field line of demarcation of i-th of camera in the visual fieldIt is taken the photograph in i+1
As head the visual field in as it can be seen that and the visual field line of demarcation of i+1 cameraIt is visible in the visual field of i-th of camera.This
Outside, remember visual field CiIn visible the m vehicle beRemember visual field Ci+1In visible n-th vehicle beWith the center-of-mass coordinate of vehicleWithIndicate vehicleWithThe coordinate in the visual field.Key be exactly determine shaped likeEquivalence
Relationship.If visual field line of demarcation is it is known that provide following regulation: as visual field CiMiddle vehicle does not pass through visual field line of demarcationWhen, vehicle
In visual field Ci+1In it is also invisible, provide at this timeAs visual field CiMiddle vehicle passes through visual field line of demarcationWhen, vehicle
In visual field Ci+1In as it can be seen that providing at this time
When vehicle passes through a visual field line of demarcation,OrValue can change, count at this time
It calculates all vehicles in adjacent fields to be ranked up all distance values to the distance in this visual field line of demarcation, apart from the smallest i.e.
For corresponding vehicle.The method expresses mathematical formulae are as follows:
Wherein, l is the marking of cars in the visual field, D (L, O) return vehicle centroid O to visual field line of demarcation L in the specified visual field away from
From.To avoid unnecessary calculating, the present invention only carries out target handoff in overlapped view region, therefore has been subject to about in formula to i
Beam.
P is visual field CjThe middle marking of cars, D (L, O) return to the distance of vehicle centroid O to straight line L.
The method can effectively solve the matching problem of the marking of cars between two cameras of arbitrary neighborhood, avoid when camera shooting
Machine is apart from each other to be difficult to the problem of completing characteristic matching.Particularly, work as i=0, when j=1, acquisition is exactly dynamic weighing system
Locate the corresponding relationship of camera and next camera vehicle.
When thering are more vehicles to cross field of view lines in the visual field, if vehicleIn l frame in Ci+1In can not
See,Enter the visual field in C in l+1 framei+1As it can be seen thatBecause of usual video (25FPS with
On) a frame time in vehicle location change it is very small, it is believed that vehicleIn the mass center line of l frame and l+1 frame
Midpoint is located at visual field line of demarcationOn, visual field cut-off rule is obtained by multiple such point fittings.
The information process analysis turn-key system includes initialization module, single camera position recognition and tracking module, more
Camera information is merged to connection module and post-processing module.
In the initialization module, the car weight information that dynamic weighing system WIM is obtained is used for the vehicle tracking of full-bridge, each
Car weight information is forced to depend on the mobile target trajectory detected in first camera monitoring video, and this relations of dependence are bases
It is established in the time of vehicle across the specified horizontal line of bridge floor, it is ensured that the correctness of matching relationship, steps are as follows:
Step 1: for each vehicle passed through, car weight information is obtained using dynamic weighing system WIM, such as net weight, axis
Weight and passes through the time at speed；
Step 2: in synchronization, whether pass through bridge floor dynamic weighing system WIM with the same vehicle of first camera detection
The inbuilt trench line of piezoelectric transducer；If so, first camera head monitor video is obtained with dynamic weighing system using time cue
The car weight information association arrived.
In the single camera position recognition and tracking module, movement that subsidiary car weight information and former frame identify
Target position information is passed in the identification of current image frame, by Kalman filter, obtains the movement of current image frame
Vehicle location, and according to image object matching relationship, each target location coordinate is correctly matched, concatenation forms the movement rail of vehicle
Mark.This transmitting movement not only carries out in first camera, but also can successively be passed to the identification of remaining camera
In obtained track, until vehicle removes bridge floor monitoring range, to form moving load identification and the tracking of full-bridge.Step is such as
Under:
Step 1: extract video camera obtain current frame image, by be no less than four serial reference point image coordinate and
Bridge floor coordinate estimated projection transformation matrix；
Step 2: the vehicle location in current image frame is identified by Kalman filter；
Step 3: resulting vehicle location will be tracked using projective transformation matrix and is converted into bridge floor coordinate.
In the step, the current frame image that video camera obtains is extracted, chooses and is no less than four image coordinates, utilize pin hole
Model foundation coordinate transfer equation, establishes coefficient Z in transfer equationciFunction expression and by respectively joining in optimization algorithm solution formula
Number.
The multi-cam information fusion is to connection module, other than carrying out each frame target position identification and being formed with track, also
Need the synchronous target handoff work for carrying out the picture frame that adjacent camera obtains.The transition of moving target is limited at finger
In fixed range --- the coincidence cross-connecting area of adjacent camera visual field.Steps are as follows:
Step 1: the image coordinate according to vehicle in video and its visibility processing visual field line of demarcation in each visual field；
Step 2: target corresponding relationship is determined by visual field boundary collimation method；
Step 3: using newly into vehicle image coordinate and visibility more New view line of demarcation.
In above-mentioned steps, each pixel in video, variation of the value in image sequence, which is regarded as, constantly generates picture
The random process of element value, for each newly generated pixel, by its pixel value compared with having multiple Gauss models, judgement should
Whether pixel belongs to prospect.
In above-mentioned steps, with state vector xkMotor behavior of the vehicle at the k moment is described, with the center of gravity (u of move vehiclek,
vk) instead of the position of load, utilize the vehicle location of Kalman filter estimation next frame picture；With i-th in kth frame image
Vehicle measures matching degree at a distance from the jth vehicle mass center in+1 frame image of kth, the smaller then consecutive frame two cars of distance value
Matching degree is higher；When finding the minimum value of distance function, Kalman state mould is updated with the vehicle movement feature of+1 frame of kth
Type, and next frame is used for as input；By above step more new model, until vehicle disappears.
In above-mentioned steps, when vehicle pass through the visual field line of demarcation when, or value change, at this time calculate adjacent fields in
All vehicles are ranked up all distance values to the distance in this visual field line of demarcation, apart from the smallest as corresponding vehicle；Limit
The handover that sets the goal only is carried out in visual field overlapping region.
The post-processing module is to differentiate whether monitoring objective removes bridge floor monitoring boundary.If out-of-bounds are deleted in monitoring
Mobile target, it is ensured that memory is not spilt over, and keeps the continuity of operation.
The present invention is applied to Shanghai City Tongji University road ramp bridge scene, calculates measured data, the present invention is to shifting
The identification of dynamic load position and size and tracking effect meet engine request, if Fig. 4 is 3 points to 4 points of afternoon on January 14th, 2018
Totally 3 moment bridge floor vehicular loads distribution.
According to multi-view angle three-dimensional simulation animation data, the analysis accuracy of the invention calculated, the results showed that, identify vehicle position
The laterally maximum mean error set is 0.212 meter (7.8 meters wide), and maximum longitudinal direction mean error is 3.580 meters (100 meters long), full
Sufficient engineer application (such as Fig. 5 and Fig. 6).
The foregoing is merely preferred embodiments of the invention, are not intended to restrict the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.All within the spirits and principles of the present invention, made any to repair
Change, equivalent replacement, improvement etc., should all be included in the protection scope of the present invention.
Claims (8)
1. a kind of full-bridge face traveling load spatial distribution merged based on dynamic weighing and more video informations monitors system, feature
It is: lays the car weight information that dynamic weighing system obtains traveling load in bridge floor initial position, arrange multiple cameras along bridge
The traffic flow video information in covering full-bridge face is obtained, there are overlapping region, site layout project information in the visual field of two neighboring camera
Acquisition and network communication system, and deployment information processing analysis turn-key system beyond the clouds, in conjunction with Kalman Filter Technology and the visual field
Boundary collimation method tracks track of vehicle in visual field and position, and crosses over dynamic weighing system piezoelectric transducer institute according to vehicle
Dynamic weighing is merged with more video informations are monitored in the time of wire casing, is realized to the traveling load within the scope of full-bridge face
Accurately identify in real time to size and location.
2. the full-bridge face traveling load spatial distribution according to claim 1 merged based on dynamic weighing and more video informations
The monitoring method of monitoring system, comprises the following steps that
Step 1: being measured first camera monitor video and dynamic weighing system by the time of dynamic weighing system according to vehicle
Information is merged；
Step 2: establishing image coordinate and bridge floor coordinate is converted；
Step 3: the detection of the mobile target of single camera；
Step 4: the tracking of single camera target is realized according to Kalman Filter Technology；
Step 5: the matching of same target in the different visuals field is realized according to visual field line of demarcation；
Step 6: calculating visual field line of demarcation.
3. the full-bridge face traveling load spatial distribution according to claim 2 merged based on dynamic weighing and more video informations
The monitoring method of monitoring system, it is characterised in that: in the step 1, the car weight information that dynamic weighing system obtains is used for full-bridge
Vehicle tracking, each car weight information is forced to depend on the mobile target trajectory detected in first camera monitoring video,
This relations of dependence are to specify the time of horizontal line to establish across bridge floor based on vehicle, and steps are as follows:
Step1: for each vehicle passed through, car weight information is obtained using dynamic weighing system, car weight information includes vehicle
Net weight, axis weight, speed and pass through the time；
Step2: in synchronization, whether pass through bridge floor dynamic weighing system piezoelectric sensing with the same vehicle of first camera detection
The inbuilt trench line of device；If so, the car weight for being obtained first camera head monitor video and dynamic weighing system using time cue
Information association.
4. the full-bridge face traveling load spatial distribution according to claim 2 merged based on dynamic weighing and more video informations
The monitoring method of monitoring system, it is characterised in that: in the step 2, extract the current frame image that video camera obtains, choose many
In four image coordinates, coordinate transfer equation is established using pin-hole model, establishes coefficient Z in transfer equationciFunction expression is simultaneously
Pass through each parameter in optimization algorithm solution formula.
5. the full-bridge face traveling load spatial distribution according to claim 2 merged based on dynamic weighing and more video informations
The monitoring method of monitoring system, it is characterised in that: in the step 3, each pixel in video, value is in image sequence
Variation regard the random process for constantly generating pixel value as, for each newly generated pixel, by its pixel value and existing
Multiple Gauss models compare, and judge whether the pixel belongs to prospect.
6. the full-bridge face traveling load spatial distribution according to claim 2 merged based on dynamic weighing and more video informations
The monitoring method of monitoring system, it is characterised in that: in the step 4, with state vector xkVehicle is described in the movement row at k moment
For with the center of gravity (u of move vehiclek,vk) instead of the position of load, utilize the vehicle of Kalman filter estimation next frame picture
Position；Matching degree is measured at a distance from the jth vehicle mass center in+1 frame image of kth with i-th vehicle in kth frame image, away from
It is higher from the matching degree for being worth smaller then consecutive frame two cars；When finding the minimum value of distance function, with the vehicle of+1 frame of kth
Motion feature updates Kalman state model, and is used for next frame as input；By above step more new model, until vehicle disappears
It loses.
7. the full-bridge face traveling load spatial distribution according to claim 2 merged based on dynamic weighing and more video informations
The monitoring method of monitoring system, it is characterised in that: in the step 5, when vehicle pass through the visual field line of demarcation when, or value occur
Variation calculates all vehicles in adjacent fields at this time and is ranked up to the distance in this visual field line of demarcation to all distance values, away from
From the smallest as corresponding vehicle；Target handoff is limited only to carry out in visual field overlapping region.
8. the full-bridge face traveling load spatial distribution according to claim 2 merged based on dynamic weighing and more video informations
The monitoring method of monitoring system, it is characterised in that: when thering are more vehicles to cross field of view lines in the step 6, in the visual field, if
VehicleIn l frame in Ci+1In it is invisible,Enter the visual field in C in l+1 framei+1As it can be seen thatThink vehicleIt is located at visual field line of demarcation at the midpoint of l frame and the mass center line of l+1 frameOn, depending on
Wild cut-off rule is obtained by multiple such point fittings.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810487110.3A CN109167956B (en) | 2018-05-21 | 2018-05-21 | Full-bridge surface moving load spatial distribution monitoring system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810487110.3A CN109167956B (en) | 2018-05-21 | 2018-05-21 | Full-bridge surface moving load spatial distribution monitoring system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109167956A true CN109167956A (en) | 2019-01-08 |
| CN109167956B CN109167956B (en) | 2021-02-02 |
Family
ID=64897144
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810487110.3A Active CN109167956B (en) | 2018-05-21 | 2018-05-21 | Full-bridge surface moving load spatial distribution monitoring system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109167956B (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109816006A (en) * | 2019-01-18 | 2019-05-28 | 深圳大学 | A kind of sea horizon detection method, device and computer readable storage medium |
| CN111354035A (en) * | 2020-03-17 | 2020-06-30 | 陕西高速机械化工程有限公司 | Automatic bridge dynamic load detection and inversion system and method |
| CN112307888A (en) * | 2020-09-21 | 2021-02-02 | 中铁第四勘察设计院集团有限公司 | Method and system for identifying dynamic load of bridge based on machine vision positioning |
| CN112444311A (en) * | 2020-11-22 | 2021-03-05 | 同济大学 | Method for monitoring space-time load of bridge vehicle |
| CN112504414A (en) * | 2020-11-27 | 2021-03-16 | 湖南大学 | Vehicle dynamic weighing method and system based on non-contact measurement of dynamic deflection of bridge |
| CN112820112A (en) * | 2021-02-05 | 2021-05-18 | 同济大学 | Bridge floor traffic flow full-view-field sensing system and method depending on bridge tower column |
| CN112885110A (en) * | 2021-02-05 | 2021-06-01 | 同济大学 | Bridge floor traffic flow full-view-field sensing system and method depending on adjacent high-rise structure |
| CN112885096A (en) * | 2021-02-05 | 2021-06-01 | 同济大学 | Bridge floor traffic flow full-view-field sensing system and method depending on bridge arch ribs |
| CN112885097A (en) * | 2021-02-07 | 2021-06-01 | 启迪云控(上海)汽车科技有限公司 | Road side fusion management method and system based on cross-point location |
| CN113022540A (en) * | 2020-04-17 | 2021-06-25 | 青岛慧拓智能机器有限公司 | Real-time remote driving system and method for monitoring multiple vehicle states |
| CN113112798A (en) * | 2021-04-09 | 2021-07-13 | 苏庆裕 | Vehicle overload detection method, system and storage medium |
| CN113114981A (en) * | 2021-03-11 | 2021-07-13 | 联想(北京)有限公司 | Region determination method, electronic device and system |
| CN113114938A (en) * | 2021-04-12 | 2021-07-13 | 滁州博格韦尔电气有限公司 | Target accurate monitoring system based on electronic information |
| CN113221682A (en) * | 2021-04-27 | 2021-08-06 | 天津大学 | Bridge vehicle load space-time distribution fine-grained identification method based on computer vision |
| CN113312744A (en) * | 2021-04-08 | 2021-08-27 | 同济大学 | Vehicle load prediction method and system |
| CN113837007A (en) * | 2021-08-23 | 2021-12-24 | 哈尔滨工业大学 | Bridge load distribution identification method based on vehicle matching deep learning in bridge monitoring video |
| CN115909223A (en) * | 2022-10-14 | 2023-04-04 | 北京科技大学 | Method and system for matching WIM system information with monitoring video data |
| CN117197760A (en) * | 2023-09-06 | 2023-12-08 | 东南大学 | Bridge vehicle load distribution long-term monitoring method based on video monitoring |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103593678A (en) * | 2013-10-16 | 2014-02-19 | 长安大学 | Long-span bridge vehicle dynamic load distribution detection method |
| CN104021676A (en) * | 2014-06-25 | 2014-09-03 | 上海交通大学 | Vehicle positioning and speed measuring method based on dynamic video feature of vehicle |
| JP2014174280A (en) * | 2013-03-07 | 2014-09-22 | Hitachi Ltd | Multi-camera photographing device |
| CN104599249A (en) * | 2015-01-13 | 2015-05-06 | 重庆大学 | Cable rope bearing bridge deck vehicle load distribution real-time detection method |
| US20160198485A1 (en) * | 2010-11-03 | 2016-07-07 | Broadcom Corporation | Network management module for a vehicle communication network |
| CN107580199A (en) * | 2017-09-08 | 2018-01-12 | 深圳市伊码泰珂电子有限公司 | The target positioning of overlapping ken multiple-camera collaboration and tracking system |
-
2018
- 2018-05-21 CN CN201810487110.3A patent/CN109167956B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160198485A1 (en) * | 2010-11-03 | 2016-07-07 | Broadcom Corporation | Network management module for a vehicle communication network |
| JP2014174280A (en) * | 2013-03-07 | 2014-09-22 | Hitachi Ltd | Multi-camera photographing device |
| CN103593678A (en) * | 2013-10-16 | 2014-02-19 | 长安大学 | Long-span bridge vehicle dynamic load distribution detection method |
| CN104021676A (en) * | 2014-06-25 | 2014-09-03 | 上海交通大学 | Vehicle positioning and speed measuring method based on dynamic video feature of vehicle |
| CN104599249A (en) * | 2015-01-13 | 2015-05-06 | 重庆大学 | Cable rope bearing bridge deck vehicle load distribution real-time detection method |
| CN107580199A (en) * | 2017-09-08 | 2018-01-12 | 深圳市伊码泰珂电子有限公司 | The target positioning of overlapping ken multiple-camera collaboration and tracking system |
Non-Patent Citations (2)
| Title |
|---|
| SOHAIB KHAN ET AL: "Consistent labeling of tracked objects in multiple cameras with overlapping fields of view", 《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》 * |
| 陈亚东: "基于视频的车辆违章行为检测研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109816006B (en) * | 2019-01-18 | 2020-11-13 | 深圳大学 | Sea-sky-line detection method and device and computer-readable storage medium |
| CN109816006A (en) * | 2019-01-18 | 2019-05-28 | 深圳大学 | A kind of sea horizon detection method, device and computer readable storage medium |
| CN111354035A (en) * | 2020-03-17 | 2020-06-30 | 陕西高速机械化工程有限公司 | Automatic bridge dynamic load detection and inversion system and method |
| CN111354035B (en) * | 2020-03-17 | 2023-09-05 | 陕西高速机械化工程有限公司 | Automatic bridge dynamic load detection and inversion system and method |
| CN113022540A (en) * | 2020-04-17 | 2021-06-25 | 青岛慧拓智能机器有限公司 | Real-time remote driving system and method for monitoring multiple vehicle states |
| CN112307888A (en) * | 2020-09-21 | 2021-02-02 | 中铁第四勘察设计院集团有限公司 | Method and system for identifying dynamic load of bridge based on machine vision positioning |
| CN112307888B (en) * | 2020-09-21 | 2022-04-26 | 中铁第四勘察设计院集团有限公司 | Method and system for identifying dynamic load of bridge based on machine vision positioning |
| CN112444311A (en) * | 2020-11-22 | 2021-03-05 | 同济大学 | Method for monitoring space-time load of bridge vehicle |
| CN112504414A (en) * | 2020-11-27 | 2021-03-16 | 湖南大学 | Vehicle dynamic weighing method and system based on non-contact measurement of dynamic deflection of bridge |
| CN112504414B (en) * | 2020-11-27 | 2021-11-12 | 湖南大学 | Vehicle dynamic weighing method and system based on non-contact measurement of dynamic deflection of bridge |
| CN112820112A (en) * | 2021-02-05 | 2021-05-18 | 同济大学 | Bridge floor traffic flow full-view-field sensing system and method depending on bridge tower column |
| CN112885096A (en) * | 2021-02-05 | 2021-06-01 | 同济大学 | Bridge floor traffic flow full-view-field sensing system and method depending on bridge arch ribs |
| CN112885110A (en) * | 2021-02-05 | 2021-06-01 | 同济大学 | Bridge floor traffic flow full-view-field sensing system and method depending on adjacent high-rise structure |
| CN112885097A (en) * | 2021-02-07 | 2021-06-01 | 启迪云控(上海)汽车科技有限公司 | Road side fusion management method and system based on cross-point location |
| CN113114981B (en) * | 2021-03-11 | 2022-09-23 | 联想(北京)有限公司 | Region determination method, electronic equipment and system |
| CN113114981A (en) * | 2021-03-11 | 2021-07-13 | 联想(北京)有限公司 | Region determination method, electronic device and system |
| CN113312744A (en) * | 2021-04-08 | 2021-08-27 | 同济大学 | Vehicle load prediction method and system |
| CN113312744B (en) * | 2021-04-08 | 2022-11-18 | 同济大学 | Vehicle load prediction method and system |
| CN113112798B (en) * | 2021-04-09 | 2023-01-24 | 苏庆裕 | Vehicle overload detection method, system and storage medium |
| CN113112798B9 (en) * | 2021-04-09 | 2023-04-07 | 苏庆裕 | Vehicle overload detection method, system and storage medium |
| CN113112798A (en) * | 2021-04-09 | 2021-07-13 | 苏庆裕 | Vehicle overload detection method, system and storage medium |
| CN113114938A (en) * | 2021-04-12 | 2021-07-13 | 滁州博格韦尔电气有限公司 | Target accurate monitoring system based on electronic information |
| CN113114938B (en) * | 2021-04-12 | 2022-07-19 | 滁州博格韦尔电气有限公司 | Target accurate monitoring system based on electronic information |
| CN113221682B (en) * | 2021-04-27 | 2022-12-27 | 天津大学 | Bridge vehicle load space-time distribution fine-grained identification method based on computer vision |
| CN113221682A (en) * | 2021-04-27 | 2021-08-06 | 天津大学 | Bridge vehicle load space-time distribution fine-grained identification method based on computer vision |
| CN113837007B (en) * | 2021-08-23 | 2022-04-12 | 哈尔滨工业大学 | Bridge load distribution identification method based on vehicle matching deep learning in bridge monitoring video |
| CN113837007A (en) * | 2021-08-23 | 2021-12-24 | 哈尔滨工业大学 | Bridge load distribution identification method based on vehicle matching deep learning in bridge monitoring video |
| CN115909223A (en) * | 2022-10-14 | 2023-04-04 | 北京科技大学 | Method and system for matching WIM system information with monitoring video data |
| CN117197760A (en) * | 2023-09-06 | 2023-12-08 | 东南大学 | Bridge vehicle load distribution long-term monitoring method based on video monitoring |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109167956B (en) | 2021-02-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109167956A (en) | The full-bridge face traveling load spatial distribution merged based on dynamic weighing and more video informations monitors system | |
| CN103473554B (en) | Artificial abortion's statistical system and method | |
| CN105157608B (en) | A kind of detection method of overrun vehicle, apparatus and system | |
| CN101075376B (en) | Intelligent video traffic monitoring system based on multi-viewpoints and its method | |
| CN103499337B (en) | Vehicle-mounted monocular camera distance and height measuring device based on vertical target | |
| CN105160702A (en) | Stereoscopic image dense matching method and system based on LiDAR point cloud assistance | |
| CN106097726A (en) | The detection determination in region, traffic information detection method and device | |
| CN104021676A (en) | Vehicle positioning and speed measuring method based on dynamic video feature of vehicle | |
| KR20200064873A (en) | Method for detecting a speed employing difference of distance between an object and a monitoring camera | |
| CN106327880B (en) | A kind of speed recognition methods and its system based on monitor video | |
| Zhang et al. | Automatic matching of construction onsite resources under camera views | |
| CN102243765A (en) | Multi-camera-based multi-objective positioning tracking method and system | |
| CN104880154A (en) | Internet-of-things binocular vision zoom dynamic target tracking test system platform and Internet-of-things binocular vision zoom dynamic target tracking ranging method | |
| Vakili et al. | Single–camera vehicle speed measurement using the geometry of the imaging system | |
| CN109410264A (en) | A kind of front vehicles distance measurement method based on laser point cloud and image co-registration | |
| CN106164930A (en) | Method for lane detection | |
| CN102523370B (en) | Multi-camera video abnormal behavior detection method based on network transmission algorithm | |
| Do et al. | A simple camera calibration method for vehicle velocity estimation | |
| Sochor et al. | Brnocompspeed: Review of traffic camera calibration and comprehensive dataset for monocular speed measurement | |
| CN109711267A (en) | A kind of pedestrian identifies again, pedestrian movement's orbit generation method and device | |
| CN106128126A (en) | Plane cognition technology is utilized to reduce the method and system meeting red light number of times rush hour | |
| CN114663473A (en) | Personnel target positioning and tracking method and system based on multi-view information fusion | |
| CN109961461A (en) | A kind of multiple mobile object tracking based on three-dimensional layered graph model | |
| CN104091450B (en) | A kind of video frequency velometer and video frequency speed-measuring method | |
| KR100994722B1 (en) | Method for tracking moving object on multiple cameras using probabilistic camera hand-off |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |