CN109190508A - A kind of multi-cam data fusion method based on space coordinates - Google Patents
A kind of multi-cam data fusion method based on space coordinates Download PDFInfo
- Publication number
- CN109190508A CN109190508A CN201810917557.XA CN201810917557A CN109190508A CN 109190508 A CN109190508 A CN 109190508A CN 201810917557 A CN201810917557 A CN 201810917557A CN 109190508 A CN109190508 A CN 109190508A
- Authority
- CN
- China
- Prior art keywords
- target
- coordinate system
- dimensional image
- camera
- dimensional space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 10
- 238000012549 training Methods 0.000 claims abstract description 28
- 238000001514 detection method Methods 0.000 claims abstract description 27
- 238000013507 mapping Methods 0.000 claims abstract description 18
- 238000012545 processing Methods 0.000 claims abstract description 8
- 238000000034 method Methods 0.000 claims description 32
- 238000012544 monitoring process Methods 0.000 claims description 18
- 238000013135 deep learning Methods 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 claims description 4
- 238000005259 measurement Methods 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 238000009827 uniform distribution Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 10
- 230000004927 fusion Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 101100354045 Caenorhabditis elegans rpn-2 gene Proteins 0.000 description 1
- 101100473185 Neurospora crassa (strain ATCC 24698 / 74-OR23-1A / CBS 708.71 / DSM 1257 / FGSC 987) rpn-1 gene Proteins 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000036544 posture Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/292—Multi-camera tracking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The multi-cam data fusion method based on space coordinates that the invention discloses a kind of, comprising the following steps: construct training dataset for the target for needing to extract, complete the training of object detection and recognition model；The location information of the classification and target of the target in the collected video data of each camera under two dimensional image coordinate system is extracted, the coordinate mapping relations between two dimensional image coordinate system and three-dimensional coordinate system are established；For the collected video stream data of camera in continuous multiple scenes, target detection and target identification processing are carried out, the location information of the classification information and target of the target occurred in each frame under two dimensional image coordinate system is extracted；Target is mapped as coordinate of the target under three-dimensional coordinate system in the location information under two dimensional image coordinate system；And according in current time node each target and a upper timing node in the distance between each target information, obtain motion trace data of the target under three-dimensional coordinate system.
Description
Technical Field
The invention relates to the field of video image processing, in particular to a monitoring data fusion method for multiple cameras in continuous multiple monitoring scenes under a space coordinate system.
Background
In recent years, with the popularization of network cameras for security, intelligent video monitoring technology has rapidly become a current research hotspot. Video data is a record of what happens in a monitored scene, with various types of information being implied. Since the background is fixed in most video surveillance data, the user is really interested in the objects appearing therein and the moving tracks of the objects for the video surveillance data.
At present, most of video monitoring data are respectively stored according to the serial numbers of cameras, and then are analyzed and processed through an intelligent video monitoring technology. Target detection, target recognition and target tracking are three important links of intelligent video monitoring analysis and processing. The target tracking is used for determining the continuous positions of the interested targets in the video sequence, and the target tracking technology is a basic technology in the field of computer vision and has wide application value.
The traditional target tracking technology records the historical motion track of a target based on a two-dimensional image space. The method is easy to implement, and the motion trail of the target in the current scene can be recorded. However, this method has disadvantages: 1. the traditional target tracking technology is limited in a two-dimensional image space and cannot reflect the position change information of a target in the space; 2. when a target moves across scenes, the conventional target tracking technology needs to compare the target in the current scene with the targets in a plurality of subsequent scenes, and then judges the moving direction of the target, so that the target cannot be well continuously tracked in a plurality of scenes.
Disclosure of Invention
The purpose of the invention is: the method comprises the steps of establishing a mapping relation between a two-dimensional image coordinate system and a real three-dimensional space coordinate system, mapping coordinate information of a target under the two-dimensional image coordinate system, which is obtained after target detection and target identification are carried out on video frames obtained by a plurality of cameras, into the space coordinate system, and providing a cross-scene tracking method for the target on the basis, so that fusion of video monitoring data collected by the plurality of cameras is realized.
The method comprises the steps of establishing a coordinate mapping equation between a two-dimensional image coordinate system and a real three-dimensional space coordinate system on the basis of analyzing a camera imaging principle and a camera calibration technology; then converting the coordinates of the target under a two-dimensional image coordinate system, which are acquired by the target detection and target identification method, into the coordinates of the target under a three-dimensional space coordinate system; and finally, tracking the target based on the coordinate of the target in the three-dimensional space coordinate system to realize the data fusion of the multiple cameras. The method can restore the position change information of the target in the real space, can better track the target across scenes, realizes the fusion of data acquired by a plurality of cameras, and provides more information for high-level target behavior analysis.
In order to achieve the above object, the present invention provides a multi-camera data fusion method based on a spatial coordinate system, which comprises the following steps:
deploying each camera in a scene to be monitored; constructing a training data set aiming at a target to be extracted, and finishing the training of a target detection and recognition model; extracting the category of a target and the position information of the target in a two-dimensional image coordinate system in video data acquired by each camera based on a target detection and identification model, and establishing a coordinate mapping relation between the two-dimensional image coordinate system and a three-dimensional space coordinate system; carrying out target detection and target identification processing on video stream data acquired by cameras in a plurality of continuous scenes, and extracting the category information of targets appearing in each frame and the position information of the targets in a two-dimensional image coordinate system; based on the coordinate mapping relation, mapping the position information of the target in the two-dimensional image coordinate system into the coordinate of the target in the three-dimensional space coordinate system; and finding the same targets in the adjacent time nodes according to the distance information between each target in the current time node and each target in the previous time node, and connecting the same targets to obtain the motion trail data of the targets in the three-dimensional space coordinate system.
Preferentially, the step of establishing a coordinate mapping relation between a two-dimensional image coordinate system and a three-dimensional space coordinate system specifically comprises the following steps:
for a scene monitored by each camera, pixel coordinates in a two-dimensional image coordinate system with 50 uniform distributions are selected and are marked as { (u)1,v1),(u2,v2),……,(u50,v50) And simultaneously acquiring corresponding space coordinates of the 50 pixel coordinates in a real three-dimensional space coordinate system, and recording as { (X)1,Y1,Z1),(X2,Y2,Z2),……,(X3,Y3,Z3) The coordinates of each point in a real three-dimensional space coordinate system can be measured manuallyOr the GPS positioning method is utilized for obtaining;
during the imaging process of the camera, the coordinate points (u, v) of the two-dimensional image and the corresponding coordinate points (X, Y, Z) of the three-dimensional space satisfy the following conditions:
considering that the targets we are interested in are all on ground level, i.e. Z ═ 0, the above equation can be converted to:
wherein, the parameter (w)1,w2,b1,w3,w4,b2) As a constant, the sum of the squares of the deviations of all the fitting results from the actual data is found by the least squares methodAndminimum parameter (w)1,w2,b1,w3,w4,b2) I.e. solving the system of equations:
preferably, the step of finding the same target in the adjacent time nodes according to the distance information between each target in the current time node and each target in the previous time node, and connecting the same targets to obtain the motion trajectory data of the target in the three-dimensional space coordinate system specifically includes:
reading coordinate data of the target in a three-dimensional space coordinate system under the monitoring scene of each camera, and sequencing according to the occurrence time of the coordinate data;
calculating the coordinates ((i +1) of each target in the (i +1) th time node under a three-dimensional space coordinate system1～(i+1)a) Coordinates (i) of each target in the ith time node in a three-dimensional space coordinate system1～ib) Distance between, noted dxy(where x ∈ [1, a ]]，y∈[1,b])；
When x is a constant value, temporarily consider d to bexyThe minimum y and x are the numbers of the same target in two adjacent time nodes;
if d isxyIf the number of the target is less than the given threshold value T, determining that x and y are the numbers of the same target in two adjacent time nodes, otherwise, determining that x is the number of a new target appearing in the monitoring scene for the first time at the (i +1) th time node;
sequentially connecting the coordinates of the same target in two adjacent time nodes under a three-dimensional space coordinate system, and simultaneously recording the category and the start-stop time node of the target to which each coordinate connecting line belongs to obtain the motion trail data of the target under the scene monitored by each camera under the three-dimensional space coordinate system.
Preferably, the training step of the target detection and recognition model is completed, which specifically includes:
step 201: for each target to be extracted, collecting 500-1000 pictures containing the target, wherein the pictures should include the targets shot from different angles as much as possible;
step 202: adding a category label to the picture acquired in step 201 by a manual marking method;
step 203: adding position information of the target in a two-dimensional image coordinate system to the picture acquired in the step 201 by a manual marking method;
step 204: randomly disordering a picture data set for training to construct a training data set, and importing a deep learning frame fast-RCNN based on Caffe;
step 205: modifying the training parameters according to the categories and the number of the categories of the training set, and setting the iteration times;
step 206: and training a target detection and recognition model.
Preferably, the step of extracting the category of the target and the position information of the target in the two-dimensional image coordinate system in the video data collected by each camera based on the target detection and recognition model specifically includes:
for video stream data acquired by cameras in a plurality of continuous scenes, target detection and target identification processing is carried out by using a target detection and identification model, and the category information of a target in a current frame and the position information of the target in a two-dimensional image coordinate system, which are acquired by each camera, are extracted every 1 second; meanwhile, the ID and the acquisition time of the camera to which the target belongs are acquired, and the information of the target is formed by the ID and the acquisition time and the position information of the target.
According to the method, a target detection and target recognition model is trained by utilizing a deep learning framework according to the actual needs of monitoring scenes, and the category information of a plurality of different targets and the position information of the targets under a two-dimensional image coordinate system can be simultaneously extracted from images acquired by cameras in a plurality of continuous monitoring scenes; establishing a coordinate mapping relation between a two-dimensional image coordinate system and a three-dimensional space coordinate system through a least square method, and converting the detected position information of each target in the two-dimensional image coordinate system into a coordinate in the target three-dimensional space coordinate system; and obtaining the motion track of the target in a three-dimensional space coordinate system by calculating the distance information between the targets under the adjacent time nodes. Compared with the traditional target tracking technology, the method can fuse the scattered monitoring data collected by each camera, and can simultaneously acquire the spatial position change information of a plurality of targets under a plurality of continuous scenes; the conversion from a two-dimensional pixel coordinate to a three-dimensional space coordinate can be completed through a coordinate mapping equation, and the tracking of the target in the three-dimensional space is realized; meanwhile, compared with the storage of continuous image data adopted by the traditional target tracking technology, the method for storing the category data and the track data of the target greatly reduces the amount of stored data and improves the calculation efficiency.
Drawings
Fig. 1 is a schematic flowchart of a multi-camera data fusion method based on a spatial coordinate system according to an embodiment of the present invention;
FIG. 2 is a flow chart of a target detection and recognition model training process.
Detailed Description
The invention relates to a multi-camera data fusion method based on a space coordinate system, which is further described in detail by combining the accompanying drawings:
fig. 1 is a schematic flow chart of a multi-camera data fusion method based on a spatial coordinate system according to an embodiment of the present invention. As shown in fig. 1, the multi-camera data fusion method based on the spatial coordinate system includes steps S1-S6:
s1, deploying multiple cameras in the scene to be monitored, wherein the edges of the scene covered by adjacent cameras should be as close as possible, and the edges of the scene covered by adjacent cameras should overlap as little as possible.
S2, training a target detection and recognition model, wherein a model training flow chart is shown in the attached figure 2, and the method comprises the following specific implementation steps:
step 201: 500 and 1000 pictures containing at least one category target are selected from each category. For the selection of the pictures, the pictures shot at different angles and the pictures containing the targets with different postures should be selected as much as possible to form a picture data set.
Step 202: adding a category label to the picture acquired in step 201 by a manual labeling method, where the category label is a category to which the object in the picture belongs.
Step 203: adding position information of the target in the two-dimensional image coordinate system to the picture acquired in step 201 by a manual marking method, wherein the position information of the target in the two-dimensional image coordinate system is coordinate information (x1, y1, x2, y2) of a rectangular enclosure frame where the target is located, wherein (x1, y1) is coordinates of an upper left corner of the rectangular enclosure frame where the target is located, and (x2, y2) is coordinates of a lower right corner of the rectangular enclosure frame where the target is located.
Step 204: randomly disordering a picture data set for training to construct a training data set, dividing the training data set, a test set and a verification set according to the proportion of 7:2:1, and importing a deep learning framework fast-RCNN.
Step 205: the training parameters are modified according to the categories of the targets in the training set and the number of the target categories, and the number of iterations is set, including rpn 1 st stage, fast rcnn 1 st stage, rpn 2 nd stage, and fast rcnn 2 nd stage.
Step 206: and training a target detection and identification model, and obtaining a target detection and identification model file with the suffix name of the coffee model after the set iteration times are finished.
And S3, carrying out target detection and target identification processing on the current frame acquired by the cameras every 1 second for the video stream data acquired by the cameras in a plurality of continuous scenes. Extracting the category information of the target in the current frame and the two-dimensional pixel coordinate position information (u) of the target under a two-dimensional image coordinate systemi,vi) Meanwhile, the ID of the camera to which the target belongs and the time information of the appearance of the target need to be stored, and the information together form the information of the target.
S4, the specific implementation steps of the calculation method of the coordinate mapping equation between the two-dimensional coordinates and the three-dimensional coordinates are as follows: during the imaging process of the camera, the following relation is satisfied between the coordinate points (u, v) of the two-dimensional image and the corresponding coordinate points (X, Y, Z) of the three-dimensional space:
considering that the targets we are interested in are all on ground level, i.e. Z ═ 0, equation (1) can be converted into:
wherein, based on the above, a coordinate mapping relationship between a two-dimensional image coordinate system and a three-dimensional space coordinate system can be established by a least square method, which comprises the following steps:
step 401: for the scene monitored by each camera, pixel coordinates in 50 uniformly distributed two-dimensional image coordinate systems are selected and are recorded as { (u)1,v1),(u2,v2),……,(u50,v50) And acquiring corresponding space coordinates of the 50 pixel coordinates in a real three-dimensional space coordinate system by manual measurement or a GPS (global positioning system) positioning method, and recording as { (X)1,Y1,Z1),(X2,Y2,Z2),……,(X3,Y3,Z3)}。
Step 402: selecting appropriate parameters (w) by least squares1,w2,b1,w3,w4,b2) Ensuring the sum of squares M of the deviations of all fitting results from the actual dataX，MYMinimum, MX，MYCan be expressed as:
make MX，MYThe minimum time parameter (w) can be taken1,w2,b1,w3,w4,b2) I.e. solving the system of equations:
will parameter (w)1,w2,b1,w3,w4,b2) And substituting the equation into the formula (2) to obtain a mapping equation between a two-dimensional image coordinate system and a three-dimensional space coordinate system in the scene monitored by the camera.
S5, two-dimensional pixel coordinate position information (u) of each object extracted in the step S3 in the two-dimensional image coordinate systemi,vi) Inputting a mapping equation between the two-dimensional image coordinate system and the three-dimensional space coordinate system obtained by calculation in the step 4:
thereby obtaining the coordinate (X) of the target detected in each monitoring scene in the three-dimensional space coordinate systemi,Yi,Zi)。
S6, fusing multi-camera data, and extracting trajectory data of the target in a three-dimensional space coordinate system based on the coordinates of the target in the three-dimensional space coordinate system under each monitoring scene, wherein the method specifically comprises the following steps:
step 601: and sequentially reading the coordinates of the target under the three-dimensional space coordinate system under each monitoring scene in two adjacent time nodes according to the sequence of the occurrence time.
Step 602: calculating the coordinates (i +1) of each target in the (i +1) th time node under the three-dimensional space coordinate system1～(i+1)a) Coordinates (i) of each target in the ith time node in a three-dimensional space coordinate system1～ib) Distance between, noted dxy(where x ∈ [1, a ]]，y∈[1,b])。
Step 603: when x is constant, d is selected such thatxyAnd the minimum y is temporarily considered as the number of the same target in two adjacent time nodes.
Step 604: if d isxyAnd if the number is less than the given threshold value T, determining that x and y are the numbers of the same object in two adjacent time nodes, otherwise, x is the number of a new object which appears in the monitoring scene for the first time at the (i +1) th time node.
Step 605: sequentially connecting the coordinates of the same target in the two adjacent time nodes judged in the step 604 under the three-dimensional space coordinate system, adding the type and the start-stop time node of the target to which the connecting line belongs to each coordinate connecting line, and forming the motion trajectory data of the target under the three-dimensional space coordinate system under the scene monitored by each camera by the data.
The embodiment of the invention can restore the position change information of the target in the real space, can better track the target across scenes, realizes the fusion of the target information acquired by a plurality of cameras, and provides more information for high-level target behavior analysis.
While particular embodiments of the present invention have been shown and described, it will be obvious to those skilled in the art that, based upon the teachings herein, that changes and modifications may be made without departing from the exemplary embodiments of the invention and its broader aspects. Accordingly, the appended claims are intended to encompass within their scope all such changes and modifications as are within the true spirit and scope of this exemplary embodiment of this invention.
Claims (5)
1. A multi-camera data fusion method based on a space coordinate system is characterized by comprising the following steps:
deploying each camera in a scene to be monitored;
constructing a training data set aiming at a target to be extracted, and finishing the training of a target detection and recognition model; extracting the category of a target and the position information of the target in a two-dimensional image coordinate system in video data acquired by each camera based on a target detection and identification model, and establishing a coordinate mapping relation between the two-dimensional image coordinate system and a three-dimensional space coordinate system;
carrying out target detection and target identification processing on video stream data acquired by cameras in a plurality of continuous scenes, and extracting the category information of targets appearing in each frame and the position information of the targets in a two-dimensional image coordinate system;
based on the coordinate mapping relation, mapping the position information of the target in the two-dimensional image coordinate system into the coordinate of the target in the three-dimensional space coordinate system; and finding the same targets in the adjacent time nodes according to the distance information between each target in the current time node and each target in the previous time node, and connecting the same targets to obtain the motion trail data of the targets in the three-dimensional space coordinate system.
2. The method according to claim 1, wherein the step of establishing a coordinate mapping relationship between a two-dimensional image coordinate system and a three-dimensional space coordinate system specifically comprises:
for a scene monitored by each camera, pixel coordinates in a two-dimensional image coordinate system with 50 uniform distributions are selected and are marked as { (u)1,v1),(u2,v2),……,(u50,v50) And simultaneously acquiring corresponding space coordinates of the 50 pixel coordinates in a real three-dimensional space coordinate system, and recording as { (X)1,Y1,Z1),(X2,Y2,Z2),……,(X3,Y3,Z3) Acquiring the coordinates of each point in a real three-dimensional space coordinate system by manual measurement or a GPS positioning method;
during the imaging process of the camera, the coordinate points (u, v) of the two-dimensional image and the corresponding coordinate points (X, Y, Z) of the three-dimensional space satisfy the following conditions:
considering that the targets we are interested in are all on ground level, i.e. Z ═ 0, the above equation can be converted to:
wherein, the parameter (w)1,w2,b1,w3,w4,b2) As a constant, the sum of the squares of the deviations of all the fitting results from the actual data is found by the least squares methodAnd the minimum parameters w1, w2, b1, w3, w4 and b2 are the solution of the equation system:
3. the method according to claim 1, wherein the step of finding the same object in the adjacent time nodes according to the distance information between each object in the current time node and each object in the previous time node, and connecting the same objects to obtain the motion trajectory data of the objects in the three-dimensional space coordinate system specifically comprises:
reading coordinate data of the target in a three-dimensional space coordinate system under the monitoring scene of each camera, and sequencing according to the occurrence time of the coordinate data;
calculating the coordinates ((i +1) of each target in the (i +1) th time node under a three-dimensional space coordinate system1～(i+1)a) Coordinates (i) of each target in the ith time node in a three-dimensional space coordinate system1～ib) Distance between, noted dxy(wherein x ∈ 2 [, [ solution ]1,a]，y∈[1,b])；
When x is a constant value, temporarily consider d to bexyThe minimum y and x are the numbers of the same target in two adjacent time nodes;
if d isxyIf the number of the target is less than the given threshold value T, determining that x and y are the numbers of the same target in two adjacent time nodes, otherwise, determining that x is the number of a new target appearing in the monitoring scene for the first time at the (i +1) th time node;
sequentially connecting the coordinates of the same target in two adjacent time nodes under a three-dimensional space coordinate system, and simultaneously recording the category and the start-stop time node of the target to which each coordinate connecting line belongs to obtain the motion trail data of the target under the scene monitored by each camera under the three-dimensional space coordinate system.
4. The method according to claim 1, wherein the step of constructing a training data set for the target to be extracted to complete training of the target detection and recognition model specifically comprises:
step 201: for each target to be extracted, collecting 500-1000 pictures containing the target, wherein the pictures should include the targets shot from different angles as much as possible;
step 202: adding a category label to the picture acquired in step 201 by a manual marking method;
step 203: adding position information of the target in a two-dimensional image coordinate system to the picture acquired in the step 201 by a manual marking method;
step 204: randomly disordering a picture data set for training to construct a training data set, and importing a deep learning frame fast-RCNN based on Caffe;
step 205: modifying the training parameters according to the categories and the number of the categories of the training set, and setting the iteration times;
step 206: and training a target detection and recognition model.
5. The method according to claim 1, wherein the step of extracting the category of the target and the position information of the target under a two-dimensional image coordinate system in the video data collected by each camera based on the target detection and recognition model specifically comprises:
for video stream data acquired by cameras in a plurality of continuous scenes, target detection and target identification processing is carried out by using a target detection and identification model, and the category information of a target in a current frame and the position information of the target in a two-dimensional image coordinate system, which are acquired by each camera, are extracted every 1 second; meanwhile, the ID and the acquisition time of the camera to which the target belongs are acquired, and the information of the target is formed by the ID and the acquisition time and the position information of the target.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810917557.XA CN109190508B (en) | 2018-08-13 | 2018-08-13 | Multi-camera data fusion method based on space coordinate system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810917557.XA CN109190508B (en) | 2018-08-13 | 2018-08-13 | Multi-camera data fusion method based on space coordinate system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109190508A true CN109190508A (en) | 2019-01-11 |
| CN109190508B CN109190508B (en) | 2022-09-06 |
Family
ID=64921676
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810917557.XA Active CN109190508B (en) | 2018-08-13 | 2018-08-13 | Multi-camera data fusion method based on space coordinate system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109190508B (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109840503A (en) * | 2019-01-31 | 2019-06-04 | 深兰科技（上海）有限公司 | A kind of method and device of determining information |
| CN109919064A (en) * | 2019-02-27 | 2019-06-21 | 湖南信达通信息技术有限公司 | Demographic method and device in real time in a kind of rail transit cars |
| CN110000793A (en) * | 2019-04-29 | 2019-07-12 | 武汉库柏特科技有限公司 | A kind of motion planning and robot control method, apparatus, storage medium and robot |
| CN110443228A (en) * | 2019-08-20 | 2019-11-12 | 图谱未来(南京)人工智能研究院有限公司 | A kind of method for pedestrian matching, device, electronic equipment and storage medium |
| CN110738846A (en) * | 2019-09-27 | 2020-01-31 | 同济大学 | Vehicle behavior monitoring system based on radar and video group and implementation method thereof |
| CN110888957A (en) * | 2019-11-22 | 2020-03-17 | 腾讯科技（深圳）有限公司 | Object positioning method and related device |
| CN111597954A (en) * | 2020-05-12 | 2020-08-28 | 博康云信科技有限公司 | Method and system for identifying vehicle position in monitoring video |
| CN111754552A (en) * | 2020-06-29 | 2020-10-09 | 华东师范大学 | Multi-camera cooperative target tracking method based on deep learning |
| CN111985307A (en) * | 2020-07-07 | 2020-11-24 | 深圳市自行科技有限公司 | Driver specific action detection method, system and device |
| CN112037159A (en) * | 2020-07-29 | 2020-12-04 | 长安大学 | Cross-camera road space fusion and vehicle target detection tracking method and system |
| CN112307912A (en) * | 2020-10-19 | 2021-02-02 | 科大国创云网科技有限公司 | Method and system for determining personnel track based on camera |
| CN114078326A (en) * | 2020-08-19 | 2022-02-22 | 北京万集科技股份有限公司 | Collision detection method, device, visual sensor and storage medium |
| CN115222920A (en) * | 2022-09-20 | 2022-10-21 | 北京智汇云舟科技有限公司 | Image-based digital twin space-time knowledge graph construction method and device |
| CN116528062A (en) * | 2023-07-05 | 2023-08-01 | 合肥中科类脑智能技术有限公司 | Multi-target tracking method |
| CN116612594A (en) * | 2023-05-11 | 2023-08-18 | 深圳市云之音科技有限公司 | Intelligent monitoring and outbound system and method based on big data |
| CN117692583A (en) * | 2023-12-04 | 2024-03-12 | 中国人民解放军92941部队 | Image auxiliary guide method and device based on position information verification |
| CN117687426A (en) * | 2024-01-31 | 2024-03-12 | 成都航空职业技术学院 | Unmanned aerial vehicle flight control method and system in low-altitude environment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104501740A (en) * | 2014-12-18 | 2015-04-08 | 杭州鼎热科技有限公司 | Handheld laser three-dimension scanning method and handheld laser three-dimension scanning equipment based on mark point trajectory tracking |
| CN106127137A (en) * | 2016-06-21 | 2016-11-16 | 长安大学 | A kind of target detection recognizer based on 3D trajectory analysis |
| CN106952289A (en) * | 2017-03-03 | 2017-07-14 | 中国民航大学 | The WiFi object localization methods analyzed with reference to deep video |
-
2018
- 2018-08-13 CN CN201810917557.XA patent/CN109190508B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104501740A (en) * | 2014-12-18 | 2015-04-08 | 杭州鼎热科技有限公司 | Handheld laser three-dimension scanning method and handheld laser three-dimension scanning equipment based on mark point trajectory tracking |
| CN106127137A (en) * | 2016-06-21 | 2016-11-16 | 长安大学 | A kind of target detection recognizer based on 3D trajectory analysis |
| CN106952289A (en) * | 2017-03-03 | 2017-07-14 | 中国民航大学 | The WiFi object localization methods analyzed with reference to deep video |
Cited By (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109840503A (en) * | 2019-01-31 | 2019-06-04 | 深兰科技（上海）有限公司 | A kind of method and device of determining information |
| CN109840503B (en) * | 2019-01-31 | 2021-02-26 | 深兰科技（上海）有限公司 | Method and device for determining category information |
| CN109919064B (en) * | 2019-02-27 | 2020-12-22 | 湖南信达通信息技术有限公司 | Real-time people counting method and device in rail transit carriage |
| CN109919064A (en) * | 2019-02-27 | 2019-06-21 | 湖南信达通信息技术有限公司 | Demographic method and device in real time in a kind of rail transit cars |
| CN110000793A (en) * | 2019-04-29 | 2019-07-12 | 武汉库柏特科技有限公司 | A kind of motion planning and robot control method, apparatus, storage medium and robot |
| CN110443228A (en) * | 2019-08-20 | 2019-11-12 | 图谱未来(南京)人工智能研究院有限公司 | A kind of method for pedestrian matching, device, electronic equipment and storage medium |
| CN110738846A (en) * | 2019-09-27 | 2020-01-31 | 同济大学 | Vehicle behavior monitoring system based on radar and video group and implementation method thereof |
| CN110738846B (en) * | 2019-09-27 | 2022-06-17 | 同济大学 | Vehicle behavior monitoring system based on radar and video group and implementation method thereof |
| CN110888957A (en) * | 2019-11-22 | 2020-03-17 | 腾讯科技（深圳）有限公司 | Object positioning method and related device |
| CN111597954A (en) * | 2020-05-12 | 2020-08-28 | 博康云信科技有限公司 | Method and system for identifying vehicle position in monitoring video |
| CN111754552A (en) * | 2020-06-29 | 2020-10-09 | 华东师范大学 | Multi-camera cooperative target tracking method based on deep learning |
| CN111985307A (en) * | 2020-07-07 | 2020-11-24 | 深圳市自行科技有限公司 | Driver specific action detection method, system and device |
| CN112037159B (en) * | 2020-07-29 | 2023-06-23 | 中天智控科技控股股份有限公司 | Cross-camera road space fusion and vehicle target detection tracking method and system |
| CN112037159A (en) * | 2020-07-29 | 2020-12-04 | 长安大学 | Cross-camera road space fusion and vehicle target detection tracking method and system |
| CN114078326A (en) * | 2020-08-19 | 2022-02-22 | 北京万集科技股份有限公司 | Collision detection method, device, visual sensor and storage medium |
| CN112307912A (en) * | 2020-10-19 | 2021-02-02 | 科大国创云网科技有限公司 | Method and system for determining personnel track based on camera |
| CN115222920A (en) * | 2022-09-20 | 2022-10-21 | 北京智汇云舟科技有限公司 | Image-based digital twin space-time knowledge graph construction method and device |
| CN116612594A (en) * | 2023-05-11 | 2023-08-18 | 深圳市云之音科技有限公司 | Intelligent monitoring and outbound system and method based on big data |
| CN116528062A (en) * | 2023-07-05 | 2023-08-01 | 合肥中科类脑智能技术有限公司 | Multi-target tracking method |
| CN116528062B (en) * | 2023-07-05 | 2023-09-15 | 合肥中科类脑智能技术有限公司 | Multi-target tracking method |
| CN117692583A (en) * | 2023-12-04 | 2024-03-12 | 中国人民解放军92941部队 | Image auxiliary guide method and device based on position information verification |
| CN117687426A (en) * | 2024-01-31 | 2024-03-12 | 成都航空职业技术学院 | Unmanned aerial vehicle flight control method and system in low-altitude environment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109190508B (en) | 2022-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109190508B (en) | Multi-camera data fusion method based on space coordinate system | |
| CN111462200B (en) | Cross-video pedestrian positioning and tracking method, system and equipment | |
| Toft et al. | Long-term visual localization revisited | |
| Chavdarova et al. | Wildtrack: A multi-camera hd dataset for dense unscripted pedestrian detection | |
| CN106897670B (en) | Express violence sorting identification method based on computer vision | |
| US11048948B2 (en) | System and method for counting objects | |
| EP2798611B1 (en) | Camera calibration using feature identification | |
| Tang et al. | Cross-camera knowledge transfer for multiview people counting | |
| CN112163537B (en) | Pedestrian abnormal behavior detection method, system, terminal and storage medium | |
| US8179440B2 (en) | Method and system for object surveillance and real time activity recognition | |
| Chen et al. | Indoor camera pose estimation via style‐transfer 3D models | |
| CN111126304A (en) | Augmented reality navigation method based on indoor natural scene image deep learning | |
| CN105938622A (en) | Method and apparatus for detecting object in moving image | |
| Zhang et al. | A swarm intelligence based searching strategy for articulated 3D human body tracking | |
| CN109409250A (en) | A kind of across the video camera pedestrian of no overlap ken recognition methods again based on deep learning | |
| Asadi-Aghbolaghi et al. | Action recognition from RGB-D data: Comparison and fusion of spatio-temporal handcrafted features and deep strategies | |
| CN113256731A (en) | Target detection method and device based on monocular vision | |
| Yang et al. | Intelligent video analysis: A Pedestrian trajectory extraction method for the whole indoor space without blind areas | |
| CN115376034A (en) | Motion video acquisition and editing method and device based on human body three-dimensional posture space-time correlation action recognition | |
| CN117036404A (en) | Monocular thermal imaging simultaneous positioning and mapping method and system | |
| CN113793362A (en) | Pedestrian track extraction method and device based on multi-lens video | |
| Panahi et al. | Automated Progress Monitoring in Modular Construction Factories Using Computer Vision and Building Information Modeling | |
| CN115565253B (en) | Dynamic gesture real-time recognition method and device, electronic equipment and storage medium | |
| CN116912877A (en) | Method and system for monitoring space-time contact behavior sequence of urban public space crowd | |
| CN115767424A (en) | Video positioning method based on RSS and CSI fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |