CN112767442A - Pedestrian three-dimensional detection tracking method and system based on top view angle - Google Patents

Pedestrian three-dimensional detection tracking method and system based on top view angle Download PDF

Info

Publication number
CN112767442A
CN112767442A CN202110064121.2A CN202110064121A CN112767442A CN 112767442 A CN112767442 A CN 112767442A CN 202110064121 A CN202110064121 A CN 202110064121A CN 112767442 A CN112767442 A CN 112767442A
Authority
CN
China
Prior art keywords
dimensional
pedestrian
height
map
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110064121.2A
Other languages
Chinese (zh)
Other versions
CN112767442B (en
Inventor
郑慧诚
苏志荣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sun Yat Sen University
Original Assignee
Sun Yat Sen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sun Yat Sen University filed Critical Sun Yat Sen University
Priority to CN202110064121.2A priority Critical patent/CN112767442B/en
Publication of CN112767442A publication Critical patent/CN112767442A/en
Application granted granted Critical
Publication of CN112767442B publication Critical patent/CN112767442B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a pedestrian three-dimensional detection tracking method and system based on a top view angle, wherein the method comprises the following steps: performing stereo correction and stereo matching processing on left and right images acquired by a binocular camera; filling the holes and converting to obtain a height map; modeling the height map to obtain a two-dimensional height foreground map; projecting and converting the image into a two-dimensional plane point cloud projection image under a top view angle, and detecting to obtain a three-dimensional coordinate point of the head of the pedestrian; mapping the three-dimensional coordinate points of the heads of the pedestrians back to a two-dimensional image coordinate system, and positioning a boundary frame of each pedestrian; and performing interframe pedestrian matching tracking by combining the three-dimensional coordinate points of the head of the pedestrian and the positioned pedestrian boundary frame. By using the invention, the problem of missed detection caused by shielding among pedestrians can be effectively solved, and the detection recall rate of the pedestrians is improved. The pedestrian three-dimensional detection tracking method and system based on the top view angle can be widely applied to the field of pedestrian detection.

Description

Pedestrian three-dimensional detection tracking method and system based on top view angle
Technical Field
The invention relates to the field of video image processing, in particular to a pedestrian three-dimensional detection tracking method and system based on a top view angle.
Background
Pedestrian detection is a popular research direction of computer vision, and is widely applied to daily life of people. And the pedestrian detection problem can be divided into top view angle pedestrian detection and squint angle pedestrian detection according to the difference of the position and the visual angle of the camera. Most of the current mainstream pedestrian detection algorithm researches are pedestrian detection at an oblique view angle with higher flexibility, the view angle has the defects that pedestrians are seriously shielded and adhered and are easy to miss detection, and the pedestrian detection based on the top view angle is a special application scene of a pedestrian detection task, but few people research the aspect at present.
Disclosure of Invention
In order to solve the technical problems, the invention aims to provide a pedestrian three-dimensional detection tracking method based on a top view angle, which can effectively overcome the problem of missed detection caused by shielding among pedestrians and improve the detection recall rate of the pedestrians.
The first technical scheme adopted by the invention is as follows: a pedestrian three-dimensional detection tracking method based on top view angle comprises the following steps:
performing stereo correction and stereo matching processing on left and right images acquired by a binocular camera to obtain a depth map;
filling holes in the depth map, and converting the depth map to obtain a height map;
modeling the height map based on an MOG2 background modeling method to obtain a two-dimensional height foreground map;
specifically, the number of frames for modeling the background model was set to 500 frames, the learning rate was 0.002, the variance threshold was set to 16, and the number of gaussian components was set to 5.
Projecting the two-dimensional height foreground image into a three-dimensional height foreground point cloud image, converting the three-dimensional height foreground image into a two-dimensional plane point cloud projection image under a top view angle, and detecting to obtain a three-dimensional coordinate point of the head of the pedestrian;
mapping the three-dimensional coordinate points of the heads of the pedestrians back to a two-dimensional image coordinate system, and positioning a boundary frame of each pedestrian in the two-dimensional image by combining the three-dimensional coordinate points of the heads;
and performing interframe pedestrian matching tracking by combining the three-dimensional coordinate point of the head of the pedestrian and the positioned pedestrian boundary frame and utilizing the interframe distance of the central point of the pedestrian and the color histogram characteristics in the boundary frame.
Further, before the step of performing stereo correction and stereo matching processing on the left and right images acquired by the binocular camera to obtain the depth map, the method further includes:
and carrying out three-dimensional calibration on the binocular camera to obtain internal and external parameters and a perspective transformation matrix of the camera.
Specifically, a perspective transformation matrix Q of 4 × 4 can be calculated by the stereo calibration step.
Further, the step of performing stereo correction and stereo matching processing on the left and right images acquired by the binocular camera to obtain a depth map specifically includes:
acquiring images based on a binocular camera and performing stereo correction on left and right images acquired by the binocular camera to obtain corrected images;
and carrying out stereo matching on the corrected image based on an SGBM algorithm to obtain a depth map.
Further, the step of filling the holes in the depth map and converting the depth map into the height map specifically includes:
selecting reliable depth points near the cavity as reference values to fill the cavity to obtain a depth map after filling;
and converting the filled depth map into a height map according to the installation height of the binocular camera.
Further, the perspective transformation matrix Q expression is as follows:
Figure BDA0002903507760000021
in the above formula, cxAnd cyThe coordinates of the principal points of the left and right lenses of the binocular camera are respectively represented, f is the normalized focal length, and T is the baseline distance between the optical centers of the left and right lenses. Using this matrix enables two-dimensional height foregroundEach pixel coordinate point in the image is projected into a three-dimensional high foreground point cloud image.
Further, the step of projecting the two-dimensional height foreground map into the three-dimensional height foreground point cloud map and converting the two-dimensional height foreground map into a two-dimensional plane point cloud projection map under a top view angle, and detecting to obtain the three-dimensional coordinate point of the head of the pedestrian specifically comprises:
projecting the two-dimensional height foreground map to form a three-dimensional height foreground point cloud map based on a perspective transformation matrix, and establishing a mapping relation table of two-dimensional pixel coordinate points and three-dimensional physical world coordinate points;
and converting the three-dimensional height foreground point cloud image into a two-dimensional plane point cloud projection image under a top view angle, and detecting and positioning the head of the pedestrian based on the two-dimensional plane point cloud projection image under the top view angle to obtain the three-dimensional coordinate point of the head of the pedestrian.
Specifically, the eight-neighborhood height information of each point is utilized to detect and locate the head of the pedestrian.
Further, the step of mapping the three-dimensional coordinate point of the head of the pedestrian back to the coordinate system of the two-dimensional image and positioning the bounding box of each pedestrian in the two-dimensional image by combining the three-dimensional coordinate point of the head specifically includes:
traversing the whole image, and calculating each pixel point P through a perspective transformation matrix Q to obtain a three-dimensional physical world coordinate point W of each pixel point P;
suppose that the pedestrian head point set S is positioned in the top view angle two-dimensional point cloud projection image { H }1,H2,..,HkIn which H is1,H2,..,HkThree-dimensional coordinates representing the heads of k pedestrians;
calculating a physical point W and three-dimensional points H of each headi(i∈[1,k]) The distance in the plane direction formed by the x axis and the y axis is obtained, and for the point W with the obtained distance smaller than a set distance threshold value, a two-dimensional pixel point P of the point W is marked to belong to the boundary frame of the ith pedestrian, so that a pixel point set of the boundary frame of each pedestrian is obtained;
and calculating the minimum area circumscribed rectangle of the pixel point set of each pedestrian boundary frame to obtain the boundary frame of each pedestrian.
The second technical scheme adopted by the invention is as follows: a pedestrian three-dimensional detection and tracking system based on top view comprises:
the depth map module is used for performing stereo correction and stereo matching processing on the left image and the right image acquired by the binocular camera to obtain a depth map;
the height map module is used for filling holes in the depth map and converting the holes to obtain a height map;
the two-dimensional height foreground map module is used for modeling the height map based on a MOG2 background modeling method to obtain a two-dimensional height foreground map;
the head three-dimensional coordinate point module is used for projecting the two-dimensional height foreground image into the three-dimensional height foreground point cloud image, converting the two-dimensional height foreground image into a two-dimensional plane point cloud projection image under a top view angle, and detecting to obtain a pedestrian head three-dimensional coordinate point;
the boundary frame module is used for mapping the three-dimensional coordinate points of the heads of the pedestrians back to a two-dimensional image coordinate system and positioning the boundary frame of each pedestrian in the two-dimensional image by combining the coordinates of the head points;
and the matching tracking module is used for combining the three-dimensional coordinate point of the head of the pedestrian and the positioned pedestrian boundary frame and carrying out interframe pedestrian matching tracking by utilizing the interframe distance of the central point of the pedestrian and the color histogram characteristics in the boundary frame.
The method and the system have the beneficial effects that: in addition, the invention projects the two-dimensional pixel points after filtering the high foreground image into the three-dimensional point cloud image, constructs the two-dimensional point cloud high foreground image with a vertex angle to detect and position the head of the pedestrian, can better overcome the problem of missed detection caused by shielding among pedestrians compared with the pedestrian detection of the traditional monocular camera, and improves the recall rate of the pedestrian detection.
Drawings
FIG. 1 is a flow chart of steps of a pedestrian three-dimensional detection and tracking method based on top view according to the invention;
fig. 2 is a structural block diagram of a pedestrian three-dimensional detection and tracking system based on a top view angle.
Detailed Description
The invention is described in further detail below with reference to the figures and the specific embodiments. The step numbers in the following embodiments are provided only for convenience of illustration, the order between the steps is not limited at all, and the execution order of each step in the embodiments can be adapted according to the understanding of those skilled in the art.
Referring to fig. 1, the invention provides a pedestrian three-dimensional detection and tracking method based on a top view angle, which comprises the following steps:
s1, performing stereo correction and stereo matching processing on the left image and the right image acquired by the binocular camera to obtain a depth map;
s2, filling holes in the depth map, and converting the holes to obtain a height map;
s3, modeling the height map based on an MOG2 background modeling method to obtain a two-dimensional height foreground map;
s4, projecting the two-dimensional height foreground image into the three-dimensional height foreground point cloud image, converting the two-dimensional height foreground image into a two-dimensional plane point cloud projection image under a top view angle, and detecting to obtain a three-dimensional coordinate point of the head of the pedestrian;
s5, mapping the three-dimensional coordinate points of the heads of the pedestrians back to a two-dimensional image coordinate system, and positioning a boundary frame of each pedestrian in the two-dimensional image by combining the three-dimensional coordinate points of the heads;
and S6, combining the three-dimensional coordinate point of the head of the pedestrian and the positioned pedestrian boundary frame, and performing inter-frame pedestrian matching tracking by using the inter-frame distance of the central point of the pedestrian and the color histogram characteristics in the boundary frame.
As a further preferred embodiment of the method, before the step of performing stereo correction and stereo matching on the left and right images acquired by the binocular camera to obtain the depth map, the method further includes:
and carrying out three-dimensional calibration on the binocular camera to obtain internal and external parameters and a perspective transformation matrix of the camera.
Further as a preferred embodiment of the method, the step of performing stereo correction and stereo matching on the left and right images acquired by the binocular camera to obtain the depth map specifically includes:
acquiring images based on a binocular camera and performing stereo correction on left and right images acquired by the binocular camera to obtain corrected images;
specifically, left and right images acquired by a binocular camera are firstly scaled to 320 × 240 resolution, and then corrected binocular images are acquired by performing stereo correction by using camera internal and external parameters.
And carrying out stereo matching on the corrected image based on an SGBM algorithm to obtain a depth map.
Further, as a preferred embodiment of the method, the step of filling the holes in the depth map and converting the depth map into the height map specifically includes:
selecting reliable depth points near the cavity as reference values to fill the cavity to obtain a depth map after filling;
and converting the filled depth map into a height map according to the installation height of the binocular camera.
Specifically, the depth map is filled with the holes, reliable depth points near the holes are selected as reference values to fill the holes, and specifically, the holes with different areas are filled with filtering by adopting average filtering kernels with different sizes, small filtering kernels are used for the holes with small areas, and large filtering kernels are used for the holes with large areas. Each depth value in the depth map is then subtracted from the camera height to convert to a height map, depending on the camera mounting height.
Further as a preferred embodiment of the method, the expression of the perspective transformation matrix Q is as follows:
Figure BDA0002903507760000051
in the above formula, cxAnd cyThe coordinates of the principal points of the left and right lenses of the binocular camera are respectively represented, f is the normalized focal length, and T is the baseline distance between the optical centers of the left and right lenses.
Further, as a preferred embodiment of the method, the step of projecting the two-dimensional height foreground map into the three-dimensional height foreground point cloud map and converting the two-dimensional height foreground map into a two-dimensional plane point cloud projection map under a top view angle to detect and obtain the three-dimensional coordinate point of the head of the pedestrian specifically includes:
projecting the two-dimensional height foreground map to form a three-dimensional height foreground point cloud map based on a perspective transformation matrix, and establishing a mapping relation table of two-dimensional pixel coordinate points and three-dimensional physical world coordinate points;
and converting the three-dimensional height foreground point cloud image into a two-dimensional plane point cloud projection image under a top view angle, and detecting and positioning the head of the pedestrian based on the two-dimensional plane point cloud projection image under the top view angle to obtain the three-dimensional coordinate point of the head of the pedestrian.
Further, as a preferred embodiment of the method, the step of mapping the three-dimensional coordinate point of the head of the pedestrian back to the coordinate system of the two-dimensional image and positioning the bounding box of each pedestrian in the two-dimensional image by combining the three-dimensional coordinate point of the head specifically includes:
traversing the whole image, and calculating each pixel point P through a perspective transformation matrix Q to obtain a three-dimensional physical world coordinate point W of each pixel point P;
suppose that the pedestrian head point set S is positioned in the top view angle two-dimensional point cloud projection image { H }1,H2,..,HkIn which H is1,H2,..,HkThree-dimensional coordinates representing the heads of k pedestrians;
calculating a physical point W and three-dimensional points H of each headi(i∈[1,k]) The distance in the plane direction formed by the x axis and the y axis is obtained, and for the point W with the obtained distance smaller than a set distance threshold value, a two-dimensional pixel point P of the point W is marked to belong to the boundary frame of the ith pedestrian, so that a pixel point set of the boundary frame of each pedestrian is obtained;
and calculating the minimum area circumscribed rectangle of the pixel point set of each pedestrian boundary frame to obtain the boundary frame of each pedestrian.
As shown in fig. 2, a pedestrian three-dimensional detection and tracking system based on top view includes:
the depth map module is used for performing stereo correction and stereo matching processing on the left image and the right image acquired by the binocular camera to obtain a depth map;
the height map module is used for filling holes in the depth map and converting the holes to obtain a height map;
the two-dimensional height foreground map module is used for modeling the height map based on a MOG2 background modeling method to obtain a two-dimensional height foreground map;
the head three-dimensional coordinate point module is used for projecting the two-dimensional height foreground image into the three-dimensional height foreground point cloud image, converting the two-dimensional height foreground image into a two-dimensional plane point cloud projection image under a top view angle, and detecting to obtain a pedestrian head three-dimensional coordinate point;
the boundary frame module is used for mapping the three-dimensional coordinate points of the heads of the pedestrians back to a two-dimensional image coordinate system and positioning the boundary frame of each pedestrian in the two-dimensional image by combining the coordinates of the head points;
and the matching tracking module is used for combining the three-dimensional coordinate point of the head of the pedestrian and the positioned pedestrian boundary frame and carrying out interframe pedestrian matching tracking by utilizing the interframe distance of the central point of the pedestrian and the color histogram characteristics in the boundary frame.
The contents in the above method embodiments are all applicable to the present system embodiment, the functions specifically implemented by the present system embodiment are the same as those in the above method embodiment, and the beneficial effects achieved by the present system embodiment are also the same as those achieved by the above method embodiment.
While the preferred embodiments of the present invention have been illustrated and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims (8)

1. A pedestrian three-dimensional detection tracking method based on a top view angle is characterized by comprising the following steps:
performing stereo correction and stereo matching processing on left and right images acquired by a binocular camera to obtain a depth map;
filling holes in the depth map, and converting the depth map to obtain a height map;
modeling the height map based on an MOG2 background modeling method to obtain a two-dimensional height foreground map;
projecting the two-dimensional height foreground image into a three-dimensional height foreground point cloud image, converting the three-dimensional height foreground image into a two-dimensional plane point cloud projection image under a top view angle, and detecting to obtain a three-dimensional coordinate point of the head of the pedestrian;
mapping the three-dimensional coordinate points of the heads of the pedestrians back to a two-dimensional image coordinate system, and positioning a boundary frame of each pedestrian in the two-dimensional image by combining the three-dimensional coordinate points of the heads;
and performing interframe pedestrian matching tracking by combining the three-dimensional coordinate point of the head of the pedestrian and the positioned pedestrian boundary frame and utilizing the interframe distance of the central point of the pedestrian and the color histogram characteristics in the boundary frame.
2. The pedestrian three-dimensional detection and tracking method based on the top view angle of claim 1, wherein before the step of performing stereo correction and stereo matching on the left and right images acquired by the binocular camera to obtain the depth map, the method further comprises:
and carrying out three-dimensional calibration on the binocular camera to obtain internal and external parameters and a perspective transformation matrix of the camera.
3. The pedestrian three-dimensional detection and tracking method based on the top view angle of claim 2, wherein the step of performing stereo correction and stereo matching on the left and right images acquired by the binocular camera to obtain the depth map specifically comprises:
acquiring images based on a binocular camera and performing stereo correction on left and right images acquired by the binocular camera to obtain corrected images;
and carrying out stereo matching on the corrected image based on an SGBM algorithm to obtain a depth map.
4. The pedestrian three-dimensional detection and tracking method based on the top view angle as claimed in claim 3, wherein the step of filling the hole in the depth map and converting the depth map into the height map specifically comprises:
selecting reliable depth points near the cavity as reference values to fill the cavity to obtain a depth map after filling;
and converting the filled depth map into a height map according to the installation height of the binocular camera.
5. The pedestrian three-dimensional detection and tracking method based on the top view angle as claimed in claim 4, wherein the perspective transformation matrix Q expression is as follows:
Figure FDA0002903507750000011
in the above formula, cxAnd cyThe coordinates of the principal points of the left and right lenses of the binocular camera are respectively represented, f is the normalized focal length, and T is the baseline distance between the optical centers of the left and right lenses.
6. The pedestrian three-dimensional detection and tracking method based on the top view angle of claim 5, wherein the step of projecting the two-dimensional height foreground map into the three-dimensional height foreground point cloud map and converting the three-dimensional height foreground map into the two-dimensional plane point cloud projection map under the top view angle to detect the three-dimensional coordinate point of the head of the pedestrian specifically comprises:
projecting the two-dimensional height foreground map to form a three-dimensional height foreground point cloud map based on a perspective transformation matrix, and establishing a mapping relation table of two-dimensional pixel coordinate points and three-dimensional physical world coordinate points;
and converting the three-dimensional height foreground point cloud image into a two-dimensional plane point cloud projection image under a top view angle, and detecting and positioning the head of the pedestrian based on the two-dimensional plane point cloud projection image under the top view angle to obtain the three-dimensional coordinate point of the head of the pedestrian.
7. The pedestrian three-dimensional detection and tracking method based on the top view angle as claimed in claim 6, wherein the step of mapping the three-dimensional coordinate point of the head of the pedestrian back to the coordinate system of the two-dimensional image and locating the bounding box of each pedestrian in the two-dimensional image in combination with the three-dimensional coordinate point of the head specifically comprises:
traversing the whole image, and calculating each pixel point P through a perspective transformation matrix Q to obtain a three-dimensional physical world coordinate point W of each pixel point P;
suppose that the pedestrian head point set S is positioned in the top view angle two-dimensional point cloud projection image { H }1,H2,..,HkIn which H is1,H2,..,HkThree-dimensional coordinates representing the heads of k pedestrians;
calculating a physical point W and three-dimensional points H of each headi(i∈[1,k]) The distance in the plane direction formed by the x axis and the y axis is obtained, and for the point W with the obtained distance smaller than a set distance threshold value, a two-dimensional pixel point P of the point W is marked to belong to the boundary frame of the ith pedestrian, so that a pixel point set of the boundary frame of each pedestrian is obtained;
and calculating the minimum area circumscribed rectangle of the pixel point set of each pedestrian boundary frame to obtain the boundary frame of each pedestrian.
8. A pedestrian three-dimensional detection tracking system based on top view angle is characterized by comprising:
the depth map module is used for performing stereo correction and stereo matching processing on the left image and the right image acquired by the binocular camera to obtain a depth map;
the height map module is used for filling holes in the depth map and converting the holes to obtain a height map;
the two-dimensional height foreground map module is used for modeling the height map based on a MOG2 background modeling method to obtain a two-dimensional height foreground map;
the head three-dimensional coordinate point module is used for projecting the two-dimensional height foreground image into the three-dimensional height foreground point cloud image, converting the two-dimensional height foreground image into a two-dimensional plane point cloud projection image under a top view angle, and detecting to obtain a pedestrian head three-dimensional coordinate point;
the boundary frame module is used for mapping the three-dimensional coordinate points of the heads of the pedestrians back to a two-dimensional image coordinate system and positioning the boundary frame of each pedestrian in the two-dimensional image by combining the coordinates of the head points;
and the matching tracking module is used for combining the three-dimensional coordinate point of the head of the pedestrian and the positioned pedestrian boundary frame and carrying out interframe pedestrian matching tracking by utilizing the interframe distance of the central point of the pedestrian and the color histogram characteristics in the boundary frame.
CN202110064121.2A 2021-01-18 2021-01-18 Pedestrian three-dimensional detection tracking method and system based on top view angle Active CN112767442B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110064121.2A CN112767442B (en) 2021-01-18 2021-01-18 Pedestrian three-dimensional detection tracking method and system based on top view angle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110064121.2A CN112767442B (en) 2021-01-18 2021-01-18 Pedestrian three-dimensional detection tracking method and system based on top view angle

Publications (2)

Publication Number Publication Date
CN112767442A true CN112767442A (en) 2021-05-07
CN112767442B CN112767442B (en) 2023-07-21

Family

ID=75702924

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110064121.2A Active CN112767442B (en) 2021-01-18 2021-01-18 Pedestrian three-dimensional detection tracking method and system based on top view angle

Country Status (1)

Country Link
CN (1) CN112767442B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116402857A (en) * 2023-04-14 2023-07-07 北京天睿空间科技股份有限公司 Moving target cross-lens tracking method based on three-dimensional calibration

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101894375A (en) * 2009-05-21 2010-11-24 富士胶片株式会社 Person tracking method and person tracking apparatus
CN102622769A (en) * 2012-03-19 2012-08-01 厦门大学 Multi-target tracking method by taking depth as leading clue under dynamic scene
CN102646309A (en) * 2012-05-18 2012-08-22 成都百威讯科技有限责任公司 Intelligent video perimeter rail system and control method thereof
CN103868460A (en) * 2014-03-13 2014-06-18 桂林电子科技大学 Parallax optimization algorithm-based binocular stereo vision automatic measurement method
CN108230351A (en) * 2016-12-15 2018-06-29 上海杰轩智能科技有限公司 Sales counter evaluation method and system based on binocular stereo vision pedestrian detection
CN109325963A (en) * 2018-08-07 2019-02-12 长安大学 A kind of bus passenger three-dimensional track classification method based on SVM
CN111223053A (en) * 2019-11-18 2020-06-02 北京邮电大学 Data enhancement method based on depth image

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101894375A (en) * 2009-05-21 2010-11-24 富士胶片株式会社 Person tracking method and person tracking apparatus
CN102622769A (en) * 2012-03-19 2012-08-01 厦门大学 Multi-target tracking method by taking depth as leading clue under dynamic scene
CN102646309A (en) * 2012-05-18 2012-08-22 成都百威讯科技有限责任公司 Intelligent video perimeter rail system and control method thereof
CN103868460A (en) * 2014-03-13 2014-06-18 桂林电子科技大学 Parallax optimization algorithm-based binocular stereo vision automatic measurement method
CN108230351A (en) * 2016-12-15 2018-06-29 上海杰轩智能科技有限公司 Sales counter evaluation method and system based on binocular stereo vision pedestrian detection
CN109325963A (en) * 2018-08-07 2019-02-12 长安大学 A kind of bus passenger three-dimensional track classification method based on SVM
CN111223053A (en) * 2019-11-18 2020-06-02 北京邮电大学 Data enhancement method based on depth image

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116402857A (en) * 2023-04-14 2023-07-07 北京天睿空间科技股份有限公司 Moving target cross-lens tracking method based on three-dimensional calibration
CN116402857B (en) * 2023-04-14 2023-11-07 北京天睿空间科技股份有限公司 Moving target cross-lens tracking method based on three-dimensional calibration

Also Published As

Publication number Publication date
CN112767442B (en) 2023-07-21

Similar Documents

Publication Publication Date Title
CN110569704B (en) Multi-strategy self-adaptive lane line detection method based on stereoscopic vision
CN107292965B (en) Virtual and real shielding processing method based on depth image data stream
JP3868876B2 (en) Obstacle detection apparatus and method
JP6955783B2 (en) Information processing methods, equipment, cloud processing devices and computer program products
CN103198488B (en) PTZ surveillance camera realtime posture rapid estimation
CN108470356B (en) Target object rapid ranging method based on binocular vision
CN109472820B (en) Monocular RGB-D camera real-time face reconstruction method and device
CN112801074B (en) Depth map estimation method based on traffic camera
US20060050788A1 (en) Method and device for computer-aided motion estimation
CN109889799B (en) Monocular structure light depth perception method and device based on RGBIR camera
CN110099268B (en) Blind area perspective display method with natural color matching and natural display area fusion
CN112184793B (en) Depth data processing method and device and readable storage medium
US20240202975A1 (en) Data processing
CN110909571B (en) High-precision face recognition space positioning method
CN112767442A (en) Pedestrian three-dimensional detection tracking method and system based on top view angle
Petrovai et al. Obstacle detection using stereovision for Android-based mobile devices
CN112509110A (en) Automatic image data set acquisition and labeling framework for land confrontation intelligent agent
CN113723432B (en) Intelligent identification and positioning tracking method and system based on deep learning
CN116597488A (en) Face recognition method based on Kinect database
TW201516965A (en) Method of detecting multiple moving objects
CN115497073A (en) Real-time obstacle camera detection method based on fusion of vehicle-mounted camera and laser radar
CN115147868A (en) Human body detection method of passenger flow camera, device and storage medium
KR101668649B1 (en) Surrounding environment modeling method and apparatus performing the same
CN111630569B (en) Binocular matching method, visual imaging device and device with storage function
CN112229381A (en) Smart phone ranging method using arm length and camera

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant