WO2023123916A1 - Target tracking method and apparatus, and electronic device and storage medium - Google Patents
Target tracking method and apparatus, and electronic device and storage medium Download PDFInfo
- Publication number
- WO2023123916A1 WO2023123916A1 PCT/CN2022/100164 CN2022100164W WO2023123916A1 WO 2023123916 A1 WO2023123916 A1 WO 2023123916A1 CN 2022100164 W CN2022100164 W CN 2022100164W WO 2023123916 A1 WO2023123916 A1 WO 2023123916A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- path
- detection
- point
- human body
- edge
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Definitions
- the present application relates to the technical field of image processing, and in particular to a target tracking method, device, electronic equipment and storage medium.
- human body tracking technology has been widely used in various aspects of social life, such as face capture in public security, face building in business scenarios, etc.
- most of the existing tracking technologies are based on human body detection under a single camera.
- human body detection is tracked under a single camera. Once the human body is blocked, human body detection will fail, which will greatly affect the tracking accuracy. , but when multiple cameras are used to shoot the human body, it is extremely difficult to fuse information between multiple cameras, resulting in insufficient tracking accuracy.
- the main purpose of this application is to provide a target tracking method, including:
- the obtained result path is determined as a trace result.
- the determining the human body detection images corresponding to the human body regions in the multiple pictures includes:
- performing feature fusion on the plurality of human detection images according to association information between the plurality of human detection images to construct the first route image includes:
- the 3D hypothetical point and the first weighted edge are calculated, and the calculation result is combined with the detection point, the 3D hypothetical point, and the first weighted edge to construct the first path image.
- the method includes:
- the epipolar distance and the feature similarity are calculated based on the Mahalanobis distance to determine a corresponding second weighted edge.
- the calculation is performed on the three-dimensional hypothetical point and the first weighted edge, and the calculation result is combined with the detection point, the three-dimensional hypothetical point and the first weighted edge to construct the first path image ,include:
- the detection point, the three-dimensional hypothetical point, the first weight edge, the second weight edge and the third weight edge are constructed to obtain the first path image.
- adding mutually exclusive information to the first path image to construct the second path image includes:
- the correct 3D real point, mutually exclusive edge, detection point, 3D hypothetical point, first weighted edge, second weighted edge, and third weighted edge are constructed to obtain the second path image.
- performing path calculation on the second path image according to a preset algorithm to obtain a resultant path includes:
- the shortest path is determined as the resulting path.
- the embodiment of the present application provides a target tracking device, including:
- a first determining module configured to determine human body detection images corresponding to human body regions in multiple pictures
- a fusion module configured to perform feature fusion of the plurality of human detection images according to the association information between the plurality of human detection images, so as to construct a first path image
- a building module configured to add mutually exclusive information to the first path image to construct a second path image
- a calculation module configured to perform path calculation on the second path image according to a preset algorithm to obtain a resultant path
- the second determining module is configured to determine the obtained result path as a tracking result.
- an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor.
- the processor executes the computer program, Steps for realizing the above-mentioned target tracking method.
- the embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned target tracking method are implemented.
- the target tracking method provided in the present application firstly determines the human body detection images corresponding to the human body regions in multiple pictures; and performs feature fusion on the multiple human body detection images according to the correlation information between the multiple human body detection images, to construct a first path image; then add mutually exclusive information to the first path image to construct a second path image; perform path calculation on the second path image according to a preset algorithm to obtain a resultant path; finally The obtained result path is determined as a tracking result; thus, the information fusion between multiple cameras can be made simpler, and the tracking accuracy can be improved.
- FIG. 1 is a schematic diagram of the overall flow of a target tracking method provided in an embodiment of the present application
- FIG. 2 is a schematic flowchart of step S10 provided in the embodiment of the present application.
- FIG. 3 is an example diagram of a target tracking method provided in an embodiment of the present application.
- FIG. 4 is another example diagram of the target tracking method provided by the embodiment of the present application.
- FIG. 5 is another example diagram of the target tracking method provided by the embodiment of the present application.
- FIG. 6 is a schematic flowchart of step S20 provided in the embodiment of the present application.
- FIG. 7 is another schematic flowchart of step S21 provided by the embodiment of the present application.
- FIG. 8 is a schematic flowchart of step S25 provided in the embodiment of the present application.
- FIG. 9 is a schematic flowchart of step S30 provided in the embodiment of the present application.
- FIG. 10 is a structural block diagram of a target tracking device provided in an embodiment of the present application.
- FIG. 11 is a structural block diagram of an electronic device provided by an embodiment of the present application.
- a specific embodiment of the present application provides a target tracking method, including:
- the multiple pictures can be multiple consecutive frames of pictures in the same video stream, and the video stream can be collected by multiple cameras at different angles in real time, and the pictures collected by each camera include the characters and the surrounding environment of the characters, and Each camera corresponds to different human body angles of people. Therefore, by performing human body detection on pictures corresponding to different human body angles of each camera, and then determining the human body detection image corresponding to the human body area, the impact of human body occlusion on tracking can be reduced.
- step S10 includes:
- the Mahalanobis distance represents the distance between a point and a distribution, and the problem of inconsistent scales of each dimension can be eliminated through the Mahalanobis distance;
- Intersection-over-Union (IoU) represents two detection frames The overlapping part is divided by the result of the set of two detection frames. When the ratio is 1, it means that the two detection frames are completely overlapped.
- IoU Intersection-over-Union
- you can use The human body area in a single frame picture is detected, and corresponding detection frames are generated respectively. For example, if there are two people in a single frame picture, two detection frames can be generated correspondingly. By matching the two detection frames, two detection frames can be calculated.
- the intersection ratio between two detection frames; and, the corresponding detection point can be determined through the center point of the detection frame, and the corresponding features in the two detection frames can be extracted by using the feature extraction model, so that the two detection frames can be calculated by calculating The cosine distance of the features in each detection frame to obtain the feature similarity between the two detection frames; after calculating the intersection ratio and feature similarity between the two detection frames, due to the intersection ratio and feature similarity
- the measurement units between are different, so the Mahalanobis distance can be used to eliminate the influence of the different scales between the intersection ratio and the feature similarity, and then calculate the first weight edge, and the corresponding human body can be constructed through the detection point and the first weight edge Detect images.
- each detection point can be set as Vd (concentric circle point in the figure), and the first weight side can be set as Ed (thin line in the figure), where the position of Vd can be determined by the position of the detection frame Center point, E d can be obtained by the following formula:
- the feature similarity can be obtained by extracting features from the detection frame with a feature extraction model, and then calculating the cosine distance of the feature; D M represents the Mahalanobis distance, and calculates all
- step S20 includes:
- different cameras correspond to the same frame during synchronous acquisition, and the two detection points corresponding to the same frame in each camera are calculated separately to obtain the epipolar distance between the two detection points in each camera , and the preset threshold can be preset; after the human body detection image is constructed through the detection point and the first weighted edge, the detection point corresponding to each camera can be used to determine the corresponding epipolar line;
- the polar constraint means that two cameras have imaging planes in three-dimensional space, and the same detection point can be projected onto the two imaging planes to form two projection points, and the polar plane can be determined by connecting the two projection points with the detection point.
- the two intersecting lines between the plane and the two imaging planes are epipolar lines.
- the poles at the shortest distance can be determined on the two epipolar lines.
- the average value, and then the three-dimensional hypothetical point can be determined.
- the three-dimensional hypothetical point does not exist in the case of epipolar mis-matching or errors in the internal and external parameters of the camera; it can be understood that when the detection point in a certain camera When there are two, two 3D hypothetical points can be determined; when there are two detection points in both cameras, four 3D hypothetical points can be generated.
- the 3D hypothetical Two points can also be generated; it can be understood that a corresponding three-dimensional hypothetical point can be generated corresponding to each frame of pictures, so as to establish a connection between multiple cameras through the three-dimensional hypothetical point.
- step S21 includes:
- the second weight edge (dotted line in the figure) is represented as a weight edge between the detection point V d (concentric circle point in the figure) and the three-dimensional hypothetical point V h (circle point in the figure), which can be obtained through the above-mentioned Feature similarity, determine the feature similarity corresponding to each camera, and the feature similarity can correspond to the same frame of the two cameras, the epipolar distance between the two detection points in each camera obtained through the above calculation , and then the Mahalanobis distance can be used to eliminate the influence of different scales between the epipolar distance and the feature similarity, and then calculate the second weight edge.
- the second weight edge can be calculated using the following formula:
- D M represents the Mahalanobis distance. Since the distance and similarity belong to different measurement units, the Mahalanobis distance is used to eliminate the influence of different measurement units, and thus calculated Second weight edge.
- step S25 includes:
- the distance between the three-dimensional hypothetical points can be determined by the Mahalanobis distance
- the Euclidean distance and the above-mentioned first weight edge and the second weight edge are calculated to eliminate the influence of the different measurement units between the Euclidean distance, the first weight edge and the second weight edge, and thus the third weight edge can be determined .
- each detection point corresponds to each detection frame, and each detection frame represents a person, since each camera can generate a corresponding three-dimensional hypothetical point, but when there is an error in the internal and external parameters of the camera, then The generated 3D hypothetical points will have errors, so it is necessary to add a mutual exclusion relationship in the first path image to determine which camera generates 3D hypothetical points as wrong 3D hypothetical points; for example, there is detection point A in camera 1, camera 2 There are detection points A and B, detection point A in camera 1 and detection point A in camera 2 can generate a three-dimensional hypothetical point AA correspondingly, detection point A in camera 1 and detection point A in camera 2 can be generated correspondingly For a three-dimensional hypothetical point AB, it can be determined that the three-dimensional hypothetical point AA is correct, and at the same time it can be determined that the three-dimensional hypothetical point AB is wrong. In this way, the three-dimensional hypothetical point AB can be removed through mutually exclusive information.
- step S30 includes:
- the 3D hypothetical points can be generated At the same time, mutually exclusive edges are correspondingly added to generate corresponding 3D real points, and at the same time, it is judged whether the 3D hypothetical points are correct. Remove, so as to get the correct 3D hypothetical point, 3D real point and mutually exclusive edge; it can be understood that when judging whether the 3D hypothetical point generated by the two cameras is correct, it can be judged by the second weight edge or directly Perform feature matching on the detection points.
- the second weight edge contains the feature similarity between the two detection points, it can be determined by the feature similarity of the two detection points or whether the two detection points match. Whether the obtained 3D hypothetical points are correct, thereby avoiding errors in camera tracking and ensuring more accurate information fusion between multiple cameras.
- the preset algorithm is the minimum cost maximum flow algorithm
- the minimum cost maximum flow algorithm means that by selecting the path and allocating the flow passing through the path, the minimum cost can be achieved under the premise of the maximum flow
- the three-dimensional real point determined in the starting frame can be used as the starting path to start the calculation, and the three-dimensional real point determined from the ending frame can be used as the ending path to end the calculation; It is understood that since there are multiple paths in the second path image, the second path image is calculated through a preset algorithm, so that the resulting path can be determined as the tracking result, making the tracking less difficult and more accurate.
- the above-mentioned path calculation on the second path image according to the preset algorithm to obtain the result path includes performing calculations in the second path image according to the minimum cost maximum flow algorithm to determine the shortest path; determining the shortest path as the result path .
- the second path image G r includes the detection point V d , the three-dimensional hypothetical point
- V h the 3D real point V r , the first weight edge E d , the second weight edge E h1 , the third weight edge E h2 , and the mutually exclusive edge E r , by combining the 3D real point in the starting frame Calculated as the starting point of the path and will end the 3D real point in the frame Calculated as the end point of the path to determine the true points that make up the 3D to The shortest path between them is determined to determine the resultant path in the second path image, which makes the camera tracking less difficult and more accurate.
- the shortest distance can be determined from the figure
- the path is the 3D real point (the upper black point in the figure), the mutually exclusive edge (the dotted line in the upper figure), the 3D hypothetical point (the upper circle point in the figure), the third weight edge, and the 3D hypothetical point (the lower circle point in the figure ), mutually exclusive edges (dotted line at the bottom of the figure), three-dimensional real points (black dots at the bottom of the figure); since the calculation of the shortest path needs to be calculated using the minimum cost maximum flow algorithm, the shortest path is only used as an example description, without any limitation.
- the person is tracked according to the shortest path, which can improve the tracking accuracy of the person, and can reduce the impact of occlusion on the person tracking when multiple cameras are tracking and shooting. Information fusion among them is easier.
- the object tracking method provided by this application first determines the human body detection images corresponding to the human body regions in multiple pictures; and according to the correlation information between the multiple human body detection images, performs feature fusion on the multiple human body detection images to construct the first A path image; then add mutually exclusive information to the first path image to construct a second path image; perform path calculation on the second path image according to a preset algorithm to obtain a result path; finally determine the obtained result path as tracking
- information fusion between multiple cameras can be made simpler, and the accuracy of tracking can be improved.
- the embodiment of the present application provides a target tracking device 10, including:
- the first determination module 11 is used to determine the human body detection image corresponding to the human body area in multiple pictures;
- the fusion module 12 is used to perform feature fusion of multiple human detection images according to the association information between the multiple human detection images, so as to construct the first path image;
- a construction module 13 configured to add mutually exclusive information to the first path image to construct a second path image
- a calculation module 14 configured to perform path calculation on the second path image according to a preset algorithm to obtain a resultant path
- the second determining module 15 is configured to determine the obtained result path as the tracking result.
- the target tracking device 10 provided by the present application firstly determines the human body detection images corresponding to the human body regions in multiple pictures; The first path image; then add mutually exclusive information in the first path image to construct the second path image; perform path calculation on the second path image according to a preset algorithm to obtain a result path; finally determine the obtained result path as Tracking results; this can make the information fusion between multiple cameras easier and improve the accuracy of tracking.
- the target tracking device 10 provided in the specific embodiment of the present application is a device corresponding to the above-mentioned target tracking method, and all embodiments of the above-mentioned target tracking method are applicable to the target tracking device 10.
- the above-mentioned target tracking device 10 embodiment There are corresponding modules corresponding to the steps in the above target tracking method, which can achieve the same or similar beneficial effects. In order to avoid too much repetition, each module in the target tracking device 2 will not be described in detail here.
- the specific embodiment of the present application also provides an electronic device 20, including a memory 202, a processor 201, and a computer program stored in the memory 202 and operable on the processor 201.
- the processor 201 The steps of realizing the above-mentioned target tracking method when the computer program is executed.
- the processor 201 is used to call the computer program stored in the memory 202, and perform the following steps:
- feature fusion is performed on the multiple human detection images to construct the first path image
- the determination of the human body detection images corresponding to the human body regions in the multiple pictures performed by the processor 201 includes:
- the detection points and the first weighted edge are constructed to obtain a human body detection image.
- the process performed by the processor 201 to perform feature fusion of multiple human detection images according to the association information between the multiple human detection images, so as to construct the first path image includes:
- the three-dimensional hypothetical point and the first weight edge are calculated, and the calculation result is combined with the detection point, the three-dimensional hypothetical point and the first weight edge to construct the first path image.
- the processor 201 determines the epipolar distance between the detection points corresponding to the same frame of at least two cameras according to the human body detection image, it includes:
- the human body detection image determine the feature similarity corresponding to each camera
- the epipolar distance and feature similarity are calculated based on the Mahalanobis distance to determine the corresponding second weight edge.
- the processor 201 executes the calculation of the three-dimensional hypothetical point and the first weighted edge, and constructs the first path image by combining the calculation result with the detection point, the three-dimensional hypothetical point, and the first weighted edge, including:
- the detection point, the three-dimensional hypothetical point, the first weight edge, the second weight edge and the third weight edge are constructed to obtain the first path image.
- adding mutual exclusion information to the first path image to construct the second path image performed by the processor 201 includes:
- the correct 3D real point, mutually exclusive edge, detection point, 3D hypothetical point, first weighted edge, second weighted edge, and third weighted edge are constructed to obtain a second path image.
- the path calculation performed by the processor 201 on the second path image according to a preset algorithm to obtain a resultant path includes:
- the steps of the above-mentioned target tracking method can be implemented, thereby making the information fusion between multiple cameras simpler and improving the accuracy of tracking Spend
- processor 201 of the electronic device 20 executes the computer program to implement the steps of the above object tracking method, all embodiments of the above object tracking method are applicable to the electronic device 20, and can achieve the same or similar Beneficial effect.
- the computer-readable storage medium provided in the embodiment of the present application stores a computer program on the computer-readable storage medium.
- each of the target tracking method or the application-side target tracking method provided in the embodiment of the present application is implemented. process, and can achieve the same technical effect, in order to avoid repetition, it will not be repeated here.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Closed-Circuit Television Systems (AREA)
- Studio Devices (AREA)
Abstract
Disclosed in the present application are a target tracking method and apparatus, and an electronic device and a storage medium. The method comprises: determining human body detection images corresponding to human body regions in a plurality of pictures; according to association information between the plurality of human body detection images, performing feature fusion on the plurality of human body detection images so as to construct a first path image; adding mutual exclusion information to the first path image so as to construct a second path image; performing path calculation on the second path image according to a preset algorithm so as to obtain a result path; and determining the obtained result path as a tracking result. The present application may make information fusion between a plurality of cameras be simpler, thereby improving the accuracy of tracking.
Description
本申请要求于2021年12月31日提交中国专利局,申请号为202111674242.5、申请名称为“目标跟踪方法、装置、电子设备及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number 202111674242.5 and the application name "target tracking method, device, electronic equipment and storage medium" submitted to the China Patent Office on December 31, 2021, the entire contents of which are incorporated by reference in this application.
本申请涉及图像处理技术领域,具体涉及一种目标跟踪方法、装置、电子设备及存储介质。The present application relates to the technical field of image processing, and in particular to a target tracking method, device, electronic equipment and storage medium.
随着人工智能技术的发展与进步,人体跟踪技术已经大量应用于社会生活的各个方面,例如公共安全中人脸抓拍,商业场景中人脸建档等。但是现有的跟踪技术大多是在单相机下,基于人体检测进行跟踪,目前在单相机下对人体检测进行跟踪,一旦人体发生遮挡,人体检测就会失效,会对跟踪准确率造成极大影响,但是采用多个相机拍摄人体时,多个相机之间的信息融合存在极大的难度,导致跟踪的准确度不足。With the development and progress of artificial intelligence technology, human body tracking technology has been widely used in various aspects of social life, such as face capture in public security, face building in business scenarios, etc. However, most of the existing tracking technologies are based on human body detection under a single camera. At present, human body detection is tracked under a single camera. Once the human body is blocked, human body detection will fail, which will greatly affect the tracking accuracy. , but when multiple cameras are used to shoot the human body, it is extremely difficult to fuse information between multiple cameras, resulting in insufficient tracking accuracy.
申请内容application content
第一方面,本申请的主要目的是提供一种目标跟踪方法,包括:In the first aspect, the main purpose of this application is to provide a target tracking method, including:
确定多张图片中的人体区域对应的人体检测图像;Determining human body detection images corresponding to human body regions in multiple pictures;
根据所述多个人体检测图像之间的关联信息,将所述多个人体检测图像进行特征融合,以构建出第一路径图像;performing feature fusion on the plurality of human detection images according to correlation information between the plurality of human detection images, to construct a first path image;
在所述第一路径图像中添加互斥信息以构建出第二路径图像;adding mutually exclusive information to the first path image to construct a second path image;
根据预设算法对所述第二路径图像进行路径计算,以得到结果路径;performing path calculation on the second path image according to a preset algorithm to obtain a resultant path;
将得到的所述结果路径确定为跟踪结果。The obtained result path is determined as a trace result.
可选地,所述确定多张图片中的人体区域对应的人体检测图像包括:Optionally, the determining the human body detection images corresponding to the human body regions in the multiple pictures includes:
获取多个相机采集的图片;Obtain pictures collected by multiple cameras;
对每张图片中的人体区域进行人体检测,以生成对应的检测框;Perform human body detection on the human body area in each picture to generate a corresponding detection frame;
根据所述检测框确定出对应的检测点,并确定每张图片中至少两个检测框之间的交并比及特征相似度;Determine the corresponding detection point according to the detection frame, and determine the intersection ratio and feature similarity between at least two detection frames in each picture;
基于马氏距离对所述交并比和所述特征相似度进行计算,以确定出对应的第一权重边;Calculating the intersection ratio and the feature similarity based on the Mahalanobis distance to determine the corresponding first weight edge;
将所述检测点和所述第一权重边进行构建以得到所述人体检测图像。Constructing the detection points and the first weighted edge to obtain the human body detection image.
可选地,所述根据所述多个人体检测图像之间的关联信息,将所述多个人体检测图像进行特征融合,以构建出第一路径图像包括:Optionally, performing feature fusion on the plurality of human detection images according to association information between the plurality of human detection images to construct the first route image includes:
根据所述人体检测图像,确定至少两个相机的相同帧对应的检测点之间的极线距离;Determine the epipolar distance between detection points corresponding to the same frame of at least two cameras according to the human body detection image;
判断所述极线距离是否小于预设阈值;judging whether the epipolar distance is less than a preset threshold;
当所述极线距离小于预设阈值时,计算所述极线之间最近点的平均值;When the epipolar line distance is less than a preset threshold, calculate the average value of the closest points between the epipolar lines;
根据所述平均值确定出三维假设点;determining a three-dimensional hypothetical point according to the average value;
将所述三维假设点及所述第一权重边进行计算,并将计算结果与所述检测点、三维假设点及所述第一权重边构建得到所述第一路径图像。The 3D hypothetical point and the first weighted edge are calculated, and the calculation result is combined with the detection point, the 3D hypothetical point, and the first weighted edge to construct the first path image.
可选地,所述根据所述人体检测图像,确定至少两个相机的相同帧对应的检测点之间的极线距离之后,包括:Optionally, after determining the epipolar distance between the detection points corresponding to the same frame of at least two cameras according to the human body detection image, the method includes:
根据所述人体检测图像,确定每个相机对应的所述特征相似度;Determine the feature similarity corresponding to each camera according to the human body detection image;
基于马氏距离对所述极线距离和所述特征相似度进行计算,以确定出对应的第二权重边。The epipolar distance and the feature similarity are calculated based on the Mahalanobis distance to determine a corresponding second weighted edge.
可选地,所述将所述三维假设点及所述第一权重边进行计算,并将计算结果与所述检测点、三维假设点及所述第一权重边构建得到所述第一路径图像,包括:Optionally, the calculation is performed on the three-dimensional hypothetical point and the first weighted edge, and the calculation result is combined with the detection point, the three-dimensional hypothetical point and the first weighted edge to construct the first path image ,include:
确定至少两个相机中相同帧对应的三维假设点之间的欧式距离;determining a Euclidean distance between three-dimensional hypothetical points corresponding to the same frame in at least two cameras;
基于马氏距离对所述欧式距离、第一权重边、所述第二权重边进行计算,以确定出所述三维假设点对应的第三权重边;Calculate the Euclidean distance, the first weight edge, and the second weight edge based on the Mahalanobis distance to determine a third weight edge corresponding to the three-dimensional hypothetical point;
将所述检测点、三维假设点、第一权重边、第二权重边及所述第三权重边进行构建以得到所述第一路径图像。The detection point, the three-dimensional hypothetical point, the first weight edge, the second weight edge and the third weight edge are constructed to obtain the first path image.
可选地,所述在所述第一路径图像中添加互斥信息以构建出第二路径图像包括:Optionally, adding mutually exclusive information to the first path image to construct the second path image includes:
将所述三维假设点添加互斥边以生成对应的三维真实点;其中,所述三维假设点与所述三维真实点之间的三维信息相同且类别信息不同;Adding a mutually exclusive edge to the three-dimensional hypothetical point to generate a corresponding three-dimensional real point; wherein, the three-dimensional information between the three-dimensional hypothetical point and the three-dimensional real point is the same and the category information is different;
对所述三维假设点进行判断以确定出正确的三维真实点及互斥边;Judging the three-dimensional hypothetical point to determine the correct three-dimensional real point and mutually exclusive edge;
将所述正确的三维真实点、互斥边以及所述检测点、三维假设点、第一权重边、第二权重边、第三权重边进行构建以得到所述第二路径图像。The correct 3D real point, mutually exclusive edge, detection point, 3D hypothetical point, first weighted edge, second weighted edge, and third weighted edge are constructed to obtain the second path image.
可选地,所述根据预设算法对所述第二路径图像进行路径计算,以得到结果路径包括:Optionally, performing path calculation on the second path image according to a preset algorithm to obtain a resultant path includes:
将所述第二路径图像中按最小成本最大流算法进行计算,以确定出最短路径;calculating the second path image according to the minimum cost maximum flow algorithm to determine the shortest path;
将所述最短路径确定为结果路径。The shortest path is determined as the resulting path.
第三方面,本申请实施例提供了一种目标跟踪装置,包括:In a third aspect, the embodiment of the present application provides a target tracking device, including:
第一确定模块,用于确定多张图片中的人体区域对应的人体检测图像;A first determining module, configured to determine human body detection images corresponding to human body regions in multiple pictures;
融合模块,用于根据所述多个人体检测图像之间的关联信息,将所述多个人体检测图像进行特征融合,以构建出第一路径图像;A fusion module, configured to perform feature fusion of the plurality of human detection images according to the association information between the plurality of human detection images, so as to construct a first path image;
构建模块,用于在所述第一路径图像中添加互斥信息以构建出第二路径图像;A building module, configured to add mutually exclusive information to the first path image to construct a second path image;
计算模块,用于根据预设算法对所述第二路径图像进行路径计算,以得到结果路径;A calculation module, configured to perform path calculation on the second path image according to a preset algorithm to obtain a resultant path;
第二确定模块,用于将得到的所述结果路径确定为跟踪结果。The second determining module is configured to determine the obtained result path as a tracking result.
第三方面,本申请实施例提供了一种电子设备,包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序,所述处理器执行所述计算机程序时实现如上述的目标跟踪方法的步骤。In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, Steps for realizing the above-mentioned target tracking method.
第四方面,本申请实施例提供了一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,所述计算机程序被处理器执行时实现如上述的目标跟踪方法的步骤。In a fourth aspect, the embodiment of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned target tracking method are implemented.
本申请的上述方案至少包括以下有益效果:The above-mentioned scheme of the present application at least includes the following beneficial effects:
本申请提供的目标跟踪方法,首先确定多张图片中的人体区域对应的人体检测图像;并根据所述多个人体检测图像之间的关联信息,将所述多个人体检测图像进行特征融合,以构建出第一路径图像;然后在所述第一路径图像中添加互斥信息以构建出第二路径图像;根据预设算法对所述第二路径图 像进行路径计算,以得到结果路径;最后将得到的所述结果路径确定为跟踪结果;由此可以使得多个相机之间的信息融合更简单,提升了跟踪的准确度。The target tracking method provided in the present application firstly determines the human body detection images corresponding to the human body regions in multiple pictures; and performs feature fusion on the multiple human body detection images according to the correlation information between the multiple human body detection images, to construct a first path image; then add mutually exclusive information to the first path image to construct a second path image; perform path calculation on the second path image according to a preset algorithm to obtain a resultant path; finally The obtained result path is determined as a tracking result; thus, the information fusion between multiple cameras can be made simpler, and the tracking accuracy can be improved.
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图示出的结构获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to the structures shown in these drawings without creative effort.
图1为本申请实施例提供的目标跟踪方法的整体流程示意图;FIG. 1 is a schematic diagram of the overall flow of a target tracking method provided in an embodiment of the present application;
图2为本申请实施例提供的步骤S10的具体流程示意图;FIG. 2 is a schematic flowchart of step S10 provided in the embodiment of the present application;
图3为本申请实施例提供的目标跟踪方法的示例图;FIG. 3 is an example diagram of a target tracking method provided in an embodiment of the present application;
图4为本申请实施例提供的目标跟踪方法的另一示例图;FIG. 4 is another example diagram of the target tracking method provided by the embodiment of the present application;
图5为本申请实施例提供的目标跟踪方法的又一示例图;FIG. 5 is another example diagram of the target tracking method provided by the embodiment of the present application;
图6为本申请实施例提供的步骤S20的具体流程示意图;FIG. 6 is a schematic flowchart of step S20 provided in the embodiment of the present application;
图7为本申请实施例提供的步骤S21的另一流程示意图;FIG. 7 is another schematic flowchart of step S21 provided by the embodiment of the present application;
图8为本申请实施例提供的步骤S25的具体流程示意图;FIG. 8 is a schematic flowchart of step S25 provided in the embodiment of the present application;
图9为本申请实施例提供的步骤S30的具体流程示意图;FIG. 9 is a schematic flowchart of step S30 provided in the embodiment of the present application;
图10为本申请实施例提供的目标跟踪装置的结构框图;FIG. 10 is a structural block diagram of a target tracking device provided in an embodiment of the present application;
图11为本申请实施例提供的电子设备的结构框图。FIG. 11 is a structural block diagram of an electronic device provided by an embodiment of the present application.
如图1所示,本申请的具体实施例提供了一种目标跟踪方法,包括:As shown in Figure 1, a specific embodiment of the present application provides a target tracking method, including:
S10、确定多张图片中的人体区域对应的人体检测图像。S10. Determine human body detection images corresponding to human body regions in the multiple pictures.
其中,多张图片可以同一视频流中的多张连续帧图片,视频流可以是由多个相机在不同角度实时采集得到的,每个相机所采集的图片中包含人物以及人物周边的环境,并且每个相机对应有人物的不同人体角度,因此,通过将每个相机对应不同人体角度的图片进行人体检测,进而确定出与人体区域对应的人体检测图像,可以减少人体遮挡对跟踪的影响。Wherein, the multiple pictures can be multiple consecutive frames of pictures in the same video stream, and the video stream can be collected by multiple cameras at different angles in real time, and the pictures collected by each camera include the characters and the surrounding environment of the characters, and Each camera corresponds to different human body angles of people. Therefore, by performing human body detection on pictures corresponding to different human body angles of each camera, and then determining the human body detection image corresponding to the human body area, the impact of human body occlusion on tracking can be reduced.
如图2所示,上述步骤S10的具体实现方式包括:As shown in Figure 2, the specific implementation of the above step S10 includes:
S11、获取多个相机采集的图片;S11. Obtain pictures collected by multiple cameras;
S12、对每张图片中的人体区域进行人体检测,以生成对应的检测框;S12. Perform human body detection on the human body area in each picture to generate a corresponding detection frame;
S13、根据检测框确定出对应的检测点,并确定每张图片中至少两个检测框之间的交并比及特征相似度;S13. Determine the corresponding detection point according to the detection frame, and determine the intersection ratio and feature similarity between at least two detection frames in each picture;
S14、基于马氏距离对交并比和特征相似度进行计算,以确定出对应的第一权重边;S14. Calculate the intersection-union ratio and the feature similarity based on the Mahalanobis distance, so as to determine the corresponding first weight edge;
S15、将检测点和第一权重边进行构建以得到人体检测图像。S15. Construct the detection points and the first weighted edge to obtain a human detection image.
在本实施例中,马氏距离表示点与一个分布之间的距离,通过马氏距离可以消除各个维度尺度不一致的问题;交并比(Intersection-over-Union,IoU),表示两个检测框重叠的部分除以两个检测框的集合部分得出的结果,在比值为1的情况下,则表示两个检测框完全重叠,在对每个相机中对应的图片进行人体检测时,可以将单帧图片中的人体区域进行检测,并分别生成对应的检测框,例如单帧图片中有两个人物,则可以对应生成两个检测框,通过将两个检测框进行匹配,进而计算出两个检测框之间的交并比;并且,可以通过检测框的中心点确定出对应的检测点,以及采用特征提取模型分别提取出两个检测框中对应的特征,由此可以通过计算出两个检测框中特征的余弦距离,以得到两个检测框之间的特征相似度;在计算出两个检测框之间的交并比和特征相似度后,由于交并比和特征相似度之间的度量单位不同,因此可以通过马氏距离消除交并比和特征相似度之间尺度不同的影响,进而计算出第一权重边,通过检测点和第一权重边则可以构建出对应的人体检测图像。In this embodiment, the Mahalanobis distance represents the distance between a point and a distribution, and the problem of inconsistent scales of each dimension can be eliminated through the Mahalanobis distance; Intersection-over-Union (IoU) represents two detection frames The overlapping part is divided by the result of the set of two detection frames. When the ratio is 1, it means that the two detection frames are completely overlapped. When performing human body detection on the corresponding pictures in each camera, you can use The human body area in a single frame picture is detected, and corresponding detection frames are generated respectively. For example, if there are two people in a single frame picture, two detection frames can be generated correspondingly. By matching the two detection frames, two detection frames can be calculated. The intersection ratio between two detection frames; and, the corresponding detection point can be determined through the center point of the detection frame, and the corresponding features in the two detection frames can be extracted by using the feature extraction model, so that the two detection frames can be calculated by calculating The cosine distance of the features in each detection frame to obtain the feature similarity between the two detection frames; after calculating the intersection ratio and feature similarity between the two detection frames, due to the intersection ratio and feature similarity The measurement units between are different, so the Mahalanobis distance can be used to eliminate the influence of the different scales between the intersection ratio and the feature similarity, and then calculate the first weight edge, and the corresponding human body can be constructed through the detection point and the first weight edge Detect images.
如图3所示,可以每个检测点设定为V
d(图中同心圆点),将第一权重边设定为E
d(图中细线),其中V
d位置可以由检测框的中心点得到,E
d可以由下面公式得到:
As shown in Figure 3, each detection point can be set as Vd (concentric circle point in the figure), and the first weight side can be set as Ed (thin line in the figure), where the position of Vd can be determined by the position of the detection frame Center point, E d can be obtained by the following formula:
可以理解的是,
代表两个检测框的交并比大小;
Understandably, Represents the intersection and union ratio of the two detection frames;
代表两个检测框之间的特征相似度,特征相似度可以通过用特征提取模型对检测框进行提取特征,然后计算特征的余弦距离得到;D
M代表马氏距离,通过马氏距离计算出所有检测点和第一权重边后,即可以通过检测点和第一权重边构建出人体检测图像G
d,G
d=(V
d,E
d);并 且,每个相机可以单独构建其对应的人体检测图像。
Represents the feature similarity between two detection frames. The feature similarity can be obtained by extracting features from the detection frame with a feature extraction model, and then calculating the cosine distance of the feature; D M represents the Mahalanobis distance, and calculates all After detecting the point and the first weight edge, the human body detection image G d can be constructed through the detection point and the first weight edge, G d = (V d , E d ); and each camera can independently construct its corresponding human body Detect images.
S20、根据多个人体检测图像之间的关联信息,将多个人体检测图像进行特征融合,以构建出第一路径图像。S20. According to the association information between the multiple human detection images, perform feature fusion on the multiple human detection images to construct a first path image.
如图6所示,上述步骤S20具体实现方式包括:As shown in Figure 6, the specific implementation of the above step S20 includes:
S21、根据人体检测图像,确定至少两个相机的相同帧对应的检测点之间的极线距离;S21. Determine the epipolar distance between the detection points corresponding to the same frame of at least two cameras according to the human body detection image;
S22、判断极线距离是否小于预设阈值;S22. Determine whether the epipolar distance is smaller than a preset threshold;
S23、当极线距离小于预设阈值时,计算极线之间最近点的平均值;S23. When the epipolar line distance is less than the preset threshold, calculate the average value of the closest point between the epipolar lines;
S24、根据平均值确定出三维假设点;S24. Determine the three-dimensional hypothetical point according to the average value;
S25、将三维假设点及第一权重边进行计算,并将计算结果与检测点、三维假设点及第一权重边构建得到第一路径图像。S25. Calculate the three-dimensional hypothetical point and the first weighted edge, and construct the calculation result with the detection point, the three-dimensional hypothetical point, and the first weighted edge to obtain a first path image.
在本实施例中,不同相机同步采集时对应有相同帧,分别将每个相机中相同帧对应的两个检测点进行计算,即可得到每个相机中两个检测点之间的极线距离,并且,预设阈值可以是预先设定的;在通过检测点和第一权重边构建出人体检测图像后,可以将每个相机对应的检测点采用对极约束确定出对应的极线;对极约束表示在两个相机在三维空间中具有成像平面,对于同一检测点可以投影至两个成像平面上形成两个投影点,将两个投影点和检测点连接则可以确定出极平面,极平面和两个成像平面之间的两条相交线为极线,由于两条极线为假定相交状态,因此,在两条极线上的可以确定出最近距离的极点,通过计算极点之间的平均值,进而可以确定出三维假设点,当然,在极线误匹配或相机的内外参存在误差的情况下,该三维假设点则不存在;可以理解的是,当某一相机中的检测点有两个时,则三维假设点也可以确定出两个;当两个相机中的检测点均有两个时,则三维假设点可以生成四个,由于三维假设点可能不存在,因此三维假设点也可以生成两个;可以理解的是,可以对应每一帧图片生成对应的三维假设点,以通过三维假设点建立多个相机之间的联系。In this embodiment, different cameras correspond to the same frame during synchronous acquisition, and the two detection points corresponding to the same frame in each camera are calculated separately to obtain the epipolar distance between the two detection points in each camera , and the preset threshold can be preset; after the human body detection image is constructed through the detection point and the first weighted edge, the detection point corresponding to each camera can be used to determine the corresponding epipolar line; The polar constraint means that two cameras have imaging planes in three-dimensional space, and the same detection point can be projected onto the two imaging planes to form two projection points, and the polar plane can be determined by connecting the two projection points with the detection point. The two intersecting lines between the plane and the two imaging planes are epipolar lines. Since the two epipolar lines are assumed to intersect, the poles at the shortest distance can be determined on the two epipolar lines. By calculating the distance between the poles The average value, and then the three-dimensional hypothetical point can be determined. Of course, the three-dimensional hypothetical point does not exist in the case of epipolar mis-matching or errors in the internal and external parameters of the camera; it can be understood that when the detection point in a certain camera When there are two, two 3D hypothetical points can be determined; when there are two detection points in both cameras, four 3D hypothetical points can be generated. Since the 3D hypothetical points may not exist, the 3D hypothetical Two points can also be generated; it can be understood that a corresponding three-dimensional hypothetical point can be generated corresponding to each frame of pictures, so as to establish a connection between multiple cameras through the three-dimensional hypothetical point.
如图7所示,上述步骤S21之后包括:As shown in Figure 7, after the above step S21 includes:
S211、根据人体检测图像,确定每个相机的对应的特征相似度;S211. Determine the corresponding feature similarity of each camera according to the human body detection image;
S212、基于马氏距离对极线距离和特征相似度进行计算,以确定出对应的第二权重边。S212. Calculate the epipolar distance and feature similarity based on the Mahalanobis distance, so as to determine the corresponding second weight edge.
在本实施例中,第二权重边(图中虚线)表示为检测点V
d(图中同心圆点)和三维假设点V
h(图中圆点)之间的权重边,可以通过上述的特征相似度,确定出每个相机对应的特征相似度,并且该特征相似度可以与两个相机的同一帧相对应,通过上述计算得到的每个相机中两个检测点之间的极线距离,进而可以通过马氏距离消除极线距离和特征相似度之间尺度不同的影响,进而计算出第二权重边。
In this embodiment, the second weight edge (dotted line in the figure) is represented as a weight edge between the detection point V d (concentric circle point in the figure) and the three-dimensional hypothetical point V h (circle point in the figure), which can be obtained through the above-mentioned Feature similarity, determine the feature similarity corresponding to each camera, and the feature similarity can correspond to the same frame of the two cameras, the epipolar distance between the two detection points in each camera obtained through the above calculation , and then the Mahalanobis distance can be used to eliminate the influence of different scales between the epipolar distance and the feature similarity, and then calculate the second weight edge.
可选地,第二权重边可以采用以下公式计算得到:Optionally, the second weight edge can be calculated using the following formula:
其中,
代表不同相机同一帧的两个检测点的极线距离;
in, Represents the epipolar distance of two detection points in the same frame of different cameras;
代表每个相机同一帧的两个检测框的特征相似度;D
M表示马氏距离,由于距离和相似度属于不同度量单位,所以使用马氏距离来消除度量单位不同的影响,由此计算出第二权重边。
Represents the feature similarity of two detection frames in the same frame of each camera; D M represents the Mahalanobis distance. Since the distance and similarity belong to different measurement units, the Mahalanobis distance is used to eliminate the influence of different measurement units, and thus calculated Second weight edge.
如图8所示,上述步骤S25的具体实现方式包括:As shown in Figure 8, the specific implementation of the above step S25 includes:
S251、确定至少两个相机中相同帧对应的三维假设点之间的欧式距离;S251. Determine the Euclidean distance between the three-dimensional hypothetical points corresponding to the same frame in at least two cameras;
S252、基于马氏距离对欧式距离、第一权重边、第二权重边进行计算,以确定出三维假设点对应的第三权重边;S252. Calculate the Euclidean distance, the first weight edge, and the second weight edge based on the Mahalanobis distance, so as to determine the third weight edge corresponding to the three-dimensional hypothetical point;
S253、将检测点、三维假设点、第一权重边、第二权重边及第三权重边进行构建以构建出第一路径图像。S253. Construct the detection point, the three-dimensional hypothetical point, the first weighted edge, the second weighted edge, and the third weighted edge to construct a first path image.
在本实施例中,在两个相机确定得到的三维假设点后,通过计算两个相机同一帧中对应的三维假设点之间的欧式距离,由此可以通过马氏距离对三维假设点之间的欧式距离以及上述的第一权重边、第二权重边进行计算,以消除欧式距离、第一权重边及第二权重边之间的度量单元不同的影响,由此可以确定出第三权重边。In this embodiment, after the two cameras determine the obtained three-dimensional hypothetical points, by calculating the Euclidean distance between the corresponding three-dimensional hypothetical points in the same frame of the two cameras, the distance between the three-dimensional hypothetical points can be determined by the Mahalanobis distance The Euclidean distance and the above-mentioned first weight edge and the second weight edge are calculated to eliminate the influence of the different measurement units between the Euclidean distance, the first weight edge and the second weight edge, and thus the third weight edge can be determined .
如图4所示,第三权重边E
h2(图中粗线)生成方式:
As shown in Figure 4, the generation method of the third weight edge E h2 (thick line in the figure):
其中,
代表每帧图片中两个圆点之间的欧氏距离;
in, Represents the Euclidean distance between two dots in each frame of the picture;
表示与圆点1相连的两条E
h1虚线;
表示与圆点2相连的两条E
h1虚线;
表示与圆点1和圆点2间接相连的两条E
d细线;可以理解的是,通过马氏距离消除所有度量单位不同的影响,因此在计算第三权重边E
h2时,需要通过第一权重边
和第二权重边E
h1 综合计算,以得到第三权重边E
h2。
Indicates the two dashed E h1 lines connected to dot 1; Indicates the two dashed E h1 lines connected to dot 2; Indicates two E d thin lines indirectly connected to dot 1 and dot 2; it can be understood that the Mahalanobis distance is used to eliminate the influence of different measurement units, so when calculating the third weight edge E h2 , it is necessary to pass the first a weighted edge and the second weight edge E h1 to obtain the third weight edge E h2 .
S30、在第一路径图像中添加互斥信息以构建出第二路径图像。S30. Add mutually exclusive information to the first path image to construct a second path image.
在本实施例中,每个检测点与每个检测框对应,并且每个检测框代表一个人,由于每个相机可以生成与其对应的三维假设点,但由于相机的内外参存在误差时,则生成的三维假设点会存在错误,因此需要在第一路径图像中添加互斥关系,以确定出哪个相机生成的三维假设点为错误的三维假设点;例如,相机1中存在检测点A,相机2存在检测点A和B,相机1中的检测点A和相机2中的检测点A可以对应生成一个三维假设点AA,相机1中的检测点A和相机2中的检测点A可以对应生成一个三维假设点AB,则可以确定出三维假设点AA为正确的,同时可以确定出三维假设点AB为错误的,如此,则可以通过互斥信息将三维假设点AB去除。In this embodiment, each detection point corresponds to each detection frame, and each detection frame represents a person, since each camera can generate a corresponding three-dimensional hypothetical point, but when there is an error in the internal and external parameters of the camera, then The generated 3D hypothetical points will have errors, so it is necessary to add a mutual exclusion relationship in the first path image to determine which camera generates 3D hypothetical points as wrong 3D hypothetical points; for example, there is detection point A in camera 1, camera 2 There are detection points A and B, detection point A in camera 1 and detection point A in camera 2 can generate a three-dimensional hypothetical point AA correspondingly, detection point A in camera 1 and detection point A in camera 2 can be generated correspondingly For a three-dimensional hypothetical point AB, it can be determined that the three-dimensional hypothetical point AA is correct, and at the same time it can be determined that the three-dimensional hypothetical point AB is wrong. In this way, the three-dimensional hypothetical point AB can be removed through mutually exclusive information.
如图9所示,上述步骤S30的具体实现方式包括:As shown in Figure 9, the specific implementation of the above step S30 includes:
S31、将三维假设点添加互斥边以生成对应的三维真实点;其中,三维假设点与三维真实点之间的三维信息相同且类别信息不同;S31. Add mutually exclusive edges to the three-dimensional hypothetical points to generate corresponding three-dimensional real points; wherein, the three-dimensional information between the three-dimensional hypothetical points and the three-dimensional real points is the same and the category information is different;
S32、对三维假设点进行判断以确定出正确的三维真实点及互斥边;S32. Judging the three-dimensional hypothetical points to determine the correct three-dimensional real points and mutually exclusive edges;
S33、将正确的三维真实点、互斥边以及检测点、三维假设点、第一权重边、第二权重边、第三权重边进行构建以得到第二路径图像。S33. Construct correct 3D real points, mutually exclusive edges, detection points, 3D hypothetical points, first weighted edges, second weighted edges, and third weighted edges to obtain a second path image.
其中,在确定出多个相机之间的三维假设点后,由于三维假设点为可能存在或可能不存在的关系,因此,为确定所生成的三维假设点是否正确,可以在生成三维假设点的同时对应添加互斥边以生成对应的三维真实点,同时判断该三维假设点是否正确,当确定出正确的三维假设点后,则可以将互斥边及其对应的三维假设点、三维真实点去除,从而得到正确的三维假设点、三维真实点及互斥边;可以理解的是,在判断两个相机中所生成的三维假设点是否正确时,可以通过第二权重边进行判断也可以直接将检测点进行特征匹配,由于第二权重边中包含有两个检测点之间的特征相似度,由此,可以由两个检测点的特征相似度或两个检测点之间是否匹配以确定出得到的三维假设点是否正确,从而避免了相机跟踪出现错误,确保多个相机之间的信息融合更为准确。Wherein, after determining the 3D hypothetical points between multiple cameras, since the 3D hypothetical points may or may not exist, in order to determine whether the generated 3D hypothetical points are correct, the 3D hypothetical points can be generated At the same time, mutually exclusive edges are correspondingly added to generate corresponding 3D real points, and at the same time, it is judged whether the 3D hypothetical points are correct. Remove, so as to get the correct 3D hypothetical point, 3D real point and mutually exclusive edge; it can be understood that when judging whether the 3D hypothetical point generated by the two cameras is correct, it can be judged by the second weight edge or directly Perform feature matching on the detection points. Since the second weight edge contains the feature similarity between the two detection points, it can be determined by the feature similarity of the two detection points or whether the two detection points match. Whether the obtained 3D hypothetical points are correct, thereby avoiding errors in camera tracking and ensuring more accurate information fusion between multiple cameras.
如图5所示,可以在假设人体图G
h的基础上,加入互斥边以生成第二路径图像;其中,三维点V
r(图中黑点),与三维假设点V
h(图中 圆点)具有相同的三维位置;当第一路径图像中一个点
生成两个假设人体点
通过加入互斥边
(图中点划线),则可以同时可以生成真实人体点
由于每个检测点只能代表一个人,所以
为一组互斥边,当其中一条互斥边对应的三维假设点正确时,则表示该互斥边存在,同理另外一条互斥边则必然不存在,由此,在确定出三维真实点V
r和互斥边E
r后,则可以构建出第二路径图像G
r=(V
d,V
h,V
r,E
d,E
h1,E
h2,E
r)。
As shown in Figure 5, on the basis of the hypothetical human body graph G h , mutually exclusive edges can be added to generate the second path image; where, the three-dimensional point V r (black point in the figure) and the three-dimensional hypothetical point V h (in the figure dot) have the same three-dimensional position; when a point in the first path image Generate two hypothetical body points By joining the mutually exclusive edge (dotted line in the figure), then real human body points can be generated at the same time Since each detection point can only represent one person, so is a group of mutually exclusive edges, when the 3D hypothetical point corresponding to one of the mutually exclusive edges is correct, it means that the mutually exclusive edge exists, and the other mutually exclusive edge must not exist in the same way, thus, when determining the 3D real point After V r and the mutually exclusive edge E r , the second path image G r =(V d , V h , V r , E d , E h1 , E h2 , E r ) can be constructed.
S40、根据预设算法对第二路径图像进行路径计算,以得到结果路径。S40. Perform path calculation on the second path image according to a preset algorithm to obtain a resultant path.
在本实施例中,预设算法为最小成本最大流算法,最小成本最大流算法表示通过选择路径、分配经过路径的流量,可以在流量最大的前提下,达到所用的费用最小的要求;在对第二路径图像中的路径进行计算时,可以由起始帧中确定得到的三维真实点作为起始路径以开始计算,并从结束帧中确定得到的三维真实点作为结束路径以结束计算;可以理解的是,由于在第二路径图像中存在多个路径,通过预设算法对第二路径图像进行计算,从而可以确定出结果路径以作为跟踪结果,使得跟踪难度更低且准确度更高。In this embodiment, the preset algorithm is the minimum cost maximum flow algorithm, and the minimum cost maximum flow algorithm means that by selecting the path and allocating the flow passing through the path, the minimum cost can be achieved under the premise of the maximum flow; When the path in the second path image is calculated, the three-dimensional real point determined in the starting frame can be used as the starting path to start the calculation, and the three-dimensional real point determined from the ending frame can be used as the ending path to end the calculation; It is understood that since there are multiple paths in the second path image, the second path image is calculated through a preset algorithm, so that the resulting path can be determined as the tracking result, making the tracking less difficult and more accurate.
具体的,上述根据预设算法对第二路径图像进行路径计算,以得到结果路径包括将第二路径图像中按最小成本最大流算法进行计算,以确定出最短路径;将最短路径确定为结果路径。Specifically, the above-mentioned path calculation on the second path image according to the preset algorithm to obtain the result path includes performing calculations in the second path image according to the minimum cost maximum flow algorithm to determine the shortest path; determining the shortest path as the result path .
其中,在第二路径图像G
r中包括有检测点V
d,三维假设点
Among them, the second path image G r includes the detection point V d , the three-dimensional hypothetical point
V
h,三维真实点V
r,第一权重边E
d,第二权重边E
h1,第三权重边E
h2,互斥边E
r,通过将起始帧中的三维真实点
作为路径的起始点计算,并将结束帧中的三维真实点
作为路径的结束点计算,以确定出构成三维真实点
至
之间的最短路径,从而确定出第二路径图像中的结果路径,使得相机的跟踪难度更低且准确率更高。
V h , the 3D real point V r , the first weight edge E d , the second weight edge E h1 , the third weight edge E h2 , and the mutually exclusive edge E r , by combining the 3D real point in the starting frame Calculated as the starting point of the path and will end the 3D real point in the frame Calculated as the end point of the path to determine the true points that make up the 3D to The shortest path between them is determined to determine the resultant path in the second path image, which makes the camera tracking less difficult and more accurate.
例如,在图5中所示,在第一帧所得到的三维真实点(图中上黑点)至第二帧对应的三维真实点(图中下黑点),从图中可以确定出最短路径为三维真实点(图中上黑点)、互斥边(图中上点划线)、三维假设点(图中上圆点)、第三权重边、三维假设点(图中下圆点)、互斥边(图中下点划线)、三维真实点(图中下黑点);由于在计算最短路径时需要采用最小成本最大 流算法进行计算,因此该最短路径仅仅作为一个示例进行说明,在此不作任何限定。For example, as shown in Figure 5, from the 3D real point obtained in the first frame (the upper black point in the figure) to the corresponding 3D real point in the second frame (the lower black point in the figure), the shortest distance can be determined from the figure The path is the 3D real point (the upper black point in the figure), the mutually exclusive edge (the dotted line in the upper figure), the 3D hypothetical point (the upper circle point in the figure), the third weight edge, and the 3D hypothetical point (the lower circle point in the figure ), mutually exclusive edges (dotted line at the bottom of the figure), three-dimensional real points (black dots at the bottom of the figure); since the calculation of the shortest path needs to be calculated using the minimum cost maximum flow algorithm, the shortest path is only used as an example description, without any limitation.
S50、将得到的结果路径确定为跟踪结果。S50. Determine the obtained result path as a tracking result.
其中,在确定出最短路径后,根据该最短路径对人物进行跟踪,可以提升人物的跟踪准确率,并且能够使得多个相机跟踪拍摄时,减小了遮挡对人物跟踪的影响,多个相机之间的信息融合更简单。Among them, after the shortest path is determined, the person is tracked according to the shortest path, which can improve the tracking accuracy of the person, and can reduce the impact of occlusion on the person tracking when multiple cameras are tracking and shooting. Information fusion among them is easier.
本申请提供的目标跟踪方法,首先确定多张图片中的人体区域对应的人体检测图像;并根据多个人体检测图像之间的关联信息,将多个人体检测图像进行特征融合,以构建出第一路径图像;然后在第一路径图像中添加互斥信息以构建出第二路径图像;根据预设算法对第二路径图像进行路径计算,以得到结果路径;最后将得到的结果路径确定为跟踪结果;由此可以使得多个相机之间的信息融合更简单,提升了跟踪的准确度。The object tracking method provided by this application first determines the human body detection images corresponding to the human body regions in multiple pictures; and according to the correlation information between the multiple human body detection images, performs feature fusion on the multiple human body detection images to construct the first A path image; then add mutually exclusive information to the first path image to construct a second path image; perform path calculation on the second path image according to a preset algorithm to obtain a result path; finally determine the obtained result path as tracking As a result, information fusion between multiple cameras can be made simpler, and the accuracy of tracking can be improved.
如图10所示,本申请实施例提供了一种目标跟踪装置10,包括:As shown in Figure 10, the embodiment of the present application provides a target tracking device 10, including:
第一确定模块11,用于确定多张图片中的人体区域对应的人体检测图像;The first determination module 11 is used to determine the human body detection image corresponding to the human body area in multiple pictures;
融合模块12,用于根据多个人体检测图像之间的关联信息,将多个人体检测图像进行特征融合,以构建出第一路径图像;The fusion module 12 is used to perform feature fusion of multiple human detection images according to the association information between the multiple human detection images, so as to construct the first path image;
构建模块13,用于在第一路径图像中添加互斥信息以构建出第二路径图像;A construction module 13, configured to add mutually exclusive information to the first path image to construct a second path image;
计算模块14,用于根据预设算法对第二路径图像进行路径计算,以得到结果路径;A calculation module 14, configured to perform path calculation on the second path image according to a preset algorithm to obtain a resultant path;
第二确定模块15,用于将得到的结果路径确定为跟踪结果。The second determining module 15 is configured to determine the obtained result path as the tracking result.
本申请提供的目标跟踪装置10,首先确定多张图片中的人体区域对应的人体检测图像;并根据多个人体检测图像之间的关联信息,将多个人体检测图像进行特征融合,以构建出第一路径图像;然后在第一路径图像中添加互斥信息以构建出第二路径图像;根据预设算法对第二路径图像进行路径计算,以得到结果路径;最后将得到的结果路径确定为跟踪结果;由此可以使得多个相机之间的信息融合更简单,提升了跟踪的准确度。The target tracking device 10 provided by the present application firstly determines the human body detection images corresponding to the human body regions in multiple pictures; The first path image; then add mutually exclusive information in the first path image to construct the second path image; perform path calculation on the second path image according to a preset algorithm to obtain a result path; finally determine the obtained result path as Tracking results; this can make the information fusion between multiple cameras easier and improve the accuracy of tracking.
需要说明的是,本申请具体实施例提供的目标跟踪装置10为与上述目标跟踪方法对应的装置,上述目标跟踪方法的所有实施例均适用于该目标跟踪装置10,上述目标跟踪装置10实施例中均有相应的模块对应上述目标跟踪 方法中的步骤,能达到相同或相似的有益效果,为避免过多重复,在此不对目标跟踪装置2中的每一模块进行过多赘述。It should be noted that the target tracking device 10 provided in the specific embodiment of the present application is a device corresponding to the above-mentioned target tracking method, and all embodiments of the above-mentioned target tracking method are applicable to the target tracking device 10. The above-mentioned target tracking device 10 embodiment There are corresponding modules corresponding to the steps in the above target tracking method, which can achieve the same or similar beneficial effects. In order to avoid too much repetition, each module in the target tracking device 2 will not be described in detail here.
如图11所示,本申请的具体实施例还提供了一种电子设备20,包括存储器202、处理器201以及存储在存储器202中并可在处理器201上运行的计算机程序,该处理器201执行计算机程序时实现上述的目标跟踪方法的步骤。As shown in FIG. 11 , the specific embodiment of the present application also provides an electronic device 20, including a memory 202, a processor 201, and a computer program stored in the memory 202 and operable on the processor 201. The processor 201 The steps of realizing the above-mentioned target tracking method when the computer program is executed.
具体的,处理器201用于调用存储器202存储的计算机程序,执行如下步骤:Specifically, the processor 201 is used to call the computer program stored in the memory 202, and perform the following steps:
确定多张图片中的人体区域对应的人体检测图像;Determining human body detection images corresponding to human body regions in multiple pictures;
根据多个人体检测图像之间的关联信息,将多个人体检测图像进行特征融合,以构建出第一路径图像;According to the association information between the multiple human detection images, feature fusion is performed on the multiple human detection images to construct the first path image;
在第一路径图像中添加互斥信息以构建出第二路径图像;Adding mutually exclusive information to the first path image to construct a second path image;
根据预设算法对第二路径图像进行路径计算,以得到结果路径;performing path calculation on the second path image according to a preset algorithm to obtain a resultant path;
将得到的结果路径确定为跟踪结果。Determine the resulting path as a trace result.
可选的,处理器201执行的确定多张图片中的人体区域对应的人体检测图像包括:Optionally, the determination of the human body detection images corresponding to the human body regions in the multiple pictures performed by the processor 201 includes:
获取多个相机采集的图片;Obtain pictures collected by multiple cameras;
对每张图片中的人体区域进行人体检测,以生成对应的检测框;Perform human body detection on the human body area in each picture to generate a corresponding detection frame;
根据检测框确定出对应的检测点,并确定每张图片中至少两个检测框之间的交并比及特征相似度;Determine the corresponding detection points according to the detection frame, and determine the intersection ratio and feature similarity between at least two detection frames in each picture;
基于马氏距离对交并比和特征相似度进行计算,以确定出对应的第一权重边;Calculate the intersection ratio and feature similarity based on the Mahalanobis distance to determine the corresponding first weight edge;
将检测点和第一权重边进行构建以得到人体检测图像。The detection points and the first weighted edge are constructed to obtain a human body detection image.
可选的,处理器201执行的根据多个人体检测图像之间的关联信息,将多个人体检测图像进行特征融合,以构建出第一路径图像包括:Optionally, the process performed by the processor 201 to perform feature fusion of multiple human detection images according to the association information between the multiple human detection images, so as to construct the first path image includes:
根据人体检测图像,确定至少两个相机的相同帧对应的检测点之间的极线距离;Determine the epipolar distance between detection points corresponding to the same frame of at least two cameras according to the human detection image;
判断极线距离是否小于预设阈值;Judging whether the epipolar distance is less than a preset threshold;
当极线距离小于预设阈值时,计算极线之间最近点的平均值;When the epipolar line distance is less than the preset threshold, calculate the average value of the nearest points between the epipolar lines;
根据平均值确定出三维假设点;Determine the three-dimensional hypothetical point according to the average value;
将三维假设点及第一权重边进行计算,并将计算结果与检测点、三维假设点及第一权重边构建得到第一路径图像。The three-dimensional hypothetical point and the first weight edge are calculated, and the calculation result is combined with the detection point, the three-dimensional hypothetical point and the first weight edge to construct the first path image.
可选的,处理器201执行的根据人体检测图像,确定至少两个相机的相同帧对应的检测点之间的极线距离之后,包括:Optionally, after the processor 201 determines the epipolar distance between the detection points corresponding to the same frame of at least two cameras according to the human body detection image, it includes:
根据人体检测图像,确定每个相机对应的特征相似度;According to the human body detection image, determine the feature similarity corresponding to each camera;
基于马氏距离对极线距离和特征相似度进行计算,以确定出对应的第二权重边。The epipolar distance and feature similarity are calculated based on the Mahalanobis distance to determine the corresponding second weight edge.
可选的,处理器201执行的将三维假设点及第一权重边进行计算,并将计算结果与检测点、三维假设点及第一权重边构建得到第一路径图像,包括:Optionally, the processor 201 executes the calculation of the three-dimensional hypothetical point and the first weighted edge, and constructs the first path image by combining the calculation result with the detection point, the three-dimensional hypothetical point, and the first weighted edge, including:
确定至少两个相机中相同帧对应的三维假设点之间的欧式距离;determining a Euclidean distance between three-dimensional hypothetical points corresponding to the same frame in at least two cameras;
基于马氏距离对欧式距离、第一权重边、第二权重边进行计算,以确定出三维假设点对应的第三权重边;Calculate the Euclidean distance, the first weight edge, and the second weight edge based on the Mahalanobis distance to determine the third weight edge corresponding to the three-dimensional hypothetical point;
将检测点、三维假设点、第一权重边、第二权重边及第三权重边进行构建以得到第一路径图像。The detection point, the three-dimensional hypothetical point, the first weight edge, the second weight edge and the third weight edge are constructed to obtain the first path image.
可选的,处理器201执行的在第一路径图像中添加互斥信息以构建出第二路径图像包括:Optionally, adding mutual exclusion information to the first path image to construct the second path image performed by the processor 201 includes:
将三维假设点添加互斥边以生成对应的三维真实点;其中,三维假设点与三维真实点之间的三维信息相同且类别信息不同;Adding mutually exclusive edges to the three-dimensional hypothetical points to generate corresponding three-dimensional real points; wherein, the three-dimensional information between the three-dimensional hypothetical points and the three-dimensional real points is the same and the category information is different;
对三维假设点进行判断以确定出正确的三维真实点及互斥边;Judging the three-dimensional hypothetical points to determine the correct three-dimensional real points and mutually exclusive edges;
将正确的三维真实点、互斥边以及检测点、三维假设点、第一权重边、第二权重边、第三权重边进行构建以得到第二路径图像。The correct 3D real point, mutually exclusive edge, detection point, 3D hypothetical point, first weighted edge, second weighted edge, and third weighted edge are constructed to obtain a second path image.
可选的,处理器201执行的根据预设算法对第二路径图像进行路径计算,以得到结果路径包括:Optionally, the path calculation performed by the processor 201 on the second path image according to a preset algorithm to obtain a resultant path includes:
将第二路径图像中按最小成本最大流算法进行计算,以确定出最短路径;Calculate the second path image according to the minimum cost maximum flow algorithm to determine the shortest path;
将最短路径确定为结果路径。Determine the shortest path as the resulting path.
即,在本申请的具体实施例中,电子设备20的处理器201执行计算机程序时实现上述目标跟踪方法的步骤,由此可以使得多个相机之间的信息融合更简单,提升了跟踪的准确度That is, in a specific embodiment of the present application, when the processor 201 of the electronic device 20 executes the computer program, the steps of the above-mentioned target tracking method can be implemented, thereby making the information fusion between multiple cameras simpler and improving the accuracy of tracking Spend
需要说明的是,由于电子设备20的处理器201执行计算机程序时实现上述目标跟踪方法的步骤,因此上述目标跟踪方法的所有实施例均适用于该电子 设备20,且均能达到相同或相似的有益效果。It should be noted that, since the processor 201 of the electronic device 20 executes the computer program to implement the steps of the above object tracking method, all embodiments of the above object tracking method are applicable to the electronic device 20, and can achieve the same or similar Beneficial effect.
本申请实施例中提供的计算机可读存储介质,计算机可读存储介质上存储有计算机程序,该计算机程序被处理器执行时实现本申请实施例提供的目标跟踪方法或应用端目标跟踪方法的各个过程,且能达到相同的技术效果,为避免重复,这里不再赘述。The computer-readable storage medium provided in the embodiment of the present application stores a computer program on the computer-readable storage medium. When the computer program is executed by the processor, each of the target tracking method or the application-side target tracking method provided in the embodiment of the present application is implemented. process, and can achieve the same technical effect, in order to avoid repetition, it will not be repeated here.
Claims (10)
- 一种目标跟踪方法,其中,包括:A target tracking method, comprising:确定多张图片中的人体区域对应的人体检测图像;Determining human body detection images corresponding to human body regions in multiple pictures;根据所述多个人体检测图像之间的关联信息,将所述多个人体检测图像进行特征融合,以构建出第一路径图像;performing feature fusion on the plurality of human detection images according to correlation information between the plurality of human detection images, to construct a first path image;在所述第一路径图像中添加互斥信息以构建出第二路径图像;adding mutually exclusive information to the first path image to construct a second path image;根据预设算法对所述第二路径图像进行路径计算,以得到结果路径;performing path calculation on the second path image according to a preset algorithm to obtain a resultant path;将得到的所述结果路径确定为跟踪结果。The obtained result path is determined as a trace result.
- 根据权利要求1所述的目标跟踪方法,其中,所述确定多张图片中的人体区域对应的人体检测图像包括:The target tracking method according to claim 1, wherein said determining the human body detection image corresponding to the human body area in the plurality of pictures comprises:获取多个相机采集的图片;Obtain pictures collected by multiple cameras;对每张图片中的人体区域进行人体检测,以生成对应的检测框;Perform human body detection on the human body area in each picture to generate a corresponding detection frame;根据所述检测框确定出对应的检测点,并确定每张图片中至少两个检测框之间的交并比及特征相似度;Determine the corresponding detection point according to the detection frame, and determine the intersection ratio and feature similarity between at least two detection frames in each picture;基于马氏距离对所述交并比和所述特征相似度进行计算,以确定出对应的第一权重边;Calculating the intersection ratio and the feature similarity based on the Mahalanobis distance to determine the corresponding first weight edge;将所述检测点和所述第一权重边进行构建以得到所述人体检测图像。Constructing the detection points and the first weighted edge to obtain the human body detection image.
- 根据权利要求2所述的目标跟踪方法,其中,所述根据所述多个人体检测图像之间的关联信息,将所述多个人体检测图像进行特征融合,以构建出第一路径图像包括:The target tracking method according to claim 2, wherein said performing feature fusion of said plurality of human detection images according to association information between said plurality of human detection images to construct a first path image comprises:根据所述人体检测图像,确定至少两个相机的相同帧对应的检测点之间的极线距离;Determine the epipolar distance between detection points corresponding to the same frame of at least two cameras according to the human body detection image;判断所述极线距离是否小于预设阈值;judging whether the epipolar distance is less than a preset threshold;当所述极线距离小于所述预设阈值时,计算所述极线之间最近点的平均值;When the epipolar line distance is less than the preset threshold, calculate the average value of the closest points between the epipolar lines;根据所述平均值确定出三维假设点;determining a three-dimensional hypothetical point according to the average value;将所述三维假设点及所述第一权重边进行计算,并将计算结果与所述检测点、三维假设点及所述第一权重边构建得到所述第一路径图像。The 3D hypothetical point and the first weighted edge are calculated, and the calculation result is combined with the detection point, the 3D hypothetical point, and the first weighted edge to construct the first path image.
- 根据权利要求3所述的目标跟踪方法,其中,所述根据所述人体检测图像,确定至少两个相机的相同帧对应的检测点之间的极线距离之后,包括:The target tracking method according to claim 3, wherein, after determining the epipolar distance between detection points corresponding to the same frame of at least two cameras according to the human body detection image, comprising:根据所述人体检测图像,确定每个相机对应的所述特征相似度;Determine the feature similarity corresponding to each camera according to the human body detection image;基于马氏距离对所述极线距离和所述特征相似度进行计算,以确定出对应的第二权重边。The epipolar distance and the feature similarity are calculated based on the Mahalanobis distance to determine a corresponding second weighted edge.
- 根据权利要求4所述的目标跟踪方法,其中,所述将所述三维假设点及所述第一权重边进行计算处理,并将计算结果与所述检测点、三维假设点及所述第一权重边构建得到所述第一路径图像,包括:The target tracking method according to claim 4, wherein the calculation process is performed on the three-dimensional hypothetical point and the first weight edge, and the calculation result is combined with the detection point, the three-dimensional hypothetical point and the first weighted edge. The weight edge is constructed to obtain the first path image, including:确定至少两个相机中相同帧对应的三维假设点之间的欧式距离;determining a Euclidean distance between three-dimensional hypothetical points corresponding to the same frame in at least two cameras;基于马氏距离对所述欧式距离、第一权重边、所述第二权重边进行计算,以确定出所述三维假设点对应的第三权重边;Calculate the Euclidean distance, the first weight edge, and the second weight edge based on the Mahalanobis distance to determine a third weight edge corresponding to the three-dimensional hypothetical point;将所述检测点、三维假设点、第一权重边、第二权重边及所述第三权重边进行构建以得到所述第一路径图像。The detection point, the three-dimensional hypothetical point, the first weight edge, the second weight edge and the third weight edge are constructed to obtain the first path image.
- 根据权利要求5所述的目标跟踪方法,其中,所述在所述第一路径图像中添加互斥信息以构建出第二路径图像包括:The target tracking method according to claim 5, wherein said adding mutually exclusive information to said first path image to construct a second path image comprises:将所述三维假设点添加互斥边以生成对应的三维真实点;其中,所述三维假设点与所述三维真实点之间的三维信息相同且类别信息不同;Adding a mutually exclusive edge to the three-dimensional hypothetical point to generate a corresponding three-dimensional real point; wherein, the three-dimensional information between the three-dimensional hypothetical point and the three-dimensional real point is the same and the category information is different;对所述三维假设点进行判断以确定出正确的三维真实点及互斥边;Judging the three-dimensional hypothetical point to determine the correct three-dimensional real point and mutually exclusive edge;将所述正确的三维真实点、互斥边以及所述检测点、三维假设点、第一权重边、第二权重边、第三权重边进行构建以得到所述第二路径图像。The correct 3D real point, mutually exclusive edge, detection point, 3D hypothetical point, first weighted edge, second weighted edge, and third weighted edge are constructed to obtain the second path image.
- 根据权利要求6所述的目标跟踪方法,其中,所述根据预设算法对所述第二路径图像进行路径计算,以得到结果路径包括:The target tracking method according to claim 6, wherein said performing path calculation on said second path image according to a preset algorithm to obtain a resulting path comprises:将所述第二路径图像中按最小成本最大流算法进行计算,以确定出最短路径;calculating the second path image according to the minimum cost maximum flow algorithm to determine the shortest path;将所述最短路径确定为结果路径。The shortest path is determined as the resulting path.
- 一种目标跟踪装置,其中,包括:A target tracking device, including:第一确定模块,用于确定多张图片中的人体区域对应的人体检测图像;A first determining module, configured to determine human body detection images corresponding to human body regions in multiple pictures;融合模块,用于根据所述多个人体检测图像之间的关联信息,将所述多个人体检测图像进行特征融合,以构建出第一路径图像;A fusion module, configured to perform feature fusion of the plurality of human detection images according to the association information between the plurality of human detection images, so as to construct a first path image;构建模块,用于在所述第一路径图像中添加互斥信息以构建出第二路径图像;A building module, configured to add mutually exclusive information to the first path image to construct a second path image;计算模块,用于根据预设算法对所述第二路径图像进行路径计算,以得 到结果路径;A calculation module, configured to perform path calculation on the second path image according to a preset algorithm to obtain a resultant path;第二确定模块,用于将得到的所述结果路径确定为跟踪结果。The second determining module is configured to determine the obtained result path as a tracking result.
- 一种电子设备,包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序,其中,所述处理器执行所述计算机程序时实现如权利要求1至7任一项所述的目标跟踪方法的步骤。An electronic device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein, when the processor executes the computer program, any of claims 1 to 7 is implemented. A step of the target tracking method.
- 一种计算机可读存储介质,所述计算机可读存储介质存储有计算机程序,其中,所述计算机程序被处理器执行时实现如权利要求1至7任一项所述的目标跟踪方法的步骤。A computer-readable storage medium, the computer-readable storage medium stores a computer program, wherein, when the computer program is executed by a processor, the steps of the target tracking method according to any one of claims 1 to 7 are realized.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111674242.5A CN114445458B (en) | 2021-12-31 | 2021-12-31 | Target tracking method, device, electronic equipment and storage medium |
CN202111674242.5 | 2021-12-31 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023123916A1 true WO2023123916A1 (en) | 2023-07-06 |
Family
ID=81365528
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/100164 WO2023123916A1 (en) | 2021-12-31 | 2022-06-21 | Target tracking method and apparatus, and electronic device and storage medium |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN114445458B (en) |
WO (1) | WO2023123916A1 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114445458B (en) * | 2021-12-31 | 2024-08-02 | 深圳云天励飞技术股份有限公司 | Target tracking method, device, electronic equipment and storage medium |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9582718B1 (en) * | 2015-06-30 | 2017-02-28 | Disney Enterprises, Inc. | Method and device for multi-target tracking by coupling multiple detection sources |
CN107169989A (en) * | 2017-04-17 | 2017-09-15 | 南京邮电大学 | A kind of multi-object tracking method assessed based on data correlation and track |
US10679362B1 (en) * | 2018-05-14 | 2020-06-09 | Vulcan Inc. | Multi-camera homogeneous object trajectory alignment |
CN114445458A (en) * | 2021-12-31 | 2022-05-06 | 深圳云天励飞技术股份有限公司 | Target tracking method and device, electronic equipment and storage medium |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110268444A (en) * | 2019-02-26 | 2019-09-20 | 武汉资联虹康科技股份有限公司 | A kind of number of people posture tracing system for transcranial magnetic stimulation diagnosis and treatment |
CN112861575A (en) * | 2019-11-27 | 2021-05-28 | 中兴通讯股份有限公司 | Pedestrian structuring method, device, equipment and storage medium |
CN113177968A (en) * | 2021-04-27 | 2021-07-27 | 北京百度网讯科技有限公司 | Target tracking method and device, electronic equipment and storage medium |
CN113610895A (en) * | 2021-08-06 | 2021-11-05 | 烟台艾睿光电科技有限公司 | Target tracking method and device, electronic equipment and readable storage medium |
-
2021
- 2021-12-31 CN CN202111674242.5A patent/CN114445458B/en active Active
-
2022
- 2022-06-21 WO PCT/CN2022/100164 patent/WO2023123916A1/en unknown
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9582718B1 (en) * | 2015-06-30 | 2017-02-28 | Disney Enterprises, Inc. | Method and device for multi-target tracking by coupling multiple detection sources |
CN107169989A (en) * | 2017-04-17 | 2017-09-15 | 南京邮电大学 | A kind of multi-object tracking method assessed based on data correlation and track |
US10679362B1 (en) * | 2018-05-14 | 2020-06-09 | Vulcan Inc. | Multi-camera homogeneous object trajectory alignment |
CN114445458A (en) * | 2021-12-31 | 2022-05-06 | 深圳云天励飞技术股份有限公司 | Target tracking method and device, electronic equipment and storage medium |
Also Published As
Publication number | Publication date |
---|---|
CN114445458B (en) | 2024-08-02 |
CN114445458A (en) | 2022-05-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2020259248A1 (en) | Depth information-based pose determination method and device, medium, and electronic apparatus | |
WO2021208372A1 (en) | Indoor visual navigation method, apparatus, and system, and electronic device | |
Stamos et al. | Integrating automated range registration with multiview geometry for the photorealistic modeling of large-scale scenes | |
TWI491849B (en) | Video object localization method using multiple cameras | |
US20170076482A1 (en) | Position searching method and apparatus based on electronic map | |
JP2018148367A (en) | Image processing device, image processing system, image processing method, and program | |
Tang et al. | ESTHER: Joint camera self-calibration and automatic radial distortion correction from tracking of walking humans | |
WO2021136386A1 (en) | Data processing method, terminal, and server | |
CN111160291B (en) | Human eye detection method based on depth information and CNN | |
CN108470356A (en) | A kind of target object fast ranging method based on binocular vision | |
TW202001892A (en) | Indoor positioning system and method based on geomagnetic signals in combination with computer vision | |
WO2023123916A1 (en) | Target tracking method and apparatus, and electronic device and storage medium | |
TWI709062B (en) | Virtuality reality overlapping method and system | |
WO2019080743A1 (en) | Target detection method and apparatus, and computer device | |
US11823326B2 (en) | Image processing method | |
JP2020067978A (en) | Floor detection program, floor detection method, and terminal device | |
WO2020133488A1 (en) | Vehicle detection method and device | |
CN114520906B (en) | Monocular camera-based three-dimensional portrait complementing method and system | |
CN114170317B (en) | Swimming pool drowning prevention head position judging method and device and computer equipment | |
JP7312026B2 (en) | Image processing device, image processing method and program | |
TW200817651A (en) | Distance measurement system and method | |
CN112634366B (en) | Method for generating position information, related device and computer program product | |
JP7192526B2 (en) | Image processing device, image processing method and program | |
CN114694137B (en) | Image detection method, three-dimensional imaging method and device | |
CN116563352A (en) | Single-line laser radar loop detection method and system integrating depth visual information |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22913205 Country of ref document: EP Kind code of ref document: A1 |