WO2020237501A1 - 一种多源协同道路车辆监控系统 - Google Patents

一种多源协同道路车辆监控系统 Download PDF

Info

Publication number
WO2020237501A1
WO2020237501A1 PCT/CN2019/088786 CN2019088786W WO2020237501A1 WO 2020237501 A1 WO2020237501 A1 WO 2020237501A1 CN 2019088786 W CN2019088786 W CN 2019088786W WO 2020237501 A1 WO2020237501 A1 WO 2020237501A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
targets
information
marked
radar detection
Prior art date
Application number
PCT/CN2019/088786
Other languages
English (en)
French (fr)
Inventor
李良群
贺勇
李小香
Original Assignee
深圳大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳大学 filed Critical 深圳大学
Priority to PCT/CN2019/088786 priority Critical patent/WO2020237501A1/zh
Publication of WO2020237501A1 publication Critical patent/WO2020237501A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/06—Systems determining position data of a target
    • G01S13/08—Systems for measuring distance only
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02—Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50—Systems of measurement based on relative movement of target
    • G01S13/58—Velocity or trajectory determination systems; Sense-of-movement determination systems
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/66—Radar-tracking systems; Analogous systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present invention relates to the field of monitoring technology, in particular to a target monitoring method, device and computer-readable storage medium.
  • Multi-sensor fusion target monitoring technology has become a research hotspot in many fields such as intelligent transportation systems, assisted driving systems, unmanned driving systems, and security systems. Relying on a single sensor is difficult to deal with various complex scenes, and the monitoring technology fusing multi-source information can make full use of the advantages of various sensors, and comprehensively improve the accuracy and stability of the target monitoring technology.
  • constructing a target detection framework for radar and camera video fusion can take advantage of the richness of video information, as well as the high precision and stability of radar information, so as to deal with various complex scenarios.
  • it is usually still mainly dependent on one of the video information and radar information to monitor the target, while the other type of information is not fully utilized, which makes the target monitoring accuracy. And efficiency is still relatively limited.
  • the main purpose of the embodiments of the present invention is to provide a target monitoring method, device, and computer-readable storage medium, which can at least solve the problem that related technologies mainly rely on one of the video information and radar information to monitor targets, and the other
  • the first type of information is not fully utilized, resulting in the problem of the accuracy and efficiency of target monitoring is still relatively limited.
  • the first aspect of the embodiments of the present invention provides a target monitoring method, which includes:
  • the video detection information includes the bounding box information (x, y, w, h) of each video detection target, category information c i, and the category of the target Probability p i
  • the radar detection information includes position information (x, y) and speed information v of each radar detection target;
  • the d kj is compared with a preset distance threshold d th , and all the detection targets in the monitoring area are respectively output as M d targets, A′ targets or B′ targets according to the comparison result; wherein, the The M d target is a target pair whose d kj is less than or equal to the d th , the A'target is the target remaining after the target pair is excluded from all radar detection targets, and the B'target is The remaining targets after excluding the target pair from all video detection targets.
  • a target monitoring device which includes:
  • the obtaining module is used to obtain the video detection information and radar detection information of the preset monitoring area at the current moment;
  • the video detection information includes the bounding box information (x, y, w, h) and category information c i of each video detection target And the probability p i of the target category, the radar detection information includes position information (x, y) and speed information v of each radar detection target;
  • the calculation module is used to calculate the centroid matrix D N ⁇ M of all the video detection targets and the radar detection targets according to the video detection information and radar detection information;
  • a determining module configured to determine the minimum value d kj of each row of the D N ⁇ M , where the d kj is used to characterize the distance of the relatively closest radar detection target of each video detection target;
  • the output module is used to compare the d kj with a preset distance threshold d th , and output all the detection targets in the monitoring area as M d targets, A′ targets or B′ targets according to the comparison result
  • the M d target is a target pair whose d kj is less than or equal to the d th
  • the A'target is the target remaining after removing the target pair from all radar detection targets
  • the Class B'targets are the remaining targets after removing the target pair from all video detection targets.
  • a third aspect of the embodiments of the present invention provides an electronic device, which includes: a processor, a memory, and a communication bus;
  • the communication bus is used to implement connection and communication between the processor and the memory
  • the processor is configured to execute one or more programs stored in the memory to implement the steps of any one of the foregoing target monitoring methods.
  • a fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs can be processed by one or more The device executes to achieve the steps of any of the above-mentioned target monitoring methods.
  • the video detection information and radar detection information of the preset monitoring area at the current moment are respectively acquired; according to the video detection information and radar detection information, all the data are calculated The video detection target and the centroid matrix of the radar detection target; determine the minimum value in each row of the centroid matrix that is used to characterize the distance of each video detection target relative to the nearest radar detection target; compare the minimum value with a preset distance threshold, according to The comparison results determine the target whose centroids are matched successfully, the video detection target and the target that is left after the successfully matched target is eliminated from the radar detection target.
  • the two types of detection information are fused, and all detected targets are divided into three types of targets through centroid matching, which effectively improves the accuracy of target monitoring. Comprehensiveness and efficiency.
  • FIG. 1 is a schematic flowchart of a target monitoring method provided by the first embodiment of the present invention
  • FIG. 2 is a schematic flowchart of a centroid fine adjustment method provided by the first embodiment of the present invention
  • FIG. 3 is a schematic flowchart of another target monitoring method provided by the first embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of a target monitoring device provided by a second embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of an electronic device provided by a third embodiment of the present invention.
  • FIG. 1 is a schematic diagram of the basic flow of the target monitoring method provided in this embodiment.
  • the target monitoring method proposed in this embodiment include the following steps:
  • Step 101 Obtain video detection information and radar detection information of the preset monitoring area at the current moment; the video detection information includes the bounding box information (x, y, w, h) of each video detection target, category information c i, and the category to which the target belongs The probability p i , the radar detection information includes position information (x, y) and velocity information v of each radar detection target.
  • cameras and radars are used to collect data on the same surveillance area at the same time. After the video data and radar data are collected, six-dimensional video detection information is obtained based on the video data and radar data respectively. And three-dimensional radar detection information, to characterize the targets they detect.
  • this embodiment is based on the deep learning algorithm to detect the target in the video data, that is, the video data is input to the constructed neural network model to output the video detection target, where the neural network used may include Any of Deep Neural Network (DNN), Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN).
  • DNN Deep Neural Network
  • CNN Convolutional Neural Network
  • RNN Recurrent Neural Network
  • the neural network model of this embodiment may be a YOLO (You Only Look Once) convolutional neural network model obtained by training a YOLO convolutional neural network based on a preset training sample set.
  • the training The sample is divided into S ⁇ S grids, and each grid predicts B bounding boxes that may include the detection target.
  • Input the video data into the YOLO convolutional neural network and output the bounding box information (x, y, w, h) corresponding to each target, the category information c i, and the probability p i of the category to which the target belongs.
  • the bounding box information of the target is also the position information of the target, where x and y are the position offset of the center of the bounding box relative to the upper left corner of the grid where it is located, and w and h are the width and height of the bounding box.
  • the YOLO convolutional neural network in this embodiment can include 24 convolutional layers and 2 fully connected layers.
  • the activation functions of the convolutional layer and the fully connected layer are the Leaky ReLU function, and the convolutional layer is used for The image features of the target in the training sample are extracted, and the fully connected layer is used to predict the bounding box information of the target.
  • Step 102 Calculate the centroid matrix D N ⁇ M of all video detection targets and radar detection targets according to the video detection information and the radar detection information.
  • the radar detection target can be set as a class A target
  • the video detection target can be set as a class B target
  • the formula for calculating the centroid matrix D N ⁇ M of the class A target and the class B target can be expressed as follows:
  • a i represents the location information of the radar detection target, that is, the center of mass of the radar detection target
  • b j represents the location information of the set center point of the bounding box corresponding to the video detection target, that is, the center of mass of the video detection target.
  • Step 103 Determine the minimum value d kj of each row of D N ⁇ M , where d kj is used to represent the distance of each video detection target to the nearest radar detection target.
  • the minimum value of each row of the centroid matrix represents the distance between the j-th video detection target closest to the k-th radar detection target position. It should be noted that the formula for calculating d kj in this embodiment can be expressed as follows:
  • D′ represents a new N ⁇ 1 dimensional matrix composed of the minimum value of each row of the centroid matrix D N ⁇ M .
  • Step 104 Compare d kj with the preset distance threshold d th , and according to the comparison result, output all the detected targets in the monitoring area as M d targets, A′ targets or B′ targets; among them, M d
  • the target is the target pair with d kj less than or equal to d th
  • the A′ target is the remaining target after removing the target pair from all radar detection targets
  • the B′ target is the remaining target after removing the target pair from all the video detection targets .
  • the type A target that excludes the successfully matched target pair is classified as one type, and it is recorded as A′
  • the type B target that removes the successfully matched target pair is classified as one type, and it is recorded as B′, which is expressed as follows: In a normal detection scene, there are fewer A′ targets, and if the external environment is bad (heavy rain, fog, etc.), there are more such targets; for B′ targets, such targets indicate the target area detected by the video. The radar does not detect the target. This type of situation may be due to radar missed detection or video misdetection, and the probability of occurrence is relatively low.
  • centroid distance matching After centroid distance matching, if the centroid distance of two targets at the same time is less than the set threshold, the two targets are judged to be the same target, otherwise they are regarded as two different targets.
  • A′ targets or B′ targets after outputting all detection targets in the monitoring area as M d targets, A′ targets or B′ targets according to the comparison result, it also includes: performing bounding boxes on each A′ target according to preset rules Designation; fine-tuning the centroid of the A'and B'targets of the designated bounding box.
  • the centroid of the target is further adjusted, so that the target can be positioned more accurately. Since the A′ target indicates the position detected by the radar, the video detection does not detect the target. For this type of target, due to the lack of video information, it is necessary to specify the bounding box for this type of target, that is, it can be specified manually or in accordance with the preset. Let the specified rules be automatically specified. After the bounding box is specified for the A'target, the centroid can be fine-tuned for the A'target.
  • FIG. 2 is a schematic flowchart of the centroid fine-tuning method provided in this embodiment, which specifically includes the following steps:
  • Step 201 draw a middle perpendicular to the bounding box, and divide the bounding box into left and right parts;
  • Step 202 Obtain color histograms of the left and right parts of the blocks
  • Step 203 Calculate the similarity of the color histograms of the left and right parts of the blocks
  • Step 204 According to the similarity calculation result, move the centroids of the A'target and the B'target to the position with the highest similarity.
  • this embodiment fine-tunes the centroid of the target based on the geometric symmetry of the target, that is, the boundary box specified by the radar detection target and the vertical line of the boundary box of the video detection target are used as the boundary, and the boundary box is divided into left and right In the two parts, obtain the color histograms of the left and right blocks respectively, calculate the similarity of the color histograms on the left and right sides of the target bounding box, and then move the centroid to the direction of increasing similarity to find the position with the highest similarity. This position is used as the final centroid of the target.
  • H l and H r are the color histogram vectors of the left and right parts respectively, and N is the dimension of the color histogram.
  • this embodiment uses the Bhattacharyya distance to calculate the similarity.
  • the value range of the Bhattacharyya distance is [0,1]. The smaller the value, the higher the identification similarity. Among them, 0 means the left and right part of the block The histogram matches perfectly, and 1 means no match at all.
  • FIG. 3 is a schematic flowchart of another target monitoring method provided in this embodiment, which specifically includes the following steps:
  • Step 301 Obtain the first characteristic information of the M d target, the A'target, and the B'target;
  • Step 302 Calculate a feature similarity measurement parameter based on the first feature information of the target marked at the current moment and the second feature information of the target marked at the historical moment;
  • Step 303 Calculate the matching degree between the marked target at each current moment and the marked target at each historical moment based on the feature similarity measurement parameter to obtain a matching degree matrix
  • Step 304 Based on the matching degree, it is determined that all consecutive n frames can be successfully matched with the existing target, and the label of the successfully matched target is set as the label of the existing target, and it is determined that it is not consistent with the existing target in n consecutive frames.
  • the target is matched with a successful target, and the label of the unmatched target is newly edited; wherein, in the initial state, the label is assigned to the marked target.
  • Heterologous features may include the following three types of information in at least one of: location information (x i, y i, w i, h i), velocity information v i and characterization information af i, af i for the location information corresponding to the boundary
  • the in-depth feature information of the image in the frame is the apparent feature of the target extracted from the region of interest.
  • calculating feature similarity measurement parameters includes:
  • v i and v j are respectively the speed information of the target marked at the current moment and the speed information of the target marked at the historical moment;
  • d 3 1-cos(af i -af j ), where af i and af j are the characterization information of the target marked at the current time and the historical time respectively Characterization information of the marked target.
  • the matching degree between the target marked at each current moment and the target marked at each historical moment is calculated, and obtaining the matching degree matrix may include: combining d 1 , d 2 and d 3 Substitute the feature fusion calculation formula to calculate the matching degree w ij between the target marked at each current moment and the target marked at each historical moment to obtain the matching degree matrix W N ⁇ M .
  • the similarity calculation formula is expressed as follows:
  • ⁇ 1 , ⁇ 2 and ⁇ 3 are the weight parameters corresponding to d 1 , d 2 and d 3 respectively, and N and M are respectively the number of marked targets at the current moment and the number of marked targets at historical moments.
  • the environmental judgment factor m p needs to be used.
  • the judgment factor of the external environment is set based on the video detection data.
  • the video detection data includes: target location information, target category information, and probability information of the target category. The higher the probability that the target belongs to the category, the better the effect of model detection, and the side reflects that the collected video is clearer.
  • this embodiment can extract the first K frames of the video detection result, and then select the probability information with the largest probability value among the probability information of the categories of all targets in each frame, and then compare the selected K maximum probabilities
  • the average value of the detection probability values of K frames before the value calculation, the calculation formula of the environmental judgment factor m p is expressed as follows:
  • p i represents the probability information of the category to which the target belongs
  • K is the number of frames.
  • the weight parameter ⁇ 3 corresponding to the characterizing similarity measure parameter d 3 is designed with f ⁇ (x) function, which is expressed as follows:
  • a speed corresponding to a similarity measure parameter d weight of 1 parameter ⁇ 1 and the speed corresponding to the similarity measure parameter d weight of 1 ⁇ 2 represent the following parameters:
  • ⁇ is a preset super parameter, with a value of [0, 1], which is used to adjust the weight parameter.
  • the traditional Hungarian algorithm can be used when matching the detection result with the existing labeled target.
  • the Hungarian algorithm is aimed at the task assignment problem, and when dealing with the target The association still applies.
  • the main rule is expressed as follows: In the initial state, the target number (ID number) is assigned to the target detected in consecutive k (for example, 5) frames; the newly detected target in the subsequent, if consecutive n (for example, 3) frames can be compared with the target If a target is successfully matched, it is set to an active state and the target ID number is assigned. If the target is not matched successfully in n consecutive frames, a new ID is assigned to the target.
  • the current video detection information and radar detection information of the preset monitoring area are respectively obtained; according to the video detection information and radar detection information, all the video detection targets and the radar are calculated The centroid matrix of the detection target; determine the minimum value in each row of the centroid matrix that characterizes the distance of the nearest radar detection target of each video detection target; compare the minimum value with the preset distance threshold, and determine the target whose centroid matches successfully according to the comparison result , Video detection targets and radar detection targets are the remaining targets after the successfully matched targets are eliminated.
  • the two types of detection information are fused, and all detected targets are divided into three types of targets through centroid matching, which effectively improves the accuracy of target monitoring. Comprehensiveness and efficiency.
  • this embodiment shows a target monitoring device.
  • the target monitoring device of this embodiment includes:
  • the obtaining module 401 is used to obtain the current video detection information and radar detection information of the preset monitoring area;
  • the video detection information includes the bounding box information (x, y, w, h) of each video detection target, the category information c i, and The probability p i of the target category,
  • the radar detection information includes the position information (x, y) and velocity information v of each radar detection target;
  • the calculation module 402 is used to calculate the centroid matrix D N ⁇ M of all video detection targets and radar detection targets according to the video detection information and radar detection information;
  • the determining module 403 is used to determine the minimum value d kj of each row of D N ⁇ M , and d kj is used to characterize the distance of each video detection target to the nearest radar detection target;
  • the output module 404 is used to compare d kj with a preset distance threshold d th , and according to the comparison result, output all the detected targets in the monitoring area as M d targets, A′ targets or B′ targets, respectively;
  • the M d target is the target pair with d kj less than or equal to d th
  • the A′ target is the remaining target after removing the target pair from all radar detection targets
  • the B′ target is the target pair after removing the target pair from all the video detection targets. The remaining goals.
  • the target monitoring device of this embodiment further includes: a centroid fine-tuning module for outputting all detection targets in the monitoring area as M d targets and A'targets according to the comparison result. Or after class B'targets, specify the bounding box of each class A'target according to preset rules; perform centroid fine-tuning processing for class A'and class B'targets of the specified bounding box.
  • a centroid fine-tuning module for outputting all detection targets in the monitoring area as M d targets and A'targets according to the comparison result. Or after class B'targets, specify the bounding box of each class A'target according to preset rules; perform centroid fine-tuning processing for class A'and class B'targets of the specified bounding box.
  • centroid fine-tuning module when it performs centroid fine-tuning processing, it is specifically used to draw a vertical line to the bounding box and divide the bounding box into two parts; Color histogram; calculate the similarity of the color histograms of the left and right parts of the block; according to the similarity calculation result, move the centroids of the A'target and the B'target to the position with the highest similarity.
  • the similarity calculation formula is expressed as follows:
  • H l and H r are the color histogram vectors of the left and right parts respectively, and N is the dimension of the color histogram.
  • the target monitoring device of this embodiment further includes: a tracking module, which is used to obtain the M d type after outputting all the detection targets in the monitoring area as the M d target, the A′ target or the B′ target according to the comparison result.
  • the first characteristic information of the target, the A'target and the B'target based on the first characteristic information of the target marked at the current moment and the second characteristic information of the target marked at the historical moment, the characteristic similarity measurement parameter is calculated; Based on the feature similarity measurement parameters, the matching degree between the marked target at each current moment and the marked target at each historical moment is calculated to obtain the matching degree matrix; based on the matching degree, it is determined that the consecutive n frames can be successfully matched with the existing target Target, and set the label of the successfully matched target as the label of the existing target, and determine the target that has not been successfully matched with the existing target in n consecutive frames, and re-edit the label of the unmatched target; among them, In the initial state, label assignments are made to the marked targets.
  • the tracking module when the first feature information includes speed information, position information, and characterization information, when the tracking module calculates the feature similarity measurement parameter, it is specifically used to calculate the speed similarity measurement parameter d 1.
  • the tracking module calculates the matching degree between the target marked at each current moment and the target marked at each historical moment based on the feature similarity measurement parameter, to obtain the matching degree matrix
  • It is specifically used to substitute d 1 , d 2 and d 3 into the feature fusion calculation formula to calculate the matching degree w ij between the target marked at each current moment and the target marked at each historical moment to obtain a matching degree matrix W N ⁇ M , similar
  • the calculation formula of degree is expressed as follows:
  • ⁇ 1 , ⁇ 2 and ⁇ 3 are the weight parameters corresponding to d 1 , d 2 and d 3 respectively, and N and M are respectively the number of marked targets at the current moment and the number of marked targets at historical moments.
  • target monitoring methods in the foregoing embodiments can all be implemented based on the target monitoring device provided in this embodiment, and those of ordinary skill in the art can clearly understand that for the convenience and conciseness of the description, the description in this embodiment For the specific working process of the described target monitoring device, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.
  • the target monitoring device provided in this embodiment is used to obtain the video detection information and radar detection information of the preset monitoring area at the current moment; according to the video detection information and radar detection information, all the video detection targets and the radar detection information are calculated
  • the centroid matrix of the target determine the minimum value in each row of the centroid matrix that is used to characterize the distance of each video detection target relative to the nearest radar detection target; compare the minimum value with the preset distance threshold, and determine the target whose centroid matches successfully according to the comparison result.
  • the video detection target and the radar detection target are the remaining targets after the successfully matched targets are eliminated.
  • the two types of detection information are fused, and all detected targets are divided into three types of targets through centroid matching, which effectively improves the accuracy of target monitoring. Comprehensiveness and efficiency.
  • This embodiment provides an electronic device, as shown in FIG. 5, which includes a processor 501, a memory 502, and a communication bus 503.
  • the communication bus 503 is used to implement connection and communication between the processor 501 and the memory 502; processing
  • the device 501 is configured to execute one or more computer programs stored in the memory 502 to implement at least one step in the target monitoring method in the first embodiment.
  • This embodiment also provides a computer-readable storage medium, which is included in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules, or other data). Volatile or non-volatile, removable or non-removable media.
  • Computer-readable storage media include but are not limited to RAM (Random Access Memory), ROM (Read-Only Memory, read-only memory), EEPROM (Electrically Erasable Programmable read only memory, charged Erasable Programmable Read-Only Memory) ), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, Or any other medium that can be used to store desired information and that can be accessed by a computer.
  • the computer-readable storage medium in this embodiment may be used to store one or more computer programs, and the stored one or more computer programs may be executed by a processor to implement at least one step of the method in the first embodiment.
  • This embodiment also provides a computer program, which can be distributed on a computer-readable medium and executed by a computer-readable device to implement at least one step of the method in the first embodiment; and in some cases At least one of the steps shown or described can be performed in a different order from the order described in the foregoing embodiment.
  • This embodiment also provides a computer program product, including a computer readable device, and the computer readable device stores the computer program as shown above.
  • the computer-readable device in this embodiment may include the computer-readable storage medium as shown above.
  • communication media usually contain computer-readable instructions, data structures, computer program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium. Therefore, the present invention is not limited to any specific combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

一种目标监控方法、装置及计算机可读存储介质，该方法包括：分别获取预设监控区域当前时刻的视频检测信息和雷达检测信息（步骤101）；根据该视频检测信息和雷达检测信息，计算所有该视频检测目标以及该雷达检测目标的质心矩阵（步骤102）；确定质心矩阵各行中用于表征各视频检测目标相对最近的雷达检测目标的距离的最小值（步骤103）；将最小值与预设距离阈值进行比较，根据比较结果确定质心匹配成功的目标、视频检测目标以及雷达检测目标中分别剔除匹配成功的目标后所剩余的目标（步骤104）。该方法对两类检测信息进行融合处理，经过质心匹配来将所有检测目标划分为三类目标，有效提高了目标监控的准确性、全面性以及效率。

Description

一种多源协同道路车辆监控系统 技术领域
本发明涉及监控技术领域，尤其涉及一种目标监控方法、装置及计算机可读存储介质。
背景技术
多传感器融合的目标监控技术已成为智能交通系统、辅助驾驶系统、无人驾驶系统、安防系统等诸多领域的研究热点。依赖单一的传感器较难处理各种复杂场景，融合多源信息的监控技术可以充分利用各类传感器的优势，全面提高目标监控技术的精度与稳定性。
其中，构建雷达和摄像机视频融合的目标检测框架能够利用视频信息的丰富性，以及雷达信息的高精度和稳定性，从而可以应对各种复杂场景。然而，目前在对目标进行监控时，通常仍是主要依赖于视频信息与雷达信息中的某一类来对目标进行监控，而另一类信息则未得到充分利用，从而使得目标监控的准确性和效率仍较为局限。
技术问题
本发明实施例的主要目的在于提供一种目标监控方法、装置及计算机可读存储介质，至少能够解决相关技术中主要依赖于视频信息与雷达信息中的某一类来对目标进行监控，而另一类信息则未得到充分利用，所导致的目标监控的准确性和效率仍较为局限的问题。
技术解决方案
为实现上述目的，本发明实施例第一方面提供了一种目标监控方法，该方法包括：
分别获取预设监控区域当前时刻的视频检测信息和雷达检测信息；所述视频检测信息包括各视频检测目标的边界框信息(x,y,w,h)、类别信息c i以及目标所属类别的概率p i，所述雷达检测信息包括各雷达检测目标的位置信息(x,y)以及速度信息v；
根据所述视频检测信息和雷达检测信息，计算所有所述视频检测目标以及所述雷达检测目标的质心矩阵D N×M；
确定所述D N×M各行的最小值d kj，所述d kj用于表征所述各视频检测目标相对最近的所述雷达检测目标的距离；
将所述d kj与预设距离阈值d th进行比较，根据比较结果将所述监控区域内的 所有检测目标分别输出为M d类目标、A′类目标或B′类目标；其中，所述M d类目标为所述d kj小于或等于所述d th的目标对，所述A′类目标为所有雷达检测目标中剔除所述目标对后所剩余的目标，所述B′类目标为所有视频检测目标中剔除所述目标对后所剩余的目标。
为实现上述目的，本发明实施例第二方面提供了一种目标监控装置，该装置包括：
获取模块，用于分别获取预设监控区域当前时刻的视频检测信息和雷达检测信息；所述视频检测信息包括各视频检测目标的边界框信息(x,y,w,h)、类别信息c i以及目标所属类别的概率p i，所述雷达检测信息包括各雷达检测目标的位置信息(x,y)以及速度信息v；
计算模块，用于根据所述视频检测信息和雷达检测信息，计算所有所述视频检测目标以及所述雷达检测目标的质心矩阵D N×M；
确定模块，用于确定所述D N×M各行的最小值d kj，所述d kj用于表征所述各视频检测目标相对最近的所述雷达检测目标的距离；
输出模块，用于将所述d kj与预设距离阈值d th进行比较，根据比较结果将所述监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标；其中，所述M d类目标为所述d kj小于或等于所述d th的目标对，所述A′类目标为所有雷达检测目标中剔除所述目标对后所剩余的目标，所述B′类目标为所有视频检测目标中剔除所述目标对后所剩余的目标。
为实现上述目的，本发明实施例第三方面提供了一种电子装置，该电子装置包括：处理器、存储器和通信总线；
所述通信总线用于实现所述处理器和存储器之间的连接通信；
所述处理器用于执行所述存储器中存储的一个或者多个程序，以实现上述任意一种目标监控方法的步骤。
为实现上述目的，本发明实施例第四方面提供了一种计算机可读存储介质，该计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现上述任意一种目标监控方法的步骤。
有益效果
根据本发明实施例提供的目标监控方法、装置及计算机可读存储介质，分别获取预设监控区域当前时刻的视频检测信息和雷达检测信息；根据所述视频 检测信息和雷达检测信息，计算所有所述视频检测目标以及所述雷达检测目标的质心矩阵；确定质心矩阵各行中用于表征各视频检测目标相对最近的雷达检测目标的距离的最小值；将最小值与预设距离阈值进行比较，根据比较结果确定质心匹配成功的目标、视频检测目标以及雷达检测目标中分别剔除匹配成功的目标后所剩余的目标。通过本发明的实施，考虑到雷达检测结果与视频检测结果存在差异，对两类检测信息进行融合处理，经过质心匹配来将所有检测目标划分为三类目标，有效提高了目标监控的准确性、全面性以及效率。
本发明其他特征和相应的效果在说明书的后面部分进行阐述说明，且应当理解，至少部分效果从本发明说明书中的记载变的显而易见。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明第一实施例提供的一种目标监控方法的流程示意图；
图2为本发明第一实施例提供的质心微调方法的流程示意图；
图3为本发明第一实施例提供的另一种目标监控方法的流程示意图；
图4为本发明第二实施例提供的目标监控装置的结构示意图；
图5为本发明第三实施例提供的电子装置的结构示意图。
本发明的实施方式
为使得本发明的发明目的、特征、优点能够更加的明显和易懂，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而非全部实施例。基于本发明中的实施例，本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
第一实施例：
为了解决相关技术中完全依赖雷达信息进行目标定位，针对雷达定位所得的目标感兴趣区域进行相应的图像处理，然而此类方法未能充分利用视频信息，容易出现目标的虚假检测以及单个目标多次检测等的技术问题，本实施例提出了一种目标监控方法，以对目标进行检测，如图1所示为本实施例提供的目标监控方法的基本流程示意图，本实施例提出的目标监控方法包括以下的步骤：
步骤101、分别获取预设监控区域当前时刻的视频检测信息和雷达检测信息；视频检测信息包括各视频检测目标的边界框信息(x,y,w,h)、类别信息c i以及目标所属类别的概率p i，雷达检测信息包括各雷达检测目标的位置信息(x,y)以及速度信息v。
具体的，本实施例中在同一时刻分别通过摄像机和雷达来对同一监控区域进行数据采集，在采集到视频数据和雷达数据之后，再分别基于视频数据和雷达数据来获取六维的视频检测信息和三维的雷达检测信息，对各自所检测的目标进行表征。
应当说明的是，本实施例基于深度学习算法是来对视频数据中的目标进行检测，也即将视频数据输入至所构建的神经网络模型来输出视频检测目标，其中，所采用的神经网络可以包括深度神经网络(DNN)，卷积神经网络(CNN)以及循环神经网络(RNN)中任意一种。
在一些实施方式中，本实施例的神经网络模型可以为基于预设的训练样本集对YOLO卷积神经网络进行训练得到的YOLO(You Only Look Once，只看一眼)卷积神经网络模型，训练样本分为S×S个网格，每个网格预测B个可能包括检测目标的边界框。将视频数据输入至该YOLO卷积神经网络，输出各目标所对应的边界框信息(x,y,w,h)、类别信息c i以及目标所属类别的概率p i，应当理解的是，这里目标的边界框信息也即目标的位置信息，其中，x和y为边界框的中心相对于所处网格的左上角的位置偏移量，w和h为边界框的宽度和高度。还应当说明的是，本实施例中的YOLO卷积神经网络可以包括24个卷积层和2个全连接层，卷积层和全连接层的激活函数为Leaky ReLU函数，卷积层用于提取训练样本中目标的图像特征，全连接层用于预测目标的边界框信息。
步骤102、根据视频检测信息和雷达检测信息，计算所有视频检测目标以及雷达检测目标的质心矩阵D N×M。
具体的，在本实施例中，可以将雷达检测目标设为A类目标，视频检测目标设为B类目标，其中，计算A类目标与B类目标的质心矩阵D N×M的公式可以表示如下：
D N×M＝{d ij|d ij＝centroid_dis(a i,b j)}
a i∈A,b j∈B 1≤i≤N,1≤j≤M
其中，a i表示雷达检测目标的位置信息，也即雷达检测目标的质心，b j表示视频检测目标所对应的边界框的集合中心点位置信息，也即视频检测目标的质 心。
步骤103、确定D N×M各行的最小值d kj，d kj用于表征各视频检测目标相对最近的雷达检测目标的距离。
具体的，在本实施例中，质心矩阵各行的最小值表示与第k个雷达检测目标位置最接近的第j个视频检测目标之间的距离。应当说明的是，本实施例中计算d kj的公式可以表示如下：
D′＝min{d kj|1≤j≤M},(1≤k≤N)
其中，D′表示质心矩阵D N×M各行的最小值组成的新的N×1维矩阵。
步骤104、将d kj与预设距离阈值d th进行比较，根据比较结果将监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标；其中，M d类目标为d kj小于或等于d th的目标对，A′类目标为所有雷达检测目标中剔除目标对后所剩余的目标，B′类目标为所有视频检测目标中剔除目标对后所剩余的目标。从而充分利用视频与雷达信息，对检测目标进行筛选与定位修正，大大降低了单个传感器的误检率，达到准确高效的目标检测效果。
具体的，本实施例中在计算得到两类目标的质心距离d kj之后，对该质心距离进行阈值判断，判断公式表示如下：M d＝{(k,j)|d kj≤d th,d kj∈D′}，其中，d th表示距离阈值，M d表示雷达目标与视频目标的质心距离小于等于设定阈值的匹配对。
而在质心距离大于设定阈值时，剔除匹配成功目标对的A类目标为一类，记为A′,剔除匹配成功目标对的B类目标为一类，记为B′，表示如下：
Figure PCTCN2019088786-appb-000001
在正常检测场景下，A′类目标较少，若在外界环境恶劣时(大雨、大雾等)此类目标较多；而对于B′类目标，此类目标表明视频检测到的目标区域，雷达并未检测到目标，此类情况可能由于雷达漏检或者视频误检，发生概率相对较低。
经过质心距离匹配，如果同一时刻某两个目标的质心距离小于设定的阈值，则判定两个目标为同一个目标，反之视为两个不同的目标。
进一步地，在根据比较结果将监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标之后，还包括：按照预设规则对各A′类目标进行边界框指定；对指定边界框的A′类目标以及B′类目标，进行质心微调处理。
具体的，本实施例中在完成对目标的分类之后，还对目标的质心进行调整，从而可以更加精确的定位目标。由于A′类目标表明在雷达检测的位置，视频检 测并未检测到该目标，针对此类目标，由于缺乏视频信息，需要对该类目标进行边界框指定，也即可以进行人为指定或按照预设指定规则自动指定。在对A′类目标完成边界框指定后，才能对A′类目标进行质心微调。
可选的，本实施例提供了一种质心微调方法，如图2为本实施例提供的质心微调方法的流程示意图，具体包括以下步骤：
步骤201、对边界框作中垂线，将边界框划分为左右两部分；
步骤202、获取左右两部分图块的颜色直方图；
步骤203、计算左右两部分图块的颜色直方图的相似度；
步骤204、根据相似度计算结果，将A′类目标以及B′类目标的质心移动至相似度最高的位置。
具体的，本实施例基于目标的几何对称性来对目标的质心进行微调，即以雷达检测目标所指定的边界框和视频检测目标的边界框的中垂线为边界，将边界框划分为左右两部分，分别求取左右部分图块的颜色直方图，计算目标边界框左右两侧的颜色直方图相似度，然后将质心向相似度增大的方向移动，求得相似度最高的位置，以此位置作为目标的最终质心点。
可选的，在进行相似度计算时，相似度的计算公式表示如下：
Figure PCTCN2019088786-appb-000002
其中，H l和H r分别为左右两部分图块的颜色直方图向量，N为颜色直方图的维度。
在算法实现中，本实施例采用巴氏距离进行相似度计算，巴氏距离的值域为[0,1]，其值越小则标识相似度越高，其中，0表示左右部分图块的直方图完美匹配，1则表示完全不匹配。
应当说明的是，由于各类目标的最终输出维度必须相同，需要为B′类目标添加模拟雷达检测信息，也即位置信息(x,y)和速度信息v，本实施例可以利用距离视频检测边界框最近的雷达检测目标充当此模拟雷达检测信息。
进一步地，针对相关技术中的目标跟踪方法仅仅依赖于视频检测信息，未能充分考虑目标的其他信息，如通过雷达检测获得的目标速度信息，容易出现定位不精确以及目标漏检等的技术问题，在根据比较结果将监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标之后，本实施例还提供了 另一种目标监控方法，以对目标进行跟踪，如图3为本实施例提供的另一种目标监控方法的流程示意图，具体包括以下步骤：
步骤301、获取M d类目标、A′类目标以及B′类目标的第一特征信息；
步骤302、基于当前时刻所标记的目标的第一特征信息，以及历史时刻所标记的目标的第二特征信息，计算特征相似性度量参数；
步骤303、基于特征相似性度量参数，计算各当前时刻所标记的目标与各历史时刻所标记的目标的匹配度，得到匹配度矩阵；
步骤304、在基于匹配度，确定连续n帧均能与已有目标匹配成功的目标，并将匹配成功的目标的标号设置为已有目标的标号，以及确定未在连续n帧均与已有目标匹配成功的目标，并将未匹配成功的目标的标号进行新编；其中，在初始状态下，对所标记的目标进行标号分配。
具体的，由于雷达检测的数据特征与视频处理提取的数据特征来自不同类型传感器，故称两类目标特征为异源特征。异源特征可以包含以下三类信息中的至少一种：位置信息(x i,y i,w i,h i)、速度信息v i和表征信息af i，af i为位置信息所对应的边界框内的图像深层特征信息，也即对感兴趣区域所提取的目标表观特征。
在本实施例一种可选的实施方式中，在第一特征信息包括速度信息、位置信息以及表征信息时，计算特征相似性度量参数包括：
计算速度相似性度量参数d 1，计算公式表示如下：
Figure PCTCN2019088786-appb-000003
其中，v i、v j分别为当前时刻所标记的目标的速度信息，以及历史时刻所标记的目标的速度信息；
计算位置相似性度量参数d 2，计算公式表示如下：d 2＝(l i-l j) TS j -1(l i-l j)，其中，l i、l j分别为当前时刻所标记的目标的位置信息，以及历史时刻所标记的目标的位置信息，l i＝[x i，y i，w i，h i]，l j＝[x j，y j，w j，h j]，S j为目标状态协方差矩阵，T为矩阵转置；
计算表征相似性度量参数d 3，计算公式表示如下：d 3＝1-cos(af i-af j)，其中，af i、af j分别为当前时刻所标记的目标的表征信息，以及历史时刻所标记的目标的表征信息。
另外，在本实施例中，基于特征相似性度量参数，计算各当前时刻所标记的目标与各历史时刻所标记的目标的匹配度，得到匹配度矩阵可以包括：将d 1、d 2与d 3代入特征融合计算公式，计算各当前时刻所标记的目标与各历史时刻所 标记的目标的匹配度w ij，得到匹配度矩阵W N×M，相似度的计算公式表示如下：
w ij＝λ 1×d 1+λ 2×d 2+λ 3×d 3
λ 1+λ 2+λ 3＝1；
1≤i≤N,1≤j≤M
其中，λ 1、λ 2与λ 3分别为d 1、d 2与d 3所对应的权重参数，N、M分别为当前时刻所标记的目标的数量，以及历史时刻所标记的目标的数量。
还应当说明的是，至于λ 1、λ 2、λ 3三个权重参数的设置，需要利用环境判断因子m p。外界环境的判断因子是基于视频检测数据进行设定，视频检测数据包括：目标的位置信息、目标类别信息以及目标所属类别的概率信息。当目标所属类别的概率越高，表明模型检测的效果越好，侧面反映采集到的视频是较为清晰的。基于以上的认知，本实施例可以提取视频检测结果的前K帧，然后选取每帧中所有目标所述类别的概率信息中，概率值最大的概率信息，再对所选取的K个最大概率值计算前K帧检测概率值的均值，环境判断因子m p的计算公式表示如下：
Figure PCTCN2019088786-appb-000004
其中，p i表示目标所属类别的概率信息，K为帧数。应当理解的是，当m p越大，表明视频检测结果越好，在进行数据融合时视频检测信息的比重越高；相反m p越小，表明视频检测结果越差，侧面反映外界环境较为恶劣(大雾、暴雨等天气)，在进行数据融合时，应降低对视频检测信息的依赖度。
应当说明的是，本实施例中对表征相似性度量参数d 3对应的权重参数λ 3进行了f λ(x)函数设计，表示如下：
Figure PCTCN2019088786-appb-000005
从而，权重参数λ 3表示如下：
Figure PCTCN2019088786-appb-000006
对应于速度相似性度量参数d 1的权重参数λ 1，以及对应于速度相似性度量参数d 1的权重参数λ 2分别表示如下：
Figure PCTCN2019088786-appb-000007
Figure PCTCN2019088786-appb-000008
其中，ξ为预设超参，取值为[0，1]，用于调配权重参数。
此外，本实施例在得到匹配度矩阵之后，进行目标匹配与管理时，具体的，匹配检测结果与已有标号目标时可以采用传统的匈牙利算法，匈牙利算法是针对任务分配问题的，在处理目标关联仍然适用。主要的规则表述如下：在初始状态下，连续k(例如5)帧均检测到的目标进行目标标号(ID号)的分配；后续新检测的目标，若连续n(例如3)帧能够与已有目标匹配成功，则设置为激活状态，分配目标ID号，而若未在连续n帧均匹配成功，则为该目标分配新ID。另外，对于已有ID号的目标，若在连续m(取大于n的值，例如20)帧内未能与新检测到的目标匹配成功，则删除此目标的ID号(视为目标已经离开检测范围)。其中，m、n、k均取大于0的正整数。从而，高效融合雷达与视频检测特征，大大改善单个传感器视频目标跟踪的虚假检测与标号跳变等问题，达到多目标的准确跟踪效果。
根据本发明实施例提供的目标监控方法，分别获取预设监控区域当前时刻的视频检测信息和雷达检测信息；根据所述视频检测信息和雷达检测信息，计算所有所述视频检测目标以及所述雷达检测目标的质心矩阵；确定质心矩阵各行中用于表征各视频检测目标相对最近的雷达检测目标的距离的最小值；将最小值与预设距离阈值进行比较，根据比较结果确定质心匹配成功的目标、视频检测目标以及雷达检测目标中分别剔除匹配成功的目标后所剩余的目标。通过本发明的实施，考虑到雷达检测结果与视频检测结果存在差异，对两类检测信息进行融合处理，经过质心匹配来将所有检测目标划分为三类目标，有效提高了目标监控的准确性、全面性以及效率。
第二实施例：
为了解决相关技术中主要依赖于视频信息与雷达信息中的某一类来对目标进行监控，而另一类信息则未得到充分利用，所导致的目标监控的准确性和效率仍较为局限的技术问题，本实施例示出了一种目标监控装置，具体请参见图4，本实施例的目标监控装置包括：
获取模块401，用于分别获取预设监控区域当前时刻的视频检测信息和雷达检测信息；视频检测信息包括各视频检测目标的边界框信息(x,y,w,h)、类别信息c i以及目标所属类别的概率p i，雷达检测信息包括各雷达检测目标的位置信息(x,y)以及速度信息v；
计算模块402，用于根据视频检测信息和雷达检测信息，计算所有视频检 测目标以及雷达检测目标的质心矩阵D N×M；
确定模块403，用于确定D N×M各行的最小值d kj，d kj用于表征各视频检测目标相对最近的雷达检测目标的距离；
输出模块404，用于将d kj与预设距离阈值d th进行比较，根据比较结果将监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标；其中，M d类目标为d kj小于或等于d th的目标对，A′类目标为所有雷达检测目标中剔除目标对后所剩余的目标，B′类目标为所有视频检测目标中剔除目标对后所剩余的目标。
在本实施例的一些实施方式中，本实施例的目标监控装置还包括：质心微调模块，用于在根据比较结果将监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标之后，按照预设规则对各A′类目标进行边界框指定；对指定边界框的A′类目标以及B′类目标，进行质心微调处理。
进一步地，在本实施例的一些实施方式中，质心微调模块在进行质心微调处理时，具体用于对边界框作中垂线，将边界框划分为左右两部分；获取左右两部分图块的颜色直方图；计算左右两部分图块的颜色直方图的相似度；根据相似度计算结果，将A′类目标以及B′类目标的质心移动至相似度最高的位置。
更进一步地，在本实施例的一些实施方式中，相似度的计算公式表示如下：
Figure PCTCN2019088786-appb-000009
其中，H l和H r分别为左右两部分图块的颜色直方图向量，N为颜色直方图的维度。
本实施例的目标监控装置还包括：跟踪模块，用于在根据比较结果将监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标之后，获取M d类目标、A′类目标以及B′类目标的第一特征信息；基于当前时刻所标记的目标的第一特征信息，以及历史时刻所标记的目标的第二特征信息，计算特征相似性度量参数；基于特征相似性度量参数，计算各当前时刻所标记的目标与各历史时刻所标记的目标的匹配度，得到匹配度矩阵；在基于匹配度，确定连续n帧均能与已有目标匹配成功的目标，并将匹配成功的目标的标号设置为已有目标的标号，以及确定未在连续n帧均与已有目标匹配成功的目标，并将未匹配成功的目标的标号进行新编；其中，在初始状态下，对所标记的目标进行标 号分配。
进一步地，在本实施例的一些实施方式中，在第一特征信息包括速度信息、位置信息以及表征信息时，跟踪模块在计算特征相似性度量参数时，具体用于计算速度相似性度量参数d 1，计算公式表示如下：
Figure PCTCN2019088786-appb-000010
其中，v i、v j分别为当前时刻所标记的目标的速度信息，以及历史时刻所标记的目标的速度信息；计算位置相似性度量参数d 2，计算公式表示如下：d 2＝(l i-l j) TS j -1(l i-l j)，其中，l i、l j分别为当前时刻所标记的目标的位置信息，以及历史时刻所标记的目标的位置信息，l i＝[x i，y i，w i，h i]，l j＝[x j，y j，w j，h j]，S j为目标状态协方差矩阵，T为矩阵转置；计算表征相似性度量参数d 3，计算公式表示如下：d 3＝1-cos(af i-af j)，其中，af i、af j分别为当前时刻所标记的目标的表征信息，以及历史时刻所标记的目标的表征信息。
更进一步地，在本实施例的一些实施方式中，跟踪模块在基于特征相似性度量参数，计算各当前时刻所标记的目标与各历史时刻所标记的目标的匹配度，得到匹配度矩阵时，具体用于将d 1、d 2与d 3代入特征融合计算公式，计算各当前时刻所标记的目标与各历史时刻所标记的目标的匹配度w ij，得到匹配度矩阵W N×M，相似度的计算公式表示如下：
w ij＝λ 1×d 1+λ 2×d 2+λ 3×d 3
λ 1+λ 2+λ 3＝1；
1≤i≤N,1≤j≤M
其中，λ 1、λ 2与λ 3分别为d 1、d 2与d 3所对应的权重参数，N、M分别为当前时刻所标记的目标的数量，以及历史时刻所标记的目标的数量。
应当说明的是，前述实施例中的目标监控方法均可基于本实施例提供的目标监控装置实现，所属领域的普通技术人员可以清楚的了解到，为描述的方便和简洁，本实施例中所描述的目标监控装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
采用本实施例提供的目标监控装置，分别获取预设监控区域当前时刻的视频检测信息和雷达检测信息；根据所述视频检测信息和雷达检测信息，计算所有所述视频检测目标以及所述雷达检测目标的质心矩阵；确定质心矩阵各行中用于表征各视频检测目标相对最近的雷达检测目标的距离的最小值；将最小值与预设距离阈值进行比较，根据比较结果确定质心匹配成功的目标、视频检测目标以及雷达检测目标中分别剔除匹配成功的目标后所剩余的目标。通过本发 明的实施，考虑到雷达检测结果与视频检测结果存在差异，对两类检测信息进行融合处理，经过质心匹配来将所有检测目标划分为三类目标，有效提高了目标监控的准确性、全面性以及效率。
第三实施例：
本实施例提供了一种电子装置，参见图5所示，其包括处理器501、存储器502及通信总线503，其中：通信总线503用于实现处理器501和存储器502之间的连接通信；处理器501用于执行存储器502中存储的一个或者多个计算机程序，以实现上述实施例一中的目标监控方法中的至少一个步骤。
本实施例还提供了一种计算机可读存储介质，该计算机可读存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、计算机程序模块或其他数据)的任何方法或技术中实施的易失性或非易失性、可移除或不可移除的介质。计算机可读存储介质包括但不限于RAM(Random Access Memory，随机存取存储器),ROM(Read-Only Memory，只读存储器),EEPROM(Electrically Erasable Programmable read only memory，带电可擦可编程只读存储器)、闪存或其他存储器技术、CD-ROM(Compact Disc Read-Only Memory，光盘只读存储器)，数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。
本实施例中的计算机可读存储介质可用于存储一个或者多个计算机程序，其存储的一个或者多个计算机程序可被处理器执行，以实现上述实施例一中的方法的至少一个步骤。
本实施例还提供了一种计算机程序，该计算机程序可以分布在计算机可读介质上，由可计算装置来执行，以实现上述实施例一中的方法的至少一个步骤；并且在某些情况下，可以采用不同于上述实施例所描述的顺序执行所示出或描述的至少一个步骤。
本实施例还提供了一种计算机程序产品，包括计算机可读装置，该计算机可读装置上存储有如上所示的计算机程序。本实施例中该计算机可读装置可包括如上所示的计算机可读存储介质。
可见，本领域的技术人员应该明白，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件(可以用计算装置可执行的计算机程序代码来实现)、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。
此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、计算机程序模块或者诸如载波或其他传输机制之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。所以，本发明不限制于任何特定的硬件和软件结合。
以上内容是结合具体的实施方式对本发明实施例所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干简单推演或替换，都应当视为属于本发明的保护范围。

Claims (10)

  1. 一种目标监控方法，其特征在于，包括：
    分别获取预设监控区域当前时刻的视频检测信息和雷达检测信息；所述视频检测信息包括各视频检测目标的边界框信息(x,y,w,h)、类别信息c i以及目标所属类别的概率p i，所述雷达检测信息包括各雷达检测目标的位置信息(x,y)以及速度信息v；
    根据所述视频检测信息和雷达检测信息，计算所有所述视频检测目标以及所述雷达检测目标的质心矩阵D N×M；
    确定所述D N×M各行的最小值d kj，所述d kj用于表征所述各视频检测目标相对最近的所述雷达检测目标的距离；
    将所述d kj与预设距离阈值d th进行比较，根据比较结果将所述监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标；其中，所述M d类目标为所述d kj小于或等于所述d th的目标对，所述A′类目标为所有雷达检测目标中剔除所述目标对后所剩余的目标，所述B′类目标为所有视频检测目标中剔除所述目标对后所剩余的目标。
  2. 如权利要求1所述的目标监控方法，其特征在于，在根据比较结果将所述监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标之后，还包括：
    按照预设规则对各所述A′类目标进行边界框指定；
    对指定边界框的所述A′类目标以及所述B′类目标，进行质心微调处理。
  3. 如权利要求2所述的目标监控方法，其特征在于，所述进行质心微调处理包括：
    对所述边界框作中垂线，将所述边界框划分为左右两部分；
    获取左右两部分图块的颜色直方图；
    计算所述左右两部分图块的颜色直方图的相似度；
    根据相似度计算结果，将所述A′类目标以及所述B′类目标的质心移动至相似度最高的位置。
  4. 如权利要求3所述的目标监控方法，其特征在于，所述相似度的计算公式表示如下：
    Figure PCTCN2019088786-appb-100001
    其中，所述H l和H r分别为所述左右两部分图块的颜色直方图向量，所述N为颜色直方图的维度。
  5. 如权利要求1所述的目标监控方法，其特征在于，在根据比较结果将所述监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标之后，还包括：
    获取所述M d类目标、所述A′类目标以及所述B′类目标的第一特征信息；
    基于当前时刻所标记的目标的第一特征信息，以及历史时刻所标记的目标的第二特征信息，计算特征相似性度量参数；
    基于所述特征相似性度量参数，计算各所述当前时刻所标记的目标与各所述历史时刻所标记的目标的匹配度，得到匹配度矩阵；
    在基于所述匹配度，确定连续n帧均能与已有目标匹配成功的目标，并将所述匹配成功的目标的标号设置为所述已有目标的标号，以及确定未在连续n帧均与已有目标匹配成功的目标，并将未匹配成功的目标的标号进行新编；其中，在初始状态下，对所标记的目标进行标号分配；其中，所述n取大于0的正整数。
  6. 如权利要求5所述的目标监控方法，其特征在于，在所述第一特征信息包括速度信息、位置信息以及表征信息时，所述表征信息为对感兴趣区域所提取的目标表观特征，所述计算特征相似性度量参数包括：
    计算速度相似性度量参数d 1，计算公式表示如下：
    Figure PCTCN2019088786-appb-100002
    其中，所述v i、v j分别为所述当前时刻所标记的目标的速度信息，以及历史时刻所标记的目标的速度信息；
    计算位置相似性度量参数d 2，计算公式表示如下：d 2＝(l i-l j) TS j -1(l i-l j)，其中，所述l i、l j分别为所述当前时刻所标记 的目标的位置信息，以及历史时刻所标记的目标的位置信息，所述l i＝[x i，y i，w i，h i]，所述l j＝[x j，y j，w j，h j]，所述S j为目标状态协方差矩阵，所述T为矩阵转置；
    计算表征相似性度量参数d 3，计算公式表示如下：d 3＝1-cos(af i-af j)，其中，所述af i、af j分别为所述当前时刻所标记的目标的表征信息，以及历史时刻所标记的目标的表征信息。
  7. 如权利要求6所述的目标监控方法，其特征在于，所述基于所述特征相似性度量参数，计算各所述当前时刻所标记的目标与各所述历史时刻所标记的目标的匹配度，得到匹配度矩阵包括：
    将所述d 1、d 2与d 3代入特征融合计算公式，计算各所述当前时刻所标记的目标与各所述历史时刻所标记的目标的匹配度w ij，得到匹配度矩阵W N×M，所述相似度的计算公式表示如下：
    w ij＝λ 1×d 1+λ 2×d 2+λ 3×d 3
    λ 1+λ 2+λ 3＝1；
    1≤i≤N,1≤j≤M
    其中，所述λ 1、λ 2与λ 3分别为所述d 1、d 2与d 3所对应的权重参数，所述N、M分别为当前时刻所标记的目标的数量，以及历史时刻所标记的目标的数量。
  8. 一种目标监控装置，其特征在于，包括：
    获取模块，用于分别获取预设监控区域当前时刻的视频检测信息和雷达检测信息；所述视频检测信息包括各视频检测目标的边界框信息(x,y,w,h)、类别信息c i以及目标所属类别的概率p i，所述雷达检测信息包括各雷达检测目标的位置信息(x,y)以及速度信息v；
    计算模块，用于根据所述视频检测信息和雷达检测信息，计算所有所述视频检测目标以及所述雷达检测目标的质心矩阵D N×M；
    确定模块，用于确定所述D N×M各行的最小值d kj，所述d kj用于表征所述各视频检测目标相对最近的所述雷达检测目标的距离；
    输出模块，用于将所述d kj与预设距离阈值d th进行比较，根据比较结果将所述监控区域内的所有检测目标分别输出为M d类目标、A′类目标或B′类目标；其中，所述M d类目标为所述d kj小于或等于所述 d th的目标对，所述A′类目标为所有雷达检测目标中剔除所述目标对后所剩余的目标，所述B′类目标为所有视频检测目标中剔除所述目标对后所剩余的目标。
  9. 一种电子装置，其特征在于，包括：处理器、存储器和通信总线；
    所述通信总线用于实现所述处理器和存储器之间的连接通信；
    所述处理器用于执行所述存储器中存储的一个或者多个程序，以实现如权利要求1至7中任意一项所述的目标监控方法的步骤。
  10. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如权利要求1至7中任意一项所述的目标监控方法的步骤。
PCT/CN2019/088786 2019-05-28 2019-05-28 一种多源协同道路车辆监控系统 WO2020237501A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/088786 WO2020237501A1 (zh) 2019-05-28 2019-05-28 一种多源协同道路车辆监控系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/088786 WO2020237501A1 (zh) 2019-05-28 2019-05-28 一种多源协同道路车辆监控系统

Publications (1)

Publication Number Publication Date
WO2020237501A1 true WO2020237501A1 (zh) 2020-12-03

Family

ID=73552456

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/088786 WO2020237501A1 (zh) 2019-05-28 2019-05-28 一种多源协同道路车辆监控系统

Country Status (1)

Country Link
WO (1) WO2020237501A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344906A (zh) * 2021-06-29 2021-09-03 阿波罗智联(北京)科技有限公司 车路协同中相机评估方法、装置、路侧设备和云控平台
CN117093872A (zh) * 2023-10-19 2023-11-21 四川数字交通科技股份有限公司 雷达目标分类模型自训练方法及系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006266927A (ja) * 2005-03-24 2006-10-05 Toyota Motor Corp 物体検出装置
CN102682453A (zh) * 2012-04-24 2012-09-19 河海大学 基于多特征融合的运动车辆跟踪方法
CN108615321A (zh) * 2018-06-07 2018-10-02 湖南安隆软件有限公司 基于雷达侦测及视频图像行为分析的安防预警系统及方法
US20180348363A1 (en) * 2015-12-04 2018-12-06 Denso Corporation Object detection apparatus and object detection method
CN109581345A (zh) * 2018-11-28 2019-04-05 深圳大学 基于毫米波雷达的目标检测与跟踪方法及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006266927A (ja) * 2005-03-24 2006-10-05 Toyota Motor Corp 物体検出装置
CN102682453A (zh) * 2012-04-24 2012-09-19 河海大学 基于多特征融合的运动车辆跟踪方法
US20180348363A1 (en) * 2015-12-04 2018-12-06 Denso Corporation Object detection apparatus and object detection method
CN108615321A (zh) * 2018-06-07 2018-10-02 湖南安隆软件有限公司 基于雷达侦测及视频图像行为分析的安防预警系统及方法
CN109581345A (zh) * 2018-11-28 2019-04-05 深圳大学 基于毫米波雷达的目标检测与跟踪方法及系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113344906A (zh) * 2021-06-29 2021-09-03 阿波罗智联(北京)科技有限公司 车路协同中相机评估方法、装置、路侧设备和云控平台
CN113344906B (zh) * 2021-06-29 2024-04-23 阿波罗智联(北京)科技有限公司 车路协同中相机评估方法、装置、路侧设备和云控平台
CN117093872A (zh) * 2023-10-19 2023-11-21 四川数字交通科技股份有限公司 雷达目标分类模型自训练方法及系统
CN117093872B (zh) * 2023-10-19 2024-01-02 四川数字交通科技股份有限公司 雷达目标分类模型自训练方法及系统

Similar Documents

Publication Publication Date Title
CN110187334B (zh) 一种目标监控方法、装置及计算机可读存储介质
CN109087510B (zh) 交通监测方法及装置
CN110349187B (zh) 基于tsk模糊分类器的目标跟踪方法、装置及存储介质
CN112883819A (zh) 多目标跟踪方法、装置、系统及计算机可读存储介质
CN111882586B (zh) 一种面向剧场环境的多演员目标跟踪方法
CN108197604A (zh) 基于嵌入式设备的快速人脸定位跟踪方法
CN114049382B (zh) 一种智能网联环境下目标融合跟踪方法、系统和介质
CN113936198B (zh) 低线束激光雷达与相机融合方法、存储介质及装置
CN111709975A (zh) 多目标跟踪方法、装置、电子设备及存储介质
CN110363165B (zh) 基于tsk模糊系统的多目标跟踪方法、装置及存储介质
CN115063454B (zh) 多目标跟踪匹配方法、装置、终端及存储介质
WO2020237501A1 (zh) 一种多源协同道路车辆监控系统
CN106599918B (zh) 车辆追踪方法及系统
CN115546705B (zh) 目标识别方法、终端设备及存储介质
CN113221750A (zh) 车辆追踪方法、装置、设备及存储介质
CN114972490B (zh) 一种数据自动标注方法、装置、设备及存储介质
CN113129335A (zh) 一种基于孪生网络的视觉跟踪算法及多模板更新策略
CN115376109A (zh) 障碍物检测方法、障碍物检测装置以及存储介质
CN115797736A (zh) 目标检测模型的训练和目标检测方法、装置、设备和介质
Mirthubashini et al. Video based vehicle counting using deep learning algorithms
CN114898314A (zh) 驾驶场景的目标检测方法、装置、设备及存储介质
CN116681730A (zh) 一种目标物追踪方法、装置、计算机设备和存储介质
CN115327529A (zh) 一种融合毫米波雷达和激光雷达的3d目标检测与追踪方法
CN113887449A (zh) 多目标跟踪方法及计算机可读存储介质
Song et al. Target tracking and 3D trajectory reconstruction based on multicamera calibration

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19931013

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19931013

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 01/04/2022)

122 Ep: pct application non-entry in european phase

Ref document number: 19931013

Country of ref document: EP

Kind code of ref document: A1