WO2023206236A1 - 一种目标的检测方法及相关装置 - Google Patents

一种目标的检测方法及相关装置 Download PDF

Info

Publication number
WO2023206236A1
WO2023206236A1 PCT/CN2022/089905 CN2022089905W WO2023206236A1 WO 2023206236 A1 WO2023206236 A1 WO 2023206236A1 CN 2022089905 W CN2022089905 W CN 2022089905W WO 2023206236 A1 WO2023206236 A1 WO 2023206236A1
Authority
WO
WIPO (PCT)
Prior art keywords
motion
key point
classifier
detection
state
Prior art date
Application number
PCT/CN2022/089905
Other languages
English (en)
French (fr)
Inventor
吴尚坤
张强
田少雄
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to PCT/CN2022/089905 priority Critical patent/WO2023206236A1/zh
Publication of WO2023206236A1 publication Critical patent/WO2023206236A1/zh

Links

Images

Definitions

  • the present application relates to the field of computer technology, and in particular, to a target detection method and related devices.
  • the camera can observe the environment around the vehicle and assist the driving system in making various driving decisions.
  • the camera monitors other moving objects (for example, vehicles, obstacles, pedestrians, etc.) in the environment around the vehicle.
  • the motion status of other objects can be estimated, thereby assisting the driving system to make various driving decisions.
  • Decision-making e.g., anti-collision braking, deceleration, etc.
  • a method for judging motion status based on multi-frame optical flow is currently proposed.
  • the principle of this method is to extract corner points on the target object, and the corner points on the images of different frames form multiple frames.
  • Optical flow select the lowest and closest optical flow point to the ground to calculate the movement distance of the target object between two frames. If the movement distance is less than the movement distance of the vehicle itself, the target object is judged to be stationary, otherwise it is judged to be in motion; then, The relative motion angle of the target object is calculated based on the optical flow convergence point and vanishing point of the optical flow on the target object, and then the motion state of the target object is determined.
  • the corner points are extreme points in the image, that is, points with particularly prominent attributes in certain aspects, such as isolated points with the highest or smallest intensity on certain attributes, and the end points of line segments.
  • the determination of the corner points is related to the shooting quality of the image and the corner point extraction algorithm. It is possible that the determined corner points cannot effectively represent the motion state of the overall target object, which will lead to a large error in the determined motion state of the target object. ; In addition, since the lowest and closest corner point to the ground is approximated as a corner point on the ground in this way, the calculated movement distance of the target object may be significantly different from the real movement distance of the target object.
  • Embodiments of the present application provide a target detection method and related devices, which can obtain more accurate motion detection results of detected targets.
  • this application provides a target detection method, which method includes: acquiring multiple images, the images including detection targets; determining at least one key point on the outline of the detection target; determining the movement of each key point Trajectory; classify the motion trajectory of each key point through multiple classifiers to obtain the motion detection result of each key point; among which, the multiple classifiers include: stationary motion classifier, non-parallel motion classifier, High-speed parallel motion classifier, low-speed motion classifier; the multiple classifiers also include one or more of the following: motion angle classifier, motion scale classifier, retrograde motion classifier; motion detection results based on each key point Determine the motion detection result of the detection target.
  • the key points of the detection target are points with specific characteristics on the outline of the detection target (for example, the key points of the vehicle may include the vehicle's rearview mirror, tire contact point, car lights, etc.)
  • the motion trajectory calculated based on this key point is more consistent with the motion of the real detection target than the motion trajectory calculated by optical flow, and can provide more reliable input for subsequent motion detection.
  • the judgment result of one classifier on a motion feature of a key point is relatively accurate. By combining the judgment results of multiple classifiers, a more accurate motion detection result of the key point can be obtained. Furthermore, by jointly determining the motion detection result of the detection target through the motion detection results of each key point, a more accurate motion detection result of the detection target can be obtained.
  • the image can be a picture, photo or video, streaming media, etc.
  • the detection target can be vehicles, pedestrians, buildings, roadblocks, etc.
  • the detection target can be understood as a person or object that may affect the action of the mobile station.
  • the motion detection result includes a motion state and a first motion probability
  • the first motion probability is used to indicate a probability of being in the motion state.
  • the first motion probability may be a confidence level.
  • the motion detection result can be used as input data for other equipment or devices.
  • it can be used as input for the adjustment module of the mobile station (for example, advanced driving assistance system, vehicle controller, etc.) to provide it with Accurately detecting the motion detection results of the target can better assist the driving of the mobile station.
  • one of the classifiers is used to determine a motion feature of the key point, and a second motion probability of the key point being in the motion feature; the motion state is based on multiple motions Features are determined, and the first motion probability is determined based on a plurality of the second motion probabilities.
  • the judgment result of one classifier on a motion feature of a key point is relatively accurate. By combining the judgment results of multiple classifiers, a more accurate motion detection result of the key point can be obtained.
  • the movement state includes one or more of the following: forward fast movement state, forward slow movement state, stationary state, forward cut-in state, forward Cut-out state, high-speed retrograde movement state, low-speed retrograde movement state, forward acceleration movement state, forward deceleration movement state, forward rapid acceleration movement state, forward rapid deceleration movement state, emergency stop state, crossing State, parallel movement state, lateral turning state, longitudinal movement state, U-turn state, parameter information describing the movement state; wherein, the parameter information describing the movement state includes one or more of the following: absolute speed information, relative speed information, collision time information.
  • the determined motion state has a more specific and detailed feature description compared to simple motion states such as stationary and moving, as well as simple speed and acceleration information. In this way, more effective Accurate motion detection results.
  • the motion characteristics determined by the stationary motion classifier include: stationary and moving; the motion characteristics determined by the non-parallel motion classifier include: lateral traveling, cutting in, and cutting out. ;
  • the motion characteristics determined by the high-speed parallel motion classifier include: fast, high-speed, acceleration, and sharp acceleration; the motion characteristics determined by the low-speed motion classifier include: slow speed, deceleration, and sharp deceleration;
  • the motion angle classifier determines
  • the movement characteristics include: judging turns, U-turns, parallel driving, forward/reverse movement; the movement characteristics determined by the movement scale classifier include: forward/reverse acceleration; the movement characteristics determined by the retrograde movement classifier include: reverse direction Driving, forward driving motion.
  • multiple motion classifiers make use of information such as pixel scale changes, real scale changes, angle changes, etc. of the detected target. Compared with methods that use multi-frame optical flow to determine motion status, they are more robust.
  • the motion detection result of the detection target corresponds to the first moment
  • the method further includes: based on the motion detection result of the detection target corresponding to the second moment and the motion detection result of the detection target.
  • the motion detection result updates the motion detection result of the detection target corresponding to the first time, and the second time is before the first time. In this way, more accurate motion detection results can be determined based on the continuity of the detected target motion.
  • a camera that acquires the multiple images is located on a mobile platform, and multiple classifiers are used to classify the motion trajectory of each key point to obtain the motion trajectory of each key point.
  • the motion detection results include: obtaining the motion parameters of the mobile station; based on the motion parameters of the mobile station, classifying the motion trajectories of each key point through multiple classifiers to obtain the motion detection results of each key point.
  • determining at least one key point on the contour of the detection target includes: performing target detection on a region of interest in the image, and determining a bounding box containing the detection target. ; Process the image in the bounding box to determine the outline of the detection target and the type of the detection target, which type includes one or more of the following: vehicles, pedestrians, buildings, roadblocks; determine the detection based on the outline and the type At least one keypoint on the contour of the target whose location on the contour is determined based on the type.
  • the key points of the detection target are points with specific characteristics on the outline of the detection target.
  • the position of the key point on the outline of the detection target can be determined more conveniently, which can improve the determination of the target. accuracy of key points and reduce processing time.
  • the bounding box of the detection target can include the detection target, which is a smaller image area than the area of interest. By setting the bounding box, you can further reduce the range of images that need to be processed, reduce processing time, and increase accuracy.
  • the camera that acquires the multiple images is located on a mobile platform.
  • the method further includes: determining at least one characteristic coordinate point in the bounding box; and using a bounding box motion classifier to The motion trajectory of each characteristic coordinate point is classified and processed to obtain the relative speed of the detection target relative to the mobile station and a third motion probability at the relative speed; the relative speed and the third motion probability are updated based on the relative speed. Detect the motion detection results of the target.
  • the bounding box motion classifier can obtain more accurate determination results.
  • the multiple images include a first image at a third time and a second image at a fourth time
  • determining the motion trajectory of each key point includes: determining the first position information of each key point in the first image; determining second position information of each key point in the second image; determining each key point according to the first position information and the second position information Movement trajectory of key points.
  • the motion trajectory can also be called a motion field, a motion vector, etc.
  • the motion trajectory of the key point can also be determined through images taken at more times, which is not limited by the embodiments of this application.
  • the multiple classifiers are connected through cascade or average integration.
  • the multiple classifiers can also be connected through partial cascade and partial average integration.
  • the judgment results obtained by each classifier can be used as a reference for each other. In this way, the correlation of the judgment results of different classifiers for a certain type or several types of motion states can be fully utilized.
  • the judgment results obtained by each classifier are obtained independently. If the judgment result of one classifier is not very accurate, it will not affect the judgment results of other classifiers.
  • classifiers with strong correlations can be connected in a cascade manner, and classifiers with weak correlations can be connected in an average integration method.
  • this application provides a detection device, which includes an acquisition unit, a first determination unit, a second determination unit, a processing unit and a third determination unit: the acquisition unit is used to acquire multiple images, the The image includes a detection target; the first determination unit is used to determine at least one key point on the outline of the detection target; the second determination unit is used to determine the motion trajectory of each key point; the processing unit is used to pass Multiple classifiers classify the motion trajectory of each key point to obtain the motion detection result of each key point; among them, the multiple classifiers include: stationary motion classifier, non-parallel motion classifier, and high-speed parallel motion Classifier, low-speed motion classifier; the multiple classifiers also include one or more of the following: motion angle classifier, motion scale classifier, retrograde motion classifier; the third determination unit is used to determine based on each key The motion detection result of the point determines the motion detection result of the detection target.
  • the motion detection result includes a motion state and a first motion probability
  • the first motion probability is used to indicate a probability of being in the motion state.
  • one of the classifiers is used to determine a motion feature of the key point, and a second motion probability of the key point being in the motion feature; the motion state is based on multiple motions Features are determined, and the first motion probability is determined based on a plurality of the second motion probabilities.
  • the movement state includes one or more of the following: forward fast movement state, forward slow movement state, stationary state, forward cut-in state, forward Cut-out state, high-speed retrograde movement state, low-speed retrograde movement state, forward acceleration movement state, forward deceleration movement state, forward rapid acceleration movement state, forward rapid deceleration movement state, emergency stop state, crossing State, parallel movement state, lateral turning state, longitudinal movement state, U-turn state, parameter information describing the movement state; wherein, the parameter information describing the movement state includes one or more of the following: absolute speed information, relative speed information, collision time information.
  • the motion characteristics determined by the stationary motion classifier include: stationary and moving; the motion characteristics determined by the non-parallel motion classifier include: lateral traveling, cutting in, and cutting out. ;
  • the motion characteristics determined by the high-speed parallel motion classifier include: fast, high-speed, acceleration, and sharp acceleration; the motion characteristics determined by the low-speed motion classifier include: slow speed, deceleration, and sharp deceleration;
  • the motion angle classifier determines
  • the movement characteristics include: judging turns, U-turns, parallel driving, forward/reverse movement; the movement characteristics determined by the movement scale classifier include: forward/reverse acceleration; the movement characteristics determined by the retrograde movement classifier include: reverse direction Drive, drive forward.
  • the motion detection result of the detection target corresponds to the first time
  • the third determination unit is further configured to: based on the motion detection result of the detection target corresponding to the second time and The motion detection result of the detection target is updated to the motion detection result of the detection target corresponding to the first time, and the second time is before the first time.
  • the camera that acquires the multiple images is located on the mobile station, and the acquisition unit is also used to: acquire the motion parameters of the mobile station; the processing unit is specifically used to: based on the The motion parameters of the mobile station are classified and processed by multiple classifiers on the motion trajectory of each key point, and the motion detection results of each key point are obtained.
  • the first determination unit is specifically configured to: perform target detection on the area of interest in the image, determine a bounding box containing the detection target; and process the bounding box in the bounding box.
  • the image determines the outline of the detection target and the type of the detection target.
  • the type includes one or more of the following: vehicles, pedestrians, buildings, roadblocks; at least one of the outlines of the detection target is determined based on the outline and the type. A keypoint whose location on this contour is determined based on this type.
  • the camera that acquires the multiple images is located on the mobile platform, and the third determination unit is further configured to: determine at least one feature coordinate point in the bounding box; The motion classifier classifies the motion trajectory of each feature coordinate point to obtain the relative speed of the detection target relative to the mobile station and the third motion probability at the relative speed; based on the relative speed, and the third motion probability The motion probability updates the motion detection result of the detected target.
  • the multiple images include a first image at a third time and a second image at a fourth time
  • the second determination unit is specifically configured to: determine each key the first position information of the point in the first image; determining the second position information of each key point in the second image; determining the position information of each key point according to the first position information and the second position information. Movement trajectory.
  • the multiple classifiers are connected through cascade or average integration.
  • the present application provides a processing device, including a processor and a memory; the memory is used to store program code; the processor is used to call the program code from the memory to execute the above-mentioned first aspect or the third aspect. Any possible implementation of an aspect of the described method.
  • the present application provides a computer-readable storage medium.
  • the computer-readable storage medium is used to store instructions. When the instructions are executed, the above-mentioned first aspect or any possible implementation of the first aspect is achieved. The described method is implemented.
  • multiple classifiers are used to process the motion trajectory of at least one key point on the detection target, and the motion detection result of each key point can be obtained, and then the detection result is determined based on the motion detection result of each key point.
  • Target motion detection results the motion trajectory calculated based on key points is less time-consuming and more accurate than the motion trajectory calculated based on optical flow. It is more consistent with the movement of the real detection target and provides more reliable input for subsequent motion detection;
  • multiple motion classifiers use information such as pixel scale changes, real scale changes, angle changes, etc. of the detected target. Compared with the method that uses multi-frame optical flow to judge motion status, it is more robust, and we get The motion detection results are more accurate. In summary, through the embodiments of the present application, more accurate motion detection results of detected targets can be obtained.
  • Figure 1 is a schematic structural diagram of a mobile station system provided by an embodiment of the present application.
  • Figure 2 is a schematic flow chart of a target detection method provided by an embodiment of the present application.
  • Figure 3 is a schematic diagram of key points of a detection target provided by an embodiment of the present application.
  • Figure 4 is a schematic diagram of a region of interest in an image and a bounding box of a detection target provided by an embodiment of the present application;
  • Figure 5 is a schematic diagram of the connection relationships of some multiple classifiers provided by the embodiment of the present application.
  • Figure 6 is a schematic diagram of a detection device provided by an embodiment of the present application.
  • Figure 7 is a schematic diagram of a detection device provided by an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a chip provided by an embodiment of the present application.
  • FIG. 1 is a schematic structural diagram of a mobile station system provided by an embodiment of the present application.
  • the mobile station can be a car, a robot, a drone, etc.
  • the system includes input module, sensor, detection module and adjustment module. It should be noted that each module shown in Figure 1 is only an example. In actual application scenarios, the mobile station may include more or fewer modules or devices, which are not limited by the embodiments of this application. Each module or device is further introduced below.
  • the input module is used to receive control information for the mobile station.
  • the control information includes: steering wheel angle information, driving gear information, driving and braking information, etc.
  • sensors can be divided into two categories. One is to measure the status of the mobile station itself, and the other is to measure the status of the environment in which the mobile station is located.
  • the former can include global positioning system (GPS), speed sensor, acceleration sensor, angular velocity sensor, torque sensor, etc.
  • GPS global positioning system
  • sensors in the mobile station to collect more information related to the movement (or driving, flight) of the mobile station, such as engine operating conditions, operating temperatures of various modules or devices, intake air pressure, Intake air temperature, etc.
  • the latter can include cameras, lidar sensors, millimeter-wave radar sensors, ultrasonic sensors, and more. These sensors can collect information about the environment in which the mobile station is located.
  • the adjustment module outputs control/adjustment instructions for the mobile station or outputs some operation suggestions to the user to assist the user in adjusting the mobile station. Take control.
  • a camera serves as a sensor and can collect information about the environment where the mobile station is located by taking images of the environment where the mobile station is located.
  • the camera is used to collect road condition information in front of the mobile station while the mobile station is traveling.
  • the camera is also used to collect reversing images of the mobile station, driving record images, etc.
  • the camera can be a monocular camera, a binocular camera, a multi-view camera, a wide-angle camera, or a surround-view camera, etc.
  • the detection module is used to determine the motion status of other objects in the environment based on the information related to the environment where the mobile station is located obtained by the camera.
  • the detection result determined by the detection module can be sent to the adjustment module, so that the adjustment module can control/adjust the driving state of the mobile station based on the detection result, or output some operation suggestions to the user.
  • the adjustment module is used to control/adjust the driving status of the mobile station.
  • the adjustment module can be an advanced driver assist system (ADAS), a vehicle control unit (VCU), an electronic stability program (ESP), etc.
  • ADAS can use environmental data inside and outside the vehicle collected by sensors such as cameras, radars, lasers, and ultrasonics to identify static and dynamic objects, and use technical processing such as detection and tracking to perceive the vehicle's driving intentions.
  • VCU is a control unit that implements vehicle control decisions. It determines the driver's driving intention by collecting signals such as accelerator pedal, gear position, brake pedal, steering wheel angle, etc.; by monitoring vehicle status (vehicle speed, temperature, etc.) information, the VCU determines and processes it.
  • ESP sends the vehicle's operating status control instructions to the power system and power battery system, and simultaneously controls the working mode of the vehicle accessory power system; it has the functions of vehicle system fault diagnosis, protection and storage.
  • ESP can analyze the vehicle driving status information sent from the vehicle sensor, and then issue correction instructions to the anti-lock brake system (anti-lock brake system, ABS), electronic brake distribution system (electrical brake distribution, EBD), etc. To help the vehicle maintain dynamic balance.
  • the adjustment module can obtain the information collected by various sensors and the detection results obtained by the detection module. After analysis, it outputs control instructions for the mobile station or outputs some operation suggestions to the user to assist the user. Mobile station for control. Therefore, the accuracy of the detection results determined by the detection module will have an impact on the control decision to adjust the module output.
  • the detection module can be an independent module, or can also be a component integrated in the adjustment module, which is not limited in the embodiment of the present application.
  • the embodiments of this application can also be applied to distributed sensor networks or non-movable platforms, such as street lights, traffic lights, etc.
  • Related fields include smart intersections, smart cities, etc.
  • the motion status of obstacles in the traffic area can be detected through cameras installed on street lights and traffic lights.
  • the accuracy of determining the motion state of the detected target can be improved.
  • the terminal's perception capability can be improved.
  • FIG 2 is a schematic flow chart of a target detection method provided by an embodiment of the present application.
  • This method can be implemented based on the system shown in Figure 1.
  • the execution subject of the method described below can be the detection module in Figure 1.
  • the detection module can be an independent module, a component integrated in the adjustment module, or integrated in the sensor. a widget.
  • the execution subject of the method described below can also be other modules in the terminal to implement the function of determining the motion detection result of the detection target described in the method described below.
  • the method includes but is not limited to the following steps.
  • the multiple images can be acquired through a camera configured on the terminal (for example, a mobile station).
  • a camera installed on the mobile station can capture images of the environment where the mobile station is located (the multiple images can be multiple frame images taken continuously), and send these images to the detection module.
  • the detection module processes the multiple received images to identify the image containing the detection target.
  • the image can be a picture, photo or video, streaming media, etc.
  • the detection target can be a vehicle, a pedestrian, a building, a roadblock, etc.
  • roadblocks refer to obstacles set up on the road to warn pedestrians or vehicles, or to convey some road condition information.
  • the detection target can be understood as a person or object that may affect the actions of the mobile station.
  • At least one key point on the contour of the detection target can be determined through a detection network model (or algorithm system) that can identify the key points of the detection target.
  • a detection network model or algorithm system
  • the camera of the mobile station can be used to capture several pictures containing various detection targets (for example, vehicles, pedestrians, buildings, roadblocks, etc.) in typical traffic scenes, and detect various types of objects that appear in the captured images. Mark its key points on the detection target (you can also mark its bounding box) to obtain a training data set.
  • the training data set includes key points on the original images and detection targets.
  • the detection network model can identify images containing detection targets.
  • the detection network model can also output a bounding box of the detection target and at least one key point on the outline of the detection target.
  • FIG. 3 is a schematic diagram of detecting key points of a target provided by an embodiment of the present application.
  • the detection target shown in Figure 3 is a vehicle.
  • the figure shows a total of 36 (only examples, more or less in actual applications) possible key points on the vehicle. During the model training process, you can Label these key points.
  • the key points include the vehicle's rearview mirror, tire contact points, lights, and other key characteristic points of the vehicle. It is understandable that for different types of detection targets, the locations of key points are set differently. For example, if the detection target is a person, the key points can be the person's top of the head, shoulders, foot ground points, hand ends, and other key feature points of the person.
  • the method of determining at least one key point on the outline of the detection target is: performing target detection on a region of interest (ROI) in the image, and determining the surrounding area that contains the detection target. box (or bounding box); process the image in the bounding box to determine the outline of the detection target and the type of the detection target, which type includes one or more of the following: vehicles, pedestrians, buildings, roadblocks; based on The contour and the type determine at least one key point on the contour of the detection target, and the position of the key point on the contour is determined based on the type.
  • ROI region of interest
  • the area of interest is an image area selected from the original image.
  • the area of interest may be a rectangular area of preset size located in the center of the original image.
  • the region of interest can also be determined through other operators (Operators) and functions, which are not limited by the embodiments of this application.
  • the bounding box of the detection target can include the detection target, which is a smaller image area than the area of interest.
  • a detection target can correspond to a bounding box.
  • the bounding box can be a two-dimensional or three-dimensional bounding box.
  • FIG. 4 is a schematic diagram of a region of interest in an image and a bounding box of a detection target provided by an embodiment of the present application.
  • the type to which the detection target belongs can have more or fewer types, or one type can be further divided.
  • vehicles can be further divided into trucks, cars, motorcycles, bicycles, etc.
  • the positions of the key points of the detection target may be different. That is to say, the positions of the key points on the outline of the detection target are determined based on the type of the detection target.
  • the corresponding relationship between the position of the key point on the outline of the detection target and the type of the detection target can be targeted during the training process of the detection network model, so that the detection network model can better Process various types of detection targets efficiently to improve judgment accuracy.
  • the image is a multi-frame image taken continuously, and the motion trajectory of the same key point is determined based on the position information of the key point in the images of different frames.
  • the motion trajectory can also be called a motion field, a motion vector, etc.
  • the multiple images include a first image at a third time and a second image at a fourth time
  • determining the motion trajectory of each key point includes: determining the location of each key point. first position information in the first image; determining the second position information of each key point in the second image; determining the motion trajectory of each key point based on the first position information and the second position information .
  • the position information may be absolute coordinate information or relative coordinate information.
  • the movement trajectory of the key point can reflect various movement characteristics of the key point. For example, if the first position information and the second position information are the same, it can indicate that the key point has not moved and is in a stationary state. If the first position information and the second position information are different, it can indicate that the key point has moved, the movement trajectory of the key point can be determined, and further the movement speed, angle, and other information can be calculated.
  • the motion trajectory of the key point is determined through at least two images taken at different times.
  • the motion trajectory of the key point can also be determined through images taken at more times, which is not limited by the embodiments of this application.
  • the characteristic coordinate point on the bounding box can also be used as a special key point, and the motion trajectory of the characteristic coordinate point can also be used as the basis for determining the motion detection result of the detection target.
  • the feature coordinate point can be the vertex of the bounding box, the midpoint of the bounding box, or the midpoint of the bounding box's border, etc.
  • the motion trajectory calculated based on key points is less time-consuming and more accurate than the motion trajectory calculated based on optical flow, and is more consistent with the movement of the real detection target; in this way, it can be used for subsequent Motion detection provides more reliable input.
  • S104 Classify the motion trajectory of each key point through multiple classifiers to obtain the motion detection result of each key point.
  • the multiple classifiers include: static motion classifier, non-parallel motion classifier, high-speed parallel motion classifier, low-speed motion classifier; the multiple classifiers also include one or more of the following: motion angle classifier, Motion scale classifier, retrograde motion classifier.
  • a classifier is used to determine a motion feature of the key point and a second motion probability that the key point is in the motion feature.
  • Each classifier is further introduced below.
  • the multiple classifiers may also include other classifiers with motion feature classification functions.
  • the embodiment of the present application does not limit the number of classifiers included.
  • two or more partial classifiers among these multiple classifiers can also be combined to form a classifier with more comprehensive functions, or one classifier can be split into multiple classifiers with more detailed and specific decision rules. , the splitting and combination of classifiers are not used to limit the scope of the embodiments of this application.
  • the functions of the multiple classifiers can be implemented based on neural networks.
  • the neural network can be a convolutional neural network (CNN), a long short-term memory network (long short-term memory, LSTM), or a convolutional long short-term memory network (convolutional long short-term memory, conv-LSTM), etc.
  • the neural network can be composed of multiple processing layers or multiple long short term memory network (long short term memory, LSTM) units.
  • the functions implemented by part of the processing layers or part of the LSTM units that make up the neural network may correspond to the functions implemented by one of the multiple classifiers. When the amount of data to be processed is large, the processing results of the neural network will be more accurate than the processing results of multiple classifiers.
  • the motion characteristics determined by the static motion classifier include: static and motion.
  • the stationary motion classifier can also derive a second motion probability that the key point is in the motion feature, which can be represented by a confidence level, for example. Among them, confidence can also be called reliability, confidence level, and confidence coefficient. Confidence indicates how credible a certain estimation result is. For example, the stationary motion classifier classifies the motion trajectory of a key point and concludes that the key point is stationary and the confidence that it is stationary is 85%; this means that the key point has an 85% probability (or called probability) is at rest. This stationary motion classifier is more accurate in determining whether a key point is in a stationary/moving state.
  • the motion characteristics determined by the non-parallel motion classifier include: lateral driving, cut in motion, and cut out motion.
  • the non-parallel motion classifier can also derive a second motion probability that the key point is in the motion feature, which can be represented by a confidence level, for example.
  • lateral travel means that the movement direction of the detection target is approximately perpendicular to the movement direction of the mobile station itself
  • cut-in movement means the movement route of the key point entering the mobile station
  • cut-out movement means the movement route of the key point leaving the mobile station.
  • the motion angle classifier determines the motion direction of key points more accurately.
  • the motion characteristics determined by the high-speed parallel motion classifier include: rapid motion, high-speed motion, accelerated motion, and rapid acceleration motion.
  • the high-speed parallel motion classifier can also obtain a second motion probability that the key point is in the motion feature, which can be represented by a confidence level, for example.
  • the determination result of the high-speed parallel motion classifier can be determined based on the calculated speed and acceleration of the key point. When the key points are in high-speed motion, the judgment results of the high-speed parallel motion classifier are more accurate.
  • the motion characteristics determined by the low-speed motion classifier include: slow motion, deceleration motion, and rapid deceleration motion.
  • the low-speed motion classifier can also derive a second motion probability that the key point is in the motion feature, which can be represented by a confidence level, for example.
  • the determination result of the low-speed parallel motion classifier can be determined based on the calculated speed and acceleration of the key point. When the key point is in low-speed motion, the judgment result of the low-speed motion classifier is more accurate.
  • the motion characteristics determined by the motion angle classifier include: turning, U-turn, parallel driving, and forward/reverse driving.
  • the motion angle classifier can also obtain a second motion probability that the key point is in the motion feature, which can be represented by a confidence level, for example.
  • the motion angle classifier determines the angular velocity of key points more accurately. In particular, if the detection target is closer to the terminal, the determination result obtained by the motion angle classifier is more accurate.
  • the moving corner point classifier can determine the movement angle of the key point based on the relationship between the convergence point generated by the key point and the optical flow convergence point of the static scene, and then calculate the angular velocity of the key point.
  • the key points of the detection target are points with specific characteristics on the contour of the detection target, they are more closely related to the detection target.
  • the convergence points generated based on the key points It will be more accurate than detecting the optical flow convergence point of the optical flow on the target. In this way, the accuracy of determining the movement angle of key points can be improved.
  • the motion characteristics determined by the motion scale classifier include: forward/reverse acceleration.
  • the motion scale classifier can also derive a second motion probability that the key point is in the motion feature, which can be represented by a confidence level, for example. It should be noted that the higher the confidence level, the greater the acceleration of the key point.
  • the motion scale classifier is more accurate in determining the acceleration of key points. In particular, if the detection target is far away from the terminal, the determination result obtained by the motion scale classifier is more accurate.
  • the motion characteristics determined by the retrograde motion classifier include: reverse driving and forward driving.
  • the retrograde motion classifier can also derive a second motion probability that the key point is in the motion feature, which can be represented by a confidence level, for example. This retrograde motion classifier is more accurate in determining whether a key point is traveling in the reverse direction/forward direction.
  • the multiple motion classifiers introduced above make use of information such as pixel scale changes, real scale changes, angle changes, etc. of the detected target. Compared with methods that use multi-frame optical flow to determine motion status, they are more robust.
  • Various motion features of the key point can be obtained through the multiple classifiers.
  • the multiple motion features can jointly determine the motion state of the key point.
  • the multiple second motion probabilities can jointly determine whether the key point is in a certain motion state.
  • First motion probability That is to say, the motion state of the key point is determined based on multiple motion characteristics, and the first motion probability is determined based on the second motion probabilities of the multiple key points.
  • the motion detection result of the key point includes the motion state of the key point and the first motion probability.
  • the movement state includes one or more of the following: forward fast movement state, forward slow movement state, stationary state, forward cut-in state, forward cut-out state , High-speed retrograde motion state, Low-speed retrograde motion state, Forward acceleration motion state, Forward deceleration motion state, Forward sharp acceleration motion state, Forward sharp deceleration motion state, Emergency stop state, Crossing state, Parallel motion state, Lateral movement Turning state, longitudinal movement state, U-turn state, parameter information describing the movement state.
  • the parameter information describing the motion state includes one or more of the following: absolute speed information, relative speed information, and time-to-collision (TTC) information.
  • TTC time-to-collision
  • the movement state determined by movement characteristics such as motion, fast movement, and forward driving may be a "forward fast movement state.”
  • the motion state determined by the motion characteristics such as motion, slow motion and forward driving can be a “forward slow motion state”.
  • the motion state determined by the motion characteristics of motion, rapid deceleration, and forward driving can be the "forward rapid deceleration motion state.”
  • the motion state determined by motion characteristics such as motion and motion angle can be a "U-turn state”.
  • the first motion probability of the key point can represent the probability that a key point is in a certain determined motion state, and can provide a basis for subsequent determination of the first motion probability of the detection target.
  • the first motion probability may be an arithmetic average or a weighted average of the plurality of second motion probabilities.
  • the first motion probability may be calculated from part of the second motion probability. For example, the second motion probability that is too small can be filtered out. For example, if the second motion probability of slow motion is 5%, it means that the key point is unlikely to be in slow motion. Calculate the detection target When the first movement probability of the slow movement is obtained, the second movement probability of the slow movement may not be considered.
  • the multiple classifiers can be connected through cascade or average integration.
  • the multiple classifiers can also be connected through partial cascade and partial average integration.
  • FIG. 5 is a schematic diagram of the connection relationships of some multiple classifiers provided by the embodiment of the present application.
  • the classifiers included in Figure 5 are: stationary motion classifier, non-parallel motion classifier, high-speed parallel motion classifier, low-speed motion classifier, motion angle classifier, motion scale classifier and retrograde motion classifier.
  • the multiple classifiers may include one or more of a motion angle classifier, a motion scale classifier, and a retrograde motion classifier.
  • (1) in Figure 5 illustrates a schematic diagram in which multiple classifiers are connected in a cascade manner.
  • the judgment results obtained by each classifier can be used as a reference for each other.
  • the correlation of the judgment results of different classifiers for a certain type or several types of motion states can be fully utilized.
  • the judgment results obtained by the high-speed parallel motion classifier (which can judge acceleration motion) and the motion scale classifier (which can judge acceleration) have strong correlation and can serve as reference for each other's judgment. Get more accurate results.
  • (2) in Figure 5 illustrates a schematic diagram in which multiple classifiers are connected through average integration. In this connection method of average integration, the judgment results obtained by each classifier are obtained independently.
  • FIG. 5 illustrates a schematic diagram in which multiple classifiers are connected through partial cascade and partial average integration.
  • classifiers with strong correlations can be connected in a cascade manner
  • classifiers with weak correlations can be connected in an average integration manner.
  • connection order of each classifier can be adjusted according to actual application requirements, and is not limited in this application.
  • the motion detection result of each key point also needs to be determined based on the motion parameters of the mobile station. That is to say, the method also includes: obtaining motion parameters of the mobile station. Based on the motion parameters of the mobile station, the motion trajectory of each key point is classified and processed through multiple classifiers to obtain the motion detection result of each key point.
  • the motion parameters of the mobile station may include the speed, acceleration, angular velocity and other information of the mobile station, which may be obtained through various sensors of the mobile station. Examples may be GPS, speed sensor, acceleration sensor, angular velocity sensor, rotation sensor, etc. Moment sensor, inertial measurement unit (IMU), global navigation satellite system (GNSS) and speed sensor (WSS), etc.
  • the motion parameters of the mobile station and the motion trajectories of key points are used as inputs to the multiple motion classifiers, and jointly determine the output results of the motion classifiers.
  • the motion parameters of the mobile station are taken into consideration, the influence of the motion of the mobile station itself on determining the motion state of the detection target can be eliminated, and more accurate motion detection results can be obtained.
  • the weights of the multiple classifiers may be different.
  • the weights of the multiple classifiers can be obtained through training using a boosting algorithm.
  • the weights of the multiple classifiers can also have different values in different driving scenarios, so as to adapt to different driving scenarios and obtain more accurate judgment results.
  • the driving scenario may include straight driving, curved driving, uphill driving, downhill driving, etc.
  • the motion detection result of the detected target includes a motion state of the detected target and a first motion probability.
  • the first motion probability is used to indicate the probability that the detected target is in a certain motion state.
  • the first motion probability and motion state of the detected target may be the first motion probability and its corresponding motion state with the largest value among the motion detection results of each key point.
  • the movement state of the first key point is a forward fast movement state, and the first movement probability is 80%;
  • the movement state of the second key point is a forward fast movement state, and the first movement probability is 80%.
  • the motion probability is 85%;
  • the motion state of the third key point is the forward acceleration motion state, and the first motion probability is 70%; then, the motion state of the detected target can be determined to be the forward rapid motion state (the first motion of the maximum value The motion state corresponding to the probability), the first motion probability of the detected target is 85%.
  • the motion detection result of the detection target may also be a motion detection result obtained by processing the motion detection result of each key point through a filter (or other algorithm system).
  • a filter or other algorithm system
  • it can be a Kalman filter (kalma filter).
  • Kalman filtering is an algorithm that uses linear system state equations to optimally estimate the system state through system input and output observation data. Since the observation data includes the influence of noise and interference in the system, the optimal estimation can also be regarded as a filtering process.
  • Data filtering is a data processing technology that removes noise and restores real data. Kalman filtering can estimate the state of a dynamic system from a series of data with measurement noise when the measurement variance is known, and can conduct real-time processing of data collected on site. updates and processing.
  • the motion detection result of the detection target can also be combined with the motion characteristics determined based on the bounding box of the detection target to jointly determine a more accurate motion detection result of the detection target.
  • the method also includes: determining at least one feature coordinate point in the bounding box; classifying the motion trajectory of each feature coordinate point through a bounding box motion classifier to obtain the detection target relative to the mobile station the relative speed, and the third motion probability at the relative speed; updating the motion detection result of the detection target based on the relative speed and the third motion probability.
  • the bounding box is a bounding box of a preset shape that may contain the detection target.
  • the bounding box may be a rectangle.
  • the feature coordinate point can be a point that is easy to find in a preset graphic and has a special positional relationship relative to the bounding box. For example, it can be the vertex of the bounding box, the center point, the midpoint of the sides that constitute the bounding box, and so on.
  • the movement trajectory of this feature coordinate point can refer to the movement trajectory of the key points introduced in the above content, which will not be described again here.
  • the bounding box motion classifier can obtain the relative speed of the detection target relative to the mobile station and the third motion probability at the relative speed based on the motion trajectory of each feature coordinate point.
  • the third motion probability can be represented by a confidence level. It should be noted that the higher the confidence level, the greater the relative speed of the detected target.
  • the bounding box motion classifier is more accurate in determining the relative speed of the detected target.
  • the relative speed of the detected target in the motion detection result of the detected target may be updated based on the relative speed and the third motion probability.
  • the key points are points with specific characteristics on the contour of the detection target, some key points may be occluded during the acquisition process, but the characteristic coordinate points on the bounding box are easy to find, and the bounding box It has a strong correlation with the detection target, and the motion characteristics of the bounding box can better reflect the motion characteristics of the detection target. Therefore, when the movement trajectories of key points are less, the bounding box motion classifier can obtain more accurate results. Accurate judgment results.
  • the motion detection result of the detection target obtained at the current time can also be combined with the motion detection result of the detection target obtained at the previous time to jointly determine a more accurate motion detection result of the detection target.
  • the motion detection result of the detection target corresponds to the first moment.
  • the multiple images used as the basis for detection correspond to the first time.
  • the multiple images are images taken within the minute of 14:01-14:02, and the first time is 14:01-14 :02.
  • the method in the embodiment of the present application also includes: based on the motion detection result of the detection target corresponding to the second time and the motion detection result of the detection target, updating the motion detection result of the detection target corresponding to the first time, the second time before that first moment.
  • the second time may be 14:00-14:01
  • the motion detection result of the detection target corresponding to the second time may be determined based on images taken within the minute of 14:00-14:01.
  • the motion detection result obtained and the motion detection result of the detection target obtained at the preceding moment of the second moment are jointly determined.
  • the second time can also be multiple times before the first time.
  • the second time can be 13:58-14:01, and the second time corresponds to the movement of the detected target.
  • the detection results include the motion detection results of the detection targets corresponding to the three times: 13:58-13:59, 13:59-14:00, and 14:00-14:01.
  • the motion detection result of the detection target corresponding to 13:58-13:59 is "86% is in a forward acceleration motion state"
  • the motion detection result of the detection target corresponding to 13:58-13:59 The result is "89% is in a state of rapid acceleration in the forward direction”
  • the motion detection result of the detection target corresponding to 14:01-14:01 is "92% is in a state of rapid acceleration in the forward direction”
  • the motion detection result of the detection target obtained at all times is "83% is in a state of forward rapid acceleration”.
  • the updated can also be understood as the final determination
  • the motion detection result of the detection target corresponding to 14:01-14:02, for example, can be "94% is in a forward rapid acceleration motion state".
  • multiple classifiers are used to process the motion trajectory of at least one key point on the detection target, and the motion detection results of each key point can be obtained.
  • the detection target is then determined based on the motion detection results of each key point.
  • motion detection results are more accurate.
  • the motion trajectory calculated based on key points is less time-consuming and more accurate than the motion trajectory calculated based on optical flow. It is more consistent with the movement of the real detection target and provides more reliable input for subsequent motion detection;
  • multiple motion classifiers use information such as pixel scale changes, real scale changes, angle changes, etc. of the detected target. Compared with the method that uses multi-frame optical flow to judge motion status, it is more robust, and we get The motion detection results are more accurate.
  • the motion state determined by the embodiment of the present application has a more detailed description of the characteristics compared to simple motion states such as stationary and moving, as well as simple speed and acceleration information.
  • more accurate motion detection results of the detection target can be obtained, thereby providing a more accurate adjustment basis for the adjustment module of the mobile station, and better assisting the driving of the mobile station.
  • the execution body of the method may include a hardware structure, a software module, and implement the above functions in the form of a hardware structure, a software module, or a hardware structure plus a software module.
  • a certain function among the above functions can be executed by a hardware structure, a software module, or a hardware structure plus a software module.
  • the detection device may be a device in the mobile station.
  • the detection device 60 includes an acquisition unit 601, a first determination unit 602, a second determination unit 603, a processing unit 604 and a third determination unit 605:
  • the acquisition unit 601 is used to acquire multiple images, where the images include detection targets.
  • the operation performed by the acquisition unit 601 can be referred to the introduction of the content of step S101 in Figure 2 above.
  • the first determining unit 602 is used to determine at least one key point on the outline of the detection target.
  • the operation performed by the first determining unit 602 may be referred to the introduction of the content of step S102 in FIG. 2 above.
  • the second determining unit 603 is used to determine the motion trajectory of each key point.
  • the operation performed by the second determination unit 603 may refer to the introduction of the content of step S103 in FIG. 2 above.
  • the processing unit 604 is used to classify the motion trajectory of each key point through multiple classifiers to obtain the motion detection result of each key point; wherein the multiple classifiers include: stationary motion classifier, non-moving classifier Parallel motion classifier, high-speed parallel motion classifier, low-speed motion classifier; the multiple classifiers also include one or more of the following: motion angle classifier, motion scale classifier, and retrograde motion classifier.
  • the operations performed by the processing unit 604 can be referred to the introduction of the content of step S104 in FIG. 2 above.
  • the third determining unit 605 is configured to determine the motion detection result of the detection target based on the motion detection result of each key point.
  • the operation performed by the third determination unit 605 may be referred to the introduction of the content of step S105 in FIG. 2 above.
  • the motion detection result includes a motion state and a first motion probability, where the first motion probability is used to indicate a probability of being in the motion state.
  • one of the classifiers is used to determine a motion feature of the key point, and a second motion probability of the key point being in the motion feature; the motion state is determined based on a plurality of the motion features, and the third A motion probability is determined based on a plurality of the second motion probabilities.
  • the movement state includes one or more of the following: forward fast movement state, forward slow movement state, stationary state, forward cut-in state, forward cut-out state out state, high-speed retrograde movement state, low-speed retrograde movement state, forward acceleration movement state, forward deceleration movement state, forward rapid acceleration movement state, forward rapid deceleration movement state, emergency stop state, crossing state, parallel movement state , transverse turning state, longitudinal movement state, U-turn state, parameter information describing the movement state; wherein, the parameter information describing the movement state includes one or more of the following: absolute speed information, relative speed information, and collision time information.
  • the motion characteristics determined by the stationary motion classifier include: stationary and moving; the motion characteristics determined by the non-parallel motion classifier include: lateral traveling, cutting in, and cutting out; the high-speed parallel motion
  • the motion characteristics determined by the classifier include: fast, high speed, acceleration, and sharp acceleration; the motion characteristics determined by the low-speed motion classifier include: slow speed, deceleration, and sharp deceleration;
  • the motion characteristics determined by the motion angle classifier include: Determine turning, U-turn, parallel driving, forward/reverse motion; the motion characteristics determined by the motion scale classifier include: forward/reverse acceleration; the motion characteristics determined by the retrograde motion classifier include: reverse driving, forward driving .
  • the motion detection result of the detection target corresponds to the first time
  • the third determination unit 605 is further configured to: based on the motion detection result of the detection target corresponding to the second time and the motion detection result of the detection target.
  • the motion detection result updates the motion detection result of the detection target corresponding to the first time, and the second time is before the first time.
  • the camera that acquires the multiple images is located on the mobile station, and the acquisition unit 601 is also used to: acquire the motion parameters of the mobile station; the processing unit 604 is specifically used to: based on the mobile station Motion parameters are used to classify the motion trajectory of each key point through multiple classifiers to obtain the motion detection result of each key point.
  • the first determination unit 602 is specifically configured to: perform target detection on the area of interest in the image, determine a bounding box containing the detection target; process the image in the bounding box to determine the detection
  • the outline of the target and the type of the detection target which type includes one or more of the following: vehicles, pedestrians, buildings, roadblocks; determine at least one key point on the outline of the detection target based on the outline and the type, the The keypoint's location on the contour is determined based on the type.
  • the camera that acquires multiple images is located on the mobile platform, and the third determination unit 605 is further configured to: determine at least one feature coordinate point in the bounding box; Classify the motion trajectories of the characteristic coordinate points to obtain the relative speed of the detection target relative to the mobile station and the third motion probability at the relative speed; update the detection target based on the relative speed and the third motion probability motion detection results.
  • the plurality of images include a first image at a third time and a second image at a fourth time
  • the second determining unit 603 is specifically configured to: determine whether each key point is at the third time. First position information in an image; determining second position information of each key point in the second image; determining the movement trajectory of each key point based on the first position information and the second position information.
  • the multiple classifiers are connected through cascade or average integration.
  • each unit of the detection device 60 shown in Figure 6 can be implemented in hardware, software, or a combination of software and hardware.
  • the functions of each unit in the above content may be implemented by one or more processors in the detection device 60 .
  • FIG 7 is a schematic structural diagram of another detection device provided by an embodiment of the present application.
  • the detection device 70 can be used to implement the method described in the above method embodiment. For details, please refer to the description in the above method embodiment.
  • the detection device 70 may include one or more processors 701 .
  • the processor 701 may be a general-purpose processor or a special-purpose processor.
  • the processor 701 can be used to control the detection device, execute software programs, and process data of the software programs.
  • the detection device 70 may include one or more memories 702, on which program code 704 may be stored.
  • the program code may be run on the processor 701, so that the detection device 70 executes the above method embodiments. method described in .
  • the memory 702 may also store data.
  • the processor 701 and the memory 702 can be set up separately or integrated together.
  • the memory 702 can also be located outside the detection device 70 and coupled with the detection device 70 in some ways.
  • the detection device 70 may also include a transceiver 705.
  • the transceiver 705 may be called a transceiver unit, a transceiver, a transceiver circuit, etc., and is used to implement transceiver functions.
  • the transceiver 705 may include a receiver and a transmitter.
  • the receiver may be called a receiver or a receiving circuit, etc., used to implement the receiving function;
  • the transmitter may be called a transmitter, a transmitting circuit, etc., used to implement the transmitting function.
  • Processor 701 configured to receive multiple images through the transceiver 705, where the images include detection targets; and to determine at least one key point on the outline of the detection target; to determine the motion trajectory of each key point; through multiple classifications
  • the motion trajectory of each key point is classified and processed to obtain the motion detection result of each key point; among them, the multiple classifiers include: static motion classifier, non-parallel motion classifier, high-speed parallel motion classifier, Low-speed motion classifier; the multiple classifiers also include one or more of the following: motion angle classifier, motion scale classifier, bounding box motion classifier, retrograde motion classifier; motion detection results based on each key point Determine the motion detection result of the detection target.
  • the detection device 70 may be a device in the mobile station, or may be a chip, chip system, or processor that supports the device in the mobile station to implement the above method.
  • the operations performed by the detection device 70 may refer to the relevant content about the detection module in the method embodiment corresponding to FIG. 2, which will not be described in detail here.
  • the transceiver 705 may be a transceiver circuit, an interface, or an interface circuit.
  • the transceiver circuits, interfaces or interface circuits used to implement the receiving and transmitting functions can be separate or integrated together.
  • the above-mentioned transceiver circuit, interface or interface circuit can be used for reading and writing codes/data, or the above-mentioned transceiver circuit, interface or interface circuit can be used for signal transmission or transfer.
  • the processor 701 can store program code 703, and the program code 703 runs on the processor 701, which can cause the detection device 70 to perform the method described in the above method embodiment.
  • the program code 703 may be solidified in the processor 701, in which case the processor 701 may be implemented by hardware.
  • the detection device 70 may include a circuit, which may implement the sending or receiving or communication functions in the foregoing method embodiments.
  • the processor and transceiver described in this application can be implemented in integrated circuits (ICs), analog ICs, radio frequency integrated circuits RFICs, mixed signal ICs, application specific integrated circuits (ASICs), printed circuit boards ( printed circuit board (PCB), electronic equipment, etc.
  • ICs integrated circuits
  • analog ICs analog ICs
  • radio frequency integrated circuits RFICs radio frequency integrated circuits
  • mixed signal ICs mixed signal ICs
  • ASICs application specific integrated circuits
  • PCB printed circuit board
  • electronic equipment etc.
  • the detection device described in the above embodiments may be a network device or a terminal device, but the scope of the detection device described in this application is not limited thereto, and the structure of the detection device may not be limited by FIG. 7 .
  • the detection device may be a stand-alone device or may be part of a larger device.
  • the detection device can be:
  • An independent integrated circuit IC, or chip, or chip system or subsystem (2) A collection of one or more ICs.
  • the IC collection may also include an integrated circuit for storing data and program codes. Storage components; (3) ASIC, such as modem; (4) Modules that can be embedded in other devices; (5) Receivers, smart terminals, wireless devices, handheld machines, mobile units, vehicle-mounted equipment, cloud equipment, Artificial Intelligence devices and more.
  • the detection device can be a chip or a chip system
  • the detection device can be a chip or a chip system
  • the schematic structural diagram of the chip shown in FIG. 8 includes a logic circuit 801 and an input and output interface 802.
  • the number of logic circuits 801 may be one or more, and the number of input-output interfaces 802 may be multiple.
  • the logic circuit 801 can be used to obtain multiple images through the input and output interface 802, and the images include detection targets.
  • the logic circuit 801 can also be used to determine at least one key point on the outline of the detection target; determine the movement trajectory of each key point; classify the movement trajectory of each key point through multiple classifiers to obtain each key point.
  • Motion detection results of key points wherein, the multiple classifiers include: stationary motion classifier, non-parallel motion classifier, high-speed parallel motion classifier, low-speed motion classifier; the multiple classifiers also include one of the following or Multiple items: motion angle classifier, motion scale classifier, bounding box motion classifier, retrograde motion classifier; determine the motion detection result of the detection target based on the motion detection result of each key point.
  • the operations performed by the logic circuit 801 may refer to the relevant content in the method embodiment corresponding to FIG. 2 above, which will not be described in detail here.
  • This application also provides a computer-readable storage medium on which a computer program is stored. When executed by a computer, the computer-readable storage medium realizes the functions of any of the above method embodiments.
  • This application also provides a computer program product, which implements the functions of any of the above method embodiments when executed by a computer.
  • the above embodiments it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application are generated in whole or in part.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted over a wired connection from a website, computer, server, or data center (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website, computer, server or data center.
  • the computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center integrated with one or more available media.
  • the available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVD)), or semiconductor media (e.g., solid state disks (SSD) ))wait.
  • magnetic media e.g., floppy disks, hard disks, tapes
  • optical media e.g., high-density digital video discs (DVD)
  • semiconductor media e.g., solid state disks (SSD)
  • Predefinition in this application can be understood as definition, pre-definition, storage, pre-storage, pre-negotiation, pre-configuration, solidification, or pre-burning.

Landscapes

  • Image Analysis (AREA)

Abstract

一种目标的检测方法及相关装置，可应用于自动驾驶或者辅助驾驶。该方法包括：获取多个图像，该图像包括检测目标；确定该检测目标的轮廓上的至少一个关键点；确定每个关键点的运动轨迹；通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果；其中，该多个分类器包括：静止运动分类器、非平行运动分类器、高速平行运动分类器、低速运动分类器；该多个分类器还包括以下一项或者多项：运动角度分类器、运动尺度分类器、逆行运动分类器；基于该每个关键点的运动检测结果确定该检测目标的运动检测结果。通过本方案，可以得到更加精确的检测目标的运动检测结果。

Description

一种目标的检测方法及相关装置 技术领域
本申请涉及计算机技术领域，尤其涉及一种目标的检测方法及相关装置。
背景技术
随着智能驾驶的不断发展，越来越多的车辆上配置有相机，可以通过相机对车辆周围的环境进行观测，进而辅助驾驶系统做出各种驾驶决策。示例性的，相机对车辆周围的环境中移动的其他物体(例如，车辆、障碍物、行人，等等)进行监测，通过计算可以估计出其他物体的运动状态，进而辅助驾驶系统做出各种驾驶决策(例如，防碰撞刹车、减速，等等)。
为了估计出其他物体的运动状态，现阶段提出了基于多帧光流进行运动状态判断的方法，该方法的原理是：在目标物体上提取角点，不同帧的图像上的角点形成多帧光流，选取最低、最靠近地面的光流点计算目标物体在两帧间的运动距离，该运动距离若小于车辆自身的运动距离，则判定该目标物体静止，否则判定为运动；之后，再根据目标物体上的光流的光流汇聚点与消失点计算目标物体的相对运动角度，进而确定出目标物体的运动状态。
但在这种方法中，由于角点是图像中的极值点，即在某方面属性特别突出的点，例如在某些属性上强度最大或者最小的孤立点、线段的终点。角点的确定与图像的拍摄质量以及角点的提取算法有关，有可能确定出的角点并不能有效代表整体目标物体的运动状态，这样会导致确定出的目标物体的运动状态的误差较大；另外，由于在这种方式中将最低、最靠近地面的角点近似为地面上的角点，可能会导致计算出的目标物体的运动距离与该目标物体真实的运动距离差异较大。
发明内容
本申请实施例提供一种目标的检测方法及相关装置，可以得到更加精确的检测目标的运动检测结果。
第一方面，本申请提供了一种目标的检测方法，该方法包括：获取多个图像，该图像包括检测目标；确定该检测目标的轮廓上的至少一个关键点；确定每个关键点的运动轨迹；通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果；其中，该多个分类器包括：静止运动分类器、非平行运动分类器、高速平行运动分类器、低速运动分类器；该多个分类器还包括以下一项或者多项：运动角度分类器、运动尺度分类器、逆行运动分类器；基于该每个关键点的运动检测结果确定该检测目标的运动检测结果。
在本方案中，该检测目标的关键点为该检测目标的轮廓上具有特定特征的点(示例性的，车辆的关键点可以包括车辆的后视镜，轮胎接地点，车灯，等等)，基于该关键点计算的运动轨迹相比光流等计算的运动轨迹，更能符合真实的检测目标的运动，可以为后续的运动检测提供更可靠的输入。一个分类器对关键点的一个运动特征的判定结果较为准确，联合多个分类器的判定结果，可以得到更加准确的该关键点的运动检测结果。进一步的，通过每个关键点的运动检测结果共同确定检测目标的运动检测结果，可以得到更加精确的检测目标的运动检测结果。
示例地，该图像可以为图片，照片或者视频，流媒体等。该检测目标可以是车辆、行人、建筑物、路障，等等。该检测目标可以理解为可能会对移动台的行动造成影响的人或物体。
结合第一方面，在一种可能的实现方式中，该运动检测结果包括运动状态和第一运动概率，该第一运动概率用于指示处于该运动状态的概率。示例性的，第一运动概率可以为置信度。通过该运动检测结果，可以确定出检测目标的运动状态，以及该检测目标处于该运动状态的概率。进一步的，该运动检测结果可以作为其他设备或者装置的输入数据，示例性的，可以作为移动台的调整模块(例如，先进驾驶辅助系统、整车控制器，等等)的输入，为其提供准确的检测目标的运动检测结果，可以更好地辅助移动台的行驶。
结合第一方面，在一种可能的实现方式中，一个该分类器用于确定该关键点的一个运动特征，和该关键点处于该运动特征的第二运动概率；该运动状态基于多个该运动特征确定，该第一运动概率基于多个该第二运动概率确定。其中，一个分类器对关键点的一个运动特征的判定结果较为准确，联合多个分类器的判定结果，可以得到更加准确的该关键点的运动检测结果。
结合第一方面，在一种可能的实现方式中，该运动状态包括以下一项或者多项：正向快速运动状态，正向慢速运动状态，静止状态，正向切入cut-in状态，正向切出cut-out状态，高速逆行运动状态，低速逆行运动状态，正向加速运动状态，正向减速运动状态，正向急加速运动状态，正向急减速运动状态，急停状态，横穿状态，平行运动状态，横向转弯状态，纵向运动状态，掉头状态，描述运动状态的参数信息；其中，该描述运动状态的参数信息包括以下一项或者多项：绝对速度信息，相对速度信息，碰撞时间信息。在本实施方式中，确定出的运动状态相比于静止、运动这类简单的运动状态以及单纯的速度、加速度信息而言，具有更加具体细节的特征描述，通过这种方式，可以提供更加有效准确的运动检测结果。
结合第一方面，在一种可能的实现方式中，该静止运动分类器确定出的运动特征包括：静止、运动；该非平行运动分类器确定出的运动特征包括：横向行驶、切入、切出；该高速平行运动分类器确定出的运动特征包括：快速、高速、加速、急加速；该低速运动分类器确定出的运动特征包括：慢速、减速、急减速；该运动角度分类器确定出的运动特征包括：判断转弯、掉头、平行行驶、正向/逆向运动；该运动尺度分类器确定出的运动特征包括：正向/逆向加速度；该逆行运动分类器确定出的运动特征包括：逆向行驶、正向行驶运动。其中，多个运动分类器利用了该检测目标的像素尺度变化，真实尺度变化，角度变化等方面的信息，相比利用多帧光流进行运动状态判断的方法，其鲁棒性更高。
结合第一方面，在一种可能的实现方式中，该检测目标的运动检测结果与第一时刻对应，该方法还包括：基于第二时刻对应的该检测目标的运动检测结果和该检测目标的运动检测结果，更新该第一时刻对应的该检测目标的运动检测结果，该第二时刻在该第一时刻之前。通过这种方式，可以依据检测目标运动的连续性，确定出更加准确的运动检测结果。
结合第一方面，在一种可能的实现方式中，获取该多个图像的相机位于移动台上，通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果，包括：获取该移动台的运动参数；基于该移动台的运动参数，通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果。通过这种方式，由于考虑到了移动台的运动参数，可以消除移动台自身的运动对判定该检测目标的运动状态的影响，该检测模块可以得到更加准确的运动检测结果。
结合第一方面，在一种可能的实现方式中，该确定该检测目标的轮廓上的至少一个关键点，包括：对该图像中的感兴趣区域进行目标检测，确定包含该检测目标的包围框；处理该 包围框中的图像确定该检测目标的轮廓和该检测目标所属的类型，该类型包括以下一项或者多项：车辆、行人、建筑物、路障；基于该轮廓和该类型确定该检测目标的轮廓上的至少一个关键点，该关键点在该轮廓上的位置基于该类型确定。
其中，该检测目标的关键点为该检测目标的轮廓上具备特定特征的点，可以通过确定检测目标所属的类型，更加便捷地确定出关键点在该检测目标的轮廓上的位置，能够提升确定关键点的准确性，并减少处理时长。通过感兴趣区域的设置，可以从原始图像中确定出需要重点关注的区域，减少处理时间，增加精度。进一步的，检测目标的包围框是可以包含检测目标的，相比于感兴趣区域面积更小的图像区域。通过包围框的设置，可以进一步的减少需要处理的图像范围，减少处理时间，增加精度。
结合第一方面，在一种可能的实现方式中，获取该多个图像的相机位于移动台上，该方法还包括：确定该包围框中的至少一个特征坐标点；通过包围框运动分类器对该每个特征坐标点的运动轨迹进行分类处理，得到该检测目标的相对该移动台的相对速度，以及处于该相对速度的第三运动概率；基于该相对速度，以及该第三运动概率更新该检测目标的运动检测结果。通过这种方式，由于包围框上的特征坐标点是易于查找到的，且该包围框与该检测目标具有强关联性，该包围框的运动特征可以较好地体现该检测目标的运动特征，因此，在关键点的运动轨迹较少的情况下，该包围框运动分类器可以得到更准确的判定结果。
结合第一方面，在一种可能的实现方式中，该多个图像包括第三时刻的第一图像，以及第四时刻的第二图像，该确定每个关键点的运动轨迹，包括：确定该每个关键点在该第一图像中的第一位置信息；确定该每个关键点在该第二图像中的第二位置信息；根据该第一位置信息和该第二位置信息确定该每个关键点的运动轨迹。可选的，该运动轨迹还可以称为运动场、运动向量，等等。为了获取更多的信息，也可以通过更多时刻拍摄的图像来确定关键点的运动轨迹，本申请实施例不作限制。
结合第一方面，在一种可能的实现方式中，该多个分类器通过级联或者平均集成的方式连接。可选的，该多个分类器还可以通过部分级联，部分平均集成的方式进行连接。在级联的这种连接方式中，各个分类器得到的判定结果可以互为参考依据，通过这种方式，可以充分利用不同分类器对于某一类或几类运动状态的判定结果的相关联性。在平均集成的这种连接方式中，各个分类器得到的判定结果单独得出，若一个分类器的判定结果并不太准确，也不会对其他分类器的判定结果造成影响。在部分级联，部分平均集成的这种连接方式中，具有强关联性的分类器可以采用级联的方式，关联性较弱的分类器之间采用平均集成的连接方式。
第二方面，本申请提供了一种检测装置，该检测装置包括获取单元、第一确定单元、第二确定单元、处理单元和第三确定单元：该获取单元，用于获取多个图像，该图像包括检测目标；该第一确定单元，用于确定该检测目标的轮廓上的至少一个关键点；该第二确定单元，用于确定每个关键点的运动轨迹；该处理单元，用于通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果；其中，该多个分类器包括：静止运动分类器、非平行运动分类器、高速平行运动分类器、低速运动分类器；该多个分类器还包括以下一项或者多项：运动角度分类器、运动尺度分类器、逆行运动分类器；该第三确定单元，用于基于该每个关键点的运动检测结果确定该检测目标的运动检测结果。
结合第二方面，在一种可能的实现方式中，该运动检测结果包括运动状态和第一运动概率，该第一运动概率用于指示处于该运动状态的概率。
结合第二方面，在一种可能的实现方式中，一个该分类器用于确定该关键点的一个运动 特征，和该关键点处于该运动特征的第二运动概率；该运动状态基于多个该运动特征确定，该第一运动概率基于多个该第二运动概率确定。
结合第二方面，在一种可能的实现方式中，该运动状态包括以下一项或者多项：正向快速运动状态，正向慢速运动状态，静止状态，正向切入cut-in状态，正向切出cut-out状态，高速逆行运动状态，低速逆行运动状态，正向加速运动状态，正向减速运动状态，正向急加速运动状态，正向急减速运动状态，急停状态，横穿状态，平行运动状态，横向转弯状态，纵向运动状态，掉头状态，描述运动状态的参数信息；其中，该描述运动状态的参数信息包括以下一项或者多项：绝对速度信息，相对速度信息，碰撞时间信息。
结合第二方面，在一种可能的实现方式中，该静止运动分类器确定出的运动特征包括：静止、运动；该非平行运动分类器确定出的运动特征包括：横向行驶、切入、切出；该高速平行运动分类器确定出的运动特征包括：快速、高速、加速、急加速；该低速运动分类器确定出的运动特征包括：慢速、减速、急减速；该运动角度分类器确定出的运动特征包括：判断转弯、掉头、平行行驶，正向/逆向运动；该运动尺度分类器确定出的运动特征包括：正向/逆向加速度；该逆行运动分类器确定出的运动特征包括：逆向行驶、正向行驶。
结合第二方面，在一种可能的实现方式中，该检测目标的运动检测结果与第一时刻对应，该第三确定单元还用于：基于第二时刻对应的该检测目标的运动检测结果和该检测目标的运动检测结果，更新该第一时刻对应的该检测目标的运动检测结果，该第二时刻在该第一时刻之前。
结合第二方面，在一种可能的实现方式中，获取该多个图像的相机位于移动台上，该获取单元还用于：获取该移动台的运动参数；该处理单元具体用于：基于该移动台的运动参数，通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果。
结合第二方面，在一种可能的实现方式中，该第一确定单元具体用于：对该图像中的感兴趣区域进行目标检测，确定包含该检测目标的包围框；处理该包围框中的图像确定该检测目标的轮廓和该检测目标所属的类型，该类型包括以下一项或者多项：车辆、行人、建筑物、路障；基于该轮廓和该类型确定该检测目标的轮廓上的至少一个关键点，该关键点在该轮廓上的位置基于该类型确定。
结合第二方面，在一种可能的实现方式中，获取该多个图像的相机位于移动台上，该第三确定单元还用于：确定该包围框中的至少一个特征坐标点；通过包围框运动分类器对该每个特征坐标点的运动轨迹进行分类处理，得到该检测目标的相对该移动台的相对速度，以及处于该相对速度的第三运动概率；基于该相对速度，以及该第三运动概率更新该检测目标的运动检测结果。
结合第二方面，在一种可能的实现方式中，该多个图像包括第三时刻的第一图像，以及第四时刻的第二图像，该第二确定单元具体用于：确定该每个关键点在该第一图像中的第一位置信息；确定该每个关键点在该第二图像中的第二位置信息；根据该第一位置信息和该第二位置信息确定该每个关键点的运动轨迹。
结合第二方面，在一种可能的实现方式中，该多个分类器通过级联或者平均集成的方式连接。
第三方面，本申请提供了一种处理装置，包括处理器和存储器；该存储器，用于存储程序代码；该处理器，用于从该存储器中调用该程序代码执行如上述第一方面或者第一方面的任一可能的实现方式所描述的方法。
第四方面，本申请提供了一种计算机可读存储介质，计算机可读存储介质用于存储指令， 当该指令被执行时，使得如上述第一方面或者第一方面的任一可能的实现方式所描述的方法被实现。
第二方面至第四方面中各可能实施方式的有益效果可参见第一方面中的相应描述，在此不赘述。
通过本申请的方案，通过多个分类器处理检测目标上的至少一个关键点的运动轨迹，可以得到每个关键点的运动检测结果，之后再根据每个关键点的运动检测结果确定出该检测目标的运动检测结果。其中，基于关键点计算的运动轨迹相比光流等计算的运动轨迹，其耗时更小，精度更高，更符合真实的检测目标的运动，为后续的运动检测提供了更可靠的输入；另外，多个运动分类器利用了该检测目标的像素尺度变化，真实尺度变化，角度变化等方面的信息，相比利用多帧光流进行运动状态判断的方法，其鲁棒性更高，得到的运动检测结果更加准确。综上所述，通过本申请实施例的方式，可以得到更加精确的检测目标的运动检测结果。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍。
图1是本申请实施例提供的一种移动台的系统的结构示意图；
图2是本申请实施例提供的一种目标的检测方法的流程示意图；
图3是本申请实施例提供的一种检测目标的关键点的示意图；
图4是本申请实施例提供的一种图像的感兴趣区域和检测目标的包围框的示意图；
图5是本申请实施例提供的一些多个分类器的连接关系的示意图；
图6是本申请实施例提供的一种检测装置的示意图；
图7是本申请实施例提供的一种检测装置的示意图；
图8是本申请实施例提供的一种芯片的结构示意图。
具体实施方式
下面对本申请实施例中的技术方案进行更详细地描述。
参见图1，是本申请实施例提供的一种移动台的系统的结构示意图。该移动台可以是汽车、机器人、无人机，等等。该系统包括输入模块、传感器、检测模块和调整模块。需要说明的是，图1中所示的各个模块仅为示例，在实际应用场景中，移动台可以包括更多或者更少的模块或者器件，本申请实施例不作限制。以下对各个模块或器件进行进一步的介绍。
其中，输入模块，用于接收对移动台的控制信息。示例性的，该控制信息包括：方向盘转角信息、驾驶档位信息、驱动和制动信息等。
传感器：按照测量对象划分，传感器可分为两大类，一类是测量移动台自身的状态，另一类是测量移动台所处环境的状态。前者可以包括全球定位系统(global positioning system，GPS)，速度传感器、加速度传感器、角速度传感器、转矩传感器，等等。另外，移动台中还可以存在一些传感器来采集更多有关于该移动台运动(或者称为行驶、飞行)相关的信息，例如，发动机运转工况、各个模块或器件的运行温度、进气压力、进气温度，等等。后者可以包括相机、激光雷达传感器、毫米波雷达传感器、超声波传感器，等等。这些传感器可以采集移动台所处的环境相关的信息。传感器采集的各类信息可以转化为电信号，传输给移动 台的调整模块，调整模块进行分析后，输出对移动台的控制/调整指令或者输出给用户一些操作建议，用以辅助用户对移动台进行控制。
具体的，相机(或者称为摄像机、摄像头)作为一种传感器，可以通过拍摄移动台所处环境的图像，来采集移动台所处的环境的信息。示例性的，相机用于采集移动台在行驶过程中移动台前方的路况信息。在另一些示例性中，相机还用于采集移动台的倒车影像，行车记录图像等等。可选的，该相机可以是单目摄像头、双目摄像头、多目摄像头、广角摄像头、或环视摄像头等。
检测模块，用于根据相机获取的移动台所处的环境相关的信息，对环境中的其他物体的运动状态进行判定。该检测模块确定出的检测结果可以发送给调整模块，以便调整模块可以根据该检测结果对移动台的行驶状态进行控制/调整，或者向用户输出一些操作建议。
调整模块，用于对移动台的行驶状态进行控制/调整。示例性的，调整模块可以是先进驾驶辅助系统(advanced driver assistant system，ADAS)，整车控制器(vehicle control unit，VCU)，车身电子稳定系统(electronic stability program，ESP)，等等。举例而言，ADAS可以采用摄像头、雷达、激光和超声波等传感器收集的车内外的环境数据，辨识静、动态物体，利用侦测与追踪等技术上的处理，对车辆的驾驶意图进行感知。VCU是实现整车控制决策的控制单元，通过采集油门踏板、挡位、刹车踏板、方向盘转角等信号来判断驾驶员的驾驶意图；通过监测车辆状态(车速、温度等)信息，由VCU判断处理后，向动力系统、动力电池系统发送车辆的运行状态控制指令，同时控制车载附件电力系统的工作模式；具有整车系统故障诊断保护与存储功能。ESP，可以通过从车辆传感器发送的车辆行驶状态信息进行分析，然后向防抱死刹车系统(anti-lock brake system，ABS)，电子刹车分配力系统(electrical brake distribution，EBD)等发出纠偏指令，来帮助车辆维持动态平衡。
在本申请实施例中，调整模块可以获取各类传感器采集的信息以及检测模块得出的检测结果，进行分析后，输出对移动台的控制指令或者输出给用户一些操作建议，用以辅助用户对移动台进行控制。因此，检测模块确定出的检测结果的准确性会对调整模块输出的控制决策造成影响。
可选的，该检测模块可以是一个独立的模块，还可以集成在调整模块中的一个部件，本申请实施例对此不作限制。
在另一些可能的实现方式中，本申请实施例还可以应用到分布式传感器网络，或非可移动平台，例如路灯、红绿灯等，相关的领域包括智慧路口、智慧城市等。示例性的，可以通过路灯和红绿灯上配置的相机对交通区域的障碍物的运动状态进行检测。
通过本申请提供的目标的检测方法和检测装置，可以提升判定检测目标的运动状态的准确性。当该方法或者装置应用于终端时，可以提高终端的感知能力。
参见图2，是本申请实施例提供的一种目标的检测方法的流程示意图。该方法可以基于图1所示的系统来实现。可选的，下面描述的方法的执行主体可以为图1中的检测模块，该检测模块可以是一个独立的模块，还可以是集成在调整模块中的一个部件，还可以是集成在传感器中的一个部件。可选的，下面描述的方法的执行主体还可以是终端中的其他模块，以实现下述方法介绍的确定检测目标的运动检测结果的功能。该方法包括但不限于如下步骤。
S101、获取多个图像，该图像包括检测目标。
在一种可能的实现方式中，可以通过终端(示例为移动台)上配置的相机获取该多个图像。示例性的，安装在移动台上的相机可以拍摄移动台所处环境的图像(该多个图像可以是 连续拍摄的多帧图像)，并向检测模块发送这些图像。检测模块对接收的多个图像进行处理，以识别出包含有检测目标的图像。示例性的，该图像可以为图片，照片或者视频，流媒体等。
示例性的，该检测目标可以是车辆、行人、建筑物、路障，等等。其中，路障是指道路中设置的障碍物，以警示行人或者车辆，或者传递一些路况信息。在本申请实施例中，该检测目标可以理解为可能会对移动台的行动造成影响的人或物体。
S102、确定该检测目标的轮廓上的至少一个关键点。
可选的，可以通过可以识别检测目标的关键点的检测网络模型(或者称为算法系统)确定该检测目标的轮廓上的至少一个关键点。接下来对一种可能的检测网络模型的生成方式进行介绍。首先，设计人员获取训练数据。示例性的，可以通过移动台的相机在典型交通场景中拍摄包含各种检测目标(例如，车辆、行人、建筑物、路障，等等)的若干张图片，对拍摄的图像中出现的各类检测目标上标注其关键点(还可以标注其包围框)，以获取训练数据集。该训练数据集中包括了原始图片和检测目标上的关键点。之后，设计检测网络模型的网络结构，进行模型训练。该检测网络模型可以识别出包含有检测目标的图像。可选的，该检测网络模型还可以输出该检测目标的包围框，以及该检测目标的轮廓上的至少一个关键点。
示例性的，参见图3，是本申请实施例提供的一种检测目标的关键点的示意图。图3示意出的检测目标为车辆，图中共示意出了该车辆上的36个(仅为示例，实际应用中可以更多或者更少)可能的关键点，在模型训练的过程中，可以对这些关键点进行标注。该关键点包括车辆的后视镜，轮胎接地点，车灯，等等车辆的关键特征点。可以理解的是，对于不同类型的检测目标，关键点的位置设置的不同。例如，若检测目标为人物，该关键点可以为人物的头顶，肩头，脚部接地点，手部尾端，等等人物的关键特征点。
在一种可能的实现方式中，确定该检测目标的轮廓上的至少一个关键点的方式为：对图像中的感兴趣区域(region of interest，ROI)进行目标检测，确定包含该检测目标的包围框(或称为边界框)；处理该包围框中的图像确定该检测目标的轮廓和该检测目标所属的类型，该类型包括以下一项或者多项：车辆、行人、建筑物、路障；基于该轮廓和该类型确定该检测目标的轮廓上的至少一个关键点，该关键点在该轮廓上的位置基于该类型确定。
其中，感兴趣区域是从原始图像中选择出的一个图像区域。例如，该感兴趣区域可以为位于原始图像中心的一个预设大小的矩形区域。另外，还可以通过其他算子(Operator)和函数来确定感兴趣区域，本申请实施例不作限制。通过感兴趣区域的设置，可以从原始图像中确定出需要重点关注的区域，减少处理时间，增加精度。进一步的，检测目标的包围框是可以包含检测目标的，相比于感兴趣区域面积更小的图像区域。可选的，一个检测目标可以对应一个包围框。可选的，包围框可以为二维或者三维形式的包围框。通过包围框的设置，可以进一步的减少需要处理的图像范围，减少处理时间，增加精度。示例性的，参见图4，是本申请实施例提供的一种图像的感兴趣区域和检测目标的包围框的示意图。
可选的，在实际运用中，检测目标所属的类型可以有更多或更少的类型，或者，针对一个类型可以有更进一步的划分。示例性的，针对车辆而言，可以进一步的划分为货车、汽车、摩托车、自行车，等等。对于不同的类型，该检测目标的关键点所处的位置可以不同，也即是说，关键点在该检测目标的轮廓上的位置基于该检测目标的类型确定。可选的，该关键点在该检测目标的轮廓上的位置与该检测目标的类型之间的对应关系，可以在检测网络模型的训练过程中进行针对性的训练，让检测网络模型可以更好地处理各个类型的检测目标，提升判断精确度。
S103、确定每个关键点的运动轨迹。
在一些实施例中，该图像为连续拍摄的多帧图像，基于同一个关键点在不同帧的图像中的位置信息确定该关键点的运动轨迹。可选的，该运动轨迹还可以称为运动场、运动向量，等等。
在一种可能的实现方式中，该多个图像包括第三时刻的第一图像，以及第四时刻的第二图像，该确定每个关键点的运动轨迹，包括：确定该每个关键点在该第一图像中的第一位置信息；确定该每个关键点在该第二图像中的第二位置信息；根据该第一位置信息和该第二位置信息确定该每个关键点的运动轨迹。其中，该位置信息可以为绝对坐标信息，也可以为相对坐标信息。该关键点的运动轨迹可以体现该关键点的各项运动特征。示例性的，若第一位置信息和第二位置信息相同，那么可以表明该关键点未发生运动，为静止状态。若第一位置信息和第二位置信息不相同，那么可以表明该关键点发生了运动，可以确定出该关键点的运动轨迹，进一步可以计算出运动速度、角度，等等信息。
需要说明的是，在本申请实施例中，至少通过两张不同时刻拍摄的图像来确定关键点的运动轨迹。为了获取更多的信息，也可以通过更多时刻拍摄的图像来确定关键点的运动轨迹，本申请实施例不作限制。另外，在一种可能的实现方式中，还可以将包围框上的特征坐标点作为一种特殊的关键点，将该特征坐标点的运动轨迹也作为确定检测目标的运动检测结果的依据，该特征坐标点可以是包围框的顶点，包围框的中点，或者包围框的边框的中点，等等。
本申请实施例中基于关键点计算的运动轨迹相比光流等计算的运动轨迹，其耗时更小，精度更高，更符合真实的检测目标的运动；通过这种方式，可以为后续的运动检测提供更可靠的输入。
S104、通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果。
其中，该多个分类器包括：静止运动分类器、非平行运动分类器、高速平行运动分类器、低速运动分类器；该多个分类器还包括以下一项或者多项：运动角度分类器、运动尺度分类器、逆行运动分类器。
具体的，一个分类器用于确定关键点的一个运动特征，和该关键点处于该运动特征的第二运动概率。以下对各个分类器进行进一步的介绍。需要说明的是，除了以下介绍的7种运动分类器，该多个分类器还可以包括其他具有运动特征分类功能的分类器，本申请实施例不限制分类器包含的数量。另外，这多个分类器中的两个或者两个以上的部分分类器也可以组合形成一个功能更加全面的分类器，或者一个分类器也可以拆分为判定规则更加细致具体的多个分类器，对于分类器的拆分与组合，并不用来限制本申请实施例的范围。
可选的，该多个分类器的功能可以基于神经网络来实现。示例性的，该神经网络可以为卷积神经网络(convolutional neural networks，CNN)，长短期记忆网络(long short-term memory，LSTM)，或者卷积长短期记忆网络(convolutional long short-term memory，conv-LSTM)，等等。示例性的，该神经网络可以由多个处理层或者多个长短期记忆网络(long short term memory，LSTM)单元组成。其中，组成该神经网络的部分处理层或者部分LSTM单元所实现的功能可以对应该多个分类器中的一个分类器所实现的功能。在需要处理的数据量较大的情况下，神经网络的处理结果会比多个分类器的处理结果更准确。
1、静止运动分类器确定出的运动特征包括：静止、运动。该静止运动分类器还可以得出关键点处于该运动特征的第二运动概率，示例性的，可以用置信度来表示。其中，置信度，还可以称为可靠度、置信水平、置信系数，置信度表示某种估计结果有多大程度可信。示例性的，静止运动分类器对一个关键点的运动轨迹进行分类处理，得出该关键点处于静止，以 及处于静止的置信度为85％；则表示该关键点有85％的可能性(或称为概率)处于静止。该静止运动分类器对关键点是否处于静止/运动状态的判定结果较为准确。
2、非平行运动分类器确定出的运动特征包括：横向行驶、切入(cut in)运动、切出(cut out)运动。该非平行运动分类器还可以得出关键点处于该运动特征的第二运动概率，示例性的，可以用置信度来表示。其中，横向行驶表示该检测目标的运动方向与移动台本身的运动方向大概是垂直的，切入运动表示关键点进入本移动台的运动路线，切出运动表示关键点离开本移动台的运动路线。该运动角度分类器对关键点的运动方向的判定结果较为准确。
3、高速平行运动分类器确定出的运动特征包括：快速运动、高速运动、加速运动、急加速运动。该高速平行运动分类器还可以得出关键点处于该运动特征的第二运动概率，示例性的，可以用置信度来表示。可选的，该高速平行运动分类器的判定结果可以基于计算得到的关键点的速度以及加速度确定。在关键点处于高速运动的情况下，该高速平行运动分类器的判定结果较为准确。
4、低速运动分类器确定出的运动特征包括：慢速运动、减速运动、急减速运动。该低速运动分类器还可以得出关键点处于该运动特征的第二运动概率，示例性的，可以用置信度来表示。可选的，该低速平行运动分类器的判定结果可以基于计算得到的关键点的速度以及加速度确定。在关键点处于低速运动的情况下，该低速运动分类器的判定结果较为准确。
5、运动角度分类器确定出的运动特征包括：转弯行驶、掉头行驶、平行行驶、正向/逆向行驶。该运动角度分类器还可以得出关键点处于该运动特征的第二运动概率，示例性的，可以用置信度来表示。该运动角度分类器对关键点的角速度的判定结果较为准确。特别的，若该检测目标距离本终端较近，该运动角度分类器得出的判定结果较为准确。需要说明的是，运动角点分类器可以根据关键点生成的汇聚点与静态场景的光流汇聚点的关系来判断关键点的运动角度，进而计算关键点的角速度。在这种方式中，由于该检测目标的关键点为该检测目标的轮廓上具有特定特征的点，与该检测目标的关联性更强，作为运动角度的判断依据，基于关键点生成的汇聚点相比于检测目标上的光流的光流汇聚点会更加准确。通过这种方式，可以提高确定关键点的运动角度的准确性。
6、运动尺度分类器确定出的运动特征包括：正向/逆向加速度。该运动尺度分类器还可以得出关键点处于该运动特征的第二运动概率，示例性的，可以用置信度来表示。需要说明的是，该置信度越高，说明该关键点的加速度越大。该运动尺度分类器对关键点的加速度的判定结果较为准确。特别的，若该检测目标距离本终端较远，该运动尺度分类器得出的判定结果较为准确。
7、逆行运动分类器确定出的运动特征包括：逆向行驶、正向行驶。该逆行运动分类器还可以得出关键点处于该运动特征的第二运动概率，示例性的，可以用置信度来表示。该逆行运动分类器对关键点是否处于逆向行驶/正向行驶的判定结果较为准确。
上述介绍的多个运动分类器利用了该检测目标的像素尺度变化，真实尺度变化，角度变化等方面的信息，相比利用多帧光流进行运动状态判断的方法，其鲁棒性更高。
通过该多个分类器可以得到关键点的各项运动特征，该多个运动特征可以共同确定该关键点的运动状态，该多个第二运动概率可以共同确定该关键点处于某个运动状态的第一运动概率。也即是说，该关键点的运动状态基于多个该运动特征确定，该第一运动概率基于多个关键点的第二运动概率确定。关键点的运动检测结果包括该关键点的运动状态和该第一运动概率。
以下对运动状态进行进一步的介绍。在本申请实施例中，该运动状态包括以下一项或者 多项：正向快速运动状态，正向慢速运动状态，静止状态，正向切入cut-in状态，正向切出cut-out状态，高速逆行运动状态，低速逆行运动状态，正向加速运动状态，正向减速运动状态，正向急加速运动状态，正向急减速运动状态，急停状态，横穿状态，平行运动状态，横向转弯状态，纵向运动状态，掉头状态，描述运动状态的参数信息。其中，该描述运动状态的参数信息包括以下一项或者多项：绝对速度信息，相对速度信息，碰撞时间(time-to-collision，TTC)信息。该碰撞时间用于指示在双方运动状态不变的情况下，当前移动台会撞上该检测目标所需的时间。
示例性的，运动、快速运动、正向行驶这些运动特征确定出的运动状态可以为“正向快速运动状态”。运动、慢速运动、正向行驶这些运动特征确定出的运动状态可以为“正向慢速运动状态”。运动、急减速运动、正向行驶这些运动特征确定出的运动状态可以为“正向急减速运动状态”。运动、运动角度这些运动特征确定出的运动状态可以为“掉头状态”。还可以存在其他更多的基于多个运动特征确定出运动状态的可能方式，本申请实施例在此不一一列举。
对于该关键点的第一运动概率而言，可以表征一个关键点处于某项确定出的运动状态的概率，可以为后续判定检测目标的第一运动概率提供依据。可选的，该第一运动概率可以是该多个第二运动概率的算术平均值或者加权平均值。需要说明的是，该第一运动概率可能由部分第二运动概率计算得出的。例如，可以对过小的第二运动概率进行滤除，举例来说，若慢速运动的第二运动概率为5％，那么表示该关键点处于慢速运动的可能性很小，计算检测目标的第一运动概率时，可以不考虑该慢速运动的第二运动概率。
可选的，该多个分类器可以通过级联或者平均集成的方式连接。可选的，该多个分类器还可以通过部分级联，部分平均集成的方式进行连接。示例性的，参见图5，是本申请实施例提供的一些多个分类器的连接关系的示意图。图5中包括的分类器为：静止运动分类器、非平行运动分类器、高速平行运动分类器、低速运动分类器、运动角度分类器、运动尺度分类器和逆行运动分类器。需要说明的是，在实际的应用中，该多个分类器可以包括运动角度分类器、运动尺度分类器、逆行运动分类器中的一个或者多个。
其中，图5中的(1)示例了一种该多个分类器通过级联的方式进行连接的示意图。在级联的这种连接方式中，各个分类器得到的判定结果可以互为参考依据，通过这种方式，可以充分利用不同分类器对于某一类或几类运动状态的判定结果的相关联性。示例性的，对于正向加速运动而言，高速平行运动分类器(可判定加速运动)和运动尺度分类器(可判定加速度)得到的判定结果具有强相关性，可以互为判定的参考依据，得到更准确的结果。图5中的(2)示例了一种该多个分类器通过平均集成的方式进行连接的示意图。在平均集成的这种连接方式中，各个分类器得到的判定结果单独得出，若一个分类器的判定结果并不太准确，也不会对其他分类器的判定结果造成影响。图5中的(3)示例了一种该多个分类器通过部分级联，部分平均集成的方式进行连接的示意图。可选的，具有强关联性的分类器可以采用级联的方式，关联性较弱的分类器之间采用平均集成的连接方式。另需要说明的是，各个分类器的连接顺序可以依照实际应用的需求进行调整，本申请不作限制。
在一种可能的实现方式中，该每个关键点的运动检测结果还需要基于移动台的运动参数来确定。也即是说，方法还包括：获取该移动台的运动参数。基于该移动台的运动参数，通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果。其中，该移动台的运动参数可以包括移动台的速度、加速度、角速度等信息，可以通过移动台的各类传感器来获取，示例性的，可以为GPS、速度传感器、加速度传感器、角速度传感 器、转矩传感器、惯性传感器(inertial measurement unit，IMU)、全球导航卫星系统(global navigation satellite system，GNSS)以及论速计(WSS)，等等。需要说明的是，在这种实现方式中，移动台的运动参数以及关键点的运动轨迹均作为该多个运动分类器的输入，共同决定运动分类器的输出结果。通过这种方式，由于考虑到了移动台的运动参数，可以消除移动台自身的运动对判定该检测目标的运动状态的影响，该可以得到更加准确的运动检测结果。
在一种可能的实现方式中，在确定关键点的运动检测结果的过程中，该多个分类器的权重可以不同。可选的，该多个分类器的权重可以采用提升算法通过训练得到。可选的，该多个分类器的权重在不同的驾驶场景中，也可以存在不同的取值，以便适配不同的驾驶场景，得到更准确的判定结果。示例性的，该驾驶场景可以包括直行行驶、弯道行驶、上坡行驶、下坡行驶等等。
S105、基于该每个关键点的运动检测结果确定该检测目标的运动检测结果。
其中，检测目标的运动检测结果包括检测目标的运动状态和第一运动概率，该第一运动概率用于指示检测目标的处于某个运动状态的概率。
可选的，该检测目标的第一运动概率和运动状态可以是该每个关键点的运动检测结果中数值最大的一个第一运动概率以及其对应的运动状态。示例性的，共有三个关键点，其中，第一关键点的运动状态为正向快速运动状态，第一运动概率为80％；第二关键点的运动状态为正向快速运动状态，第一运动概率为85％；第三关键点的运动状态为正向加速运动状态，第一运动概率为70％；那么，检测目标的运动状态可以确定为正向快速运动状态(最大数值的第一运动概率对应的运动状态)，该检测目标的第一运动概率为85％。
可选的，该检测目标的运动检测结果还可以是，基于该每个关键点的运动检测结果通过滤波器(或者其他算法系统)处理得到的运动检测结果。示例性的，可以为卡尔曼滤波器(kalma filter)。卡尔曼滤波(Kalman filtering)是一种利用线性系统状态方程，通过系统输入输出观测数据，对系统状态进行最优估计的算法。由于观测数据中包括系统中的噪声和干扰的影响，所以最优估计也可看作是滤波过程。数据滤波是去除噪声还原真实数据的一种数据处理技术，Kalman滤波在测量方差已知的情况下能够从一系列存在测量噪声的数据中，估计动态系统的状态，能够对现场采集的数据进行实时的更新和处理。
在一些实施例中，该检测目标的运动检测结果还可以联合基于该检测目标的包围框确定出的运动特征，共同确定出更加准确的检测目标的运动检测结果。可选的，该方法还包括：确定该包围框中的至少一个特征坐标点；通过包围框运动分类器对该每个特征坐标点的运动轨迹进行分类处理，得到该检测目标的相对该移动台的相对速度，以及处于该相对速度的第三运动概率；基于该相对速度，以及该第三运动概率更新该检测目标的运动检测结果。
其中，该包围框是可以包含检测目标的一个预设形状的边界框，示例性的，该包围框可以为矩形。对于包围框的介绍可以参照上述图4有关内容中介绍。该特征坐标点可以是在预设图形中容易查找到的点，具有特殊的相对于包围框的位置关系。示例性的，可以是包围框的顶点、中心点、构成包围框的边的中点，等等。该特征坐标点的运动轨迹可以参照上述内容中介绍的关键点的运动轨迹，此处不再赘述。具体的，包围框运动分类器可以基于每个特征坐标点的运动轨迹，得到该检测目标的相对该移动台的相对速度，以及处于该相对速度的第三运动概率。示例性的，第三运动概率可以用置信度来表示。需要说明的是，该置信度越高，说明该检测目标的相对速度越大。该包围框运动分类器对检测目标的相对速度的判定结果较为准确。可选的，可以基于该相对速度，以及该第三运动概率更新该检测目标的运动检测结果中的检测目标的相对速度。
由于关键点为该检测目标的轮廓上具有特定特征的点，在获取过程中，部分关键点可以会存在被遮挡的情况，而包围框上的特征坐标点是易于查找到的，且该包围框与该检测目标具有强关联性，该包围框的运动特征可以较好地体现该检测目标的运动特征，因此，在关键点的运动轨迹较少的情况下，该包围框运动分类器可以得到更准确的判定结果。
在一些实施例中，当前时刻得出的检测目标的运动检测结果还可以联合前序时刻得出的检测目标的运动检测结果，共同确定出更加准确的检测目标的运动检测结果。示例性的，该检测目标的运动检测结果与第一时刻对应。可以理解为，作为检测依据的该多个图像与第一时刻对应，例如，该多个图像是14:01-14:02这一分钟内拍摄的图像，该第一时刻为14:01-14:02。本申请实施例的方法还包括：基于第二时刻对应的该检测目标的运动检测结果和该检测目标的运动检测结果，更新该第一时刻对应的该检测目标的运动检测结果，该第二时刻在该第一时刻之前。示例性的，该第二时刻可以为14:00-14:01，该第二时刻对应的该检测目标的运动检测结果可以是，基于14:00-14:01这一分钟内拍摄的图像确定出的运动检测结果以及该第二时刻的前序时刻得出的检测目标的运动检测结果，共同确定出的。
可选的，该第二时刻也可以是在第一时刻之前的多个时刻，示例性的，该第二时刻可以为13:58-14:01，该第二时刻对应的该检测目标的运动检测结果包含13:58-13:59、13:59-14:00、14:00-14:01这三个时刻分别对应的检测目标的运动检测结果。以一个具体的示例而言，若13:58-13:59对应的检测目标的运动检测结果为“86％处于正向加速运动状态”、13:58-13:59对应的检测目标的运动检测结果为“89％处于正向急加速运动状态”、14:00-14:01对应的检测目标的运动检测结果为“92％处于正向急加速运动状态”，14:01-14:02当前时刻得出的检测目标的运动检测结果为“83％处于正向急加速运动状态”，将上述运动检测结果输入到训练完成的时序神经网络中，可以得到更新后的(也可以理解为最终确定出的)14:01-14:02对应的检测目标的运动检测结果，示例性的，可以为“94％处于正向急加速运动状态”。
在本方案中，通过多个分类器处理检测目标上的至少一个关键点的运动轨迹，可以得到每个关键点的运动检测结果，之后再根据每个关键点的运动检测结果确定出该检测目标的运动检测结果。其中，基于关键点计算的运动轨迹相比光流等计算的运动轨迹，其耗时更小，精度更高，更符合真实的检测目标的运动，为后续的运动检测提供了更可靠的输入；另外，多个运动分类器利用了该检测目标的像素尺度变化，真实尺度变化，角度变化等方面的信息，相比利用多帧光流进行运动状态判断的方法，其鲁棒性更高，得到的运动检测结果更加准确。此外，本申请实施例确定出的运动状态相比于静止、运动这类简单的运动状态以及单纯的速度、加速度信息而言，具有更加具体细节的特征描述。综上所述，通过本申请实施例的方式，可以得到更加精确的检测目标的运动检测结果，进而为移动台的调整模块提供更准确的调整依据，可以更好地辅助移动台的行驶。
为了实现上述本申请实施例提供的方法中的各功能，该方法的执行主体可以包括硬件结构、软件模块，以硬件结构、软件模块、或硬件结构加软件模块的形式来实现上述各功能。上述各功能中的某个功能可以以硬件结构、软件模块、或者硬件结构加软件模块的方式来执行。
参见图6，是本申请实施例提供的一种检测装置的结构示意图。该检测装置可以是移动台中的装置。检测装置60包括获取单元601、第一确定单元602、第二确定单元603、处理单元604和第三确定单元605：
该获取单元601，用于获取多个图像，该图像包括检测目标。该获取单元601所执行的 操作可以参照上述图2的步骤S101的内容的介绍。
该第一确定单元602，用于确定该检测目标的轮廓上的至少一个关键点。该第一确定单元602所执行的操作可以参照上述图2的步骤S102的内容的介绍。
该第二确定单元603，用于确定每个关键点的运动轨迹。该第二确定单元603所执行的操作可以参照上述图2的步骤S103的内容的介绍。
该处理单元604，用于通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果；其中，该多个分类器包括：静止运动分类器、非平行运动分类器、高速平行运动分类器、低速运动分类器；该多个分类器还包括以下一项或者多项：运动角度分类器、运动尺度分类器、逆行运动分类器。该处理单元604所执行的操作可以参照上述图2的步骤S104的内容的介绍。
该第三确定单元605，用于基于该每个关键点的运动检测结果确定该检测目标的运动检测结果。该第三确定单元605所执行的操作可以参照上述图2的步骤S105的内容的介绍。
在一种可能的实现方式中，该运动检测结果包括运动状态和第一运动概率，该第一运动概率用于指示处于该运动状态的概率。
在一种可能的实现方式中，一个该分类器用于确定该关键点的一个运动特征，和该关键点处于该运动特征的第二运动概率；该运动状态基于多个该运动特征确定，该第一运动概率基于多个该第二运动概率确定。
在一种可能的实现方式中，该运动状态包括以下一项或者多项：正向快速运动状态，正向慢速运动状态，静止状态，正向切入cut-in状态，正向切出cut-out状态，高速逆行运动状态，低速逆行运动状态，正向加速运动状态，正向减速运动状态，正向急加速运动状态，正向急减速运动状态，急停状态，横穿状态，平行运动状态，横向转弯状态，纵向运动状态，掉头状态，描述运动状态的参数信息；其中，该描述运动状态的参数信息包括以下一项或者多项：绝对速度信息，相对速度信息，碰撞时间信息。
在一种可能的实现方式中，该静止运动分类器确定出的运动特征包括：静止、运动；该非平行运动分类器确定出的运动特征包括：横向行驶、切入、切出；该高速平行运动分类器确定出的运动特征包括：快速、高速、加速、急加速；该低速运动分类器确定出的运动特征包括：慢速、减速、急减速；该运动角度分类器确定出的运动特征包括：判断转弯、掉头、平行行驶，正向/逆向运动；该运动尺度分类器确定出的运动特征包括：正向/逆向加速度；该逆行运动分类器确定出的运动特征包括：逆向行驶、正向行驶。
在一种可能的实现方式中，该检测目标的运动检测结果与第一时刻对应，该第三确定单元605还用于：基于第二时刻对应的该检测目标的运动检测结果和该检测目标的运动检测结果，更新该第一时刻对应的该检测目标的运动检测结果，该第二时刻在该第一时刻之前。
在一种可能的实现方式中，获取该多个图像的相机位于移动台上，该获取单元601还用于：获取该移动台的运动参数；该处理单元604具体用于：基于该移动台的运动参数，通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果。
在一种可能的实现方式中，该第一确定单元602具体用于：对该图像中的感兴趣区域进行目标检测，确定包含该检测目标的包围框；处理该包围框中的图像确定该检测目标的轮廓和该检测目标所属的类型，该类型包括以下一项或者多项：车辆、行人、建筑物、路障；基于该轮廓和该类型确定该检测目标的轮廓上的至少一个关键点，该关键点在该轮廓上的位置基于该类型确定。
在一种可能的实现方式中，获取多个图像的相机位于移动台上，第三确定单元605还用 于：确定该包围框中的至少一个特征坐标点；通过包围框运动分类器对该每个特征坐标点的运动轨迹进行分类处理，得到该检测目标的相对该移动台的相对速度，以及处于该相对速度的第三运动概率；基于该相对速度，以及该第三运动概率更新该检测目标的运动检测结果。
在一种可能的实现方式中，该多个图像包括第三时刻的第一图像，以及第四时刻的第二图像，该第二确定单元603具体用于：确定该每个关键点在该第一图像中的第一位置信息；确定该每个关键点在该第二图像中的第二位置信息；根据该第一位置信息和该第二位置信息确定该每个关键点的运动轨迹。
在一种可能的实现方式中，该多个分类器通过级联或者平均集成的方式连接。
具体的，图6所示的检测装置60的各个单元执行的操作可以参照上述图2对应的方法实施例中的相关内容，此处不再详述。上述各个单元可以以硬件，软件或者软硬件结合的方式来实现。在一个实施例中，上述内容中的各个单元的功能可以由检测装置60中的一个或多个处理器来实现。
参见图7，是本申请实施例提供的又一种检测装置的结构示意图。该检测装置70可用于实现上述方法实施例中描述的方法，具体可以参见上述方法实施例中的说明。
检测装置70可以包括一个或多个处理器701。该处理器701可以是通用处理器或者专用处理器等。该处理器701可以用于对检测装置进行控制，执行软件程序，处理软件程序的数据。
可选的，该检测装置70中可以包括一个或多个存储器702，其上可以存有程序代码704，该程序代码可在该处理器701上被运行，使得该检测装置70执行上述方法实施例中描述的方法。可选的，该存储器702中还可以存储有数据。该处理器701和存储器702可以单独设置，也可以集成在一起。可选的，存储器702还可以位于检测装置70之外，通过一些方式与检测装置70耦合。
可选的，该检测装置70还可以包括收发器705。该收发器705可以称为收发单元、收发机、或收发电路等，用于实现收发功能。收发器705可以包括接收器和发送器，接收器可以称为接收机或接收电路等，用于实现接收功能；发送器可以称为发送机或发送电路等，用于实现发送功能。
处理器701，用于通过收发器705接收多个图像，该图像包括检测目标；还用于确定该检测目标的轮廓上的至少一个关键点；确定每个关键点的运动轨迹；通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果；其中，该多个分类器包括：静止运动分类器、非平行运动分类器、高速平行运动分类器、低速运动分类器；该多个分类器还包括以下一项或者多项：运动角度分类器、运动尺度分类器、包围框运动分类器、逆行运动分类器；基于该每个关键点的运动检测结果确定该检测目标的运动检测结果。
需要说明的是，在上述实施例中，检测装置70可以是移动台中的装置，也可以是支持移动台中的装置中实现上述方法的芯片、芯片系统、或处理器等。
具体的，检测装置70执行的操作可以参照上述图2对应的方法实施例中有关于检测模块的相关内容，此处不再详述。
在另一种可能的设计中，该收发器705可以是收发电路，或者是接口，或者是接口电路。用于实现接收和发送功能的收发电路、接口或接口电路可以是分开的，也可以集成在一起。上述收发电路、接口或接口电路可以用于代码/数据的读写，或者，上述收发电路、接口或接口电路可以用于信号的传输或传递。
在又一种可能的设计中，可选的，处理器701可以存有程序代码703，程序代码703在处理器701上运行，可使得该检测装置70执行上述方法实施例中描述的方法。程序代码703可能固化在处理器701中，该种情况下，处理器701可能由硬件实现。
在又一种可能的设计中，检测装置70可以包括电路，该电路可以实现前述方法实施例中发送或接收或者通信的功能。
本申请中描述的处理器和收发器可实现在集成电路(integrated circuit，IC)、模拟IC、射频集成电路RFIC、混合信号IC、专用集成电路(application specific integrated circuit，ASIC)、印刷电路板(printed circuit board，PCB)、电子设备等上。
以上实施例描述中的检测装置可以是网络设备或者终端设备，但本申请中描述的检测装置的范围并不限于此，而且检测装置的结构可以不受图7的限制。检测装置可以是独立的设备或者可以是较大设备的一部分。例如该检测装置可以是：
(1)独立的集成电路IC，或芯片，或，芯片系统或子系统；(2)具有一个或多个IC的集合，可选的，该IC集合也可以包括用于存储数据，程序代码的存储部件；(3)ASIC，例如调制解调器(Modem)；(4)可嵌入在其他设备内的模块；(5)接收机、智能终端、无线设备、手持机、移动单元、车载设备、云设备、人工智能设备等等。
对于检测装置可以是芯片或芯片系统的情况，可参见图8所示的芯片的结构示意图。图8所示的芯片80包括逻辑电路801和输入输出接口802。其中，逻辑电路801的数量可以是一个或多个，输入输出接口802的数量可以是多个。
逻辑电路801，可以用于通过输入输出接口802获取多个图像，该图像包括检测目标。
逻辑电路801，还可以用于确定该检测目标的轮廓上的至少一个关键点；确定每个关键点的运动轨迹；通过多个分类器对该每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果；其中，该多个分类器包括：静止运动分类器、非平行运动分类器、高速平行运动分类器、低速运动分类器；该多个分类器还包括以下一项或者多项：运动角度分类器、运动尺度分类器、包围框运动分类器、逆行运动分类器；基于该每个关键点的运动检测结果确定该检测目标的运动检测结果。
具体的，在这种情况中，逻辑电路801所执行的操作可以参照上述图2对应的方法实施例中的相关内容，此处不再详述。
本领域技术人员还可以了解到本申请实施例列出的各种说明性逻辑块(illustrative logical block)和步骤(step)可以通过电子硬件、电脑软件，或两者的结合进行实现。这样的功能是通过硬件还是软件来实现取决于特定的应用和整个系统的设计要求。本领域技术人员可以对于每种特定的应用，可以使用各种方法实现该的功能，但这种实现不应被理解为超出本申请实施例保护的范围。
本申请还提供了一种计算机可读存储介质，其上存储有计算机程序，该计算机可读存储介质被计算机执行时实现上述任一方法实施例的功能。
本申请还提供了一种计算机程序产品，该计算机程序产品被计算机执行时实现上述任一方法实施例的功能。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。该计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行该计算机指令时，全部或部分地产生按照本申请实施例该的流程或功能。该计算机指令可以存储在计算机可读存储介质中，或者从一个 计算机可读存储介质向另一个计算机可读存储介质传输，例如，该计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(digital subscriber line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。该计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。该可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，高密度数字视频光盘(digital video disc，DVD))、或者半导体介质(例如，固态硬盘(solid state disk，SSD))等。
本领域普通技术人员可以理解：本申请中涉及的第一、第二等各种数字编号仅为描述方便进行的区分，并不用来限制本申请实施例的范围，先后顺序。
本申请中的预定义可以理解为定义、预先定义、存储、预存储、预协商、预配置、固化、或预烧制。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。

Claims (24)

  1. 一种目标的检测方法，其特征在于，包括：
    获取多个图像，所述图像包括检测目标；
    确定所述检测目标的轮廓上的至少一个关键点；
    确定每个关键点的运动轨迹；
    通过多个分类器对所述每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果；
    其中，所述多个分类器包括：静止运动分类器、非平行运动分类器、高速平行运动分类器、低速运动分类器；所述多个分类器还包括以下一项或者多项：运动角度分类器、运动尺度分类器、逆行运动分类器；
    基于所述每个关键点的运动检测结果确定所述检测目标的运动检测结果。
  2. 根据权利要求1所述的方法，其特征在于，所述运动检测结果包括运动状态和第一运动概率，所述第一运动概率用于指示处于所述运动状态的概率。
  3. 根据权利要求2所述的方法，其特征在于，一个所述分类器用于确定所述关键点的一个运动特征，和所述关键点处于所述运动特征的第二运动概率；
    所述运动状态基于多个所述运动特征确定，所述第一运动概率基于多个所述第二运动概率确定。
  4. 根据权利要求2或3所述的方法，其特征在于，所述运动状态包括以下一项或者多项：正向快速运动状态，正向慢速运动状态，静止状态，正向切入cut-in状态，正向切出cut-out状态，高速逆行运动状态，低速逆行运动状态，正向加速运动状态，正向减速运动状态，正向急加速运动状态，正向急减速运动状态，急停状态，横穿状态，平行运动状态，横向转弯状态，纵向运动状态，掉头状态，描述运动状态的参数信息；
    其中，所述描述运动状态的参数信息包括以下一项或者多项：绝对速度信息，相对速度信息，碰撞时间信息。
  5. 根据权利要求3所述的方法，其特征在于，
    所述静止运动分类器确定出的运动特征包括：静止、运动；
    所述非平行运动分类器确定出的运动特征包括：横向行驶、切入、切出；
    所述高速平行运动分类器确定出的运动特征包括：快速、高速、加速、急加速；
    所述低速运动分类器确定出的运动特征包括：慢速、减速、急减速；
    所述运动角度分类器确定出的运动特征包括：判断转弯、掉头、平行行驶、正向/逆向运动；
    所述运动尺度分类器确定出的运动特征包括：正向/逆向加速度；
    所述逆行运动分类器确定出的运动特征包括：逆向行驶、正向行驶运动。
  6. 根据权利要求1-5任一项所述的方法，其特征在于，所述检测目标的运动检测结果与第一时刻对应，所述方法还包括：
    基于第二时刻对应的所述检测目标的运动检测结果和所述检测目标的运动检测结果，更新所述第一时刻对应的所述检测目标的运动检测结果，所述第二时刻在所述第一时刻之前。
  7. 根据权利要求1-6任一项所述的方法，其特征在于，获取所述多个图像的相机位于移动台上，通过多个分类器对所述每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果，包括：
    获取所述移动台的运动参数；
    基于所述移动台的运动参数，通过多个分类器对所述每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果。
  8. 根据权利要求1-7任一项所述的方法，其特征在于，所述确定所述检测目标的轮廓上的至少一个关键点，包括：
    对所述图像中的感兴趣区域进行目标检测，确定包含所述检测目标的包围框；
    处理所述包围框中的图像确定所述检测目标的轮廓和所述检测目标所属的类型，所述类型包括以下一项或者多项：车辆、行人、建筑物、路障；
    基于所述轮廓和所述类型确定所述检测目标的轮廓上的至少一个关键点，所述关键点在所述轮廓上的位置基于所述类型确定。
  9. 根据权利要求8所述的方法，其特征在于，获取所述多个图像的相机位于移动台上，所述方法还包括：
    确定所述包围框中的至少一个特征坐标点；
    通过包围框运动分类器对所述每个特征坐标点的运动轨迹进行分类处理，得到所述检测目标的相对所述移动台的相对速度，以及处于所述相对速度的第三运动概率；
    基于所述相对速度，以及所述第三运动概率更新所述检测目标的运动检测结果。
  10. 根据权利要求1-9任一项所述的方法，其特征在于，所述多个图像包括第三时刻的第一图像，以及第四时刻的第二图像，所述确定每个关键点的运动轨迹，包括：
    确定所述每个关键点在所述第一图像中的第一位置信息；
    确定所述每个关键点在所述第二图像中的第二位置信息；
    根据所述第一位置信息和所述第二位置信息确定所述每个关键点的运动轨迹。
  11. 根据权利要求1-10任一项所述的方法，其特征在于，所述多个分类器通过级联或者平均集成的方式连接。
  12. 一种检测装置，其特征在于，所述检测装置包括获取单元、第一确定单元、第二确定单元、处理单元和第三确定单元：
    所述获取单元，用于获取多个图像，所述图像包括检测目标；
    所述第一确定单元，用于确定所述检测目标的轮廓上的至少一个关键点；
    所述第二确定单元，用于确定每个关键点的运动轨迹；
    所述处理单元，用于通过多个分类器对所述每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果；
    其中，所述多个分类器包括：静止运动分类器、非平行运动分类器、高速平行运动分类器、低速运动分类器；所述多个分类器还包括以下一项或者多项：运动角度分类器、运动尺度分类器、包围框运动分类器、逆行运动分类器；
    所述第三确定单元，用于基于所述每个关键点的运动检测结果确定所述检测目标的运动检测结果。
  13. 根据权利要求12所述的检测装置，其特征在于，所述运动检测结果包括运动状态和第一运动概率，所述第一运动概率用于指示处于所述运动状态的概率。
  14. 根据权利要求13所述的检测装置，其特征在于，一个所述分类器用于确定所述关键点的一个运动特征，和所述关键点处于所述运动特征的第二运动概率；
    所述运动状态基于多个所述运动特征确定，所述第一运动概率基于多个所述第二运动概率确定。
  15. 根据权利要求13或14所述的检测装置，其特征在于，所述运动状态包括以下一项或者多项：正向快速运动状态，正向慢速运动状态，静止状态，正向切入cut-in状态，正向切出cut-out状态，高速逆行运动状态，低速逆行运动状态，正向加速运动状态，正向减速运动状态，正向急加速运动状态，正向急减速运动状态，急停状态，横穿状态，平行运动状态，横向转弯状态，纵向运动状态，掉头状态，描述运动状态的参数信息；
    其中，所述描述运动状态的参数信息包括以下一项或者多项：绝对速度信息，相对速度信息，碰撞时间信息。
  16. 根据权利要求14所述的检测装置，其特征在于，
    所述静止运动分类器确定出的运动特征包括：静止、运动；
    所述非平行运动分类器确定出的运动特征包括：横向行驶、切入、切出；
    所述高速平行运动分类器确定出的运动特征包括：快速、高速、加速、急加速；
    所述低速运动分类器确定出的运动特征包括：慢速、减速、急减速；
    所述运动角度分类器确定出的运动特征包括：判断转弯、掉头、平行行驶，正向/逆向运动；
    所述运动尺度分类器确定出的运动特征包括：正向/逆向加速度；
    所述逆行运动分类器确定出的运动特征包括：逆向行驶、正向行驶。
  17. 根据权利要求12-16任一项所述的检测装置，其特征在于，所述检测目标的运动检测结果与第一时刻对应，所述第三确定单元还用于：
    基于第二时刻对应的所述检测目标的运动检测结果和所述检测目标的运动检测结果，更新所述第一时刻对应的所述检测目标的运动检测结果，所述第二时刻在所述第一时刻之前。
  18. 根据权利要求12-17任一项所述的检测装置，其特征在于，获取所述多个图像的相机位于移动台上，所述获取单元还用于：获取所述移动台的运动参数；
    所述处理单元具体用于：基于所述移动台的运动参数，通过多个分类器对所述每个关键点的运动轨迹进行分类处理，得到每个关键点的运动检测结果。
  19. 根据权利要求12-18任一项所述的检测装置，其特征在于，所述第一确定单元具体用于：
    对所述图像中的感兴趣区域进行目标检测，确定包含所述检测目标的包围框；
    处理所述包围框中的图像确定所述检测目标的轮廓和所述检测目标所属的类型，所述类型包括以下一项或者多项：车辆、行人、建筑物、路障；
    基于所述轮廓和所述类型确定所述检测目标的轮廓上的至少一个关键点，所述关键点在所述轮廓上的位置基于所述类型确定。
  20. 根据权利要求12-19任一项所述的检测装置，其特征在于，获取所述多个图像的相机位于移动台上，所述第三确定单元还用于：
    确定所述包围框中的至少一个特征坐标点；
    通过包围框运动分类器对所述每个特征坐标点的运动轨迹进行分类处理，得到所述检测目标的相对所述移动台的相对速度，以及处于所述相对速度的第三运动概率；
    基于所述相对速度，以及所述第三运动概率更新所述检测目标的运动检测结果。
  21. 根据权利要求12-20任一项所述的检测装置，其特征在于，所述多个图像包括第三时刻的第一图像，以及第四时刻的第二图像，所述第二确定单元具体用于：
    确定所述每个关键点在所述第一图像中的第一位置信息；
    确定所述每个关键点在所述第二图像中的第二位置信息；
    根据所述第一位置信息和所述第二位置信息确定所述每个关键点的运动轨迹。
  22. 根据权利要求12-21任一项所述的检测装置，其特征在于，所述多个分类器通过级联或者平均集成的方式连接。
  23. 一种检测装置，其特征在于，包括处理器和存储器；
    所述存储器，用于存储程序代码；
    所述处理器，用于从所述存储器中调用所述程序代码执行如权利要求1-11任一项所述的方法。
  24. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质用于存储指令，当所述指令被执行时，使得如权利要求1-11任一项所述的方法被实现。
PCT/CN2022/089905 2022-04-28 2022-04-28 一种目标的检测方法及相关装置 WO2023206236A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/089905 WO2023206236A1 (zh) 2022-04-28 2022-04-28 一种目标的检测方法及相关装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/089905 WO2023206236A1 (zh) 2022-04-28 2022-04-28 一种目标的检测方法及相关装置

Publications (1)

Publication Number Publication Date
WO2023206236A1 true WO2023206236A1 (zh) 2023-11-02

Family

ID=88516869

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/089905 WO2023206236A1 (zh) 2022-04-28 2022-04-28 一种目标的检测方法及相关装置

Country Status (1)

Country Link
WO (1) WO2023206236A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105447490A (zh) * 2015-11-19 2016-03-30 浙江宇视科技有限公司 基于梯度回归树的车辆关键点检测方法及装置
CN106295620A (zh) * 2016-08-28 2017-01-04 乐视控股（北京）有限公司 发型识别方法及发型识别装置
CN110231605A (zh) * 2019-05-09 2019-09-13 深圳市速腾聚创科技有限公司 人体行为的识别方法、装置、计算机设备和存储介质
CN110674785A (zh) * 2019-10-08 2020-01-10 中兴飞流信息科技有限公司 一种基于人体关键点跟踪的多人姿态分析方法
CN111914676A (zh) * 2020-07-10 2020-11-10 泰康保险集团股份有限公司 人体摔倒检测方法、装置、电子设备和存储介质
CN112200044A (zh) * 2020-09-30 2021-01-08 北京四维图新科技股份有限公司 异常行为检测方法、装置及电子设备
WO2021213742A1 (en) * 2020-04-22 2021-10-28 Continental Automotive Gmbh Method and system for keypoint detection based on neural networks
CN114170580A (zh) * 2021-12-01 2022-03-11 连云港杰瑞电子有限公司 一种面向高速公路的异常事件检测方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105447490A (zh) * 2015-11-19 2016-03-30 浙江宇视科技有限公司 基于梯度回归树的车辆关键点检测方法及装置
CN106295620A (zh) * 2016-08-28 2017-01-04 乐视控股（北京）有限公司 发型识别方法及发型识别装置
CN110231605A (zh) * 2019-05-09 2019-09-13 深圳市速腾聚创科技有限公司 人体行为的识别方法、装置、计算机设备和存储介质
CN110674785A (zh) * 2019-10-08 2020-01-10 中兴飞流信息科技有限公司 一种基于人体关键点跟踪的多人姿态分析方法
WO2021213742A1 (en) * 2020-04-22 2021-10-28 Continental Automotive Gmbh Method and system for keypoint detection based on neural networks
CN111914676A (zh) * 2020-07-10 2020-11-10 泰康保险集团股份有限公司 人体摔倒检测方法、装置、电子设备和存储介质
CN112200044A (zh) * 2020-09-30 2021-01-08 北京四维图新科技股份有限公司 异常行为检测方法、装置及电子设备
CN114170580A (zh) * 2021-12-01 2022-03-11 连云港杰瑞电子有限公司 一种面向高速公路的异常事件检测方法

Similar Documents

Publication Publication Date Title
US11288524B2 (en) Estimating object properties using visual image data
US11748620B2 (en) Generating ground truth for machine learning from time series elements
US20240304003A1 (en) Predicting three-dimensional features for autonomous driving
US11727668B2 (en) Using captured video data to identify pose of a vehicle
US20220182498A1 (en) System making decision based on data communication
US11308357B2 (en) Training data generation apparatus
CN114495064A (zh) 一种基于单目深度估计的车辆周围障碍物预警方法
CN116137655A (zh) 用于环绕视图增强的智能车辆系统和控制逻辑
CN115205311A (zh) 图像处理方法、装置、车辆、介质及芯片
WO2021233154A1 (zh) 可行驶区域检测方法、装置、设备及存储介质
WO2023206236A1 (zh) 一种目标的检测方法及相关装置
CN114842455A (zh) 障碍物检测方法、装置、设备、介质、芯片及车辆
WO2022204867A1 (zh) 一种车道线检测方法及装置
CN115082886A (zh) 目标检测的方法、装置、存储介质、芯片及车辆