WO2022161139A1

WO2022161139A1 - Driving direction test method and apparatus, computer device, and storage medium

Info

Publication number: WO2022161139A1
Application number: PCT/CN2022/070674
Authority: WO
Inventors: 程光亮; 石建萍
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-01-29
Filing date: 2022-01-07
Publication date: 2022-08-04
Also published as: CN112861683A

Abstract

The present disclosure provides a driving direction test method and apparatus, a computer device, and a storage medium. The method comprises: acquiring a driving image of vehicles surrounding a target vehicle during a driving process; identifying, from the image, at least one vehicle to be tested and at least two wheels of the vehicles to be tested; and for each vehicle to be tested, determining, on the basis of the at least two wheels of the vehicles to be detected, the driving directions of the vehicles to be tested. As such, the real driving direction of surrounding vehicles may be effectively tested, accuracy is high, robustness is good, and a target vehicle may be facilitated in determining the driving situation of the surrounding vehicles, so as to prepare a response strategy in advance, which can effectively reduce the probability of accidents occurring.

Description

Driving direction detection method, device, computer equipment and storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure claims priority to Chinese Patent Publication No. 202110129565.X filed on January 29, 2021, the entire contents of which are incorporated herein by reference.

technical field

The present disclosure relates to the technical field of automatic driving, and in particular, to a driving orientation detection method, device, computer equipment, and storage medium.

Background technique

With the development and progress of science and technology, cars have gradually entered more and more homes, bringing great convenience to users' travel. In recent years, with the development of automatic driving and assisted driving technology, people are more Driving requirements are also getting higher and higher, and the demand is gradually changing from L (level, level) 0 level assisted driving that provides alarm function to L1 and L2 level assisted driving that adds control function. Different from the alarm function provided by L0 level, L2-level assisted driving requires the assisted driving system to more accurately understand the distance, speed, driving direction, and intention of surrounding targets, and has higher requirements for perception and recognition.

In view of this, in automatic driving, how to effectively and accurately identify the driving directions of surrounding vehicles has become an urgent problem to be solved.

SUMMARY OF THE INVENTION

The embodiments of the present disclosure provide at least a driving direction detection method, apparatus, computer equipment, and storage medium.

In a first aspect, an embodiment of the present disclosure provides a method for detecting a driving direction, the method comprising: acquiring a driving image of surrounding vehicles of a target vehicle during driving; identifying at least one vehicle to be detected and each vehicle to be detected from the image at least two wheels of the vehicle to be detected; for each vehicle to be detected, the driving direction of the vehicle to be detected is determined based on the at least two wheels of the vehicle to be detected.

In this way, the wheels of the vehicle can be identified through the collected images, and the driving direction of the vehicle can be detected by the wheels, which can effectively detect the driving direction of the vehicle. Preparing coping strategies in advance can effectively reduce the probability of accidents.

In an optional implementation manner, determining the driving direction of the vehicle to be detected based on at least two wheels of the vehicle to be detected includes: determining attributes of each of the at least two wheels, wherein the The attributes include whether the wheel is a front wheel or a rear wheel; based on the attributes of each wheel, two to-be-detected wheels in the at least two wheels are determined; based on the preset reference point position information on each to-be-detected wheel and each to-be-detected wheel The attribute of the wheel is detected, and the driving direction of the vehicle to be detected is determined.

In this way, the wheel to be detected is screened out by the attributes of the wheel, and the driving direction of the vehicle to be detected is further determined by the information of the wheel to be detected, and the driving direction of the vehicle can be determined by screening the wheels in a targeted manner, with high accuracy and robustness. it is good.

In an optional implementation manner, the identifying at least one vehicle to be detected and at least two wheels of each vehicle to be detected from the image includes: identifying at least one vehicle and a plurality of vehicles from the image. wheels; determine the position information of each vehicle and the position information of each wheel; determine the vehicle to be detected and the position information of each vehicle to be detected in the at least one vehicle based on the position information of each vehicle and the position information of each wheel At least two wheels.

In this way, at least two wheels belonging to the same vehicle are matched by the position information, and the corresponding wheels of the vehicle can be accurately detected, which is simple and fast, has high accuracy and good robustness, and is beneficial to improve the accuracy of subsequent detection of the driving direction of the vehicle.

In an optional embodiment, the vehicle to be detected in the at least one vehicle and at least two wheels of each vehicle to be detected are determined based on the position information of each vehicle and the position information of each wheel. The method includes: matching the at least one vehicle and the plurality of wheels according to a preset distance threshold and an area coincidence ratio threshold to obtain a wheel corresponding to each vehicle in the at least one vehicle; The position information of the vehicle is used to determine the vehicle to be detected.

In an optional implementation manner, the at least one vehicle and the plurality of wheels are matched according to a preset distance threshold and an area coincidence ratio threshold, so as to obtain the at least one vehicle and each vehicle in the at least one vehicle. The wheel corresponding to the vehicle includes: for the wheel and vehicle to be matched, if the area overlap ratio between the wheel detection frame of the wheel and the vehicle detection frame of the vehicle reaches the preset area overlap ratio threshold, the wheel is used as the The candidate wheel of the vehicle; among the multiple candidate wheels of the vehicle, for any candidate wheel, if the relative distance between the wheel and other candidate wheels is greater than the preset distance threshold, the candidate wheel is determined as the vehicle's wheel.

In an optional implementation manner, the identifying at least one vehicle to be detected and at least two wheels of each vehicle to be detected from the image includes: determining each vehicle to be detected in the image; The image content corresponding to the vehicle to be detected in the image is used to identify at least two wheels of the vehicle to be detected.

In this way, the wheel is identified by the image corresponding to the identified vehicle, so that the matching accuracy between the vehicle and the wheel can be ensured.

In an optional implementation manner, the following steps are used to determine the attributes of the at least two wheels of the vehicle to be detected: according to the relative positional relationship between each of the at least two wheels and the vehicle to be detected, determine each wheel. properties of a wheel.

In an optional implementation manner, the vehicle to be detected is determined by the following steps: a vehicle that has one or more of the following correspondences with the target vehicle in the image is determined as the vehicle to be detected: The target vehicle is located in the same lane, is located in an adjacent lane of the lane where the target vehicle is located, is located in front of or behind the target vehicle and the distance between the two is less than a first preset distance, and is located on the side of the target vehicle And the distance between the two is smaller than the second preset distance, and the target vehicle travels in the opposite direction.

In this way, the vehicle whose driving direction needs to be detected in the surrounding environment can be effectively determined, which is beneficial to reduce the probability of possible traffic accident by detecting the driving direction of the vehicle, and help the target vehicle to respond in advance.

In an optional implementation manner, the determining the driving direction of the vehicle to be detected based on the preset reference point position information on each wheel to be detected and the attributes of each wheel to be detected includes: Detect the preset reference point position information on the wheels and the attributes of each wheel to be detected, as well as related driving information, and determine the driving direction of the vehicle to be detected, and the related driving information includes one or more of the following information: The road attribute of the lane where the vehicle to be detected is located, the distance between the vehicle to be detected and the lane line of the lane where the vehicle to be detected is located, and the current position of the vehicle to be detected in the lane where it is located.

In an optional implementation manner, in the case that the two wheels to be detected include two rear wheels or two front wheels, the method based on the preset reference point position information on each wheel to be detected and Attributes of each wheel to be detected, and determining the driving direction of the vehicle to be detected includes: determining the contact points of the two rear wheels or the two front wheels with the ground, and the distance between the two contact points The midpoint of the first connection line; determine the vertical line that intersects the first connection line with the midpoint as the intersection, wherein the vertical line is parallel to the ground; determine that the vertical line faces the front of the vehicle to be detected. The direction is the driving direction of the vehicle to be detected.

In an optional implementation manner, in the case that the two wheels to be detected include a front wheel and a rear wheel located on the same side of the vehicle to be detected, the preset reference point based on each wheel to be detected The location information and the attributes of each wheel to be detected, and determining the driving direction of the vehicle to be detected includes: determining the contact point between the front wheel and the ground and the contact point between the rear wheel and the ground, and determining the contact point between the front wheel and the ground The second connection line between the contact point of the front wheel and the contact point of the front wheel, wherein the starting point of the second connection line is the contact point of the rear wheel, and the end point of the second connection line is the front wheel contact point of the wheel; determine the direction indicated by the second connection line as the driving direction of the vehicle to be detected.

In an optional implementation manner, the method further includes: according to the determined driving direction of the vehicle to be detected, sending a prompt message or controlling the target vehicle.

In a second aspect, an embodiment of the present disclosure further provides a driving direction detection device, the device includes: a driving image acquisition module for acquiring driving images of surrounding vehicles of the target vehicle during driving; a vehicle information identification module for obtaining images from At least one vehicle to be detected and at least two wheels of each vehicle to be detected are identified in the image; a driving direction determination module is configured to, for each vehicle to be detected, based on the at least two wheels of the vehicle to be detected, determine The driving direction of the vehicle to be detected.

In a third aspect, embodiments of the present disclosure further provide a computer device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the computer device runs, the processing The processor and the memory communicate through a bus, and the machine-readable instructions execute the steps of the above-mentioned driving direction detection method when executed by the processor.

In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned driving direction detection method are executed.

The driving direction detection method, device, computer equipment and storage medium provided by the embodiments of the present disclosure, by acquiring the driving images of surrounding vehicles of the target vehicle during the driving process; at least one to-be-detected vehicle and each to-be-detected vehicle are identified from the images. at least two wheels of the vehicle; for each vehicle to be detected, determining the driving direction of the vehicle to be detected based on the at least two wheels of the vehicle to be detected.

In this way, detecting the driving direction of the surrounding vehicles according to the identified wheels of the surrounding vehicles can effectively detect the real driving direction of the surrounding vehicles, with high accuracy and good robustness, which is beneficial to help the target vehicle to judge the driving conditions of the surrounding vehicles, so that the Preparing coping strategies in advance can effectively reduce the probability of accidents.

In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to explain the technical solutions of the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings required in the embodiments, which are incorporated into the specification and constitute a part of the specification. The drawings illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.

1 is a flowchart of a method for detecting a driving direction provided by an embodiment of the present disclosure;

FIG. 2 is a flowchart of vehicle information identification in a driving direction detection method provided by an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of an image recognition model used in a driving direction detection method provided by an embodiment of the present disclosure;

4 is a flowchart of vehicle information identification in another driving direction detection method provided by an embodiment of the present disclosure;

FIG. 5 is one of the schematic diagrams of a driving orientation detection device according to an embodiment of the present disclosure;

FIG. 6 is the second schematic diagram of a driving direction detection device provided by an embodiment of the present disclosure;

FIG. 7 shows a schematic diagram of a computer device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only These are some, but not all, embodiments of the present disclosure. The components of the disclosed embodiments generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the disclosure provided in the accompanying drawings is not intended to limit the scope of the disclosure as claimed, but is merely representative of selected embodiments of the disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present disclosure.

In recent years, with the development of autonomous driving and assisted driving technology, people's requirements for autonomous driving and assisted driving experience are getting higher and higher. In autonomous driving, how to effectively and accurately identify the driving direction of surrounding vehicles has become one of the important research directions of L2-level assisted driving. Most of the current research schemes are given a picture with a vehicle, and the model is trained by manually marking the orientation of the vehicle (for example, dividing the orientation into 32 sub-directions) for vehicle orientation recognition, but this method has obvious defects. First, due to the difference in perspective and the distance of the vehicle, it is difficult to label the orientation accurately, which will increase the difficulty of network training. Secondly, the orientation labeling method takes the vehicle as a mass point and the actual center of the vehicle as the mass center, but the mass point is an idealized model, and the default object size and shape have little response to the research problem; but in actual use, The shapes of various vehicle models are quite different, and there will be a large deviation in converting the obtained orientation to the world coordinate system, resulting in inaccurate orientation prediction in practical applications, which will cause greater difficulty for assisted driving or automatic driving control, and even Driving safety may be affected by giving wrong signals.

Based on the above research, the present disclosure provides a driving direction detection method. By collecting the driving images of surrounding vehicles of a target vehicle, the wheels of surrounding vehicles are identified from the driving images of surrounding vehicles. According to the preset reference point position information on the wheels and the wheels It can effectively detect the real driving direction of the surrounding vehicles, which is more suitable for the real driving conditions of the surrounding vehicles. It has high accuracy and good robustness, which is beneficial to help the target vehicle to judge the driving conditions of the surrounding vehicles. , to prepare coping strategies in advance, which can effectively improve the safety of autonomous driving or assisted driving.

Like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

In order to facilitate the understanding of this embodiment, a driving direction detection method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the driving direction detection method provided by the embodiment of the present disclosure is generally a computer device with a certain computing capability. The computer equipment includes, for example: terminal equipment, server, automatic driving equipment, assisted driving equipment or other processing equipment, and the terminal equipment can be user equipment (User Equipment, UE), mobile equipment, user terminal, terminal, cellular phone, personal digital assistant (Personal Digital Assistant, PDA), handheld devices, computing devices, in-vehicle devices, wearable devices, personal computers, notebook computers, etc. In some possible implementations, the driving direction detection method may be implemented by a processor in a computer device calling computer-readable instructions stored in a memory.

The following describes the driving direction detection method provided by the embodiment of the present disclosure by taking the execution subject as a terminal device as an example.

Please refer to FIG. 1 , which is a flowchart of a method for detecting a driving direction according to an embodiment of the present disclosure. As shown in FIG. 1, the method includes steps S101 to S103.

S101: Acquire a running image of surrounding vehicles in the running process of the target vehicle.

In this step, while the target vehicle is traveling, images of surrounding vehicles of the target vehicle may be acquired.

The driving image of the surrounding vehicles may be collected by an image acquisition device installed on the target vehicle, or may be captured by a user who rides the target vehicle using a user terminal, or may be captured by a user terminal installed in the target vehicle. The image is collected by the image acquisition equipment, and then obtained from the image storage location such as the server through the Internet and other connection methods.

The driving image of the surrounding vehicle may be a panoramic image around the target vehicle, such as a 360-degree panoramic image centered on the target vehicle, or may be an image in one or more directions of the target vehicle. For example, if the target vehicle is driving in the leftmost lane of the road, only one or more of the right image, front image, rear image, right front image and right rear image of the target vehicle can be collected.

For example, an autonomous driving device or an assisted driving device includes, in addition to a processor and memory, various sensors, such as one or more cameras. The camera can be installed on the target vehicle, for example, it can be fixed on the rearview mirror, and the lens faces the direction of the front of the vehicle. During the driving process of the target vehicle, the camera can capture the image of the front of the vehicle. Other cameras can also be installed at different positions of the target vehicle to obtain images in different directions. The present disclosure does not limit this. The vehicle images captured by each camera are respectively sent back to the processor and memory for subsequent processing.

There may be multiple driving images of surrounding vehicles obtained in this step. Since the subsequent processing procedures of these images are similar, for the sake of simplicity, a driving image of surrounding vehicles is used as an example for description in the following.

S102: Identify at least one vehicle to be detected and at least two wheels of each vehicle to be detected from the image.

In this step, after the image is acquired, the image can be recognized by using an image recognition model, or traditional image recognition technology or other means, so as to identify one or more vehicles to be detected from the image. , and at least two wheels of each vehicle to be detected.

For example, the vehicle detection algorithm can be used to detect the target of the vehicle in the image to obtain the vehicle detection frame, and the wheel detection algorithm can be used to detect the target of the visible wheel in the image to obtain the wheel detection frame. Based on the rule that the wheels in the vehicle detection frame belong to the same vehicle, at least two wheels of each vehicle to be detected can be obtained. In some examples, for the sake of accuracy, if a vehicle detection frame of a certain vehicle has only one wheel detection frame, it may be considered that the vehicle is not a vehicle to be detected.

In actual situations, due to the positional relationship between vehicles during driving, it is generally difficult to see the whole picture of the surrounding vehicles in the driving state, and only a part of the surrounding vehicles may be observed from the position of the target vehicle. Therefore, from the image In most cases, only part of the vehicle can be recognized. For example, for an ordinary domestic vehicle, it generally has only four tires, so generally only two wheels of a vehicle can be seen from an image (for example, from a front view, a rear view, a left view, or a right view) , or three wheels (eg from a front right, rear right, front left, or rear left).

S103: For each vehicle to be detected, determine the driving direction of the vehicle to be detected based on at least two wheels of the vehicle to be detected.

In this step, after identifying at least two wheels of a vehicle to be detected, the vehicle to be detected can be obtained by using the relevant information of the at least two wheels, such as the position, belonging to the front wheel or the rear wheel, etc. driving direction.

Specifically, in some possible embodiments, the driving direction of the vehicle to be detected is determined based on at least two wheels of the vehicle to be detected, and the driving direction may be determined by the following steps.

determining an attribute of each of the at least two wheels, wherein the attribute of the wheel includes whether the wheel is a front wheel or a rear wheel; based on the attribute of each wheel, determining two to-be-detected wheels in the at least two wheels; based on Preset reference point position information on each to-be-detected wheel and attributes of each to-be-detected wheel determine the driving direction of the to-be-detected vehicle.

In the above steps, after at least two wheels are identified, the attributes of each wheel can be further determined, for example, the attributes of each wheel can be determined by factors such as the position of the wheel, the relative positional relationship between the wheel and the body, etc., and then Two to-be-detected wheels that can be used for judging the driving direction of the vehicle can be determined. The driving direction of the vehicle to be detected is determined by combining the attributes of each wheel to be detected and the position information of the preset reference point on each wheel to be detected.

In a specific application, when determining the attributes of each wheel, the following steps may be used to determine the attributes of at least two wheels of the vehicle to be detected: according to the relationship between each of the at least two wheels and the vehicle to be detected The relative position relationship determines the properties of each wheel.

In this step, the position information of each wheel in the image and the position information of the preset vehicle part of the vehicle to be detected in the image can be used to determine the relationship between each wheel and the to-be-detected vehicle. The relative positional relationship of the vehicle is detected to derive the properties of each wheel.

Among them, the attributes of the wheel include the front wheel and the rear wheel, that is, the wheel is the front wheel, or the wheel is the rear wheel. The preset vehicle parts may be the front, the rear, and the like. Taking the preset vehicle part as the front of the vehicle as an example, according to the relative positional relationship between each of the at least two wheels and the front of the vehicle to be detected, it can be determined that the wheel with a smaller relative distance from the front of the vehicle is the front wheel, and the wheel with a larger relative distance can be determined as the front wheel. The wheels are rear wheels, resulting in the properties of each wheel.

Wherein, the position information of the preset reference point on the wheel to be detected may refer to the position information of the preset reference point on the wheel to be detected, and the position of the preset reference point on each wheel to be detected is the same, for example, on each wheel , the preset reference points are at the center of each wheel, or the frontmost point of each wheel, or the rearmost point of each wheel, or the bottommost point of each wheel, etc. . In addition, the wheel is dynamic, but in the embodiment of the present disclosure, the preset reference point refers to a static reference point. For example, no matter where the wheel rotates, the preset reference point may refer to the moment corresponding to the captured image. The reference point for the recorded preset position on the wheel.

The present disclosure provides a driving direction detection method. By collecting driving images of surrounding vehicles of a target vehicle, the wheels of surrounding vehicles are identified from the driving images of surrounding vehicles, and the surrounding vehicles are determined according to preset reference point position information on the wheels and attributes of the wheels. It can effectively detect the real driving direction of the surrounding vehicles, which is more suitable for the real driving conditions of the surrounding vehicles, with high accuracy and good robustness, which is conducive to helping the target vehicle to judge the driving conditions of the surrounding vehicles, so as to prepare the countermeasures in advance. Improve the safety of autonomous or assisted driving.

The foregoing S101 to S103 will be described in detail below with reference to specific embodiments.

For the above step S102, please refer to FIG. 2, which is a flowchart of vehicle information identification in a driving direction detection method provided by an embodiment of the present disclosure. As shown in FIG. 2, step S102 includes the following steps.

S1021: Identify at least one vehicle and multiple wheels from the image.

Wherein, recognizing the image may include constructing an image recognition model through a deep learning-based neural network, and recognizing the image through a trained image recognition model to obtain at least one vehicle and multiple wheels. time, improving the recognition speed, etc., the embodiment of the present disclosure can perform the recognition of the vehicle and the recognition of the wheel in parallel.

Specifically, for the consideration of processing speed and model structure deployment, the identification of the vehicle and the identification of the wheel can be performed in parallel, that is, the two tasks can be combined into the same network, as shown in Figure 3, 3 is a schematic diagram of an image recognition model used in the driving direction detection method provided by the embodiment of the present disclosure. The image recognition model 300 constructed in the embodiment of the present disclosure may include a feature extraction network 310, a vehicle detection network 320, and a wheel detection network 330. The vehicle detection network 320 and the wheel detection network 330 share a common feature extraction network 310, and the vehicle detection network 320 and the wheel detection network 330 can use the current mainstream two-stage network (faster-RCNN (faster-Region Convolutional Neural Network, region-based update). Fast convolutional neural network) and other networks) or one-stage network (RetinaNet (retina network), FCOS (Fully Convolutional One-Stage Object Detection, first-order fully convolutional target detection) network), etc. This solution is not limited to the corresponding network The structure, residual networks ResNet50, ResNet18, and self-designed network structures can also be incorporated into the usage scheme of the embodiment of the present disclosure, and the detection scheme and the backbone network structure can be set according to project needs.

After the above network is constructed, each vehicle and the wheel of each vehicle in the sample image can be marked by annotating the sample image. For the wheels corresponding to each vehicle, only the visible wheels can be marked, and when a certain wheel is not visible, the marking can be omitted. For each wheel, its attributes are marked as front wheel and rear wheel. When there are multiple wheels on one side of the vehicle, only the front wheel is marked as the front wheel, and the rest are all rear wheels. The sample image is used to train the constructed network, and the trained network is used as an image recognition model to recognize the obtained driving images of surrounding vehicles, and then the recognition results including at least one vehicle and multiple wheels in the image can be obtained. , and in addition, the identification result may further include: vehicle information including the position information of each vehicle, and wheel information including the position information of each wheel. In some examples, the recognition results may also contain attributes for each wheel.

S1022: Determine the position information of each vehicle and the position information of each wheel.

Wherein, to determine the position information of each vehicle and the position information of each wheel, the position information of each vehicle in the image and the position information of each wheel in the image may be obtained, or the position information of each wheel in the image may be obtained. The position information of each vehicle in the actual scene, and the position information of each wheel in the actual scene. As long as the reference frame of the position information of the vehicle is the same as the reference frame of the position information of the wheels.

Exemplarily, on the premise that the vehicle and the wheel in the image are identified by the image recognition model, the position information of the identified vehicle and the position information of the wheel can be obtained from the identification result. For example, the vehicle detection frame of each vehicle can be obtained from the recognition result, which contains the position information of the vehicle, and the wheel detection frame of each wheel can be obtained from the recognition result, and the detection frame contains the position information of the wheel. location information.

S1023: Based on the position information of each vehicle and the position information of each wheel, determine a vehicle to be detected and at least two wheels of each vehicle to be detected in the at least one vehicle.

In this step, after determining the position information of each vehicle and the position information of each wheel, the vehicle and the wheel can be matched according to the preset distance threshold and area coincidence ratio threshold, so as to obtain the corresponding vehicle-to-vehicle relationship. wheel. For example, based on the rule that the wheels in the vehicle detection frame belong to the same vehicle, for the wheel and vehicle to be matched, if the overlap ratio of the area between the wheel detection frame of the wheel and the vehicle detection frame of the vehicle reaches a preset area If the coincidence ratio threshold is exceeded, the wheel is regarded as the candidate wheel of the vehicle. Among multiple candidate wheels of a certain vehicle, for a certain candidate wheel, if the relative distance between the wheel and other candidate wheels is greater than the preset distance threshold, the candidate wheel can be determined as the wheel of the vehicle, so that we can obtain At least two wheels of each vehicle to be inspected. In combination with the position information of each vehicle, the relevant vehicles to be detected are determined, thereby obtaining at least two wheels of the vehicle to be detected and attributes of the wheels. Alternatively, the vehicle to be detected may be determined first, and then at least two wheels of the vehicle to be detected and attributes of the corresponding wheels are obtained by matching between the position information of the vehicle to be detected and the position information of each wheel.

Further, in the case where two wheels of the vehicle to be detected are determined, these two wheels may be used as the wheels to be detected. In the case where three wheels of the vehicle to be detected are determined, it can be determined that two wheels on the same side of the vehicle to be detected are the wheels to be detected. For example, one front wheel and one rear wheel on the same side of the vehicle to be tested may be used as the wheel to be tested; or, if the three wheels include two front wheels, the two front wheels may be used as the wheel to be tested; or, if The three wheels include two rear wheels, which can be used as the wheels to be detected.

For the above step S102, please refer to FIG. 4, which is a flowchart of vehicle information identification in another driving direction detection method provided by an embodiment of the present disclosure. As shown in FIG. 4, step S102 includes the following steps.

S1024: Determine each vehicle to be detected in the image.

Wherein, to determine the vehicle to be detected in the image, the image may be identified through a neural network, etc., so as to identify the vehicle in the image, and then the identified vehicle is used as the vehicle to be detected. Alternatively, at least one vehicle may be selected from the identified multiple vehicles as the vehicle to be detected, and the specific processing process will be described in detail later.

S1025: Identify at least two wheels of the vehicle to be detected from the image content corresponding to the vehicle to be detected in the image.

In this step, after the vehicle to be detected in the image is determined, the image content of the vehicle to be detected is further identified to obtain at least two wheels of the vehicle to be detected.

In some possible embodiments, for the vehicle to be detected in step S102, the vehicle to be detected may be determined through the following steps: a vehicle that has one or more of the following correspondences between the image and the target vehicle Determined as the vehicle to be detected: located in the same lane as the target vehicle, located in the adjacent lane of the lane where the target vehicle is located, located in front of or behind the target vehicle, and the distance between the two is less than the first preset distance , located on the side of the target vehicle and the distance between the two is less than a second preset distance, and travels in the opposite direction of the target vehicle.

In some possible embodiments, for step S103, step S103 includes:

The driving direction of the vehicle to be detected is determined based on the preset reference point position information on each wheel to be detected and the attributes of each wheel to be detected, as well as relevant driving information, where the relevant driving information includes one or more of the following information Type: the road attribute of the lane where the vehicle to be detected is located, the distance between the vehicle to be detected and the lane line of the lane where it is located, and the current position of the vehicle to be detected in the lane where it is located.

In this step, in order to more accurately predict the driving direction of the vehicle to be detected, on the basis of the preset reference point position information on each wheel to be detected and the attributes of the wheel, the driving information of the vehicle to be detected can be combined with the above-mentioned relevant driving information to predict the driving direction of the vehicle to be detected. Driving direction.

Exemplarily, when the vehicle to be detected on a ramp or other road needs to merge into the road where the target vehicle is located, the real-time driving direction of the vehicle to be detected can be predicted in combination with the above information. For example, the road on which the vehicle to be detected is located is a road that cannot be steered. At this time, even if it is determined by the wheels to be detected that the driving direction of the vehicle to be detected and the current driving direction of the target vehicle intersect, the road attribute represents The vehicle cannot be steered during normal driving, so if the vehicle to be detected has no problems in normal driving, it can be temporarily considered that the vehicle to be detected is only fine-tuning the driving state and will still drive along the extension direction of the road where it is located. The extension direction of the road is considered to be the driving direction that the vehicle to be detected will still maintain. For another example, the vehicle to be detected is currently driving on the right lane line of the lane where it is located, which is far from the left lane line, and the detected driving direction has little deviation from the extension direction of the lane it is in, then it is also possible to It is considered that the vehicle to be detected only needs to fine-tune the driving state, and its final driving direction is also the extension direction of the lane in which it is located.

Further, in order to enhance security, continuous monitoring can also be performed for the above-mentioned situations. For example, if it is detected in the first frame of images that the driving direction of the vehicle to be detected intersects with the current driving direction of the target vehicle or the vehicle to be detected is driving on the lane line of the lane where it is located, the vehicle to be detected can be temporarily put into monitoring list, and further detect the driving direction of the vehicle to be detected in the second frame image, the third frame image, and the i-th frame image (i can be set according to the current speed of the target vehicle and/or the distance of the vehicle to be detected), if If it is found that the driving direction of the vehicle to be detected has returned to normal, the vehicle to be detected will no longer be placed on the monitoring list. If there is still a problem with the driving direction of the vehicle to be detected, the assisted driving device or the automatic driving device can issue a prompt message or control The target vehicle evades.

In some possible embodiments, when the two to-be-detected wheels include two rear wheels or two front wheels, for step S103, step S103 includes:

Determine the contact points of the two rear wheels or the two front wheels with the ground, and the midpoint of the first connection line between the two contact points; determine the first connection line with the midpoint as the intersection point The vertical line is parallel to the ground; the direction of the vertical line toward the front of the vehicle to be detected is determined as the driving direction of the vehicle to be detected.

In this step, when it is identified that the two to-be-detected wheels of the to-be-detected vehicle are the two rear wheels or the two front wheels, the preset reference points on the two rear wheels or the front wheels may be determined first, such as The contact point between the rear wheel or the front wheel and the ground is used as the preset reference point, so that the first connection can be obtained through the two contact points, and then the orientation of the vertical line parallel to the ground at the midpoint of the first connection is The driving direction of the vehicle to be detected.

Further, since the vertical line indicates two directions, the two directions indicated by the vertical line can be combined with the components located at the front and rear positions of the vehicle to be detected, the direction of the road where the vehicle to be detected is located, etc. A direction is determined in the , as the driving direction of the vehicle to be detected.

In the embodiment of the present disclosure, the contact point between the rear wheel or the front wheel and the ground is used as an example for description, but it is not limited to this. In other embodiments, the center of the hub of the front wheel or the rear wheel may also be used. Points etc. are preset reference points.

In some possible embodiments, when the two to-be-detected wheels include a front wheel and a rear wheel located on the same side of the to-be-detected vehicle, for step S103, step S103 includes: determining that the front wheel is connected to the ground The contact point of the rear wheel and the contact point of the ground, and the second connecting line between the contact point of the rear wheel and the contact point of the front wheel, wherein the starting point of the second connecting line is the rear wheel The end point of the second connection line is the contact point of the front wheel; it is determined that the direction indicated by the second connection line is the driving direction of the vehicle to be detected.

In this step, when it is recognized that the wheels of the vehicle to be detected are one rear wheel and one front wheel on the same side, preset reference points on the rear wheel and the front wheel may be determined first, such as the rear wheel and the front wheel The contact point with the ground is used as a preset reference point, so that a second connection line can be obtained through the two contact points, and then the direction of the second connection line is used as the driving direction of the vehicle to be detected. The direction of the second connection line is the direction from the start point of the second connection line to the end point of the second connection line.

In some possible embodiments, the method further includes: according to the determined driving direction of the vehicle to be detected, sending out prompt information or controlling the target vehicle.

In this way, the influence on the target vehicle can be judged according to the determined driving direction of the vehicle to be detected, so that when there is influence or hidden danger, a prompt message can be issued or the target vehicle can be controlled, thereby helping Improve the driving safety of the target vehicle.

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.

Based on the same inventive concept, the embodiment of the present disclosure also provides a driving direction detection device corresponding to the driving direction detection method. For the implementation of the apparatus, reference may be made to the implementation of the method, and the repetition will not be repeated.

Please refer to FIG. 5 and FIG. 6 , FIG. 5 is one of the schematic diagrams of a driving direction detection device provided by an embodiment of the present disclosure, and FIG. 6 is the second schematic diagram of a driving direction detection device provided by an embodiment of the present disclosure. As shown in FIG. 5 , the driving direction detection device 500 provided by the embodiment of the present disclosure includes: a driving image acquisition module 510 , configured to acquire driving images of surrounding vehicles during the driving process of the target vehicle. The vehicle information identification module 520 is configured to identify at least one vehicle to be detected and at least two wheels of each vehicle to be detected from the image. The driving direction determination module 530 is configured to, for each vehicle to be detected, determine the driving direction of the vehicle to be detected based on at least two wheels of the vehicle to be detected.

In an optional implementation manner, the driving direction determination module 530 is specifically configured to: determine attributes of each of the at least two wheels, wherein the attributes of the wheels include whether the wheel is a front wheel or a rear wheel; the attributes of the wheels, to determine the two to-be-detected wheels in the at least two wheels; based on the preset reference point position information on each of the to-be-detected wheels and the attributes of each to-be-detected wheel, to determine the driving of the to-be-detected vehicle towards.

In an optional embodiment, the vehicle information identification module 520 is specifically configured to: identify at least one vehicle and multiple wheels from the image; determine the position information of each vehicle and the position information of each wheel; Based on the position information of each vehicle and the position information of each wheel, a vehicle to be detected in the at least one vehicle and at least two wheels of each vehicle to be detected are determined.

In an optional implementation manner, the vehicle information identification module 520 is specifically configured to: match the at least one vehicle and the plurality of wheels according to a preset distance threshold and an area coincidence ratio threshold to obtain the The wheel corresponding to each vehicle in the at least one vehicle is determined; the vehicle to be detected is determined according to the position information of each vehicle.

In an optional implementation manner, the vehicle information identification module 520 is specifically configured to: for the wheel to be matched with the vehicle, if the overlap ratio of the area between the wheel detection frame of the wheel and the vehicle detection frame of the vehicle reaches the required The preset area coincidence ratio threshold, the wheel is used as the candidate wheel of the vehicle; among the multiple candidate wheels of the vehicle, for any candidate wheel, if the relative distance between the wheel and other candidate wheels is greater than the predetermined wheel Set the distance threshold to determine that the candidate wheel is the wheel of the vehicle.

In an optional implementation manner, the vehicle information identification module 520 is specifically configured to: determine each vehicle to be detected in the image; identify the image content corresponding to the vehicle to be detected in the image; at least two wheels of the vehicle to be detected.

In an optional implementation manner, the vehicle information identification module 520 is configured to use the following steps to determine attributes of at least two wheels of the vehicle to be detected: according to the relationship between each wheel of the at least two wheels and the The relative positional relationship of the vehicle is detected, and the attributes of each wheel are determined.

In an optional implementation manner, the vehicle information identification module 520 is configured to determine the vehicle to be detected through the following steps: identifying the vehicle in the image with the target vehicle having one or more of the following correspondences: Determined as the vehicle to be detected: located in the same lane as the target vehicle, located in the adjacent lane of the lane where the target vehicle is located, located in front of or behind the target vehicle, and the distance between the two is less than the first preset distance , located on the side of the target vehicle and the distance between the two is less than a second preset distance, and travels in the opposite direction of the target vehicle.

In an optional embodiment, the driving direction determination module 530 is used to determine the driving of the vehicle to be detected based on the preset reference point position information on each wheel to be detected and the attributes of each wheel to be detected. When the direction is used, it is specifically used to: determine the driving direction of the vehicle to be detected based on the preset reference point position information on each wheel to be detected, the attributes of each wheel to be detected, and the relevant driving information, and the relevant driving information Including one or more of the following information: road attributes of the lane where the vehicle to be detected is located, the distance between the vehicle to be detected and the lane line of the lane where the vehicle to be detected is located, the vehicle to be detected in the lane where it is located 's current location.

In an optional implementation manner, in the case that the two wheels to be detected include two rear wheels or two front wheels, the driving direction determination module 530 is used to determine the driving direction based on the predetermined value on each wheel to be detected. Assuming the reference point position information and the attributes of each wheel to be detected, when determining the driving direction of the vehicle to be detected, it is specifically used to: determine the contact points of the two rear wheels or the two front wheels with the ground respectively, and the midpoint of the first connection line between the two contact points; determine the vertical line that intersects the first connection line with the midpoint as the intersection point; determine the direction of the vertical line toward the front of the vehicle to be detected is the driving direction of the vehicle to be detected.

In an optional implementation manner, in the case that the two wheels to be detected include front wheels and rear wheels located on the same side of the vehicle to be detected, the driving direction determination module 530 is used to determine the driving direction based on each wheel to be detected. When detecting the preset reference point position information on the wheels and the attributes of each wheel to be detected, and determining the driving direction of the vehicle to be detected, it is specifically used for: determining the contact point between the front wheel and the ground and the contact point between the rear wheel and the rear wheel. a contact point of the ground, and a second connecting line between the contact point of the rear wheel and the contact point of the front wheel, wherein the starting point of the second connecting line is the contact point of the rear wheel, so The end point of the second connection line is the contact point of the front wheel; the direction indicated by the second connection line is determined as the driving direction of the vehicle to be detected.

In an optional implementation manner, as shown in FIG. 6 , the driving direction detection device 500 further includes: a vehicle prompting control module 540, configured to issue prompt information or control the vehicle according to the determined driving direction of the vehicle to be detected. the target vehicle.

The driving direction detection device provided by the embodiment of the present disclosure obtains the driving images of surrounding vehicles of the target vehicle during the driving process; at least one vehicle to be detected and at least two wheels of each vehicle to be detected are identified from the images, wherein , the attributes of the wheels include front wheels and rear wheels; for each vehicle to be detected, based on the preset reference point position information on each wheel of the vehicle to be detected and the attributes of each wheel, determine the vehicle to be detected driving direction.

For the description of the processing flow of each module in the apparatus and the interaction flow between the modules, reference may be made to the relevant descriptions in the foregoing method embodiments, which will not be described in detail here.

An embodiment of the present disclosure further provides a computer device 700 , as shown in FIG. 7 , which is a schematic structural diagram of the computer device 700 provided by the embodiment of the present disclosure, including: a processor 710 , a memory 720 , and a bus 730 . The memory 720 stores machine-readable instructions executable by the processor 710. When the computer device 700 is running, the processor 710 communicates with the memory 720 through the bus 730, and the machine-readable instructions are executed. When executed, the processor 710 may execute the steps of the driving direction detection method in the above method embodiments.

For the specific execution process of the above instruction, reference may be made to the steps of the driving direction detection method described in the embodiment of the present disclosure, which will not be repeated here.

Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the driving direction detection method described in the above method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

Wherein, the computer program product can be specifically implemented by hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the system and device described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided by the present disclosure, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of software products in essence, or the parts that contribute to the prior art or the parts of the technical solutions. The computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure rather than limit them. The protection scope of the present disclosure is not limited thereto, although referring to the foregoing The embodiments describe the present disclosure in detail. Those of ordinary skill in the art should understand that: any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure. Changes can be easily thought of, or equivalent replacements are made to some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered in the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

A driving direction detection method, characterized in that the method comprises:

Obtain the driving images of surrounding vehicles during the driving process of the target vehicle;

identifying from the image at least one vehicle to be inspected and at least two wheels of each vehicle to be inspected;

For each vehicle to be detected, the driving direction of the vehicle to be detected is determined based on at least two wheels of the vehicle to be detected.
The method according to claim 1, wherein determining the driving direction of the vehicle to be detected based on at least two wheels of the vehicle to be detected comprises:

determining an attribute of each of the at least two wheels, wherein the attribute of the wheel includes whether the wheel is a front wheel or a rear wheel;

determining two of the at least two wheels to be detected based on attributes of the respective wheels;

Based on the preset reference point position information on each wheel to be detected and the attributes of each wheel to be detected, the driving direction of the vehicle to be detected is determined.
The method according to claim 1 or 2, wherein the identifying at least one vehicle to be detected and at least two wheels of each vehicle to be detected from the image comprises:

identifying at least one vehicle and a plurality of wheels from the image;

Determine the position information of each vehicle and the position information of each wheel;

Based on the position information of each vehicle and the position information of each wheel, a vehicle to be detected in the at least one vehicle and at least two wheels of each of the vehicles to be detected are determined.
The method according to claim 3, wherein the vehicle to be detected in the at least one vehicle and the position information of each vehicle to be detected are determined based on the position information of each vehicle and the position information of each wheel. At least two wheels include:

matching the at least one vehicle and the plurality of wheels according to a preset distance threshold and an area coincidence ratio threshold to obtain a wheel corresponding to each vehicle in the at least one vehicle and the vehicle;

The vehicle to be detected is determined according to the position information of each vehicle.
The method according to claim 4, wherein the at least one vehicle and the plurality of wheels are matched according to a preset distance threshold and an area coincidence ratio threshold to obtain the at least one vehicle The wheels of each vehicle corresponding to that vehicle include:

For the wheel and vehicle to be matched, if the area overlap ratio between the wheel detection frame of the wheel and the vehicle detection frame of the vehicle reaches the preset area overlap ratio threshold, the wheel is used as the candidate wheel of the vehicle;

Among the multiple candidate wheels of the vehicle, for any candidate wheel, if the relative distance between the wheel and other candidate wheels is greater than the preset distance threshold, the candidate wheel is determined to be the wheel of the vehicle.
The method according to claim 1 or 2, wherein the identifying at least one vehicle to be detected and at least two wheels of each vehicle to be detected from the image comprises:

determining each vehicle to be detected in the image;

At least two wheels of the vehicle to be detected are identified from the image content corresponding to the vehicle to be detected in the image.
The method according to claim 2, wherein the following steps are used to determine the attributes of at least two wheels of the vehicle to be detected:

The attribute of each wheel is determined according to the relative positional relationship between each of the at least two wheels and the vehicle to be detected.
The method according to any one of claims 1 to 7, wherein the vehicle to be detected is determined through the following steps:

A vehicle in the image that has one or more of the following correspondences with the target vehicle is determined as the vehicle to be detected:

It is located in the same lane as the target vehicle, is located in the adjacent lane of the lane where the target vehicle is located, is located in front of or behind the target vehicle and the distance between the two is less than the first preset distance, and is located on the side of the target vehicle and the distance between the two is less than the second preset distance, and the target vehicle travels in the opposite direction.
The method according to any one of claims 2 to 8, wherein the vehicle to be detected is determined based on preset reference point position information on each wheel to be detected and attributes of each wheel to be detected direction of travel, including:

The driving direction of the vehicle to be detected is determined based on the preset reference point position information on each wheel to be detected, the attributes of each wheel to be detected, and related driving information, where the relevant driving information includes one of the following information Or multiple: road attributes of the lane where the vehicle to be detected is located, the distance between the vehicle to be detected and the lane line of the lane where it is located, and the current position of the vehicle to be detected in the lane where it is located.
The method according to any one of claims 2 to 8, wherein, in the case that the two wheels to be detected include two rear wheels or two front wheels, the The preset reference point position information and the attributes of each wheel to be detected, determine the driving direction of the vehicle to be detected, including:

determining the contact points of the two rear wheels or the two front wheels with the ground, and the midpoint of the first connecting line between the two contact points;

determining a vertical line intersecting the first connecting line with the midpoint as an intersection, wherein the vertical line is parallel to the ground;

The direction of the vertical line toward the front of the vehicle to be detected is determined as the driving direction of the vehicle to be detected.
The method according to any one of claims 2 to 8, wherein, in the case that the wheels to be detected include front wheels and rear wheels located on the same side of the vehicle to be detected, the Detecting the preset reference point position information on the wheels and the attributes of each wheel to be detected, and determining the driving direction of the vehicle to be detected, including:

Determining the point of contact of the front wheel with the ground and the point of contact of the rear wheel with the ground, and a second connection from the point of contact of the rear wheel and the point of contact of the front wheel, wherein the The starting point of the second connection line is the contact point of the rear wheel, and the end point of the second connection line is the contact point of the front wheel;

It is determined that the direction indicated by the second connection line is the driving direction of the vehicle to be detected.
The method according to any one of claims 1 to 11, wherein the method further comprises:

According to the determined driving direction of the vehicle to be detected, a prompt message is issued or the target vehicle is controlled.
A driving direction detection device, characterized in that the device comprises:

The driving image acquisition module is used to acquire the driving images of surrounding vehicles during the driving process of the target vehicle;

a vehicle information identification module for identifying at least one vehicle to be detected and at least two wheels of each vehicle to be detected from the image;

The driving direction determination module is configured to, for each vehicle to be detected, determine the driving direction of the vehicle to be detected based on at least two wheels of the vehicle to be detected.
A computer device, characterized in that it includes: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the computer device runs, the processor and the memory communicate with each other. The machine-readable instructions are executed by the processor to perform the steps of the driving direction detection method according to any one of claims 1 to 12.
A computer-readable storage medium, characterized in that, a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the driving direction detection method according to any one of claims 1 to 12 is executed. step.