CN114519848A

CN114519848A - Movement intention determining method, device and equipment and storage medium

Info

Publication number: CN114519848A
Application number: CN202210122826.XA
Authority: CN
Inventors: 何园; 蒋沁宏
Original assignee: Sensetime Group Ltd; Honda Motor Co Ltd
Current assignee: Sensetime Group Ltd; Honda Motor Co Ltd
Priority date: 2022-02-09
Filing date: 2022-02-09
Publication date: 2022-05-20
Also published as: WO2023151241A1

Abstract

The embodiment of the application provides a movement intention determining method, a movement intention determining device, movement intention determining equipment and a storage medium, wherein a traffic image is acquired; determining, based on the traffic image, headlight information of a vehicle and orientation information of the vehicle in the traffic image; determining an intent to move of the vehicle based on the headlight information and the orientation information.

Description

Movement intention determining method, device, equipment and storage medium

Technical Field

The embodiment of the application relates to the technical field of intelligent driving, and relates to but is not limited to a movement intention determining method, a movement intention determining device, movement intention determining equipment and a storage medium.

Background

In recent years, with the rise of driving lamps and wide lamps, the lamps have a complicated situation, and whether the vehicle is braking or turning cannot be accurately judged only according to the brightness of the left and right whole lamps.

Disclosure of Invention

The embodiment of the application provides a technical scheme for determining the movement intention.

The technical scheme of the embodiment of the application is realized as follows:

the embodiment of the application provides a movement intention determining method, which comprises the following steps: acquiring a traffic image; determining, based on the traffic image, headlight information of a vehicle and orientation information of the vehicle in the traffic image; determining an intent to move of the vehicle based on the headlight information and the orientation information.

In some embodiments, the determining, based on the traffic image, headlight information of a vehicle in the traffic image and orientation information of the vehicle comprises: determining, based on the traffic image, location information of illuminated target headlights of the vehicle in the traffic image; determining appearance information of vehicles in the traffic image based on the traffic image; determining orientation information of a head of the vehicle based on the appearance information of the vehicle; determining an intent to move of the vehicle based on the headlight information and the orientation information, comprising: determining the movement intention of the vehicle based on the position information of the target vehicle lamp of the vehicle and the orientation information of the head of the vehicle. In this way, the position information of the target vehicle lamp is combined with the orientation information of the vehicle, and the steering of the vehicle can be predicted more accurately.

In some embodiments, the target headlight is a single turn light, and the determining the movement intention of the vehicle based on the position information of the target headlight of the vehicle and the orientation information of the head of the vehicle includes: determining steering information indicated by a steering lamp based on the position information of the single steering lamp and the orientation information of the head of the vehicle; determining a steering intent of the vehicle according to the steering information. Therefore, the steering information indicated by the steering lamp can be accurately obtained by analyzing the position of the single steering lamp and the orientation of the vehicle head, and the steering intention of the vehicle can be accurately predicted.

In some embodiments, the determining the movement intention of the vehicle based on the position information of the target headlight of the vehicle and the orientation information of the head of the vehicle includes: and determining that the vehicle is in a braking state in response to that brake light information is not included in the light information and the target light is a plurality of steering lights. Thus, by identifying whether a plurality of turn signals are simultaneously in an on state, whether the vehicle is in a braking state can be accurately predicted.

In some embodiments, the method further comprises: determining vehicle type information of a vehicle in the traffic image based on the traffic image; determining an intent to move of the vehicle based on the headlight information and the orientation information, comprising: determining the movement intention of the vehicle based on the vehicle light information, the orientation information and the vehicle type information. In this way, by combining the vehicle light information, the direction information, and the vehicle type information, it is possible to accurately obtain the steering information indicated by the steering lamp of the vehicle, that is, to predict the movement intention of the vehicle.

In some embodiments, in determining the intent-to-move of the vehicle, a confidence level of the intent-to-move of the vehicle is determined, the method further comprising: in response to the orientation information indicating that the vehicle is lateral, reducing the confidence level of the intent-to-move. In this way, when the orientation information is in the lateral direction of the vehicle, the confidence of the movement intention is reduced, and the accuracy of prediction of the movement intention of the vehicle can be improved.

In some embodiments, in determining the intent-to-move of the vehicle, a confidence level of the intent-to-move of the vehicle is determined; the method further comprises the following steps: acquiring an application demand for predicting the movement intention of the vehicle; determining a confidence threshold value matching the application requirement; after determining the intent-to-move of the vehicle, the method further comprises: and taking the movement intention with the confidence coefficient larger than the confidence coefficient threshold value as the determined movement intention of the vehicle. In this way, the confidence threshold is set according to the application requirement, so that the predicted movement intention can better meet the requirement of the user.

In some embodiments, determining the headlight information, the orientation information, and the movement intent of the vehicle is performed by a neural network; the first classifier in the neural network is obtained by utilizing the sample image which is marked with the car light information and the orientation information in a training mode, and the second classifier in the neural network is obtained by utilizing the sample image which is marked with the movement intention of the vehicle in a training mode. In this way, the accuracy of the movement intention prediction can be improved by identifying the movement intention of the vehicle using the identification network including the plurality of classifiers.

In some embodiments, the second classifier comprises at least one of: the system comprises a basic classifier and an extended classifier, wherein the basic classifier is used for classifying basic movement intentions of the vehicle, and the extended classifier is used for classifying extended movement intentions of the vehicle, and the basic classifier is trained on sample images which mark the state of the integral vehicle lamp of the vehicle; the extended classifier is trained based on sample images that are labeled with the state of the headlights of the turn lights of the vehicle. Therefore, the basic classifier and the extended classifier are mutually assisted in the training process, the state of the whole car lamp is considered in the network in the training process, the turning of the steering lamp in the whole car lamp is considered, and the movement intention of the car can be predicted more accurately.

In some embodiments, determining, with the neural network, headlight information of a vehicle and orientation information of the vehicle in the traffic image based on the traffic image comprises: determining an attention mask for the traffic image using the convolutional layer of the neural network; determining spatial features of the traffic image based on the attention mask; merging the spatial features and the time features of the traffic images to obtain image features of the traffic images; determining, with the first classifier, headlight information of the vehicle and heading information of the vehicle based on the image features. As such, by using multitask learning, a classifier that assists in classification of the state of the vehicle lights, such as a classifier of the vehicle orientation, the vehicle type, etc., may be used; and the accuracy of predicting the display state of the vehicle lamp can be improved.

In some embodiments, the determining the movement intent of the vehicle based on the headlight information and the orientation information includes: inputting the headlight information and the heading information into the second classifier, the second classifier outputting a predicted movement intention of the vehicle; in response to the predicted movement intent not matching the classification result output by the first classifier, determining a first confidence level of the predicted movement intent and a second confidence level of the classification result, respectively; and determining the movement intention of the vehicle based on the prediction result corresponding to the higher confidence coefficient in the first confidence coefficient of the prediction movement intention and the second confidence coefficient of the classification result. In this way, when the prediction results of the plurality of classifiers conflict with each other, the final prediction result is selected with a high degree of confidence, and the movement intention of the vehicle can be predicted more accurately.

An embodiment of the present application provides an exercise intention determination device, which includes: the image acquisition module is used for acquiring a traffic image; an information determination module for determining vehicle light information of a vehicle and orientation information of the vehicle in the traffic image based on the traffic image; an intent determination module to determine a movement intent of the vehicle based on the light information and the orientation information.

Correspondingly, an embodiment of the present application provides a computer storage medium, where computer-executable instructions are stored on the computer storage medium, and after being executed, the computer-executable instructions can implement the above-mentioned method steps.

An embodiment of the present application provides a computer device, where the computer device includes a memory and a processor, where the memory stores computer-executable instructions, and the processor executes the computer-executable instructions on the memory to implement the above-mentioned method steps.

The embodiment of the application provides a movement intention determining method, a movement intention determining device and a storage medium, wherein the movement intention determining method is used for determining car light information and orientation information of a vehicle in an acquired traffic image; combining the vehicle lamp information with the orientation information, and judging whether the vehicle has the movement intention of braking or steering and the like; in this way, the movement intention of the vehicle can be predicted more accurately.

Drawings

Fig. 1 is a schematic flow chart illustrating an implementation of a movement intention determining method according to an embodiment of the present application;

fig. 2 is a schematic flow chart of another implementation of the movement intention determining method provided in the embodiment of the present application;

fig. 3 is a schematic view of an application scenario of the movement intention determining method according to the embodiment of the present application;

fig. 4 is a frame diagram illustrating an implementation of a motion intention determining method according to an embodiment of the present application;

FIG. 5 is a schematic structural diagram of an exercise intention determining apparatus according to an embodiment of the present application;

fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, specific technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings in the embodiments of the present application. The following examples are intended to illustrate the present application, but are not intended to limit the scope of the present application.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.

In the following description, references to the terms "first \ second \ third" are only used to distinguish similar objects and do not denote a particular order, but rather the terms "first \ second \ third" are used to interchange specific orders or sequences, where permissible, so that the embodiments of the present application described in some embodiments may be practiced in other than the order shown or described in some embodiments.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the application.

Before further detailed description of the embodiments of the present application, terms and expressions referred to in the embodiments of the present application will be described, and the terms and expressions referred to in the embodiments of the present application will be used for the following explanation.

1) Convolutional Neural Networks (CNN): is a kind of feedforward neural network containing convolution calculation and having depth structure; the method has the characteristic learning capability, and can carry out translation invariant classification on the input information according to the hierarchical structure.

2) Autonomous vehicle (ego vehicle): a vehicle including a sensory ambient sensor. The vehicle coordinate system is fixedly connected to the autonomous vehicle, wherein the x axis is the advancing direction of the vehicle, the y axis points to the left side of the advancing direction of the vehicle, and the z axis is vertical to the ground and upward and accords with a right-hand coordinate system. The origin of the coordinate system is located on the ground below the midpoint of the rear axle.

An exemplary application of the device for determining the exercise intention provided by the embodiment of the present application is described below, and the device provided by the embodiment of the present application may be implemented as a notebook computer, a tablet computer or other vehicle-mounted device with an image capturing function, and may also be implemented as a server. In the following, an exemplary application will be explained when the device is implemented as a terminal or a server.

The method can be applied to a computer device, and the functions realized by the method can be realized by calling a program code by a processor in the computer device, although the program code can be stored in a computer storage medium, which at least comprises the processor and the storage medium.

Fig. 1 is a schematic flow chart of an implementation of the method for determining a movement intention according to the embodiment of the present application, as shown in fig. 1, and is described with reference to the steps shown in fig. 1:

and step S101, acquiring a traffic image.

In some embodiments, the traffic image may be an image of any road acquisition, may be an image including complicated picture contents, and may be an image including simple picture contents. Such as an image of a street scene taken late at night, or an image of a street scene taken during the day, etc. The traffic image may include a vehicle, wherein the vehicle includes: vehicles with various functions (such as trucks, automobiles, motorcycles, and the like) and vehicles with various wheel counts (such as four-wheel vehicles, two-wheel vehicles, and the like). The following description will not be made by taking a car as an example. For example, the traffic image is an image acquired from a car on a road.

Step S102, determining vehicle light information of the vehicle and orientation information of the vehicle in the traffic image based on the traffic image.

In some embodiments, the traffic image is input into a trained neural network, and a full convolution network in the network is adopted to perform feature extraction on the traffic image to obtain image features; by inputting the image features into a plurality of classifiers of a neural network, headlight information of each headlight of the vehicle, and orientation information of the vehicle are identified. The vehicle lamp information includes: the display status of the vehicle lights and the location of the vehicle lights on the vehicle. Taking a vehicle as an ordinary car as an example, the lamp of the ordinary car comprises: headlamps, fog lights, backup lights, license plate lights, clearance lights, running lights, turn signals, dome lights, high mounted stop lights, high/low beam lights, warning lights, trunk lights, and the like. The lamp information of the ordinary car is the display state and the position of each lamp included in the car body. The orientation information is the head orientation of the vehicle, is used for representing the vehicle orientation of the vehicle, and comprises: for the self-vehicle acquiring the images, the head of the self-vehicle faces forwards or backwards, namely, the tail of the self-vehicle faces the self-vehicle with the head facing forwards, the tail of the self-vehicle is shown in the acquired traffic images, the head of the self-vehicle faces the self-vehicle with the head facing backwards, and the head of the self-vehicle is shown in the acquired traffic images. The orientation information also includes the vehicle lateral direction, e.g. the vehicle head is towards the left side or the right side of the road, i.e. the vehicle is across the road.

Step S103, determining the movement intention of the vehicle based on the vehicle light information and the orientation information.

In some embodiments, the movement pattern indicated by the lighted headlights on the vehicle is determined by combining the headlight display state of the vehicle with the heading information of the vehicle head. I.e. the intent of the sport includes: left turn, right turn, forward, reverse, or braking, etc. The traffic image is input into a classifier for classifying the car light information in the neural network and the classifier for predicting the direction of the car head, so that classification results output by the classifiers of the plurality of car lights and classification results output by the classifiers of the direction of the car are obtained; and judging whether the output results of the classifiers conflict with each other or not, and finally outputting the vehicle lamp information and the orientation information with higher confidence coefficient. In this way, by comprehensively considering the lamp information and the orientation information, the movement intention of the vehicle can be predicted more accurately.

In the embodiment of the application, the vehicle light information and the orientation information of the vehicle are determined in the traffic image, and the vehicle light information and the orientation information are combined to judge whether the vehicle intends to perform movement such as braking or steering; thus, the movement intention of the vehicle can be analyzed more accurately.

In some embodiments, the determination of whether the turn signal is a left turn signal or a right turn signal and thus the movement intention of the vehicle is performed by analyzing the two-class orientation of the vehicle, i.e. the above step S102 can be implemented by the following steps S121 to S123 (not shown in the figure):

step S121, determining position information of the lighted target vehicle lamp of the vehicle in the traffic image based on the traffic image.

Here, a neural network is employed to detect the illuminated headlights of the vehicle in the traffic image. And if the car light is a steering lamp, determining the coordinates of the detection frame of the steering lamp, thereby obtaining the position information of the target steering lamp.

In some possible implementations, first, in the headlight information, the target headlight that is lit is determined; here, the target vehicle lamp that is lit in the display state is selected from the vehicle lamp information. The target vehicle light may be any vehicle light on the vehicle. Then, when the target vehicle light is a turn signal, position information of the turn signal is determined based on the orientation information. Here, the orientation information may include a vehicle head orientation, i.e., vehicle head forward or vehicle head backward. For example, for the same vehicle, when the vehicle head faces forward, the position of the left turn light in the image is the left side, and when the vehicle head faces backward, the position of the left turn light in the image is the right side, so that whether the target turn light in the turn light information is the left turn light or the right turn light is judged by combining the direction of the vehicle head. If the target vehicle light is not the steering light, the target vehicle light can be represented in the traffic image in a detection frame mode; the coordinates of the detection frame are the position information of the target vehicle lamp. Therefore, whether the target steering lamp is the left steering lamp or the right steering lamp can be accurately predicted by considering the orientation information of the vehicle head.

And step S122, determining appearance information of vehicles in the traffic image based on the traffic image.

In some embodiments, a single frame of traffic image is input into a neural network, and appearance information of the vehicle is extracted; wherein the appearance information includes: a picture presented by the vehicle in the traffic image; for example, the traffic image is collected at the rear of the vehicle, and the appearance information is the appearance corresponding to the tail of the vehicle (including the turn light and the high-mount stop light, etc. at the rear of the vehicle).

And S123, determining the direction information of the head of the vehicle based on the appearance information of the vehicle.

In some embodiments, by analyzing the screen content of the vehicle presented in the image, the heading, i.e., whether the vehicle heading of the vehicle is forward, backward, or sideways, can be determined.

The above steps S121 to S123 provide a way to determine the heading information of the vehicle head and the position information of the target vehicle lights, so that the heading information of the vehicle is predicted by using a single frame image, which can simplify the network model and reduce the processing delay.

After the position information of the target vehicle lights and the orientation information of the vehicle head are determined, the movement intention of the vehicle is determined by the following step S124.

And step S124, determining the movement intention of the vehicle based on the position information of the target vehicle lamp of the vehicle and the direction information of the head of the vehicle.

Here, the turning of the vehicle, that is, the target turning is predicted in conjunction with the lighted target turn lamp on the basis of the orientation information of the vehicle. For example, if the target turn signal is a left turn signal and the vehicle is oriented forward, then the target turn is a left turn forward. In this way, the position information of the target vehicle lamp is combined with the orientation information of the vehicle, and the steering of the vehicle can be predicted more accurately.

In some embodiments, in the case that the target vehicle lamp is a single turn signal lamp, the movement intention of the vehicle can be predicted by analyzing the position information of the single turn signal lamp and the orientation information of the vehicle head, that is, the step S124 can be implemented by:

the method comprises the first step of determining steering information indicated by the steering lamps on the basis of the position information of the single steering lamp and the orientation information of the head of the vehicle.

In some embodiments, the position information of the single turn signal is the position of the single turn signal in the vehicle presented on the traffic image; the turn signal indicated by the turn signal may be left turn, right turn, double flash, etc. In a specific example, when the head of the vehicle is forward in the traffic image, if the position information of a single turn signal is that the position in the image is the left side, the turn signal is indicated as a left turn signal, and then the turn signal indicated by the turn signal is determined as a left turn. Similarly, if the position information of a single turn signal lamp is that the position in the image is on the left side when the vehicle head faces backwards, the turn signal lamp is a right turn signal lamp, and then the turn signal indicated by the turn signal lamp is determined to be a right turn.

And secondly, determining the steering intention of the vehicle according to the steering information.

In some embodiments, the steering information indicated by the steering lamp may be used to derive the steering indicated by the steering lamp, which in turn may predict the next steering of the vehicle, i.e., determine the intent of the vehicle to steer.

In the embodiment of the application, the steering information indicated by the steering lamp can be accurately obtained by analyzing the position of the single steering lamp and the orientation of the vehicle head, and the steering intention of the vehicle can be accurately predicted.

In some embodiments, in the case that the target vehicle light is a plurality of steering lights, whether the vehicle is in a braking state may be analyzed by analyzing whether brake light information is included in the vehicle light information, that is, the step S124 may be implemented by:

and determining that the vehicle is in a braking state in response to that brake light information is not included in the light information and the target light is a plurality of steering lights.

In some embodiments, if the brake light information is not included in the vehicle light information, it is indicated that no brake light information is collected in the vehicle light information.

In some possible implementation manners, firstly, whether brake light information exists in the vehicle light information is judged; then, if there is no brake light information, in the lamp information, display states of left and right turn lights of the vehicle are determined. For example, a truck or a bus does not have a dome lamp, and the braking state of the truck or the bus can be judged through left and right steering lamps. In response to both the left and right turn signals being illuminated, it is determined that the vehicle is in a braking state. That is, if the target vehicle light to be turned on is a plurality of turn signals, that is, the plurality of turn signals are simultaneously turned on, it can be predicted that the vehicle is in a braking state. In this way, in the case where the brake light information is not included in the lamp information, if it is recognized that a plurality of turn lights are simultaneously in the on state, it is possible to accurately predict that the vehicle is in the braking state.

In other embodiments, if the brake light information is included in the lamp information, it can be predicted whether the vehicle is in a braking state by analyzing the on-off state of the brake light information.

In some embodiments, determining the steering of the vehicle by identifying the vehicle type of the vehicle and combining the vehicle type with the position information of the target vehicle lights and the orientation information of the vehicle can be achieved by:

the method comprises the first step of determining vehicle type information of vehicles in the traffic image based on the traffic image.

In some embodiments, a neural network is employed to identify the vehicle type of the vehicle. The image features of the traffic image are input into a vehicle type classifier, and the vehicle type of the vehicle is identified, for example, the vehicle is a car, a truck, a bus or the like.

And secondly, determining the movement intention of the vehicle based on the vehicle light information, the orientation information and the vehicle type information.

In some embodiments, the vehicle type information, the vehicle light information, and the orientation information are combined to determine whether the target vehicle light is a single turn signal, and if so, further specifically a left turn signal or a right turn signal. For example, first, the appearance and the position of the turn signal of the vehicle lamp can be determined based on the vehicle type information, so that it can be determined whether the position information of the target vehicle lamp in the vehicle lamp information is the position information of the turn signal; then, if the target vehicle lamp is determined to be a single steering lamp, the steering lamp of the vehicle can be accurately predicted to be a left steering lamp or a right steering lamp by combining the orientation information of the vehicle, and further the steering information indicated by the steering lamp of the vehicle can be accurately obtained, namely the movement intention of the vehicle is predicted.

In some embodiments, to improve the accuracy of the predicted movement intention of the vehicle, a confidence level of the movement intention of the vehicle is determined when determining the movement intention of the vehicle, and the confidence level of the movement intention is decreased in response to the orientation information indicating that the vehicle is lateral.

In some possible implementations, if the orientation classifier identifies that the vehicle orientation is lateral, to improve the accuracy of the prediction of the vehicle movement intention, the confidence of the movement intention is reduced if the orientation information is lateral. Here, since it is not easy to distinguish the left and right turning of the vehicle when the vehicle is inclined, the confidence of the predicted movement intention is lowered. Since the state of the lamp on only one side of the vehicle can be seen when the vehicle is in the lateral direction, there are many cases, and therefore, the confidence level predicted in such a case is lowered.

In some embodiments, the car light information, the orientation information of the car head and the movement intention of the vehicle in the traffic image are identified by adopting a neural network; through obtaining the trained neural network, the traffic image is input into a plurality of classifiers of the network, the display state and the head direction of each lamp of the vehicle are predicted, and the movement intention of the vehicle is further predicted.

In some embodiments, a first classifier in the neural network is trained using sample images labeled with headlight information and heading information, and a second classifier in the neural network is trained using sample images labeled with movement intent of the vehicle.

In some possible implementations, the first classifier includes at least one classifier for classifying the headlight information, the vehicle type, and the orientation information of each headlight of the sample vehicle, respectively; for example, the headlight information, the vehicle type, and the heading information of each headlight are classified based on 3 different classifiers, or the headlight information, the vehicle type, and the heading information of each headlight are classified based on the same classifier. The second classifier is used for classifying the movement intention of the sample vehicle. Inputting the acquired traffic image into a neural network, and respectively identifying the display state of each lamp of the vehicle by adopting a plurality of classifiers in the neural network so as to obtain the lamp information of the vehicle; and identifying the direction of the head of the vehicle by adopting the direction classifier in the network to obtain the direction information of the head of the vehicle.

In the process of training the neural network, a small batch (mini-batch) is used for training the second classifier, namely two parts of data with different label types are adopted for training the second classifier. The second classifier includes at least one of: the system comprises a basic classifier and an extended classifier, wherein the basic classifier is used for classifying basic movement intentions of the vehicle, and the extended classifier is used for classifying extended movement intentions of the vehicle, and the basic classifier is trained on sample images which mark the state of the integral vehicle lamp of the vehicle; the extended classifier is trained on sample images marking the state of the headlight of a steering lamp of the vehicle, the basic movement intention and the extended movement intention both represent the movement intention of the vehicle, and the confidence of the basic movement intention is lower than that of the extended movement intention. Alternatively, the base movement intention is a movement intention of the vehicle roughly predicted based on the state of the overall vehicle lights, and the extended movement intention is a movement intention of the vehicle accurately predicted based on the display state of the turn lights of the vehicle. Namely, the training sample data of the basic classifier is the data marked with the display states of all the vehicle lamps on the vehicle; training sample data of the expanded classifier is data marked with whether the left/right steering lamp is bright or dark.

In the case where the second classifier includes a basic classifier, roughly predicting a basic movement intention of the vehicle by a display state based on a whole vehicle light of the vehicle;

accurately predicting an extension movement intention of the vehicle by a display state based on a turn signal of the vehicle in a case where the second classifier includes an extension classifier;

under the condition that the second classifier comprises a basic classifier and an expanded classifier, firstly, roughly predicting a basic movement intention of the vehicle by adopting the basic classifier based on the display state of the integral vehicle lamp of the vehicle; on the basis of the basic movement intention, an extended movement intention of the vehicle is further predicted more accurately by adopting an extended classifier in combination with a display state of a turn light of the vehicle. Therefore, the basic movement intention based on the display state of the whole car light and the extended movement intention based on the display state of the left/right steering light are trained, the basic classifier and the extended classifier are mutually assisted in the training process, the state of the whole car light is considered in the training process in a network mode, the turning of the steering light in the whole car light is further considered, and the movement intention of the car can be predicted more accurately.

In some possible implementations, at least the dome light information and the turn light information of the vehicle are identified by inputting the traffic image into a first following and laying of the neural network to obtain the headlight information of the vehicle.

Here, after feature extraction is performed on the traffic image, the traffic image is input into a first classifier; and the left turn light classifier, the right turn light classifier, the top light classifier and the like in the first classifier classify the display state of the car light based on the extracted image features, so that the display state of the left turn light, the display state of the right turn light, the display state of the top light and the like are obtained. And taking the car light display state and the car light position identified by each car light classifier in the first classifier as the car light information. Therefore, by at least identifying the turn light information and the ceiling light information of the vehicle, the data amount of identification can be reduced, and abundant basis can be provided for prediction of vehicle movement intention.

In some embodiments, by using the neural network to perform feature extraction on the traffic image, and using a plurality of classifiers to predict the headlight information and the heading information of the vehicle based on the extracted image features, that is, the step S102 may be implemented by the steps shown in fig. 2, and fig. 2 is a flowchart of another implementation of the movement intention determining method provided by the embodiment of the present application, and the following description is performed in conjunction with the steps shown in fig. 2:

step S201, determining an attention mask of the traffic image by using the convolutional layer of the neural network.

In some embodiments, the traffic image is input to a full convolution network of a neural network, predicting an attention mask for the image.

Step S202, based on the attention mask, determining the spatial characteristics of the traffic image.

In some embodiments, the traffic image is multiplied element by element with the attention mask, and the product is output to the resulting spatial features for spatial feature extraction based on CNN.

And step S203, merging the spatial characteristics and the time characteristics of the traffic image to obtain the image characteristics of the traffic image.

In some embodiments, the extracted spatial features are input into a special Recurrent Neural Network (RNN), Long short-term memory (LSTM), and combined with the temporal features, and the combined features are used as image features to facilitate subsequent recognition of the vehicle lamp state and the vehicle head orientation based on the features.

Step S204, determining the car light information of the vehicle and the orientation information of the vehicle based on the image characteristics by adopting the first classifier.

In some embodiments, the image features are input to the first classifier separately, resulting in at least predicted headlight information for each headlight of the vehicle and predicted heading information for the vehicle. And respectively inputting the image characteristics into each first classifier to predict the vehicle lamp state and the vehicle head orientation. For example, the first classifier includes: the device comprises a classifier for classifying the display state of a dome lamp, a classifier for classifying the display state of a left turn lamp, a classifier for classifying the display state of a right turn lamp, a classifier for classifying the type of a vehicle and a classifier for classifying the head orientation of the vehicle. The classification result of the first classifier includes display state of each lamp of the vehicle, vehicle type, and heading information of the vehicle head. Selecting predicted vehicle light information with confidence coefficient greater than or equal to a confidence coefficient threshold value from the classification results of the vehicle light information of the same vehicle light as the vehicle light information of the vehicle; likewise, the predicted orientation information in which the confidence of the orientation information is greater than the confidence threshold in the classification result is selected as the orientation information of the vehicle. For example, the classification result about the left turn signal includes: light, dark, and none; and if the bright confidence coefficient is greater than the confidence coefficient threshold value, taking the left steering lamp in a bright state as the vehicle light information of the left steering lamp of the vehicle. Similarly, the orientation information includes: forward, rearward and transverse; and if the confidence of the orientation is greater than the confidence threshold value, taking the forward head as the orientation information of the vehicle.

In the embodiment of the present application, by using multitask learning, a classifier that assists classification of the state of the vehicular lamp, such as a classifier of the orientation of the vehicle, the type of the vehicle, or the like, may be used; and the accuracy of predicting the display state of the vehicle lamp can be improved. For example, an orientation classifier is used to help the vehicle light model determine left and right, and a vehicle type classifier is used to help the vehicle light information classifier determine vehicle light position and vehicle light shape.

In some embodiments, the better exercise intention can be selected from the predicted plurality of exercise intentions in the following two ways, i.e. the above step S103 can be implemented by:

the first method is as follows: by analyzing the user requirements, the movement intention of the classifier with higher confidence coefficient is output, namely, more accurate movement intention can be predicted through the following steps:

in a first step, application requirements for predicting the movement intention of the vehicle are obtained.

In some embodiments, the application requirements may be user-defined, such as a defined maximum number of brake misdetections or a predefined vehicle steering supervision. In one specific example, the application requirement may be that the number of false positives for a right-turning vehicle is less than 5.

And secondly, determining a confidence threshold value matched with the application requirement.

In some embodiments, a confidence threshold is set based on the application requirements. For example, if the application requirement is that the number of false positives for a right-turning vehicle is less than 5, the confidence threshold may be set to a larger value (for example, the confidence threshold is set to 0.9); if the application requirement is that the number of false positives for a right-turning vehicle is less than 20, a confidence threshold of 0.8 may be set, etc.

And thirdly, taking the movement intention with the confidence coefficient larger than the confidence coefficient threshold value as the determined movement intention of the vehicle.

In some embodiments, after determining the movement intent of the vehicle, the movement intent with a confidence level greater than the confidence level threshold is taken as the determined movement intent of the vehicle. The predicted movement intent may be to turn left, turn right, advance, reverse, or brake, etc.; of these predicted movement intentions, a predicted movement intention having a confidence greater than a confidence threshold for the application requirement matching is determined as the movement intention of the vehicle. In this way, the confidence threshold is set according to the application requirement, so that the predicted movement intention can better meet the requirement of the user.

The second method comprises the following steps: whether a plurality of obtained classification results conflict or not is analyzed, and an output result of a classifier with higher confidence coefficient is output, namely a more accurate movement intention can be predicted through the following steps:

the first step is to input the headlight information and the direction information into the second classifier, and the second classifier outputs the predicted movement intention of the vehicle.

In some embodiments, if the second classifier is a basic classifier, the basic classifier combines all the vehicle light information as a whole with the movement intention of the vehicle towards the predicted vehicle to obtain the predicted movement intention of the whole vehicle; if the second classifier is an extended classifier, the extended classifier predicts the movement intention of the vehicle based on the state of the turn signal in the vehicle light information in combination with the orientation information to obtain an extended movement intention of the vehicle.

In response to the predicted movement intention not matching the classification result output by the first classifier, a first confidence degree of the predicted movement intention and a second confidence degree of the classification result are respectively determined.

In some embodiments, the predicted movement intent does not match the classification result output by the first classifier, and the predicted movement intent conflicts with the classification result output by the first classifier. In one specific example, the second classifier is a basic classifier, the predicted movement intention is left turn, the classifier output of the left turn light in the first classification is off, the classifier output of the right turn light is on, and the classifier output of the dome light is off; therefore, the movement intention of the vehicle is right turn according to the on-off state of each part of the lamps output by the first classifier, and the output results of the plurality of classifiers conflict. In this case, the confidence of the predicted movement intention at which the collision occurs, and the second confidence of the classification result are acquired, respectively. The second confidence of the classification result may be understood as the confidence of determining the vehicle movement intention based on the classification result, or may also be understood as the confidence of the classification result as a whole.

And thirdly, determining the movement intention of the vehicle based on the prediction result corresponding to the higher confidence coefficient in the first confidence coefficient of the prediction movement intention and the second confidence coefficient of the classification result.

In some embodiments, the movement intention of the vehicle with the highest confidence level is selected from the predicted movement intention output by the second classifier and the movement intention determined by the classification result. In this way, when the prediction results of the plurality of classifiers conflict with each other, the final prediction result is selected with a high degree of confidence, and the movement intention of the vehicle can be predicted more accurately.

In the following, an exemplary application of the embodiment of the present application in an actual application scenario will be described, taking the determination of the vehicle lamp state based on the multi-task learning and the multi-stage learning as an example.

In order to determine the intention and the future Driving track of the vehicle in Advanced Driving Assistance System (ADAS) and automatic Driving tasks, the state of the lights of the vehicle needs to be detected so as to help the vehicle to perform tasks such as collision warning and decision planning. In the related art, the ADAS product is basically blank in dynamic prediction, and dynamic prediction of a vehicle lamp model for various driving scenes is also rarely performed in an automatic driving system.

In the related art, it is not accurate enough to determine which one of the winker lamps is turned to the left or to the right depending on the lighting position of the recognized lamp, as shown in fig. 3, the right winker lamp of the vehicle is turned on both of the front left winker lamp in the image 31 and the rear right winker lamp in the image 32.

With the rise of driving lamps and width indicating lamps, the driving lamps are complicated, and whether the vehicle brakes or turns cannot be judged only according to the brightness of the left whole lamp and the right whole lamp. Based on this, the embodiment of the present application provides a vehicle light state prediction method, which performs a vehicle light intention determination task by using deep learning, and can be cognitively decomposed into a vehicle intention expressed by a single light-dark non-state and a vehicle intention expressed by a whole light-on/off state, and two levels, and meanwhile, a classifier of a vehicle orientation and a vehicle type is supplemented. Therefore, the auxiliary multitask and multi-level processing is beneficial to learning of the vehicle lamp network, and the reasoning precision of the final model can be greatly improved.

In the embodiment of the present application, the left-right turn indicated by the turn signal of the vehicle is predicted by acquiring the information of the on-off position of the lamp and the advancing direction of the vehicle, which are shown on the picture. In some embodiments, for features where the positions and colors of the turn lights and the brake lights of the vehicle in some regions are different, the embodiment of the present application provides a method for predicting the state of the vehicle lights, which can be implemented by the following processes:

firstly, utilizing multi-task learning, inputting a single-frame model by using a vehicle picture to carry out end-to-end training, simultaneously outputting orientation/a vehicle tail top lamp/a left vehicle lamp/a right vehicle lamp/a vehicle type/a left and right steering lamp, and judging the instant state of the single vehicle lamp.

In some possible implementation modes, because the position and the shape of the car lamp are complex and changeable, the combination of the light and the dark is more; therefore, a classifier is separately provided for each lamp, that is, a classification of whether the left/right lamp, the left/right turn lamp, and the roof lamp are bright or dark is judged. On the basis, the additional information can assist in judging the positions and the shape types of the vehicle lamps, for example, a classifier of the vehicle direction can assist in judging the left and right vehicle lamps, and a classifier of the vehicle type can assist in judging the shapes of the vehicle lamps.

Second, with multi-level learning, the braking state and the steering intention are further judged on the basis that the model learns the state of a single lamp.

Here, the judgment of the braking state and the steering intention has a plurality of levels, and the steering intention and the braking intention expressed by the whole vehicle lamp are obtained by inference according to the states of a plurality of single vehicle lamps. And then the intention of the whole vehicle is judged by combining the left lamp state, the right lamp state and the ceiling lamp state. In addition, vehicle occlusion and motion also introduce a great deal of uncertainty in the single frame determination.

In some embodiments, the movement intent determination method may be implemented by:

in the first step, a vehicle single frame input is used for multi-task training to obtain a plurality of classifiers including orientation/vehicle type/dome light state/left light state/right light state.

In some possible implementations, the movement intention may be determined by the method shown in fig. 4, as shown in fig. 4, fig. 4 is a schematic diagram of an implementation framework of the movement intention determination method provided in the embodiment of the present application, and a single-frame image 400 of a vehicle is input into a vehicle detector 401 to identify the vehicle in the image; the detection frame of the recognized vehicle is input to the CNN402, and feature extraction is performed to obtain a feature map 403. The dimensions of the feature map 403 are 7 × 7 × 2048. The feature map 403 is processed to obtain a 2048-dimensional feature vector 404.

And secondly, training the network by using a mini-batch mode on the basis of the classifier of the first step.

Here, in one batch, half uses data labeled as left/right vehicle light entirety (where the entirety means that the entire light: any one of the sub lights (brake/fog/turn lights) is lit but unlit), and the other half uses data of left/right turn light. Training a base vehicle intent based on left and right full light states and an extended vehicle intent based on left/right turn lights.

The two groups of classifiers assist each other in training, wherein the mutual assistance means that different tasks in multi-task learning have correlation to promote each other, and the mutual promotion process is automatically completed in the model training process. In some possible implementations, the network model may input the state of the whole lights, then integrate the states of the turn lights in the whole lights, and finally output the vehicle intention of the vehicle lights. In this way, not only the final result is improved, but also the problem that training is impossible due to iteration of labeling (for example, data which are not labeled with each lamp state are labeled again) can be solved. Therefore, multi-level learning is used, the difficulty setting of the task is from shallow to difficult, the judgment of the state of a single vehicle lamp to the judgment of the whole braking state and the steering intention accords with the natural cognitive level, and model learning is facilitated; and the mini-batch mode is used for training the network, so that the training difficulty of data caused by different labels can be solved to a great extent, the same model can obtain multiple functions under different label information, and the label cost is greatly reduced.

As shown in fig. 4, the extracted feature vectors 404 are input to a plurality of fully-connected layers (fc) for classification; wherein the full connection layer 451 is used to classify on, off and non-state of the vehicle dome lamp; the full connection layer 452 is used for classifying on, off and no states of the vehicle left turn signal; the full connection layer 453 is used to classify on, off, and non-states of the right turn signal of the vehicle; the fully-connected layer 454 is used to classify vehicle heading (e.g., forward and reverse); the fully connected layer 455 is used to classify the type of vehicle (e.g., car, truck, bus, taxi, emergency vehicle, or other vehicle light); fully connected layer 456 for a vehicle oriented forward, vehicle-basedClassifying the intention of the basic vehicle according to the left and right whole light states of the vehicle; the full link layer 457 is used to classify a base vehicle intention based on a vehicle right and left full light state for a vehicle with a vehicle orientation facing rearward; that is, the fully-connected layer 456 and the fully-connected layer 457 train a basic vehicle intention based on the left and right whole light states, which is suitable for simple scenarios; the full-link layer 458 is used to classify an expanded vehicle intent based on the left and right turn signals of the vehicle for a vehicle oriented forward; the full-connection layer 459 is used for classifying the intention of the expanded vehicle based on the left and right turn signals of the vehicle for the vehicle with the vehicle facing backward; that is, the fully-connected layer 458 and the fully-connected layer 459 train the intention of extending the vehicle based on the left and right turn signal pairs of the vehicle, and are suitable for complex scenes. The outputs of the classifiers formed by the fully connected layers 451 to 459 are combined to obtain the probability distribution of the vehicle under a plurality of categories. For example, the probability of vehicle intention

Probability of vehicle turning left

Probability of vehicle turning right

And probability of vehicle orientation

In some embodiments, the above-mentioned determination of the movement intention by the manner shown in fig. 4 is only one possible implementation, and the manner in which the embodiment of the present application determines the movement intention is not limited thereto; for example, the movement intention may also be determined by a residual error network or a deep neural network; and are not described in detail herein.

And thirdly, on the basis of the first step and the second step, post-processing and logic addition can be conveniently carried out. For example, the following various post-processing and logical additions may be made:

a. judging whether the vehicle is a left lamp or a right lamp according to the dichotomy orientation (front/back) of the vehicle;

b. the truck/bus has no top light and running light, and whether braking or double flashing is performed is judged according to the brightness and darkness of the left light and the right light;

c. the taxi top lamp is generally provided with an advertisement or a taxi identification lamp, and braking is judged according to the time sequence states of the left lamp and the right lamp;

d. when the prediction results of the classifiers conflict, selecting the prediction result with the highest confidence coefficient; or the like, or, alternatively,

e. and finally outputting a prediction result with the confidence coefficient higher than a certain threshold value aiming at the requirement of less steering and brake false detection in application layer prediction. Thus, the flexible post-processing enables the applicability of the training method to be stronger, and false positive samples can be reduced by setting a confidence threshold.

And fourthly, supplementing the training process.

In some possible implementation manners, the first step and the second step can be combined into one step, and the mini-batch manner is directly used for training. If the labels of the data set are consistent, the mini-batch can be not used, and the data set is adopted to train the network training in one step to obtain a trained network model.

In the embodiment of the present application, first, an image captured by a camera mounted on the own vehicle is acquired, then, it is determined whether the direction light in the lighting of the other vehicle is turning left or turning right based on the image, and finally, based on this, it is further determined whether the direction light in the lighting of the other vehicle is turning left or turning right using the vehicle light information and the advancing direction (advancing/backing) of the other vehicle displayed in the image. In this way, when the intention of left-right turn is determined, the lamp lighting position information of the other vehicle displayed in the image is combined with the forward direction (front/back) information, and the robustness of determining left-right turn of the vehicle can be enhanced.

An exercise intention determining apparatus according to an embodiment of the present application is provided, fig. 5 is a schematic structural composition diagram of the exercise intention determining apparatus according to the embodiment of the present application, and as shown in fig. 5, the exercise intention determining apparatus 600 includes:

an image acquisition module 601, configured to acquire a traffic image;

an information determination module 602, configured to determine, based on the traffic image, headlight information of a vehicle in the traffic image and orientation information of the vehicle;

an intention determining module 603 configured to determine an intention of movement of the vehicle based on the headlight information and the orientation information.

In some embodiments, the information determining module 602 includes:

a position information determination sub-module for determining position information of an illuminated target headlight of the vehicle in the traffic image based on the traffic image;

an appearance information determination sub-module for determining appearance information of vehicles in the traffic image based on the traffic image;

the orientation information determining submodule is used for determining orientation information of the head of the vehicle based on the appearance information of the vehicle;

the intent determination module 603 is further configured to:

determining the movement intention of the vehicle based on the position information of the target vehicle lamp of the vehicle and the orientation information of the head of the vehicle.

In some embodiments, the target vehicle light is a single turn light, and the intent determination module 603 includes:

the steering information determining submodule is used for determining steering information indicated by the steering lamps on the basis of the position information of the single steering lamp and the orientation information of the head of the vehicle;

and the intention determining submodule is used for determining the steering intention of the vehicle according to the steering information.

In some embodiments, the intent determination module 603 includes:

and the braking state determining submodule is used for determining that the vehicle is in a braking state in response to the fact that the vehicle lamp information does not comprise brake lamp information and the target vehicle lamp is a plurality of steering lamps.

In some embodiments, the apparatus further comprises:

the vehicle type information determining module is used for determining vehicle type information of the vehicle in the traffic image based on the traffic image;

the intent determination module 603 is further configured to:

determining the movement intention of the vehicle based on the vehicle light information, the orientation information and the vehicle type information.

In some embodiments, in determining the intent-to-move of the vehicle, the apparatus further comprises: a confidence determination module to determine a confidence of the intent of movement of the vehicle;

a confidence adjustment module to decrease a confidence of the intent to move in response to the orientation information indicating that the vehicle is lateral.

In some embodiments, the apparatus further comprises:

the demand acquisition module is used for acquiring application demands for predicting the movement intention of the vehicle;

the confidence threshold matching module is used for determining a confidence threshold matched with the application requirement;

after determining the movement intent of the vehicle, the intent determination module 603 is further configured to:

and taking the movement intention with the confidence coefficient larger than the confidence coefficient threshold value as the determined movement intention of the vehicle.

In some embodiments, determining the headlight information, the orientation information, and the movement intent of the vehicle is performed by a neural network; the first classifier in the neural network is obtained by utilizing the sample image which is marked with the car light information and the orientation information in a training mode, and the second classifier in the neural network is obtained by utilizing the sample image which is marked with the movement intention of the vehicle in a training mode.

In some embodiments, the second classifier comprises at least one of: the system comprises a basic classifier and an extended classifier, wherein the basic classifier is used for classifying basic movement intentions of the vehicle, and the extended classifier is used for classifying extended movement intentions of the vehicle, and the basic classifier is trained on sample images which mark the state of the integral vehicle lamp of the vehicle; the extended classifier is trained based on sample images that are labeled with the state of the headlights of the turn lights of the vehicle.

In some embodiments, the information determining module 602 is further configured to determine, by using the neural network, headlight information of a vehicle in the traffic image and orientation information of the vehicle based on the traffic image; the information determining module 602 includes:

a mask determination sub-module for determining an attention mask for the traffic image using the convolutional layer of the neural network;

a spatial feature determination sub-module for determining spatial features of the traffic image based on the attention mask;

the characteristic merging submodule is used for merging the spatial characteristic and the time characteristic of the traffic image to obtain the image characteristic of the traffic image;

an information determination sub-module to determine, with the first classifier, the headlight information of the vehicle and the heading information of the vehicle based on the image feature.

In some embodiments, the intent determination module 603 includes:

an information input sub-module for inputting the headlight information and the orientation information into the second classifier, the second classifier outputting a predicted movement intention of the vehicle;

a confidence level determination sub-module, configured to determine a first confidence level of the predicted movement intention and a second confidence level of the classification result, respectively, in response to a mismatch between the predicted movement intention and the classification result output by the first classifier;

and the confidence degree comparison sub-module is used for determining the movement intention of the vehicle based on the prediction result corresponding to the higher confidence degree in the first confidence degree of the prediction movement intention and the second confidence degree of the classification result.

It should be noted that the above description of the embodiment of the apparatus, similar to the above description of the embodiment of the method, has similar beneficial effects as the embodiment of the method. For technical details not disclosed in the embodiments of the apparatus of the present application, reference is made to the description of the embodiments of the method of the present application for understanding.

It should be noted that, in the embodiment of the present application, if the movement intention determination method is implemented in the form of a software functional module and sold or used as a standalone product, it may also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for causing a computer device (which may be a terminal, a server, etc.) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a hard disk drive, a Read Only Memory (ROM), a magnetic disk, or an optical disk. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.

Correspondingly, the embodiment of the present application further provides a computer program product, where the computer program product includes computer-executable instructions, and the computer-executable instructions, when executed, can implement the steps in the movement intention determination method provided by the embodiment of the present application. Accordingly, an embodiment of the present application further provides a computer storage medium, on which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the steps of the movement intention determination method provided by the above embodiment are implemented. Accordingly, an embodiment of the present application provides a computer device, fig. 6 is a schematic structural diagram of the computer device in the embodiment of the present application, and as shown in fig. 6, the computer device 700 includes: a processor 701, at least one communication bus, a communication interface 702, at least one external communication interface, and a memory 703. Wherein communication interface 702 is configured to enable connectivity communications between these components. The communication interface 702 may include a display screen, and the external communication interface may include a standard wired interface and a wireless interface, among others. The processor 701 is configured to execute an image processing program in a memory to implement the steps of the movement intention determining method provided by the above embodiments.

The above descriptions of the embodiments of the motion-intent determination apparatus, the computer device, and the storage medium are similar to the above descriptions of the embodiments of the method, have similar technical descriptions and advantages as the corresponding embodiments of the method, and are limited by the text, so that the description of the above embodiments of the method is omitted here for brevity. For technical details not disclosed in the embodiments of the apparatus for determining an exercise intention, the computer device and the storage medium of the present application, reference is made to the description of the embodiments of the method of the present application for understanding. It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Thus, the appearances of the phrases "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in the various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described device embodiments are merely illustrative, for example, the division of the unit is only a logical functional division, and there may be other division ways in actual implementation, such as: multiple units or components may be combined, or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling, direct coupling or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between the devices or units may be electrical, mechanical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units; can be located in one place or distributed on a plurality of network units; some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment. In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be separately regarded as one unit, or two or more units may be integrated into one unit; the integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit. Those of ordinary skill in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.

Alternatively, the integrated units described above in the present application may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as independent products. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented or portions thereof contributing to the prior art may be embodied in the form of a software product stored in a storage medium, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code. The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for determining an exercise intention, the method comprising:

acquiring a traffic image;

determining, based on the traffic image, headlight information of a vehicle and orientation information of the vehicle in the traffic image;

determining an intent to move of the vehicle based on the headlight information and the orientation information.

2. The method of claim 1, wherein the determining, based on the traffic image, headlight information for a vehicle in the traffic image and heading information for the vehicle comprises:

determining, based on the traffic image, location information of illuminated target headlights of the vehicle in the traffic image;

determining appearance information of vehicles in the traffic image based on the traffic image;

determining orientation information of a head of the vehicle based on the appearance information of the vehicle;

the determining the movement intention of the vehicle based on the vehicle light information and the orientation information includes:

3. The method of claim 2, wherein the target vehicle light is a single turn light, and wherein determining the movement intent of the vehicle based on the position information of the target vehicle light of the vehicle and the orientation information of the nose of the vehicle comprises:

determining steering information indicated by a steering lamp based on the position information of the single steering lamp and the orientation information of the head of the vehicle;

determining a steering intent of the vehicle according to the steering information.

4. The method of claim 2, wherein determining the movement intent of the vehicle based on the position information of the target headlights of the vehicle and the orientation information of the nose of the vehicle comprises:

5. The method of any of claims 1 to 4, further comprising:

determining vehicle type information of a vehicle in the traffic image based on the traffic image;

6. The method of any of claims 1 to 4, wherein in determining the intent to move of the vehicle, a confidence level of the intent to move of the vehicle is determined, the method further comprising:

in response to the orientation information indicating that the vehicle is lateral, reducing the confidence level of the intent-to-move.

7. The method according to any one of claims 1 to 4, characterized in that in determining the movement intention of the vehicle, a confidence level of the movement intention of the vehicle is determined; the method further comprises the following steps:

acquiring an application demand for predicting the movement intention of the vehicle;

determining a confidence threshold value matching the application requirement;

after determining the intent-to-move of the vehicle, the method further comprises:

8. The method of claim 1, wherein determining the headlight information, the orientation information, and the movement intent of the vehicle is performed by a neural network;

the first classifier in the neural network is obtained by utilizing the sample image which is marked with the car light information and the orientation information in a training mode, and the second classifier in the neural network is obtained by utilizing the sample image which is marked with the movement intention of the vehicle in a training mode.

9. The method of claim 8, wherein the second classifier comprises at least one of: the system comprises a basic classifier and an extended classifier, wherein the basic classifier is used for classifying basic movement intentions of the vehicle, and the extended classifier is used for classifying extended movement intentions of the vehicle, and the basic classifier is trained on sample images which mark the state of the integral vehicle lamp of the vehicle; the extended classifier is trained based on sample images that are labeled with the state of the headlights of the turn lights of the vehicle.

10. The method of claim 8, wherein determining, with the neural network, headlight information for a vehicle and heading information for the vehicle in the traffic image based on the traffic image comprises:

determining an attention mask for the traffic image using the convolutional layer of the neural network;

determining spatial features of the traffic image based on the attention mask;

merging the spatial features and the time features of the traffic images to obtain image features of the traffic images;

determining, with the first classifier, headlight information of the vehicle and heading information of the vehicle based on the image features.

11. The method of claim 10, wherein the determining the intent to move of the vehicle based on the headlight information and the orientation information comprises:

inputting the headlight information and the heading information into the second classifier, the second classifier outputting a predicted movement intention of the vehicle;

in response to the predicted movement intent not matching the classification result output by the first classifier, determining a first confidence level of the predicted movement intent and a second confidence level of the classification result, respectively;

and determining the movement intention of the vehicle based on the prediction result corresponding to the higher confidence coefficient in the first confidence coefficient of the prediction movement intention and the second confidence coefficient of the classification result.

12. An exercise intent determination device, characterized in that the device comprises:

the image acquisition module is used for acquiring a traffic image;

an information determination module for determining vehicle light information of a vehicle and orientation information of the vehicle in the traffic image based on the traffic image;

an intent determination module to determine a movement intent of the vehicle based on the light information and the orientation information.

13. A computer storage medium having computer-executable instructions stored thereon that, when executed, perform the method steps of any of claims 1 to 11.

14. A computer device comprising a memory having computer-executable instructions stored thereon and a processor operable to perform the method steps of any of claims 1 to 11 when the processor executes the computer-executable instructions on the memory.