WO2023151241A1 - 一种运动意图确定方法、装置、设备及存储介质 - Google Patents

一种运动意图确定方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2023151241A1
WO2023151241A1 PCT/CN2022/108284 CN2022108284W WO2023151241A1 WO 2023151241 A1 WO2023151241 A1 WO 2023151241A1 CN 2022108284 W CN2022108284 W CN 2022108284W WO 2023151241 A1 WO2023151241 A1 WO 2023151241A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
information
intention
determining
classifier
Prior art date
Application number
PCT/CN2022/108284
Other languages
English (en)
French (fr)
Inventor
何园
蒋沁宏
Original Assignee
商汤集团有限公司
本田技研工业株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 商汤集团有限公司, 本田技研工业株式会社 filed Critical 商汤集团有限公司
Publication of WO2023151241A1 publication Critical patent/WO2023151241A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • Embodiments of the present disclosure relate to the technical field of intelligent driving, and relate to, but are not limited to, a method, device, device, and storage medium for determining a motion intention.
  • An embodiment of the present disclosure provides a technical solution for determining a motion intention.
  • An embodiment of the present disclosure provides a method for determining a motion intention, the method comprising: acquiring a traffic image; based on the traffic image, determining the vehicle light information and the orientation information of the vehicle in the traffic image; based on the The vehicle light information and the orientation information determine the movement intention of the vehicle.
  • the determining the vehicle light information and the orientation information of the vehicle in the traffic image based on the traffic image includes: determining the vehicle light information in the traffic image based on the traffic image.
  • the vehicle light information and the orientation information, and determining the vehicle's movement intention includes: determining the vehicle's movement intention based on the position information of the target vehicle light of the vehicle and the orientation information of the vehicle's front. In this way, by combining the position information of the target vehicle light with the orientation information of the vehicle, the steering of the vehicle can be predicted more accurately.
  • the target lamp is a single turn signal
  • determining the movement intention of the vehicle based on the position information of the target lamp of the vehicle and the orientation information of the head of the vehicle includes: The position information of the single turn signal and the orientation information of the head of the vehicle determine the turn information indicated by the turn signal; and determine the turn intention of the vehicle according to the turn information.
  • the steering information indicated by the turn signal can be accurately obtained, and thus the steering intention of the vehicle can be accurately predicted.
  • the determining the movement intention of the vehicle based on the position information of the target lamp of the vehicle and the orientation information of the head of the vehicle includes: responding that the vehicle lamp information does not include braking light information, and the target vehicle light is a plurality of turn lights, it is determined that the vehicle is in a braking state. In this way, by identifying whether multiple turn signals are on at the same time, it can be accurately predicted whether the vehicle is in a braking state.
  • the method further includes: based on the traffic image, determining the vehicle type information of the vehicle in the traffic image; based on the vehicle light information and the orientation information, determining the movement intention of the vehicle, The method includes: determining the movement intention of the vehicle based on the vehicle light information, orientation information and vehicle type information. In this way, by combining the vehicle light information, orientation information, and vehicle type information, the steering information indicated by the vehicle's turn signal can be accurately obtained, that is, the movement intention of the vehicle can be predicted.
  • the method when determining the vehicle's intention to move, determining a confidence level of the vehicle's intention to move, the method further comprises: in response to the orientation information indicating that the vehicle is lateral, reducing the movement Intent Confidence. In this way, when the orientation information is the lateral direction of the vehicle, the confidence level of the motion intention can be reduced, and the prediction accuracy of the vehicle motion intention can be improved.
  • the method when determining the motion intention of the vehicle, determine the confidence level of the vehicle's motion intention; the method further includes: acquiring application requirements for predicting the vehicle's motion intention; The confidence threshold of the application requirement matching; after determining the movement intention of the vehicle, the method further includes: taking the movement intention with a confidence greater than the confidence threshold as the determined movement intention of the vehicle. In this way, the confidence threshold is set according to application requirements, so that the predicted motion intention can better meet user requirements.
  • determining the vehicle light information, the orientation information and the vehicle's motion intention is performed by a neural network; the first classifier in the neural network utilizes the samples marked with the vehicle light information and orientation information Image training is obtained, and the second classifier in the neural network is trained by using sample images marked with the movement intention of the vehicle.
  • a recognition network including multiple classifiers to identify the motion intention of the vehicle can improve the accuracy of motion intention prediction.
  • the second classifier includes at least one of the following: a basic classifier for classifying the basic motion intention of the vehicle, an extended classifier for classifying the extended motion intention of the vehicle, wherein, The basic classifier is trained based on the sample images marked with the status of the vehicle's overall lights; the extended classifier is trained based on the sample images marked with the status of the lights of the vehicle's turn signals.
  • the basic classifier and the extended classifier are mutually assisted during the training process.
  • the network first considers the status of the overall headlights, and then considers the turn of the turn signal in the overall headlights, which can predict the movement of the vehicle more accurately. intention.
  • using the neural network to determine the vehicle light information and the orientation information of the vehicle in the traffic image based on the traffic image includes: using the convolutional layer of the neural network, Determine the attention mask of the traffic image; determine the spatial features of the traffic image based on the attention mask; combine the spatial features with the temporal features of the traffic image to obtain the traffic image Image features: using the first classifier, based on the image features, to determine the light information of the vehicle and the orientation information of the vehicle.
  • classifiers such as vehicle orientation and vehicle type classifiers can be used to assist vehicle light state classification; furthermore, the accuracy of predicting the display state of vehicle lights can be improved.
  • the determining the movement intention of the vehicle based on the vehicle light information and the orientation information includes: inputting the vehicle light information and the orientation information into the second classifier, The second classifier outputs a predicted motion intention of the vehicle; in response to the predicted motion intention not matching the classification result output by the first classifier, determining a first confidence level and the predicted motion intention, respectively.
  • the second confidence degree of the classification result based on the prediction result corresponding to the greater confidence among the first confidence degree of the predicted motion intention and the second confidence degree of the classification result, determine the motion intention of the vehicle. In this way, when the prediction results of multiple classifiers conflict, the one with higher confidence is selected as the final prediction result, so that the motion intention of the vehicle can be predicted more accurately.
  • An embodiment of the present disclosure provides an apparatus for determining a movement intention, the apparatus comprising: an image acquisition part configured to acquire a traffic image; an information determination part configured to determine the vehicle's position in the traffic image based on the traffic image The vehicle light information and the orientation information of the vehicle; an intention determination part configured to determine the movement intention of the vehicle based on the vehicle light information and the orientation information.
  • an embodiment of the present disclosure provides a computer storage medium, on which computer executable instructions are stored. After the computer executable instructions are executed, the above-mentioned method steps can be implemented.
  • An embodiment of the present disclosure also provides a computer program product, the computer program product includes a computer program or an instruction, and when the computer program or instruction is run on an electronic device, the electronic device performs any of the above-mentioned first aspects. Steps in one possible implementation.
  • An embodiment of the present disclosure provides a computer device.
  • the computer device includes a memory and a processor.
  • Computer-executable instructions are stored in the memory.
  • the processor runs the computer-executable instructions in the memory, it can realize the above-mentioned The method steps described above.
  • Embodiments of the present disclosure provide a method, device, device, and storage medium for determining a motion intention, by determining the vehicle light information and orientation information of a vehicle in the acquired traffic image; combining the vehicle light information and orientation information to determine the vehicle Whether it is necessary to brake or turn and other motion intentions; in this way, the motion intention of the vehicle can be predicted more accurately.
  • FIG. 1A is a schematic diagram of a system architecture that can be applied to a method for determining a motion intention according to an embodiment of the present disclosure
  • FIG. 1B is a schematic diagram of the implementation flow of the method for determining the motion intention of the embodiment of the present disclosure
  • FIG. 2 is a schematic flow diagram of another implementation of the method for determining the motion intention provided by the embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of an application scenario of a method for determining a motion intention provided by an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of an implementation framework of a method for determining a motion intention provided by an embodiment of the present disclosure
  • FIG. 5 is a schematic diagram of the structure and composition of a device for determining a motion intention according to an embodiment of the present disclosure
  • FIG. 6 is a schematic diagram of the composition and structure of a computer device according to an embodiment of the present disclosure.
  • first ⁇ second ⁇ third is only used to distinguish similar objects, and does not represent a specific order for objects. Understandably, “first ⁇ second ⁇ third” is used in Where permitted, the specific order or sequence may be interchanged such that embodiments of the disclosure described in some embodiments can be practiced in sequences other than those illustrated or described in some embodiments.
  • CNN Convolutional Neural Networks
  • Ego vehicle A vehicle that includes sensors that perceive the surrounding environment.
  • the vehicle coordinate system is fixed on the autonomous vehicle, where the x-axis is the direction in which the vehicle is moving, the y-axis points to the left side of the vehicle's direction of progress, and the z-axis is perpendicular to the ground and upwards, conforming to the right-handed coordinate system.
  • the origin of the coordinate system is located on the ground below the midpoint of the rear axle.
  • the device provided by the embodiment of the present disclosure may be implemented as a notebook computer, a tablet computer or other vehicle-mounted devices with an image collection function, and may also be implemented as a server.
  • a server an exemplary application when the device is implemented as a terminal or a server will be described.
  • FIG. 1A is a schematic diagram of a system architecture of a method for determining a motion intention provided by an embodiment of the present disclosure.
  • the system architecture includes: an image acquisition device 11 , a network 12 and a vehicle control terminal 13 .
  • the image acquisition device 11 and the vehicle control terminal 13 establish a communication connection through the network 12.
  • the image acquisition device 11 reports the acquired traffic image to the vehicle control terminal 13 through the network 12, and the vehicle control terminal 13 is responsible for traffic images.
  • the image is recognized to determine the vehicle light information and orientation information in the traffic image; and the vehicle light information and orientation information are combined to determine the vehicle's movement intention.
  • the image acquisition device 11 may include a visual processing device having visual information processing capabilities.
  • the network 12 can be wired or wirelessly connected.
  • the vehicle-mounted control terminal 13 may communicate with the visual processing device through a wired connection, such as performing data communication through a bus.
  • the image acquisition device 11 may be a vision processing device with a video capture module, or a host computer with a camera.
  • the augmented reality data presentation method of the embodiment of the present disclosure may be executed by the image acquisition device 11 , and the above-mentioned system architecture may not include the network 12 and the vehicle control terminal 13 .
  • the method can be applied to a computer device, and the functions realized by the method can be realized by calling a program code by a processor in the computer device.
  • the program code can be stored in a computer storage medium.
  • the computer device includes at least a processor and a storage device. medium.
  • FIG. 1B is a schematic diagram of the implementation process of the method for determining the motion intention of the embodiment of the present disclosure, as shown in FIG. 1B , combined with the steps shown in FIG. 1B for description:
  • Step S101 acquiring traffic images.
  • the traffic image may be an image collected from any road, and may be an image including complex picture content or an image including simple picture content.
  • the traffic image may include vehicles, wherein the vehicles include: vehicles with various functions (such as trucks, automobiles, motorcycles, etc.) and vehicles with various numbers of wheels (such as four-wheel vehicles, two-wheel vehicles, etc.). Let's take a car as an example to illustrate.
  • the traffic image is an image collected from cars on the road.
  • Step S102 based on the traffic image, determine the vehicle light information and the orientation information of the vehicle in the traffic image.
  • the traffic image is input into the trained neural network, and the full convolution network in the network is used to extract the features of the traffic image to obtain image features;
  • the car light information includes: the display state of the car light and the position of the car light on the car.
  • the lights of the ordinary car include: headlights, fog lights, reversing lights, license plate lights, position marker lights, running lights, turn signals, dome lights, high-mount brake lights, far/near Lights, warning lights and trunk lights, etc.
  • the light information of the ordinary car is the display state and position of each light included in the vehicle body.
  • the orientation information is the orientation of the vehicle's head, which is configured to characterize the orientation of the vehicle, including: relative to the self-vehicle that collects the image, the vehicle's head is facing forward or the vehicle's head is facing backward, that is, the vehicle's head is facing forward when the rear of the vehicle is facing
  • the traffic image collected in this way presents the rear of the vehicle
  • the front of the vehicle faces the self-vehicle, so the traffic image collected in this way presents the head of the vehicle.
  • the orientation information also includes the lateral direction of the vehicle, for example, the front of the vehicle is facing the left side of the road or the right side of the road, that is, the vehicle is crossing the road.
  • Step S103 based on the vehicle light information and the orientation information, determine the vehicle's movement intention.
  • the movement mode indicated by the lighted lights on the vehicle is determined by combining the display state of the lights of the vehicle with the orientation information of the head of the vehicle. That is, the motion intention includes: the vehicle turns left, turns right, moves forward, moves backward, or brakes.
  • the classifier that classifies the car light information, and the classifier that predicts the head orientation obtains the classification results output by the classifier of multiple car lights and the classification output by the classifier of the vehicle orientation. Result: judge whether the output results of these multiple classifiers are in conflict, and finally output the car light information and orientation information with higher confidence. In this way, by comprehensively considering the vehicle light information and orientation information, it is possible to more accurately predict the movement intention of the vehicle.
  • Step S121 based on the traffic image, determine the position information of the illuminated target vehicle light of the vehicle in the traffic image.
  • a neural network is used to detect the illuminated headlights of vehicles in traffic images. If the vehicle light is a turn signal, the coordinates of the detection frame of the turn signal are determined, so as to obtain the position information of the target turn signal.
  • the target vehicle light that is turned on is determined; here, in the vehicle light information, the target vehicle light that is turned on in a display state is screened out.
  • the target light can be any light on the vehicle.
  • the target vehicle light is a turn signal
  • position information of the turn signal is determined based on the orientation information.
  • the orientation information may include the orientation of the vehicle head, that is, the vehicle head is facing forward or the vehicle is facing backward. For example, for the same vehicle, when the head of the car is facing forward, the position of the left turn signal in the image is on the left side; Whether the target turn signal in the information is left turn signal or right turn signal.
  • the target vehicle light is not a turn signal, it can be represented by a detection frame in the traffic image; the coordinates of the detection frame are the position information of the target vehicle light. In this way, by considering the orientation information of the vehicle head, it is possible to accurately predict whether the target turn signal is a left turn signal or a right turn signal.
  • Step S122 based on the traffic image, determine appearance information of vehicles in the traffic image.
  • the traffic image of a single frame is input into the neural network to extract the appearance information of the vehicle; wherein, the appearance information includes: the picture presented by the vehicle in the traffic image; for example, the traffic image is collected behind the vehicle
  • the appearance information is the appearance corresponding to the rear of the vehicle (including: turn signals and high-mounted brake lights at the rear of the vehicle, etc.).
  • Step S123 based on the appearance information of the vehicle, determine the orientation information of the front of the vehicle.
  • the heading of the vehicle can be determined, that is, it can be determined whether the vehicle heading of the vehicle is forward, backward or sideways.
  • the above steps S121 to S123 provide a way to determine the orientation information of the vehicle head and the position information of the target vehicle light. In this way, using a single frame image to predict the orientation information of the vehicle can simplify the network model and reduce the processing delay.
  • the vehicle's movement intention is determined through the following step S124.
  • Step S124 based on the position information of the target lamp of the vehicle and the orientation information of the head of the vehicle, determine the movement intention of the vehicle.
  • the steering of the vehicle that is, the target steering is predicted. For example, if the target turn signal is a left turn signal, and the vehicle is facing forward, then the target turn signal is a left turn ahead. In this way, by combining the position information of the target vehicle light with the orientation information of the vehicle, the steering of the vehicle can be predicted more accurately.
  • the vehicle's movement intention can be predicted, that is, the above step S124 can be realized by the following steps :
  • the first step is to determine the steering information indicated by the steering lamp based on the position information of the single steering lamp and the orientation information of the head of the vehicle.
  • the position information of a single turn signal is the position of the single turn signal in the vehicle shown on the traffic image; the turn information indicated by the turn signal may be turning left, turning right, or double flashing, etc. .
  • the position information of a single turn signal in the traffic image, when the head of the car is facing forward, if the position information of a single turn signal is on the left side in the image, it means that the turn signal is a left turn signal, and then the turn information indicated by the turn signal is determined to turn left.
  • the turn signal is a right turn signal, and then it is determined that the turn information indicated by the turn signal is a right turn.
  • the second step is to determine the steering intention of the vehicle according to the steering information.
  • the turning indicated by the turning indicator can be obtained according to the turning information indicated by the turning indicator, and then the next turning of the vehicle can be predicted, that is, the turning intention of the vehicle can be determined.
  • the steering information indicated by the turn signal can be accurately obtained, and thus the steering intention of the vehicle can be accurately predicted.
  • step S124 when the target vehicle lights are multiple turn signals, by analyzing whether the vehicle light information includes brake light information, it can be analyzed whether the vehicle is in a braking state, that is, the above step S124 can be implemented through the following process:
  • the vehicle light information does not include brake light information and the target vehicle light is a plurality of turn lights, it is determined that the vehicle is in a braking state.
  • the vehicle light information does not include brake light information, it means that no brake light information is collected in the vehicle light information.
  • the display status of the left and right turn signals of the vehicle is determined. For example, if a truck or bus does not have a dome light, its braking status can be judged by the left and right turn signals. In response to both the left and right turn signal lights being on, it is determined that the vehicle is in a braking state. That is, if the lighted target lights are multiple turn signals, that is, the multiple turn lights are on at the same time, and then it can be predicted that the vehicle is in a braking state. In this way, if the brake light information is not included in the vehicle light information, if it is recognized that a plurality of turn signal lights are on at the same time, it can be accurately predicted that the vehicle is in a braking state.
  • the vehicle light information includes brake light information
  • the first step is to determine the vehicle type information of the vehicles in the traffic image based on the traffic image.
  • a neural network is used to identify the vehicle model. Input the image features of the traffic image into the vehicle type classifier to identify the type of the vehicle, for example, the vehicle is a car, truck or bus.
  • the second step is to determine the movement intention of the vehicle based on the vehicle light information, orientation information and vehicle type information.
  • the vehicle type information, vehicle light information and orientation information are combined to determine whether the target vehicle light is a single turn signal, and if it is a single turn signal, further specifically, whether it is a left turn signal or a right turn signal.
  • the appearance and position of the turn signal of the car light can be determined based on the vehicle model information, so it can be determined whether the position information of the target car light in the car light information is the position information of the turn signal;
  • the light is a single turn signal, combined with the orientation information of the vehicle, it can be accurately predicted whether the turn signal of the vehicle is a left turn signal or a right turn signal, and then the steering information indicated by the vehicle's turn signal can be accurately obtained, that is, the vehicle movement intention.
  • the confidence level of the motion intention is reduced.
  • the orientation classifier recognizes that the orientation of the vehicle is lateral, then in order to improve the prediction accuracy of the vehicle's motion intention, if the orientation information is the vehicle's lateral orientation, the confidence of the motion intention is reduced Spend.
  • the confidence of the predicted motion intention is reduced. Because only the state of the lights on one side of the vehicle can be seen when the vehicle is in the lateral direction, there may be many situations, so the confidence level of the prediction in this case is reduced.
  • the vehicle light information, the orientation information of the vehicle head, and the movement intention of the vehicle in the traffic image are identified by using a neural network; by obtaining a trained neural network, the traffic image is input to multiple classifications of the network In the device, the display state of each lamp of the vehicle and the direction of the head of the vehicle are predicted, and then the movement intention of the vehicle is predicted.
  • the first classifier in the neural network is trained by using sample images marked with vehicle light information and orientation information, and the second classifier in the neural network is obtained by using samples marked with the vehicle's movement intention image training.
  • the first classifier includes at least one classifier, which is used to classify the lamp information, vehicle type and orientation information of each lamp of the sample vehicle respectively; for example, the vehicle lamp information of each lamp
  • the lamp information, vehicle type and orientation information are classified based on three different classifiers, or the lamp information, vehicle type and orientation information of each lamp are classified based on the same classifier.
  • the second classifier is used to classify the movement intention of the sample vehicle.
  • the collected traffic images are input into the neural network, and the display status of each lamp of the vehicle is recognized by using multiple classifiers in the neural network, so as to obtain the lamp information of the vehicle;
  • the orientation classifier of the vehicle recognizes the orientation of the vehicle's head, and obtains the orientation information of the vehicle's head.
  • the second classifier is trained using a mini-batch, that is, the second classifier is trained by using two parts of data with different label types.
  • the second classifier includes at least one of the following: a basic classifier for classifying the basic motion intention of the vehicle, an extended classifier for classifying the extended motion intention of the vehicle, wherein the basic classifier is based on the annotation obtained by training the sample images of the vehicle’s overall headlight status; the extended classifier is trained based on the sample images of the vehicle’s turn signal lamp status; the basic motion intention and the extended motion intention are both A motoring intent of the vehicle is characterized, and the confidence of the base motoring intent is lower than the confidence of the extended motoring intent.
  • the basic motion intention is a roughly predicted motion intention of the vehicle based on the state of the overall vehicle lights
  • the extended motion intention is an accurately predicted motion intention of the vehicle based on the display state of the turn signal of the vehicle. That is, the training sample data of the basic classifier is the data marked with the display status of all lights on the vehicle; the training sample data of the extended classifier is the data marked with the left/right turn signal on or off.
  • the second classifier includes a basic classifier, roughly predicting the basic movement intention of the vehicle based on the display state of the vehicle's overall vehicle lights;
  • the second classifier includes an extended classifier, accurately predicting the extended movement intention of the vehicle based on the display state of the turn signal of the vehicle;
  • the second classifier includes a basic classifier and an extended classifier
  • the basic classifier is used to roughly predict the basic motion intention of the vehicle based on the display state of the vehicle's overall vehicle lights; on the basis of the basic motion intention , combined with the display state of the vehicle's turn signal, the extended classifier is used to further predict the extended motion intention of the vehicle more accurately.
  • the basic classifier and the extended classifier will assist each other during the training process.
  • the network first considers the state of the overall car lights, and then further considers the turning of the turn signal in the overall car lights, so that it can more accurately predict the movement intention of the vehicle.
  • the traffic image into the first foundation of the neural network, at least the dome light information and the turn signal information of the vehicle are recognized, so as to obtain the vehicle light information.
  • the first classifier After the feature extraction of the traffic image, it is input into the first classifier; the left turn signal classifier, the right turn signal classifier and the dome light classifier in the first classifier, etc., based on the extracted image features
  • the display states are classified, so as to obtain the display states of the left turn signal, the display state of the right turn signal, and the display state of the dome light.
  • the vehicle light display state and vehicle light position recognized by each vehicle light classifier in the first classifier are used as the vehicle light information. In this way, by at least identifying the turn signal information and the dome light information of the vehicle, it is possible to reduce the amount of data to be identified, and also provide a rich basis for predicting the vehicle's movement intention.
  • FIG. Figure 2 is a schematic flowchart of another implementation of the motion intention determination method provided by the embodiment of the present disclosure, and the following description is made in conjunction with the steps shown in Figure 2:
  • Step S201 using the convolutional layer of the neural network to determine the attention mask of the traffic image.
  • a traffic image is input to a fully convolutional network of neural networks that predicts an attention mask for the image.
  • Step S202 based on the attention mask, determine the spatial features of the traffic image.
  • the traffic image is multiplied element-by-element by the attention mask, and the product is output to the obtained spatial features based on spatial feature extraction in CNN.
  • Step S203 combining the spatial features with the temporal features of the traffic image to obtain image features of the traffic image.
  • the extracted spatial features are input into a special recurrent neural network (Recurrent Neural Network, RNN), long short-term memory (Long short-term memory, LSTM), merged with temporal features, and
  • RNN Recurrent Neural Network
  • Long short-term memory Long short-term memory
  • merged features are used as image features to facilitate subsequent recognition of the status of the vehicle lights and the orientation of the vehicle based on the features.
  • Step S204 using the first classifier to determine the vehicle light information and the orientation information of the vehicle based on the image features.
  • the image features are respectively input into the first classifier to obtain at least predicted vehicle light information of each vehicle light of the vehicle and predicted orientation information of the vehicle.
  • the image features are respectively input into each first classifier to predict the state of the vehicle light and the direction of the vehicle head.
  • the first classifier includes: a classifier for classifying the display state of the dome light, a classifier for classifying the display state of the left turn signal, a classifier for classifying the display state of the right turn signal, and a classifier for classifying the vehicle type and a classifier that classifies the vehicle's head orientation.
  • the classification result of the first classifier includes the display status of each lamp of the vehicle, the vehicle type and the orientation information of the vehicle head.
  • the classification results of the headlight information of the same headlight select the predicted headlight information with a confidence degree greater than or equal to the confidence threshold value as the headlight information of the vehicle; similarly, select the confidence degree of the orientation information in the classification result greater than the confidence degree threshold value
  • the predicted heading information of is used as the heading information of the vehicle.
  • the classification results about the left turn signal include: bright, dark and none; wherein, if the confidence level of bright is greater than the confidence threshold, then the left turn signal light is in a bright state as the light information of the vehicle's left turn signal light.
  • the orientation information includes: forward, backward and lateral; where, if the confidence of the orientation is greater than the confidence threshold, then the head of the vehicle is taken as the orientation information of the vehicle.
  • classifiers such as vehicle orientation and vehicle type classifiers can be used to assist vehicle light state classification; thus, the accuracy of predicting the display state of vehicle lights can be improved.
  • use the orientation classifier to help the car light model judge left and right
  • use the vehicle type classifier to help the car light information classifier to judge the position and shape of the car light.
  • step S103 can be implemented by the following steps:
  • Method 1 By analyzing user needs, output the motion intention of a classifier with a high degree of confidence, that is, a more accurate motion intention can be predicted through the following steps:
  • the first step is to obtain the application requirements for predicting the motion intention of the vehicle.
  • the application requirement can be set by the user independently, for example, the set maximum number of brake false detections or the supervision of the preset steering of the vehicle.
  • the application requirement may be that the number of false detections for right-turning vehicles is less than 5.
  • the second step is to determine a confidence threshold matching the application requirement.
  • a confidence threshold is set. For example, if the application requirement is that the number of false detections of right-turning vehicles is less than 5, you can set the confidence threshold to a larger value (for example, set the confidence threshold to 0.9); if the application requirement is false detections of right-turning vehicles If the number of times is less than 20, you can set the confidence threshold to 0.8 and so on.
  • the motion intention with a confidence degree greater than the confidence threshold is taken as the determined motion intention of the vehicle.
  • a movement intention with a confidence degree greater than the confidence threshold is taken as the determined movement intention of the vehicle.
  • the predicted motion intention may be turning left, turning right, moving forward, reversing, or braking, etc.; among these predicted motion intentions, determine the predicted motion intention whose confidence is greater than the confidence threshold of application requirement matching as the vehicle’s motion intention.
  • the confidence threshold is set according to application requirements, so that the predicted motion intention can better meet user requirements.
  • Method 2 By analyzing whether there are conflicts among the multiple classification results obtained, output the output result of a classifier with higher confidence, that is, more accurate motion intentions can be predicted through the following steps:
  • the vehicle light information and the orientation information are input into the second classifier, and the second classifier outputs the predicted motion intention of the vehicle.
  • the second classifier if the second classifier is a basic classifier, then the basic classifier takes all the lamp information as a whole, combines the vehicle orientation to predict the vehicle's motion intention, and obtains the overall predicted motion intention of the vehicle; if the second If the classifier is an extended classifier, then the extended classifier predicts the vehicle's movement intention based on the state of the turn signal in the vehicle light information, combined with the orientation information, and obtains the vehicle's extended movement intention.
  • a second step in response to the fact that the predicted motion intention does not match the classification result output by the first classifier, respectively determine a first confidence level of the predicted motion intention and a second confidence level of the classification result.
  • the predicted motion intention does not match the classification result output by the first classifier, that is, the predicted motion intention conflicts with the classification result output by the first classifier.
  • the second classifier is the basic classifier, and the predicted motion intention is to turn left.
  • the output of the classifier of the left turn signal is off
  • the output of the classifier of the right turn signal is on
  • the output of the classifier of the dome light is The output is off; in this way, according to the on-off state of each part of the lights output by the first classifier, it can be concluded that the vehicle's motion intention is to turn right, and at this time, the output results of multiple classifiers conflict.
  • the confidence level of the conflicting predicted motion intention and the second confidence level of the classification result are obtained respectively.
  • the second confidence level of the classification result can be understood as the confidence level of determining the vehicle movement intention based on the classification result, or can also be understood as the confidence level of the entire classification result in the classification result.
  • the third step is to determine the movement intention of the vehicle based on the prediction result corresponding to the higher confidence degree among the first confidence degree of the predicted movement intention and the second confidence degree of the classification result.
  • the one with the highest confidence is selected as the motion intent of the vehicle. In this way, when the prediction results of multiple classifiers conflict, the one with higher confidence is selected as the final prediction result, so that the motion intention of the vehicle can be predicted more accurately.
  • ADAS Advanced Driving Assistance System
  • ADAS products are basically blank in terms of dynamic prediction, and there are few dynamic predictions of car light models for various driving scenarios in automatic driving systems.
  • an embodiment of the present disclosure provides a method for predicting the state of a vehicle light, which utilizes deep learning to perform the task of judging the intention of a vehicle light, which can be cognitively decomposed into the vehicle intention and the overall vehicle light as expressed by the bright and dark state of a single light.
  • the auxiliary multi-task and multi-level processing is beneficial to the learning of the headlight network, and can also greatly improve the inference accuracy of the final model.
  • the left and right turns indicated by the turn signal of the vehicle are predicted by acquiring the location information of the lights on the picture and the direction of the vehicle.
  • the embodiments of the present disclosure propose a method for predicting the status of vehicle lights, which can be implemented through the following process:
  • a separate classifier is set for each light, that is, to judge left/right lights, left/right steering Stateless classification of dimming of lights and dome lights.
  • additional information can assist in determining the position and shape of vehicle lights.
  • a vehicle direction classifier can assist in identifying left and right vehicle lights
  • a vehicle type classifier can assist in determining the shape of vehicle lights.
  • the method for determining the motion intention may be implemented through the following steps:
  • the first step is to use a single frame input of the vehicle for multi-task training to obtain multiple classifiers, including orientation/vehicle type/top light status/left car light status/right car light status.
  • the motion intention can be determined in the manner shown in FIG. 4.
  • FIG. 4 input into the vehicle detector 401 to identify the vehicle in the image; input the detection frame of the identified vehicle into the CNN 402 for feature extraction to obtain a feature map 403 .
  • the dimension of the feature map 403 is 7 ⁇ 7 ⁇ 2048.
  • the feature map 403 is processed to obtain a 2048-dimensional feature vector 404 .
  • the mini-batch method is used for network training.
  • half of the left/right car lights are marked as a whole (wherein, the whole refers to the whole light: any sub-light (brake light/fog light/turn signal) is bright) data that is bright and dark , and the other half uses the left/right turn signal bright and dark data.
  • the basic vehicle intent based on the state of the left and right full lights and the extended vehicle intent based on the left/right turn signal.
  • the two groups of classifiers assist each other during training.
  • Mutual assistance means that different tasks in multi-task learning are related to promote each other, and the process of mutual promotion is automatically completed during the model training process.
  • the network model can first input the state of the whole light, and then collect the state of the turn signal in the whole light, and finally output the vehicle intention of the light. In this way, not only the final result will be improved, but also the problem of being unable to train due to the iteration of labeling (for example, re-labeling the data that has not been labeled with the status of each light) can be solved.
  • multi-level learning is used to set the difficulty of the task from shallow to difficult, from the judgment of the state of a single car light to the judgment of the overall braking state and steering intention, which conforms to the natural cognitive level and is conducive to model learning; and uses mini-
  • the batch method of training the network can largely solve the training difficulties caused by different labeling of data, so that the same model can obtain multiple functions under different labeling information, thereby greatly reducing the cost of labeling.
  • the extracted feature vector 404 is input to multiple fully connected layers (fc) for classification; wherein, the fully connected layer 451 is used to classify the bright, extinguished and stateless state of the vehicle dome light; the fully connected layer 452 is used to classify the bright, off and stateless of the left turn signal of the vehicle; the fully connected layer 453 is used to classify the light on, off and stateless of the right turn signal of the vehicle; the fully connected layer 454 is used to classify the direction of the vehicle ( For example, forward and backward) are classified; fully connected layer 455 is used to classify vehicle types (such as cars, trucks, buses, taxis, emergency vehicles or other lights); fully connected layer 456 is used for vehicle For a vehicle facing forward, classify the basic vehicle intent based on the state of the vehicle's left and right lights; the fully connected layer 457 is used to classify the basic vehicle intent based on the state of the vehicle's left and right lights for a vehicle whose orientation is backward; that is, The fully connected layer 456
  • the above-mentioned determination of motion intention through the method shown in FIG. The network and the like determine the movement intention; no more details here.
  • the third step on the basis of the first and second steps, it is convenient to perform post-processing and logic addition.
  • post-processing and logic additions are possible:
  • Trucks/buses do not have dome lights and running lights, and judge whether to brake or double flash according to the brightness of the left and right lights;
  • the fourth step is to supplement the training process.
  • the first step and the second step can be combined into one step, directly using the mini-batch method for training. If the labels of the data sets are consistent, the mini-batch may not be used, and the data set is used for one-step training of the network training to obtain the trained network model.
  • the image captured by the camera installed in the own car is obtained, and then, based on the image, it is judged whether the turn signal in the lights of the other car is turning left or right, and finally, based on this, the The car light information and the forward direction (forward/backward) of other cars, and further judge whether the direction lights in the lights of other cars turn left or right.
  • the intention to turn left and right combining the information on the lighting position of the lights of other cars displayed in the image with the information on the forward direction (front/back) can enhance the robustness of judging the turning left and right of the vehicle.
  • the writing order of each step does not mean a strict execution order and constitutes any limitation on the implementation process.
  • the specific execution order of each step should be based on its function and possible
  • the inner logic is OK.
  • the embodiment of the present disclosure also provides a sports intention determination device corresponding to the sports intention determination method. Since the problem-solving principle of the device in the embodiment of the present disclosure is similar to the above-mentioned method of determining the sports intention in the embodiment of the present disclosure, therefore The implementation of the device can refer to the implementation of the method.
  • FIG. 5 is a schematic diagram of the structure of the device for determining exercise intent according to an embodiment of the present disclosure.
  • the device for determining exercise intent 600 includes:
  • An image acquisition part 601 configured to acquire traffic images
  • the information determining part 602 is configured to determine, based on the traffic image, the vehicle light information of the vehicle in the traffic image and the orientation information of the vehicle;
  • the intention determination part 603 is configured to determine the movement intention of the vehicle based on the vehicle light information and the orientation information.
  • the information determining part 602 includes:
  • a position information determination subsection configured to determine position information of a lighted target vehicle light of the vehicle in the traffic image based on the traffic image
  • an appearance information determining subsection configured to determine appearance information of vehicles in the traffic image based on the traffic image
  • an orientation information determination subpart configured to determine orientation information of the front of the vehicle based on the appearance information of the vehicle
  • the intention determining part 603 is also used for:
  • the movement intention of the vehicle is determined.
  • the target vehicle light is a single turn signal light
  • the intention determining part 603 includes:
  • the steering information determination subpart is configured to determine the steering information indicated by the steering lamp based on the position information of the single steering lamp and the orientation information of the front of the vehicle;
  • the intention determination subpart is configured to determine the steering intention of the vehicle according to the steering information.
  • the intent determining part 603 includes:
  • the braking state determination subpart is configured to determine that the vehicle is in a braking state in response to the fact that the vehicle light information does not include brake light information and the target vehicle light is a plurality of turn lights.
  • the device also includes:
  • a vehicle type information determining part configured to determine vehicle type information of vehicles in the traffic image based on the traffic image
  • the intention determining part 603 is also used for:
  • the movement intention of the vehicle is determined.
  • the device when determining the movement intention of the vehicle, the device further includes: a confidence determination part configured to determine the confidence of the vehicle's movement intention;
  • a confidence adjustment section configured to reduce the confidence of the movement intention in response to the orientation information indicating that the vehicle is lateral.
  • the device also includes:
  • a requirement acquiring part configured to acquire an application requirement for predicting the movement intention of the vehicle
  • a confidence threshold matching part configured to determine a confidence threshold matching the application requirement
  • the intention determining part 603 is further configured to:
  • determining the vehicle light information, the orientation information and the vehicle's motion intention is performed by a neural network; the first classifier in the neural network utilizes the samples marked with the vehicle light information and orientation information Image training is obtained, and the second classifier in the neural network is trained by using sample images marked with the movement intention of the vehicle.
  • the second classifier includes at least one of the following: a basic classifier for classifying the basic motion intention of the vehicle, an extended classifier for classifying the extended motion intention of the vehicle, wherein, The basic classifier is trained based on the sample images marked with the status of the vehicle's overall lights; the extended classifier is trained based on the sample images marked with the status of the lights of the vehicle's turn signals.
  • the information determining part 602 is further configured to use the neural network to determine the vehicle light information and the orientation information of the vehicle in the traffic image based on the traffic image; the The information determination part 602 includes:
  • a mask determination subsection configured to determine an attention mask of the traffic image using a convolutional layer of the neural network
  • a spatial feature determination subsection configured to determine spatial features of the traffic image based on the attention mask
  • a feature merging subsection configured to combine the spatial features with the temporal features of the traffic image to obtain image features of the traffic image
  • the information determination subpart is configured to determine the vehicle light information of the vehicle and the orientation information of the vehicle based on the image features by using the first classifier.
  • the intent determining part 603 includes:
  • an information input subsection configured to input the vehicle light information and the orientation information into the second classifier, and the second classifier outputs the predicted motion intention of the vehicle;
  • a confidence determination subpart configured to determine a first confidence level of the predicted motion intention and a second confidence level of the classification result, respectively, in response to a mismatch between the predicted motion intention and the classification result output by the first classifier.
  • the confidence comparison subpart is configured to determine the vehicle's movement intention based on the prediction result corresponding to the greater confidence among the first confidence level of the predicted movement intention and the second confidence level of the classification result.
  • a “module” may be a circuit, a processor, a program, or software, etc., of course, may also be a unit, and may also be non-modular.
  • the above method for determining the exercise intention is implemented in the form of software function modules and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
  • the essence of the technical solutions of the embodiments of the present disclosure or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for A computer device (which may be a terminal, a server, etc.) is made to execute all or part of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage media include: various media that can store program codes such as U disk, sports hard disk, read-only memory (Read Only Memory, ROM), magnetic disk or optical disk.
  • embodiments of the present disclosure are not limited to any specific combination of hardware and software.
  • an embodiment of the present disclosure further provides a computer program product, the computer program product includes computer-executable instructions, and after the computer-executable instructions are executed, the steps in the method for determining the exercise intention provided by the embodiments of the present disclosure can be realized .
  • an embodiment of the present disclosure further provides a computer storage medium, on which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, the method for determining the motion intention provided by the above-mentioned embodiments is implemented.
  • an embodiment of the present disclosure provides a computer device.
  • FIG. 6 is a schematic diagram of the composition and structure of a computer device in an embodiment of the present disclosure. As shown in FIG.
  • the computer device 700 includes: a processor 701, at least one communication bus, A communication interface 702, at least one external communication interface and a memory 703.
  • the communication interface 702 is configured to realize connection and communication between these components.
  • the communication interface 702 may include a display screen, and the external communication interface may include a standard wired interface and a wireless interface.
  • the processor 701 is configured to execute the image processing program in the memory, so as to realize the steps of the method for determining the motion intention provided in the above-mentioned embodiments.
  • the disclosed devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division.
  • the coupling, or direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms of.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may be used as a single unit, or two or more units may be integrated into one unit; the above-mentioned integration
  • the unit can be realized in the form of hardware or in the form of hardware plus software functional unit.
  • the above-mentioned integrated units of the present disclosure are realized in the form of software function modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.
  • a software product which is stored in a storage medium and includes several instructions for Make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage medium includes various media capable of storing program codes such as removable storage devices, ROMs, magnetic disks or optical disks.
  • An embodiment of the present disclosure provides a method, device, device, and storage medium for determining a motion intention, wherein a traffic image is acquired; based on the traffic image, the vehicle light information and the orientation information of the vehicle in the traffic image are determined ; Based on the vehicle light information and the orientation information, determine the movement intention of the vehicle.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Resources & Organizations (AREA)
  • Economics (AREA)
  • Software Systems (AREA)
  • Game Theory and Decision Science (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Biology (AREA)
  • Biomedical Technology (AREA)
  • Development Economics (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

本公开实施例提供一种运动意图确定方法、装置、设备及存储介质，其中，获取交通图像；基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息；基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图。如此，能够更加准确地预测出车辆的运动意图。

Description

一种运动意图确定方法、装置、设备及存储介质
相关申请的交叉引用
本公开基于申请号为202210122826.X、申请日为2022年2月9日、申请名称为“一种运动意图确定方法、装置、设备及存储介质”的中国专利申请提出，并要求该中国专利申请的优先权，该中国专利申请的全部内容在此引入本公开作为参考。
技术领域
本公开实施例涉及智能驾驶技术领域，涉及但不限于一种运动意图确定方法、装置、设备及存储介质。
背景技术
近些年来，随着行车灯和示宽灯的兴起，车灯呈现着复杂化的态势，仅根据左右两个整灯的亮暗情况无法准确判断车辆是否在刹车或者转向。
发明内容
本公开实施例提供一种运动意图确定技术方案。
本公开实施例的技术方案是这样实现的：
本公开实施例提供一种运动意图确定方法，所述方法包括：获取交通图像；基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息；基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图。
在一些实施例中，所述基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息，包括：基于所述交通图像，确定所述交通图像中所述车辆的点亮的目标车灯的位置信息；基于所述交通图像，确定所述交通图像中的车辆的外观信息；基于所述车辆的外观信息，确定所述车辆的车头的朝向信息；基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图，包括：基于所述车辆的目标车灯的位置信息和所述车辆的车头的朝向信息，确定所述车辆的运动意图。如此，将目标车灯的位置信息与车辆的朝向信息相结合，能够更加准确地预测出车辆的转向。
在一些实施例中，所述目标车灯为单个转向灯，所述基于所述车辆的目标车灯的位置信息和所述车辆的车头的朝向信息，确定所述车辆的运动意图，包括：基于所述单个转向灯的位置信息和所述车辆的车头的朝向信息，确定转向灯指示的转向信息；根据所述转向信息，确定所述车辆的转向意图。如此，通过分析单个转向灯的位置和车头的朝向能够准确得到该转向灯所指示的转向信息，进而能够精准预测车辆的转向意图。
在一些实施例中，所述基于所述车辆的目标车灯的位置信息和所述车辆的车头的朝向信息，确定所述车辆的运动意图，包括：响应于所述车灯信息中未包括刹车灯信息、且所述目标车灯为多个转向灯，确定所述车辆处于制动状态。如此，通过识别多个转向灯是否同时处于点亮状态，可准确预测该车辆是否处于制动状态。
在一些实施例中，所述方法还包括：基于所述交通图像，确定所述交通图像中的车辆的车型信息；基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图，包括：基于所述车灯信息、朝向信息和车型信息，确定所述车辆的运动意图。如此，将车灯信息、朝向信息以及车型信息相结合，能够准确得到车辆的转向灯所指示的转向信息，即预测到该车辆的运动意图。
在一些实施例中，在确定所述车辆的运动意图时，确定所述车辆的运动意图的置信度，所述方法还包括：响应于所述朝向信息指示所述车辆为横向，降低所述运动意图的置信度。如此，在所述朝向信息为车辆横向的情况下，降低所述运动意图的置信度，能够提高对车辆运动意图的预测准确度。
在一些实施例中，在确定所述车辆的运动意图时，确定所述车辆的运动意图的置信度；所述方法还包括：获取对所述车辆的运动意图进行预测的应用需求；确定与所述应用需求匹配的置信度阈值；在确定所述车辆的运动意图之后，所述方法还包括：将置信度大于所述置信度阈值的运动意图作为所述车辆的确定运动意图。如此，按照应用需求设定置信度阈值，从而能够使得预测的运动意图更好地满足用户需求。
在一些实施例中，确定所述车灯信息、所述朝向信息和所述车辆的运动意图由神经网络执行；所述神经网络中的第一分类器利用标注了车灯信息和朝向信息的样本图像训练得到，所述神经网络中的第二分类器利用标注了车辆的运动意图的样本图像训练得到。如此，采用包括多个分类器的识别网络对车辆的运动意图进行识别，能够提高运动意图 预测的准确度。
在一些实施例中，所述第二分类器包括以下至少之一：用于对车辆的基础运动意图进行分类的基础分类器、用于对车辆的扩展运动意图进行分类的扩展分类器，其中，所述基础分类器是基于标注了车辆的整体车灯状态的样本图像训练得到的；所述扩展分类器是基于标注了车辆的转向灯的车灯状态的样本图像训练得到的。如此，将基础分类器和扩展分类器在训练过程中互相辅助，在训练的过程中网络先考虑整体车灯状态，再考虑整体车灯中转向灯的转头，能够更加准确地预测车辆的运动意图。
在一些实施例中，利用所述神经网络，基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息，包括：利用所述神经网络的卷积层，确定所述交通图像的注意力掩模；基于所述注意力掩模，确定所述交通图像的空间特征；将所述空间特征与所述交通图像的时间特征进行合并，得到所述交通图像的图像特征；采用所述第一分类器，基于所述图像特征，确定所述车辆的车灯信息和所述车辆的朝向信息。如此，通过使用多任务学习，可以使用车辆朝向、车辆类型的分类器等辅助车灯状态分类的分类器；进而能够提高对车灯的显示状态进行预测的准确度。
在一些实施例中，所述基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图，包括：将所述车灯信息和所述朝向信息，输入所述第二分类器，所述第二分类器输出所述车辆的预测运动意图；响应于所述预测运动意图与所述第一分类器输出的分类结果不匹配，分别确定所述预测运动意图的第一置信度和所述分类结果的第二置信度；基于所述预测运动意图的第一置信度和所述分类结果的第二置信度中较大置信度对应的预测结果，确定所述车辆的运动意图。如此，在多个分类器的预测结果产生冲突时，选择置信度较大的作为最终的预测结果，能够更加准确地预测出车辆的运动意图。
本公开实施例提供一种运动意图确定装置，所述装置包括：图像获取部分，被配置为获取交通图像；信息确定部分，被配置为基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息；意图确定部分，被配置为基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图。
对应地，本公开实施例提供一种计算机存储介质，所述计算机存储介质上存储有计算机可执行指令，该计算机可执行指令被执行后，能够实现上述所述的方法步骤。
本公开实施例还提供一种计算机程序产品，所述计算机程序产品包括计算机程序或指令，在所述计算机程序或指令在电子设备上运行的情况下，使得所述电子设备执行上述第一方面任一种可能的实施方式中的步骤。
本公开实施例提供一种计算机设备，所述计算机设备包括存储器和处理器，所述存储器上存储有计算机可执行指令，所述处理器运行所述存储器上的计算机可执行指令时可实现上述所述的方法步骤。
本公开实施例提供一种运动意图确定方法、装置、设备及存储介质，通过在获取的交通图像中，确定车辆的车灯信息和朝向信息；将车灯信息与朝向信息相结合，判断该车辆是否要制动或者转向等运动意图；如此，能够更加准确地预测出车辆的运动意图。
为使本公开实施例的上述目的、特征和优点能更明显易懂，下文特举较佳实施例，并配合所附附图，作详细说明如下。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，此处的附图被并入说明书中并构成本说明书中的一部分，这些附图示出了符合本公开的实施例，并与说明书一起用于说明本公开的技术方案。应当理解，以下附图仅示出了本公开的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1A为可以应用本公开实施例的运动意图确定方法的一种系统架构示意图；
图1B为本公开实施例运动意图确定方法的实现流程示意图；
图2为本公开实施例提供的运动意图确定方法的另一实现流程示意图；
图3为本公开实施例提供的运动意图确定方法的应用场景示意图；
图4为本公开实施例提供的运动意图确定方法的实现框架示意图；
图5为本公开实施例运动意图确定装置结构组成示意图；
图6为本公开实施例计算机设备的组成结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对发明的具体技术方案做进一步详细描述。以下实施例用于说明本公开，但不用来限制本公开的范围。
在以下的描述中，涉及到“一些实施例”，其描述了所有可能实施例的子集，但是可以理解，“一些实施例”可以是所有可能实施例的相同子集或不同子集，并且可以在不冲突的情况下相互结合。
在以下的描述中，所涉及的术语“第一\第二\第三”仅仅是区别类似的对象，不代表针对对象的特定排序，可以理解地，“第一\第二\第三”在允许的情况下可以互换特定的顺序或先后次序，以使在一些实施例中描述的本公开实施例能够以除了在一些实施例中图示或描述的以外的顺序实施。
除非另有定义，本文所使用的所有的技术和科学术语与属于本公开的技术领域的技术人员通常理解的含义相同。本文中所使用的术语只是为了描述本公开实施例的目的，不是旨在限制本公开。
对本公开实施例进行进一步详细说明之前，对本公开实施例中涉及的名词和术语进行说明，本公开实施例中涉及的名词和术语适用于如下的解释。
1)卷积神经网络(Convolutional Neural Networks，CNN)：是一类包含卷积计算且具有深度结构的前馈神经网络；具有表征学习能力，能够按其阶层结构对输入信息进行平移不变分类。
2)自主车辆(ego vehicle)：包含感知周围环境传感器的车辆。车辆坐标系固连在自主车辆上，其中，x轴为汽车前进的方向，y轴指向车辆前进方向的左侧，z轴垂直于地面向上，符合右手坐标系。坐标系原点位于后轴中点下方的大地上。
下面说明本公开实施例提供的运动意图确定的设备的示例性应用，本公开实施例提供的设备可以实施为具有图像采集功能的笔记本电脑，平板电脑或其他车载设备，也可以实施为服务器。下面，将说明设备实施为终端或服务器时示例性应用。
图1A为本公开实施例提供的运动意图确定方法的一种系统架构示意图，如图1A所示，该系统架构中包括：图像获取设备11、网络12和车载控制终端13。为实现支撑一个示例性应用，图像获取设备11和车载控制终端13通过网络12建立通信连接，首先，图像获取设备11通过网络12向车载控制终端13上报获取的交通图像，车载控制终端13对交通图像进行识别，以确定交通图像中车辆的车灯信息和朝向信息；并将车灯信息和朝向信息相结合，来确定车辆的运动意图。
作为示例，图像获取设备11可以包括具有视觉信息处理能力的视觉处理设备。网络12可以采用有线或无线连接方式。其中，当图像获取设备11为视觉处理设备时，车载控制终端13可以通过有线连接的方式与视觉处理设备通信连接，例如通过总线进行数据通信。
或者，在一些场景中，图像获取设备11可以是带有视频采集模组的视觉处理设备，可以是带有摄像头的主机。这时，本公开实施例的增强现实数据展示方法可以由图像获取设备11执行，上述系统架构可以不包含网络12和车载控制终端13。
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对发明的具体技术方案做进一步详细描述。以下实施例配置为说明本公开，但不用来限制本公开的范围。
在以下的描述中，涉及到“一些实施例”，其描述了所有可能实施例的子集，但是可以理解，“一些实施例”可以是所有可能实施例的相同子集或不同子集，并且可以在不冲突的情况下相互结合。
该方法可以应用于计算机设备，该方法所实现的功能可以通过计算机设备中的处理器调用程序代码来实现，当然程序代码可以保存在计算机存储介质中，可见，该计算机设备至少包括处理器和存储介质。
图1B为本公开实施例运动意图确定方法的实现流程示意图，如图1B所示，结合如图1B所示步骤进行说明：
步骤S101，获取交通图像。
在一些实施例中，交通图像可以是任意道路采集的图像，可以是包括画面内容复杂的图像还可以是包括画面内容简单的图像。比如，在深夜采集的街道场景的图像，或者在白天采集的街道场景的图像等。该交通图像中可以包括车辆，其中，车辆包括：各种各样功能的车辆(如卡车、汽车、摩托车等)和各种轮数的车辆(如四轮车辆、两轮车辆等)等。下面不妨以轿车为例进行说明。比如，交通图像为在道路上，对轿车进行采集的图像。
步骤S102，基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息。
在一些实施例中，将交通图像输入训练好的神经网络中，采用该网络中的全卷积网络对交通图像进行特征提取，得到图像特征；通过将该图像特征输入到神经网络的多个分类器中，识别该车辆的每一车灯的车灯信息，以及该车辆 的朝向信息。车灯信息包括：车灯的显示状态和车灯在车上的位置。以车辆为普通轿车为例，该普通轿车的车灯包括：前照灯、雾灯、倒车灯、牌照灯、示廓灯、行车灯、转向灯、顶灯、高位刹车灯、远/近光灯、警示灯和行李箱灯等等。该普通轿车的车灯信息为车身包括的每一车灯的显示状态和位置。朝向信息为该车辆的车头朝向，被配置为表征该车辆的车辆朝向，包括：相对于采集图像的自车来说，车头朝前或车头朝后，即，车头朝前为车辆的尾部面对自车，这样采集到的交通图像中呈现的是车辆尾部，车头朝后为车辆的车头面对自车，这样采集到的交通图像中呈现的是车辆头部。朝向信息还包括车辆横向，比如，车辆的车头朝向道路左侧或者道路右侧，即车辆横在道路上。
步骤S103，基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图。
在一些实施例中，通过将车辆的车灯显示状态与车头的朝向信息结合，判断该车辆上点亮的车灯所指示的运动方式。即该运动意图包括：车辆左转、右转、前进、后退或者刹车等。通过将交通图像输入到神经网络中对车灯信息进行分类的分类器，以及对车头朝向进行预测的分类器，得到多个车灯的分类器输出的分类结果和车辆朝向的分类器输出的分类结果；判断这多个分类器的输出结果之间是否相冲突，最后输出置信度较大的车灯信息和朝向信息。如此，通过综合考虑车灯信息和朝向信息，能够更加准确地预测车辆的运动意图。
在本公开实施例中，通过在交通图像中，确定车辆的车灯信息和朝向信息，将车灯信息与朝向信息相结合，判断该车辆是否要进行刹车或者转向等运动意图；如此，能够更加准确地分析出车辆的运动意图。
在一些实施例中，通过分析车辆的二分类朝向来确定转向灯是左转向灯还是右转向灯，进而确定车辆的运动意图，即上述步骤S102可以通过以下步骤S121至S123(图示未示出)实现：
步骤S121，基于所述交通图像，确定所述交通图像中所述车辆的点亮的目标车灯的位置信息。
这里，采用神经网络，检测交通图像中车辆的点亮的车灯。如果该车灯为转向灯，并确定该转向灯的检测框的坐标，从而得到目标转向灯的位置信息。
在一些可能的实现方式中，首先，在所述车灯信息中，确定点亮的所述目标车灯；这里，在车灯信息中，筛选出显示状态点亮的目标车灯。该目标车灯可以是车辆上的任意车灯。然后，在所述目标车灯为转向灯的情况下，基于所述朝向信息，确定所述转向灯的位置信息。这里，该朝向信息可以包括车头朝向，即车头朝前或者车头朝后。比如，对于同一车辆，车头朝前时，左转向灯在图像中的位置为左侧，车头朝后时，左转向灯在图像中的位置为右侧，这样，结合车头朝向，来判断转向灯信息中目标转向灯是左转向灯还是右转向灯。如果目标车灯不是转向灯，那么可以通过在交通图像中对该目标车灯以检测框的方式表示出来；该检测框的坐标即为目标车灯的位置信息。如此，通过考虑车头的朝向信息，能够准确预测出目标转向灯是左转向灯还是右转向灯。
步骤S122，基于所述交通图像，确定所述交通图像中的车辆的外观信息。
在一些实施例中，将单帧的交通图像输入到神经网络中，提取该车辆的外观信息；其中，外观信息包括：车辆在交通图像中呈现的画面；比如，交通图像是在车辆的后面采集到的，那么该外观信息为车辆尾部所对应的外观(包括：车辆后方的转向灯和高位刹车灯等)。
步骤S123，基于所述车辆的外观信息，确定所述车辆的车头的朝向信息。
在一些实施例中，通过对图像中呈现的车辆的画面内容进行分析，能够确定车头朝向，即确定出该车辆的车辆朝向是前进、后退还是横向行驶。
上述步骤S121至S123提供了一种确定车头朝向信息和目标车灯的位置信息的方式，这样采用单帧图像预测车辆的朝向信息，能够简化网络模型，降低处理延时。
在确定目标车灯的位置信息和车头的朝向信息之后，通过以下步骤S124确定车辆的运动意图。
步骤S124，基于所述车辆的目标车灯的位置信息和所述车辆的车头的朝向信息，确定所述车辆的运动意图。
这里，在车辆的朝向信息的基础上，结合点亮的目标转向灯，预测该车辆的转向，即目标转向。比如，目标转向灯为左转向灯，且车辆朝向为朝前，那么目标转向为前方的左转。如此，将目标车灯的位置信息与车辆的朝向信息相结合，能够更加准确地预测出车辆的转向。
在一些实施例中，在目标车灯为单个转向灯的情况下，通过分析该单个转向灯的位置信息和车头的朝向信息，即可预测车辆的运动意图，即上述步骤S124可以通过以下步骤实现：
第一步，基于所述单个转向灯的位置信息和所述车辆的车头的朝向信息，确定转向灯指示的转向信息。
在一些实施例中，单个转向灯的位置信息即为该单个转向灯在交通图像上所呈现的车辆中的位置；转向灯指示的转向信息可能是向左转、向右转或者是双闪等。在一个具体例子中，在交通图像中，车头朝前时，如果单个转向灯的 位置信息为在图像中的位置为左侧，说明该转向灯为左转向灯，进而确定转向灯指示的转向信息为左转。同样在车头朝后的情况下，如果单个转向灯的位置信息为在图像中的位置为左侧，说明该转向灯为右转向灯，进而确定转向灯指示的转向信息为右转。
第二步，根据所述转向信息，确定所述车辆的转向意图。
在一些实施例中，按照转向灯指示的转向信息可以得到该转向灯所指示的转向，进而可以预测车辆接下来的转向，即确定车辆的转向意图。
在本公开实施例中，通过分析单个转向灯的位置和车头的朝向能够准确得到该转向灯所指示的转向信息，进而能够精准预测车辆的转向意图。
在一些实施例中，在目标车灯为多个转向灯的情况下，通过分析车灯信息中是否包括刹车灯信息，可以分析车辆是否处于制动状态，即上述步骤S124可以通过以下过程实现：
响应于所述车灯信息中未包括刹车灯信息、且所述目标车灯为多个转向灯，确定所述车辆处于制动状态。
在一些实施例中，如果车灯信息中未包括刹车灯信息，说明在该车灯信息中未采集到刹车灯信息。
在一些可能的实现方式中，首先，在车灯信息中判断是否有刹车灯信息；然后，如果没有刹车灯信息，在所述车灯信息中，确定所述车辆的左右转向灯的显示状态。比如，卡车或者公交车没有顶灯，可以通过左右转向灯判断其刹车状态。响应于左右转向灯均点亮，确定车辆处于制动状态。即如果点亮的目标车灯为多个转向灯，即多个转向灯同时处于点亮状态，进而可预测出该车辆处于制动状态。如此，在车灯信息中未包括刹车灯信息的情况下，如果识别到多个转向灯同时处于点亮状态，那么可准确预测该车辆处于制动状态。
在其他实施例中，如果车灯信息中包括刹车灯信息，那么通过分析刹车灯信息的点灭状态，即可预测车辆是否处于制动状态。
在一些实施例中，通过识别该车辆的车型，将车型与目标车灯的位置信息以及车辆的朝向信息相结合，来判断车辆的转向，可以通过以下步骤实现：
第一步，基于所述交通图像，确定所述交通图像中的车辆的车型信息。
在一些实施例中，采用神经网络识别该车辆的车型。将交通图像的图像特征输入到车型分类器中，识别该车辆的车型，比如，该车辆为轿车、卡车或公交车等。
第二步，基于所述车灯信息、朝向信息和车型信息，确定所述车辆的运动意图。
在一些实施例中，将车型信息、车灯信息与朝向信息相结合，确定目标车灯是否为单个转向灯，如果是单个转向灯的情况，进一步具体是左转向灯还是右转向灯。比如，首先，基于车型信息能够确定该车灯的转向灯的外观以及位置，所以能够确定车灯信息中的目标车灯的位置信息是否为转向灯的位置信息；然后，如果确定出该目标车灯为单个转向灯，结合车辆的朝向信息，能够准确的预测到该车辆的转向灯为左转向灯还是右转向灯，进而能够准确得到车辆的转向灯所指示的转向信息，即预测到该车辆的运动意图。
在一些实施例中，为提高预测的车辆的运动意图的准确度，在确定所述车辆的运动意图时，确定所述车辆的运动意图的置信度，响应于所述朝向信息指示所述车辆为横向，降低所述运动意图的置信度。
在一些可能的实现方式中，如果朝向分类器识别到车辆朝向为横向，那么为提高对车辆运动意图的预测准确度，在所述朝向信息为车辆横向的情况下，降低所述运动意图的置信度。这里，由于车辆倾斜时，不易区分车辆左右转，所以预测的运动意图的降低置信度。因为车辆横向时只能看到车辆一侧的车灯状态，可能出现多种情况，所以降低这种情况下预测到的置信度。
在一些实施例中，通过采用神经网络对交通图像中的车灯信息、车头的朝向信息以及车辆的运动意图进行识别；通过获取已训练的神经网络，将交通图像输入到该网络的多个分类器中，对车辆的每一车灯的显示状态以及车头朝向进行预测，进而预测车辆的运动意图。
在一些实施例中，所述神经网络中的第一分类器利用标注了车灯信息和朝向信息的样本图像训练得到，所述神经网络中的第二分类器利用标注了车辆的运动意图的样本图像训练得到。
在一些可能的实现方式中，第一分类器包括至少一个分类器，分别用于对样本车辆的每一车灯的车灯信息、车辆类型和朝向信息进行分类；比如，每一车灯的车灯信息、车辆类型和朝向信息分别是基于3个不同的分类器进行分类得到的，或者每一车灯的车灯信息、车辆类型和朝向信息是基于同一个分类器进行分类得到的。所述第二分类器用于对所述样本车辆的运动意图进行分类。将采集到的交通图像输入到神经网络中，通过采用该神经网络中的多个分类器分 别对车辆的每一车灯的显示状态进行识别，从而得到该车辆的车灯信息；采用该网络中的朝向分类器对该车辆的车头朝向进行识别，得到车辆的车头的朝向信息。
在训练神经网络的过程中，使用小批量(mini-batch)对第二分类器进行训练，即采用两部分不同标注类型的数据对第二分类器进行训练。第二分类器包括以下至少之一：用于对车辆的基础运动意图进行分类的基础分类器、用于对车辆的扩展运动意图进行分类的扩展分类器，其中，所述基础分类器是基于标注了车辆的整体车灯状态的样本图像训练得到的；所述扩展分类器是基于标注了车辆的转向灯的车灯状态的样本图像训练得到的，所述基础运动意图和所述扩展运动意图均表征所述车辆的运动意图，且所述基础运动意图的置信度低于所述扩展运动意图的置信度。或者是，该基础运动意图为基于整体车灯状态粗略预测的车辆的运动意图，该扩展运动意图为基于车辆的转向灯的显示状态精确预测的车辆的运动意图。即，基础分类器的训练样本数据为标注了车辆上所有车灯的显示状态的数据；扩展分类器的训练样本数据为标注了左/右转向灯亮暗无的数据。
在第二分类器包括基础分类器的情况下，通过基于车辆的整体车灯的显示状态，粗略预测该车辆的基础运动意图；
在第二分类器包括扩展分类器的情况下，通过基于车辆的转向灯的显示状态，精确预测该车辆的扩展运动意图；
在第二分类器包括基础分类器和扩展分类器的情况下，首先，采用基础分类器基于车辆的整体车灯的显示状态，粗略预测该车辆的基础运动意图；在该基础运动意图的基础上，结合车辆的转向灯的显示状态，采用扩展分类器进一步更加精确地预测车辆的扩展运动意图。这样，训练基于整体车灯的显示状态的基础运动意图，和基于左/右转向灯的显示状态的扩展运动意图，将基础分类器和扩展分类器在训练过程中互相辅助，在训练的过程中网络先考虑整体车灯状态，再进一步考虑整体车灯中转向灯的转头，能够更加准确地预测车辆的运动意图。
在一些可能的实现方式中，通过将交通图像输入到神经网络的第一跟垒砌中，至少识别所述车辆的顶灯信息和转向灯信息，以得到车辆的车灯信息。
这里，将交通图像进行特征提取之后，输入到第一分类器中；第一分类器中的左转向灯分类器、右转向灯分类器和顶灯分类器等，基于提取的图像特征对车灯的显示状态进行分类，从而得到左转向灯的显示状态、右转向灯的显示状态和顶灯的显示状态等。将第一分类器中的每一车灯分类器识别到的车灯显示状态和车灯位置，作为该车灯信息。如此，通过至少识别车辆的转向灯信息和顶灯信息，既能够减少识别的数据量，还能够为车辆运动意图的预测，提供丰富的依据。
在一些实施例中，通过利用所述神经网络，对交通图像进行特征提取，采用多个分类器基于提取的图像特征，预测车辆的车灯信息和朝向信息，即上述步骤S102可以通过图2所示的步骤实现，图2为本公开实施例提供的运动意图确定方法的另一实现流程示意图，结合图2所示的步骤进行以下说明：
步骤S201，利用所述神经网络的卷积层，确定所述交通图像的注意力掩模。
在一些实施例中，将交通图像输入到神经网络的全卷积网络，预测该图像的注意力掩模。
步骤S202，基于所述注意力掩模，确定所述交通图像的空间特征。
在一些实施例中，将交通图像与注意力掩模进行逐元素乘积，将该乘积输出到基于CNN中进行空间特征提取的得到空间特征。
步骤S203，将所述空间特征与所述交通图像的时间特征进行合并，得到所述交通图像的图像特征。
在一些实施例中，将提取到的空间特征输入到一种特殊的循环神经网络(Recurrent Neural Network，RNN)，长短期记忆(Long short-term memory，LSTM)中，与时间特征进行合并，将合并后的特征作为图像特征，以便于后续基于该特征识别车灯状态和车头朝向。
步骤S204，采用所述第一分类器，基于所述图像特征，确定所述车辆的车灯信息和所述车辆的朝向信息。
在一些实施例中，将所述图像特征分别输入所述第一分类器，至少得到所述车辆的每一车灯的预测车灯信息和所述车辆的预测朝向信息。将图像特征分别输入到每一个第一分类器中，进行车灯状态和车头朝向的预测。比如，第一分类器包括：对顶灯的显示状态进行分类的分类器、对左转向灯的显示状态进行分类的分类器、对右转向灯的显示状态进行分类的分类器、对车辆类型进行分类的分类器以及对车辆的车头朝向进行分类的分类器。第一分类器的分类结果包括车辆的每一车灯的显示状态、车辆类型和车头的朝向信息。在同一车灯的车灯信息的分类结果中，选择置信度大于或者等于置信度阈值的预测车灯信息，作为车辆的车灯信息；同样，选择分类结果中朝向信息的置信度大于置信度阈值的预测朝向信息，作为车辆的朝向信息。比如，关于左转向灯的分类结果包括：亮、暗和无；其中，亮的置信度大于置信度阈值，那么将左转向灯处于亮的状态作为该车辆的左转向灯的车灯信息。同理，关于朝向信息包括：朝 前、朝后和横向；其中，朝向的置信度大于置信度阈值，那么将车头朝前作为该车辆的朝向信息。
在本公开实施例中，通过使用多任务学习，可以使用车辆朝向、车辆类型的分类器等辅助车灯状态分类的分类器；进而能够提高对车灯的显示状态进行预测的准确度。比如，使用朝向分类器帮助车灯模型判断左右，使用车辆类型分类器帮助车灯信息分类器判断车灯位置和车灯形状。
在一些实施例中，可以通过以下两种方式，从预测的多个运动意图中选择较优的运动意图，即上述步骤S103可以通过以下步骤实现：
方式一：通过分析用户需求，输出置信度较高的分类器的运动意图，即可以通过以下步骤预测更加准确的运动意图：
第一步，获取对所述车辆的运动意图进行预测的应用需求。
在一些实施例中，应用需求可以是用户自主设定的，比如，设定的对刹车误检最高次数或者对车辆预设转向的监督等。在一个具体例子中，应用需求可以是对右转车辆的误检次数小于5。
第二步，确定与所述应用需求匹配的置信度阈值。
在一些实施例中，基于该应用需求，设定置信度阈值。比如，如果应用需求为对右转车辆的误检次数小于5，可以设定置信度阈值为较大值(比如，设定置信度阈值为0.9)；如果应用需求为对右转车辆的误检次数小于20，可以设定置信度阈值为0.8等。
第三步，将置信度大于所述置信度阈值的运动意图作为所述车辆的确定运动意图。
在一些实施例中，在确定所述车辆的运动意图之后，将置信度大于所述置信度阈值的运动意图作为所述车辆的确定运动意图。预测的运动意图可以是左转、右转、前进、倒退或者刹车等；在这些预测的运动意图中，确定置信度大于应用需求匹配的置信度阈值的预测运动意图，作为车辆的运动意图。如此，按照应用需求设定置信度阈值，从而能够使得预测的运动意图更好地满足用户需求。
方式二：通过分析得到的多个分类结果之间是否有冲突，输出置信度较高的分类器的输出结果，即可以通过以下步骤预测更加准确的运动意图：
第一步，将所述车灯信息和所述朝向信息，输入所述第二分类器，所述第二分类器输出所述车辆的预测运动意图。
在一些实施例中，如果第二分类器为基础分类器，那么基础分类器将全部车灯信息作为一个整体，结合车辆朝向预测车辆的运动意图，得到该车辆整体的预测运动意图；如果第二分类器为扩展分类器，那么扩展分类器基于车灯信息中的转向灯的状态，结合朝向信息预测车辆的运动意图，得到该车辆的扩展运动意图。
第二步，响应于所述预测运动意图与所述第一分类器输出的分类结果不匹配，分别确定所述预测运动意图的第一置信度和所述分类结果的第二置信度。
在一些实施例中，预测运动意图与所述第一分类器输出的分类结果不匹配，为预测运动意图与所述第一分类器输出的分类结果相冲突。在一个具体例子中，第二分类器为基础分类器，预测运动意图为左转，第一分类中左转向灯的分类器输出为灭，右转向灯的分类器输出为亮，顶灯的分类器输出为灭；这样根据第一分类器输出的各个部分车灯亮灭状态，得出该车辆的运动意图为右转，这时出现多个分类器的输出结果冲突。在这种情况下，分别获取到发生冲突的预测运动意图的置信度，以及，分类结果的第二置信度。其中，分类结果的第二置信度可以理解为基于该分类结果确定车辆运动意图的置信度，或者还可以理解为是该分类结果中分类结果整体的置信度。
第三步，基于所述预测运动意图的第一置信度和所述分类结果的第二置信度中较大置信度对应的预测结果，确定所述车辆的运动意图。
在一些实施例中，在第二分类器输出的预测运动意图和分类结果确定出的运动意图中，选择置信度最大的作为该车辆的运动意图。如此，在多个分类器的预测结果产生冲突时，选择置信度较大的作为最终的预测结果，能够更加准确地预测出车辆的运动意图。
下面，将说明本公开实施例在一个实际的应用场景中的示例性应用，以针对基于多任务学习和多阶段学习，确定车灯状态为例，进行说明。
高级驾驶辅助系统(Advanced Driving Assistance System，ADAS)以及自动驾驶任务中为了判断他车意图和未来行驶轨迹，需要进行他车车灯状态的检测，以帮助自车做出碰撞预警和决策规划等任务。在相关技术中，ADAS产品在动态预测方面基本是空白，自动驾驶系统中也鲜有针对多种驾驶场景的车灯模型的动态预测。
在相关技术中，根据识别出的灯的点亮位置来确定哪个方向指示灯是向左转还是向右转是不够准确的，如图3所示， 车辆的右转向灯，在图像31中的左前转向灯和图像32中的右后转向灯的上都亮。
随着行车灯和示宽灯的兴起，车灯呈现着复杂化的态势，仅根据左右两个整灯的亮暗情况已无法判断车辆是否在刹车或者转向。基于此，本公开实施例提供一种车灯状态的预测方法，利用深度学习进行车灯意图判断任务，可以在认知上分解为单灯的亮暗无状态表现出来的车辆意图和整体车灯的亮灭状态表现出来的车辆意图，两个层次，同时辅以车辆朝向和车辆类型的分类器。如此，辅助多任务和多层级的处理有益于车灯网络的学习，还能够大幅提高最终模型的推理精度。
在本公开实施例中，通过获取图片上表现出来的车灯点灭位置信息和车辆前进方向，预测车辆转向灯所表示的左右转。在一些实施例中，对于部分地区车辆转向灯与刹车灯位置颜色均不同的特征，本公开实施例提出一种车灯状态的预测方法，可以通过以下过程实现：
第一，利用多任务学习，使用车辆图片输入单帧模型进行端到端训练，同时输出朝向/车尾顶灯/左车灯/右车灯/车辆类型/左右转向灯，并进行单个车灯即时状态的判断。
在一些可能的实现方式中，由于车灯的位置和形状复杂多变，亮暗组合形式较多；因此，对每个灯都单独设置分类器，即判断左/右车灯、左/右转向灯和车尾顶灯的亮暗无状态分类。在此基础上，额外的信息可以辅助判断车灯的位置和形状类型，比如，车辆方向的分类器可以辅助判别左右车灯，车辆类型的分类器可以辅助判断车灯的形态。
第二，利用多层级学习，在模型学习到单个灯状态的基础上进一步判断刹车状态和转向意图。
这里，刹车状态和转向意图的判断有多个层次，根据多个单个车灯的状态推理得到车灯整体表现出来的转向意图和刹车意图。再联合左车灯状态/右车灯状态/顶灯状态判断整个车的意图。另外，车辆遮挡和运动也给单帧的判断带来了很大不确定性。
在一些实施例中，运动意图确定方法可以通过以下步骤实现：
第一步，使用车辆单帧输入，进行多任务训练，得到多个分类器，包括朝向/车辆类型/顶灯状态/左车灯状态/右车灯状态。
在一些可能的实现方式中，可以通过图4所示的方式确定运动意图，如图4所示，图4为本公开实施例提供的运动意图确定方法的实现框架示意图，将车辆单帧图像400输入车辆检测器401中，以图像中的识别车辆；将识别到的车辆的检测框输入到CNN402中，进行特征提取，得到特征图403。特征图403的维度为7×7×2048。对该特征图403进行处理得到2048维的特征向量404。
第二步，在第一步的分类器的基础上，使用mini-batch的方式进行网络训练。
这里，在一个批量中，一半使用标注为左/右车灯整体(其中，整体指整灯：任何一个子灯(刹车灯/雾灯/转向灯)亮了都算亮)亮暗无的数据，另一半使用左/右转向灯亮暗无的数据。训练基于左右整灯状态的基础车辆意图和基于左/右转向灯的扩展车辆意图。
两组分类器在训练中互相辅助，其中，互相辅助指多任务学习中不同任务有相关性进行相互促进，而且相互促进的过程是在模型训练过程中自动完成的。在一些可能的实现方式中，网络模型可以先输入整灯状态，再集合整灯中转向灯的状态，最终输出车灯的车辆意图。这样，不仅最终结果会有提升，也还能够解决因为标注的迭代(比如，对没有标注各个灯状态的数据进行再次标注)而无法训练的问题。如此，使用多层级学习，在任务的难度设置上由浅至难，从单个车灯的状态的判断到整体刹车状态和转向意图的判断，符合自然的认知层级，利于模型学习；而且使用mini-batch的方式训练网络，能够很大程度上解决数据由于标注不同造成的训练困难，使得同一个模型获得不同标注信息下的多种功能，从而大大减小标注成本。
如图4所示，将提取到的特征向量404输入到多个全连接层(fc)进行分类；其中，全连接层451用于对车辆顶灯的亮、灭和无状态进行分类；全连接层452用于对车辆左转向灯的亮、灭和无状态进行分类；全连接层453用于对车辆右转向灯的亮、灭和无状态进行分类；全连接层454用于对车辆车辆朝向(比如，前进和后退)进行分类；全连接层455用于对车辆类型(比如，小汽车、卡车、公共汽车、出租车、急救车辆或其他车灯)进行分类；全连接层456用于针对车辆朝向为朝前的车辆，基于车辆左右整灯状态对基础车辆意图进行分类；全连接层457用于针对车辆朝向为朝后的车辆，基于车辆左右整灯状态对基础车辆意图进行分类；即，全连接层456和全连接层457训练基于左右整灯状态的基础车辆意图，适用于简单场景；全连接层458用于针对车辆朝向为朝前的车辆，基于车辆的左右转向灯对扩展车辆意图进行分类；全连接层459用于针对车辆朝向为朝后的车辆，基于车辆的左右转向灯对扩展车辆意图进行分类；即，全连接层458和全连接层459训练基于车辆的左右转向灯对扩展车辆意图，适用于复杂场景。将全连接层451至459形成的分类 器的输出进行合并，得到车辆在多个类别下的概率分布。比如，车辆意图的概率
Figure PCTCN2022108284-appb-000001
车辆左转的概率
Figure PCTCN2022108284-appb-000002
车辆右转的概率
Figure PCTCN2022108284-appb-000003
和车辆朝向的概率
Figure PCTCN2022108284-appb-000004
在一些实施例中，上述通过图4所示的方式确定运动意图仅为一种可行的实施方式，本公开实施例确定运动意图的方式不限于此；比如，还可以通过残差网络或深度神经网络等确定运动意图；这里不再一一赘述。
第三步，在第一步和第二步的基础上，能够方便地进行后处理和逻辑添加。比如，可以进行以下多种后处理和逻辑添加：
a、根据车辆的二分类朝向(front/back)判断是左灯还是右灯；
b、卡车/公交没有顶灯和行车灯，根据左右灯的亮暗无状态判断是否刹车或者双闪；
c、出租车的顶灯处一般有广告或者出租车标识灯，根据左右灯的时序状态判断刹车；
d、当多个分类器的预测结果产生冲突时，选择置信度最高的预测结果；或，
e、针对应用层预测中转向和刹车误检少的需求，最终输出置信度高于某个阈值的预测结果。如此，灵活的后处理使得训练方法的应用性更强，通过设置置信度阈值能够减少错误正样本。
第四步，对于训练过程进行补充。
在一些可能的实现方式中，第一步和第二步可以合为一步，直接使用mini-batch的方式进行训练。如果数据集的标签一致，可以不使用mini-batch，采用该数据集对网络训练进行一步训练得到已训练的网络模型。
在本公开实施例中，首先，获取安装于自车摄像机拍摄的图像，然后，基于该图像判断他车亮灯中的方向灯是左转还是右转，最后，基于此，使用图像中显示的他车的车灯信息和前进方向(前进/后退)，进一步判断该他车点灯中的方向灯是左转还是右转。如此，在判断左右转的意图时，将图像中显示的他车的车灯点灯位置信息与前进方向(front/back)信息相结合，能够加强判断车辆左右转的鲁棒性。
本领域技术人员可以理解，在具体实施方式的上述方法中，各步骤的撰写顺序并不意味着严格的执行顺序而对实施过程构成任何限定，各步骤的具体执行顺序应当以其功能和可能的内在逻辑确定。
基于同一发明构思，本公开实施例中还提供了与运动意图确定方法对应的运动意图确定装置，由于本公开实施例中的装置解决问题的原理与本公开实施例上述运动意图确定方法相似，因此装置的实施可以参见方法的实施。
本公开实施例提供一种运动意图确定装置，图5为本公开实施例运动意图确定装置结构组成示意图，如图5所示，所述运动意图确定装置600包括：
图像获取部分601，被配置为获取交通图像；
信息确定部分602，被配置为基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息；
意图确定部分603，被配置为基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图。
在一些实施例中，所述信息确定部分602，包括：
位置信息确定子部分，被配置为基于所述交通图像，确定所述交通图像中所述车辆的点亮的目标车灯的位置信息；
外观信息确定子部分，被配置为基于所述交通图像，确定所述交通图像中的车辆的外观信息；
朝向信息确定子部分，被配置为基于所述车辆的外观信息，确定所述车辆的车头的朝向信息；
所述意图确定部分603，还用于：
基于所述车辆的目标车灯的位置信息和所述车辆的车头的朝向信息，确定所述车辆的运动意图。
在一些实施例中，所述目标车灯为单个转向灯，所述意图确定部分603，包括：
转向信息确定子部分，被配置为基于所述单个转向灯的位置信息和所述车辆的车头的朝向信息，确定转向灯指示的转向信息；
意图确定子部分，被配置为根据所述转向信息，确定所述车辆的转向意图。
在一些实施例中，所述意图确定部分603，包括：
制动状态确定子部分，被配置为响应于所述车灯信息中未包括刹车灯信息、且所述目标车灯为多个转向灯，确定所述车辆处于制动状态。
在一些实施例中，所述装置还包括：
车型信息确定部分，被配置为基于所述交通图像，确定所述交通图像中的车辆的车型信息；
所述意图确定部分603，还用于：
基于所述车灯信息、朝向信息和车型信息，确定所述车辆的运动意图。
在一些实施例中，在确定所述车辆的运动意图时，所述装置还包括：置信度确定部分，被配置为确定所述车辆的运动意图的置信度；
置信度调整部分，被配置为响应于所述朝向信息指示所述车辆为横向，降低所述运动意图的置信度。
在一些实施例中，所述装置还包括：
需求获取部分，被配置为获取对所述车辆的运动意图进行预测的应用需求；
置信度阈值匹配部分，被配置为确定与所述应用需求匹配的置信度阈值；
在确定所述车辆的运动意图之后，所述意图确定部分603，还用于：
将置信度大于所述置信度阈值的运动意图作为所述车辆的确定运动意图。
在一些实施例中，确定所述车灯信息、所述朝向信息和所述车辆的运动意图由神经网络执行；所述神经网络中的第一分类器利用标注了车灯信息和朝向信息的样本图像训练得到，所述神经网络中的第二分类器利用标注了车辆的运动意图的样本图像训练得到。
在一些实施例中，所述第二分类器包括以下至少之一：用于对车辆的基础运动意图进行分类的基础分类器、用于对车辆的扩展运动意图进行分类的扩展分类器，其中，所述基础分类器是基于标注了车辆的整体车灯状态的样本图像训练得到的；所述扩展分类器是基于标注了车辆的转向灯的车灯状态的样本图像训练得到的。
在一些实施例中，所述信息确定部分602，还用于利用所述神经网络，基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息；所述信息确定部分602，包括：
掩模确定子部分，被配置为利用所述神经网络的卷积层，确定所述交通图像的注意力掩模；
空间特征确定子部分，被配置为基于所述注意力掩模，确定所述交通图像的空间特征；
特征合并子部分，被配置为将所述空间特征与所述交通图像的时间特征进行合并，得到所述交通图像的图像特征；
信息确定子部分，被配置为采用所述第一分类器，基于所述图像特征，确定所述车辆的车灯信息和所述车辆的朝向信息。
在一些实施例中，所述意图确定部分603，包括：
信息输入子部分，被配置为将所述车灯信息和所述朝向信息，输入所述第二分类器，所述第二分类器输出所述车辆的预测运动意图；
置信度确定子部分，被配置为响应于所述预测运动意图与所述第一分类器输出的分类结果不匹配，分别确定所述预测运动意图的第一置信度和所述分类结果的第二置信度；
置信度对比子部分，被配置为基于所述预测运动意图的第一置信度和所述分类结果的第二置信度中较大置信度对应的预测结果，确定所述车辆的运动意图。
需要说明的是，以上装置实施例的描述，与上述方法实施例的描述是类似的，具有同方法实施例相似的有益效果。对于本公开装置实施例中未披露的技术细节，请参照本公开方法实施例的描述而理解。
在本公开实施例以及其他的实施例中，“模块”可以是电路、处理器、程序或软件等等，当然也可以是单元，还可以是非模块化的。
需要说明的是，本公开实施例中，如果以软件功能模块的形式实现上述的运动意图确定方法，并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是终端、服务器等)执行本公开各个实施例所述方法的全部或部分。而前述的存储介质包括：U盘、运动硬盘、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。这样，本公开实施例不限制于任何特定的硬件和软件结合。
对应地，本公开实施例再提供一种计算机程序产品，所述计算机程序产品包括计算机可执行指令，该计算机可执行指令被执行后，能够实现本公开实施例提供的运动意图确定方法中的步骤。相应的，本公开实施例再提供一种计算机存储介质，所述计算机存储介质上存储有计算机可执行指令，所述该计算机可执行指令被处理器执行时实现上述实施例提供的运动意图确定方法的步骤。相应的，本公开实施例提供一种计算机设备，图6为本公开实施例计算机设备的组成结构示意图，如图6所示，所述计算机设备700包括：一个处理器701、至少一个通信总线、通信接口702、至少一 个外部通信接口和存储器703。其中，通信接口702配置为实现这些组件之间的连接通信。其中，通信接口702可以包括显示屏，外部通信接口可以包括标准的有线接口和无线接口。其中所述处理器701，配置为执行存储器中图像处理程序，以实现上述实施例提供的运动意图确定方法的步骤。
以上运动意图确定装置、计算机设备和存储介质实施例的描述，与上述方法实施例的描述是类似的，具有同相应方法实施例相似的技术描述和有益效果，限于篇幅，可案件上述方法实施例的记载，故在此不再赘述。对于本公开运动意图确定装置、计算机设备和存储介质实施例中未披露的技术细节，请参照本公开方法实施例的描述而理解。应理解，说明书通篇中提到的“一个实施例”或“一实施例”意味着与实施例有关的特定特征、结构或特性包括在本公开的至少一个实施例中。因此，在整个说明书各处出现的“在一个实施例中”或“在一实施例中”未必一定指相同的实施例。此外，这些特定的特征、结构或特性可以任意适合的方式结合在一个或多个实施例中。应理解，在本公开的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本公开实施例的实施过程构成任何限定。上述本公开实施例序号仅仅为了描述，不代表实施例的优劣。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
在本公开所提供的几个实施例中，应该理解到，所揭露的设备和方法，可以通过其它的方式实现。以上所描述的设备实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，如：多个单元或组件可以结合，或可以集成到另一个系统，或一些特征可以忽略，或不执行。另外，所显示或讨论的各组成部分相互之间的耦合、或直接耦合、或通信连接可以是通过一些接口，设备或单元的间接耦合或通信连接，可以是电性的、机械的或其它形式的。
上述作为分离部件说明的单元可以是、或也可以不是物理上分开的，作为单元显示的部件可以是、或也可以不是物理单元；既可以位于一个地方，也可以分布到多个网络单元上；可以根据实际的需要选择其中的部分或全部单元来实现本实施例方案的目的。另外，在本公开各实施例中的各功能单元可以全部集成在一个处理单元中，也可以是各单元分别单独作为一个单元，也可以两个或两个以上单元集成在一个单元中；上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。本领域普通技术人员可以理解：实现上述方法实施例的全部或部分步骤可以通过程序指令相关的硬件来完成，前述的程序可以存储于计算机可读取存储介质中，该程序在执行时，执行包括上述方法实施例的步骤；而前述的存储介质包括：移动存储设备、只读存储器(Read Only Memory，ROM)、磁碟或者光盘等各种可以存储程序代码的介质。
或者，本公开上述集成的单元如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开实施例的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机、服务器、或者网络设备等)执行本公开各个实施例所述方法的全部或部分。而前述的存储介质包括：移动存储设备、ROM、磁碟或者光盘等各种可以存储程序代码的介质。以上所述，仅为本公开的具体实施方式，但本公开的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本公开揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本公开的保护范围之内。因此，本公开的保护范围应以所述权利要求的保护范围为准。
工业实用性
本公开实施例提供一种运动意图确定方法、装置、设备及存储介质，其中，获取交通图像；基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息；基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图。

Claims (15)

  1. 一种运动意图确定方法，所述方法由电子设备执行，所述方法包括：
    获取交通图像；
    基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息；
    基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图。
  2. 根据权利要求1所述的方法，其中，所述基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息，包括：
    基于所述交通图像，确定所述交通图像中所述车辆的点亮的目标车灯的位置信息；
    基于所述交通图像，确定所述交通图像中的车辆的外观信息；
    基于所述车辆的外观信息，确定所述车辆的车头的朝向信息；
    所述基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图，包括：
    基于所述车辆的目标车灯的位置信息和所述车辆的车头的朝向信息，确定所述车辆的运动意图。
  3. 根据权利要求2所述的方法，其中，所述目标车灯为单个转向灯，所述基于所述车辆的目标车灯的位置信息和所述车辆的车头的朝向信息，确定所述车辆的运动意图，包括：
    基于所述单个转向灯的位置信息和所述车辆的车头的朝向信息，确定转向灯指示的转向信息；
    根据所述转向信息，确定所述车辆的转向意图。
  4. 根据权利要求2所述的方法，其中，所述基于所述车辆的目标车灯的位置信息和所述车辆的车头的朝向信息，确定所述车辆的运动意图，包括：
    响应于所述车灯信息中未包括刹车灯信息、且所述目标车灯为多个转向灯，确定所述车辆处于制动状态。
  5. 根据权利要求1至4任一所述的方法，其中，所述方法还包括：
    基于所述交通图像，确定所述交通图像中的车辆的车型信息；
    所述基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图，包括：
    基于所述车灯信息、朝向信息和车型信息，确定所述车辆的运动意图。
  6. 根据权利要求1至4任一所述的方法，其中，在确定所述车辆的运动意图时，确定所述车辆的运动意图的置信度，所述方法还包括：
    响应于所述朝向信息指示所述车辆为横向，降低所述运动意图的置信度。
  7. 根据权利要求1至4任一所述的方法，其中，在确定所述车辆的运动意图时，确定所述车辆的运动意图的置信度；所述方法还包括：
    获取对所述车辆的运动意图进行预测的应用需求；
    确定与所述应用需求匹配的置信度阈值；
    在确定所述车辆的运动意图之后，所述方法还包括：
    将置信度大于所述置信度阈值的运动意图作为所述车辆的确定运动意图。
  8. 根据权利要求1所述的方法，其中，确定所述车灯信息、所述朝向信息和所述车辆的运动意图由神经网络执行；
    所述神经网络中的第一分类器利用标注了车灯信息和朝向信息的样本图像训练得到，所述神经网络中的第二分类器利用标注了车辆的运动意图的样本图像训练得到。
  9. 根据权利要求8所述的方法，其中，所述第二分类器包括以下至少之一：用于对车辆的基础运动意图进行分类的基础分类器、用于对车辆的扩展运动意图进行分类的扩展分类器，其中，所述基础分类器是基于标注了车辆的整体车灯状态的样本图像训练得到的；所述扩展分类器是基于标注了车辆的转向灯的车灯状态的样本图像训练得到的。
  10. 根据权利要求8所述的方法，其中，利用所述神经网络，基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息，包括：
    利用所述神经网络的卷积层，确定所述交通图像的注意力掩模；
    基于所述注意力掩模，确定所述交通图像的空间特征；
    将所述空间特征与所述交通图像的时间特征进行合并，得到所述交通图像的图像特征；
    采用所述第一分类器，基于所述图像特征，确定所述车辆的车灯信息和所述车辆的朝向信息。
  11. 根据权利要求10所述的方法，其中，所述基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图，包括：
    将所述车灯信息和所述朝向信息，输入所述第二分类器，所述第二分类器输出所述车辆的预测运动意图；
    响应于所述预测运动意图与所述第一分类器输出的分类结果不匹配，分别确定所述预测运动意图的第一置信度和所述分类结果的第二置信度；
    基于所述预测运动意图的第一置信度和所述分类结果的第二置信度中较大置信度对应的预测结果，确定所述车辆的运动意图。
  12. 一种运动意图确定装置，其中，所述装置包括：
    图像获取部分，被配置为获取交通图像；
    信息确定部分，被配置为基于所述交通图像，确定所述交通图像中的车辆的车灯信息和所述车辆的朝向信息；
    意图确定部分，被配置为基于所述车灯信息和所述朝向信息，确定所述车辆的运动意图。
  13. 一种计算机存储介质，其中，所述计算机存储介质上存储有计算机可执行指令，该计算机可执行指令被执行后，能够实现权利要求1至11任一项所述的方法步骤。
  14. 一种计算机设备，其中，所述计算机设备包括存储器和处理器，所述存储器上存储有计算机可执行指令，所述处理器运行所述存储器上的计算机可执行指令时可实现权利要求1至11任一项所述的方法步骤。
  15. 一种计算机程序产品，所述计算机程序产品包括计算机程序或指令，在所述计算机程序或指令在电子设备上运行的情况下，使得所述电子设备执行权利要求1至11中任意一项所述的方法。
PCT/CN2022/108284 2022-02-09 2022-07-27 一种运动意图确定方法、装置、设备及存储介质 WO2023151241A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210122826.X 2022-02-09
CN202210122826.XA CN114519848A (zh) 2022-02-09 2022-02-09 一种运动意图确定方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023151241A1 true WO2023151241A1 (zh) 2023-08-17

Family

ID=81597612

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/108284 WO2023151241A1 (zh) 2022-02-09 2022-07-27 一种运动意图确定方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN114519848A (zh)
WO (1) WO2023151241A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114519848A (zh) * 2022-02-09 2022-05-20 商汤集团有限公司 一种运动意图确定方法、装置、设备及存储介质
CN115402352B (zh) * 2022-08-24 2024-08-20 东软睿驰汽车技术（沈阳）有限公司 预测车辆刹车的方法、装置、设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105975915A (zh) * 2016-04-28 2016-09-28 大连理工大学 一种基于多任务卷积神经网络的前方车辆参数识别方法
US20200234066A1 (en) * 2019-01-18 2020-07-23 Toyota Research Institute, Inc. Attention-based recurrent convolutional network for vehicle taillight recognition
CN112307833A (zh) * 2019-07-31 2021-02-02 浙江商汤科技开发有限公司 识别智能行驶设备的行驶状态的方法及装置、设备
CN113050615A (zh) * 2019-12-26 2021-06-29 浙江吉利汽车研究院有限公司 一种驾驶安全控制方法、装置、电子设备及存储介质
CN113291320A (zh) * 2021-06-15 2021-08-24 苏州智加科技有限公司 一种车辆轨迹预测方法、装置、设备及存储介质
CN114519848A (zh) * 2022-02-09 2022-05-20 商汤集团有限公司 一种运动意图确定方法、装置、设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105975915A (zh) * 2016-04-28 2016-09-28 大连理工大学 一种基于多任务卷积神经网络的前方车辆参数识别方法
US20200234066A1 (en) * 2019-01-18 2020-07-23 Toyota Research Institute, Inc. Attention-based recurrent convolutional network for vehicle taillight recognition
CN112307833A (zh) * 2019-07-31 2021-02-02 浙江商汤科技开发有限公司 识别智能行驶设备的行驶状态的方法及装置、设备
CN113050615A (zh) * 2019-12-26 2021-06-29 浙江吉利汽车研究院有限公司 一种驾驶安全控制方法、装置、电子设备及存储介质
CN113291320A (zh) * 2021-06-15 2021-08-24 苏州智加科技有限公司 一种车辆轨迹预测方法、装置、设备及存储介质
CN114519848A (zh) * 2022-02-09 2022-05-20 商汤集团有限公司 一种运动意图确定方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN114519848A (zh) 2022-05-20

Similar Documents

Publication Publication Date Title
US12067764B2 (en) Brake light detection
US10877485B1 (en) Handling intersection navigation without traffic lights using computer vision
WO2023151241A1 (zh) 一种运动意图确定方法、装置、设备及存储介质
Ohn-Bar et al. On surveillance for safety critical events: In-vehicle video networks for predictive driver assistance systems
US11688179B2 (en) Inferring intent using computer vision
Satzoda et al. Looking at vehicles in the night: Detection and dynamics of rear lights
Devi et al. A comprehensive survey on autonomous driving cars: A perspective view
CN108357418B (zh) 一种基于尾灯识别的前车驾驶意图分析方法
Haltakov et al. Semantic segmentation based traffic light detection at day and at night
Rezaei et al. Computer vision for driver assistance
CN115104138A (zh) 多模态、多技术载具信号检测
WO2021017341A1 (zh) 识别智能行驶设备的行驶状态的方法及装置、设备
CN110969647B (zh) 一种整合车辆的识别跟踪与车灯检测的方法
US20220410931A1 (en) Situational awareness in a vehicle
US11900690B2 (en) Apparatus, method, and computer program for identifying state of signal light, and controller
CN112733818B (zh) 基于注意力机制的车灯状态识别方法、装置、终端和介质
Jeon et al. A deep learning framework for robust and real-time taillight detection under various road conditions
Oldenziel et al. Provident detection of vehicles at night
Ewecker et al. Provident vehicle detection at night for advanced driver assistance systems
KR20080004833A (ko) 주간 및 야간 주행 차량을 조도상황에 따라 검출하는 방법및 장치
CN113989772A (zh) 一种交通灯检测方法、装置、车辆和可读存储介质
Varma et al. Vision Based Advanced Driver Assistance System Using Deep Learning
CN113715827A (zh) 行车提醒方法、装置、电子设备及存储介质
Sun et al. Deep Hierarchical Rear‐Lamp Tracking at Nighttime
Kushwaha et al. Forward vehicle brake light detection for avoiding road accidents using Yolov7 and IR sensor

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22925600

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE