CN113673412A - Key target object identification method and device, computer equipment and storage medium - Google Patents

Key target object identification method and device, computer equipment and storage medium Download PDF

Info

Publication number
CN113673412A
CN113673412A CN202110944797.0A CN202110944797A CN113673412A CN 113673412 A CN113673412 A CN 113673412A CN 202110944797 A CN202110944797 A CN 202110944797A CN 113673412 A CN113673412 A CN 113673412A
Authority
CN
China
Prior art keywords
target object
vehicle
target
full
key
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110944797.0A
Other languages
Chinese (zh)
Other versions
CN113673412B (en
Inventor
刘洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Uisee Shanghai Automotive Technologies Ltd
Original Assignee
Uisee Shanghai Automotive Technologies Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Uisee Shanghai Automotive Technologies Ltd filed Critical Uisee Shanghai Automotive Technologies Ltd
Priority to CN202110944797.0A priority Critical patent/CN113673412B/en
Publication of CN113673412A publication Critical patent/CN113673412A/en
Application granted granted Critical
Publication of CN113673412B publication Critical patent/CN113673412B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Abstract

The embodiment of the invention discloses a method and a device for identifying a key target object, computer equipment and a storage medium. The method comprises the following steps: acquiring a target object in the surrounding environment of the vehicle when the vehicle is in an automatic driving starting mode; acquiring a target object association description parameter set corresponding to the target object; and inputting the target object association description parameter set into an LSTM network to obtain a key target object identification result corresponding to the target object. By the technical scheme of the embodiment of the invention, an ideal target selection model can be obtained only by performing data training according to the framework of the embodiment of the invention without requiring engineering designers to perform complex algorithm design, and the key target identification is performed on each target existing around the vehicle, so that the identification accuracy of the key target in the automatic driving system is improved.

Description

Key target object identification method and device, computer equipment and storage medium
Technical Field
The embodiment of the invention relates to a computer technology, in particular to an automatic driving and artificial intelligence technology, and particularly relates to a method and a device for identifying a key target object, computer equipment and a storage medium.
Background
The automatic driving system has the environment sensing capacity and the control capacity of the motion state of the vehicle. The ability of an autonomous driving system to control the state of motion of a vehicle is based on its cognitive ability to perceive results, an important aspect of which is the selection of targets in various circumstances, namely: the target selection is carried out according to the principle or method of the targets which are accurately tracked and positioned in front of or around the system, and the safe and effective passing of the vehicle through the current area can be controlled.
The target selection function under the autopilot system needs to meet timeliness and accuracy. The existing target selection algorithm mainly comprises the following steps: the method comprises a grid method, a method for selecting a target by calculating the overlapping condition of the motion tracks of the vehicle and the target vehicle, a method for correcting the motion track of the vehicle by using the motion track of a front target, a method for selecting the target according to the potential energy field of the environment where the system is located, and the like.
In the process of implementing the invention, the inventor finds that the prior art mainly has the following defects: the grid method is mainly suitable for low-speed motion scenes, and in high-speed motion scenes, the function of a mainstream low-dominant-frequency vehicle-mounted computing platform cannot be realized due to the undersized grid size requirement; the method for selecting the target by calculating the overlapping condition of the motion tracks of the vehicle and the target vehicle is easy to cause target selection errors due to the change of the curvature information of the front road and the lag of the motion posture information of the vehicle; the method for correcting the motion trail of the vehicle by using the motion trail of the front target has the same difficulty in ensuring the effect when the front target is less; the method for selecting the target according to the potential energy field of the environment where the system is located has high difficulty in engineering implementation, and the accuracy depends on the degree of accurate measurement of the energy of the target in each field.
Disclosure of Invention
The embodiment of the invention provides a method and a device for identifying a key target object, computer equipment and a storage medium, aiming at realizing a new method for identifying the key target object of each target object existing around a vehicle in the running process of the vehicle and improving the identification accuracy of the key target object.
In a first aspect, an embodiment of the present invention provides a method for identifying a key target, including:
acquiring a target object in the surrounding environment of the vehicle when the vehicle is in an automatic driving starting mode;
acquiring a target object association description parameter set corresponding to the target object;
and inputting the target object association description parameter set into an LSTM (Long Short-Term Memory network) to obtain a key target object identification result corresponding to the target object.
In a second aspect, an embodiment of the present invention further provides a preceding vehicle following method in an automatic driving start mode, including:
when a vehicle enters a preceding vehicle following scene in an automatic driving starting mode, identifying each target object appearing in the surrounding environment of the vehicle by adopting the method for identifying the key target object in any embodiment of the invention to obtain the key target object;
and taking the key target object as a following target, and carrying out matched vehicle speed following control on the vehicle according to the real-time driving speed of the following target.
In a third aspect, an embodiment of the present invention further provides a device for identifying a key target, where the device includes:
the system comprises a target object acquisition module, a target object acquisition module and a target object acquisition module, wherein the target object acquisition module is used for acquiring a target object in the surrounding environment of the vehicle when the vehicle is in an automatic driving starting mode;
a parameter set obtaining module, configured to obtain a target object association description parameter set corresponding to the target object;
and the result acquisition module is used for inputting the target object association description parameter set into the LSTM network to obtain a key target object identification result corresponding to the target object.
In a fourth aspect, an embodiment of the present invention further provides a leading vehicle following device in an automatic driving start mode, where the device includes:
the key target object identification module is used for identifying and obtaining a key target object in each target object appearing in the surrounding environment of the vehicle by adopting the identification method of the key target object in any embodiment of the invention when the vehicle enters a preceding vehicle following scene in the automatic driving starting mode;
and the vehicle speed control module is used for taking the key target object as a following target and carrying out matched vehicle speed following control on the vehicle according to the real-time driving speed of the following target.
In a fifth aspect, an embodiment of the present invention further provides a computer device, where the computer device includes:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a method of identifying a key target object or a method of preceding vehicle following in an autonomous driving initiation mode as described in any of the embodiments of the invention.
In a sixth aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method for identifying a key object or the method for following a preceding vehicle in an automatic driving start mode according to any embodiment of the present invention.
According to the technical scheme of the embodiment of the invention, an effective method for obtaining the target selection function applicable to the automatic driving system is creatively provided by a technical means of inputting the target object associated description parameter set corresponding to the target object in the vehicle surrounding environment in the automatic driving mode into the LSTM network and obtaining the key target object identification result corresponding to the target object according to the output result of the LSTM network, so that the method can be well applicable to various sensors and various environments where the automatic driving system is located, a specific target selection algorithm is not required to be designed, the method is simple and convenient to realize, and great improvement is brought to the target selection performance of the automatic driving system.
Drawings
FIG. 1a is a flowchart of a method for identifying a key target according to a first embodiment of the present invention;
FIG. 1b is a schematic diagram of an LSTM-based learning model structure to which an embodiment of the present invention is applied;
FIG. 2a is a flowchart of a method for identifying a key target according to a second embodiment of the present invention;
FIG. 2b is a schematic structural diagram of an LSTM-based learning model to which the second embodiment of the present invention is applied;
FIG. 3a is a flowchart of a method for identifying a key target according to a third embodiment of the present invention;
FIG. 3b is a schematic structural diagram of an LSTM-based learning model to which a third embodiment of the present invention is applied;
FIG. 4a is a flowchart of a method for identifying a key target according to a fourth embodiment of the present invention;
FIG. 4b is a schematic structural diagram of an LSTM-based learning model to which the fourth embodiment of the present invention is applied;
FIG. 5 is a flow chart of a preceding vehicle following method in an automatic driving start mode according to a fifth embodiment of the present invention;
fig. 6 is a schematic structural diagram of an apparatus for identifying a key target in a sixth embodiment of the present invention;
fig. 7 is a schematic structural diagram of a preceding vehicle following device in an automatic driving start mode in a seventh embodiment of the invention;
fig. 8 is a schematic structural diagram of a computer device in an eighth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1a is a flowchart of a method for identifying a key target object according to an embodiment of the present invention, where the embodiment is applicable to a case where a vehicle is in an automatic driving start mode to identify a key target object for each target object existing around the vehicle, and the method may be executed by a device for identifying a key target object, where the device may be implemented in a hardware and/or software manner, and may be generally integrated in a vehicle end of a vehicle with an automatic driving function, where the method specifically includes the following steps:
and S110, acquiring the target object in the surrounding environment of the vehicle when the vehicle is in the automatic driving starting mode.
The target object refers to any object appearing or existing around the vehicle in the automatic driving mode, and the target object may be a dynamic object, such as a pedestrian, a non-motor vehicle, or another vehicle in driving, or may be a static object, such as a tree or a building, which is not limited in this embodiment.
In this embodiment, the target object in the surrounding environment of the vehicle may be identified and acquired through various sensors configured on the vehicle, for example, a vision sensor, a ranging sensor, a laser radar sensor, or a millimeter wave radar sensor, and specifically, the target object may be identified according to a detection result of a single sensor, or the target object may be identified through a fusion detection result of multiple sensors, which is not limited in this embodiment.
And S120, acquiring a target object association description parameter set corresponding to the target object.
The object-related description parameter set includes a plurality of object-related description parameters, and the object-related description parameters refer to parameter information that corresponds to an object and can be used for describing a motion state of the object and a motion state of a current vehicle. By way of example and not limitation, the object association description parameter may include: the vehicle speed and running state information comprises distance description information of the target object from the vehicle, speed description information of the target object, position relation information of the target object and the lane line, the speed and running state information of the vehicle and the like.
It should be noted that, in the technical solutions of the embodiments of the present invention, the LSTM network is used to identify the key object of each object, but the LSTM network is required to input feature sets at different time points for long-term fusion, so that the matched identification result can be obtained correspondingly.
Correspondingly, the object association description parameter set may specifically include multiple object association description parameters acquired at multiple time points and corresponding to the identified object, or may also be a parameter matrix formed by multiple parameter vectors after parameter vectors are respectively constructed according to the object association description parameters acquired at different time points.
And S130, inputting the target object association description parameter set into an LSTM network to obtain a key target object identification result corresponding to the target object.
The key target object refers to a target object which can interfere with the running track of the vehicle in the automatic driving process of the vehicle, namely the target object which needs to be avoided in the running process of the vehicle. The key object recognition result refers to a result of recognizing whether or not the object is a key object, and the result is generally possible only for two kinds, that is, the object is a key object and the object is not a key object.
In this embodiment, the LSTM network is configured to perform long-term and short-term feature fusion according to an input object-related description parameter set corresponding to an object, and output a classification result that the object is identified as a key object according to a feature fusion result.
Fig. 1b is a schematic diagram of an LSTM-based learning model structure according to an embodiment of the present invention. In this embodiment, to avoid unnecessary computation on the basis of ensuring the accuracy of the computation result, the hidden state unit inside the LSTM model is taken as 4 × 1 dimension, and the LSTM model is actually combined from the preceding 4 model states and the current model state after being expanded (that is, the LSTM model includes 5 sequentially connected LSTM units in total).
In fig. 1b, a vector consisting of L1 to L4 may be formed according to a plurality of object-related description parameters corresponding to an object obtained at different time points, and if the LSTM model uses 5 model states, 5 vectors consisting of L1 to L4 formed by the object-related description parameters obtained at 5 time points need to be input to the LSTM network.
Wherein T is T0Represents the current time, and Δ t represents the operation period. So that T is T0-n Δ t represents the first n periodic instants with reference to the current instant. Each LSTM ticketThe element forwards a 4 x 1-dimensional vector representing the output of the cell, and exemplary LSTM1 forwards the LSTM2 vector
Figure BDA0003216394940000051
LSTM2 vector to LSTM3
Figure BDA0003216394940000052
LSTM3 vector to LSTM4
Figure BDA0003216394940000053
LSTM4 vector to LSTM5
Figure BDA0003216394940000054
Correspondingly, after the 4 x 1 dimensional vector of the target object at five cycle times is successfully input into the LSTM network and the network calculation is carried out, the output state quantity result of the LSTM network at the time t0 is [ H ]1,H2,H3,H4]。
It will be appreciated that in the LSTM network, a classification layer (not shown) may also be included for deriving H from the output state quantity result1,H2,H3,H4]And acquiring a key target object identification result corresponding to the target object, namely classifying the target object as the key target object or classifying the target object as not the key target object.
According to the technical scheme of the embodiment of the invention, an effective method for obtaining the target selection function applicable to the automatic driving system is creatively provided by a technical means of inputting the target object associated description parameter set corresponding to the target object in the vehicle surrounding environment in the automatic driving mode into the LSTM network and obtaining the key target object identification result corresponding to the target object according to the output result of the LSTM network, so that the method can be well applicable to various sensors and various environments where the automatic driving system is located, a specific target selection algorithm is not required to be designed, the method is simple and convenient to realize, and great improvement is brought to the target selection performance of the automatic driving system.
On the basis of the foregoing embodiments, acquiring the target object association description parameter set corresponding to the target object may further include:
acquiring a plurality of target object associated description parameters of a target object at a current detection time point to form a real-time description parameter vector, and acquiring historical description parameter vectors of the target object at a preset number of continuous historical detection time points; and forming a description parameter time matrix corresponding to the target object as the target object associated description parameter set according to the real-time description parameter vector and a plurality of historical description parameter vectors.
Correspondingly, inputting the target object association description parameter set into the LSTM network to obtain a key target object identification result corresponding to the target object, which may specifically include:
in the description parameter time matrix, sequentially acquiring a description parameter vector according to the sequence of detection time points from near to far, and inputting the description parameter vector into the LSTM network;
and after detecting that the LSTM unit in the LSTM network finishes the calculation processing of the description parameter vector, returning to execute the operation of sequentially obtaining one description parameter vector until the input processing of all the description parameter vectors in the description parameter time matrix is finished and the complete input of the description parameter time matrix is finished, and obtaining the identification result of the key target object output by the LSTM network.
On the basis of the foregoing embodiments, before inputting the target object association description parameter set into the LSTM long-short term memory network, the method may further include:
and normalizing or normalizing and expanding each object related description parameter included in the object related description parameter set.
In the previous example, the manner of forming the real-time description parameter vector after obtaining the multiple target object associated description parameters of the target object at the current detection time point may be: carrying out normalization processing or normalization expansion processing on each target object association description parameter to form a normalization result or a normalization expansion result respectively corresponding to each target object association description parameter; and sequencing the normalization results or the normalization expansion results according to a preset description parameter arrangement sequence to form a real-time description parameter vector corresponding to the target object.
On the basis of the foregoing embodiments, a manner of forming a description parameter time matrix corresponding to the target object according to the real-time description parameter vector and the plurality of historical description parameter vectors may be:
judging whether a target alternative description parameter time matrix matched with the target object exists in an alternative description parameter time matrix set updated at a previous historical detection time point of a current detection time point;
if so, removing the historical description parameter vector at the historical detection time point farthest from the current detection time point in the target alternative description parameter time matrix, and forming a description parameter time matrix corresponding to the target object according to the remaining historical description parameter vectors and the target real-time description parameter vector;
if not, forming a description parameter time matrix corresponding to the currently processed object according to a preset number of all-zero vectors and the target real-time description parameter vector.
Meanwhile, the updating of the candidate description parameter time matrix set corresponding to the current detection time point may include:
adding the description parameter time matrix of each object at the current detection time point into the alternative description parameter time matrix set, or replacing the matched alternative description parameter time matrix in the alternative description parameter time matrix set; acquiring each associated alternative description parameter time matrix corresponding to other objects in the alternative description parameter time matrix set; removing the historical description parameter vector at the historical detection time point farthest from the current detection time point in each associated target candidate description parameter time matrix, and obtaining each new associated target candidate description parameter time matrix according to the remaining historical description parameter vectors and all-zero vectors; and removing all-zero matrixes included in the candidate description parameter time matrix set.
In a specific example, in order to enable target-related description parameters at multiple historical times of a target to be commonly input to an LSTM network for identification processing after the target is detected, a description parameter time matrix corresponding to each object detected by a vehicle may be maintained in real time, and a matching target-related description parameter set is formed for the target detected in real time based on the description parameter time matrix, that is, the description parameter time matrix is input to the LSTM model.
For example, the maintenance process of the description parameter time matrix corresponding to each object may be as follows:
if at t1Identifying a target object A and a target object B for the first time in the moment, and inputting input characteristics under 5 time points by the LSTM, correspondingly constructing a description parameter time matrix [ A', 0,0] corresponding to the target object A]The time matrix of the description parameter corresponding to the object B is [ B', 0,0,0]](ii) a Wherein A 'and B' are determined by the ratio of target A to target B at t1And (3) a target real-time description parameter vector formed by the associated description parameters of each target object at the moment, wherein 0 is a zero vector with the same number of vector elements contained in A 'or B'.
If at the next time point t2When only the object B is detected but the object a is not detected, the time matrix of the description parameter corresponding to the object a is [0, a', 0]The time matrix of the description parameters corresponding to the object B is [ B ", B', 0](ii) a If at the next time point t3When only the target object A is detected but the target object B is not detected, the time matrix of the description parameters corresponding to the target object A is [ A ", 0, A', 0]]The parameter time matrix corresponding to the object B is [0, B ', B', 0]]。
It should be noted that, when a certain object is not detected at a certain time point, only the description parameter time matrix corresponding to the object is updated without being input into the LSTM network, and only when a certain object is detected to exist around the vehicle at the current time point, the description parameter time matrix corresponding to the object is input into the LSTM network for identification.
Example two
Fig. 2a is a flowchart of a method for identifying a key target according to a second embodiment of the present invention. In this embodiment, the target object association description parameter set is input into an LSTM network to obtain a key target object identification result corresponding to the target object, which is specifically: inputting the target object associated description parameter set of the target object into a full-connection network, and acquiring a global fusion abstract result output by the full-connection network aiming at the target object associated description parameter set; and inputting the global fusion abstract result of the target object into the LSTM network to obtain a key target object identification result corresponding to the target object.
As shown in fig. 2a, the method comprises the following specific steps:
s210, when the vehicle is in the automatic driving starting mode, the target object in the surrounding environment of the vehicle is obtained.
And S220, acquiring a target object association description parameter set corresponding to the target object.
In a specific example, the target object association description parameters stored in the target object association description parameter set and respectively corresponding to different time points may include:
the system comprises a vehicle, a radar reflecting section, a vehicle sensor, a vehicle speed sensor, a radar reflecting section and a vehicle type attribute; a distance from a target object representing lane line information to the left and right lane lines and an effective extension distance of the left and right lane lines; the vehicle speed, the vehicle yaw rate, the vehicle steering angle, and the like that indicate the vehicle motion state information are not limited in the present embodiment.
Wherein the fusion state of the at least one sensor that detects the target object refers to the number or type of sensors that can detect the target object. For example, if the target is detected only by the visual sensor, the fusion state is 1; if the target object is detected only by the laser radar sensor, the fusion state is 2; if the target object is detected by the laser radar sensor and the vision sensor at the same time, the fusion state is 3. The class attribute of the object may be: pedestrian 0, vehicle 1, and tree 2, etc.
And S230, inputting the target object associated description parameter set of the target object into a fully-connected network, and acquiring a global fusion abstract result output by the fully-connected network aiming at the target object associated description parameter set.
The fully-connected network refers to a network in which any node of a certain layer has a connection relation with all nodes of a layer above the certain layer, and is used for integrating features extracted from a previous layer. The global fusion abstract result refers to an output result of the target object association description parameter set after full connection processing in the full connection network.
S240, inputting the global fusion abstract result of the target object into the LSTM network to obtain a key target object identification result corresponding to the target object.
Fig. 2b is a schematic diagram of an LSTM-based learning model structure according to an embodiment of the present invention. The parameters in the target object association description parameter set are C1-C5For example, the fully-connected network includes three layers, in this embodiment, the number of nodes in the first layer of the fully-connected network is not limited as long as the number of nodes in the first layer of the fully-connected network is greater than the number of input nodes, and this embodiment takes the first layer of the fully-connected network as 6 nodes as an example. Wherein L is1-1-L1-6A first layer being a fully connected network portion for fully connecting the input parameters; l is2-1-L2-3A second layer being a part of the fully connected network for fully connecting the first layer network; l is3-1And the third layer is a full connection network part and is used for performing full connection on the second layer network. Inputting the target object associated description parameter set corresponding to the target object into the first layer of the full-connection network in a full-connection mode, connecting all nodes of the first layer of the full-connection network with all nodes of the second layer of the full-connection network, and connecting all nodes of the second layer of the full-connection network with all nodes of the third layer of the full-connection network, so as to obtain a final global fusion abstract result, namely, the final global fusion abstract result is stored in the node L3-1All of (1)And (4) locally fusing the abstract result, and then inputting the global fusion abstract result into the LSTM network to obtain a key target object identification result corresponding to the target object.
According to the technical scheme of the embodiment of the invention, the target object associated description parameter set corresponding to the target object is input into the fully-connected network, and the obtained global fusion abstract result is input into the LSTM network to obtain the key target object identification result corresponding to the target object, so that the calculation speed of the LSTM network on the input parameters is increased, the reliability is improved, the delay time is reduced, the complex algorithm design of engineering designers can be avoided, and the ideal target selection model can be obtained only by the data training of the engineering personnel according to the frame in the embodiment of the invention, the key target object identification is carried out on each target object existing around the vehicle, and the identification accuracy of the key target object in the automatic driving system is improved.
EXAMPLE III
Fig. 3a is a flowchart of a method for identifying a key target according to a third embodiment of the present invention. In this embodiment, the target object association description parameter set of the target object is input into the full-connection network, specifically: and grouping the object associated description parameters included in the object associated description parameter set according to a preset classification rule to obtain a plurality of parameter groups, and inputting the parameter groups into a full-connection network in parallel.
Correspondingly, the global fusion abstract result of the target object is input into the LSTM network, and a key target object identification result corresponding to the target object is obtained, specifically: inputting the global fusion abstract result of the target object into the LSTM network to obtain a plurality of long-term and short-term fusion results output by the LSTM network; and inputting each long-term and short-term fusion result into a classification network, and acquiring the key target object identification result output by the classification network.
As shown in fig. 3a, the method comprises the following specific steps:
and S310, acquiring the target object in the surrounding environment of the vehicle when the vehicle is in the automatic driving starting mode.
And S320, acquiring a target object association description parameter set corresponding to the target object.
S330, grouping the object associated description parameters included in the object associated description parameter set according to a preset classification rule to obtain a plurality of parameter groups, inputting the parameter groups into a full-connection network in parallel, and acquiring a global fusion abstract result output by the full-connection network aiming at the object associated description parameter set.
The preset classification rule refers to a preset rule for classifying parameters in the target object association description parameter set, and for example, the parameters may be classified according to the sending object of the parameters, for example, the parameters are divided into target attribute information and vehicle attribute information, which is not limited in this embodiment.
Optionally, the fully connected network specifically includes: the system comprises a plurality of local sub-networks and a unified full-connection layer respectively connected with the local sub-networks, wherein different local sub-networks are used for inputting different parameter groups; the local sub-network is used for carrying out at least one stage of full connection processing on each target object associated description parameter in the input parameter grouping to obtain a local fusion abstract result; and the uniform full-connection layer is used for performing full-connection processing on the local fusion abstract results output by each local sub-network to obtain the global fusion abstract results.
Optionally, the local subnetwork comprises: the system comprises a plurality of sequentially connected local full-connection layers, wherein each local full-connection layer comprises a plurality of full-connection nodes; each full-link node in the next full-link layer is connected with each full-link node in the previous full-link layer, and each target object associated description parameter included in the parameter grouping is input into each full-link node in the first full-link layer.
According to the technical scheme, the target object association description parameter set is classified, gathered and input, so that the calculation burden is effectively relieved, and the processing speed of the algorithm is improved.
S340, inputting the global fusion abstract result of the target object into the LSTM network, and acquiring a plurality of long-term and short-term fusion results output by the LSTM network.
The long-term and short-term fusion result refers to a result obtained after processing by the LSTM network, and generally refers to an output state quantity result of the LSTM network.
And S350, inputting each long-term and short-term fusion result into a classification network, and acquiring the key target object identification result output by the classification network.
The classification network refers to a network for processing the long-term and short-term fusion results to obtain classification results; the classification result indicates whether the target is to be selected.
Fig. 3b is a schematic diagram of a LSTM-based learning model structure according to an embodiment of the present invention. In this embodiment, each target related description parameter in the target related description parameter set is divided into two groups, where one group is C1-C5Another set of parameters is C6-C9. Local full connection layer L1-1-L1-6And a local full connection layer L2-1-L2-3Forming a local sub-network for inputting the parameter C1-C5(ii) a Local full connection layer L1-7-L1-11And a local full connection layer L2-4-L2-6Forming a local sub-network for inputting the parameter C6-C9，L3-1And L3-2Forming a uniform full connection layer.
Wherein two local sub-networks are connected with L in a unified full connection layer3-1And L3-2And carrying out full connection to form a full connection network. Storing of fully connected network outputs at node L3-1And L3-2The global fusion abstract result in (1) is input into the LSTM network, and then a plurality of long-short term fusion results [ H ] output by the LSTM network are input1,H2,H3,H4]And inputting the result into a classification network capable of classifying the result to obtain a key target object identification result.
According to the technical scheme of the embodiment of the invention, the target object association description parameters grouped according to the preset classification rule are grouped and input into the fully-connected network in parallel, then the overall fusion abstract result output by the fully-connected network is input into the LSTM network, and finally a plurality of long-term and short-term fusion results output by the LSTM network are input into the classification network to obtain the key target object identification result.
On the basis of the above embodiments, the fully-connected network, the LSTM network, and the classification network may be pre-trained in advance using training samples.
Specifically, the training samples used include: and presetting a sample object associated description parameter set of the sample object in the detection scene and a labeling result of whether the sample object is a key object.
The sample object association description parameter set is consistent with the data format included in the object association description parameter set according to the embodiments of the present invention.
Optionally, the detection scenario of the sample object in the training sample includes at least one of the following:
detecting a detection scene in which the sample object is detected at a plurality of continuous detection time points; detecting the detection scene obtained by interval detection under a plurality of continuous detection time points of the sample object; and a detection scene in which the sample object is detected in at least one preceding detection time point among the plurality of successive detection time points and is not detected in all the following detection time points.
In a specific example, a sequence unit is composed of 8 consecutive time instances, and the sequence parameters are sent to a full-connection network, an LSTM network and a classification network. The tissue types can be classified into the following three types according to different situations:
(1)#N-#N-#N-#N-#N-#N-#N-#N
(2)#N-#N-0-0-0-0-0-0
(3)#N-0-0-0-0-#N-#N-#N
where # N represents a sample object whose sample object ID (identification document) is # N. The first case represents that the sample object with ID # N appears for 8 frames (i.e., 8 cycle times) consecutively. The second case represents the target object with ID # N that has just recently appeared for 2 frames, and does not appear in the first 6 frames, but can also be represented by # N-0-0-0-0-0-0 or # N-0, but they all represent the initial case of target object ID # N. The third case represents the target with ID ═ N, which is present in the initial stage, absent in the middle, and present subsequently, and can also be represented by # N-0-0-0-0-0- # N or # N- # N- # N- # N- # N- # N-0- # N.
The marked parameters are sent to the full-connection network, the LSTM network and the classification network for training, and finally, the trained networks are obtained, so that the combination of the networks is suitable for recognizing key targets in various scenes.
Example four
Fig. 4a is a flowchart of a method for identifying a key target according to a fourth embodiment of the present invention. In this embodiment, each target object association description parameter included in the target object association description parameter set is grouped according to a preset classification rule to obtain a plurality of parameter groups, which specifically is: acquiring each object associated description parameter set matched with the running state of the object in the object associated description parameter set to construct a first parameter group; acquiring each target object associated description parameter set matched with the auxiliary detection attribute of the target object in the target object associated description parameter set to construct a second parameter group; acquiring all the object associated description parameter sets matched with the lane lines in the object associated description parameter set to construct a third parameter group; and acquiring each object associated description parameter set matched with the vehicle state of the vehicle in the object associated description parameter set to construct a fourth parameter group.
Inputting the target object association description parameter set into the LSTM long-short term memory network, specifically: before inputting the target object association description parameter set into the LSTM long-short term memory network, the method further includes: and normalizing or normalizing and expanding each object related description parameter included in the object related description parameter set.
As shown in fig. 4a, the method comprises the following specific steps:
and S410, acquiring the target object in the surrounding environment of the vehicle when the vehicle is in the automatic driving starting mode.
And S420, acquiring a target object association description parameter set corresponding to the target object.
And S430, normalizing or normalizing and expanding each object related description parameter in the object related description parameter set.
Wherein, normalization refers to an operation of ensuring that the inputted information can be normalized in the range of [ -1,1] area, improving the accuracy of the algorithm. Since some attributes are single values, the present implementation expands them at normalization. For example, the fusion state is a single attribute, and the value range thereof is 0,1, 2, and 3, and the total value is 4 isolated values. But when normalized, the corresponding [0,0,0], [0,0,1], [0,1,0], [1,0,0] are generated, totaling four vectors of 1 x 3, i.e. from one single input to one 3-dimensional input. Similarly, the class attribute of the object is changed from a single input end to a 5-dimensional input end.
S440, acquiring the object associated description parameter sets matched with the running state of the object in the object associated description parameter set to construct a first parameter group.
Optionally, the first parameter grouping includes: a lateral-longitudinal distance of the target from the vehicle, a lateral-longitudinal velocity of the target, and a fusion status of at least one sensor detecting the target.
S450, acquiring each target object associated description parameter set matched with the auxiliary detection attribute of the target object in the target object associated description parameter set to construct a second parameter group.
Optionally, the second parameter grouping includes: a radar reflection cross section, a fusion state of at least one sensor detecting the target object, and a category attribute of the target object.
And S460, acquiring each target object associated description parameter set matched with the lane line in the target object associated description parameter set to construct a third parameter group.
Optionally, the third parameter grouping includes: the distance from the target to the left and right lane lines and the effective extension distance of the left and right lane lines.
And S470, acquiring the object associated description parameter sets matched with the vehicle state of the vehicle in the object associated description parameter set to construct a fourth parameter group.
Optionally, the fourth parameter grouping includes: vehicle speed, vehicle yaw rate, and vehicle steering wheel angle.
And S480, parallelly inputting each parameter group into a full-connection network, and acquiring a global fusion abstract result output by the full-connection network aiming at the target object associated description parameter set.
And S490, inputting the global fusion abstract result of the target object into the LSTM network, and acquiring a plurality of long-term and short-term fusion results output by the LSTM network.
S4100, inputting each long-term and short-term fusion result into a classification network, and acquiring the key target object identification result output by the classification network.
Optionally, the classification network includes: the system comprises a first fully-connected node, a second fully-connected node and a softmax classifier, wherein the softmax classifier is respectively connected with the first fully-connected node and the second fully-connected node; and the first full-connection node and the second full-connection node are respectively connected with each output end of the LSTM network.
Optionally, the first full-connection node is configured to perform full-connection processing on each input long-term and short-term fusion result to obtain a first processing result; the second full-connection node is used for performing full-connection processing on each input long-term and short-term fusion result to obtain a second processing result; and the softmax classifier is used for obtaining an identification result of whether the target object is a key target object according to the first processing result and the second processing result.
Fig. 4b is a schematic diagram of an LSTM-based learning model structure according to an embodiment of the present invention. In the present embodiment, the target is located at a lateral-longitudinal distance (R) from the vehicleXAnd RY) The transverse and longitudinal speed (V) of the objectXAnd VY) And setting a fusion state (Fuse Status) of the at least one sensor that detected the target object as a first parameter group, setting a radar reflection section (RCS), a fusion state (Fuse Status) of the at least one sensor that detected the target object, and a Class attribute (Class) of the target object as a second parameter group, setting distances (Distance to Left and Distance to Right) of the target object and effective extension distances (Left Range and Right Range) of the Left and Right lane lines as a third parameter group, and setting a vehicle speed (Host Velocity), a vehicle Yaw Rate (Host Yaw Rate), and a vehicle Steering Wheel Angle (Host Steering Wheel Angle) as a fourth parameter group.
The LSTM-based learning model shown in fig. 4b is mainly composed of three major parts, where the first part is a fully connected network, the second part is an LSTM network, and the third part is a softmax and class output network. L is1-1To L1-17The first layer belonging to the fully-connected network part is mainly used for fully connecting data of an input layer. Wherein L is1-1To L1-6Fully connecting motion state information Rx, Ry, Vx and Vy of a target object with fusion state information Fuse Status; l is1-7To L1-11Fully connecting Class information Class of a target object, radar reflection cross section information RCS and fusion state information Fuse Status; l is1-12To L1-15Fully connecting lane line information; l is1-16To L1-17The motion state information of the main vehicle is fully connected. L is2-1To L2-10The second layer, which belongs to the part of the fully connected network, mainly abstracts the first layer network. Wherein L is2-1To L2-3Is to the upper layer network L1-1To L1-6Carrying out full connection; l is2-4To L2-6Is to the upper layer network L1-7To L1-11Carrying out full connection; l is2-7To L2-8Is to the upper layer network L1-12To L1-15Carrying out full connection; l is2-9To L2-10Is to the upper layer network L1-16To L1-17And carrying out full connection. L is3-1To L3-4The third layer, which belongs to the part of the fully-connected network, mainly abstracts the second layer network. Wherein L is3-1Is to the upper layer network L2-1To L2-3Abstracting; l is3-2Is to the upper layer network L2-4To L2-6Abstracting; l is3-3Is to the upper layer network L2-7To L2-8Abstracting; l is3-4Is to the upper layer network L2-9To L2-10And (6) abstracting. Then, L3-1To L3-4Form a vector, and send it into the long and short term memory network LSTM. The last part of the model is a first fully-connected node St-1 and a second fully connected node S t2, and sending the obtained first processing result and second processing result into the softmax layer. And finally, obtaining the identification result of whether the target object is a key target object.
The technical scheme of the embodiment of the invention can realize that engineering designers do not need to carry out complex algorithm design, and only the engineering personnel need to carry out data training according to the framework in the embodiment of the invention to obtain a very ideal target selection model, the method and the device have the advantages that the key target objects are identified for all the target objects existing around the vehicle, and the identification accuracy of the key target objects in the automatic driving system is improved.
EXAMPLE five
Fig. 5 is a flowchart of a preceding vehicle following method in an automatic driving start mode according to a fifth embodiment of the present invention, where this embodiment is applicable to a situation where a preceding vehicle is followed in the automatic driving start mode, and the method may be executed by a preceding vehicle following device in the automatic driving start mode, where the device may be implemented in a hardware and/or software manner, and may be generally integrated in a computer device, such as a terminal device or a server, having a preceding vehicle following function in the automatic driving start mode, where the method specifically includes the following steps:
and S510, when the vehicle enters a preceding vehicle following scene in the automatic driving starting mode, identifying the key target object in each target object appearing in the surrounding environment of the vehicle by adopting the identification method of the key target object in any embodiment.
And S520, taking the key target object as a following target, and performing matched vehicle speed following control on the vehicle according to the real-time driving speed of the following target.
For example, when the vehicle is in an automatic driving start mode, if the vehicle is going to go straight through a crossing following a preceding vehicle, a vehicle turns left to go between the vehicle and a following target object vehicle, at this time, a method for identifying a key target object can be used to identify the key target object vehicle in each target object appearing in the surrounding environment of the vehicle, and the vehicle is subjected to matched vehicle speed following control according to the real-time driving speed of the following vehicle, so that the vehicle can smoothly go straight through the crossing following the preceding vehicle.
According to the technical scheme, the problem of target object interference when the vehicle is followed in the automatic driving starting mode is solved by using the method for identifying the key target object, so that the vehicle can accurately identify the vehicle to be followed as soon as possible, and the adaptability of automatic driving is improved.
EXAMPLE six
Fig. 6 is a schematic structural diagram of an apparatus for identifying a key target according to a sixth embodiment of the present invention, which can execute the method for identifying a key target according to the foregoing embodiments. The device can be implemented in software and/or hardware, and as shown in fig. 6, the device for identifying a key target specifically includes: an object acquisition module 610, a parameter set acquisition module 620, and a result acquisition module 630.
The target object acquiring module 610 is configured to acquire a target object in an environment around a vehicle when the vehicle is in an automatic driving start mode;
a parameter set obtaining module 620, configured to obtain a target object association description parameter set corresponding to the target object;
the result obtaining module 630 is configured to input the target object association description parameter set into an LSTM network, so as to obtain a key target object identification result corresponding to the target object.
According to the technical scheme of the embodiment of the invention, an effective method for obtaining the target selection function applicable to the automatic driving system is creatively provided by a technical means of inputting the target object associated description parameter set corresponding to the target object in the vehicle surrounding environment in the automatic driving mode into the LSTM network and obtaining the key target object identification result corresponding to the target object according to the output result of the LSTM network, so that the method can be well applicable to various sensors and various environments where the automatic driving system is located, a specific target selection algorithm is not required to be designed, the method is simple and convenient to realize, and great improvement is brought to the target selection performance of the automatic driving system.
Optionally, the result obtaining module 630 may specifically include a global fusion abstract result obtaining unit and a result obtaining unit;
the global fusion abstract result obtaining unit is configured to input a target object associated description parameter set of the target object into a fully connected network, and obtain a global fusion abstract result output by the fully connected network for the target object associated description parameter set;
and the result acquisition unit is used for inputting the global fusion abstract result of the target object into the LSTM network to obtain a key target object identification result corresponding to the target object.
Optionally, the global fusion abstraction result obtaining unit may include a parameter grouping subunit, configured to group, according to a preset classification rule, each target object association description parameter included in the target object association description parameter set, to obtain a plurality of parameter groups, and input each parameter group to the full-connection network in parallel.
Optionally, the result obtaining unit may be specifically configured to input the global fusion abstract result of the target object to the LSTM network, and obtain a plurality of long-term and short-term fusion results output by the LSTM network; and inputting each long-term and short-term fusion result into a classification network, and acquiring the key target object identification result output by the classification network.
Optionally, the fully connected network specifically includes: the system comprises a plurality of local sub-networks and a unified full-connection layer respectively connected with the local sub-networks, wherein different local sub-networks are used for inputting different parameter groups; the local sub-network is used for carrying out at least one stage of full connection processing on each target object associated description parameter in the input parameter grouping to obtain a local fusion abstract result; and the uniform full-connection layer is used for performing full-connection processing on the local fusion abstract results output by each local sub-network to obtain the global fusion abstract results.
Optionally, the local sub-network includes: the system comprises a plurality of sequentially connected local full-connection layers, wherein each local full-connection layer comprises a plurality of full-connection nodes; each full-link node in the next full-link layer is connected with each full-link node in the previous full-link layer, and each target object associated description parameter included in the parameter grouping is input into each full-link node in the first full-link layer.
Optionally, the classification network includes: the system comprises a first fully-connected node, a second fully-connected node and a softmax classifier, wherein the softmax classifier is respectively connected with the first fully-connected node and the second fully-connected node; and the first full-connection node and the second full-connection node are respectively connected with each output end of the LSTM network.
Optionally, the first full-connection node is configured to perform full-connection processing on each input long-term and short-term fusion result to obtain a first processing result; the second full-connection node is used for performing full-connection processing on each input long-term and short-term fusion result to obtain a second processing result; and the softmax classifier is used for obtaining an identification result of whether the target object is a key target object according to the first processing result and the second processing result.
Optionally, the parameter grouping subunit may be specifically configured to, in the target object association description parameter set, obtain each target object association description parameter set that matches the operating state of the target object to construct a first parameter group; acquiring each target object associated description parameter set matched with the auxiliary detection attribute of the target object in the target object associated description parameter set to construct a second parameter group; acquiring all the object associated description parameter sets matched with the lane lines in the object associated description parameter set to construct a third parameter group; and acquiring each object associated description parameter set matched with the vehicle state of the vehicle in the object associated description parameter set to construct a fourth parameter group.
Optionally, the first parameter grouping includes: the lateral-longitudinal distance of the target from the vehicle, the lateral-longitudinal speed of the target and the fusion state of at least one sensor for detecting the target; the second parameter packet includes: the radar reflection cross section, the fusion state of at least one sensor for detecting the target object and the class attribute of the target object; the third parameter packet includes: the distance from the target object to the left lane line and the right lane line and the effective extension distance of the left lane line and the right lane line; the fourth parameter packet includes: vehicle speed, vehicle yaw rate, and vehicle steering wheel angle.
Optionally, the identification apparatus for a key object may further include a normalization or normalization extension module, configured to normalize or normalize and extend each object related description parameter included in the object related description parameter set before inputting the object related description parameter set into the LSTM long-short term memory network.
The identification device of the key target object provided by the embodiment of the invention can execute the identification method of the key target object provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE seven
Fig. 7 is a schematic structural diagram of a preceding vehicle following device in an automatic driving start mode according to a seventh embodiment of the present invention, where the device can execute the preceding vehicle following method in the automatic driving start mode according to the foregoing embodiments. The device can be implemented in software and/or hardware, and as shown in fig. 7, the preceding vehicle following device in the automatic driving start mode specifically includes: a key target identification module 710, a vehicle speed control module 720.
The key target object identification module 710 is configured to identify, when the vehicle enters a preceding vehicle following scene in the automatic driving start mode, a key target object among target objects appearing in the surrounding environment of the vehicle by using the identification method for the key target object described in any one of the embodiments above;
and the vehicle speed control module 720 is used for taking the key target object as a following target and performing matched vehicle speed following control on the vehicle according to the real-time driving speed of the following target.
According to the technical scheme, the problem of target object interference when the vehicle is followed in the automatic driving starting mode is solved by using the method for identifying the key target object, so that the vehicle can accurately identify the vehicle to be followed as soon as possible, and the adaptability of automatic driving is improved.
The front vehicle following device in the automatic driving starting mode provided by the embodiment of the invention can execute the front vehicle following method in the automatic driving starting mode provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
Example eight
Fig. 8 is a schematic structural diagram of a computer apparatus according to an eighth embodiment of the present invention, as shown in fig. 8, the computer apparatus includes a processor 810, a memory 820, an input device 830, and an output device 840; the number of the processors 810 in the computer device may be one or more, and one processor 810 is taken as an example in fig. 8; the processor 810, the memory 820, the input device 830 and the output device 840 in the computer apparatus may be connected by a bus or other means, and fig. 8 illustrates the connection by a bus as an example.
The memory 820 may be used as a computer-readable storage medium for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the identification method of the key object in the embodiment of the present invention (for example, the object acquisition module 610, the parameter set acquisition module 620, and the result acquisition module 630 in the identification device of the key object) or program instructions/modules corresponding to the preceding vehicle following method in the automatic driving start mode in the embodiment of the present invention (for example, the key object identification module 710 and the vehicle speed control module 720 in the preceding vehicle following device in the automatic driving start mode). The processor 810 executes various functional applications of the computer device and data processing, i.e., the above-described recognition method of a key object or the preceding vehicle following method in the automatic driving start mode, by executing software programs, instructions, and modules stored in the memory 620.
The method for identifying the key target object comprises the following steps:
acquiring a target object in the surrounding environment of the vehicle when the vehicle is in an automatic driving starting mode;
acquiring a target object association description parameter set corresponding to the target object;
and inputting the target object association description parameter set into an LSTM network to obtain a key target object identification result corresponding to the target object.
The preceding vehicle following method in the automatic driving starting mode comprises the following steps:
when a vehicle enters a preceding vehicle following scene in an automatic driving starting mode, identifying and obtaining a key target object in each target object appearing in the surrounding environment of the vehicle by adopting the method of any one of claims 1 to 11;
and taking the key target object as a following target, and carrying out matched vehicle speed following control on the vehicle according to the real-time driving speed of the following target.
The memory 820 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 820 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 820 may further include memory located remotely from the processor 810, which may be connected to a computer device through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 830 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the computer apparatus. The output device 840 may include a display device such as a display screen.
Example nine
Embodiments of the present invention also provide a storage medium containing computer-executable instructions that, when executed by a computer processor, perform a method of identifying a key target or a method of preceding vehicle following in an automatic driving initiation mode.
The method for identifying the key target object comprises the following steps:
acquiring a target object in the surrounding environment of the vehicle when the vehicle is in an automatic driving starting mode;
acquiring a target object association description parameter set corresponding to the target object;
and inputting the target object association description parameter set into an LSTM network to obtain a key target object identification result corresponding to the target object.
The preceding vehicle following method in the automatic driving starting mode comprises the following steps:
when a vehicle enters a preceding vehicle following scene in an automatic driving starting mode, identifying and obtaining a key target object in each target object appearing in the surrounding environment of the vehicle by adopting the method of any one of claims 1 to 11;
and taking the key target object as a following target, and carrying out matched vehicle speed following control on the vehicle according to the real-time driving speed of the following target.
Of course, the storage medium provided by the embodiment of the present invention contains computer executable instructions, and the computer executable instructions are not limited to the operations of the method described above, and may also execute the relevant operations in the method for identifying a key target object or the method for following a preceding vehicle in the automatic driving start mode provided by any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the above embodiment of the recognition device for the key target object or the preceding vehicle following device in the automatic driving start mode, the included units and modules are only divided according to the functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims (16)

1. A method for identifying a key target object, comprising:
acquiring a target object in the surrounding environment of the vehicle when the vehicle is in an automatic driving starting mode;
acquiring a target object association description parameter set corresponding to the target object;
and inputting the target object association description parameter set into an LSTM network to obtain a key target object identification result corresponding to the target object.
2. The method of claim 1, wherein inputting the set of object association description parameters into an LSTM network to obtain a key object identification result corresponding to the object comprises:
inputting the target object associated description parameter set of the target object into a full-connection network, and acquiring a global fusion abstract result output by the full-connection network aiming at the target object associated description parameter set;
and inputting the global fusion abstract result of the target object into the LSTM network to obtain a key target object identification result corresponding to the target object.
3. The method of claim 2, wherein inputting the set of object association description parameters for the object into the fully connected network comprises:
and grouping the object associated description parameters included in the object associated description parameter set according to a preset classification rule to obtain a plurality of parameter groups, and inputting the parameter groups into a full-connection network in parallel.
4. The method of claim 2, wherein inputting the global fusion abstraction result of the object into the LSTM network to obtain a key object recognition result corresponding to the object comprises:
inputting the global fusion abstract result of the target object into the LSTM network to obtain a plurality of long-term and short-term fusion results output by the LSTM network;
and inputting each long-term and short-term fusion result into a classification network, and acquiring the key target object identification result output by the classification network.
5. The method according to claim 4, wherein the fully connected network comprises in particular: the system comprises a plurality of local sub-networks and a unified full-connection layer respectively connected with the local sub-networks, wherein different local sub-networks are used for inputting different parameter groups;
the local sub-network is used for carrying out at least one stage of full connection processing on each target object associated description parameter in the input parameter grouping to obtain a local fusion abstract result;
and the uniform full-connection layer is used for performing full-connection processing on the local fusion abstract results output by each local sub-network to obtain the global fusion abstract results.
6. The method of claim 5, wherein the local subnetwork comprises: the system comprises a plurality of sequentially connected local full-connection layers, wherein each local full-connection layer comprises a plurality of full-connection nodes;
each full-link node in the next full-link layer is connected with each full-link node in the previous full-link layer, and each target object associated description parameter included in the parameter grouping is input into each full-link node in the first full-link layer.
7. The method of claim 4, wherein the classification network comprises: the system comprises a first fully-connected node, a second fully-connected node and a softmax classifier, wherein the softmax classifier is respectively connected with the first fully-connected node and the second fully-connected node; and the first full-connection node and the second full-connection node are respectively connected with each output end of the LSTM network.
8. The method according to claim 7, wherein the first fully-connected node is configured to perform fully-connected processing on each input long-term and short-term fusion result to obtain a first processing result;
the second full-connection node is used for performing full-connection processing on each input long-term and short-term fusion result to obtain a second processing result;
and the softmax classifier is used for obtaining an identification result of whether the target object is a key target object according to the first processing result and the second processing result.
9. The method according to claim 3, wherein grouping the object associated description parameters included in the object associated description parameter set according to a preset classification rule to obtain a plurality of parameter groups comprises:
acquiring each object associated description parameter set matched with the running state of the object in the object associated description parameter set to construct a first parameter group;
acquiring each target object associated description parameter set matched with the auxiliary detection attribute of the target object in the target object associated description parameter set to construct a second parameter group;
acquiring all the object associated description parameter sets matched with the lane lines in the object associated description parameter set to construct a third parameter group;
and acquiring each object associated description parameter set matched with the vehicle state of the vehicle in the object associated description parameter set to construct a fourth parameter group.
10. The method of claim 9, wherein the first grouping of parameters comprises: the lateral-longitudinal distance of the target from the vehicle, the lateral-longitudinal speed of the target and the fusion state of at least one sensor for detecting the target;
the second parameter packet includes: the radar reflection cross section, the fusion state of at least one sensor for detecting the target object and the class attribute of the target object;
the third parameter packet includes: the distance from the target object to the left lane line and the right lane line and the effective extension distance of the left lane line and the right lane line;
the fourth parameter packet includes: vehicle speed, vehicle yaw rate, and vehicle steering wheel angle.
11. The method of claim 1, further comprising, prior to inputting the set of object association description parameters into the LSTM long-short term memory network:
and normalizing or normalizing and expanding each object related description parameter included in the object related description parameter set.
12. A preceding vehicle following method in an automatic driving start mode, comprising:
when a vehicle enters a preceding vehicle following scene in an automatic driving starting mode, identifying and obtaining a key target object in each target object appearing in the surrounding environment of the vehicle by adopting the method of any one of claims 1 to 11;
and taking the key target object as a following target, and carrying out matched vehicle speed following control on the vehicle according to the real-time driving speed of the following target.
13. An apparatus for identifying a key object, comprising:
the system comprises a target object acquisition module, a target object acquisition module and a target object acquisition module, wherein the target object acquisition module is used for acquiring a target object in the surrounding environment of the vehicle when the vehicle is in an automatic driving starting mode;
a parameter set obtaining module, configured to obtain a target object association description parameter set corresponding to the target object;
and the result acquisition module is used for inputting the target object association description parameter set into the LSTM network to obtain a key target object identification result corresponding to the target object.
14. A leading vehicle following device in an automatic driving start mode, comprising:
a key target object identification module, configured to identify a key target object among target objects appearing in an environment around a vehicle by using the method according to any one of claims 1 to 11 when the vehicle enters a preceding vehicle following scene in an automatic driving start mode;
and the vehicle speed control module is used for taking the key target object as a following target and carrying out matched vehicle speed following control on the vehicle according to the real-time driving speed of the following target.
15. A computer device, characterized in that the computer device comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement a method of identifying a key object as recited in any of claims 1-11 or a method of preceding vehicle following in an autopilot mode as recited in claim 12.
16. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out a method for identifying a key object as claimed in any one of claims 1 to 11 or a method for preceding vehicle following in an automatic driving start mode as claimed in claim 12.
CN202110944797.0A 2021-08-17 2021-08-17 Method and device for identifying key target object, computer equipment and storage medium Active CN113673412B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110944797.0A CN113673412B (en) 2021-08-17 2021-08-17 Method and device for identifying key target object, computer equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110944797.0A CN113673412B (en) 2021-08-17 2021-08-17 Method and device for identifying key target object, computer equipment and storage medium

Publications (2)

Publication Number Publication Date
CN113673412A true CN113673412A (en) 2021-11-19
CN113673412B CN113673412B (en) 2023-09-26

Family

ID=78543360

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110944797.0A Active CN113673412B (en) 2021-08-17 2021-08-17 Method and device for identifying key target object, computer equipment and storage medium

Country Status (1)

Country Link
CN (1) CN113673412B (en)

Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006092395A (en) * 2004-09-27 2006-04-06 Satoshi Izumoya Automatic control system of traffic regulation information
CN108172025A (en) * 2018-01-30 2018-06-15 东软集团股份有限公司 A kind of auxiliary driving method, device, car-mounted terminal and vehicle
CN108229292A (en) * 2017-07-28 2018-06-29 北京市商汤科技开发有限公司 target identification method, device, storage medium and electronic equipment
US20180275667A1 (en) * 2017-03-27 2018-09-27 Uber Technologies, Inc. Machine Learning for Event Detection and Classification in Autonomous Vehicles
CN109635793A (en) * 2019-01-31 2019-04-16 南京邮电大学 A kind of unmanned pedestrian track prediction technique based on convolutional neural networks
CN110021165A (en) * 2019-03-18 2019-07-16 浙江工业大学 A kind of traffic flow forecasting method based on Autoencoder-LSTM Fusion Model
US20200042776A1 (en) * 2018-08-03 2020-02-06 Baidu Online Network Technology (Beijing) Co., Ltd. Method and apparatus for recognizing body movement
CN111079590A (en) * 2019-12-04 2020-04-28 东北大学 Peripheral vehicle behavior pre-judging method of unmanned vehicle
CN111310583A (en) * 2020-01-19 2020-06-19 中国科学院重庆绿色智能技术研究院 Vehicle abnormal behavior identification method based on improved long-term and short-term memory network
US20200324794A1 (en) * 2020-06-25 2020-10-15 Intel Corporation Technology to apply driving norms for automated vehicle behavior prediction
CN111860269A (en) * 2020-07-13 2020-10-30 南京航空航天大学 Multi-feature fusion tandem RNN structure and pedestrian prediction method
CN112085165A (en) * 2020-09-02 2020-12-15 中国第一汽车股份有限公司 Decision information generation method, device, equipment and storage medium
CN112767682A (en) * 2020-12-18 2021-05-07 南京航空航天大学 Multi-scale traffic flow prediction method based on graph convolution neural network
CN112884742A (en) * 2021-02-22 2021-06-01 山西讯龙科技有限公司 Multi-algorithm fusion-based multi-target real-time detection, identification and tracking method
CN112937603A (en) * 2019-12-10 2021-06-11 三星电子株式会社 System and method for predicting position of target vehicle
CN113139446A (en) * 2021-04-12 2021-07-20 长安大学 End-to-end automatic driving behavior decision method, system and terminal equipment

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006092395A (en) * 2004-09-27 2006-04-06 Satoshi Izumoya Automatic control system of traffic regulation information
US20180275667A1 (en) * 2017-03-27 2018-09-27 Uber Technologies, Inc. Machine Learning for Event Detection and Classification in Autonomous Vehicles
CN108229292A (en) * 2017-07-28 2018-06-29 北京市商汤科技开发有限公司 target identification method, device, storage medium and electronic equipment
CN108172025A (en) * 2018-01-30 2018-06-15 东软集团股份有限公司 A kind of auxiliary driving method, device, car-mounted terminal and vehicle
US20200042776A1 (en) * 2018-08-03 2020-02-06 Baidu Online Network Technology (Beijing) Co., Ltd. Method and apparatus for recognizing body movement
CN109635793A (en) * 2019-01-31 2019-04-16 南京邮电大学 A kind of unmanned pedestrian track prediction technique based on convolutional neural networks
CN110021165A (en) * 2019-03-18 2019-07-16 浙江工业大学 A kind of traffic flow forecasting method based on Autoencoder-LSTM Fusion Model
CN111079590A (en) * 2019-12-04 2020-04-28 东北大学 Peripheral vehicle behavior pre-judging method of unmanned vehicle
CN112937603A (en) * 2019-12-10 2021-06-11 三星电子株式会社 System and method for predicting position of target vehicle
CN111310583A (en) * 2020-01-19 2020-06-19 中国科学院重庆绿色智能技术研究院 Vehicle abnormal behavior identification method based on improved long-term and short-term memory network
US20200324794A1 (en) * 2020-06-25 2020-10-15 Intel Corporation Technology to apply driving norms for automated vehicle behavior prediction
CN111860269A (en) * 2020-07-13 2020-10-30 南京航空航天大学 Multi-feature fusion tandem RNN structure and pedestrian prediction method
CN112085165A (en) * 2020-09-02 2020-12-15 中国第一汽车股份有限公司 Decision information generation method, device, equipment and storage medium
CN112767682A (en) * 2020-12-18 2021-05-07 南京航空航天大学 Multi-scale traffic flow prediction method based on graph convolution neural network
CN112884742A (en) * 2021-02-22 2021-06-01 山西讯龙科技有限公司 Multi-algorithm fusion-based multi-target real-time detection, identification and tracking method
CN113139446A (en) * 2021-04-12 2021-07-20 长安大学 End-to-end automatic driving behavior decision method, system and terminal equipment

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
张燕咏;张莎;张昱;吉建民;段逸凡;黄奕桐;彭杰;张宇翔;: "基于多模态融合的自动驾驶感知及计算", 计算机研究与发展, no. 09 *
李为斌;刘佳;: "基于视觉的动态手势识别概述", 计算机应用与软件, no. 03 *

Also Published As

Publication number Publication date
CN113673412B (en) 2023-09-26

Similar Documents

Publication Publication Date Title
Chen et al. Attention-based hierarchical deep reinforcement learning for lane change behaviors in autonomous driving
Fang et al. 3d-siamrpn: An end-to-end learning method for real-time 3d single object tracking using raw point cloud
Pizzati et al. Enhanced free space detection in multiple lanes based on single CNN with scene identification
Chen et al. Driving maneuvers prediction based autonomous driving control by deep Monte Carlo tree search
CN110781949B (en) Asynchronous serial multi-sensor-based flight path data fusion method and storage medium
CN111161315A (en) Multi-target tracking method and system based on graph neural network
US11472444B2 (en) Method and system for dynamically updating an environmental representation of an autonomous agent
Ji et al. A method for LSTM-based trajectory modeling and abnormal trajectory detection
CN114913386A (en) Training method of multi-target tracking model and multi-target tracking method
Yu et al. Deep temporal model-based identity-aware hand detection for space human–robot interaction
CN115860102B (en) Pre-training method, device, equipment and medium for automatic driving perception model
CN115862136A (en) Lightweight filler behavior identification method and device based on skeleton joint
CN113740837A (en) Obstacle tracking method, device, equipment and storage medium
CN114913206A (en) Multi-target tracking method and system based on multi-mode fusion
JP2018010568A (en) Image recognition system
Nakamura et al. An effective combination of loss gradients for multi-task learning applied on instance segmentation and depth estimation
CN112651294A (en) Method for recognizing human body shielding posture based on multi-scale fusion
CN116975781A (en) Automatic driving vehicle behavior decision system and method
CN113673412A (en) Key target object identification method and device, computer equipment and storage medium
Lu et al. Hybrid deep learning based moving object detection via motion prediction
Shukla et al. UBOL: User-Behavior-aware one-shot learning for safe autonomous driving
She et al. Multi-obstacle detection based on monocular vision for UAV
CN115014366A (en) Target fusion method and device, vehicle and storage medium
CN113762043A (en) Abnormal track identification method and device
Li et al. Smart IoT-based visual target enabled track and field training using image recognition

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant