CN111341102A - Motion primitive library construction method and device and motion primitive connection method and device - Google Patents
Motion primitive library construction method and device and motion primitive connection method and device Download PDFInfo
- Publication number
- CN111341102A CN111341102A CN202010136047.6A CN202010136047A CN111341102A CN 111341102 A CN111341102 A CN 111341102A CN 202010136047 A CN202010136047 A CN 202010136047A CN 111341102 A CN111341102 A CN 111341102A
- Authority
- CN
- China
- Prior art keywords
- data
- over
- segmentation
- calculating
- characteristic
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0125—Traffic data processing
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
- G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
Abstract
The specification provides a construction method and a device of a driving-like motion primitive library and a method and a device for connecting motion primitives, wherein the construction method comprises the following steps: acquiring vehicle characteristic data of each sampling moment under the condition of driving by a person; the driving data comprises course characteristic data, speed characteristic data and position characteristic data; determining the heading change zero crossing point of the vehicle, and segmenting the vehicle characteristic data to obtain an over-segmentation data segment; calculating an attribute feature set corresponding to each over-segmentation data segment according to the speed feature data and the position feature data of each over-segmentation data segment; and screening the over-segmentation data segments serving as the moving primitives in the moving primitive library and the corresponding attribute feature sets by adopting an expected maximum algorithm based on the attribute feature sets corresponding to all the over-segmentation data segments. The automatic driving path constructed by the motion primitives in the motion primitive library can better meet the riding experience of passengers.
Description
Technical Field
The invention relates to the technical field of automatic driving, in particular to a method and a device for constructing a motion primitive library and a method and a device for connecting motion primitives.
Background
In the field of automatic driving, an automatic driving method mainly applied at present plans a feasible driving track and operation characteristics of a vehicle based on environmental characteristic data and vehicle operation characteristic data detected by a vehicle sensor, and selects a certain feasible driving track and motion characteristics as a final driving track according to driving safety and high efficiency requirements. Because the current automatic driving control method only plans the driving track and the motion characteristic according to the environmental characteristic data, the safety requirement and the high efficiency requirement, the adaptability of the personnel in the vehicle is not considered.
Disclosure of Invention
The construction method and the device of the human-like driving element library are used for extracting motion elements forming the motion element library according to vehicle characteristic data; in addition, the application also provides a method and a device for connecting the motion primitives.
On one hand, the application provides a construction method of a human-like driving motion primitive library, which comprises the following steps:
acquiring vehicle characteristic data of each sampling moment under the condition of driving by a person; the driving data comprises course characteristic data, speed characteristic data and position characteristic data;
determining the heading change zero crossing point of the vehicle running according to the heading characteristic data at each sampling moment; dividing the vehicle characteristic data by using the course change zero crossing point as an over-division point to obtain an over-division data segment;
calculating an attribute feature set corresponding to each over-segmentation data segment according to the speed feature data and the position feature data of each over-segmentation data segment;
and screening the over-segmentation data segments serving as the moving primitives in the moving primitive library and the corresponding attribute feature sets by adopting an expected maximum algorithm based on the attribute feature sets corresponding to all the over-segmentation data segments.
Optionally, determining an attribute feature set corresponding to each over-segmented data segment according to the speed feature data and the position feature data in each over-segmented data segment, where the attribute feature set includes:
constructing an attenuation function and a Gaussian kernel function;
calculating deviation data according to the speed characteristic data and the position characteristic data of the over-segmentation data segment and the set critical damping system data;
calculating a weight coefficient corresponding to each Gaussian kernel function by adopting a local weighted regression algorithm according to the deviation data, the attenuation function and the Gaussian kernel function;
and constructing the attribute feature set by adopting the starting point speed of the over-segmentation data segment and the weight coefficient corresponding to each Gaussian kernel function.
Optionally, the attenuation function is:
wherein αzT is the duration of the over-segmentation data segment and delta T is the sampling interval of the sampling moment.
In another aspect, the present specification further provides a method for connecting a motion primitive selected from the aforementioned human-like driving motion primitive library; the method comprises the following steps:
calculating a total duration from the durations of each selected motion primitive;
calculating an attenuation function using the total duration;
respectively calculating the Gaussian kernel center and the kernel distribution amplitude of each selected motion element according to the duration and the total duration of each selected motion element, constructing a Gaussian kernel center sequence according to the Gaussian kernel centers of all selected motion elements, and constructing a kernel distribution amplitude sequence according to the kernel distribution amplitudes of all selected motion elements; constructing a primitive track characteristic sequence according to the weight coefficients in the attribute characteristic set of all the selected motion primitives;
calculating a nonlinear part corresponding to each moment point according to the attenuation function, the Gaussian kernel center sequence, the kernel distribution amplitude sequence and the element track characteristic sequence, and converting the nonlinear part into a global coordinate system;
calculating a critical damping system part corresponding to each moment point by adopting the speed characteristic data of the selected motion element and the speed parameter of the selected motion element; and summing the nonlinear part and the critical damping system to obtain the track coordinate corresponding to each time point.
Optionally, converting the non-linear part into a global coordinate system includes:
according to the included angle between the starting point speed of each selected motion element and the coordinate axis of the global coordinate system;
and adjusting the nonlinear part according to the included angle, and converting the nonlinear part into a global coordinate system.
In one aspect, the present specification provides a construction apparatus of a human-like driving motion primitive library, including:
the data acquisition unit is used for acquiring vehicle characteristic data at each sampling moment under the condition of driving by a person; the driving data comprises course characteristic data, speed characteristic data and position characteristic data;
the segmentation unit is used for determining the heading change zero crossing point of the vehicle running according to the heading characteristic data at each sampling moment; dividing the vehicle characteristic data by using the course change zero crossing point as an over-division point to obtain an over-division data segment;
the feature set construction unit is used for calculating an attribute feature set corresponding to each over-segmentation data segment according to the speed feature data and the position feature data of each over-segmentation data segment;
and the motion element screening unit is used for screening the over-segmentation data segments serving as the motion elements in the motion element library and the corresponding attribute feature sets by adopting an expected maximum algorithm based on the attribute feature sets corresponding to all the over-segmentation data segments.
Optionally, the feature set constructing unit includes:
the function constructing subunit is used for constructing an attenuation function and a Gaussian kernel function;
the deviation data calculating subunit is used for calculating deviation data according to the speed characteristic data and the position characteristic data of the over-segmentation data segment and set critical damping system data;
the weight coefficient calculation unit is used for calculating the weight coefficient corresponding to each Gaussian kernel function by adopting a local weighted regression algorithm according to the deviation data, the attenuation function and the Gaussian kernel function;
and the characteristic set constructing subunit is used for constructing the attribute characteristic set by adopting the starting point speed of the over-segmentation data segment and the weight coefficient corresponding to each Gaussian kernel function.
Optionally, the attenuation function is:
wherein αzT is the duration of the over-segmentation data segment and delta T is the sampling interval of the sampling moment.
In yet another aspect, the present specification provides a method of connecting selected motion primitives from the aforementioned library of human-like driving motion primitives; the method comprises the following steps:
a time calculation unit calculating a total duration from the durations of each selected motion primitive;
an attenuation function calculation unit that calculates an attenuation function using the total duration;
the mixed calculation unit is used for respectively calculating the Gaussian kernel center and the kernel distribution amplitude of each selected motion element according to the duration and the total duration of each selected motion element, constructing a Gaussian kernel center sequence according to the Gaussian kernel centers of all selected motion elements, and constructing a kernel distribution amplitude sequence according to the kernel distribution amplitudes of all selected motion elements; constructing a primitive track characteristic sequence according to the weight coefficients in the attribute characteristic set of all the selected motion primitives;
the nonlinear part calculating unit calculates a nonlinear part corresponding to each moment point according to the attenuation function, the Gaussian kernel center sequence, the kernel distribution amplitude sequence and the element track characteristic sequence, and converts the nonlinear part into a global coordinate system;
the combination unit is used for calculating a critical damping system part corresponding to each moment point by adopting the speed characteristic data of the selected motion element and the speed parameter of the selected motion element; and summing the nonlinear part and the critical damping system to obtain the track coordinate corresponding to each time point.
Optionally, the nonlinear part calculating unit converts the nonlinear part into a global coordinate system, and includes:
according to the included angle between the starting point speed of each selected motion element and the coordinate axis of the global coordinate system;
and adjusting the nonlinear part according to the included angle, and converting the nonlinear part into a global coordinate system.
The construction method and the construction device for the human-like driving motion primitive database provided by the specification divide complex driving behaviors into over-segmentation data segments, realize the characterization of the characteristics of each over-segmentation data segment by using a Gaussian mixture model, and form an attribute characteristic set corresponding to each over-segmentation data segment; the representation method of each over-segmentation data segment has good generalization capability on the basis of representing the driving behavior with certain precision. And then, adopting an expectation maximization algorithm, extracting from the over-segmentation data segments without labels based on interdependencies of element characterization and feature extraction, and determining the over-segmentation data segments which can be used as a motion element library and a corresponding data feature set.
In the invention, the technical schemes can be combined with each other to realize more preferable combination schemes. Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, wherein like reference numerals are used to designate like parts throughout.
FIG. 1 is a flowchart of a method for constructing a human-like driving movement primitive library provided by an embodiment;
FIG. 2 is a flowchart of a method for concatenating motion primitives provided by an embodiment;
FIG. 3 is a schematic diagram of a construction device of a human-like driving movement primitive library provided by an embodiment;
FIG. 4 is a diagram of an apparatus for connecting motion primitives according to an embodiment;
wherein: 11-a data acquisition unit, 12-a segmentation unit, 13-a feature set construction unit, 14-a motion element screening unit, 21-a time calculation unit, 22-an attenuation function calculation unit, 23-a mixing calculation unit, 24-a nonlinear part calculation unit and 25-a combination unit.
Detailed Description
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate preferred embodiments of the invention and together with the description, serve to explain the principles of the invention and not to limit the scope of the invention.
The embodiment of the invention provides a construction method of a human-like driving motion primitive library. FIG. 1 is a flowchart of a method for constructing a human-like driving movement primitive library according to an embodiment. As shown in fig. 1, the method provided by the present embodiment includes steps S101-S104.
S101: and acquiring vehicle characteristic data of each sampling moment under the condition of driving by a person.
Step S101 is a data sampling process. In specific application, the sensor for collecting the vehicle characteristic data can be installed in a vehicle driven by a person, so that the vehicle driven by the person is operated by a driver, the speed and the like meet the requirements of typical scenes, and the vehicle can run under the conditions of typical road conditions such as sharp turning, lane changing, normal running, overtaking and the like to collect the vehicle characteristic data.
In practical application, the vehicle characteristic data is represented in a data set mode, and a data segment of the whole sampling moment is formed. Each point in the data set corresponds to a respective sampling time point. The data of each sampling time point comprises course characteristic data, speed characteristic data and position characteristic data.
S102: determining the heading change zero crossing point of the vehicle running according to the heading characteristic data at each sampling moment; and dividing the vehicle characteristic data by using the course change zero crossing point as a dividing point to obtain an over-divided data segment.
As described above, the aforementioned vehicle characteristic data is embodied as travel of a data set, and the data set includes travel data of various states; therefore, the vehicle characteristic data needs to be reasonably divided to form data segments for subsequent various driving state characteristic extraction.
Because the vehicle running is a continuous process, the vehicle characteristic data at a single sampling time cannot completely reflect the characteristics of a certain vehicle running state, so the vehicle characteristic data at a single sampling time is not adopted as a data segment in the embodiment. It is not reasonable to adopt the vehicle characteristic data corresponding to the vehicle running state for a long time as a data piece data amount for subsequent calculation, which may last for a long time.
In the embodiment of the present specification, considering that a driver needs to operate a steering wheel (rudder) at any time during the vehicle driving process to realize the operation of the vehicle driving direction, and the difference of the vehicle driving state can be reflected by the difference of the operation action of the steering wheel, so that the segmentation of the vehicle characteristic data can be realized based on the operation of the steering wheel by the driver.
The operation of the steering wheel by the driver is directly embodied as the heading characteristic data, so that the vehicle characteristic data can be divided according to the heading characteristic data. And because the typical data point in the course characteristic data is the zero crossing point of the course change deviation, the time point corresponding to the course change zero crossing point can be used as a segmentation point to segment the vehicle characteristic data to obtain a segmentation data segment.
As described above, since some pieces of vehicle characteristic data in the same state may be divided into a plurality of pieces, the pieces of data formed by the aforementioned division are over-divided pieces of data. Also, since the running characteristics of the vehicles are the same in the same state, it can be assumed that the over-divided data pieces have the same characteristics in the same state.
S103: and calculating the attribute feature set corresponding to each over-segmentation data segment according to the speed feature data and the position feature data in each over-segmentation data segment.
Step S103 is a process of performing feature extraction on the feature data segment captured in step S102 to form a corresponding attribute feature set.
Through analysis, the over-segmentation data segment can be characterized by a typical spring damping system and a nonlinear function. In the case of reasonably setting the parameters of the spring damping system so that the spring damping system can be converged quickly and stably (namely, the spring damping system is a critical damping system), the characterization core of the over-segmentation data segment is to determine the nonlinear function part.
Therefore, the embodiment of the specification expresses each over-segmentation data segment by using a critical damping system with the same characteristics and a corresponding nonlinear function; accordingly, the core of the representation of each over-segmented data segment is to determine the features that can represent the non-linear function portion.
In one embodiment, the nonlinear function portion is constructed using a Gaussian mixture model. Determining the attribute feature set corresponding to each over-segmented data segment according to the speed feature data and the position feature data in the over-segmented data segment may adopt steps S1031 to S1034.
S1031: an attenuation function and a gaussian kernel function are constructed.
In order to enable the nonlinear function representing each over-segmented data segment to have a convergence characteristic and enable the nonlinear function representing each over-segmented data segment to reasonably converge to an end point of the over-segmented data segment when the over-segmented data segment is represented by adopting a Gaussian mixture model, an attenuation function is set in the embodiment of the specification. In one particular application, the attenuation function may be a function expressed by equation one.
α in formula IzFor preset parameters, by setting α appropriatelyzThe attenuation function can be enabled to have better convergence characteristics; t is the duration of the over-segmentation data segments, T is the duration corresponding to each over-segmentation data segment, and delta T is the sampling time interval. In other embodiments, the attenuation function may be other types of functions as long as they have reasonable convergence characteristics, so that the model formed by combining the gaussian mixture model and the critical damping system model can converge to the end point of the over-segmented data segment when the over-segmented data segment is characterized.
In this embodiment, the gaussian kernel function can be represented by formula two.
In the second formula, μnCharacterizing the position parameter, p, of the Gaussian kernelnThe distribution amplitude of the kernel is represented, tau is a time scale parameter and is used for the telescopic adjustment of a time element on a time scale, and T is the duration of the over-segmentation data segment.
By adopting the attenuation function and the Gaussian kernel function, the nonlinear function corresponding to each over-segmentation data segment can be constructed; wherein the nonlinear function is expressed by formula three.
In the third formula, ωnAnd the weight coefficients corresponding to the Gaussian kernel functions are finally used as the data of the attribute feature set in each over-segmentation data segment and are calculation objects when the nonlinear functions are represented. I.e. during the subsequent processing, mainly used for finding omegan。
S1032: and calculating deviation data according to the data characteristic data and the position characteristic data of the over-segmentation data segment and the set critical damping system data.
The deviation data calculated in step S1032 is the aforementioned non-linear function f (t, z)m) Corresponding data. Therefore, the temperature of the molten metal is controlled,the deviation data can be directly adopted as f (t, z)m) And (4) showing.
In this embodiment, the deviation data may be obtained by calculation using a formula four-formula six.
In the formula four to the formula six,
position and speed parameters during characterization of the over-segmented data segment, bm＝[xm(0),ym(0)]And gm＝[xm(T),ym(T)]Is the position coordinate of the over-divided data segment whose initial time is the termination time T, rmPosition characteristic data (target point equation) of over-segmented data segments for corresponding time instantsmFor over-segmenting the velocity profile of the data segment at the corresponding time, αmAnd βmIs a characteristic parameter of the critical damping system.
S1033: and calculating to obtain a weight coefficient corresponding to each Gaussian kernel by adopting a local weighted regression algorithm according to the deviation data, the attenuation function and the Gaussian inner-haar function.
The specific steps a-C in step S1033.
Step A: calculating the deviation value fdev(t)；
And B: setting matrix z ═ zm(1),...,zm(T)]T,
And fdev＝[fdev(1),...,fdev(T)]T；
And C: using a locally weighted regression of ω (n) ═ zTψz)-1zTψfdevω (n) is calculated.
It should be noted that the foregoing description is only illustrative of the calculation method, and in the actual calculation process, it is necessary to perform calculation on the x coordinate data and the y coordinate data in the divided data segments to obtain the corresponding ωxAnd ωy，ωxAnd ωyEach of which is a data sequence composed of a plurality of weight coefficients.
S1034: and constructing the attribute feature set by adopting the starting point speed of the over-segmentation data segment and the weight coefficient corresponding to each Gaussian kernel function.
The weight coefficients determined in steps S1031 to S1033 mainly represent trajectory feature information of the over-segmented data segment, and the attribute features of the over-segmented data segment include a velocity feature in addition to the trajectory feature. Because the speed characteristic of each over-segmentation data segment has great relation with the speed of the starting point of each over-segmentation data segment, the speed of the starting point of each over-segmentation data segment can be matched with the weight coefficient to form an attribute characteristic set phim＝[vinit,ωx,ωy]。
Step S104: and screening each over-segmentation data segment by adopting an expectation maximization algorithm based on the attribute feature sets corresponding to all over-segmentation data segments to obtain the over-segmentation data segments for forming the motion primitive database and the corresponding attribute feature sets.
Each data segment determined in step S103 is an over-segmented data segment, and a reasonable over-segmented data segment needs to be screened, that is, a suitable zero-crossing segmentation point needs to be extracted.
Assuming that each over-segmented data segment s is part of the finally established trajectory primitive library, the trajectory primitive library is different types of primitives mmThe probability that the extracted over-segmentation data segment s belongs to the primitive library is expressed by a formula seven.
In formula seven, M is the total number of primitive types in the primitive library, λmAre a mixture of coefficients of different types of over-segmented data segments. All observed trajectory data o can be represented using a parametric model represented by equation eight.
In the formula VIII, S*A set of suitable over-segmented data segments obtained based on the most suitable segmentation point. Since the over-segmentation point is a parameter requiring solution, the over-segmentation point is taken as an implicit variable to be solved and is brought into a parametric model expressed by a formula eight to obtain a formula nine.
p(o|Φ)＝∑S∈Dp(S)∏s∈Sp (s | Φ) equation nine
In the formula nine, D is all possible segmentation results in the vehicle feature data o. Correspondingly, each trajectory primitive characterization parameter in the trajectory primitive library to be solved can be obtained by adopting the maximum likelihood function calculation shown in the formula ten.
Accordingly, the auxiliary functions used for iterative maximization until convergence in the expectation-maximization algorithm are defined as formula eleven and formula twelve.
Q(Φ,Φ′)＝∑o∈O∑S∈Dp (S | o, Φ')/log (p (S) p (o | Φ, S)) formula twelve.
In formula eleven and formula twelve, Φ' is a model optimization parameter obtained in the previous iteration process, and the prior probability p(s) is expressed as formula twelve and formula thirteen.
Formula twelve and formula thirteen, csFor over-segmenting the initial number of segmentation points, p, contained in the data segment sc∈[0,1]The probability that each initial segmentation point is a suitable segmentation point is characterized.
Under the condition of the determined relation, a weight coefficient α of the suitability degree of the segmentation result S under the characteristic parameter phi' of the existing track element library is represented by adopting an expectation maximization algorithm (EM algorithm)s＝∑S∈Dp (S | o, Φ') is updated in step E; and the characteristic parameter phi is updated in the M step. When the maximum algorithm convergence is expected, a suitable zero-crossing division point d can be obtained at the same time*And a set of characteristic parameters phi characterizing over-segmented data segments of different types*。
In step S104, over-segmented data segments that can be used as the motion primitives in the human-like driving motion primitive library and the attribute feature set of the over-segmented data segments can be determined.
According to the construction method of the human-like driving motion element library, complex driving behaviors are divided into over-segmentation data segments, the representation of the characteristics of each over-segmentation data segment is realized by using a Gaussian mixture model, and the attribute characteristic set corresponding to each over-segmentation data segment is formed.
The characterization method of each over-segmentation data segment has good generalization capability on the basis of representing the driving behavior on a certain precision; since each over-segmented data segment is a data segment collected in a real-person driving state, it can represent a path characteristic and a speed characteristic formed by a human driving habit.
Then, an expectation maximization algorithm is adopted, and based on the interdependence relationship between element representation and feature extraction, the element representation and feature extraction are carried out to extract from the over-segmentation data segments without labels, and the over-segmentation data segments which can be used as motion elements in a motion element library and the corresponding data feature set are determined; because the over-segmentation data segments and the corresponding data feature set which form the motion primitive database are the path feature data and the speed feature data which represent the driving behaviors of human beings, when the motion primitive database is adopted to construct the automatic driving path, the constructed path is more in line with the driving habits of human beings. Compared with the existing automatic driving path which is determined by simply fitting a mathematical curve (such as a cubic spline curve), the automatic driving path constructed by the motion elements in the motion element library can better meet the riding experience of passengers.
In addition to providing the aforementioned construction method of the human-like driving movement primitive library, an embodiment of the present specification further provides a method for implementing movement primitive connection based on the aforementioned movement primitive library. It should be noted that the connection method of the motion primitive library mentioned here is to implement connection of each motion primitive on the basis of the motion primitive for which a route selection has been determined, and the starting point and the target point of each selected primitive have been determined.
Fig. 2 is a flowchart of a method for connecting motion primitives according to an embodiment. As shown in fig. 2, the method provided by the present embodiment includes steps S201 to S204.
S201: the total duration is calculated from the duration of each selected motion primitive.
In this step, the total duration is the sum of the durations of the selected motion primitives, and may be specifically expressed by the formula fourteen.
In the fourteen formula, TkIs the duration of the kth selected motion primitive, K being the number of selected motion primitives.
S202: the total duration is used to calculate the decay function.
The attenuation function in step S202 is the same as the calculation method of the aforementioned attenuation function, and in a specific application, the attenuation function may be as shown in formula fifteen.
The parameters in equation fifteen are the same as the aforementioned equations except that T' is changed to the total duration, and will not be repeated here.
S203: respectively calculating the Gaussian kernel center and the kernel distribution amplitude of each selected motion element according to the duration and the total duration of each selected motion element, constructing a Gaussian kernel center sequence according to the Gaussian kernel centers of all selected motion elements, constructing a kernel distribution amplitude sequence according to the kernel distribution amplitudes of all selected motion elements, and constructing an element track characteristic sequence according to the weight coefficients in the attribute characteristic set of all motion elements.
In step S203, the gaussian kernel center of each selected motion primitive may be calculated by using formula sixteen, and correspondingly, the sequence of the gaussian kernel centers is represented by formula seventeen.
In the formula sixteen and the formula seventeen, N is the number of gaussian kernel centers in each motion primitive.
In step S203, the kernel distribution amplitude of each selected motion primitive may be calculated by using the formula eighteen, and the corresponding kernel distribution amplitude sequence is expressed by using the formula nineteen.
In the eighteenth formula and the nineteen formula,
the distribution amplitude of the original Gaussian kernel center of the selected motion element is obtained.
In step S203, a primitive trajectory feature sequence is constructed according to the weight coefficients in the attribute feature set of all the selected motion primitives, where the weight coefficients corresponding to the gaussian kernels in all the selected motion primitives are arranged according to the arrangement order of the selected motion primitives, and the primitive trajectory feature sequence may be represented by a formula twenty.
In the formula twenty, ω' represents a primitive trajectory feature sequence.
S204: and calculating a nonlinear part corresponding to each moment point according to the attenuation function, the Gaussian kernel center sequence, the kernel distribution amplitude sequence and the element track characteristic sequence, and converting the nonlinear part into a global coordinate system.
In step S204, a nonlinear part corresponding to each time point is calculated according to the attenuation function, the gaussian kernel center sequence, the kernel distribution amplitude sequence, and the primitive trajectory feature sequence, that is, the gaussian kernel function is reconstructed, and the primitive trajectory feature sequence and the attenuation function are combined to obtain a corresponding coefficient, so as to obtain a corresponding nonlinear part.
It should be noted that the above nineteen-twenty formula is only a teaching of the calculation method of the present solution, and in practical applications, it is necessary to consider the non-linear portions on different coordinate axes in each selected motion primitive coordinate system. In this embodiment, the calculation of the non-linear portion corresponding to each time point and the conversion of the non-linear portion to the global coordinate system are performed simultaneously, so that the calculation of the coordinate point of the later selected motion primitive is calculated on the basis of the determined, preceding coordinate point.
In this embodiment, the calculation formulas of the non-linear portions of the time points are as formulas twenty-one to twenty-four.
In the formulae twenty-one to twenty-four, Ftran_xThe abscissa component in the global coordinate system for the non-linear part of each time point, Ftran_yThe vertical coordinate component of the non-linear part in the global coordinate system is taken as each time point. Wherein, longitudinal and transverse Gaussian nuclei
Respectively adopting mu 'and p' determined in the seventeenth formula and the nineteen formula, determining according to the calculation method of the second formula,
is the triggering instant of the k-th kinematic primitive gaussian kernel function,
is that
The end time of the gaussian kernel function of each motion primitive. Sigma'kAnd σ'k-1Is the standard deviation of the corresponding gaussian kernel for the motion primitive; thetakThe angle between the starting velocity of the kth selected motion primitive and the x-axis of the global coordinate system is shown.
S205: and calculating a critical damping system part corresponding to each moment point, and summing the nonlinear part and the critical damping part to obtain a track coordinate corresponding to each moment point.
In step S205, in order to correspond to the foregoing formula four, in the present embodiment, a process of calculating the critical damping system portion and summing the nonlinear portion and the critical damping portion is simultaneously expressed by using formulas twenty-five to twenty-eight.
Twenty-five of the formulas to twenty-eight of the formulas αmAnd βmIs a characteristic parameter r 'of a critical damping system'xIs an objective function of the spliced primitive in the x direction, v'xSpeed of the spliced element in the x direction, r'yIs the target function of the tiled primitive in the y-direction, and v'yIs the objective function of the spliced primitive in the y direction.
The method for connecting the motion primitives provided by the embodiment of the description converts the splicing connection problem of the selected motion primitives into the re-characterization problem of the motion primitives, so that the smooth transition of the selected motion primitives at the connection point is realized, and the spliced track has good smooth characteristics.
In addition to providing the aforementioned construction method of the human-like driving motion primitive library and the method of connecting motion primitives, the embodiments of the present specification also provide a construction device of the human-like driving motion primitive library and a device of connecting motion primitives. Because the construction device of the human-like driving movement primitive library and the construction method adopt the same inventive concept, and the device for connecting the movement primitives and the method for connecting the movement primitives adopt the same inventive concept, the following only describes the structures of the two devices, and the corresponding technical effect expression can be referred to the foregoing.
FIG. 3 is a schematic diagram of a device for constructing a human-like driving movement primitive library according to an embodiment. As shown in fig. 3, the construction apparatus includes a data acquisition unit 11, a segmentation unit 12, a feature set construction unit 13, and a motion primitive screening unit 14.
The data acquisition unit 11 is used for acquiring vehicle characteristic data at each sampling moment under the condition of driving by a person; the travel data includes heading characteristic data, speed characteristic data, and location characteristic data.
The segmentation unit 12 is configured to determine a heading change zero crossing point of vehicle driving according to the heading feature data at each sampling time; and dividing the vehicle characteristic data by using the course change zero crossing point as an over-division point to obtain an over-division data segment.
The feature set constructing unit 13 is configured to calculate an attribute feature set corresponding to each over-segmented data segment according to the speed feature data and the position feature data of each over-segmented data segment. In one application, the feature set constructing unit 13 includes a function constructing subunit, a deviation data calculating subunit, a weight coefficient calculating unit, and a feature set constructing subunit.
The function constructing subunit is used for constructing an attenuation function and a Gaussian kernel function; the deviation data calculating subunit is used for calculating deviation data according to the speed characteristic data and the position characteristic data of the over-segmentation data segment and the set critical damping system data; the weight coefficient calculation unit is used for calculating the weight coefficient corresponding to each Gaussian kernel function by adopting a local weighted regression algorithm according to the deviation data, the attenuation function and the Gaussian kernel function; the feature set constructing subunit is used for constructing an attribute feature set by using the starting point speed of the over-segmentation data segment and the weight coefficient corresponding to each Gaussian kernel function. The moving element screening unit 14 is configured to screen, based on the attribute feature sets corresponding to all over-segmented data segments, an expected maximum algorithm to obtain over-segmented data segments serving as moving elements in the moving element library and corresponding attribute feature sets.
Fig. 4 is a schematic diagram of an apparatus for connecting motion primitives according to an embodiment. As shown in fig. 4, it includes a time calculation unit 21, an attenuation function calculation unit 22, a mixture calculation unit 23, a nonlinear part calculation unit 24, and a combination unit 25.
The time calculation unit 21 is arranged to calculate the total duration from the duration of each selected motion primitive.
The decay function calculation unit 22 is adapted to calculate the decay function using the total duration.
The hybrid calculation unit 23 is configured to calculate a gaussian kernel center and a kernel distribution amplitude of each selected motion primitive according to the duration and the total duration of each selected motion primitive, construct a gaussian kernel center sequence according to the gaussian kernel centers of all selected motion primitives, and construct a kernel distribution amplitude sequence according to the kernel distribution amplitudes of all selected motion primitives; and constructing a primitive track characteristic sequence according to the weight coefficients in the attribute characteristic set of all the selected motion primitives.
The nonlinear part calculating unit 24 is configured to calculate a nonlinear part corresponding to each time point according to the attenuation function, the gaussian kernel center sequence, the kernel distribution amplitude sequence, and the primitive trajectory feature sequence, and convert the nonlinear part into a global coordinate system.
The combination unit 25 is configured to calculate a critical damping system portion corresponding to each time point by using the speed feature data of the selected motion primitive and the speed parameter of the selected motion primitive; and summing the nonlinear part and the critical damping system to obtain the corresponding track coordinates of each moment point.
In one application, the non-linear part calculating unit 24 converts the non-linear part into a global coordinate system, including: according to the included angle between the starting point speed of each selected motion element and the coordinate axis of the global coordinate system; and adjusting the nonlinear part according to the included angle, and converting the nonlinear part into a global coordinate system.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are also included in the scope of the present invention.
Claims (10)
1. A construction method of a driving movement primitive-like library is characterized by comprising the following steps:
acquiring vehicle characteristic data of each sampling moment under the condition of driving by a person; the driving data comprises course characteristic data, speed characteristic data and position characteristic data;
determining the heading change zero crossing point of the vehicle running according to the heading characteristic data at each sampling moment; dividing the vehicle characteristic data by using the course change zero crossing point as an over-division point to obtain an over-division data segment;
calculating an attribute feature set corresponding to each over-segmentation data segment according to the speed feature data and the position feature data of each over-segmentation data segment;
and screening the over-segmentation data segments serving as the moving primitives in the moving primitive library and the corresponding attribute feature sets by adopting an expected maximum algorithm based on the attribute feature sets corresponding to all the over-segmentation data segments.
2. The method for constructing the human-like driving motion primitive database according to claim 1, wherein determining the attribute feature set corresponding to each over-segmented data segment according to the speed feature data and the position feature data in each over-segmented data segment comprises:
constructing an attenuation function and a Gaussian kernel function;
calculating deviation data according to the speed characteristic data and the position characteristic data of the over-segmentation data segment and the set critical damping system data;
calculating a weight coefficient corresponding to each Gaussian kernel function by adopting a local weighted regression algorithm according to the deviation data, the attenuation function and the Gaussian kernel function;
and constructing the attribute feature set by adopting the starting point speed of the over-segmentation data segment and the weight coefficient corresponding to each Gaussian kernel function.
3. According to the claimsSolving 2 the construction method of the human-like driving motion primitive database, which is characterized in that the attenuation function is as follows:
wherein αzT is the duration of the over-segmentation data segment and delta T is the sampling interval of the sampling moment.
4. A method of concatenating motion primitives for concatenating selected motion primitives from the pool of human-like motion primitives of any one of claims 1-3; it is characterized by comprising:
calculating a total duration from the durations of each selected motion primitive;
calculating an attenuation function using the total duration;
respectively calculating the Gaussian kernel center and the kernel distribution amplitude of each selected motion element according to the duration and the total duration of each selected motion element, constructing a Gaussian kernel center sequence according to the Gaussian kernel centers of all selected motion elements, and constructing a kernel distribution amplitude sequence according to the kernel distribution amplitudes of all selected motion elements; constructing a primitive track characteristic sequence according to the weight coefficients in the attribute characteristic set of all the selected motion primitives;
calculating a nonlinear part corresponding to each moment point according to the attenuation function, the Gaussian kernel center sequence, the kernel distribution amplitude sequence and the element track characteristic sequence, and converting the nonlinear part into a global coordinate system;
calculating a critical damping system part corresponding to each moment point by adopting the speed characteristic data of the selected motion element and the speed parameter of the selected motion element; and summing the nonlinear part and the critical damping system to obtain the track coordinate corresponding to each time point.
5. The method of claim 4, wherein converting the non-linear portion to a global coordinate system comprises:
according to the included angle between the starting point speed of each selected motion element and the coordinate axis of the global coordinate system;
and adjusting the nonlinear part according to the included angle, and converting the nonlinear part into a global coordinate system.
6. A construction device of a driving-like motion primitive library, comprising:
the data acquisition unit is used for acquiring vehicle characteristic data at each sampling moment under the condition of driving by a person; the driving data comprises course characteristic data, speed characteristic data and position characteristic data;
the segmentation unit is used for determining the heading change zero crossing point of the vehicle running according to the heading characteristic data at each sampling moment; dividing the vehicle characteristic data by using the course change zero crossing point as an over-division point to obtain an over-division data segment;
the feature set construction unit is used for calculating an attribute feature set corresponding to each over-segmentation data segment according to the speed feature data and the position feature data of each over-segmentation data segment;
and the motion element screening unit is used for screening the over-segmentation data segments serving as the motion elements in the motion element library and the corresponding attribute feature sets by adopting an expected maximum algorithm based on the attribute feature sets corresponding to all the over-segmentation data segments.
7. The apparatus of claim 6, wherein the feature set constructing unit comprises:
the function constructing subunit is used for constructing an attenuation function and a Gaussian kernel function;
the deviation data calculating subunit is used for calculating deviation data according to the speed characteristic data and the position characteristic data of the over-segmentation data segment and set critical damping system data;
the weight coefficient calculation unit is used for calculating the weight coefficient corresponding to each Gaussian kernel function by adopting a local weighted regression algorithm according to the deviation data, the attenuation function and the Gaussian kernel function;
and the characteristic set constructing subunit is used for constructing the attribute characteristic set by adopting the starting point speed of the over-segmentation data segment and the weight coefficient corresponding to each Gaussian kernel function.
8. The apparatus for constructing a human-like driving motion primitive library according to claim 7, wherein said attenuation function is:
wherein αzT is the duration of the over-segmentation data segment and delta T is the sampling interval of the sampling moment.
9. A means for connecting a selected motion primitive in the human-like driving motion primitive library of any one of claims 1-3; it is characterized by comprising:
a time calculation unit calculating a total duration from the durations of each selected motion primitive;
an attenuation function calculation unit that calculates an attenuation function using the total duration;
the mixed calculation unit is used for respectively calculating the Gaussian kernel center and the kernel distribution amplitude of each selected motion element according to the duration and the total duration of each selected motion element, constructing a Gaussian kernel center sequence according to the Gaussian kernel centers of all selected motion elements, and constructing a kernel distribution amplitude sequence according to the kernel distribution amplitudes of all selected motion elements; constructing a primitive track characteristic sequence according to the weight coefficients in the attribute characteristic set of all the selected motion primitives;
the nonlinear part calculating unit calculates a nonlinear part corresponding to each moment point according to the attenuation function, the Gaussian kernel center sequence, the kernel distribution amplitude sequence and the element track characteristic sequence, and converts the nonlinear part into a global coordinate system;
the combination unit is used for calculating a critical damping system part corresponding to each moment point by adopting the speed characteristic data of the selected motion element and the speed parameter of the selected motion element; and summing the nonlinear part and the critical damping system to obtain the track coordinate corresponding to each time point.
10. The apparatus according to claim 9, wherein the non-linear portion calculating unit converts the non-linear portion into a global coordinate system, comprising:
according to the included angle between the starting point speed of each selected motion element and the coordinate axis of the global coordinate system;
and adjusting the nonlinear part according to the included angle, and converting the nonlinear part into a global coordinate system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010136047.6A CN111341102B (en) | 2020-03-02 | 2020-03-02 | Motion primitive library construction method and device and motion primitive connection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010136047.6A CN111341102B (en) | 2020-03-02 | 2020-03-02 | Motion primitive library construction method and device and motion primitive connection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111341102A true CN111341102A (en) | 2020-06-26 |
| CN111341102B CN111341102B (en) | 2021-04-23 |
Family
ID=71185934
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010136047.6A Active CN111341102B (en) | 2020-03-02 | 2020-03-02 | Motion primitive library construction method and device and motion primitive connection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111341102B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113741179A (en) * | 2021-11-08 | 2021-12-03 | 北京理工大学 | Heterogeneous vehicle-oriented unified motion planning method and system |
| CN115495882A (en) * | 2022-08-22 | 2022-12-20 | 北京科技大学 | Method and device for constructing robot motion primitive library under non-flat terrain |
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080120025A1 (en) * | 2006-11-22 | 2008-05-22 | Denso Corporation | Driving behavior prediction method and apparatus |
| US7839292B2 (en) * | 2007-04-11 | 2010-11-23 | Nec Laboratories America, Inc. | Real-time driving danger level prediction |
| CN102160100A (en) * | 2008-09-18 | 2011-08-17 | 日产自动车株式会社 | Driving maneuver assisting apparatus and method for assisting driving maneuver |
| CN102449991A (en) * | 2009-05-29 | 2012-05-09 | 高通股份有限公司 | Method and apparatus for movement detection by evaluating elementary movement patterns |
| CN103444163A (en) * | 2011-02-05 | 2013-12-11 | 苹果公司 | Method and apparatus for mobile location determination |
| US9177475B2 (en) * | 2013-11-04 | 2015-11-03 | Volkswagen Ag | Driver behavior based parking availability prediction system and method |
| CN105046677A (en) * | 2015-08-27 | 2015-11-11 | 安徽超远信息技术有限公司 | Enhancement processing method and apparatus for traffic video image |
| US9443153B1 (en) * | 2015-06-12 | 2016-09-13 | Volkswagen Ag | Automatic labeling and learning of driver yield intention |
| CN107289941A (en) * | 2017-06-14 | 2017-10-24 | 湖南格纳微信息科技有限公司 | A kind of indoor orientation method and device based on inertial navigation |
| US20180053102A1 (en) * | 2016-08-16 | 2018-02-22 | Toyota Jidosha Kabushiki Kaisha | Individualized Adaptation of Driver Action Prediction Models |
| US20180082242A1 (en) * | 2016-09-22 | 2018-03-22 | LucidLift LLC | Data-driven training and coaching system and method |
| US20180113458A1 (en) * | 2016-10-24 | 2018-04-26 | International Business Machines Corporation | Method of effective driving behavior extraction using deep learning |
| CN108153310A (en) * | 2017-12-22 | 2018-06-12 | 南开大学 | A kind of Mobile Robot Real-time Motion planing method based on human behavior simulation |
| CN108805292A (en) * | 2017-05-05 | 2018-11-13 | 英特尔公司 | For the instant deep learning in the machine learning of autonomous machine |
| CN108944930A (en) * | 2018-07-05 | 2018-12-07 | 合肥工业大学 | A kind of automatic follow the bus method and system of drive simulating person's characteristic based on LSTM |
| CN108986545A (en) * | 2018-08-16 | 2018-12-11 | 武汉理工大学 | Personalized driving inducible system based on V2V |
| CN109263639A (en) * | 2018-08-24 | 2019-01-25 | 武汉理工大学 | Driving path planing method based on state Grid Method |
| CN109278758A (en) * | 2018-08-28 | 2019-01-29 | 武汉理工大学 | A kind of intelligent vehicle personalized driving learning system based on smart phone |
| CN109871010A (en) * | 2018-12-25 | 2019-06-11 | 南方科技大学 | Method and system based on intensified learning type |
-
2020
- 2020-03-02 CN CN202010136047.6A patent/CN111341102B/en active Active
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080120025A1 (en) * | 2006-11-22 | 2008-05-22 | Denso Corporation | Driving behavior prediction method and apparatus |
| US7839292B2 (en) * | 2007-04-11 | 2010-11-23 | Nec Laboratories America, Inc. | Real-time driving danger level prediction |
| CN102160100A (en) * | 2008-09-18 | 2011-08-17 | 日产自动车株式会社 | Driving maneuver assisting apparatus and method for assisting driving maneuver |
| CN102449991A (en) * | 2009-05-29 | 2012-05-09 | 高通股份有限公司 | Method and apparatus for movement detection by evaluating elementary movement patterns |
| CN103444163A (en) * | 2011-02-05 | 2013-12-11 | 苹果公司 | Method and apparatus for mobile location determination |
| US9177475B2 (en) * | 2013-11-04 | 2015-11-03 | Volkswagen Ag | Driver behavior based parking availability prediction system and method |
| US9443153B1 (en) * | 2015-06-12 | 2016-09-13 | Volkswagen Ag | Automatic labeling and learning of driver yield intention |
| CN105046677A (en) * | 2015-08-27 | 2015-11-11 | 安徽超远信息技术有限公司 | Enhancement processing method and apparatus for traffic video image |
| US20180053102A1 (en) * | 2016-08-16 | 2018-02-22 | Toyota Jidosha Kabushiki Kaisha | Individualized Adaptation of Driver Action Prediction Models |
| US20180082242A1 (en) * | 2016-09-22 | 2018-03-22 | LucidLift LLC | Data-driven training and coaching system and method |
| US20180113458A1 (en) * | 2016-10-24 | 2018-04-26 | International Business Machines Corporation | Method of effective driving behavior extraction using deep learning |
| CN108805292A (en) * | 2017-05-05 | 2018-11-13 | 英特尔公司 | For the instant deep learning in the machine learning of autonomous machine |
| CN107289941A (en) * | 2017-06-14 | 2017-10-24 | 湖南格纳微信息科技有限公司 | A kind of indoor orientation method and device based on inertial navigation |
| CN108153310A (en) * | 2017-12-22 | 2018-06-12 | 南开大学 | A kind of Mobile Robot Real-time Motion planing method based on human behavior simulation |
| CN108944930A (en) * | 2018-07-05 | 2018-12-07 | 合肥工业大学 | A kind of automatic follow the bus method and system of drive simulating person's characteristic based on LSTM |
| CN108986545A (en) * | 2018-08-16 | 2018-12-11 | 武汉理工大学 | Personalized driving inducible system based on V2V |
| CN109263639A (en) * | 2018-08-24 | 2019-01-25 | 武汉理工大学 | Driving path planing method based on state Grid Method |
| CN109278758A (en) * | 2018-08-28 | 2019-01-29 | 武汉理工大学 | A kind of intelligent vehicle personalized driving learning system based on smart phone |
| CN109871010A (en) * | 2018-12-25 | 2019-06-11 | 南方科技大学 | Method and system based on intensified learning type |
Non-Patent Citations (9)
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113741179A (en) * | 2021-11-08 | 2021-12-03 | 北京理工大学 | Heterogeneous vehicle-oriented unified motion planning method and system |
| CN113741179B (en) * | 2021-11-08 | 2022-03-25 | 北京理工大学 | Heterogeneous vehicle-oriented unified motion planning method and system |
| CN115495882A (en) * | 2022-08-22 | 2022-12-20 | 北京科技大学 | Method and device for constructing robot motion primitive library under non-flat terrain |
| CN115495882B (en) * | 2022-08-22 | 2024-02-27 | 北京科技大学 | Method and device for constructing robot motion primitive library under uneven terrain |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111341102B (en) | 2021-04-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7075366B2 (en) | Methods, devices, equipment and media for classifying driving scene data | |
| Liebel et al. | Auxiliary tasks in multi-task learning | |
| US11243532B1 (en) | Evaluating varying-sized action spaces using reinforcement learning | |
| Nishi et al. | Merging in congested freeway traffic using multipolicy decision making and passive actor-critic learning | |
| Fernando et al. | Deep inverse reinforcement learning for behavior prediction in autonomous driving: Accurate forecasts of vehicle motion | |
| Chen et al. | Driving maneuvers prediction based autonomous driving control by deep Monte Carlo tree search | |
| CN112162555A (en) | Vehicle control method based on reinforcement learning control strategy in hybrid vehicle fleet | |
| US20210056863A1 (en) | Hybrid models for dynamic agents in a simulation environment | |
| CN110406530A (en) | A kind of automatic Pilot method, apparatus, equipment and vehicle | |
| US20230196909A1 (en) | Systems and Methods for Simulating Traffic Scenes | |
| JP2022547611A (en) | Simulation of various long-term future trajectories in road scenes | |
| Li et al. | Oil: Observational imitation learning | |
| CN111341102B (en) | Motion primitive library construction method and device and motion primitive connection method and device | |
| Zhou et al. | Learning the car-following behavior of drivers using maximum entropy deep inverse reinforcement learning | |
| WO2022156181A1 (en) | Movement trajectory prediction method and apparatus | |
| Heinrich | Planning universal on-road driving strategies for automated vehicles | |
| Cheng et al. | Convolutional neural network-based lane-change strategy via motion image representation for automated and connected vehicles | |
| CN113239986A (en) | Training method and device for vehicle track evaluation network model and storage medium | |
| Feher et al. | Q-learning based reinforcement learning approach for lane keeping | |
| Sun et al. | Human-like highway trajectory modeling based on inverse reinforcement learning | |
| Mirus et al. | An investigation of vehicle behavior prediction using a vector power representation to encode spatial positions of multiple objects and neural networks | |
| Saha et al. | Practical self-driving cars: Survey of the state-of-the-art | |
| Selvaraj et al. | Edge learning of vehicular trajectories at regulated intersections | |
| CN111507499B (en) | Method, device and system for constructing model for prediction and testing method | |
| Rao et al. | Spatio-temporal look-ahead trajectory prediction using memory neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |