CN117079242B

CN117079242B - Deceleration strip determining method and device, storage medium, electronic equipment and vehicle

Info

Publication number: CN117079242B
Application number: CN202311268312.6A
Authority: CN
Inventors: 毛金为; 钟晓云; 赵伟冰; 王有为; 黄毕有
Original assignee: BYD Co Ltd
Current assignee: BYD Co Ltd
Priority date: 2023-09-28
Filing date: 2023-09-28
Publication date: 2024-01-26
Anticipated expiration: 2043-09-28
Also published as: CN117079242A

Abstract

The disclosure relates to a deceleration strip determining method, a deceleration strip determining device, a storage medium, electronic equipment and a vehicle. The method comprises the following steps: acquiring a target image to be processed; generating a feature map corresponding to the target image by utilizing a pre-generated key point identification model; determining position information of a plurality of key points linearly distributed on a deceleration strip according to the feature map; and determining a target deceleration strip in the target image according to the position information of the plurality of key points. Therefore, the key point positions of the deceleration strips in the target image can be rapidly determined through the feature map, the speed of the deceleration strips can be improved and determined, the deceleration strips are simplified to be linear through the key points of the deceleration strips, the deceleration strips are further determined through the key points of the deceleration strips, the deceleration strips are fitted with high precision through the connection of the key points of the deceleration strips, simplicity and high efficiency are achieved, and accuracy can be guaranteed.

Description

Deceleration strip determining method and device, storage medium, electronic equipment and vehicle

Technical Field

The disclosure relates to the technical field of vehicles, in particular to a deceleration strip determining method and device, a storage medium, electronic equipment and a vehicle.

Background

The deceleration strip is used as an important ground traffic sign and is mainly used for reducing traffic accidents and road control. In driving scenes, the deceleration strip detection technology is an important component, through recognition of the deceleration strip, a series of auxiliary actions can be executed when a vehicle passes through the deceleration strip, for example, the speed of the vehicle is reduced before the vehicle passes through the deceleration strip, the state of a front suspension system and a rear suspension system is adjusted and measured before and after the vehicle passes through the deceleration strip, and the like, so that the safety of the vehicle is ensured, and the riding experience of drivers and passengers in the vehicle is improved. It can be seen that this is very important for determining the deceleration strip in driving situations. In the related art, when determining the deceleration strip, the outline or the frame of the deceleration strip is usually determined, the accuracy is not enough, and meanwhile, the stability is not enough in various and complex actual driving scenes.

Disclosure of Invention

The invention aims to provide a deceleration strip determining method, a deceleration strip determining device, a storage medium, electronic equipment and a vehicle, so that detection of the deceleration strip is rapidly and accurately realized.

In order to achieve the above object, according to a first aspect of the present disclosure, there is provided a deceleration strip determination method including:

acquiring a target image to be processed;

Generating a feature map corresponding to the target image by utilizing a pre-generated key point identification model;

determining position information of a plurality of key points linearly distributed on a deceleration strip according to the feature map;

and determining a target deceleration strip in the target image according to the position information of the plurality of key points.

Optionally, the generating a feature map corresponding to the target image by using a pre-generated key point recognition model includes:

inputting a processed image into the key point identification model to obtain a first number of feature images output by the key point identification model, wherein one feature image corresponds to one image area in the target image, and the one feature image is used for indicating feature information of the image area corresponding to the feature image, the feature information comprises first information used for indicating whether a deceleration strip key point exists or not and second information used for indicating the position of the deceleration strip key point, and the processed image is an image in a target format obtained by processing the target image.

Optionally, the keypoint identification model is generated by:

obtaining training samples, wherein each training sample comprises a training image and a first number of training feature images corresponding to the training image, the training image is in the target format, one training feature image corresponds to one image area in the training image, the training feature image is used for indicating training feature information of the image area corresponding to the feature image, and the training feature information comprises first label information used for indicating whether a deceleration strip key point exists or not and second label information used for indicating the position of the deceleration strip key point;

And performing model training by taking the training images as the input of the model and taking the first number of training feature images as the target output of the model so as to obtain the trained key point recognition model.

Optionally, the training image is obtained by:

acquiring initial samples, wherein each initial sample comprises an initial image and deceleration strip marking information corresponding to the initial image, the deceleration strip marking information comprises at least one point set, and one point set is used for indicating one deceleration strip in the initial image;

extracting at least one partial image from the initial image for each initial image, wherein each partial image comprises a deceleration strip;

and processing the local images aiming at each local image to obtain the training image in the target format.

Optionally, the first number of training feature maps corresponding to the training image is determined by:

extracting, for each partial image, partial mark information corresponding to a deceleration strip in the partial image from target deceleration strip mark information corresponding to an initial image in which the partial image is located, wherein the partial mark information comprises a subset of at least one point set in the target deceleration strip mark information;

And dividing the local images into a first number of training feature images aiming at each local image, and determining first label information and second label information of each training feature image according to local mark information corresponding to the local images.

Optionally, if the number of the point sets in the target deceleration strip mark information exceeds the second number, deleting part of the point sets in the target deceleration strip mark information so that the number of the point sets in the deceleration strip mark information does not exceed the second number.

Optionally, in each model training, an output result obtained by processing an input training image by the model includes a first number of output feature maps, one output feature map corresponds to one image area in the input training image, and one output feature map is used for indicating output feature information of the image area corresponding to the output feature map, where the output feature information includes third information for indicating whether a deceleration strip key point exists and fourth information for indicating a deceleration strip key point position;

in one model training, the loss value for updating the model is determined by at least one of:

According to a first training feature map and a first loss value determined by the first feature map, the first training feature map is a training feature map with deceleration strip key points in a training feature map corresponding to a training image used in the model training, and the first feature map is an output feature map corresponding to the first training feature map in an output result of the training;

according to a second training feature map and a second loss value determined by the second feature map, the second training feature map is a training feature map which is corresponding to a training image used in the model training and does not have a deceleration strip key point, and the second feature map is an output feature map which is corresponding to the second training feature map in an output result of the training;

a third loss value determined according to the first label information of the training feature map corresponding to the training image used in the model training and the third information;

a fourth loss value is determined according to the coordinate value of the deceleration strip key point position indicated by the second label information on the first coordinate axis and the coordinate value of the deceleration strip key point position indicated by the fourth label information on the first coordinate axis;

and determining a fifth loss value according to the coordinate value of the deceleration strip key point position indicated by the second label information on the second coordinate axis and the coordinate value of the deceleration strip key point position indicated by the fourth label information on the second coordinate axis.

Optionally, the processed image is obtained by:

converting the target image into a gray scale image to obtain a first intermediate image;

normalizing the first intermediate image to obtain a second intermediate image;

and generating a new image according to the second intermediate image through interpolation processing, and taking the new image as the processing image.

Optionally, the feature map is a multi-channel image, and the feature map includes a plurality of first channels for storing first information and a plurality of second channels for storing second information, where the number of the first channels is the sum of a second number and 1, and the number of the second channels is two times of the second number, and the second number is the maximum number of deceleration strips that can be identified by the key point identification model through one feature map;

among the plurality of first channels, a first channel corresponding to a first confidence representing whether a feature map has a deceleration strip key point and a second number of first channels corresponding to a second confidence representing whether an Nth deceleration strip has a deceleration strip key point are included;

the plurality of second channels comprise a second channel corresponding to the coordinate value of the Nth deceleration strip in the first coordinate axis and a second channel corresponding to the coordinate value of the Nth deceleration strip in the second coordinate axis;

Wherein N comprises a positive integer from 1 to a second number.

Optionally, determining, according to the feature map, position information of a plurality of key points linearly distributed on the deceleration strip includes:

determining target feature graphs with first confidence coefficient reaching a first threshold and second confidence coefficient reaching a second threshold in a first number of feature graphs corresponding to the target image;

determining the position coordinates of deceleration strip key points in the target feature images corresponding to each target feature image through coordinate values of target channels, wherein the target channels are second channels corresponding to the first channels reaching the second threshold;

and clustering the deceleration strip key points of which the position coordinates are determined to determine at least one deceleration strip key point sequence so as to determine the position information of the plurality of key points.

Optionally, the clustering processing is performed on the deceleration strip keypoints determined by the position coordinates to determine at least one deceleration strip keypoint sequence, so as to determine position information of the plurality of keypoints, including:

adding deceleration strip key points with determined position coordinates into a key point matrix corresponding to the target feature map aiming at each target feature map, wherein the key point matrix comprises a non-empty point column, and the non-empty point column comprises deceleration strip key points with determined position coordinates;

Determining a first non-empty point column in the key point matrix along the direction of the first coordinate axis;

generating a deceleration strip key point sequence with deceleration strip key points according to deceleration strip key points contained in the first non-empty point sequence respectively aiming at each deceleration strip key point;

determining a next non-empty point column along the direction of the first coordinate axis to serve as a target point column;

taking each deceleration strip key point in the target point row as a target key point, and executing the following steps: determining the distance between a target key point and a target element in the direction of the second coordinate axis, wherein the target element is a tail element of each existing deceleration strip key point sequence; comparing the minimum value in the determined distance with a preset threshold value; if the minimum value is smaller than the preset threshold value, adding the target key point to the tail part of the deceleration strip key point sequence corresponding to the minimum value; if the minimum value is greater than or equal to the preset threshold value, generating a deceleration strip key point sequence with the target key point;

under the condition that a non-empty point column which is not used as a target point column exists in the key point matrix, the step of determining the next non-empty point column along the direction of the first coordinate axis as the target point column is executed again until the non-empty point column which is not used as the target point column does not exist in the key point matrix;

And determining the position information of the key points in the existing deceleration strip key point sequence under the condition that the non-empty point sequence which is not used as the target point sequence no longer exists in the key point matrix.

Optionally, the determining the location information of the plurality of key points in the existing deceleration strip key point sequence includes:

determining the number of elements in each existing deceleration strip key point sequence;

and taking the deceleration strip key point sequence with the number of elements reaching a preset value as an alternative sequence, and determining the position information of the key points according to the key points contained in the alternative sequence.

According to a second aspect of the present disclosure, there is provided a deceleration strip determination apparatus, the apparatus comprising:

the first acquisition module is used for acquiring a target image to be processed;

the generation module is used for generating a feature map corresponding to the target image by utilizing a pre-generated key point identification model;

the first determining module is used for determining position information of a plurality of key points linearly distributed on the deceleration strip according to the characteristic diagram;

and the second determining module is used for determining a target deceleration strip in the target image according to the position information of the plurality of key points.

According to a third aspect of the present disclosure there is provided a computer readable storage medium having stored thereon computer program instructions which when executed by a processor implement the steps of the method of the first aspect of the present disclosure.

According to a fourth aspect of the present disclosure, there is provided an electronic device comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the method of the first aspect of the disclosure.

According to a fifth aspect of the present disclosure, there is provided a vehicle comprising the electronic device of the fourth aspect of the present disclosure.

According to the technical scheme, the target image to be processed is obtained, the feature map corresponding to the target image is generated by utilizing the pre-generated key point identification model, the position information of a plurality of key points distributed linearly on the deceleration strip is determined according to the feature map, and the target deceleration strip in the target image is determined according to the position information of the plurality of key points. Therefore, the key point positions of the deceleration strips in the target image can be rapidly determined through the feature map, the speed of the deceleration strips can be improved and determined, the deceleration strips are simplified to be linear through the key points of the deceleration strips, the deceleration strips are further determined through the key points of the deceleration strips, the deceleration strips are fitted with high precision through the connection of the key points of the deceleration strips, simplicity and high efficiency are achieved, and accuracy can be guaranteed.

Additional features and advantages of the present disclosure will be set forth in the detailed description which follows.

Drawings

The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate the disclosure and together with the description serve to explain, but do not limit the disclosure. In the drawings:

FIG. 1 is a flow chart of a deceleration strip determination method provided in accordance with one embodiment of the present disclosure;

FIG. 2 is an exemplary schematic illustration of an initial image and a partial image of the present disclosure;

FIG. 3 is an exemplary schematic diagram of a partial image and training feature map of the present disclosure;

FIG. 4 is a block diagram of a deceleration strip determination apparatus provided in accordance with one embodiment of the present disclosure;

fig. 5 is a block diagram of an electronic device, according to an example embodiment.

Detailed Description

Specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating and illustrating the disclosure, are not intended to limit the disclosure.

In a road scene, a deceleration strip is generally disposed in a road in a form of a long line and perpendicular to the extending direction of the road. As described in the background art, determining the deceleration strip plays a very important role in driving scenarios. In the related art, two ways are generally adopted to determine the position of the deceleration strip in the image, one way is to determine the outer frame information of the deceleration strip through a target detection way, so as to obtain whether the deceleration strip exists in the image, the position of the deceleration strip in the image and the like; the other is to obtain a semantic segmentation image by an image segmentation mode, and then to obtain the position information of the deceleration strip outline in the image by performing image processing on the semantic segmentation image. However, in the former method, the circumscribed rectangular frame of the plurality of deceleration strips is determined by a target detection method, when the length difference between the deceleration strips is large, the method is extremely easy to generate the condition of missing detection, and when the deceleration strips are bent or inclined, the problems of low detection precision, complex processing and the like are also caused; in the latter way, in the actual application scene, the method is extremely easy to be interfered by factors such as pavement aging, illumination, rainwater, sand and the like, and has the problem of low precision.

In order to solve the technical problems, the present disclosure provides a deceleration strip determining method, a deceleration strip determining device, a storage medium, an electronic device and a vehicle, so as to rapidly and accurately detect a deceleration strip.

Fig. 1 is a flowchart of a deceleration strip determination method provided according to one embodiment of the present disclosure. As shown in fig. 1, the method may include steps 11 to 14.

In step 11, a target image to be processed is acquired.

The target image may be an image reflecting the surrounding environment of the vehicle, such as a road image. For example, the target image may be acquired by a forward-looking camera of the vehicle during the travel of the vehicle.

In step 12, a feature map corresponding to the target image is generated using the previously generated key point recognition model.

In one possible implementation, step 12 may be implemented by:

and inputting the processed image into the key point identification model to obtain a first number of feature images output by the key point identification model.

The processing image is an image in a target format obtained by processing the target image. The target format is the input format of the keypoint identification model. In the model processing process, a certain requirement is usually imposed on the format of the input content, and the target image is converted into a processed image in the target format, so that the reasoning of the model is facilitated.

Alternatively, the processed image may be obtained by:

normalizing the first intermediate image to obtain a second intermediate image;

according to the second intermediate image, a new image is generated by interpolation processing, and the new image is taken as a processed image.

For the target image, the values of the RGB channels of each pixel may be weighted and averaged to obtain a gray value corresponding to the pixel, so as to convert the target image into a gray image, that is, a first intermediate image.

Based on the first intermediate image, the normalization may be achieved by mapping the pixel values to a range between 0 and 1 by dividing each pixel value by 255, resulting in a second intermediate image.

After the second intermediate image is obtained, a new image may be generated by interpolation processing to obtain a processed image. Alternatively, the second intermediate image may be converted into a new image of the first size by interpolation processing based on the first size to obtain the above-described processed image. The above interpolation processing may be performed by bilinear interpolation, for example. For example, if the target image has a height 1080 and a width 1920, the first dimension may be the height 320 and the width 640.

One feature map may correspond to one image region in the target image, and one feature map may be used to indicate feature information of the image region to which the feature map corresponds.

Wherein, the characteristic information may include first information for indicating whether a deceleration strip key point exists and second information for indicating a deceleration strip key point position. Therefore, the feature map can reflect whether a certain image area in the target image contains the deceleration strip or not, and can further reflect the position of the deceleration strip when the deceleration strip is contained.

The first number may be 1 or an integer greater than 1. In the case that the first number is 1, the feature map corresponding to the target image generated by the key point recognition model may be the entire image area of the target image.

The feature map may have an image size that is smaller than the image size of the processed image (i.e., the first size), which is advantageous for reducing the model output dimension and reducing the computational complexity during data processing. The smaller the feature map is, the more remarkable the above-described effect is. For example, if the image size of the processed image is height 320, width 640, the feature map size may be height 10, width 20.

In one possible implementation, the feature map may be a multi-channel image, and the feature map may include a plurality of first channels for storing first information and a plurality of second channels for storing second information. The number of the first channels may be the sum of the second number and 1, and the number of the second channels may be twice the second number. And:

among the plurality of first channels, a first channel corresponding to a first confidence representing whether a feature map has a deceleration strip key point and a second number of first channels corresponding to a second confidence representing whether an nth deceleration strip has a deceleration strip key point may be included;

among the plurality of second channels, a second channel corresponding to a coordinate value representing an nth speed bump in the first coordinate axis and a second channel corresponding to a coordinate value representing an nth speed bump in the second coordinate axis may be included.

Where N may comprise a positive integer from 1 to a second number. The second number may be the maximum number of deceleration strips that the keypoint identification model can identify through one feature map.

For example, if the key point recognition model is set to recognize at most 3 deceleration strips in the process of generating the key point recognition model, N is 3, and accordingly:

The feature map is a 10-channel image, which includes 4 first channels and 6 second channels. The first channel comprises: the method comprises the steps of determining whether a first channel for representing whether a first confidence coefficient of a deceleration strip key point exists in a feature map, determining whether a second confidence coefficient of a 1 st deceleration strip exists in the feature map, determining whether a second confidence coefficient of a 2 nd deceleration strip exists in the feature map, and determining whether a second confidence coefficient of a 3 rd deceleration strip exists in the feature map. The second channel comprises: the second channel corresponding to the coordinate value representing the 1 st speed bump in the first coordinate axis, the second channel corresponding to the coordinate value representing the 1 st speed bump in the second coordinate axis, the second channel corresponding to the coordinate value representing the 2 nd speed bump in the first coordinate axis, the second channel corresponding to the coordinate value representing the 2 nd speed bump in the second coordinate axis, the second channel corresponding to the coordinate value representing the 3 rd speed bump in the first coordinate axis, and the second channel corresponding to the coordinate value representing the 3 rd speed bump in the second coordinate axis.

The first coordinate axis may be an x axis in the image, and the second coordinate axis may be a y axis in the image.

Based on this, the feature map can reflect, not only whether there are deceleration strip key points in the feature map through its first confidence, but also the key points of several deceleration strips through the second confidence, for example, if two of the 3 second confidence levels in the above example can represent that there are deceleration strip key points, and another second confidence level represents that there are no deceleration strip key points, it may be determined that two deceleration strip key points are identified through the feature map. And then, coordinate values are acquired through the corresponding second channels to respectively determine the positions of the key points of the two deceleration strips.

Alternatively, the keypoint identification model may be generated by:

obtaining a training sample;

model training is carried out by taking training images as the input of the model and taking a first number of training feature images as the target output of the model, so that a trained key point recognition model is obtained.

Wherein each training sample may include a training image and a first number of training feature maps corresponding to the training image. The training image is in a target format, one training feature image corresponds to one image area in the training image, and the training feature image is used for indicating training feature information of the image area corresponding to the feature image, wherein the training feature information comprises first label information used for indicating whether a deceleration strip key point exists and second label information used for indicating the position of the deceleration strip key point.

It should be noted that, the training feature map is substantially the same as the description related to the feature map, and the difference is that the feature information in the training stage is known, so that the first confidence coefficient and the second confidence coefficient of the training feature map are 0 (the feature has no deceleration strip key point) or 1 (the feature has deceleration strip key point), and the first confidence coefficient and the second confidence coefficient of the feature map corresponding to the target image generated by the key point recognition model in the actual reasoning stage are real numbers in the [0,1] numerical interval. For the same parts, the description thereof will not be repeated here.

In one possible embodiment, the training image may be obtained by:

acquiring an initial sample;

extracting at least one partial image from the initial image for each initial image;

and processing the local images aiming at each local image to obtain a training image in a target format.

Alternatively, the image in the target format may be the first size.

Each initial sample may include an initial image and deceleration strip marking information corresponding to the initial image, and the deceleration strip marking information may include at least one point set, where the point set is used to indicate a deceleration strip in the initial image.

The initial image may be obtained from a road video acquired during actual driving of the vehicle, for example, image frames may be extracted from the road video at certain time intervals, and the image frames including the deceleration strip may be retained as the initial image.

Deceleration strip marking information may be used to mark no less than two points in the deceleration strip, including its two end points. For example, for a linear deceleration strip, the corresponding set of points in its deceleration strip label information may include two endpoints of the deceleration strip. For another example, for a deceleration strip having a curve, more points are required for the corresponding point set in the deceleration strip mark information, and, in order to ensure accuracy, the greater the degree of curve of the deceleration strip, the greater the number of points in the deceleration strip corresponding to the point set in the deceleration strip mark information should be. In addition, when determining the deceleration strip marking information, the deceleration strip can be subjected to difference processing based on the points marked on the deceleration strip, so that the distance between adjacent points is smaller than 2px, and the subsequent data processing is facilitated. Thus, the distance between adjacent points in the same point set in the deceleration strip flag information is less than 2px.

After the initial sample is acquired, at least one partial image may be extracted from the initial image for each of the initial images in the initial sample. Wherein the partial image includes a deceleration strip. Alternatively, the partial image may be of the first size.

Alternatively, the aspect ratio of the first size may be the same as the initial image.

Optionally, the ratio of the area corresponding to the first size to the image area of the initial image may be greater than a preset ratio, so as to ensure that the partial image can extract as much deceleration strip information in the initial image as possible without excessive omission. The preset ratio may be 1/4, for example.

In one possible implementation, a sliding window of a first size may be generated and the generated sliding window slid over the initial image to extract at least one partial image.

After a plurality of local images are obtained through multiple extractions, the local images can be preprocessed for each local image to obtain a training image in a target format.

The preprocessing method is described in the content of preprocessing the target image, and is not repeated here.

In one possible implementation, the first number of training feature maps corresponding to the training images may be determined by:

extracting local mark information corresponding to a deceleration strip in each local image from target deceleration strip mark information corresponding to an initial image in which the local image is positioned;

And dividing the local images into a first number of training feature images aiming at each local image, and determining first label information and second label information of each training feature image according to the local mark information corresponding to the local images.

Wherein the training feature map may be of a second size. For example, if the image size of the partial image is height 320, width 640, the training feature map may be sized to be a second size (e.g., height 10, width 20).

Wherein the local signature information may include a subset of at least one set of points in the target deceleration strip signature information. The subset of the set of points may comprise a set of elements having a number less than the total number of elements comprised by the set of points, or may comprise the set of points itself.

As shown in fig. 2, which shows a scene when a partial image is extracted from an initial image, a is the initial image, B is a sliding window for extracting the partial image, C1, C2, C3, and C4 are all deceleration strips, assuming that the sliding window extracts the partial image through the current position, it is known from fig. 2 that part of deceleration strip key points of deceleration strips C1 and C2 are located in the partial image, all of deceleration strip key points of deceleration strip C3 are located in the partial image, deceleration strip C4 is not located in the partial image, based on this, from among the point sets included in the target deceleration strip mark information, the point set F1 corresponding to deceleration strip C3 can be extracted, and the subset F3 corresponding to the E1 to E2 part in the point set F2 of deceleration strip C1 can be extracted, and the subset F5 corresponding to the E3 to E4 part in the point set F4 of deceleration strip C2 can be extracted, and thus the partial mark information F1, F3, and F5 can be obtained.

In addition, since the partial image may include a plurality of pieces of deceleration strip information, the influence degrees of the deceleration strips on the vehicle are different, and determining an excessive number of deceleration strips may also reduce the accuracy of identifying the deceleration strips, which is not beneficial to determining the deceleration strips. Therefore, if the number of the point sets in the target deceleration strip mark information exceeds the second number, the deletion processing may be performed on the partial point sets in the target deceleration strip mark information so that the number of the point sets in the deceleration strip mark information does not exceed the second number.

For example, since the road image during the running of the vehicle is processed, the deceleration strip on the y-axis (i.e., the second coordinate axis) can generally represent the distance of the deceleration strip from the vehicle, that is, the smaller the coordinate value of the deceleration strip on the y-axis, the closer the deceleration strip is to the vehicle, the more should be noticed, so when the number of the point sets in the target deceleration strip marking information exceeds the second number, the point sets may be sorted in the y-axis direction, and the second number of the point sets with smaller y-axis coordinate values may be reserved as the target deceleration strip information that finally needs to participate in the data processing. Therefore, the determination of the deceleration strip is more accurate, the processing process can be simplified, the deceleration strip nearer to the vehicle can be preferentially determined, and the vehicle running safety can be improved.

After extracting the local mark information, dividing the local image into a first number of training feature images with a second size for each local image, and determining the first label information and the second label information of each training feature image according to the local mark information corresponding to the local image.

After the partial image is divided into the first number of training feature images, for each deceleration strip, the training feature images passed by the deceleration strip can be marked as the key points of the deceleration strip, namely the first label information, and meanwhile, the second label information can be generated based on the number of the deceleration strips in the training feature images. For the training feature map with the key training points of the deceleration strips, second label information can be further generated for each deceleration strip in the training feature map. For example, boundary points or center points in the training feature map may be used to generate the second tag information.

Fig. 3 shows a scenario in which a partial image is divided into a plurality of training feature maps, in which two deceleration strip key points G1 and G2 of a deceleration strip exist, it is seen that the deceleration strip passes through a grid where G1, H2, G2 are located (i.e., training feature maps), and therefore, tag information corresponding to the deceleration strip can be generated for the training feature maps where G1, H2, G2 are located, while tag information without the deceleration strip exists is generated for the other training feature maps in fig. 3. For example, the label information of the deceleration strip in one training feature diagram shown in fig. 3 may be [1, x_, y_ ], where 1 represents that there is a deceleration strip key point, x_ represents a coordinate value of the deceleration strip key point corresponding to the x axis in the training feature diagram, and y_ represents a coordinate value of the deceleration strip key point corresponding to the y axis in the training feature diagram, which in turn corresponds to the above second confidence coefficient, the coordinate value of the deceleration strip in the first coordinate axis, and the coordinate value of the deceleration strip in the second coordinate axis. Based on this, the related information of the deceleration strip included in each training feature map can be determined.

It should be noted that, the first tag information is the same as the first information principle, and the second tag information is the same as the second information principle, which is not described herein.

According to the method, based on the initial sample, multiple partial images are extracted through multiple sampling, multiple images with deceleration strips are obtained, and therefore large-scale data enhancement can be achieved without multiple marking.

After obtaining the training sample for generating the key point recognition model, the training sample can be used for generating the key point recognition model. That is, the training image is taken as the input of the model, and the model training is performed in a mode that the first number of training feature images are taken as the target output of the model, so that the trained key point recognition model is obtained.

Alternatively, the keypoint identification model may be trained on the basis of a neural network model. For example, a pytorch build deep learning full convolutional neural network may be employed for the generation of the keypoint identification model. The initialized key point identification model can comprise a plurality of convolution layers and finally does not comprise a full connection layer, wherein the last convolution layer normalizes the result by using a sigmoid function, and the rest convolution layers are connected with BN and Relu functions. The BN and the Relu function can better generate a characteristic selection effect, meanwhile, the full-connection layer is not arranged, so that the loss of the characteristics of the full-connection layer can be avoided, the comprehensiveness and the accuracy of characteristic extraction are ensured, and the accuracy of key point identification is improved.

In each model training, an output result obtained by processing an input training image by the model may include a first number of output feature maps, one output feature map may correspond to one image area in the input training image, and one output feature map may be used to indicate output feature information of the image area corresponding to the output feature map, where the output feature information may include third information for indicating whether a deceleration strip key point exists and fourth information for indicating a deceleration strip key point position. The output feature map may refer to the description of the feature map corresponding to the target image in the foregoing, the third information may refer to the description of the first information in the foregoing, and the fourth information may refer to the description of the second information in the foregoing, which is not described herein.

In one model training, the loss value for updating the model may be determined by at least one of:

according to a first training feature map and a first loss value determined by the first feature map, the first training feature map is a training feature map with deceleration strip key points in the training feature map corresponding to a training image used in the model training, and the first feature map is an output feature map corresponding to the first training feature map in the output result of the training;

According to a second training feature map and a second loss value determined by the second feature map, the second training feature map is a training feature map which does not have key points of a deceleration strip in the training feature map corresponding to a training image used in the model training, and the second feature map is an output feature map corresponding to the second training feature map in an output result of the training;

a third loss value is determined according to the first label information and the third information of the training feature map corresponding to the training image used in the model training;

a fourth loss value is determined according to the coordinate value of the deceleration strip key point position indicated by the second label information in the first coordinate axis and the coordinate value of the deceleration strip key point position indicated by the fourth information in the first coordinate axis;

and determining a fifth loss value according to the coordinate value of the deceleration strip key point position indicated by the second label information in the second coordinate axis and the coordinate value of the deceleration strip key point position indicated by the fourth information in the second coordinate axis.

For example, the sum of the first loss value, the second loss value, the third loss value, the fourth loss value, and the fifth loss value may be used as the loss value for updating the model. For another example, a weight may be assigned to each of the loss values, and the loss values for updating the model may be generated by means of weighted summation.

In this way, in the training process of the model, factors possibly causing poor model reasoning performance are considered from multiple angles, and loss values are calculated through different angles so as to update the model, so that the performance of the model is improved, and the accuracy of the key point recognition model is ensured.

Referring back to fig. 1, in step 13, position information of a plurality of key points linearly distributed on the deceleration strip is determined from the feature map.

Each deceleration strip is composed of a plurality of key points distributed in a shape, and the deceleration strips can be determined based on a certain position connection sequence according to the position information of the key points.

In one possible embodiment, step 13 may comprise the steps of:

for each target feature map, determining the position coordinates of the deceleration strip key points in the target feature map, which correspond to the deceleration strip key points in the target image, through the coordinate values of the target channels;

and clustering the deceleration strip key points of which the position coordinates are determined to determine at least one deceleration strip key point sequence so as to determine the position information of a plurality of key points.

As described above, the first confidence characterizes whether a deceleration strip exists in the feature map, and the second confidence characterizes whether an nth deceleration strip exists, i.e., reflects the number of deceleration strips in the feature map. Thus, the first confidence level reaching the first threshold value and the second confidence level reaching the second threshold value, indicating that at least one deceleration strip is present, such a feature map is extracted as a target feature map for further determining the deceleration strip.

As for the feature map whose first confidence coefficient does not reach the first threshold value, whether or not there is the second confidence coefficient that reaches the second threshold value in the feature map, it is explained that there is no deceleration strip in the feature map, and therefore, it can be determined that there is no deceleration strip in such a feature map, and no further processing is necessary.

And determining the position coordinates of the deceleration strip key points in the target feature images corresponding to the target feature images according to the coordinate values of the target channels aiming at each target feature image. Wherein the target channel is a second channel corresponding to the first channel reaching the second threshold. That is, in the case where there is a deceleration strip in the feature map, there may be one or more (depending on the second number), and therefore, it is possible to determine that there are several deceleration strip key points by the second confidence, and extract position information from the corresponding second channel, respectively, for each deceleration strip key point, respectively.

And then, determining at least one deceleration strip key point sequence through a clustering method according to the deceleration strip key points of the determined position coordinates.

In one possible implementation manner, clustering may be performed based on the positions of the deceleration strip key points, so as to cluster out a deceleration strip key point sequence corresponding to at least one deceleration strip, so as to determine position information of a plurality of key points.

In another possible implementation manner, clustering is performed on the deceleration strip keypoints determined to position coordinates to determine at least one deceleration strip keypoint sequence, so as to determine position information of a plurality of keypoints, and the method may include the following steps:

for each target feature map, adding deceleration strip key points for determining the position coordinates into a key point matrix corresponding to the target feature map, wherein the key point matrix comprises non-empty point columns which comprise deceleration strip key points for determining the position coordinates;

Determining a next non-empty point column along the direction of the first coordinate axis as a target point column;

taking each deceleration strip key point in the target point row as a target key point, and executing the following steps: determining the distance between a target key point and a target element in the direction of a second coordinate axis, wherein the target element is a tail element of each existing deceleration strip key point sequence; comparing the minimum value in the determined distance with a preset threshold value; if the minimum value is smaller than a preset threshold value, adding the target key point to the tail part of the deceleration strip key point sequence corresponding to the minimum value; if the minimum value is greater than or equal to a preset threshold value, generating a deceleration strip key point sequence with target key points;

under the condition that a non-empty point column which is not used as a target point column exists in the key point matrix, the method comprises the steps of executing the direction along the first coordinate axis again, determining the next non-empty point column as the target point column, and until the non-empty point column which is not used as the target point column does not exist in the key point matrix;

when a non-empty dot sequence which is not used as a target dot sequence no longer exists in the key dot matrix, position information of a plurality of key dots is determined in an existing deceleration strip key dot sequence.

For example, if the target feature map is multiple, the above-mentioned process may be performed for each target feature map, that is, the position information of multiple key points linearly distributed on the deceleration strip may be determined for each target feature map, or the target feature map may be restored to a large key point matrix according to the respective relative positions, and then the above-mentioned process may be performed, so as to obtain the position information of multiple key points linearly distributed on the deceleration strip. The latter key point matrix is larger than the former key point matrix.

Illustratively, the direction along the first coordinate axis may be a direction along the forward direction of the first coordinate axis.

And for the first non-empty point row, generating a deceleration strip key point sequence with deceleration strip key points according to deceleration strip key points contained in the non-empty point row, namely initializing a plurality of deceleration strips, aiming at each deceleration strip key point. Further, in the next non-empty dot column, the distance between each deceleration strip key point in the next non-empty dot column and the tail element of the generated deceleration strip key point sequence (i.e., the last deceleration strip key point in the deceleration strip key point sequence) in the second coordinate axis direction can be determined, and the minimum value in the distances can be selected. The deceleration strip key point sequence with the calculated minimum value is the sequence to which the deceleration strip key point is most likely to be added, so the minimum value can be compared with a preset threshold value, if the minimum value is smaller than the preset threshold value, the probability that the deceleration strip key point belongs to the deceleration strip key point sequence with the calculated minimum value is extremely high, and therefore the deceleration strip key point sequence can be added to the tail part of the deceleration strip key point sequence, and if the minimum value is larger than or equal to the preset threshold value, the deceleration strip key point still has a certain distance with an existing deceleration strip, and is possibly a new deceleration strip, so that a new deceleration strip can be initialized based on the deceleration strip. The preset threshold value can be set according to actual requirements.

Based on the above processing procedure, each point column in the key point matrix is traversed, and a series of deceleration strip key point queues can be generated based on deceleration strip key points.

Further, the position information of a plurality of key points can be determined in the existing deceleration strip key point sequence.

In one possible implementation, the existing deceleration strip keypoint sequences may each be determined as location information for a plurality of keypoints on the deceleration strip.

In another possible implementation manner, in an existing deceleration strip key point sequence, determining the position information of a plurality of key points may include the following steps:

and taking the deceleration strip key point sequence with the number of elements reaching a preset value as an alternative sequence, and determining the position information of a plurality of key points according to the key points contained in the alternative sequence.

In consideration of the fact that a part of the area in the image is mistakenly identified as a deceleration strip possibly caused by light rays and the like, after the deceleration strip key point sequences are determined, the number of elements in each existing deceleration strip key point sequence can be determined, and the deceleration strip key point sequences with the number of elements reaching a preset value can be determined as position information of a plurality of key points on the deceleration strip. Therefore, in the case of false recognition, the elements in the deceleration strip key point sequence can be deleted because the number of the elements does not reach a preset value, so that the accuracy of deceleration strip determination can be ensured. For example, the preset value may be set to 4.

In step 14, a target deceleration strip in the target image is determined based on the position information of the plurality of key points.

After the position information of the plurality of key points is determined according to the step 13, the key points of the deceleration strip can be connected to form a line representing the deceleration strip, and the deceleration strip in the target image, namely the target deceleration strip, can be further determined based on the formed line representing the deceleration strip. For example, since the deceleration strip has a certain width, after the line representing the deceleration strip is formed, the deceleration strip having a certain width may be determined with the line as an axis (for example, a central axis) to achieve positioning of the deceleration strip in the image.

Fig. 4 is a block diagram of a deceleration strip determination apparatus provided according to one embodiment of the present disclosure. As shown in fig. 4, the apparatus 50 includes:

a first acquiring module 51, configured to acquire a target image to be processed;

a generating module 52, configured to generate a feature map corresponding to the target image using a pre-generated key point recognition model;

a first determining module 53, configured to determine, according to the feature map, position information of a plurality of key points linearly distributed on a deceleration strip;

and a second determining module 54, configured to determine a target deceleration strip in the target image according to the position information of the plurality of key points.

Optionally, the generating module 52 is configured to input a processed image to the keypoint identification model, obtain a first number of feature maps output by the keypoint identification model, one feature map corresponds to one image area in the target image, and one feature map is configured to indicate feature information of the image area corresponding to the feature map, where the feature information includes first information for indicating whether a deceleration strip keypoint exists and second information for indicating a deceleration strip keypoint position, and the processed image is an image in a target format obtained by processing the target image.

Optionally, the keypoint identification model is generated by:

the second acquisition module is used for acquiring training samples, each training sample comprises a training image and a first number of training feature images corresponding to the training images, the training images are in the target format, one training feature image corresponds to one image area in the training images, the training feature images are used for indicating training feature information of the image area corresponding to the feature images, and the training feature information comprises first label information used for indicating whether deceleration strip key points exist and second label information used for indicating the positions of the deceleration strip key points;

and the training module is used for carrying out model training by taking the training image as the input of the model and taking the first number of training feature images as the target output of the model so as to obtain the trained key point identification model.

Optionally, the training image is obtained by:

the third acquisition module is used for acquiring initial samples, each initial sample comprises an initial image and deceleration strip marking information corresponding to the initial image, the deceleration strip marking information comprises at least one point set, and one point set is used for indicating one deceleration strip in the initial image;

The first extraction module is used for extracting at least one local image from the initial image aiming at each initial image, wherein each local image comprises a deceleration strip;

the first processing module is used for processing the local images aiming at each local image to obtain the training image in the target format.

the second extraction module is used for extracting local mark information corresponding to a speed bump in the local image from target speed bump mark information corresponding to an initial image where the local image is located for each local image, wherein the local mark information comprises a subset of at least one point set in the target speed bump mark information;

and the third determining module is used for dividing the local images into a first number of training feature images aiming at each local image, and determining the first label information and the second label information of each training feature image according to the local mark information corresponding to the local image.

Optionally, the method further includes a deleting module, configured to delete a part of the point sets in the target deceleration strip mark information if the number of the point sets in the target deceleration strip mark information exceeds a second number, so that the number of the point sets in the deceleration strip mark information does not exceed the second number.

Optionally, the processed image is obtained by:

the second processing module is used for converting the target image into a gray level image to obtain a first intermediate image;

the third processing module is used for carrying out normalization processing on the first intermediate image to obtain a second intermediate image;

and a fourth processing module, configured to generate a new image according to the second intermediate image through interpolation processing, and take the new image as the processed image.

wherein N comprises a positive integer from 1 to a second number.

Optionally, the first determining module 53 includes:

the first determining submodule is used for determining target feature graphs with first confidence reaching a first threshold and second confidence reaching a second threshold in a first number of feature graphs corresponding to the target image;

the second determining submodule is used for determining the position coordinates of the deceleration strip key points in the target feature diagrams corresponding to the target images according to the coordinate values of the target channels for each target feature diagram, wherein the target channels are second channels corresponding to the first channels reaching the second threshold;

and the third determining submodule is used for carrying out clustering processing on the deceleration strip key points for determining the position coordinates so as to determine at least one deceleration strip key point sequence and determine the position information of the plurality of key points.

Optionally, the third determining sub-module includes:

the adding sub-module is used for adding the deceleration strip key points with the determined position coordinates into a key point matrix corresponding to the target feature map according to each target feature map, wherein the key point matrix comprises a non-empty point row, and the non-empty point row comprises deceleration strip key points with the determined position coordinates;

a fourth determining submodule, configured to determine a first non-null point column in the key point matrix along the direction of the first coordinate axis;

the generation sub-module is used for respectively generating a deceleration strip key point sequence with deceleration strip key points according to deceleration strip key points contained in the first non-empty point row and aiming at each deceleration strip key point;

a fifth determining submodule, configured to determine a next non-null point column along the direction of the first coordinate axis, as a target point column;

and the fifth processing sub-module is used for taking each deceleration strip key point in the target point row as a target key point respectively and executing the following steps: determining the distance between a target key point and a target element in the direction of the second coordinate axis, wherein the target element is a tail element of each existing deceleration strip key point sequence; comparing the minimum value in the determined distance with a preset threshold value; if the minimum value is smaller than the preset threshold value, adding the target key point to the tail part of the deceleration strip key point sequence corresponding to the minimum value; if the minimum value is greater than or equal to the preset threshold value, generating a deceleration strip key point sequence with the target key point;

A circulation sub-module, configured to execute the direction along the first coordinate axis again, and determine a next non-empty point column as a target point column if there is a non-empty point column that is not used as a target point column in the key point matrix, until there is no non-empty point column that is not used as a target point column in the key point matrix;

a sixth determining sub-module, configured to determine, in the existing deceleration strip key point sequence, location information of the plurality of key points when there are no non-empty point columns in the key point matrix that are not used as target point columns.

Optionally, the sixth determining submodule includes:

a seventh determining submodule, configured to determine a number of elements in each existing deceleration strip key point sequence;

and the eighth determining submodule is used for taking a deceleration strip key point sequence with the number of elements reaching a preset value as an alternative sequence and determining the position information of the key points according to the key points contained in the alternative sequence.

The specific manner in which the various modules perform the operations in the apparatus of the above embodiments have been described in detail in connection with the embodiments of the method, and will not be described in detail herein.

The present disclosure also provides an electronic device, including:

a memory having a computer program stored thereon;

and a processor, configured to execute the computer program in the memory, so as to implement the steps of the deceleration strip determining method provided by any embodiment of the disclosure.

The present disclosure also provides a vehicle including the electronic device provided by any of the embodiments of the present disclosure.

Fig. 5 is a block diagram of an electronic device 700, according to an example embodiment. As shown in fig. 5, the electronic device 700 may include: a processor 701, a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input/output (I/O) interface 704, and a communication component 705.

The processor 701 is configured to control the overall operation of the electronic device 700, so as to complete all or part of the steps in the deceleration strip determination method described above. The memory 702 is used to store various types of data to support operation on the electronic device 700, which may include, for example, instructions for any application or method operating on the electronic device 700, as well as application-related data, such as contact data, messages sent and received, pictures, audio, video, and so forth. The Memory 702 may be implemented by any type or combination of volatile or non-volatile Memory devices, such as static random access Memory (Static Random Access Memory, SRAM for short), electrically erasable programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), erasable programmable Read-Only Memory (Erasable Programmable Read-Only Memory, EPROM for short), programmable Read-Only Memory (Programmable Read-Only Memory, PROM for short), read-Only Memory (ROM for short), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia component 703 can include a screen and an audio component. Wherein the screen may be, for example, a touch screen, the audio component being for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 702 or transmitted through the communication component 705. The audio assembly further comprises at least one speaker for outputting audio signals. The I/O interface 704 provides an interface between the processor 701 and other interface modules, which may be a keyboard, mouse, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, bluetooth, near field communication (Near Field Communication, NFC for short), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or one or a combination of more of them, is not limited herein. The corresponding communication component 705 may thus comprise: wi-Fi module, bluetooth module, NFC module, etc.

In an exemplary embodiment, the electronic device 700 may be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as ASIC), digital signal processors (Digital Signal Processor, abbreviated as DSP), digital signal processing devices (Digital Signal Processing Device, abbreviated as DSPD), programmable logic devices (Programmable Logic Device, abbreviated as PLD), field programmable gate arrays (Field Programmable Gate Array, abbreviated as FPGA), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described speed bump determination method.

In another exemplary embodiment, a computer readable storage medium is also provided that includes program instructions that, when executed by a processor, implement the steps of the deceleration strip determination method described above. For example, the computer readable storage medium may be the memory 702 including program instructions described above, which are executable by the processor 701 of the electronic device 700 to perform the deceleration strip determination method described above.

In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable apparatus, the computer program having code portions for performing the above-described deceleration strip determination method when executed by the programmable apparatus.

The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, but the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solutions of the present disclosure within the scope of the technical concept of the present disclosure, and all the simple modifications belong to the protection scope of the present disclosure.

In addition, the specific features described in the above embodiments may be combined in any suitable manner without contradiction. The various possible combinations are not described further in this disclosure in order to avoid unnecessary repetition.

Moreover, any combination between the various embodiments of the present disclosure is possible as long as it does not depart from the spirit of the present disclosure, which should also be construed as the disclosure of the present disclosure.

Claims

1. A deceleration strip determination method, characterized in that the method comprises:

acquiring a target image to be processed;

inputting a processed image into a key point identification model to obtain a first number of feature images output by the key point identification model, wherein one feature image corresponds to one image area in the target image, and the one feature image is used for indicating feature information of the image area corresponding to the feature image, wherein the feature information comprises first information for indicating whether a deceleration strip key point exists or not and second information for indicating the position of the deceleration strip key point, and the processed image is an image in a target format obtained by processing the target image;

determining a target deceleration strip in the target image according to the position information of the key points;

the feature map is a multi-channel image, the feature map comprises a plurality of first channels for storing first information and a plurality of second channels for storing second information, the number of the first channels is the sum of a second number and 1, the number of the second channels is twice the second number, and the second number is the maximum number of deceleration strips which can be identified by the key point identification model through one feature map; among the plurality of first channels, a first channel corresponding to a first confidence representing whether a feature map has a deceleration strip key point and a second number of first channels corresponding to a second confidence representing whether an Nth deceleration strip has a deceleration strip key point are included; the plurality of second channels comprise a second channel corresponding to the coordinate value of the Nth deceleration strip in the first coordinate axis and a second channel corresponding to the coordinate value of the Nth deceleration strip in the second coordinate axis; n comprises a positive integer from 1 to a second number.

2. The method of claim 1, wherein the keypoint identification model is generated by:

3. The method according to claim 2, characterized in that the training image is obtained by:

4. A method according to claim 3, wherein the first number of training feature maps corresponding to the training image is determined by:

5. The method according to claim 4, wherein if the number of the dot sets in the target deceleration strip flag information exceeds a second number, deleting the partial dot sets in the target deceleration strip flag information so that the number of the dot sets in the deceleration strip flag information does not exceed the second number.

6. The method according to claim 2, wherein in each model training, an output result obtained by processing an input training image by the model includes a first number of output feature maps, one of the output feature maps corresponds to one image area in the input training image, and one of the output feature maps is used for indicating output feature information of the image area corresponding to the output feature map, wherein the output feature information includes third information for indicating whether a deceleration strip key point exists and fourth information for indicating a deceleration strip key point position;

7. The method according to claim 1, wherein the processed image is obtained by:

normalizing the first intermediate image to obtain a second intermediate image;

8. The method according to claim 1, wherein determining the position information of the plurality of key points linearly distributed on the deceleration strip according to the feature map includes:

9. The method of claim 8, wherein the clustering the deceleration strip keypoints that determine the position coordinates to determine at least one deceleration strip keypoint sequence to determine position information for the plurality of keypoints comprises:

for each target feature map, adding deceleration strip key points with determined position coordinates into a key point matrix corresponding to the target feature map, wherein the key point matrix comprises non-empty point columns, and the non-empty point columns comprise deceleration strip key points with determined position coordinates;

10. The method of claim 9, wherein determining the location information of the plurality of keypoints in the existing deceleration strip keypoint sequence comprises:

11. A deceleration strip determination apparatus, characterized in that the apparatus comprises:

the generation module is used for inputting a processed image into the key point recognition model to obtain a first number of feature images output by the key point recognition model, wherein one feature image corresponds to one image area in the target image, and the other feature image is used for indicating feature information of the image area corresponding to the feature image, wherein the feature information comprises first information for indicating whether a deceleration strip key point exists or not and second information for indicating the position of the deceleration strip key point, and the processed image is an image in a target format obtained by processing the target image;

the second determining module is used for determining a target deceleration strip in the target image according to the position information of the plurality of key points;

12. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the steps of the method of any of claims 1-10.

13. An electronic device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the method of any one of claims 1-10.

14. A vehicle comprising the electronic device of claim 13.