CN114140773A - Ground identification category confirmation method and device and vehicle - Google Patents
Ground identification category confirmation method and device and vehicle Download PDFInfo
- Publication number
- CN114140773A CN114140773A CN202111463157.4A CN202111463157A CN114140773A CN 114140773 A CN114140773 A CN 114140773A CN 202111463157 A CN202111463157 A CN 202111463157A CN 114140773 A CN114140773 A CN 114140773A
- Authority
- CN
- China
- Prior art keywords
- ground
- category
- image
- top view
- identifier
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The application provides a method and a device for confirming the category of a ground identifier and a vehicle, and relates to the technical field of computers. Wherein, the method comprises the following steps: acquiring an image comprising a ground identifier to be determined; performing inverse perspective transformation on the image to acquire a top view corresponding to the image; and confirming the category of the ground identifier to be determined by using the top view and the ground identifier recognition model, wherein the ground identifier recognition model is trained by using a top view set, and each top view in the top view set is obtained by performing inverse perspective transformation on an image comprising the ground identifier with known category. By utilizing the method provided by the application, the detection capability of the ground mark is improved, particularly the detection capability of the ground mark at the far end is improved, and further better technical support is provided for path planning, real-time decision and driving control in the subsequent automatic driving process, and the method has higher practicability.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for determining a category of a ground identifier, and a vehicle.
Background
In the automatic driving technology, environmental perception is used as a first link, and the core of the automatic driving technology lies in that a vehicle can better simulate the perception capability of a human driver and accurately understand the surrounding driving conditions. In practical application, the detection of the ground identification of the vehicle driving road is accurately realized, and the method has important significance for subsequent path planning, real-time decision and driving control.
Currently, the class identification method for the ground identifier is generally a method based on a Region of Interest (ROI) and artificial features, such as a ground identifier horizontal projection class identification method. The method uses Histogram of Oriented (HOG) to extract features from candidate ground tags, and then uses a Total Error Rate (TER) classifier to classify the candidate ground tags.
However, this method is only suitable for processing the situation when the ground identifier is located at the near end of the current vehicle, because the position of the camera device of the vehicle is fixed and is affected by the camera angle of view of the camera device, for the ground identifier located at the far end of the vehicle, the candidate ground identifier obtained by horizontal projection detection generally has severe deformation and distortion, which makes it difficult to accurately determine the type of the ground identifier. Therefore, the ground identification detection capability of the method is poor, the ground identification detection can be realized only nearby, and the practicability is low.
Disclosure of Invention
In order to solve the technical problem, the application provides a method and a device for confirming the category of a ground identifier and a vehicle, which improve the detection capability of the ground identifier, especially the detection capability of a far-end ground identifier, and have higher practicability.
In a first aspect, the present application provides a method for identifying a category of a ground identifier, where the method includes: acquiring an image comprising a ground identifier to be determined; performing Inverse Perspective transformation (IPM) on the image to acquire a top view corresponding to the image; and confirming the category of the ground identifier to be determined by utilizing the top view and the ground identifier recognition model, wherein the ground identifier recognition model is trained by utilizing a top view set, and each top view in the top view set is obtained by carrying out inverse perspective transformation on an image comprising the ground identifier with known category.
By utilizing the method provided by the application, the image comprising the ground mark to be determined is subjected to inverse perspective transformation, the characteristics of the ground mark to be determined can be better reserved on the obtained top view through the inverse perspective transformation, the influence of the height and the visual angle of a shooting device on a vehicle on the characteristics of the ground mark is reduced, and the ground mark recognition model is effectively recognized. And when the ground identification recognition model is trained, the top view is obtained by utilizing the inverse perspective transformation for training, so that the accuracy of the recognition type of the ground identification recognition model is improved. By utilizing the scheme, the detection capability of the ground mark is improved, particularly the detection capability of the far-end ground mark is improved, and further better technical support is provided for path planning, real-time decision and driving control in the follow-up automatic driving process, so that the method has higher practicability.
In a possible implementation manner, performing inverse perspective transformation on an image to obtain a top view corresponding to the image specifically includes:
and performing inverse perspective transformation on the image by using the camera calibration file to acquire a top view corresponding to the image.
In a possible implementation manner, the determining, by using the top view and the ground identifier recognition model, the category of the ground identifier to be determined specifically includes:
extracting a characteristic diagram corresponding to the ground identifier to be determined from the top view by using a ground identifier recognition model;
and confirming that the category of the ground identifier to be determined is a first category by using the ground identifier recognition model and the characteristic diagram, and when the prediction confidence coefficient of the first category is greater than or equal to a preset threshold value, taking the first category as the category of the ground identifier to be determined.
In one possible implementation, the method further includes:
and when the prediction confidence of the first category is smaller than a preset threshold, determining that the category confirmation result of the ground identifier to be determined is unknown.
In one possible implementation, the ground identity recognition model is YOLOV3, and the backbone network backbone of the ground identity recognition model is Xception.
In a possible implementation manner, after confirming the category of the ground identifier to be determined, the method further includes:
the category of the ground identification to be determined is displayed on a display device.
In a second aspect, the present application further provides a ground identifier detection device, where the category confirmation device includes: the device comprises an acquisition unit, a transformation unit and a recognition unit. The acquisition unit is used for acquiring an image comprising a ground mark to be determined. The transformation unit is used for carrying out inverse perspective transformation on the image so as to acquire a corresponding top view of the image. The recognition unit is used for confirming the category of the ground mark to be determined by utilizing the top view and the ground mark recognition model, the ground mark recognition model is trained by utilizing a top view set, and each top view in the top view set is obtained by carrying out inverse perspective transformation on an image comprising the ground mark with known category.
The device that this application provided, through the transform unit to including waiting to confirm the image of ground sign to carry out the inverse perspective transform, through the inverse perspective transform, can keep the characteristic of waiting to confirm ground sign better on the plan view that obtains, the influence of the height that has reduced ground sign and the visual angle that the characteristic received shooting device on the vehicle to make ground sign recognition model carry out effectual discernment. And when the ground identification recognition model is trained, the top view is obtained by utilizing the inverse perspective transformation for training, so that the accuracy of the recognition type of the ground identification recognition model is improved. By utilizing the device, the detection capability of the ground mark is improved, particularly the detection capability of the far-end ground mark is improved, and further better technical support is provided for path planning, real-time decision and driving control in the follow-up automatic driving process, so that the device has higher practicability.
In a possible implementation manner, the transformation unit is specifically configured to perform inverse perspective transformation on the image by using the camera calibration file to obtain a corresponding top view of the image.
In a possible implementation manner, the recognition unit is specifically configured to extract a feature map corresponding to the ground identifier to be determined from a top view by using a ground identifier recognition model; and confirming that the category of the ground identifier to be determined is a first category by using the ground identifier recognition model and the characteristic diagram, and when the prediction confidence coefficient of the first category is greater than or equal to a preset threshold value, taking the first category as the category of the ground identifier to be determined.
In a possible implementation manner, the identification unit further specifically determines that the class confirmation result of the ground identifier to be determined is unknown when the prediction confidence of the first class is smaller than a preset threshold.
In one possible implementation, the category confirmation apparatus further includes a display unit. The display unit is used for displaying the category of the ground mark to be determined on the display equipment.
In a third aspect, the present application further provides a vehicle including the photographing device and the category identifying device of the ground identifier provided in the above implementation manner. The shooting device is used for shooting an image including the ground mark to be determined and transmitting the image to the category confirmation device of the ground mark.
Drawings
FIG. 1 is a schematic diagram of a vehicle undergoing ground mark detection;
fig. 2 is a flowchart of a method for identifying a category of a ground identifier according to an embodiment of the present disclosure;
FIG. 3 is an image including a ground identification to be determined provided by an embodiment of the present application;
FIG. 4 is a top view of FIG. 3 after an inverse perspective transformation;
fig. 5 is a flowchart of another method for identifying a category of a ground identifier according to an embodiment of the present application;
fig. 6 is a schematic diagram of a category identification device for ground identification according to an embodiment of the present disclosure;
fig. 7 is a schematic diagram of another category identification device for ground identification according to an embodiment of the present disclosure;
fig. 8 is a schematic view of a vehicle according to an embodiment of the present application.
Detailed Description
In order to make the technical solution more clearly understood by those skilled in the art, an application scenario of the technical solution of the present application is first described below.
Referring to fig. 1, a schematic diagram of a vehicle for ground mark detection is shown.
The ground mark in the following description of the present application refers to lines, marks, characters, and the like drawn on a road surface on which a vehicle travels, for indicating the travel of the vehicle, and is distinguished from a signboard arranged perpendicular to the ground, on which the ground mark is horizontally drawn.
Common ground marks are, for example, straight marks, turning marks, parking prohibition marks and the like.
When the vehicle is automatically driven, the detection of the ground identification of the driving road of the vehicle is accurately realized, and the method has important significance for subsequent path planning, real-time decision and driving control.
The method for identifying the ground mark is generally a method based on ROI and artificial feature, such as ground mark horizontal projection type identification method. The method utilizes the HOG to extract features from the candidate ground identifications, and then utilizes the total error rate TER classifier to classify the candidate ground identifications.
However, the above method is only suitable for processing the situation when the ground mark is located at the near end of the current vehicle, for example, for the vehicle 11 in the figure, when the ground mark in front of the vehicle is located at the near end of the vehicle, the feature of the ground mark is obvious, and therefore the recognition can be normally performed.
However, for the ground mark at the far end of the vehicle, the detection capability of this method is significantly reduced, for example, for the vehicle 10 in the figure, when the ground mark at the left front is recognized, because the ground mark is at the far end of the vehicle 10, the candidate ground mark detected by horizontal projection generally has severe deformation and distortion, which makes it difficult to accurately determine the type of the ground mark, and the candidate ground mark can only be recognized when the vehicle 10 continues to drive to approach the ground mark. This may cause that only the near-end ground identifier can be always recognized during the automatic driving of the vehicle, leaving a short action time for subsequent path planning, real-time decision-making and driving control, which may result in the untimely subsequent path planning, real-time decision-making and driving control.
Therefore, the ground identification detection capability of the method is poor, the ground identification detection can be realized only nearby, and the practicability is low.
In order to solve the above problem, embodiments of the present application provide a method and an apparatus for confirming a category of a ground identifier, and a vehicle. The method carries out inverse perspective transformation on the image including the ground mark to be determined, can better reserve the characteristics of the ground mark to be determined on the obtained top view through the inverse perspective transformation, and reduces the influence of the height and the visual angle of a shooting device on a vehicle on the characteristics of the ground mark so as to effectively identify the ground mark identification model. And when the ground identification recognition model is trained, the top view is obtained by utilizing the inverse perspective transformation for training, so that the accuracy of the recognition category of the ground identification recognition model is improved. By utilizing the scheme, the detection capability of the ground mark is improved, particularly the detection capability of the far-end ground mark is improved, and further better technical support is provided for path planning, real-time decision and driving control in the follow-up automatic driving process, so that the method has higher practicability.
In order to make the technical solutions more clearly understood by those skilled in the art, the technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
The terms "first", "second", and the like in the description of the present application are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implying any number of technical features indicated
The embodiment of the present application provides a method for identifying a category of a ground identifier, which is specifically described below with reference to the accompanying drawings.
Referring to fig. 2, the figure is a flowchart of a method for identifying a category of a ground identifier according to an embodiment of the present application.
In an implementation manner, the scheme provided by the embodiment of the application can be applied to a scene that the vehicle automatically confirms the type of the ground identifier in the automatic driving process; in another possible implementation manner, the method can also be applied to a non-automatic driving scene of the vehicle, namely, a driver actively drives the vehicle, and the ground identifier is recognized by the method and then displayed or prompted, so that the driver knows the ground identifier, and the decision making of the driver is further assisted.
The method is described below by taking as an example the application of the method in an automatic driving process, and the method comprises the following steps:
s201: an image including a ground identity to be determined is acquired.
The vehicle comprises a shooting device, wherein the shooting device is used for shooting the road condition in front of the driving direction of the vehicle, and then the image of the ground mark to be determined can be obtained in real time.
In one possible implementation, the shooting device is configured to shoot a video, intercept a video frame from the video, and further obtain an image including the ground identifier to be determined.
The ground marks in the embodiment of the application refer to marks and the like which are drawn on a running road surface of a vehicle and used for indicating the running of the vehicle, such as straight running marks, turning marks, parking forbidding marks and the like, and determine the types of the ground marks, that is, which type of marks the ground marks are specifically, and each type of ground marks represents indication information of the running of the vehicle.
For example, a left turn type ground sign for indicating that the road ahead of the vehicle can be turned left; and the straight-going class mark is used for indicating the vehicle to go straight on the road.
It is understood that there may be some difference in marks indicating the same information drawn on the actual road surface, for example, a difference in size, a difference between proportions of parts of the marks, a difference in color of the marks, or a difference in wear condition, so in the embodiment of the present application, the marks indicating the same information are determined to be a type of mark, and when the type of the ground mark is determined, the information indicated by the ground mark is determined.
S202: and carrying out inverse perspective transformation on the image to acquire a corresponding top view of the image.
S203: and confirming the category of the ground mark to be determined by utilizing the top view and the ground mark recognition model.
The ground logo recognition model is trained by using a top view set, wherein each top view in the top view set is obtained by Inverse Perspective transformation (IPM) from an image comprising ground logos with known types.
The inventor finds out through tests and comparisons that the best characteristics of the ground mark cannot be obtained if the shot image is simply and directly used. Similarly, the training of the ground identification recognition model by directly using the acquired images is limited by the practical problems of camera height, vehicle view angle and the like, so that the ground identification recognition model cannot learn the significant features of the ground identification, and further cannot provide sufficient technical support for automatic driving.
The method creatively uses the top view to train the model, and solves the problem that the ground identification cannot be effectively identified due to the camera imaging principle. The following description is made with reference to examples.
Referring to fig. 3, an image including a ground mark to be determined is provided according to an embodiment of the present application.
At this time, because the ground mark to be determined is located at the far-end position in front of the vehicle, the feature display of the ground mark to be determined is not obvious in the image acquired by the vehicle, and the ground mark to be determined is easily confused with the lane line, so that when the detection and identification are directly carried out by using the still image, the error identification is easy to occur or the correct type is difficult to identify.
Referring to fig. 4, it is a top view of fig. 3 after inverse perspective transformation.
The embodiment of the application provides a high scheme, and the image is subjected to inverse perspective transformation to obtain a corresponding top view. After the image under the scene is subjected to inverse perspective transformation, the ground identification characteristics can be well kept on a top view, the deformation is reduced, and then the ground identification model can be effectively identified.
In summary, with the method provided by the present application, the image including the ground identifier to be determined is subjected to inverse perspective transformation, and through the inverse perspective transformation, the features of the ground identifier to be determined can be better retained on the obtained top view, so that the influence of the height and the angle of view of the shooting device on the vehicle on the features of the ground identifier is reduced, and the ground identifier recognition model can perform effective recognition. And when the ground identification recognition model is trained, the top view is obtained by utilizing inverse perspective transformation for training, so that the accuracy of the recognition category of the ground identification recognition model is improved, and the robustness is strong when the ground identification recognition model is applied to a complex real scene. By using the method, the detection capability of the ground mark is improved, particularly the detection capability of the remote ground mark is improved, and further better technical support is provided for path planning, real-time decision and driving control in the subsequent automatic driving process, so that the method has higher practicability.
The following description is made with reference to specific implementations.
Referring to fig. 5, this figure is a flowchart of another method for identifying a category of a ground identifier according to an embodiment of the present application.
The method comprises the following steps:
s501: an image including a ground identity to be determined is acquired with a camera.
S502: and performing inverse perspective transformation on the image by using the camera calibration file to acquire a top view corresponding to the image.
S503: and extracting a characteristic diagram corresponding to the ground identifier to be determined from the top view by using the ground identifier recognition model.
The ground identification model is a convolutional neural network model, automatically defines the characteristic types required by the pavement identification, and realizes the real-time detection and the class output of the ground identification through deep learning.
The ground identification recognition model is specifically YOLOV3 (young Only Look Once V3, a model for fast and accurate real-time object detection), and the backbone network (backbone) of the ground identification recognition model is Xception.
The backbone network is a network used for feature extraction in the model, and is generally used for extracting image information at the front end to generate a feature map for the use of the following network.
Xception is a feature extraction network implemented with deep separable convolution. The deep separable convolution is an algorithm obtained by improving standard convolution in a convolutional neural network, and by splitting the correlation between space dimensionality and channel dimensionality, the number of parameters of convolution calculation is reduced, and the use efficiency of convolution kernel parameters is improved
S504: and confirming that the category of the ground identifier to be determined is a first category by using the ground identifier recognition model and the characteristic diagram, and determining whether the prediction confidence coefficient of the first category is greater than or equal to a preset threshold value.
If yes, executing S505; otherwise, S506 is performed.
S505: and taking the first category as the category of the ground identification to be determined.
And when the prediction confidence of the first category is greater than or equal to a preset threshold, the determined first category is credible, and the result is taken as the category of the ground identifier to be determined. The prediction confidence may be set according to actual conditions, and the embodiment of the present application is not particularly limited.
S506: and determining that the class confirmation result of the ground identifier to be determined is unknown.
And when the prediction confidence of the first category is smaller than a preset threshold, the surface cannot determine the category of the ground identifier to be determined according to the image.
In some implementations of the above-described embodiments,
it should be understood that the above steps in the embodiments of the present application are only for convenience of description, and do not form a limitation on the technical solution of the present application, and in practical applications, the above solution may be adjusted according to practical situations, for example, when the above solution further includes the following steps:
and after the category of the ground identifier to be determined is confirmed, displaying the category of the ground identifier to be determined on display equipment.
The steps can be applied to an automatic driving scene or a non-automatic driving scene, for example, when non-automatic driving is carried out, a driver can accurately acquire the type of the ground mark in advance on the display equipment of the vehicle, and then the driving strategy is made in time.
The ground identifier recognition model provided in the embodiment of the present application is trained using a top view obtained after inverse perspective transformation, and a training process of the model is specifically described below.
First, the top view acquisition method used for training the model will be described.
And acquiring a ground identification image of the real scene by using a driving camera. In order to improve the training effect, the ground identification images under different shooting conditions need to be acquired, and the diversity of training data is increased. Wherein the different photographing conditions include different lane conditions, different weather conditions, and different timestamp conditions.
The lane condition of difference, the visual angle and the deformation degree that influence the image, different weather condition and timestamp condition, the definition degree and the light and shade degree etc. that influence the image.
And classifying and sorting the images of each ground mark collected under different lane conditions, different weather conditions and different time stamp conditions into the same folder.
And generating an image corresponding to the top view by using the acquired image and combining a calibration file of the corresponding camera in an inverse perspective transformation mode. The specific implementation of the inverse perspective transformation is a mature technology, and is not described herein again.
Marking the generated top view image with a target frame according to different ground marks and marking corresponding categories. The target frame is the minimum circumscribed rectangle of the ground mark, and the categories are marks such as straight line, left turn, right turn and the like.
The following describes the generation process of model training data.
Firstly, generating a rectangular frame of a target area including a ground mark, and randomly expanding the rectangular frame of the ground mark in proportion according to a model training strategy.
Aiming at the existing data distribution, data are balanced, the principle is that the color of an image cannot be influenced, and the shape of a ground mark in the image cannot be influenced, and the data preprocessing flow can be as follows: traversing a training data set, iteratively inputting a ground identification top view image and an annotation file, carrying out data equalization according to data conditions, and storing the image and the annotation file after the data equalization.
Based on the consideration of algorithm embedded deployment, the ground identification recognition model is specifically Yolov3, and the backbone network of the ground identification recognition model is Xconvergence.
The backbone network is a network used for feature extraction in the model, and is generally used for extracting image information at the front end to generate a feature map for the use of the following network.
And after the framework of the ground identification recognition model is confirmed, training the ground identification recognition model by using the training data.
Through the mode, the ground identification recognition model obtained by training is used for inputting the top view obtained by carrying out inverse perspective transformation on the image including the ground identification to be determined as the model in practical application, so that the accuracy of the recognition category of the ground identification recognition model is improved, the robustness is strong when the ground identification recognition model is applied to a complex real scene, and the ground identification recognition model has higher practicability.
Based on the method for confirming the category of the ground identifier provided by the above embodiment, the embodiment of the present application further provides a detection apparatus for the ground identifier, which is specifically described below with reference to the accompanying drawings.
Referring to fig. 6, the figure is a schematic view of a category identification device for ground identification according to an embodiment of the present application.
The category identifying device of the illustrated ground sign includes an acquisition unit 601, a conversion unit 602, and a recognition unit 603.
The obtaining unit 601 is configured to obtain an image including a ground identifier to be determined.
The transformation unit 602 is configured to perform an inverse perspective transformation on the image to obtain a corresponding top view of the image.
The recognition unit 603 is configured to identify the category of the ground identifier to be determined using the top view and the ground identifier recognition model.
The ground mark recognition model is trained by utilizing an overhead view set, and each overhead view in the overhead view set is obtained by performing inverse perspective transformation on an image comprising the ground mark with known category.
Further, the transformation unit 602 is specifically configured to perform inverse perspective transformation on the image by using the camera calibration file to obtain a top view corresponding to the image.
The recognition unit 603 is specifically configured to extract a feature map corresponding to the ground identifier to be determined from the top view by using the ground identifier recognition model; and confirming that the category of the ground identifier to be determined is a first category by using the ground identifier recognition model and the characteristic diagram, and when the prediction confidence coefficient of the first category is greater than or equal to a preset threshold value, taking the first category as the category of the ground identifier to be determined. The identifying unit 603 further specifically determines that the class confirmation result of the ground identifier to be determined is unknown when the prediction confidence of the first class is smaller than the preset threshold.
Referring to fig. 7, the figure is a schematic view of another category identification device for ground identification according to an embodiment of the present application.
The apparatus shown in fig. 7 differs from that shown in fig. 6 in that it further includes a display unit 604. The display unit 604 is configured to display the category of the ground identifier to be determined on the display device, so as to prompt the driver.
The device for confirming the ground mark type comprises a processor and a memory, wherein the acquisition unit, the transformation unit, the identification unit, the display unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more, and the class confirmation of the ground identification is realized by adjusting kernel parameters.
The embodiment of the application also provides a storage medium, wherein a program is stored on the storage medium, and the program realizes the category confirmation method of the ground identification when being executed by a processor.
The embodiment of the application provides a processor, wherein the processor is used for running a program, and the program executes the method for confirming the category of the ground identifier when running.
The embodiment of the invention provides electronic equipment, which comprises a bus, a processor and a memory. The processor and the memory complete mutual communication through the bus. The processor is used for calling the program instructions in the memory to execute the above-mentioned ground identification class confirmation method. The electronic device herein may be a vehicle-mounted computer, an Automatic Driving Control Unit (ADCU), and the like, and the embodiments of the present application are not limited in particular. In addition, the number of the controller and the memory may be one or more.
In a typical configuration, an electronic device includes one or more processors (CPUs), memory, and a bus. The device may also include input/output interfaces, network interfaces, and the like.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
Based on the method for confirming the category of the ground identifier and the device for detecting the ground identifier provided by the above embodiments, the embodiments of the present application further provide a vehicle, which is specifically described below with reference to the accompanying drawings.
Referring to fig. 8, the figure is a schematic view of a vehicle according to an embodiment of the present application.
The illustrated vehicle 800 includes a camera 801 and a category identifying device 802 for ground identification.
The shooting device 801 is configured to obtain an image including a ground identifier to be determined by shooting, and transmit the image to the category confirmation device 802 of the ground identifier.
The camera 801 may be a driving camera of a vehicle.
For specific working principle and implementation of the category confirmation device 802, reference may be made to the relevant description in the above embodiments, and the principle in the category confirmation method that may be combined with the ground identifier, which is not described herein again in this embodiment of the present application.
The vehicle that this application embodiment provided can be gasoline vehicle, diesel vehicle or new forms of energy vehicle, and this application embodiment does not do specifically and restricts.
In summary, the vehicle provided in the embodiment of the present application includes the category identification device of the ground identifier provided in the foregoing implementation manner. The device carries out inverse perspective transformation on the image comprising the ground mark to be determined through the transformation unit, and can better reserve the characteristics of the ground mark to be determined on the obtained top view through the inverse perspective transformation, thereby reducing the influence of the height and the visual angle of a shooting device on a vehicle on the characteristics of the ground mark so as to effectively identify the ground mark identification model. And when the ground identification recognition model is trained, the top view is obtained by utilizing the inverse perspective transformation for training, so that the accuracy of the recognition type of the ground identification recognition model is improved. By utilizing the device, the detection capability of the ground mark is improved, particularly the detection capability of the far-end ground mark is improved, and further better technical support is provided for path planning, real-time decision and driving control in the follow-up automatic driving process of the vehicle, so that the device has higher practicability.
It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. The above-described apparatus embodiments are merely illustrative, and units and modules illustrated as separate components may or may not be physically separate. In addition, some or all of the units and modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
The foregoing is merely a detailed description of the present application, and it should be noted that modifications and embellishments could be made by those skilled in the art without departing from the principle of the present application, and these should also be considered as the protection scope of the present application.
Claims (10)
1. A method for category identification of a ground identifier, the method comprising:
acquiring an image comprising a ground identifier to be determined;
performing inverse perspective transformation on the image to acquire a top view corresponding to the image;
and confirming the category of the ground identifier to be determined by using the top view and the ground identifier recognition model, wherein the ground identifier recognition model is trained by using a top view set, and each top view in the top view set is obtained by performing inverse perspective transformation on an image comprising the ground identifier with known category.
2. The method for confirming the category of the ground identifier according to claim 1, wherein the performing inverse perspective transformation on the image to obtain the top view corresponding to the image specifically includes:
and carrying out inverse perspective transformation on the image by utilizing a camera calibration file so as to obtain a top view corresponding to the image.
3. The method for confirming the category of the ground identifier according to claim 1, wherein the confirming the category of the ground identifier to be determined by using the overhead view and the ground identifier recognition model specifically comprises:
extracting a feature map corresponding to the ground identifier to be determined from the top view by using the ground identifier recognition model;
and confirming that the category of the ground identifier to be determined is a first category by using the ground identifier recognition model and the feature map, and when the prediction confidence coefficient of the first category is greater than or equal to a preset threshold value, taking the first category as the category of the ground identifier to be determined.
4. The method for class validation of ground identifications as claimed in claim 4, further comprising:
and when the prediction confidence of the first category is smaller than the preset threshold, determining that the category confirmation result of the ground identifier to be determined is unknown.
5. The method of claim 1, wherein the ground identity recognition model is YOLOV3, and the backbone network backbone of the ground identity recognition model is Xception.
6. The method for confirming the category of the ground identifier according to any one of claims 1 to 5, wherein after confirming the category of the ground identifier to be confirmed, the method further comprises:
and displaying the category of the ground identifier to be determined on a display device.
7. A category identification device for a ground sign, the category identification device comprising: an acquisition unit, a transformation unit and an identification unit;
the acquisition unit is used for acquiring an image comprising a ground identifier to be determined;
the transformation unit is used for carrying out inverse perspective transformation on the image so as to acquire a top view corresponding to the image;
the identification unit is used for confirming the category of the ground identifier to be determined by utilizing the top view and the ground identifier identification model, the ground identifier identification model is trained by utilizing a top view set, and each top view in the top view set is obtained by performing inverse perspective transformation on an image comprising the ground identifier with known category.
8. The apparatus for confirming the category of a ground sign according to claim 7, wherein the transforming unit is specifically configured to perform an inverse perspective transformation on the image by using a camera calibration file to obtain a top view corresponding to the image.
9. The apparatus according to claim 7, wherein the recognition unit is specifically configured to extract a feature map corresponding to the to-be-determined ground identifier from the top view by using the ground identifier recognition model; and confirming that the category of the ground identifier to be determined is a first category by using the ground identifier recognition model and the feature map, and when the prediction confidence coefficient of the first category is greater than or equal to a preset threshold value, taking the first category as the category of the ground identifier to be determined.
10. A vehicle characterized in that the vehicle comprises a camera and a category confirmation device of the ground sign of any one of claims 7 to 9;
the shooting device is used for shooting the image including the ground identifier to be determined and transmitting the image to the category confirmation device of the ground identifier.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111463157.4A CN114140773A (en) | 2021-12-02 | 2021-12-02 | Ground identification category confirmation method and device and vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111463157.4A CN114140773A (en) | 2021-12-02 | 2021-12-02 | Ground identification category confirmation method and device and vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114140773A true CN114140773A (en) | 2022-03-04 |
Family
ID=80387312
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111463157.4A Pending CN114140773A (en) | 2021-12-02 | 2021-12-02 | Ground identification category confirmation method and device and vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114140773A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114626462A (en) * | 2022-03-16 | 2022-06-14 | 小米汽车科技有限公司 | Pavement mark recognition method, device, equipment and storage medium |
-
2021
- 2021-12-02 CN CN202111463157.4A patent/CN114140773A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114626462A (en) * | 2022-03-16 | 2022-06-14 | 小米汽车科技有限公司 | Pavement mark recognition method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103824066B (en) | A kind of licence plate recognition method based on video flowing | |
| EP4109331A1 (en) | Obstacle detection method and apparatus, computer device, and storage medium | |
| CN107563372B (en) | License plate positioning method based on deep learning SSD frame | |
| CN110659539B (en) | Information processing method and device and machine-readable storage medium | |
| US11205276B2 (en) | Object tracking method, object tracking device, electronic device and storage medium | |
| US9454692B2 (en) | Method for identifying and positioning building using outline region restraint of mountain | |
| US20180018528A1 (en) | Detecting method and device of obstacles based on disparity map and automobile driving assistance system | |
| KR101778724B1 (en) | Device and method for reconition of road markings | |
| Ohgushi et al. | Road obstacle detection method based on an autoencoder with semantic segmentation | |
| CN108268867B (en) | License plate positioning method and device | |
| CN112528878A (en) | Method and device for detecting lane line, terminal device and readable storage medium | |
| CN105512683A (en) | Target positioning method and device based on convolution neural network | |
| CN111191611B (en) | Traffic sign label identification method based on deep learning | |
| EP3852061A1 (en) | Method and device for damage segmentation of vehicle damage image | |
| CN103310231A (en) | Auto logo locating and identifying method | |
| CN109840463B (en) | Lane line identification method and device | |
| CN109635737A (en) | Automobile navigation localization method is assisted based on pavement marker line visual identity | |
| CN110826415A (en) | Method and device for re-identifying vehicles in scene image | |
| CN113255444A (en) | Training method of image recognition model, image recognition method and device | |
| CN104463238B (en) | A kind of automobile logo identification method and system | |
| CN114140773A (en) | Ground identification category confirmation method and device and vehicle | |
| CN110727269B (en) | Vehicle control method and related product | |
| EP4287137A1 (en) | Method, device, equipment, storage media and system for detecting drivable space of road | |
| CN110751163B (en) | Target positioning method and device, computer readable storage medium and electronic equipment | |
| CN105205444A (en) | Vehicle logo identification method based on dot pair characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |