CN115601728A - Vehicle identification method, device, equipment and storage medium - Google Patents

Vehicle identification method, device, equipment and storage medium Download PDF

Info

Publication number
CN115601728A
CN115601728A CN202211338722.9A CN202211338722A CN115601728A CN 115601728 A CN115601728 A CN 115601728A CN 202211338722 A CN202211338722 A CN 202211338722A CN 115601728 A CN115601728 A CN 115601728A
Authority
CN
China
Prior art keywords
vehicle
image
color
identification
detected
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211338722.9A
Other languages
Chinese (zh)
Inventor
石兆涵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Agricultural Bank of China
Original Assignee
Agricultural Bank of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Agricultural Bank of China filed Critical Agricultural Bank of China
Priority to CN202211338722.9A priority Critical patent/CN115601728A/en
Publication of CN115601728A publication Critical patent/CN115601728A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/56Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/46Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462Salient features, e.g. scale invariant feature transforms [SIFT]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/56Extraction of image or video features relating to colour
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07Target detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08Detecting or categorising vehicles

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a vehicle identification method, a device, equipment and a storage medium, wherein the method comprises the following steps: acquiring an image to be detected; carrying out target identification on an image to be detected to obtain a key area of the vehicle; identifying the vehicle type of the key area of the vehicle to obtain the vehicle type; carrying out vehicle color identification on an image to be detected to obtain a vehicle color; and determining the key area of the vehicle, the type of the vehicle and the color of the vehicle as the identification result of the image to be detected. According to the technical scheme, the key area, the type and the color of the vehicle are obtained from the image to be detected, the vehicle is identified according to a plurality of characteristics of the vehicle, the problem that the vehicle with high vehicle type similarity cannot be accurately identified is solved, and the accuracy of vehicle identification is improved.

Description

Vehicle identification method, device, equipment and storage medium
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a vehicle identification method, apparatus, device, and storage medium.
Background
With the development of computer technology and artificial intelligence, image processing technology is widely applied in many fields, and the field of vehicle identification is one of the application fields.
Most of the conventional vehicle identification methods identify vehicles based on a single attribute of the vehicle, and mainly include a license plate-based identification algorithm, a vehicle contour-based identification algorithm, a license plate color-based identification algorithm, and a vehicle texture-based identification algorithm. The vehicle actually has a plurality of attributes, and the vehicle with high vehicle type similarity cannot be accurately identified based on the identification algorithm of a certain single attribute of the vehicle, so that the accuracy of vehicle identification is low.
Disclosure of Invention
The invention provides a vehicle identification method, a vehicle identification device, vehicle identification equipment and a storage medium, which are used for improving the accuracy of vehicle identification.
According to an aspect of the present invention, there is provided a vehicle identification method including:
acquiring an image to be detected;
carrying out target identification on an image to be detected to obtain a key area of the vehicle;
identifying the vehicle type of the key area of the vehicle to obtain the vehicle type;
carrying out vehicle color identification on an image to be detected to obtain a vehicle color;
and determining the key area of the vehicle, the type of the vehicle and the color of the vehicle as the identification result of the image to be detected.
According to another aspect of the present invention, there is provided a vehicle identification device including:
the image acquisition module is used for acquiring an image to be detected;
the key area acquisition module is used for carrying out target identification on the image to be detected to obtain a key area of the vehicle;
the vehicle type acquisition module is used for identifying the vehicle type of the key area of the vehicle to obtain the vehicle type;
the vehicle color acquisition module is used for identifying the vehicle color of the image to be detected to obtain the vehicle color;
and the identification result determining module is used for determining the key area of the vehicle, the type of the vehicle and the color of the vehicle as the identification result of the image to be detected.
According to another aspect of the present invention, there is provided an electronic apparatus including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the vehicle identification method of any of the embodiments of the invention.
According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the vehicle identification method of any one of the embodiments of the present invention when executed.
According to the technical scheme of the embodiment of the invention, the image to be detected is obtained; carrying out target identification on an image to be detected to obtain a key area of the vehicle; identifying the vehicle type of the key area of the vehicle to obtain the vehicle type; carrying out vehicle color identification on an image to be detected to obtain a vehicle color; and determining the key area of the vehicle, the type of the vehicle and the color of the vehicle as the identification result of the image to be detected. According to the technical scheme, the key area, the type and the color of the vehicle are obtained from the image to be detected, the vehicle is identified according to a plurality of characteristics of the vehicle, the problem that the vehicle with high vehicle type similarity cannot be accurately identified is solved, and the accuracy of vehicle identification is improved.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present invention, nor do they necessarily limit the scope of the invention. Other features of the present invention will become apparent from the following description.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a flowchart of a vehicle identification method according to an embodiment of the present invention;
FIG. 2A is a flow chart of a vehicle identification method according to a second embodiment of the present invention;
FIG. 2B is a flow chart of another vehicle identification method according to a second embodiment of the invention;
FIG. 2C is a training procedure of fast-R-CNN according to a second embodiment of the present invention;
FIG. 2D is a flowchart of a vehicle type identification method according to a second embodiment of the present invention;
FIG. 3 is a schematic diagram of a flow chart apparatus provided in accordance with a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device implementing the flowchart method of the embodiment of the present invention.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "target" and "initial" and the like in the description and claims of the invention and the drawings described above are used for distinguishing similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example one
Fig. 1 is a flowchart of a vehicle identification method according to an embodiment of the present invention, where the embodiment is applicable to a case of identifying a vehicle in an image, and the method may be executed by a vehicle identification device, where the vehicle identification device may be implemented in a form of hardware and/or software, and the vehicle identification device may be configured in an electronic device, where the electronic device may be a large computer. As shown in fig. 1, the method includes:
and S101, acquiring an image to be detected.
The image to be detected may refer to an image including a vehicle. Specifically, an image acquisition device can be installed at a preset position according to business requirements, and an image to be detected is acquired through the image acquisition device. In an exemplary embodiment, an image acquisition device is installed at a toll station at a highway intersection, and an image to be detected is acquired through the image acquisition device. For another example, the user may capture an image to be detected by holding the image capture device.
And S102, carrying out target identification on the image to be detected to obtain a key area of the vehicle.
The target identification may refer to identifying a specific object in the image to be detected, such as identifying a vehicle in the image to be detected, identifying a human, and identifying an animal. The vehicle key zone may be a zone most representative of the vehicle, and the vehicle key zone may refer to a zone including vehicle key information; wherein the vehicle critical information may comprise at least one of: license plate number, vehicle model, vehicle color, vehicle brand and the like.
Specifically, the vehicle in the image to be detected can be positioned through a vehicle positioning algorithm, and the positioned vehicle is identified to obtain a key area of the vehicle.
And S103, identifying the vehicle type of the vehicle key area to obtain the vehicle type.
The vehicle type may include a brand of the vehicle, a model of the vehicle, a year of the vehicle, and the like. The vehicle type is used to distinguish vehicles associated with a critical area of the vehicle from other vehicles. Specifically, the vehicle type may be obtained by identifying the image to be detected through a special algorithm model (for example, an algorithm based on a convolutional neural network).
And S104, identifying the color of the vehicle of the image to be detected to obtain the color of the vehicle.
The vehicle color refers to the color of the vehicle body, may include all colors on the vehicle body, and may also refer to only the main color of the vehicle body. The vehicle colors may include black, blue, brown, gray, green, red, white, yellow, and the like.
Specifically, the vehicle color may be obtained by performing vehicle color recognition on the image to be detected through a special algorithm model (e.g., an algorithm based on a convolutional neural network).
And S105, determining the key area of the vehicle, the type of the vehicle and the color of the vehicle as the identification result of the image to be detected.
Specifically, the identification result of the image to be detected is to identify the vehicle key area, the vehicle type and the vehicle color of the vehicle in the image to be detected.
According to the technical scheme of the embodiment of the invention, the image to be detected is obtained; carrying out target identification on an image to be detected to obtain a key area of the vehicle; identifying the vehicle type of the key area of the vehicle to obtain the vehicle type; carrying out vehicle color identification on an image to be detected to obtain a vehicle color; and determining the key area of the vehicle, the type of the vehicle and the color of the vehicle as the identification result of the image to be detected. According to the technical scheme, the key area, the type and the color of the vehicle are obtained from the image to be detected, the vehicle is identified according to a plurality of characteristics of the vehicle, the problem that the vehicle with high vehicle type similarity cannot be accurately identified is solved, and the accuracy of vehicle identification is improved.
On the basis of the foregoing embodiment, as an optional manner of the embodiment of the present invention, the method may further include: after determining a key area of a vehicle, a vehicle type and a vehicle color as a recognition result of an image to be detected, acquiring vehicle information corresponding to the image to be detected, which is provided by an automatic road payment system; comparing the vehicle information with the identification result of the image to be detected; and sending a passing instruction to the automatic road payment system under the condition that the comparison result is consistent, so that the automatic road payment system controls to release the vehicle corresponding to the image to be detected.
The automatic road payment system may be an Electronic Toll Collection (ETC) system, and is used for automatic road Toll Collection. The vehicle information may include a license plate number, a vehicle key zone, a vehicle type, a vehicle color, and the like.
For example, the default vehicle is installed with the ETC before leaving the factory, and information such as the color, brand, model, year, and license plate number of the vehicle is stored in the ETC. After the ETC system receives an ETC signal sent by a vehicle in an image to be detected, vehicle information corresponding to the ETC signal is extracted from a database corresponding to the ETC system according to the received ETC signal; comparing the vehicle type and the vehicle color in the vehicle information with the vehicle type and the vehicle color in the image recognition result to be detected; under the condition that the comparison results are consistent, sending a passing instruction to the ETC system so that the ETC system controls to pass the vehicle corresponding to the image to be detected; and under the condition that the comparison results are inconsistent, forbidding the vehicle corresponding to the image to be detected to pass.
According to the technical scheme of the embodiment, after the key area of the vehicle, the type of the vehicle and the color of the vehicle are determined as the identification result of the image to be detected, the vehicle information corresponding to the image to be detected, which is provided by the automatic road payment system, is obtained; comparing the vehicle information with the identification result of the image to be detected; and sending a passing instruction to the automatic road payment system under the condition that the comparison result is consistent, so that the automatic road payment system controls to release the vehicle corresponding to the image to be detected. The accuracy of vehicle identification is further improved, and the occurrence of vehicle brush stealing is avoided.
On the basis of the foregoing embodiment, as an optional manner of the embodiment of the present invention, the method may further include: after determining the key area, the type and the color of the vehicle as the identification result of the image to be detected, acquiring vehicle verification information provided by a vehicle owner; comparing the vehicle authentication information with the identification result; and in case of inconsistency, determining that the vehicle corresponding to the vehicle verification information has a risk.
The vehicle verification information may include information such as a license plate number, a vehicle model, a vehicle color, a vehicle brand, a vehicle age, and the like.
Illustratively, the vehicle information provided by the owner of the vehicle is taken as the vehicle authentication information; and comparing the vehicle verification information with the recognition result of the image to be detected, determining that the vehicle corresponding to the vehicle verification information has risk under the condition of inconsistency, and prompting related business personnel that the vehicle is probably stolen or robbed or the vehicle is probably modified. And under the condition of consistency, determining that the vehicle corresponding to the vehicle verification information has no risk. In the case of coincidence, the request of the owner of the vehicle can be processed on the basis of the vehicle information provided.
According to the technical scheme of the embodiment, after the key area, the type and the color of the vehicle are determined as the identification result of the image to be detected, vehicle verification information provided by a vehicle owner is acquired; comparing the vehicle authentication information with the identification result; and in case of inconsistency, determining that the vehicle corresponding to the vehicle verification information has a risk. The application scene of the vehicle identification method is provided, and the risk of vehicle-related business operation is reduced.
Example two
Fig. 2A is a flowchart of a vehicle identification method according to a second embodiment of the present invention, where in this embodiment, "target identification is performed on an image to be detected" on the basis of the second embodiment, so as to obtain a key area of a vehicle; identifying the vehicle type of the key area of the vehicle to obtain the vehicle type; and (3) carrying out vehicle color identification on the image to be detected to obtain further detailed vehicle colors, and providing an optional implementation scheme. In the embodiments of the present invention, detailed descriptions of the related embodiments are referred to. As shown in fig. 2A, the method includes:
s201, obtaining an image to be detected.
S202, carrying out target identification on the image to be detected through a target identification module of the vehicle identification model to obtain a vehicle key area.
Wherein the vehicle identification model may be preconfigured. The target identification module is used for identifying a key area of the vehicle in the image to be detected. The vehicle key zones may include: vehicle side doors, vehicle side bodies, vehicle front faces, vehicle rear portions, and vehicle roof portions.
The object recognition module can be the Fast-R-CNN, which includes Region-generated networks (RPN) and Fast-RCNN (Fast Region Convolutional Neural networks).
Specifically, target recognition is carried out on an image to be detected through the Faster-R-CNN of the vehicle recognition model, and a vehicle key area is obtained. The specific working principle is as follows: inputting an image to be detected into a Convolutional Neural Network (CNN) in Fast-RCNN; extracting image features from the image to be detected by utilizing a convolutional layer in the convolutional neural network so as to obtain a feature map; inputting the feature map into the RPN to obtain a region of interest (which may be one or more); scoring the interest areas, and selecting the interest area with the highest score as a target area according to the scores of the interest areas; projecting the target area onto the feature map to obtain the feature map at the corresponding position of the target area; inputting the obtained feature map into Fast-RCNN, and processing the feature map through a ROI Pooling (Regions Of Interest Pooling) layer to obtain a key region Of the vehicle.
And S203, identifying the type of the key area of the vehicle through a classification module of the vehicle identification model to obtain the type of the vehicle.
The classification module is used for classifying the types of the vehicles in the key areas of the vehicles. Wherein, the classification module can be Alex-Net; alex-Net has 8 layers, the first five layers are Convolutional layers (volumetric layer), the last three layers are fully connected layers (Full Connection layer), and the last fully connected layer is used for outputting classification results.
Specifically, the key area of the vehicle is input into a vehicle identification model, and the type of the key area of the vehicle is identified by Alex-Net to obtain the type of the vehicle. The specific working principle is as follows: assuming that the image size specified by Alex-Net is 227 x 3, the size of the key area of the vehicle is normalized, the key area of the vehicle is adjusted to the image size specified by Alex-Net, and Alex-Net is input for training, so that the vehicle type is obtained.
And S204, carrying out color identification on the image to be detected through a classification module to obtain the color of the vehicle.
Specifically, the classification module is further used for recognizing the color of the vehicle body of the vehicle in the image to be detected. Illustratively, color recognition is carried out on the image to be detected by using Alex-Net to obtain the color of the vehicle. The specific working principle is the same as the principle of obtaining the vehicle type by carrying out type identification on the key area of the vehicle through the classification module, only the input is changed into an image to be detected, and the output is changed into the color of the vehicle.
S205, determining the key area of the vehicle, the type of the vehicle and the color of the vehicle as the recognition result of the image to be detected.
According to the technical scheme of the embodiment of the invention, the target identification module of the vehicle identification model is used for carrying out target identification on the image to be detected to obtain a key area of the vehicle; the method comprises the steps that type recognition is carried out on a key area of a vehicle through a classification module of a vehicle recognition model to obtain the type of the vehicle; and carrying out color identification on the image to be detected through a classification module to obtain the color of the vehicle. According to the technical scheme, the key area, the type and the color of the vehicle are obtained through the target recognition module and the classification module of the vehicle recognition model, the target recognition module in the vehicle recognition model can optimize and normalize the size of the input image to be detected, the normalization processing process of the image to be detected in the early stage is omitted, and the running speed of a vehicle recognition algorithm is guaranteed.
On the basis of the above embodiment, the present invention further provides a flowchart of another vehicle identification method, referring to fig. 2B, obtaining a vehicle to be detected, wherein a body color of the vehicle to be detected is pure red; generating interest regions (regions framed by candidate frames in the graph) by using the RPN, wherein 4 interest regions are generated in the graph exemplarily; scoring the interest areas, and selecting the interest area with the highest score as a vehicle key area according to the scores of the interest areas; and inputting the key areas of the vehicles and images in a pre-established training database into a classification module in the vehicle identification model to finally obtain the types of the vehicles and the colors (A01, red) of the vehicles. The vehicle images in the training database comprise images of the front face of the vehicle, images of the tail of the vehicle, images of the side faces of the vehicle, images of vehicles of various models under various vehicle brands and the like.
On the basis of the above embodiment, as an optional manner of the embodiment of the present invention, the vehicle identification model is obtained by: acquiring an initial vehicle image; marking an initial vehicle image, adding the initial vehicle image into a vehicle image sample set, marking a vehicle key area, a vehicle type and a vehicle color on the vehicle image sample, wherein the vehicle type comprises a vehicle logo and a model; obtaining samples in a vehicle image sample set; and training the initial model by adopting a sample to obtain the vehicle identification module.
The initial vehicle image may refer to an unprocessed vehicle image, and the initial vehicle image is used for training the vehicle recognition model. The emblem is used to uniquely identify the brand of the vehicle. The model number may refer to a model number under a brand of a vehicle to which the vehicle belongs. The initial model may be an untrained model. And training the initial model to obtain a vehicle identification model.
Specifically, an initial vehicle image is obtained from a pre-established vehicle identification training database; marking initial vehicle images at different time, different positions and different colors, and selecting key parts of the vehicle for marking when marking the color of the initial vehicle images, wherein the key parts can comprise the color of a vehicle door on the side surface of the vehicle, the color of a trunk at the tail part of the vehicle, the color of a hood on the front face of the vehicle, the color of the top of the vehicle and the like; adding the marked initial vehicle image to a vehicle image sample set; randomly selecting a sample from a vehicle image sample set; and training the initial model by adopting the selected sample to obtain the vehicle identification module. Wherein the training of the initial model comprises training of fast-R-CNN and Alex-Net.
Fast-R-CNN was trained using the Caffe (Convolutional structure for Fast Feature Embedding) framework, the training steps of which are shown in FIG. 2C. Firstly, performing one-to-one training on RPN by adopting an ImageNet model, and then adjusting a target region candidate frame generated by the RPN in an end-to-end mode; secondly, initializing Fast-RCNN by adopting an ImageNet model, and training the Fast-RCNN model by taking a target region candidate box generated by RPN as input data of the Fast-RCNN. The parameters of RPN and Fast-RCNN are not shared in this step; then, after training the parameters of Fast-RCNN, the new RPN2 is initialized by using the parameters, and the network layers except the shared partial convolution layer are updated. In the step, RPN and Fast-RCNN share a convolution layer; finally, training is accomplished using both RPN and Fast-RCNN. And taking the image marked with the key area of the vehicle as data input, and training the Fast-RCNN model. After training the Faster-R-CNN, the model converges according to the model training method described above. ImageNet is a large visual database for visual object recognition software research. The Caffe framework is used as a training tool of the Faster-R-CNN, the inner part of the Caffe framework comprises a Blob part, a Layer part, a Net part and a Solver part, and the process of training the Faster-R-CNN can be abstracted. The Blob is used for storing data such as parameters and training data in each layer of the network layer in the convolutional neural network. Layer represents convolutional layers in a neural network, and a forward propagation algorithm and a backward propagation algorithm are implemented in each Layer, and abstractly contains each network Layer commonly used in CNN. Net assembles Layer together and integrates the Layer into a network model. The Solver is an iterative optimization algorithm, and parameters are updated by alternately calling a forward propagation algorithm and a backward propagation algorithm, so that a loss function is minimized.
Illustratively, the Alex-Net is used for training key areas of the vehicle to obtain the vehicle type. Assuming that the image size specified by Alex-Net is 227X 3, a training database for vehicle identification is established in advance, the size of the image and the size of a vehicle key region in the training database are normalized, the size of the image and the size of the vehicle key region in the training database are adjusted to the image size specified by Alex-Net, and Alex-Net is input for training, so that the vehicle type is obtained. Specific training procedures, see e.g. 2D, assume that the first five layers of Alex-Net are denoted by Conv1, conv2, conv3, conv4 and Conv5, respectively, and the last three layers are denoted by FC6, FC7 and FC8, respectively. Conv1 is input into the images and the key areas of the vehicles in the training database, convolution operation and ReLU (Linear update Unit) operation are carried out on the images through convolution kernel, then pooling operation and normalization processing are carried out, and finally the scale of the obtained pixel layer is 27 × 96; taking the pixel layer as the input of Conv2, performing convolution operation and ReLU operation on data in the pixel layer through convolution kernel, and then performing pooling operation and normalization processing to obtain a final pixel layer with a scale of 13 × 256; by analogy, after the data in the pixel layer is subjected to Conv3, conv4, conv5 and ReLU operations, the data is subjected to pooling operation and normalization processing, and the finally obtained pixel layer size is 6 × 256; and (3) fully connecting the data in the pixel layer with 4096 neurons of FC6, then fully connecting with 4096 neurons of FC7 and 196 neurons of FC8, and finally outputting the type of the vehicle. In the model, the initialization operation of the algorithm weight is performed by the trained model of Alex-Net on ImageNet, and only the trained weight needs to be correspondingly adjusted to ensure better recognition effect. During the use of the model, the actual required image loading size was determined to be 227 x 3. Although the first five layers of convolution are more computationally intensive, there are not as many fully connected layers behind as there are parameters, and the front convolutional layer is far more important than the fully connected layer in terms of function. The ReLU function converges and it is determined that the training is complete.
According to the technical scheme of the embodiment, the initial vehicle image is obtained; marking the initial vehicle image, and adding the marked initial vehicle image into a vehicle image sample set; and training the initial model through samples in the vehicle image sample set to obtain the vehicle identification module. The method for acquiring the vehicle identification model is provided, and the subsequent identification of the image to be detected is facilitated.
On the basis of the above embodiment, as an optional manner of the embodiment of the present invention, the acquiring of the initial vehicle image may be: acquiring a vehicle image acquired by an automatic road payment system; acquiring an image of a vehicle acquired at a preset inclination angle; acquiring an image including a vehicle on a network; and an image generated by transforming the collected images of the vehicle.
Wherein, automatic road payment system can be ETC system, both can shoot the vehicle, can be used for the automatic charging of road again. The preset inclination angle may refer to an angle inclined in a vertical direction with respect to a front view of the vehicle, for example, inclined by 45 degrees in the vertical direction with respect to the front view of the vehicle, so as to obtain vehicle information of a maximum area, so as to enrich identifiable vehicle information.
Specifically, a vehicle image shot by an automatic road payment system is obtained; acquiring an image of the vehicle acquired by inclining 45 degrees to the vertical direction relative to the front view of the vehicle; acquiring an image including a vehicle from a network by using a crawler technology; and an image obtained by performing a series of transformations such as motion blur processing and brightness adjustment on the collected vehicle image. The crawler technology is a technology for automatically capturing data from a network according to a certain rule.
According to the technical scheme of the embodiment, multiple acquisition sources of the initial vehicle image are provided, the number of training databases in the vehicle recognition model is enriched, and the vehicle recognition result is more accurate.
On the basis of the foregoing embodiment, as an optional manner of the embodiment of the present invention, labeling the initial vehicle image, and adding the labeled initial vehicle image to the vehicle image sample set may be: acquiring the body color and the corresponding color area of a vehicle in an initial vehicle image; obtaining the ratio of the color area corresponding to the body color of the vehicle to the total color coverage area; and under the condition that the ratio is greater than or equal to the preset ratio threshold, marking the body color of the vehicle as the body color of the initial vehicle image.
The body color may include a door color of a side of the vehicle, a trunk color of a rear of the vehicle, a hood color of a front of the vehicle, a roof color of a roof of the vehicle, and the like.
Specifically, the body color and the corresponding color area of the vehicle in the initial vehicle image are obtained. If the body color of the vehicle in the initial vehicle image is a pure color, the ratio of the color area corresponding to the body color of the vehicle to the total color coverage area is 100%; the body color of the vehicle is labeled as the body color of the initial vehicle image. If the body color of the vehicle in the initial vehicle image has two or more colors at the same time, acquiring the ratio of the color area corresponding to the body color of the vehicle to the total color coverage area; and under the condition that the ratio is greater than or equal to the preset ratio threshold, marking the body color of the vehicle as the body color of the initial vehicle image.
For example, assuming that the body colors of the vehicles in the initial vehicle image include white and black, the preset ratio threshold is 85%; wherein, white accounts for 90% of the total area covered by the color of the vehicle body, and black accounts for 10% of the total area covered by the color of the vehicle body. Acquiring the body colors (white and black) and the corresponding color areas of the vehicle in the initial vehicle image; obtaining the ratio (90% and 10%) of the color area corresponding to the body color of the vehicle to the total area covered by the color; and (3) marking the white as the body color of the vehicle in the initial vehicle image, wherein the ratio (90%) of the white to the total covered area of the body color is greater than a preset ratio threshold (85%).
According to the technical scheme of the embodiment, multiple possibilities of vehicle body color composition of the vehicle in the initial vehicle image are fully considered, the corresponding method for determining the vehicle body color in the initial vehicle image is provided, and the accuracy of vehicle color identification in the image to be detected is further improved.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a vehicle identification device according to a third embodiment of the present invention. As shown in fig. 3, the apparatus includes:
an image obtaining module 301, configured to obtain an image to be detected;
a key area obtaining module 302, configured to perform target identification on an image to be detected to obtain a key area of a vehicle;
the vehicle type obtaining module 303 is configured to perform vehicle type identification on a vehicle key area to obtain a vehicle type;
the vehicle color acquisition module 304 is configured to perform vehicle color identification on the image to be detected to obtain a vehicle color;
and the identification result determining module 305 is used for determining the key area of the vehicle, the type of the vehicle and the color of the vehicle as the identification result of the image to be detected.
According to the technical scheme of the embodiment of the invention, the image to be detected is obtained through the image obtaining module; obtaining a key area of the vehicle through a key area obtaining module; the method comprises the steps that vehicle type identification is carried out on a key area of a vehicle through a vehicle type acquisition module to obtain the vehicle type; the method comprises the steps that vehicle color identification is carried out on an image to be detected through a vehicle color acquisition module to obtain vehicle colors; and acquiring the identification result of the image to be detected through an identification result determining module. According to the technical scheme, the key area, the type and the color of the vehicle are obtained from the image to be detected, the vehicle is identified according to a plurality of characteristics of the vehicle, the problem that the vehicle with high vehicle type similarity cannot be accurately identified is solved, and the accuracy of vehicle identification is improved.
Optionally, the key area obtaining module 302 is specifically configured to:
carrying out target identification on an image to be detected through a target identification module of a vehicle identification model to obtain a vehicle key area;
the vehicle type obtaining module 303 is specifically configured to: the method comprises the steps that type recognition is carried out on a key area of a vehicle through a classification module of a vehicle recognition model to obtain the type of the vehicle;
the vehicle color obtaining module 304 is specifically configured to: and carrying out color identification on the image to be detected through a classification module to obtain the color of the vehicle.
Optionally, the vehicle identification model is obtained through the following steps:
acquiring an initial vehicle image;
marking an initial vehicle image, adding the initial vehicle image into a vehicle image sample set, wherein the vehicle image sample is marked with a vehicle key area, a vehicle type and a vehicle color, and the vehicle type comprises a vehicle logo and a model;
obtaining samples in a vehicle image sample set;
and training the initial model by adopting a sample to obtain the vehicle identification module.
Optionally, the acquiring an initial vehicle image includes:
acquiring a vehicle image acquired by an automatic road payment system;
acquiring an image of a vehicle acquired at a preset inclination angle;
acquiring an image including a vehicle on a network; and
and transforming the collected images of the vehicle to generate the images.
Optionally, the labeling the initial vehicle image and adding the labeled initial vehicle image to the vehicle image sample set include:
acquiring the body color and the corresponding color area of a vehicle in an initial vehicle image;
obtaining the ratio of the color area corresponding to the body color of the vehicle to the total color coverage area;
and under the condition that the ratio is greater than or equal to a preset ratio threshold value, marking the body color of the vehicle as the body color of the initial vehicle image.
Optionally, the apparatus may further include:
the vehicle information acquisition module is used for acquiring vehicle information corresponding to the image to be detected, which is provided by the automatic road payment system, after determining a key area of the vehicle, the type of the vehicle and the color of the vehicle as the identification result of the image to be detected;
the information comparison module is used for comparing the vehicle information with the identification result of the image to be detected;
and the traffic instruction sending module is used for sending a traffic instruction to the automatic road payment system under the condition that the comparison result is consistent so that the automatic road payment system controls to release the vehicle corresponding to the image to be detected.
Optionally, the apparatus may further include:
the verification information acquisition module is used for acquiring vehicle verification information provided by a vehicle owner after determining a key area, a vehicle type and a vehicle color of the vehicle as a recognition result of the image to be detected;
the verification information comparison module is used for comparing the vehicle verification information with the identification result;
and the risk determining module is used for determining that the vehicle corresponding to the vehicle verification information has risks under the condition of inconsistency.
The vehicle identification device provided by the embodiment of the invention can execute the vehicle identification method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing each vehicle identification method.
Example four
FIG. 4 shows a schematic block diagram of an electronic device 400 that may be used to implement embodiments of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 4, the electronic device 400 includes at least one processor 401, and a memory communicatively connected to the at least one processor 401, such as a Read Only Memory (ROM) 402, a Random Access Memory (RAM) 403, and the like, wherein the memory stores computer programs executable by the at least one processor, and the processor 401 may perform various suitable actions and processes according to the computer programs stored in the Read Only Memory (ROM) 402 or the computer programs loaded from a storage unit 408 into the Random Access Memory (RAM) 403. In the RAM403, various programs and data necessary for the operation of the electronic apparatus 400 can also be stored. The processor 401, ROM402 and RAM403 are connected to each other by a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.
A number of components in the electronic device 400 are connected to the I/O interface 405, including: an input unit 406 such as a keyboard, a mouse, or the like; an output unit 407 such as various types of displays, speakers, and the like; a storage unit 408 such as a magnetic disk, optical disk, or the like; and a communication unit 409 such as a network card, modem, wireless communication transceiver, etc. The communication unit 409 allows the electronic device 400 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
Processor 401 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of processor 401 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The processor 401 performs the various methods and processes described above, such as the vehicle identification method.
In some embodiments, the vehicle identification method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 400 via the ROM402 and/or the communication unit 409. When the computer program is loaded into RAM403 and executed by processor 401, one or more steps of the vehicle identification method described above may be performed. Alternatively, in other embodiments, the processor 401 may be configured to perform the vehicle identification method in any other suitable manner (e.g., by way of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be performed. A computer program can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user may provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the Internet.
The computing system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server may be a cloud Server, which is also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the conventional physical host and VPS (Virtual Private Server) service.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present invention may be executed in parallel, sequentially, or in different orders, and are not limited herein as long as the desired results of the technical solution of the present invention can be achieved.
The above-described embodiments should not be construed as limiting the scope of the invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A vehicle identification method, characterized in that the method comprises:
acquiring an image to be detected;
carrying out target identification on the image to be detected to obtain a key area of the vehicle;
identifying the vehicle type of the key area of the vehicle to obtain the vehicle type;
carrying out vehicle color identification on the image to be detected to obtain a vehicle color;
and determining the key area of the vehicle, the type of the vehicle and the color of the vehicle as the recognition result of the image to be detected.
2. The method according to claim 1, wherein the step of performing target recognition on the image to be detected to obtain a key area of the vehicle comprises:
performing target identification on the image to be detected through a target identification module of a vehicle identification model to obtain a key area of the vehicle;
the type identification of the key area of the vehicle to obtain the vehicle type comprises the following steps:
the type of the key area of the vehicle is identified through a classification module of the vehicle identification model, so that the type of the vehicle is obtained;
the right wait to detect the image and carry out color identification, obtain the vehicle colour, include:
and carrying out color identification on the image to be detected through the classification module to obtain the color of the vehicle.
3. The method of claim 2, wherein the vehicle identification model is obtained by:
acquiring an initial vehicle image;
marking the initial vehicle image, and adding the initial vehicle image into a vehicle image sample set, wherein the vehicle image sample is marked with a vehicle key area, a vehicle type and a vehicle color, and the vehicle type comprises a vehicle logo and a model;
obtaining samples in the vehicle image sample set;
and training the initial model by adopting the sample to obtain the vehicle identification module.
4. The method of claim 3, wherein the acquiring an initial vehicle image comprises:
acquiring a vehicle image acquired by an automatic road payment system;
acquiring an image of a vehicle acquired at a preset inclination angle;
acquiring an image including a vehicle on a network; and
and transforming the collected images of the vehicle to generate the images.
5. The method of claim 3, wherein said annotating said initial vehicle image, added to a vehicle image sample set, comprises:
acquiring the body color and the corresponding color area of a vehicle in an initial vehicle image;
obtaining the ratio of the color area corresponding to the body color of the vehicle to the total color coverage area;
and under the condition that the ratio is greater than or equal to a preset ratio threshold, marking the body color of the vehicle as the body color of the initial vehicle image.
6. The method according to claim 1, characterized in that after determining the vehicle key area, the vehicle type and the vehicle color as the identification result of the image to be detected, the method further comprises:
acquiring vehicle information corresponding to the image to be detected, which is provided by an automatic road payment system;
comparing the vehicle information with the recognition result of the image to be detected;
and sending a passing instruction to the automatic road payment system under the condition that the comparison result is consistent, so that the automatic road payment system controls to release the vehicle corresponding to the image to be detected.
7. The method according to claim 1, after determining the vehicle key area, the vehicle type and the vehicle color as the identification result of the image to be detected, further comprising:
acquiring vehicle verification information provided by a vehicle owner;
comparing the vehicle authentication information with the identification result;
and determining that the vehicle corresponding to the vehicle verification information is in risk under the condition of inconsistency.
8. A vehicle identification device, characterized by comprising:
the image acquisition module is used for acquiring an image to be detected;
the key area acquisition module is used for carrying out target identification on the image to be detected to obtain a key area of the vehicle;
the vehicle type acquisition module is used for identifying the vehicle type of the key area of the vehicle to obtain the vehicle type;
the vehicle color acquisition module is used for carrying out vehicle color identification on the image to be detected to obtain a vehicle color;
and the identification result determining module is used for determining the key area of the vehicle, the type of the vehicle and the color of the vehicle as the identification result of the image to be detected.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,
the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the vehicle identification method of any one of claims 1-7.
10. A computer-readable storage medium storing computer instructions for causing a processor to perform the vehicle identification method of any one of claims 1-7 when executed.
CN202211338722.9A 2022-10-28 2022-10-28 Vehicle identification method, device, equipment and storage medium Pending CN115601728A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211338722.9A CN115601728A (en) 2022-10-28 2022-10-28 Vehicle identification method, device, equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211338722.9A CN115601728A (en) 2022-10-28 2022-10-28 Vehicle identification method, device, equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115601728A true CN115601728A (en) 2023-01-13

Family

ID=84850979

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211338722.9A Pending CN115601728A (en) 2022-10-28 2022-10-28 Vehicle identification method, device, equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115601728A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117456473A (en) * 2023-12-25 2024-01-26 杭州吉利汽车数字科技有限公司 Vehicle assembly detection method, device, equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117456473A (en) * 2023-12-25 2024-01-26 杭州吉利汽车数字科技有限公司 Vehicle assembly detection method, device, equipment and storage medium
CN117456473B (en) * 2023-12-25 2024-03-29 杭州吉利汽车数字科技有限公司 Vehicle assembly detection method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN108776772B (en) Cross-time building change detection modeling method, detection device, method and storage medium
WO2021139324A1 (en) Image recognition method and apparatus, computer-readable storage medium and electronic device
CN107633204A (en) Face occlusion detection method, apparatus and storage medium
CN111667001B (en) Target re-identification method, device, computer equipment and storage medium
CN110222686B (en) Object detection method, object detection device, computer equipment and storage medium
CN110765954A (en) Vehicle weight recognition method, equipment and storage device
CN114119676B (en) Target detection tracking identification method and system based on multi-feature information fusion
CN110379020A (en) A kind of laser point cloud painting methods and device based on generation confrontation network
WO2021120961A1 (en) Brain addiction structure map evaluation method and apparatus
CN105760898A (en) Vision mapping method based on mixed group regression method
CN111985367A (en) Pedestrian re-recognition feature extraction method based on multi-scale feature fusion
CN113239807B (en) Method and device for training bill identification model and bill identification
CN115330940B (en) Three-dimensional reconstruction method, device, equipment and medium
CN112016545A (en) Image generation method and device containing text
CN110910445A (en) Object size detection method and device, detection equipment and storage medium
CN109784171A (en) Car damage identification method for screening images, device, readable storage medium storing program for executing and server
CN114550051A (en) Vehicle loss detection method and device, computer equipment and storage medium
CN115601728A (en) Vehicle identification method, device, equipment and storage medium
CN115100469A (en) Target attribute identification method, training method and device based on segmentation algorithm
CN113496148A (en) Multi-source data fusion method and system
CN111310720A (en) Pedestrian re-identification method and system based on graph metric learning
CN113673527B (en) License plate recognition method and system
Muchtar et al. Convolutional network and moving object analysis for vehicle detection in highway surveillance videos
CN111709377B (en) Feature extraction method, target re-identification method and device and electronic equipment
Li et al. A novel approach for vehicle detection using an AND–OR-graph-based multiscale model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination