CN111738212A - Traffic signal lamp identification method, device, equipment and medium based on artificial intelligence - Google Patents
Traffic signal lamp identification method, device, equipment and medium based on artificial intelligence Download PDFInfo
- Publication number
- CN111738212A CN111738212A CN202010695795.8A CN202010695795A CN111738212A CN 111738212 A CN111738212 A CN 111738212A CN 202010695795 A CN202010695795 A CN 202010695795A CN 111738212 A CN111738212 A CN 111738212A
- Authority
- CN
- China
- Prior art keywords
- target
- traffic signal
- image
- anchor box
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000012545 processing Methods 0.000 claims abstract description 35
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 32
- 238000001514 detection method Methods 0.000 claims abstract description 28
- 238000010586 diagram Methods 0.000 claims abstract description 26
- 238000012549 training Methods 0.000 claims description 42
- 230000006870 function Effects 0.000 claims description 22
- 238000013507 mapping Methods 0.000 claims description 19
- 239000002131 composite material Substances 0.000 claims description 17
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 16
- 238000000605 extraction Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 230000006835 compression Effects 0.000 claims description 4
- 238000007906 compression Methods 0.000 claims description 4
- 230000002194 synthesizing effect Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 239000000284 extract Substances 0.000 abstract description 11
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000010276 construction Methods 0.000 abstract 1
- 230000001737 promoting effect Effects 0.000 abstract 1
- 238000004590 computer program Methods 0.000 description 12
- 238000007726 management method Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 230000003068 static effect Effects 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 230000007246 mechanism Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention relates to the technical field of artificial intelligence and image detection, and provides a traffic signal lamp identification method, a device, equipment and a medium based on artificial intelligence, which can extract target characteristic information of a target image after resize processing by utilizing a dark net53 network, input the target characteristic information into a traffic signal lamp identification model trained by a Mixup algorithm and a residual attention network, output a target characteristic diagram to accurately extract detailed characteristics, improve recall rate and accuracy, acquire the identification of the target anchor box on each target characteristic diagram, output a target anchor box coordinate and a target score, take the target anchor box coordinate with the highest score as a predicted position coordinate and map the predicted position coordinate onto an image to be identified to obtain an identification result, further realize automatic identification of a traffic signal lamp based on artificial intelligence means, and have higher identification accuracy. The invention also relates to a block chain technology, and the identification result and the traffic signal lamp identification model can be stored in the block chain. The invention can also be applied to smart traffic scenes, thereby promoting the construction of smart cities.
Description
Technical Field
The invention relates to the technical field of artificial intelligence and image detection, in particular to a traffic signal lamp identification method, a device, equipment and a medium based on artificial intelligence.
Background
With the increasing number of illegal behaviors such as red light running of vehicles, the rapid positioning and identification of signal lamps in traffic checkpoint images become an extremely important and challenging task in urban traffic management.
The traditional Yolov3 detection algorithm has the advantages of high detection speed and high precision, and becomes one of mainstream target detection algorithms in the industry at present, the Yolov3 detection algorithm has a good detection effect on large targets, but has a great promotion space for the detection effect on small targets such as similar traffic lights, and especially has a weak effect on detection and identification under complex scenes such as night or haze.
Disclosure of Invention
In view of the above, it is necessary to provide a traffic signal lamp identification method, device, apparatus and medium based on artificial intelligence, which can extract the detailed features of the traffic signal lamp more accurately based on the Mixup algorithm and the residual attention network, so as to improve the recall rate and accuracy of the traffic signal lamp detection, and further realize automatic identification of the traffic signal lamp based on artificial intelligence means, and the identification accuracy is higher.
An artificial intelligence based traffic signal lamp identification method, comprising:
responding to the received image to be recognized, and performing resize processing on the image to be recognized to obtain a target image;
extracting target characteristic information of the target image by using a dark net53 network;
inputting the target characteristic information into a pre-trained traffic signal lamp identification model, and outputting a target characteristic map with at least one scale, wherein the traffic signal lamp identification model is obtained by adopting a Mixup algorithm and residual error attention network training;
acquiring a target anchor box of the traffic signal lamp identification model;
identifying each target feature map by using the target anchor box, outputting a target anchor box coordinate corresponding to each target feature map and a target score of each target anchor box coordinate, and taking the target anchor box coordinate with the highest target score as a predicted position coordinate of the traffic signal lamp;
and mapping the predicted position coordinates of the traffic signal lamp to the image to be identified to obtain an identification result of the image to be identified, and storing the identification result in a block chain.
According to a preferred embodiment of the present invention, before inputting the target feature information into a traffic signal recognition model trained in advance and outputting a target feature map of at least one scale, the artificial intelligence-based traffic signal recognition method further includes:
obtaining a training sample;
performing combined operation on the training samples by taking a preset number as a group to obtain at least one group of training data;
synthesizing each group of training data by using a Mixup algorithm to obtain a synthesized image corresponding to each group of training data;
extracting characteristic information of each composite image by using a dark net53 network;
obtaining a residual block of at least one scale in the darknet53 network;
splicing the characteristic information of each synthetic image and the residual block with the corresponding scale based on a residual attention network to obtain a splicing result;
performing convolution operation on the splicing result to obtain a characteristic diagram of at least one scale of each synthetic image;
obtaining an anchor box obtained by pre-clustering;
identifying on each characteristic diagram by using the anchor box, and outputting the anchor box coordinate corresponding to each characteristic diagram and the score of each anchor box coordinate;
acquiring an anchor box coordinate with the highest score in the feature map of at least one scale of each synthetic image as a prediction coordinate of each synthetic image;
determining the actual coordinates of each composite image;
calculating accuracy and recall rate based on the actual coordinates of each composite image and the predicted coordinates of each composite image;
calculating a value of a loss function;
and when the accuracy reaches a preset accuracy, the recall rate reaches a preset recall rate, and the value of the loss function is lower than a preset loss, stopping training to obtain the traffic signal lamp identification model, and storing the traffic signal lamp identification model on a block chain.
According to the preferred embodiment of the present invention, the splicing the feature information of each synthesized image and the residual block with the corresponding scale based on the residual attention network, and obtaining a splicing result includes:
obtaining a plurality of feature information having the same scale in a plurality of layers of the darknet53 network;
for a plurality of feature information of each scale, performing compression transformation on each feature information to obtain a plurality of compressed data;
performing squeeze processing on the plurality of compressed data based on the residual blocks with corresponding sizes to obtain a plurality of processing results;
calculating the score of each processing result by adopting an attention algorithm;
and determining the processing result with the highest score as the splicing result.
According to a preferred embodiment of the present invention, the performing convolution operation on the stitching result to obtain at least one scale feature map of each composite image includes:
and sequentially inputting the splicing result to a conv _ layer, a conv _ block layer and a conv layer for convolution operation, and outputting a characteristic diagram of at least one scale of each synthetic image.
According to the preferred embodiment of the present invention, the artificial intelligence-based traffic signal light recognition method further comprises:
determining the center point coordinate of the anchor box corresponding to each characteristic diagram, the width and height coordinates of the anchor box corresponding to each characteristic diagram and the score of each anchor box coordinate;
calculating a central point coordinate error according to the central point coordinate of the anchor box corresponding to each feature map;
calculating a width and height coordinate error according to the width and height coordinates of the anchor box corresponding to each feature map;
calculating a target error according to the score of each anchor box coordinate;
calculating a sum of the center point coordinate error, the width-to-height coordinate error, and the target error as the loss function.
According to the preferred embodiment of the present invention, the mapping the predicted position coordinates of the traffic signal lamp to the image to be recognized to obtain the recognition result of the image to be recognized includes:
determining an offset;
converting the position coordinate according to the offset to obtain a conversion coordinate;
determining a first scale of the image to be recognized and determining a second scale of a target feature map corresponding to the position coordinates;
calculating a quotient of the first scale and the second scale as a coefficient;
and multiplying the coefficient and the position coordinate to obtain the corresponding position of the position coordinate on the image to be identified, and obtaining the identification result of the image to be identified.
According to the preferred embodiment of the present invention, the artificial intelligence-based traffic signal light recognition method further comprises:
responding to a received detection instruction, and determining a terminal corresponding to the detection instruction;
and sending the identification result to the terminal.
An artificial intelligence based traffic signal light identification apparatus, comprising:
the processing unit is used for responding to the received image to be recognized and carrying out resize processing on the image to be recognized to obtain a target image;
an extraction unit configured to extract target feature information of the target image using a darknet53 network;
the input unit is used for inputting the target characteristic information into a traffic signal lamp recognition model trained in advance and outputting a target characteristic map with at least one scale, wherein the traffic signal lamp recognition model is obtained by adopting a Mixup algorithm and residual error attention network training;
the acquisition unit is used for acquiring a target anchor box of the traffic signal lamp identification model;
the identification unit is used for identifying each target feature map by using the target anchor box, outputting a target anchor box coordinate corresponding to each target feature map and a target score of each target anchor box coordinate, and taking the target anchor box coordinate with the highest target score as the predicted position coordinate of the traffic signal lamp;
and the mapping unit is used for mapping the predicted position coordinates of the traffic signal lamp to the image to be identified to obtain the identification result of the image to be identified.
An electronic device, the electronic device comprising:
a memory storing at least one instruction; and
a processor executing instructions stored in the memory to implement the artificial intelligence based traffic signal identification method.
A computer-readable storage medium having at least one instruction stored therein, the at least one instruction being executable by a processor in an electronic device to implement the artificial intelligence based traffic signal identification method.
According to the technical scheme, the method can respond to the received image to be recognized, resize the image to be recognized to obtain a target image, extract target characteristic information of the target image by utilizing a dark net53 network, input the target characteristic information into a traffic signal lamp recognition model trained in advance, and output a target characteristic image with at least one scale, wherein the traffic signal lamp recognition model is obtained by adopting a Mixup algorithm and a residual attention network training, and can more accurately extract the detailed characteristics of a signal lamp based on the Mixup algorithm and the residual attention network, so that the recall rate and the accuracy of traffic signal lamp detection are improved, the target anchor box of the traffic signal lamp recognition model is obtained, the target anchor box is used for recognizing on each target characteristic image, and the target anchor box coordinate corresponding to each target characteristic image and the target score of each target anchor box coordinate are output, and taking the target anchor box coordinate with the highest target score as a predicted position coordinate of the traffic signal lamp, mapping the predicted position coordinate of the traffic signal lamp to the image to be recognized to obtain a recognition result of the image to be recognized, further realizing automatic recognition of the traffic signal lamp based on an artificial intelligence means, and having higher recognition accuracy.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of the traffic signal light identification method based on artificial intelligence of the present invention.
Fig. 2 is a functional block diagram of a preferred embodiment of the artificial intelligence based traffic signal light recognition apparatus of the present invention.
Fig. 3 is a schematic structural diagram of an electronic device according to a preferred embodiment of the present invention for implementing an artificial intelligence-based traffic signal light recognition method.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in detail with reference to the accompanying drawings and specific embodiments.
Fig. 1 is a flow chart of a traffic signal light recognition method based on artificial intelligence according to a preferred embodiment of the present invention. The order of the steps in the flow chart may be changed and some steps may be omitted according to different needs.
The traffic Signal lamp identification method based on artificial intelligence is applied to one or more electronic devices, wherein the electronic devices are devices capable of automatically performing numerical calculation and/or information processing according to preset or stored instructions, and hardware of the electronic devices includes but is not limited to a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device and the like.
The electronic device may be any electronic product capable of performing human-computer interaction with a user, for example, a Personal computer, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a game machine, an interactive Internet Protocol Television (IPTV), an intelligent wearable device, and the like.
The electronic device may also include a network device and/or a user device. The network device includes, but is not limited to, a single network server, a server group consisting of a plurality of network servers, or a cloud computing (cloud computing) based cloud consisting of a large number of hosts or network servers.
The Network where the electronic device is located includes, but is not limited to, the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.
S10, responding to the received image to be recognized, and performing resize processing on the image to be recognized to obtain a target image.
Through resize processing, the image to be recognized meets the requirement of the model for the size of the image, and automatic recognition by the model is convenient to follow-up use.
And S11, extracting target characteristic information of the target image by using the darknet53 network.
Wherein the target feature information includes, but is not limited to, one or more of the following features in combination:
color features, edge features, shape features, and the like.
In addition, the characteristic information is in a matrix form.
And S12, inputting the target characteristic information into a traffic signal light recognition model trained in advance, and outputting a target characteristic map with at least one scale, wherein the traffic signal light recognition model is obtained by adopting a Mixup algorithm and residual attention network training.
In this embodiment, the traffic signal recognition model also needs to be trained.
Specifically, before the target feature information is input into a traffic signal recognition model trained in advance and a target feature map of at least one scale is output, the artificial intelligence-based traffic signal recognition method further includes:
obtaining a training sample;
performing combined operation on the training samples by taking a preset number as a group to obtain at least one group of training data;
synthesizing each group of training data by using a Mixup algorithm to obtain a synthesized image corresponding to each group of training data;
extracting characteristic information of each composite image by using a dark net53 network;
obtaining a residual block of at least one scale in the darknet53 network;
splicing the characteristic information of each synthetic image and the residual block with the corresponding scale based on a residual attention network to obtain a splicing result;
performing convolution operation on the splicing result to obtain a characteristic diagram of at least one scale of each synthetic image;
obtaining an anchor box obtained by pre-clustering;
identifying on each characteristic diagram by using the anchor box, and outputting the anchor box coordinate corresponding to each characteristic diagram and the score of each anchor box coordinate;
acquiring an anchor box coordinate with the highest score in the feature map of at least one scale of each synthetic image as a prediction coordinate of each synthetic image;
determining the actual coordinates of each composite image;
calculating accuracy and recall rate based on the actual coordinates of each composite image and the predicted coordinates of each composite image;
calculating a value of a loss function;
and when the accuracy reaches a preset accuracy, the recall rate reaches a preset recall rate, and the value of the loss function is lower than a preset loss, stopping training to obtain the traffic signal lamp identification model, and storing the traffic signal lamp identification model on a block chain.
For example: the preset number may be 2, and when the training sample includes a picture 1, a picture 2, and a picture 3, three groups of training data obtained through the combination operation are respectively: picture 1, picture 2; picture 2, picture 3; picture 1, picture 3.
In view of the fact that the color of some automobile tail lamps is yellow, and the influence of various factors such as light and the like is easy to be confused with the traffic signal lamp, a new image enhancement algorithm Mixup is introduced into the traditional YOLOv3 framework, so that the characteristics can be enhanced, the situation that non-signal lamps such as the automobile tail lamps and the like are mistakenly identified as the traffic signal lamps is effectively reduced, and the accuracy of the detection of the traffic signal lamp is improved.
In addition, an attention mechanism (attention) is added into a traditional residual error network, the obtained residual error attention network is introduced into a YOLOv3 frame, it needs to be noted that the original YOLOv3 frame adopts static residual error connection, a residual error value predicted in one step is taken as a final residual error, the original static residual error connection is changed into dynamic residual error connection, namely, residual errors predicted in multiple steps are spliced by using the residual error attention network to be taken as a final residual error value, so that the detailed characteristics of a signal lamp can be extracted more accurately, the accuracy of traffic signal lamp detection is improved due to more accurate extraction of the detailed characteristics, meanwhile, the recall rate represents how many positive examples in sample data are correctly predicted, the probability that the positive examples in the sample data are correctly predicted (namely, the pictures with traffic signal lamps are correctly identified with the traffic signal lamps) is improved, that is, the recall rate of traffic light detection will also increase.
Specifically, the splicing of the feature information of each synthesized image and the residual block with the corresponding scale based on the residual attention network to obtain a splicing result includes:
obtaining a plurality of feature information having the same scale in a plurality of layers of the darknet53 network;
for a plurality of feature information of each scale, performing compression transformation on each feature information to obtain a plurality of compressed data;
performing squeeze processing on the plurality of compressed data based on the residual blocks with corresponding sizes to obtain a plurality of processing results;
calculating the score of each processing result by adopting an attention algorithm;
and determining the processing result with the highest score as the splicing result.
Through the implementation mode, the important information can be more pertinently concentrated on the basis of the attention algorithm, and the accuracy of identification is further improved.
Specifically, the performing convolution operation on the stitching result to obtain at least one scale of the feature map of each composite image includes:
and sequentially inputting the splicing result to a conv _ layer, a conv _ block layer and a conv layer for convolution operation, and outputting a characteristic diagram of at least one scale of each synthetic image.
It should be noted that the layer composition of the conv _ layer, the conv _ block, and the conv layer may be set according to actual requirements, and the present invention is not limited thereto.
For example: it is known from the historical data that the conv _ layer may include 5 layers of convolution, 1 layer of BN layer (Batch Normalization) and 1 layer of active layer, the conv _ block layer may include 1 layer of convolution, 1 layer of BN layer and 1 layer of active layer, and the conv layer may include 1 layer of convolution.
Further, the artificial intelligence-based traffic signal lamp identification method further comprises the following steps:
determining the center point coordinate of the anchor box corresponding to each characteristic diagram, the width and height coordinates of the anchor box corresponding to each characteristic diagram and the score of each anchor box coordinate;
calculating a central point coordinate error according to the central point coordinate of the anchor box corresponding to each feature map;
calculating a width and height coordinate error according to the width and height coordinates of the anchor box corresponding to each feature map;
calculating a target error according to the score of each anchor box coordinate;
calculating a sum of the center point coordinate error, the width-to-height coordinate error, and the target error as the loss function.
The loss function constructed in the above embodiment can evaluate the loss of the model from a plurality of levels, and further improve the effect of the model.
And S13, acquiring a target anchor box of the traffic signal lamp identification model.
And the number of the target anchor boxes is a multiple of the at least one scale, so as to ensure that the target feature map of each scale can obtain the same target anchor boxes.
And S14, identifying each target feature map by using the target anchor box, outputting a target anchor box coordinate corresponding to each target feature map and a target score of each target anchor box coordinate, and taking the target anchor box coordinate with the highest target score as the predicted position coordinate of the traffic signal.
Through the embodiment, the target anchor box coordinate with the highest score can be acquired from the target anchor box coordinates corresponding to each output target feature map and serves as the predicted position coordinate of the traffic signal lamp, and the score is used for further screening, so that the recognition accuracy is improved again.
S15, mapping the predicted position coordinates of the traffic signal lamp to the image to be recognized to obtain the recognition result of the image to be recognized, and storing the recognition result in a block chain.
In this embodiment, the mapping the predicted position coordinates of the traffic signal lamp to the image to be recognized to obtain the recognition result of the image to be recognized includes:
determining an offset;
converting the position coordinate according to the offset to obtain a conversion coordinate;
determining a first scale of the image to be recognized and determining a second scale of a target feature map corresponding to the position coordinates;
calculating a quotient of the first scale and the second scale as a coefficient;
and multiplying the coefficient and the position coordinate to obtain the corresponding position of the position coordinate on the image to be identified, and obtaining the identification result of the image to be identified.
Through the embodiment, the identified position of the traffic signal lamp is mapped on the original image, so that a user can conveniently check the identification result.
Further, the artificial intelligence-based traffic signal lamp identification method further comprises the following steps:
responding to a received detection instruction, and determining a terminal corresponding to the detection instruction;
and sending the identification result to the terminal.
After the identification result is sent to the terminal, the identification result can be used for assisting in detecting whether the traffic signal lamp is damaged or not, whether a red light running behavior exists or not and the like.
In the present embodiment, in order to ensure data security and improve privacy, the recognition result and the traffic signal recognition model are stored in the block chain.
According to the technical scheme, the method can respond to the received image to be recognized, resize the image to be recognized to obtain a target image, extract target characteristic information of the target image by utilizing a dark net53 network, input the target characteristic information into a traffic signal lamp recognition model trained in advance, and output a target characteristic image with at least one scale, wherein the traffic signal lamp recognition model is obtained by adopting a Mixup algorithm and a residual attention network training, and can more accurately extract the detailed characteristics of a signal lamp based on the Mixup algorithm and the residual attention network, so that the recall rate and the accuracy of traffic signal lamp detection are improved, the target anchor box of the traffic signal lamp recognition model is obtained, the target anchor box is used for recognizing on each target characteristic image, and the target anchor box coordinate corresponding to each target characteristic image and the target score of each target anchor box coordinate are output, and taking the target anchor box coordinate with the highest target score as a predicted position coordinate of the traffic signal lamp, mapping the predicted position coordinate of the traffic signal lamp to the image to be recognized to obtain a recognition result of the image to be recognized, further realizing automatic recognition of the traffic signal lamp based on an artificial intelligence means, and having higher recognition accuracy.
Fig. 2 is a functional block diagram of a traffic signal light recognition device based on artificial intelligence according to a preferred embodiment of the present invention. The artificial intelligence-based traffic signal light recognition device 11 comprises a processing unit 110, an extraction unit 111, an input unit 112, an acquisition unit 113, a recognition unit 114, a mapping unit 115, an arithmetic unit 116, a synthesis unit 117, a splicing unit 118, a determination unit 119, a training unit 120 and a sending unit 121. The module/unit referred to in the present invention refers to a series of computer program segments that can be executed by the processor 13 and that can perform a fixed function, and that are stored in the memory 12. In the present embodiment, the functions of the modules/units will be described in detail in the following embodiments.
The processing unit 110 performs resize processing on the image to be recognized in response to the received image to be recognized, so as to obtain a target image.
Through resize processing, the image to be recognized meets the requirement of the model for the size of the image, and automatic recognition by the model is convenient to follow-up use.
The extraction unit 111 extracts the target feature information of the target image using the darknet53 network.
Wherein the target feature information includes, but is not limited to, one or more of the following features in combination:
color features, edge features, shape features, and the like.
In addition, the characteristic information is in a matrix form.
The input unit 112 inputs the target feature information into a traffic signal light recognition model trained in advance, and outputs a target feature map of at least one scale, wherein the traffic signal light recognition model is obtained by using a Mixup algorithm and residual attention network training.
In this embodiment, the traffic signal recognition model also needs to be trained.
Specifically, before the target feature information is input into a traffic signal light recognition model trained in advance and a target feature map of at least one scale is output, the obtaining unit 113 obtains a training sample;
the arithmetic unit 116 performs a combination operation on the training samples in a group of preset numbers to obtain at least one group of training data;
the synthesizing unit 117 synthesizes each set of training data by using a Mixup algorithm to obtain a synthesized image corresponding to each set of training data;
the extraction unit 111 extracts the feature information of each synthetic image using the darknet53 network;
the obtaining unit 113 obtains at least one scale of residual block in the darknet53 network;
the splicing unit 118 splices the feature information of each synthesized image and the residual block with the corresponding scale based on the residual attention network to obtain a splicing result;
the operation unit 116 performs convolution operation on the splicing result to obtain a feature map of at least one scale of each composite image;
the acquiring unit 113 acquires an anchor box obtained by pre-clustering;
the recognition unit 114 recognizes each feature map by using the anchor box, and outputs an anchor box coordinate corresponding to each feature map and a score of each anchor box coordinate;
the acquisition unit 113 acquires the anchorbox coordinate with the highest score in the feature map of at least one scale of each synthetic image as the predicted coordinate of each synthetic image;
the determination unit 119 determines the actual coordinates of each synthesized image;
the arithmetic unit 116 calculates an accuracy and a recall rate based on the actual coordinates of each synthesized image and the predicted coordinates of each synthesized image;
the arithmetic unit 116 calculates the value of the loss function;
when the accuracy reaches a preset accuracy, the recall rate reaches a preset recall rate, and the value of the loss function is lower than a preset loss, the training unit 120 stops training to obtain the traffic signal lamp identification model, and stores the traffic signal lamp identification model on a block chain.
For example: the preset number may be 2, and when the training sample includes a picture 1, a picture 2, and a picture 3, three groups of training data obtained through the combination operation are respectively: picture 1, picture 2; picture 2, picture 3; picture 1, picture 3.
In view of the fact that the color of some automobile tail lamps is yellow, and the influence of various factors such as light and the like is easy to be confused with the traffic signal lamp, a new image enhancement algorithm Mixup is introduced into the traditional YOLOv3 framework, so that the characteristics can be enhanced, the situation that non-signal lamps such as the automobile tail lamps and the like are mistakenly identified as the traffic signal lamps is effectively reduced, and the accuracy of the detection of the traffic signal lamp is improved.
In addition, an attention mechanism (attention) is added into a traditional residual error network, the obtained residual error attention network is introduced into a YOLOv3 frame, it needs to be noted that the original YOLOv3 frame adopts static residual error connection, a residual error value predicted in one step is taken as a final residual error, the original static residual error connection is changed into dynamic residual error connection, namely, residual errors predicted in multiple steps are spliced by using the residual error attention network to be taken as a final residual error value, so that the detailed characteristics of a signal lamp can be extracted more accurately, the accuracy of traffic signal lamp detection is improved due to more accurate extraction of the detailed characteristics, meanwhile, the recall rate represents how many positive examples in sample data are correctly predicted, the probability that the positive examples in the sample data are correctly predicted (namely, the pictures with traffic signal lamps are correctly identified with the traffic signal lamps) is improved, that is, the recall rate of traffic light detection will also increase.
Specifically, the splicing unit 118 splices the feature information of each synthesized image and the residual block with the corresponding scale based on the residual attention network, and the obtained splicing result includes:
obtaining a plurality of feature information having the same scale in a plurality of layers of the darknet53 network;
for a plurality of feature information of each scale, performing compression transformation on each feature information to obtain a plurality of compressed data;
performing squeeze processing on the plurality of compressed data based on the residual blocks with corresponding sizes to obtain a plurality of processing results;
calculating the score of each processing result by adopting an attention algorithm;
and determining the processing result with the highest score as the splicing result.
Through the implementation mode, the important information can be more pertinently concentrated on the basis of the attention algorithm, and the accuracy of identification is further improved.
Specifically, the performing a convolution operation on the stitching result by the operation unit 116 to obtain at least one scale feature map of each composite image includes:
and sequentially inputting the splicing result to a conv _ layer, a conv _ block layer and a conv layer for convolution operation, and outputting a characteristic diagram of at least one scale of each synthetic image.
It should be noted that the layer composition of the conv _ layer, the conv _ block, and the conv layer may be set according to actual requirements, and the present invention is not limited thereto.
For example: it is known from the historical data that the conv _ layer may include 5 layers of convolution, 1 layer of BN layer (batch normalization), and 1 layer of active layer, the conv _ block layer may include 1 layer of convolution, 1 layer of BN layer, and 1 layer of active layer, and the conv layer may include 1 layer of convolution.
Further, the determining unit 119 determines the center point coordinate of the anchor box corresponding to each feature map, the width and height coordinates of the anchor box corresponding to each feature map, and the score of each anchor box coordinate;
the arithmetic unit 116 calculates a central point coordinate error according to the central point coordinate of the anchor box corresponding to each feature map;
the arithmetic unit 116 calculates the width and height coordinate error according to the width and height coordinates of the anchor box corresponding to each feature map;
the arithmetic unit 116 calculates a target error according to the score of each anchor box coordinate;
the arithmetic unit 116 calculates the sum of the center point coordinate error, the width and height coordinate error, and the target error as the loss function.
The loss function constructed in the above embodiment can evaluate the loss of the model from a plurality of levels, and further improve the effect of the model.
The acquisition unit 113 acquires a target anchor box of the traffic signal light recognition model.
And the number of the target anchor boxes is a multiple of the at least one scale, so as to ensure that the target feature map of each scale can obtain the same target anchor boxes.
The recognition unit 114 recognizes each target feature map by using the target anchor box, outputs a target anchor box coordinate corresponding to each target feature map and a target score of each target anchor box coordinate, and takes the target anchor box coordinate with the highest target score as the predicted position coordinate of the traffic signal.
Through the embodiment, the target anchor box coordinate with the highest score can be acquired from the target anchor box coordinates corresponding to each output target feature map and serves as the predicted position coordinate of the traffic signal lamp, and the score is used for further screening, so that the recognition accuracy is improved again.
The mapping unit 115 maps the predicted position coordinates of the traffic signal lamp to the image to be recognized, obtains a recognition result of the image to be recognized, and stores the recognition result in a block chain.
In this embodiment, the mapping unit 115 maps the predicted position coordinates of the traffic signal lamp to the image to be recognized, and obtaining the recognition result of the image to be recognized includes:
determining an offset;
converting the position coordinate according to the offset to obtain a conversion coordinate;
determining a first scale of the image to be recognized and determining a second scale of a target feature map corresponding to the position coordinates;
calculating a quotient of the first scale and the second scale as a coefficient;
and multiplying the coefficient and the position coordinate to obtain the corresponding position of the position coordinate on the image to be identified, and obtaining the identification result of the image to be identified.
Through the embodiment, the identified position of the traffic signal lamp is mapped on the original image, so that a user can conveniently check the identification result.
Further, the determination unit 119 determines, in response to the received detection instruction, a terminal corresponding to the detection instruction;
the transmitting unit 121 transmits the recognition result to the terminal.
After the identification result is sent to the terminal, the identification result can be used for assisting in detecting whether the traffic signal lamp is damaged or not, whether a red light running behavior exists or not and the like.
In the present embodiment, in order to ensure data security and improve privacy, the recognition result and the traffic signal recognition model are stored in the block chain.
According to the technical scheme, the method can respond to the received image to be recognized, resize the image to be recognized to obtain a target image, extract target characteristic information of the target image by utilizing a dark net53 network, input the target characteristic information into a traffic signal lamp recognition model trained in advance, and output a target characteristic image with at least one scale, wherein the traffic signal lamp recognition model is obtained by adopting a Mixup algorithm and a residual attention network training, and can more accurately extract the detailed characteristics of a signal lamp based on the Mixup algorithm and the residual attention network, so that the recall rate and the accuracy of traffic signal lamp detection are improved, the target anchor box of the traffic signal lamp recognition model is obtained, the target anchor box is used for recognizing on each target characteristic image, and the target anchor box coordinate corresponding to each target characteristic image and the target score of each target anchor box coordinate are output, and taking the target anchor box coordinate with the highest target score as a predicted position coordinate of the traffic signal lamp, mapping the predicted position coordinate of the traffic signal lamp to the image to be recognized to obtain a recognition result of the image to be recognized, further realizing automatic recognition of the traffic signal lamp based on an artificial intelligence means, and having higher recognition accuracy.
Fig. 3 is a schematic structural diagram of an electronic device according to a preferred embodiment of the present invention for implementing an artificial intelligence-based traffic signal light recognition method.
The electronic device 1 may comprise a memory 12, a processor 13 and a bus, and may further comprise a computer program, such as an artificial intelligence based traffic signal recognition program, stored in the memory 12 and executable on the processor 13.
It will be understood by those skilled in the art that the schematic diagram is merely an example of the electronic device 1, and does not constitute a limitation to the electronic device 1, the electronic device 1 may have a bus-type structure or a star-type structure, the electronic device 1 may further include more or less hardware or software than those shown in the figures, or different component arrangements, for example, the electronic device 1 may further include an input and output device, a network access device, and the like.
It should be noted that the electronic device 1 is only an example, and other existing or future electronic products, such as those that can be adapted to the present invention, should also be included in the scope of the present invention, and are included herein by reference.
The memory 12 includes at least one type of readable storage medium, which includes flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory 12 may in some embodiments be an internal storage unit of the electronic device 1, for example a removable hard disk of the electronic device 1. The memory 12 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the electronic device 1. Further, the memory 12 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 12 may be used not only to store application software installed in the electronic device 1 and various types of data, such as codes of an artificial intelligence-based traffic light recognition program, etc., but also to temporarily store data that has been output or is to be output.
The processor 13 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 13 is a Control Unit (Control Unit) of the electronic device 1, connects various components of the electronic device 1 by using various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (for example, executing an artificial intelligence-based traffic light recognition program, etc.) stored in the memory 12 and calling data stored in the memory 12.
The processor 13 executes an operating system of the electronic device 1 and various installed application programs. The processor 13 executes the application program to implement the steps in each of the above-described embodiments of artificial intelligence based traffic signal identification methods, such as the steps shown in fig. 1.
Illustratively, the computer program may be divided into one or more modules/units, which are stored in the memory 12 and executed by the processor 13 to accomplish the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a processing unit 110, an extraction unit 111, an input unit 112, an acquisition unit 113, a recognition unit 114, a mapping unit 115, an arithmetic unit 116, a synthesis unit 117, a concatenation unit 118, a determination unit 119, a training unit 120, a transmission unit 121.
Alternatively, the processor 13, when executing the computer program, implements the functions of the modules/units in the above device embodiments, for example:
responding to the received image to be recognized, and performing resize processing on the image to be recognized to obtain a target image;
extracting target characteristic information of the target image by using a dark net53 network;
inputting the target characteristic information into a pre-trained traffic signal lamp identification model, and outputting a target characteristic map with at least one scale, wherein the traffic signal lamp identification model is obtained by adopting a Mixup algorithm and residual error attention network training;
acquiring a target anchor box of the traffic signal lamp identification model;
identifying each target feature map by using the target anchor box, outputting a target anchor box coordinate corresponding to each target feature map and a target score of each target anchor box coordinate, and taking the target anchor box coordinate with the highest target score as a predicted position coordinate of the traffic signal lamp;
and mapping the predicted position coordinates of the traffic signal lamp to the image to be identified to obtain an identification result of the image to be identified, and storing the identification result in a block chain.
The integrated unit implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a computer device, or a network device) or a processor (processor) to execute the portions of the artificial intelligence based traffic signal identification method according to the embodiments of the present invention.
The integrated modules/units of the electronic device 1 may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented.
Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
Further, the computer-usable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the blockchain node, and the like.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one arrow is shown in FIG. 3, but this does not indicate only one bus or one type of bus. The bus is arranged to enable connection communication between the memory 12 and at least one processor 13 or the like.
Although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 13 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Further, the electronic device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices.
Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
Fig. 3 only shows the electronic device 1 with components 12-13, and it will be understood by a person skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components.
Referring to fig. 1, the memory 12 of the electronic device 1 stores a plurality of instructions to implement an artificial intelligence based traffic signal recognition method, and the processor 13 can execute the plurality of instructions to implement:
responding to the received image to be recognized, and performing resize processing on the image to be recognized to obtain a target image;
extracting target characteristic information of the target image by using a dark net53 network;
inputting the target characteristic information into a pre-trained traffic signal lamp identification model, and outputting a target characteristic map with at least one scale, wherein the traffic signal lamp identification model is obtained by adopting a Mixup algorithm and residual error attention network training;
acquiring a target anchor box of the traffic signal lamp identification model;
identifying each target feature map by using the target anchor box, outputting a target anchor box coordinate corresponding to each target feature map and a target score of each target anchor box coordinate, and taking the target anchor box coordinate with the highest target score as a predicted position coordinate of the traffic signal lamp;
and mapping the predicted position coordinates of the traffic signal lamp to the image to be identified to obtain an identification result of the image to be identified, and storing the identification result in a block chain.
Specifically, the processor 13 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the instruction, which is not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A traffic signal lamp identification method based on artificial intelligence is characterized by comprising the following steps:
responding to the received image to be recognized, and performing resize processing on the image to be recognized to obtain a target image;
extracting target characteristic information of the target image by using a dark net53 network;
inputting the target characteristic information into a pre-trained traffic signal lamp identification model, and outputting a target characteristic map with at least one scale, wherein the traffic signal lamp identification model is obtained by adopting a Mixup algorithm and residual error attention network training;
acquiring a target anchor box of the traffic signal lamp identification model;
identifying each target feature map by using the target anchor box, outputting a target anchor box coordinate corresponding to each target feature map and a target score of each target anchor box coordinate, and taking the target anchor box coordinate with the highest target score as a predicted position coordinate of the traffic signal lamp;
and mapping the predicted position coordinates of the traffic signal lamp to the image to be identified to obtain an identification result of the image to be identified, and storing the identification result in a block chain.
2. The artificial intelligence based traffic signal recognition method according to claim 1, wherein before inputting the target feature information into a pre-trained traffic signal recognition model and outputting at least one scale of target feature map, the artificial intelligence based traffic signal recognition method further comprises:
obtaining a training sample;
performing combined operation on the training samples by taking a preset number as a group to obtain at least one group of training data;
synthesizing each group of training data by using a Mixup algorithm to obtain a synthesized image corresponding to each group of training data;
extracting characteristic information of each composite image by using a dark net53 network;
obtaining a residual block of at least one scale in the darknet53 network;
splicing the characteristic information of each synthetic image and the residual block with the corresponding scale based on a residual attention network to obtain a splicing result;
performing convolution operation on the splicing result to obtain a characteristic diagram of at least one scale of each synthetic image;
obtaining an anchor box obtained by pre-clustering;
identifying on each characteristic diagram by using the anchor box, and outputting the anchor box coordinate corresponding to each characteristic diagram and the score of each anchor box coordinate;
acquiring an anchor box coordinate with the highest score in the feature map of at least one scale of each synthetic image as a prediction coordinate of each synthetic image;
determining the actual coordinates of each composite image;
calculating accuracy and recall rate based on the actual coordinates of each composite image and the predicted coordinates of each composite image;
calculating a value of a loss function;
and when the accuracy reaches a preset accuracy, the recall rate reaches a preset recall rate, and the value of the loss function is lower than a preset loss, stopping training to obtain the traffic signal lamp identification model, and storing the traffic signal lamp identification model on a block chain.
3. The method for identifying traffic signal lamps based on artificial intelligence as claimed in claim 2, wherein the splicing the feature information of each synthesized image and the residual block with the corresponding scale based on the residual attention network to obtain the spliced result comprises:
obtaining a plurality of feature information having the same scale in a plurality of layers of the darknet53 network;
for a plurality of feature information of each scale, performing compression transformation on each feature information to obtain a plurality of compressed data;
performing squeeze processing on the plurality of compressed data based on the residual blocks with corresponding sizes to obtain a plurality of processing results;
calculating the score of each processing result by adopting an attention algorithm;
and determining the processing result with the highest score as the splicing result.
4. The method for identifying traffic signal lamps based on artificial intelligence as claimed in claim 2, wherein the performing convolution operation on the stitching result to obtain at least one scale of feature map of each composite image comprises:
and sequentially inputting the splicing result to a conv _ layer, a conv _ block layer and a conv layer for convolution operation, and outputting a characteristic diagram of at least one scale of each synthetic image.
5. The artificial intelligence based traffic signal recognition method of claim 2, wherein the artificial intelligence based traffic signal recognition method further comprises:
determining the center point coordinate of the anchor box corresponding to each characteristic diagram, the width and height coordinates of the anchor box corresponding to each characteristic diagram and the score of each anchor box coordinate;
calculating a central point coordinate error according to the central point coordinate of the anchor box corresponding to each feature map;
calculating a width and height coordinate error according to the width and height coordinates of the anchor box corresponding to each feature map;
calculating a target error according to the score of each anchor box coordinate;
calculating a sum of the center point coordinate error, the width-to-height coordinate error, and the target error as the loss function.
6. The artificial intelligence based traffic signal lamp identification method according to claim 1, wherein the mapping the predicted location coordinates of the traffic signal lamp onto the image to be identified to obtain the identification result of the image to be identified comprises:
determining an offset;
converting the position coordinate according to the offset to obtain a conversion coordinate;
determining a first scale of the image to be recognized and determining a second scale of a target feature map corresponding to the position coordinates;
calculating a quotient of the first scale and the second scale as a coefficient;
and multiplying the coefficient and the position coordinate to obtain the corresponding position of the position coordinate on the image to be identified, and obtaining the identification result of the image to be identified.
7. The artificial intelligence based traffic signal recognition method of claim 1, wherein the artificial intelligence based traffic signal recognition method further comprises:
responding to a received detection instruction, and determining a terminal corresponding to the detection instruction;
and sending the identification result to the terminal.
8. An artificial intelligence-based traffic signal light recognition apparatus, comprising:
the processing unit is used for responding to the received image to be recognized and carrying out resize processing on the image to be recognized to obtain a target image;
an extraction unit configured to extract target feature information of the target image using a darknet53 network;
the input unit is used for inputting the target characteristic information into a traffic signal lamp recognition model trained in advance and outputting a target characteristic map with at least one scale, wherein the traffic signal lamp recognition model is obtained by adopting a Mixup algorithm and residual error attention network training;
the acquisition unit is used for acquiring a target anchor box of the traffic signal lamp identification model;
the identification unit is used for identifying each target feature map by using the target anchor box, outputting a target anchor box coordinate corresponding to each target feature map and a target score of each target anchor box coordinate, and taking the target anchor box coordinate with the highest target score as the predicted position coordinate of the traffic signal lamp;
and the mapping unit is used for mapping the predicted position coordinates of the traffic signal lamp to the image to be identified to obtain the identification result of the image to be identified.
9. An electronic device, characterized in that the electronic device comprises:
a memory storing at least one instruction; and
a processor executing instructions stored in the memory to implement the artificial intelligence based traffic signal identification method of any one of claims 1 to 7.
10. A computer-readable storage medium characterized by: the computer-readable storage medium has stored therein at least one instruction that is executed by a processor in an electronic device to implement the artificial intelligence based traffic signal identification method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010695795.8A CN111738212B (en) | 2020-07-20 | 2020-07-20 | Traffic signal lamp identification method, device, equipment and medium based on artificial intelligence |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010695795.8A CN111738212B (en) | 2020-07-20 | 2020-07-20 | Traffic signal lamp identification method, device, equipment and medium based on artificial intelligence |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111738212A true CN111738212A (en) | 2020-10-02 |
| CN111738212B CN111738212B (en) | 2020-11-20 |
Family
ID=72654978
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010695795.8A Active CN111738212B (en) | 2020-07-20 | 2020-07-20 | Traffic signal lamp identification method, device, equipment and medium based on artificial intelligence |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111738212B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112132032A (en) * | 2020-09-23 | 2020-12-25 | 平安国际智慧城市科技股份有限公司 | Traffic sign detection method and device, electronic equipment and storage medium |
| CN112418278A (en) * | 2020-11-05 | 2021-02-26 | 中保车服科技服务股份有限公司 | Multi-class object detection method, terminal device and storage medium |
| CN112488083A (en) * | 2020-12-24 | 2021-03-12 | 杭州电子科技大学 | Traffic signal lamp identification method, device and medium for extracting key points based on heatmap |
| CN114630396A (en) * | 2021-12-31 | 2022-06-14 | 厦门阳光恩耐照明有限公司 | Intelligent lamp Bluetooth configuration method and system based on image recognition |
| WO2023273358A1 (en) * | 2021-06-30 | 2023-01-05 | 上海商汤智能科技有限公司 | Traffic sign recognition method and apparatus, electronic device, and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650641A (en) * | 2016-12-05 | 2017-05-10 | 北京文安智能技术股份有限公司 | Traffic light positioning and identification method, device and system |
| CN108108761A (en) * | 2017-12-21 | 2018-06-01 | 西北工业大学 | A kind of rapid transit signal lamp detection method based on depth characteristic study |
| CN109344821A (en) * | 2018-08-30 | 2019-02-15 | 西安电子科技大学 | Small target detecting method based on Fusion Features and deep learning |
| CN109508580A (en) * | 2017-09-15 | 2019-03-22 | 百度在线网络技术（北京）有限公司 | Traffic lights recognition methods and device |
| CN109657622A (en) * | 2018-12-21 | 2019-04-19 | 广东工业大学 | A kind of detection of traffic lights and recognition methods, device and equipment |
| CN109740662A (en) * | 2018-12-28 | 2019-05-10 | 成都思晗科技股份有限公司 | Image object detection method based on YOLO frame |
| CN110069986A (en) * | 2019-03-13 | 2019-07-30 | 北京联合大学 | A kind of traffic lights recognition methods and system based on mixed model |
| CN110197152A (en) * | 2019-05-28 | 2019-09-03 | 南京邮电大学 | A kind of road target recognition methods for automated driving system |
| CN110210621A (en) * | 2019-06-06 | 2019-09-06 | 大连理工大学 | A kind of object detection method based on residual error network improvement |
| CN110472467A (en) * | 2019-04-08 | 2019-11-19 | 江西理工大学 | The detection method for transport hub critical object based on YOLO v3 |
| CN110852347A (en) * | 2019-10-17 | 2020-02-28 | 天津大学 | Fire detection method using improved YOLO v3 |
| CN111310861A (en) * | 2020-03-27 | 2020-06-19 | 西安电子科技大学 | License plate recognition and positioning method based on deep neural network |
-
2020
- 2020-07-20 CN CN202010695795.8A patent/CN111738212B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106650641A (en) * | 2016-12-05 | 2017-05-10 | 北京文安智能技术股份有限公司 | Traffic light positioning and identification method, device and system |
| CN109508580A (en) * | 2017-09-15 | 2019-03-22 | 百度在线网络技术（北京）有限公司 | Traffic lights recognition methods and device |
| CN108108761A (en) * | 2017-12-21 | 2018-06-01 | 西北工业大学 | A kind of rapid transit signal lamp detection method based on depth characteristic study |
| CN109344821A (en) * | 2018-08-30 | 2019-02-15 | 西安电子科技大学 | Small target detecting method based on Fusion Features and deep learning |
| CN109657622A (en) * | 2018-12-21 | 2019-04-19 | 广东工业大学 | A kind of detection of traffic lights and recognition methods, device and equipment |
| CN109740662A (en) * | 2018-12-28 | 2019-05-10 | 成都思晗科技股份有限公司 | Image object detection method based on YOLO frame |
| CN110069986A (en) * | 2019-03-13 | 2019-07-30 | 北京联合大学 | A kind of traffic lights recognition methods and system based on mixed model |
| CN110472467A (en) * | 2019-04-08 | 2019-11-19 | 江西理工大学 | The detection method for transport hub critical object based on YOLO v3 |
| CN110197152A (en) * | 2019-05-28 | 2019-09-03 | 南京邮电大学 | A kind of road target recognition methods for automated driving system |
| CN110210621A (en) * | 2019-06-06 | 2019-09-06 | 大连理工大学 | A kind of object detection method based on residual error network improvement |
| CN110852347A (en) * | 2019-10-17 | 2020-02-28 | 天津大学 | Fire detection method using improved YOLO v3 |
| CN111310861A (en) * | 2020-03-27 | 2020-06-19 | 西安电子科技大学 | License plate recognition and positioning method based on deep neural network |
Non-Patent Citations (1)
| Title |
|---|
| HUIBING ZHANG 等: ""Real-Time Detection Method for Small Traffic Signs Based on Yolov3"", 《IEEE ACCESS》 * |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112132032A (en) * | 2020-09-23 | 2020-12-25 | 平安国际智慧城市科技股份有限公司 | Traffic sign detection method and device, electronic equipment and storage medium |
| CN112418278A (en) * | 2020-11-05 | 2021-02-26 | 中保车服科技服务股份有限公司 | Multi-class object detection method, terminal device and storage medium |
| CN112488083A (en) * | 2020-12-24 | 2021-03-12 | 杭州电子科技大学 | Traffic signal lamp identification method, device and medium for extracting key points based on heatmap |
| CN112488083B (en) * | 2020-12-24 | 2024-04-05 | 杭州电子科技大学 | Identification method, device and medium of traffic signal lamp based on key point extraction of hetmap |
| WO2023273358A1 (en) * | 2021-06-30 | 2023-01-05 | 上海商汤智能科技有限公司 | Traffic sign recognition method and apparatus, electronic device, and storage medium |
| CN114630396A (en) * | 2021-12-31 | 2022-06-14 | 厦门阳光恩耐照明有限公司 | Intelligent lamp Bluetooth configuration method and system based on image recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111738212B (en) | 2020-11-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111738212B (en) | Traffic signal lamp identification method, device, equipment and medium based on artificial intelligence | |
| CN111476324B (en) | Traffic data labeling method, device, equipment and medium based on artificial intelligence | |
| CN111476225B (en) | In-vehicle human face identification method, device, equipment and medium based on artificial intelligence | |
| CN111931729B (en) | Pedestrian detection method, device, equipment and medium based on artificial intelligence | |
| CN112052850A (en) | License plate recognition method and device, electronic equipment and storage medium | |
| CN112231586A (en) | Course recommendation method, device, equipment and medium based on transfer learning | |
| CN112699775A (en) | Certificate identification method, device and equipment based on deep learning and storage medium | |
| CN112396005A (en) | Biological characteristic image recognition method and device, electronic equipment and readable storage medium | |
| CN112329777B (en) | Character recognition method, device, equipment and medium based on direction detection | |
| CN112507934A (en) | Living body detection method, living body detection device, electronic apparatus, and storage medium | |
| CN113705462A (en) | Face recognition method and device, electronic equipment and computer readable storage medium | |
| CN111950621A (en) | Target data detection method, device, equipment and medium based on artificial intelligence | |
| CN115063632A (en) | Vehicle damage identification method, device, equipment and medium based on artificial intelligence | |
| CN111985545A (en) | Target data detection method, device, equipment and medium based on artificial intelligence | |
| CN112115890B (en) | Drunk driving identification method, device, equipment and medium based on artificial intelligence | |
| CN112101191A (en) | Expression recognition method, device, equipment and medium based on frame attention network | |
| CN111932413A (en) | Case element extraction method, case element extraction device, case element extraction equipment and case element extraction medium | |
| CN111488876B (en) | License plate recognition method, device, equipment and medium based on artificial intelligence | |
| CN115346095A (en) | Visual question answering method, device, equipment and storage medium | |
| CN112905817B (en) | Image retrieval method and device based on sorting algorithm and related equipment | |
| CN115239958A (en) | Wheel hub damage detection method based on artificial intelligence and related equipment | |
| CN112132037B (en) | Pavement detection method, device, equipment and medium based on artificial intelligence | |
| CN114913518A (en) | License plate recognition method, device, equipment and medium based on image processing | |
| CN113128440A (en) | Target object identification method, device, equipment and storage medium based on edge equipment | |
| CN112132140B (en) | Vehicle brand identification method, device, equipment and medium based on artificial intelligence |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |