CN117496486A - Traffic light shape recognition method, readable storage medium and intelligent device - Google Patents
Traffic light shape recognition method, readable storage medium and intelligent device Download PDFInfo
- Publication number
- CN117496486A CN117496486A CN202311817951.3A CN202311817951A CN117496486A CN 117496486 A CN117496486 A CN 117496486A CN 202311817951 A CN202311817951 A CN 202311817951A CN 117496486 A CN117496486 A CN 117496486A
- Authority
- CN
- China
- Prior art keywords
- traffic light
- image
- identification
- shape
- original image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000012549 training Methods 0.000 claims description 37
- 238000001514 detection method Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 abstract description 7
- 230000000694 effects Effects 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 5
- 238000002372 labelling Methods 0.000 description 5
- 238000013461 design Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000033228 biological regulation Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 238000012800 visualization Methods 0.000 description 2
- 101100248200 Arabidopsis thaliana RGGB gene Proteins 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Traffic Control Systems (AREA)
Abstract
The application relates to the technical field of automatic driving, and particularly provides a traffic light shape recognition method, a readable storage medium and intelligent equipment, which aim at solving the problem of how to effectively improve the recognition effect of the traffic light shape at night. For the purpose, according to the method, an original image file of the acquired traffic light is acquired, an identification image for image identification is acquired according to the original image file, image identification is carried out according to the identification image, and a shape identification result of the traffic light is obtained. Because the original image file contains richer information, the identification image obtained based on the original image file also has richer information, and further, more accurate traffic light shape identification results can be obtained according to the identification image, and the conditions that ISP (Internet service provider) processing exposure is too high and traffic light shapes are difficult to recognize due to too low night environment brightness can be effectively improved. Therefore, more accurate traffic light identification attributes are obtained according to the traffic light shape identification result.
Description
Technical Field
The application relates to the technical field of automatic driving, and particularly provides a traffic light shape recognition method, a readable storage medium and intelligent equipment.
Background
In the real vehicle road test process, the situation that traffic light semantic categories cannot be identified due to traffic light overexposure frequently occurs in a night scene. While the current automatic driving technology of unbinding high-precision map (HDmap-Free) needs the function of sensing traffic lights. Among them, identifying the properties of traffic lights (the semantics of circles, arrows, etc.) is a very important ring. Thus, it is extremely important to improve the ability to recognize traffic light shape in night scenes.
However, the existing scheme for improving the recognition effect of the night traffic light is generally to adjust the parameters of the ISP (Image Signal Processor ). While adjusting the ISP parameters can improve the recognition of night-time traffic lights, repeated adjustments of the ISP parameters may have an impact on other tasks and the improvement is not generally apparent.
Accordingly, there is a need in the art for a new traffic light shape recognition scheme to address the above-described problems.
Disclosure of Invention
The present application has been made to overcome the above drawbacks, and provides a solution or at least partially solves the problem of how to effectively improve the recognition effect of the night-time traffic light shape.
In a first aspect, the present application provides a traffic light shape recognition method, the method comprising:
acquiring an original image file of an acquired traffic light;
acquiring an identification image for image identification based on the original image file;
based on the identification image, performing image identification to obtain a shape identification result of the traffic light
In one technical scheme of the traffic light shape recognition method, the performing image recognition based on the recognition image to obtain a shape recognition result of the traffic light includes:
based on the identification image, applying a trained image identification model to carry out image identification, and obtaining a shape identification result of the traffic light;
the trained image recognition model is obtained through training based on a training data set, and the training data set contains annotation images generated based on an original image file of a traffic light used for training.
In one technical scheme of the traffic light shape recognition method, the method further comprises the step of generating the training data set according to the following steps:
converting the original image file into a plurality of RGB images with different brightness aiming at each original image file for training so as to obtain a plurality of RGB images;
splicing the RGB images to obtain a spliced image;
and marking the shape of the traffic light according to the spliced image so as to generate the training data set.
In one technical scheme of the traffic light shape recognition method, the marking the shape of the traffic light according to the spliced image includes:
and selecting an RGB image with the highest definition of the traffic light shape from a plurality of RGB images according to the shape of each traffic light on the spliced image, and marking the shape of the traffic light.
In one technical scheme of the traffic light shape recognition method, the converting the original image file into a plurality of RGB images with different brightness includes:
according to a plurality of preset brightness parameters, the original image file is adjusted to obtain a plurality of adjusted original image files;
and performing color space conversion on the adjusted original image file to obtain a plurality of RGB images with different brightness.
In one technical scheme of the traffic light shape recognition method, the obtaining a recognition image for image recognition based on the original image file includes:
acquiring data of four channels of the original image file;
according to the data of the four channels, data combination is carried out to obtain RGB image form data;
and acquiring the identification image according to the RGB image form data.
In one technical scheme of the traffic light shape recognition method, the obtaining the recognition image according to the RGB image form data includes:
the RGB image form data is resized to obtain the identification image.
In one technical scheme of the traffic light shape recognition method, the performing image recognition based on the recognition image to obtain a shape recognition result of the traffic light includes:
based on the identification image, carrying out traffic light target detection to obtain a target detection result;
acquiring a local image of the traffic light according to the target detection result;
and carrying out classification and identification according to the local images, and obtaining the shape identification result of the traffic lights.
In a second aspect, a computer readable storage medium is provided, in which a plurality of program codes are stored, the program codes being adapted to be loaded and run by a processor to perform the traffic light shape recognition method according to any one of the above-mentioned traffic light shape recognition methods.
In a third aspect, there is provided a smart device comprising:
at least one processor;
and a memory communicatively coupled to the at least one processor;
the memory stores a computer program, and the computer program when executed by the at least one processor implements the traffic light shape recognition method according to any one of the above-mentioned traffic light shape recognition methods.
The technical scheme has at least one or more of the following beneficial effects:
in the technical scheme of implementing the application, the method acquires the identification image for image identification according to the acquired original image file of the traffic light, and performs image identification according to the identification image to acquire the shape identification result of the traffic light. Through the configuration mode, the original image file contains richer information, so that the identification image obtained based on the original image file also has richer information, and further, a more accurate traffic light shape identification result can be obtained according to the identification image, and the conditions that ISP processing exposure is too high and traffic light shapes are difficult to recognize due to too low night environment brightness can be effectively improved. Therefore, more accurate traffic light identification attributes such as round cakes, arrows, arrow directions and the like are obtained according to the traffic light shape identification result, and accurate driving prediction, planning and the like are performed based on the traffic light identification attributes.
Further, the method and the device apply the trained image recognition model to carry out image recognition on the recognition image, and because the trained image recognition model is obtained based on training data set training, and the training data set contains the annotation file generated based on the original image file of the traffic light used for training, the trained recognition model can obtain more accurate shape recognition results of the traffic light, and further recognition of the shape of the traffic light at night is improved.
Further, the training data set for training the image recognition model converts each original image file for training into a plurality of RGB images with different brightness, and then the RGB images are spliced, so that the spliced RGB images are mutually referenced, the shape marking of the traffic light is realized, more accurate and rich label information can be obtained, the image recognition model obtained through training can obtain higher recognition accuracy, and more accurate shape recognition results of the traffic light are obtained in the recognition process of the recognition image.
Drawings
The disclosure of the present application will become more readily understood with reference to the accompanying drawings. As will be readily appreciated by those skilled in the art: these drawings are for illustrative purposes only and are not intended to limit the scope of the present application. Wherein:
FIG. 1 is a flow chart illustrating the main steps of a traffic light shape recognition method according to one embodiment of the present application;
FIG. 2 is a schematic diagram of a comparison of an identification image acquired in accordance with an embodiment of the present application and a RGB image of a directly acquired traffic light;
FIG. 3 is a flow chart of main steps of traffic light shape recognition according to RGB images in the prior art;
FIG. 4 is a flow chart illustrating the main steps of a traffic light shape recognition method according to one implementation of an embodiment of the present application;
FIG. 5 is a schematic illustration of an original image file;
FIG. 6 is a schematic view of stitched images according to an implementation of an embodiment of the present application;
FIG. 7 is a visual illustration of a shape recognition result of an acquired traffic light according to one implementation of an embodiment of the present application;
FIG. 8 is a visual representation of the prior art traffic light shape recognition results from RGB images;
FIG. 9 is a schematic diagram of a connection of a processor and a memory according to one embodiment of the present application.
Detailed Description
Some embodiments of the present application are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are merely for explaining the technical principles of the present application, and are not intended to limit the scope of the present application.
In the description of the present application, a "module," "processor" may include hardware, software, or a combination of both. A module may comprise hardware circuitry, various suitable sensors, communication ports, memory, or software components, such as program code, or a combination of software and hardware. The processor may be a central processor, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and/or signal processing functions. The processor may be implemented in software, hardware, or a combination of both. Non-transitory computer readable storage media include any suitable medium that can store program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, and the like. The term "a and/or B" means all possible combinations of a and B, such as a alone, B alone or a and B. The term "at least one A or B" or "at least one of A and B" has a meaning similar to "A and/or B" and may include A alone, B alone or A and B. The singular forms "a", "an" and "the" include plural referents.
Some terms involved in the present application are explained first.
An autopilot system (Automated Driving Systems, ADS), which means that the system will continue to perform all dynamic driving tasks (Dynamic Driving Task, DDT) within its design operating range (Operational Domain Design, ODD). That is, the machine system is allowed to fully take over the task of the vehicle autonomous handling under the prescribed appropriate driving scenario conditions, i.e., the system is activated when the vehicle satisfies the ODD condition, which replaces the human driver as the driving subject of the vehicle. Among them, the dynamic driving task DDT refers to continuous lateral (left and right steering) and longitudinal motion control (acceleration, deceleration, uniform) of the vehicle and detection and response of targets and events in the running environment of the vehicle. The design operation range ODD refers to a condition under which the automatic driving system can safely operate, and the set condition may include a geographical location, a road type, a speed range, weather, time, country and local traffic laws and regulations, and the like.
RAW (RAW) image files, i.e., RAW image files, contain data processed from an image sensor of a digital camera, scanner, or motion picture film scanner. This is so named because they have not been processed, printed or used for editing.
RGB images, also called true color images, use R, G, B components to identify the color of a pixel, R, G, B represents the 3 different base colors red, green, and blue, respectively, and any color can be synthesized from the 3 primary colors.
At present, the conventional traffic light shape recognition is generally carried out by adopting an RGB image acquired by image acquisition equipment. Referring to fig. 3, fig. 3 is a schematic flow chart of main steps of traffic light shape recognition according to RGB images in the prior art. As shown in fig. 3, in the prior art, a lamp cap is generally cut out from an RGB image according to a detection frame (box) coordinate, and then sent into a recognition model, and the shape recognition result is obtained by performing shape recognition on the cut lamp cap through the recognition model. The disadvantage of this method is that the RGB image contains lost information, especially at night, the RGB image loses image information of the low-luminance part, making it difficult for the recognition model to train and obtain a correct shape recognition result.
In this case, the prediction error is caused by the image information contained in the model input being incomplete. The present application provides a new traffic light shape recognition solution to the above-mentioned problems.
Referring to fig. 1, fig. 1 is a schematic flow chart of main steps of a traffic light shape recognition method according to an embodiment of the present application. As shown in fig. 1, the traffic light shape recognition method in the embodiment of the present application mainly includes the following steps S101 to S103.
Step S101: and acquiring an original image file of the acquired traffic light.
In this embodiment, an original image file of the collected traffic light may be obtained.
In one embodiment, the original image file of the traffic light, such as an on-board camera, can be acquired through the image acquisition device.
Step S102: based on the original image file, an identification image for image identification is acquired.
In this embodiment, since the original image file is acquired processing data, which is in an unprocessed file format, the original image file can be adjusted according to actual requirements, so as to obtain an identification image for image identification. Since the obtained identification image is generated based on the original image file, it contains more abundant data. Referring to fig. 2, fig. 2 is a schematic diagram illustrating comparison of an identification image obtained according to an embodiment of the present application and an RGB image of a directly collected traffic light. As shown in fig. 2, in a night scene, RGB images directly generated by an image capturing device are exposed too high after ISP processing, and it is difficult to form original shapes in brighter places. The identification image obtained according to the original image file retains richer information and is clearer than the RGB image.
Step S103: and carrying out image recognition based on the recognition image to obtain a shape recognition result of the traffic light.
In this embodiment, image recognition may be performed based on the recognition image, so as to obtain a shape recognition result of the traffic light. The shape recognition result of the traffic light can comprise an arrow, a cake and the like. Wherein the arrows may also include arrows of different directions.
In one embodiment, a neural network model can be constructed, and image recognition is performed based on the neural network model to obtain a shape recognition result of the traffic light.
In one embodiment, an original image file may be converted into RGB images with different brightness, and the RGB images with different brightness are used as identification images to perform image identification, so as to obtain a shape identification result of a traffic light of each RGB image and a confidence level of the shape identification result, and obtain a final shape identification result of the traffic light according to the confidence level. For example, the shape recognition result with the highest confidence is selected as the final shape recognition result.
Based on the above steps S101-S103, in the embodiment of the present application, according to the acquired original image file of the traffic light, an identification image for image identification is acquired according to the original image file, and image identification is performed according to the identification image, so as to obtain a shape identification result of the traffic light. Through the configuration mode, the original image file contains richer information, so that the identification image obtained based on the original image file also has richer information, and further, a more accurate traffic light shape identification result can be obtained according to the identification image, and the conditions that ISP processing exposure is too high and traffic light shapes are difficult to recognize due to too low night environment brightness can be effectively improved. Therefore, more accurate traffic light identification attributes such as round cakes, arrows, arrow directions and the like are obtained according to the traffic light shape identification result, and accurate driving prediction, planning and the like are performed based on the traffic light identification attributes.
Step S102 and step S103 are further described below, respectively.
In one implementation of the embodiment of the present application, step S102 may further include the following steps S1021 to S1023:
step S1021: data of four channels of an original image file is acquired.
In this embodiment, referring to fig. 5, fig. 5 is a schematic view (color not shown) of an original image file. As shown in fig. 5, since the image information is stored in the original image file in a manner of repeating and alternating with four channels of RGGB, in order to realize the visualization of the image information, the data of the four channels of the original image file may be first fetched. The data of the four channels can be fetched by the following code:
r = bayer[::2, ::2]
g1 = bayer[::2, 1::2]
g2 = bayer[1::2, ::2]
b = bayer[1::2, 1::2]
wherein, the bayer is the data ordering mode of the original image file, and r, g1, g2 and b are the data of the 4 channels which are taken out and correspond to red, green 1, green 2 and blue respectively.
Step S1022: based on the four channels of data, data combination is performed to obtain RGB image form data.
In the present embodiment, data of four channels may be data-combined to obtain RGB image form data. Wherein the RGB image form data is image data generated based on the original image file and visualized in RGB form.
Step S1023: and acquiring the identification image according to the RGB image form data.
In this embodiment, adjustment may be performed according to RGB image format data to obtain data meeting the actual application requirements as the identification image. The RGB image form data may be resized or shaped to obtain a recognition image, for example, based on the requirements of the input data of the image recognition model.
Specifically, in order to implement multiplexing of the related art recognition model (e.g., fig. 3) for traffic light shape recognition based on RGB images, the model input data may be adjusted to the same size and shape as the model input data of the recognition model in fig. 3. Assuming that the original image file is 16 bits and the model input data in fig. 3 is 8 bits, the obtained RGB image form data can be reduced to 8 bits. Meanwhile, since the channel data are spaced, the shape of the obtained RGB image form data becomes small, and it is necessary to adjust (restore) to the original size, thereby achieving alignment of the obtained recognition image with the model input data in fig. 3.
The identification image can be obtained specifically by:
green = (g1 + g2) × 0.5
rgb = np.concatenate(
(red[..., None], green[..., None], blue[..., None]), axis=2
)
# float32 47435.0 2384.0
rgb = (rgb / 256).clip(0, 255)
rgb = cv2.resize(rgb, (width, height))
firstly, combining g1 and g2 to obtain green data, and then combining r, g and b to obtain RGB image form data.
In one implementation of the embodiment of the present invention, step S103 may be further configured to:
based on the identification image, applying a trained image identification model to carry out image identification, and obtaining a shape identification result of the traffic light; the trained image recognition model is obtained through training based on a training data set, and the training data set contains annotation images generated based on an original image file of a traffic light used for training.
In this embodiment, a training data set may be generated by labeling according to an original image file of a traffic light for training, and the training data set is applied to train an image recognition model, so as to obtain a trained image recognition model.
In one embodiment, in order to realize more efficient and convenient traffic light shape recognition, an image recognition model for performing traffic light shape recognition based on RGB images can be multiplexed, and a training data set is applied to train the model. The training process can also multiplex the training process of the image recognition model for carrying out the shape recognition of the traffic lights based on the RGB images, and the specific training process is not specifically limited in the application.
In one embodiment, the training data set may be generated according to the following steps S201 to S203:
step S201: for each original image file used for training, the original image file is converted into a plurality of RGB images of different brightnesses to obtain a plurality of RGB images.
In the present embodiment, step S201 may further include the following steps S2011 and S2012:
step S2011: and adjusting the original image file according to a plurality of preset brightness parameters to obtain a plurality of adjusted original image files.
Step S2012: and performing color space conversion on the adjusted original image file to obtain a plurality of RGB images with different brightness.
In the present embodiment, since the original image file itself is not visualized, the original image file for training can be converted into an RGB image. It is considered that if the brightness of the RGB image is too bright during the conversion process, the traffic light shape is not seen clearly, and if the brightness is too dark, it may not be possible to determine whether the traffic light is present. Therefore, a plurality of different brightness parameters can be set, the original image file is adjusted according to the brightness parameters, and the adjusted original image file is subjected to color space conversion, so that a plurality of RGB images with different brightness are obtained. Specifically, the color space conversion may be performed by the following code:
cv2.cvtColor(raw, cv2.COLOR_BayerRG2BGR_EA)
in one embodiment, the luminance maximum may be set to ensure that the original image file is adjusted within a range of 0 to the luminance maximum.
Step S202: and splicing the RGB images to obtain a spliced image.
In this embodiment, reference may be made to fig. 6, and fig. 6 is a schematic diagram of a stitched image according to an embodiment of the present application. Fig. 6 shows an example of a stitched image, where 3 luminance parameters may be selected to obtain three RGB images with different luminance, and stitching is performed to obtain a stitched image.
Step S203: and marking the shape of the traffic light according to the spliced image to generate a training data set.
In this embodiment, on the stitched image, for each traffic light shape, an RGB image with the highest definition of the traffic light shape among the RGB images is selected, and the traffic light shape is marked. In other words, when the shape of the traffic light is marked, the plurality of RGB images on the spliced image are referred to, so that the shape of the traffic light is marked more accurately. If the traffic light shape in the RGB image with the highest brightness is not visible, the RGB image with other brightness can be referred to for marking.
In one embodiment, when labeling the spliced image, a labeling process and a labeling tool for labeling the RGB image in the prior art can be multiplexed.
In one implementation of the embodiment of the present invention, step S103 may further include the following steps S1031 to S1033:
step S1031: and carrying out traffic light target detection based on the identification image to obtain a target detection result.
Step S1032: according to the target detection result, obtaining a local image of the traffic light
Step S1033: and carrying out classification and identification according to the local images, and obtaining the shape identification result of the traffic lights.
In this embodiment, referring to fig. 4, fig. 4 is a schematic flow chart of main steps of a traffic light shape recognition method according to an embodiment of the present application. As shown in fig. 4, the preprocessing may be performed first based on the original image file to obtain an identification image, the target detection may be performed based on the identification image to obtain a target detection result, the local image of the traffic light may be cut out based on the detection frame (box) coordinates of the target detection result, and the shape recognition result of the traffic light may be obtained by performing classification recognition according to the local image.
In one embodiment, a residual network may be applied to achieve classification recognition of the partial images.
Referring to fig. 7 and 8, fig. 7 is a schematic view of a visualization of the obtained shape recognition result of the traffic light according to an embodiment of the present application; fig. 8 is a schematic view showing a visual recognition result of a traffic light obtained from an RGB image in the prior art. As shown in fig. 7, in a night scene, different shapes (a cake, an arrow and different directions of the arrow) can be clearly distinguished by using the shape recognition result obtained by the traffic light shape recognition method. As shown in fig. 8, the details cannot be recognized by the shape recognition result of the traffic light obtained directly based on the acquired RGB image under the same condition. The evaluation data shows that the accuracy of traffic light shape recognition based on the acquired RGB image is 72.7%, the accuracy of traffic light shape recognition based on the original image file is 93.6%, and the accuracy is improved by 30% compared with that based on the acquired RGB image when evaluating at night.
It should be noted that, although the foregoing embodiments describe the steps in a specific sequential order, it should be understood by those skilled in the art that, in order to achieve the effects of the present application, different steps need not be performed in such an order, and may be performed simultaneously (in parallel) or in other orders, and these variations are within the scope of protection of the present application.
It will be appreciated by those skilled in the art that the present application may implement all or part of the above-described methods according to the above-described embodiments, or may be implemented by means of a computer program for instructing relevant hardware, where the computer program may be stored in a computer readable storage medium, and the computer program may implement the steps of the above-described embodiments of the method when executed by a processor. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable storage medium may include: any entity or device, medium, usb disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier wave signals, telecommunications signals, software distribution media, and the like capable of carrying the computer program code. It should be noted that the computer readable storage medium may include content that is subject to appropriate increases and decreases as required by jurisdictions and by jurisdictions in which such computer readable storage medium does not include electrical carrier signals and telecommunications signals.
Further, the present application also provides a computer-readable storage medium. In one computer-readable storage medium embodiment according to the present application, the computer-readable storage medium may be configured to store a program for performing the traffic light shape recognition method of the above-described method embodiment, which may be loaded and executed by a processor to implement the traffic light shape recognition method described above. For convenience of explanation, only those portions relevant to the embodiments of the present application are shown, and specific technical details are not disclosed, refer to the method portions of the embodiments of the present application. The computer readable storage medium may be a storage device including various electronic devices, and optionally, in embodiments of the present application, the computer readable storage medium is a non-transitory computer readable storage medium.
Another aspect of the present application also provides a smart device that may include at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores a computer program which, when executed by the at least one processor, implements the method of any of the embodiments described above. The intelligent device can comprise driving equipment, intelligent vehicles, robots and the like. Referring to fig. 9, fig. 9 is a schematic diagram of a connection relationship between a processor and a memory according to an embodiment of the present application, and fig. 9 illustrates that the memory and the processor are connected by a bus communication.
In some embodiments of the present application, the smart device further comprises at least one sensor for sensing information. The sensor is communicatively coupled to any of the types of processors referred to herein. Optionally, the intelligent device further comprises an automatic driving system, and the automatic driving system is used for guiding the intelligent device to drive by itself or assist driving. The processor communicates with the sensors and/or the autopilot system for performing the method of any one of the embodiments described above.
Further, it should be understood that, since the respective modules are merely set to illustrate the functional units of the apparatus of the present application, the physical devices corresponding to the modules may be the processor itself, or a part of software in the processor, a part of hardware, or a part of a combination of software and hardware. Accordingly, the number of individual modules in the figures is merely illustrative.
Those skilled in the art will appreciate that the various modules in the apparatus may be adaptively split or combined. Such splitting or combining of specific modules does not lead to a deviation of the technical solution from the principles of the present application, and therefore, the technical solution after splitting or combining will fall within the protection scope of the present application.
The personal information of the relevant user possibly related in each embodiment of the application is personal information which is strictly required by laws and regulations, is processed actively provided by the user in the process of using the product/service or is generated by using the product/service and is obtained by authorization of the user according to legal, legal and necessary principles and based on reasonable purposes of business scenes.
The personal information of the user processed by the application may be different according to the specific product/service scene, and the specific scene that the user uses the product/service may be referred to as account information, equipment information, driving information, vehicle information or other related information of the user. The applicant would treat the user's personal information and its processing with a high diligence.
The method and the device have the advantages that safety of personal information of the user is very important, and safety protection measures which meet industry standards and are reasonable and feasible are adopted to protect the information of the user and prevent the personal information from unauthorized access, disclosure, use, modification, damage or loss.
Thus far, the technical solution of the present application has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present application is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present application, and such modifications and substitutions will be within the scope of the present application.
Claims (10)
1. A traffic light shape recognition method, the method comprising:
acquiring an original image file of an acquired traffic light;
acquiring an identification image for image identification based on the original image file;
and carrying out image recognition based on the recognition image to obtain a shape recognition result of the traffic light.
2. The traffic light shape recognition method according to claim 1, wherein,
the step of carrying out image recognition based on the recognition image to obtain the shape recognition result of the traffic light comprises the following steps:
based on the identification image, applying a trained image identification model to carry out image identification, and obtaining a shape identification result of the traffic light;
the trained image recognition model is obtained through training based on a training data set, and the training data set contains annotation images generated based on an original image file of a traffic light used for training.
3. The traffic light shape recognition method according to claim 2, wherein,
the method further comprises generating the training data set according to the steps of:
converting the original image file into a plurality of RGB images with different brightness aiming at each original image file for training so as to obtain a plurality of RGB images;
splicing the RGB images to obtain a spliced image;
and marking the shape of the traffic light according to the spliced image so as to generate the training data set.
4. The traffic light shape recognition method according to claim 3, wherein,
the marking the shape of the traffic light according to the spliced image comprises the following steps:
and selecting an RGB image with the highest definition of the traffic light shape from a plurality of RGB images according to the shape of each traffic light on the spliced image, and marking the shape of the traffic light.
5. The traffic light shape recognition method according to claim 3, wherein,
the converting the original image file into a plurality of RGB images with different brightness includes:
according to a plurality of preset brightness parameters, the original image file is adjusted to obtain a plurality of adjusted original image files;
and performing color space conversion on the adjusted original image file to obtain a plurality of RGB images with different brightness.
6. The traffic light shape recognition method according to claim 1, wherein,
the acquiring the identification image for image identification based on the original image file comprises the following steps:
acquiring data of four channels of the original image file;
according to the data of the four channels, data combination is carried out to obtain RGB image form data;
and acquiring the identification image according to the RGB image form data.
7. The traffic light shape recognition method according to claim 6, wherein,
the obtaining the identification image according to the RGB image form data includes:
the RGB image form data is resized to obtain the identification image.
8. The traffic light shape recognition method according to claim 1, wherein,
the step of carrying out image recognition based on the recognition image to obtain the shape recognition result of the traffic light comprises the following steps:
based on the identification image, carrying out traffic light target detection to obtain a target detection result;
acquiring a local image of the traffic light according to the target detection result;
and carrying out classification and identification according to the local images, and obtaining the shape identification result of the traffic lights.
9. A computer readable storage medium having stored therein a plurality of program codes, wherein the program codes are adapted to be loaded and executed by a processor to perform the traffic light shape recognition method of any one of claims 1 to 8.
10. An intelligent device, the intelligent device comprising:
at least one processor;
and a memory communicatively coupled to the at least one processor;
wherein the memory has stored therein a computer program which, when executed by the at least one processor, implements the traffic light shape recognition method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311817951.3A CN117496486B (en) | 2023-12-27 | 2023-12-27 | Traffic light shape recognition method, readable storage medium and intelligent device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311817951.3A CN117496486B (en) | 2023-12-27 | 2023-12-27 | Traffic light shape recognition method, readable storage medium and intelligent device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117496486A true CN117496486A (en) | 2024-02-02 |
| CN117496486B CN117496486B (en) | 2024-03-26 |
Family
ID=89680326
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311817951.3A Active CN117496486B (en) | 2023-12-27 | 2023-12-27 | Traffic light shape recognition method, readable storage medium and intelligent device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117496486B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130110386A1 (en) * | 2011-10-28 | 2013-05-02 | Thinkware Systems Corporation | Method and apparatus for providing analysis index of roadway section based on road and traffic conditions |
| CN112150364A (en) * | 2020-10-10 | 2020-12-29 | 厦门理工学院 | Pairing and splicing method for split type candidate image areas of arrow-shaped traffic signal lamp |
| CN112861748A (en) * | 2021-02-22 | 2021-05-28 | 奥特酷智能科技(南京)有限公司 | Traffic light detection system and method in automatic driving |
| WO2021159279A1 (en) * | 2020-02-11 | 2021-08-19 | 深圳元戎启行科技有限公司 | Exposure control method and apparatus, and readable storage medium and computer device |
| CN115116035A (en) * | 2022-06-22 | 2022-09-27 | 南京佑驾科技有限公司 | Road traffic light identification system and method based on neural network |
| CN115497076A (en) * | 2022-10-09 | 2022-12-20 | 江苏智能无人装备产业创新中心有限公司 | High-precision and high-efficiency signal identification detection method, device and medium |
-
2023
- 2023-12-27 CN CN202311817951.3A patent/CN117496486B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130110386A1 (en) * | 2011-10-28 | 2013-05-02 | Thinkware Systems Corporation | Method and apparatus for providing analysis index of roadway section based on road and traffic conditions |
| WO2021159279A1 (en) * | 2020-02-11 | 2021-08-19 | 深圳元戎启行科技有限公司 | Exposure control method and apparatus, and readable storage medium and computer device |
| CN112150364A (en) * | 2020-10-10 | 2020-12-29 | 厦门理工学院 | Pairing and splicing method for split type candidate image areas of arrow-shaped traffic signal lamp |
| CN112861748A (en) * | 2021-02-22 | 2021-05-28 | 奥特酷智能科技(南京)有限公司 | Traffic light detection system and method in automatic driving |
| CN115116035A (en) * | 2022-06-22 | 2022-09-27 | 南京佑驾科技有限公司 | Road traffic light identification system and method based on neural network |
| CN115497076A (en) * | 2022-10-09 | 2022-12-20 | 江苏智能无人装备产业创新中心有限公司 | High-precision and high-efficiency signal identification detection method, device and medium |
Non-Patent Citations (1)
| Title |
|---|
| 楚婉艺等: "基于视觉的红绿灯识别", 《信息技术与信息化》, no. 5, 31 May 2021 (2021-05-31), pages 66 - 68 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117496486B (en) | 2024-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110009913B (en) | Intelligent checking method and system for off-site law enforcement picture of vehicle running red light | |
| US11195258B2 (en) | Device and method for automatic image enhancement in vehicles | |
| CN106469309B (en) | The method and apparatus of vehicle monitoring, processor, image capture device | |
| CN109886210A (en) | A kind of traffic image recognition methods, device, computer equipment and medium | |
| DE102020104363A1 (en) | DETECTION OF AN ABNORMAL IMAGE IN CONNECTION WITH IMAGE DATA FROM ONE OR MORE CAMERAS OF A COMPUTER-AIDED OR AUTONOMOUS VEHICLE | |
| CN110532903B (en) | Traffic light image processing method and equipment | |
| CN108965726B (en) | Vehicular camera system | |
| CN110390314B (en) | Visual perception method and equipment | |
| US20190197328A1 (en) | Method and apparatus for outputting information | |
| US20200210765A1 (en) | Semantic segmentation using driver attention information | |
| US11416707B2 (en) | Information processing method, information processing system, and information processing apparatus | |
| CN113792600B (en) | Video frame extraction method and system based on deep learning | |
| CN117495847B (en) | Intersection detection method, readable storage medium and intelligent device | |
| CN117496486B (en) | Traffic light shape recognition method, readable storage medium and intelligent device | |
| CN112784675B (en) | Target detection method and device, storage medium and terminal | |
| CN113126252B (en) | Low-light-level imaging system | |
| CN110378184A (en) | A kind of image processing method applied to automatic driving vehicle, device | |
| US20180262678A1 (en) | Vehicle camera system | |
| US11823305B2 (en) | Method and device for masking objects contained in an image | |
| US20150055858A1 (en) | Systems and methods for color recognition in computer vision systems | |
| KR102220643B1 (en) | CCTV for recognizing car number based on deep learning | |
| US10735660B2 (en) | Method and device for object identification | |
| CN115439509B (en) | Multi-target tracking method and device, computer equipment and storage medium | |
| CN111402610A (en) | Method, device, equipment and storage medium for identifying lighting state of traffic light | |
| CN111724349B (en) | Image smudge recognition method based on HSV and SVM |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |