TWI832270B - Method for detecting road condition, electronic device and storage medium - Google Patents

Abstract

The present application provides a method for detecting road condition, an electronic device and a storage medium. The method includes: acquiring images in front of the vehicle as detection images; inputting the detection images into a trained semantic segmentation model, which includes a backbone network and a head network; and extracting features from the detection images by using the backbone network and obtaining a plurality of feature maps. The plurality of feature maps are input into the head network, and a first segmentation network of the head network processes the plurality of feature maps to obtain a first recognition result, and a second segmentation network of the head network performs the plurality of feature images and obtaining a second identification result. The vehicle can be driven based on the first identification result and the second identification result. The application can improve the safety of autonomous driving.

Description

Road condition detection method, electronic device and computer-readable storage medium

The present application relates to the field of computer vision technology, and in particular to a road condition detection method, electronic equipment and computer-readable storage media.

In autonomous driving of vehicles, environmental perception is an extremely important technology. Currently, most environmental awareness functions are achieved using semantic segmentation methods based on deep learning. Semantic segmentation methods use deep learning segmentation models to identify objects in images. However, the above method can only recognize pre-defined object categories, such as identifying pre-defined categories of roads, pedestrians, cars, etc. However, the scene on the road is extremely complex in actual application. If an unknown object category appears on the road scene, the trained Models often recognize errors or fail to recognize them, causing the vehicle to directly hit objects on the road and cause traffic accidents.

In view of the above, it is necessary to provide road condition detection methods, electronic devices and computer-readable storage media to solve the problem of the applied model being unable to identify unknown objects and causing the vehicle to be unable to make corresponding judgments, thereby avoiding the occurrence of traffic accidents.

Embodiments of the present application provide a road condition detection method. The road condition detection method includes: obtaining an image in front of the vehicle as a detection image; inputting the detection image into a trained semantic segmentation model, where the semantic segmentation model includes a backbone network Road and head network; use the backbone network to perform feature extraction on the detection image to obtain multiple feature maps; input the multiple feature maps The head network, the first segmentation network of the head network processes the plurality of feature maps to obtain a first recognition result, and the second segmentation network of the head network processes the plurality of feature maps. The characteristic image is processed to obtain a second recognition result; based on the first recognition result and the second recognition result, it is determined whether the vehicle can continue driving.

In an optional implementation, the method further includes: constructing a semantic segmentation model and completing training of the semantic segmentation model, including: obtaining training images; inputting the training images into the backbone network Perform feature extraction to obtain multiple training feature maps; input the multiple training feature maps into the head network, and the first segmentation network processes the multiple training feature maps to obtain the training image the first training result; according to the first training result and the preset first expected result, use the preset loss function to calculate the first loss value of the first segmentation network; the second segmentation network pairs The plurality of training feature maps are processed to obtain a second training result of the training image; according to the second training result and the preset second expected result, the preset loss function is used to calculate the third The second loss value of the binary segmentation network; adjust the parameters of the semantic segmentation model according to the first loss value and the second loss value to obtain the trained semantic segmentation model.

In an optional implementation, the method of obtaining the first training result and the second training result includes: using the first segmentation network to upsample and deconvolve the multiple training feature maps. Process to obtain a plurality of first training feature maps with the same size as the training image, use the first softmax layer to classify each first training feature map according to the first preset pixel category, and obtain the first training feature Probability of each pixel classification in the picture, select the category corresponding to the maximum probability value as the category corresponding to the pixel, and output the first training result, wherein the first preset pixel category includes multiple predefined object categories ; Use the second segmentation network to perform upsampling and deconvolution processing on the plurality of training feature maps to obtain a plurality of second training feature maps with the same size as the detection image, and use the second softmax layer to Each training feature map is classified according to the second preset pixel category, the probability of each pixel classification in the second training feature map is obtained, and the category corresponding to the maximum probability value is selected as the Category corresponding to the pixel, and output the second training result, wherein the second preset pixel category includes two predefined road categories: lane or non-lane.

In an optional implementation, adjusting the parameters of the semantic segmentation model according to the first loss value and the second loss value to obtain the trained semantic segmentation model includes: Add a loss value to the second loss value to obtain the loss value of the semantic segmentation model; use the gradient descent method to adjust the parameters of the semantic segmentation model to minimize the loss value of the semantic segmentation model, and obtain the training Completed semantic segmentation model.

In an optional implementation, the method further includes: using the encoding network of the segnet network as the backbone network; using the decoding network of the segnet network as the first segmentation network in the head network. road; and add a decoding network of the segnet network as the second segmented network in the head network.

In an optional implementation, the first segmentation network of the head network processes the multiple feature maps, and obtaining the first recognition result includes: inputting the detection image to the backbone network Convolution operations and maximum pooling operations are performed to obtain multiple feature maps of the detection image; the first segmentation network performs upsampling and deconvolution processing on the multiple feature maps to obtain multiple features corresponding to the The first feature map of the same size in the detection image is used; the first softmax layer is used to classify each of the first feature maps according to the first preset pixel category, and the pixel to which each pixel in the detection image belongs is output. Category information: determine the categories of all objects in the detection image according to the category information to which each pixel belongs, and use the categories of all objects in the detection image as the first recognition result.

In an optional implementation, the second segmentation network of the head network processes the plurality of characteristic images, and obtaining the second recognition result includes: the second segmentation network processes the plurality of feature images. The feature map is subjected to upsampling and deconvolution processing to obtain multiple second feature maps with the same size as the detection image; the second softmax layer is used to perform processing on each second feature map according to the second preset pixel category. Classify and determine the road category corresponding to the detected image as the second recognition result.

In an optional implementation, determining whether the vehicle can continue driving based on the first recognition result and the second recognition result includes: if the first recognition result indicates that the detection pattern has been recognized All object categories in the image, determine whether the vehicle can continue driving based on the categories of all objects in the first recognition result; or if the first recognition result indicates that there are unrecognizable objects in the detected image and The second recognition result indicates that the road category is a lane, and it is determined that the vehicle can continue driving; or if the first recognition result indicates that there are unrecognizable objects in the detected image and the second recognition result indicates that the road category It is a non-lane, and it is determined that the vehicle cannot continue driving.

An embodiment of the present application also provides an electronic device. The electronic device includes a processor and a memory. The processor is configured to execute a computer program stored in the memory to implement the road condition detection method.

Embodiments of the present application also provide a computer-readable storage medium that stores at least one instruction. When the at least one instruction is executed by a processor, the road condition detection method is implemented.

The above technical solution provided by the embodiment of the present application can redefine and retrain the object category in front of the vehicle when the preset model cannot recognize the object category in front of the vehicle, thereby ensuring that the self-driving vehicle can recognize it again, by identifying it twice. The combined method of assistance makes the recognition results more accurate, thus improving the safety of autonomous driving.

5: Electronic equipment

501:Memory

502: Processor

503: Computer program

504: Communication bus

101-106: Steps

Figure 1 is a flow chart of a road condition detection method provided by an embodiment of the present application.

Figure 2 is a structural diagram of a semantic segmentation model provided by an embodiment of the present application.

Figure 3 is a schematic diagram of the first recognition result provided by the embodiment of the present application.

Figure 4 is a schematic diagram of the second recognition result provided by the embodiment of the present application.

FIG. 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

In order to more clearly understand the above objects, features and advantages of the present application, the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

In order to more clearly understand the above objects, features and advantages of the present application, the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

Many specific details are set forth in the following description to facilitate a full understanding of the present application. The described embodiments are only some, rather than all, of the embodiments of the present application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this application.

Hereinafter, the terms “first” and “second” are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of some embodiments of the present application, words such as "exemplary" or "such as" are used to represent examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "such as" in some embodiments of the application is not intended to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the words "exemplary" or "such as" is intended to present the concept in a concrete manner.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the application is for the purpose of describing specific embodiments only and is not intended to limit the application.

Refer to Figure 1, which is a flow chart of a road condition detection method provided by an embodiment of the present application. The method is applied to electronic devices (for example, the electronic device 5 shown in Figure 5). The electronic device can be any electronic product that can interact with the user, such as a personal computer, a tablet computer, a smart phone. , Personal Digital Assistant (PDA), game consoles, interactive Internet Protocol Television (IPTV), smart wearable devices, etc.

The electronic device is a device that can automatically perform numerical calculations and/or information processing according to preset or stored instructions. Its hardware includes, but is not limited to: microprocessors, Application Specific Integrated Circuits (ASICs) ), Field-Programmable Gate Array (FPGA), Digital Signal Processor (DSP), embedded devices, etc.

The electronic equipment may also include network equipment and/or user equipment. The network equipment includes, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing.

The network where the electronic device is located includes, but is not limited to: the Internet, wide area network, metropolitan area network, regional network, virtual private network (Virtual Private Network, VPN), etc.

The method specifically includes the following steps.

101. Obtain the front image of the vehicle as the detection image.

In at least one embodiment of the present application, a camera installed inside or outside the vehicle is used to capture an area in front of the vehicle (for example, a field of view area), and the captured image is used as a detection image.

In other embodiments, OpenCV technology can also be used to obtain images from the driving recorder's video as the detection image. The method of obtaining detection images in this application is not specifically limited.

102. Construct a semantic segmentation model and complete training of the semantic segmentation model.

In at least one embodiment of the present application, the structure of the semantic segmentation model is shown with reference to Figure 2. The semantic segmentation model includes a backbone network and a head network, wherein the head network includes a first segmentation network and a head network. Second segmentation network.

In at least one embodiment of the present application, completing the training of the semantic segmentation model includes: obtaining training images; inputting the training images into the backbone network for feature extraction to obtain multiple training features Figure: Input the multiple training feature maps into the head network, and the first segmentation network processes the multiple training feature maps to obtain the first training result of the training image; according to the The first training result and the preset first expected result are used to calculate the first loss value of the first segmentation network using the preset loss function; the second segmentation network processes the plurality of training feature maps. , obtain the second training result of the training image; according to the second training result and the preset second expected result, use the preset loss function to calculate the second loss value of the second segmentation network ; According to the first loss value and the second loss value, adjust the parameters of the semantic segmentation model to obtain the trained semantic segmentation model.

In at least one embodiment of the present application, the obtaining training images includes: using images in the PASCAL Visual Object Classes (VOC) data set as training images, or using images in the Cityscapes data set as training images, or The semantic segmentation model is trained using self-photographed traffic images as training images. This application does not specifically limit this. For example, images of various road scenes can be used as training images. The training images include different objects as detection objects, such as vehicles, pedestrians, trees, roadblocks, etc.

In at least one embodiment of the present application, if the training image uses a self-photographed road condition image as the training image, the method includes: performing data enhancement processing on the self-photographed road condition image to increase the training image. Image, wherein the data enhancement processing includes flipping, rotating, scaling, and shifting the training sample image. In this embodiment, data enhancement operations are performed on self-photographed traffic images, and training images can be added to improve the robustness of the semantic segmentation model.

In at least one embodiment of the present application, inputting the training image into the backbone network for feature extraction, and obtaining multiple training feature maps includes: using the encoding network of the segnet network as the backbone of the semantic segmentation model. network; the coding network of the segnet network includes a convolution layer, a batch normalization (Batch Normalization, BN) layer, a ReLU activation layer and a maximum pooling layer (max-Pooling); input the training image to the The convolution layer performs a convolution operation to extract the feature values of the training image. The BN layer normalizes the feature values to calculate the current learning rate, and after processing by the ReLU activation layer and the max-Pooling layer, Output multiple training feature maps.

In at least one embodiment of the present application, the plurality of training feature maps are input into the head network, and the first segmentation network processes the plurality of training feature maps to obtain the training map. The first training result of the image includes: adopting the decoding network of the segnet network as the first segmentation network in the head network of the semantic segmentation model; wherein the decoding layer network of the segnet network includes an upsampling layer , convolutional layer and the first softmax layer. In this embodiment, the multiple training feature maps are input to the upsampling layer to perform an upsampling operation, the training feature maps are enlarged to the same size as the training image, and then the upsampled training feature maps are input The convolution layer performs a convolution operation to obtain the first training feature map after the operation, and inputs the first training feature map into the first softmax layer for classification according to the first preset pixel category to obtain each of the training images. Probability A _ik of pixel classification, where A _ik represents the probability that the i- th pixel in the training image is the k -th category, select the category corresponding to the maximum probability value as the category corresponding to the pixel, and output the training image The category information to which each pixel in the image belongs is used as the first training result, and the categories of all objects in the training image are determined based on the category information to which each pixel belongs.

In this embodiment, the semantic segmentation model is trained based on training images and corresponding pixel category annotations, and multiple pixel categories can be predetermined. For example, the first preset pixel category predicted by the first softmax layer includes 19 predefined object categories, including vehicles, pedestrians, trees, roadblocks, street lights, buildings, etc. For example, the categories of pixel classification include vehicles ( k =0), pedestrians ( k =1), trees ( k =2), roadblocks ( k =3), street lights ( k =4), buildings ( k =5), After the first softmax layer is classified according to the first preset pixel category, the probability values of the i-th pixel are: A _{i 0} =0.94, A _{i 1} =0.23, A _{i 2} =0.13, A _{i 3} =0.03, A _{i 4} =0.02, A _{i 5} =0.01, and the maximum probability value is 0.94. Since the corresponding k =0, it can be confirmed that the object category is a vehicle. Therefore, in this example, by calculating and comparing the probability of classification of the i - th pixel, it can be determined that the i- th pixel is a vehicle.

In at least one embodiment of the present application, calculating the first loss value of the first segmentation network using a preset loss function based on the first training result and a preset first expected result includes: The loss function is:

代表第一訓練結果。 Among them, LOSS represents the first loss, y represents the preset first expected result,

Represents the first training result.

In at least one embodiment of the present application, the second segmentation network processes the plurality of training feature maps, and obtaining the second training result of the training image includes: adding a decoding network of a segnet network. path as the second segmentation network in the head network of the semantic segmentation model, wherein the decoding layer network of the newly added segnet network includes an upsampling layer, a convolution layer and a second softmax layer; Multiple training feature maps are input to the upsampling layer to perform an upsampling operation, the training feature map is enlarged to the same size as the training image, and then the upsampled training feature map is input to the convolution layer to perform a convolution operation, Obtain the second training feature map, and finally input the second training feature map into the second softmax layer to classify according to the second preset pixel category, and obtain the probability A _bq of each pixel classification in the training image, where, the A _bq represents the probability that the b- th pixel in the training image is of the b- th category, select the category corresponding to the maximum probability value as the category corresponding to the pixel, and determine the road category corresponding to the training image as the second training result.

In this embodiment, the second preset pixel category includes two predefined road categories: lane or non-lane. For example, the second softmax layer predicts two object categories, lane or non-lane. For example, the categories of pixel classification include lanes ( q =10) and non-lanes ( q =15). After the second softmax layer is classified according to the second preset pixel category, the probability values of the b -th pixel are: A _{b 10} =0.86, A _{b 15} =0.33, and the maximum probability value is 0.86. Since it corresponds to q =10, the category can be determined to be a lane. Therefore, in this example, by calculating and comparing the probability of the classification of the b- th pixel, it can be obtained that the road category of the b -th pixel is a lane. In this embodiment, if the object in the training image is recognized as a lane category, it indicates that the object is not an obstacle; if the object in the training image is recognized as a non-lane category, it indicates that the object is an obstacle.

In at least one embodiment of the present application, the second loss value and the second loss value of the second segmentation network are calculated using the preset loss function based on the second training result and the preset second expected result. The method of calculating the first loss value of the first segmentation network using the preset loss function is similar and will not be described again in this application.

In at least one embodiment of the present application, adjusting the parameters of the semantic segmentation model according to the first loss value and the second loss value to obtain the trained semantic segmentation model includes: Add the first loss value and the second loss value to obtain the loss value of the semantic segmentation model; use the gradient descent method to adjust the parameters of the semantic segmentation model to minimize the loss value of the semantic segmentation model, Obtain the trained semantic segmentation model. In this embodiment, the gradient descent algorithm used includes stochastic gradient descent method (Stochastic Gradient Descent) or mini-batch gradient descent method (Mini-batch Gradient Descent). This application does not specifically limit this. In this embodiment, adjusting the parameters of the semantic segmentation model includes adjusting the learning rate of the semantic segmentation model or the number of iterative training times of the training images.

103. Input the detection image into the backbone network in the trained semantic segmentation model for feature extraction to obtain multiple feature maps.

In at least one embodiment of the present application, the detection image is input to the convolution layer in the backbone network to perform a convolution operation to extract the feature values of the detection image, and the feature values are standardized and calculated through the BN layer. The current learning rate is processed by the ReLU activation layer and the max-Pooling layer, and multiple feature maps are output.

104. Input the plurality of feature maps into the head network, and the first segmentation network of the head network processes the plurality of feature maps and outputs a first recognition result.

In at least one embodiment of the present application, the plurality of feature maps are input into the head network, and the first segmentation network of the head network processes the plurality of feature maps and outputs a first recognition result. It includes: inputting the detection image to the backbone network to perform convolution operation and maximum pooling operation to obtain multiple feature maps of the detection image; the first segmentation network performs a convolution operation on the multiple features The image is subjected to upsampling and deconvolution processing to obtain multiple feature maps with the same size as the detection image; The first softmax layer is used to classify the plurality of feature maps according to a first preset pixel category, and the category information to which each pixel in the detection image belongs is output; and the category information to which each pixel belongs is determined according to the category information. Detect the categories of all objects in the image, and use the categories of all objects in the detected image as the first recognition result. For example, as shown in Figure 3, Figure 3 is a schematic diagram of the first recognition result provided by an embodiment of the present application. The figure shows the recognition result of the detection image obtained through the first segmentation network. The object categories in the detection image are classified by pixels to obtain the object categories on the detection image.

105. The second segmentation network of the head network processes the multiple feature maps and outputs a second recognition result.

In at least one embodiment of the present application, processing the plurality of feature maps by the second segmentation network of the head network, and outputting the second recognition result includes: the second segmentation network processes the plurality of feature maps. Perform upsampling and deconvolution processing on the feature maps to obtain multiple feature maps with the same size as the detection image; use the second softmax layer to classify the multiple feature maps according to the second preset pixel category , determine the road category corresponding to the detection image as the second recognition result, wherein the road category is lane or non-lane. For example, as shown in Figure 4, Figure 4 is a schematic diagram of the second recognition result provided by an embodiment of the present application. The figure shows the recognition result of the detection image obtained through the second segmentation network. The object categories in the detection image are classified by pixels to determine the road category corresponding to the detection image. In this embodiment, the lane is regarded as a non-obstacle, and the non-lane is regarded as an obstacle.

The above process of obtaining the first recognition result can refer to the above process of obtaining the first training result. Similarly, the process of obtaining the second recognition result can refer to the above process of obtaining the second training result, which will not be repeated here. Repeat instructions.

It should be noted that the first segmentation network and the second segmentation network process the received feature maps at the same time. When the first segmentation network obtains the first recognition result, the recognition The identified category is judged to determine the next operation of the vehicle. When the first recognition result shows that there is an unrecognizable category, the second recognition result is called, and the next operation of the vehicle is determined based on the second recognition result.

106. Determine whether the vehicle can continue driving based on the first recognition result and the second recognition result.

In at least one embodiment of the present application, determining whether the vehicle can continue driving based on the first recognition result and the second recognition result includes: if the first recognition result indicates that the vehicle has been recognized Detect all object categories in the image, and determine whether the vehicle can continue driving based on the categories of all objects in the first recognition result; or if the first recognition result indicates that there are unrecognizable objects in the detected image object and the second recognition result indicates that the road type is a lane, it is deemed that there is no obstacle in front of the vehicle, and it is determined that the vehicle can continue driving; or if the first recognition result indicates that there is an unrecognizable object in the detection image object and the second recognition result indicates that the road category is non-lane, it is considered that there is an obstacle in front of the vehicle, and it is determined that the vehicle cannot continue driving.

In at least one embodiment of the present application, when the first recognition result is obtained through the first segmentation network, if the first recognition result cannot identify the object type, the second recognition result is called to determine whether the vehicle can continue driving. For example, if there is a stroller in front of the vehicle, and the category of the stroller is not included in the training of the first segmentation network, the first segmentation network cannot recognize the stroller in front of the vehicle, and the second recognition result is called. The second recognition result indicates that the road category is non-lane, which indicates that there is an obstacle in front of the vehicle, and it is determined that the vehicle cannot continue driving.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. For those of ordinary skill in the art, without departing from the creative concept of the present application, they can also make Improvements, but these all fall within the protection scope of this application.

As shown in Figure 5, Figure 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device 5 includes a memory 501, at least one processor 502, a computer program 503 stored in the memory 501 and executable on the at least one processor 502, and at least one communication bus 504.

Those skilled in the art can understand that the schematic diagram shown in FIG. 5 is only an example of the electronic device 5 and does not constitute a limitation of the electronic device 5. It may include more or less components than those shown in the figure, or a combination thereof. Certain components, or different components, for example, the electronic device 5 may also include input and output devices, network access devices, etc.

The at least one processor 502 may be a Central Processing Unit (CPU), or other general-purpose processor, a Digital Signal Processor (DSP), or an Application Specific Integrated Circuit (ASIC). ), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The at least one processor 502 may be a microprocessor or the at least one processor 502 may also be any conventional processor, etc. The at least one processor 502 is the control center of the electronic device 5 and utilizes various interfaces and circuits. Connect various parts of the entire electronic device 5.

The memory 501 can be used to store the computer program 503. The at least one processor 502 runs or executes the computer program 503 stored in the memory 501 and calls the data stored in the memory 501, Various functions of the electronic device 5 are realized. The memory 501 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area The area may store data created according to the use of the electronic device 5 (such as audio data) and the like. In addition, the memory 501 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart memory card (Smart Media Card, SMC), Secure Digital (Secure Digital, SD) card, flash memory card (Flash Card), at least one disk memory device, flash memory device, or other non-volatile solid-state memory device.

If the integrated modules/units of the electronic device 5 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the above embodiment methods by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by the processor, the computer program can implement the steps of each of the above method embodiments. Wherein, the computer program includes computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable media may include: any entity or device capable of carrying the computer program code, a recording medium, a flash drive, a mobile hard drive, a magnetic disk, an optical disk, computer memory, and read-only memory (ROM, Read-Only Memory). Memory).

It is obvious to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as illustrative and non-restrictive from any point of view, and the scope of the present application is defined by the appended claims rather than the above description, and it is therefore intended that those falling within the claims All changes within the meaning and scope of the equivalent elements are included in this application. Any associated association markup in a request item should not be considered to limit the request item in question.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present application can be modified. Modifications or equivalent substitutions may be made without departing from the spirit and scope of the technical solution of the present application.

101-106: Steps

Claims (9)

A road condition detection method, wherein the road condition detection method includes: obtaining an image in front of the vehicle as a detection image; inputting the detection image into a trained semantic segmentation model, the semantic segmentation model including a backbone network and a head Network; use the backbone network to perform feature extraction on the detection image to obtain multiple feature maps; use the decoding network of the segnet network as the first segmentation network in the head network; and add a new The decoding network of a segnet network serves as the second segmentation network in the head network; the plurality of feature maps are input to the head network, and the first segmentation network of the head network is responsible for the Multiple feature images are processed to obtain a first recognition result, and the second segmentation network of the head network processes the multiple feature images to obtain a second recognition result; based on the first recognition result and The second recognition result determines whether the vehicle can continue driving, including: if the first recognition result indicates that all object categories in the detection image have been recognized, based on all objects in the first recognition result determine whether the vehicle can continue driving; or if the first recognition result indicates that there is an unrecognizable object in the detection image and the second recognition result indicates that the road category is a lane, determine whether the vehicle can continue driving Driving; or if the first recognition result shows that there is an unrecognizable object in the detected image and the second recognition result shows that the road type is non-lane, it is determined that the vehicle cannot continue to drive. The road condition detection method according to claim 1, wherein the method further includes: constructing a semantic segmentation model and completing training of the semantic segmentation model, including: obtaining training images; inputting the training images to the The backbone network is used for feature extraction to obtain multiple training feature maps; The plurality of training feature maps are input into the head network, and the first segmentation network processes the plurality of training feature maps to obtain a first training result of the training image; according to the first The training results and the preset first expected result are calculated using the preset loss function to calculate the first loss value of the first segmentation network; the second segmentation network processes the multiple training feature maps to obtain The second training result of the training image; according to the second training result and the preset second expected result, use the preset loss function to calculate the second loss value of the second segmentation network; according to The first loss value and the second loss value are used to adjust the parameters of the semantic segmentation model to obtain the trained semantic segmentation model. The road condition detection method according to claim 2, wherein the method of obtaining the first training result and the second training result includes: using the first segmentation network to upsample the plurality of training feature maps. and deconvolution processing to obtain a plurality of first training feature maps with the same size as the training image, and use the first softmax layer to classify each first training feature map according to the first preset primitive category to obtain the The probability of classifying each graphic element in the training image is selected, the category corresponding to the maximum probability value is selected as the category corresponding to the graphic element, and the first training result is output, wherein the first preset graphic element category includes a preset Multiple defined object categories; use the second segmentation network to perform upsampling and deconvolution processing on the multiple training feature maps to obtain multiple second training feature maps that are consistent in size with the detection image, Use the second softmax layer to classify each training feature map according to the second preset primitive category, obtain the probability of each primitive classification in the training image, and select the category corresponding to the maximum probability value as the corresponding primitive category, and output the second training result, wherein the second preset primitive category includes two predefined road categories: lane or non-lane. The road condition detection method as described in claim 2, wherein adjusting the parameters of the semantic segmentation model according to the first loss value and the second loss value to obtain the trained semantic segmentation model includes: Add the first loss value and the second loss value to obtain the loss value of the semantic segmentation model; use the gradient descent method to adjust the parameters of the semantic segmentation model to minimize the loss value of the semantic segmentation model, The trained semantic segmentation model is obtained. The road condition detection method as described in any one of claims 1 to 4, wherein the method further includes: using a coding network of a segnet network as the backbone network. The road condition detection method according to claim 5, wherein the first segmentation network of the head network processes the plurality of feature maps, and obtaining the first recognition result includes: inputting the detection image to the The backbone network performs convolution operations and maximum pooling operations to obtain multiple feature maps of the detection image; the first segmentation network performs upsampling and deconvolution processing on the multiple feature maps to obtain A plurality of first feature maps with the same size as the detection image; use the first softmax layer to classify each first feature map according to the first preset element category, and output the first feature map in the detection image Category information to which each graphic element belongs; determine the categories of all objects in the detected image based on the category information to which each graphic element belongs, and use the categories of all objects in the detected image as the first recognition result. The road condition detection method as described in claim 5, wherein the second segmentation network of the head network processes the plurality of feature images, and the second recognition result obtained includes: the second segmentation network pairs The plurality of feature maps are subjected to upsampling and deconvolution processing to obtain a plurality of second feature maps that are consistent in size with the detection image; The second softmax layer is used to classify each second feature map according to the second preset element category, and the road category corresponding to the detection image is determined as the second recognition result. An electronic device, wherein the electronic device includes a processor and a memory, and the processor is used to execute a computer program stored in the memory to implement the road condition detection method as described in any one of claims 1 to 7. A computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the road condition detection method as described in any one of claims 1 to 7 is implemented.

Images