CN117315624A

CN117315624A - Obstacle detection method, vehicle control method, device, apparatus, and storage medium

Info

Publication number: CN117315624A
Application number: CN202311289595.2A
Authority: CN
Inventors: 张宝丰; 刘浩; 王丹
Original assignee: Beijing Jingdong Qianshi Technology Co Ltd
Current assignee: Beijing Jingdong Qianshi Technology Co Ltd
Priority date: 2023-10-08
Filing date: 2023-10-08
Publication date: 2023-12-29

Abstract

The disclosure provides an obstacle detection method, a vehicle control method, a device, equipment and a storage medium, and relates to the fields of artificial intelligence, intelligent driving and intelligent logistics. The detection method comprises the following steps: inputting a driving environment image related to the vehicle to a semantic segmentation layer, and outputting an environment semantic image; performing obstacle detection on the environment semantic image to obtain at least one initial obstacle detection result; according to the position relation between the initial obstacle detection result and the passing semantic image block, determining a candidate obstacle detection result which at least partially coincides with the passing semantic image block from at least one initial obstacle detection result; and inputting the candidate obstacle image blocks associated with the candidate obstacle detection result into the traffic obstacle detection model in the running environment image, and outputting the traffic obstacle detection result.

Description

Obstacle detection method, vehicle control method, device, apparatus, and storage medium

Technical Field

The present disclosure relates to the field of artificial intelligence, intelligent driving, and intelligent logistics, and more particularly, to an obstacle detection method, a vehicle control method, an apparatus, a device, and a storage medium.

Background

Along with the rapid development of technology, the automatic driving function is widely applied to vehicles such as passenger cars, logistics carriers and automatic inspection vehicles, so that the vehicles can realize the automatic driving functions such as automatic parking and automatic inspection according to the generated motion path, and the running efficiency and the operation efficiency of the vehicles are improved.

In the process of implementing the disclosed concept, the inventor finds that at least the following problems exist in the related art: it is difficult for the related vehicle to accurately recognize an obstacle in the running environment, resulting in a low running efficiency of the vehicle during automatic running.

Disclosure of Invention

In view of this, the present disclosure provides an obstacle detection method, a vehicle control method, an apparatus, a device, and a storage medium.

One aspect of the present disclosure provides an obstacle detection method including:

inputting a running environment image related to a vehicle to a semantic segmentation layer, and outputting an environment semantic image, wherein the environment semantic image comprises a passing semantic image block representing a passing area in the running environment, and the passing area is an area suitable for passing the vehicle;

performing obstacle detection on the environment semantic image to obtain at least one initial obstacle detection result;

Determining candidate obstacle detection results which at least partially coincide with the passing semantic image blocks from at least one initial obstacle detection result according to the position relation between the initial obstacle detection results and the passing semantic image blocks; and

and inputting a candidate obstacle image block associated with the candidate obstacle detection result into a traffic obstacle detection model in the running environment image, and outputting a traffic obstacle detection result.

According to an embodiment of the present disclosure, performing obstacle detection on the environmental semantic image to obtain at least one initial obstacle detection result includes:

inputting the environment semantic image into a first obstacle detection model, and outputting a first detection result, wherein the first detection result comprises a plurality of first obstacle classification results and first confidence degrees corresponding to the plurality of first obstacle classification results;

under the condition that the plurality of first confidence coefficients meet the preset condition, processing the plurality of first confidence coefficients according to an information entropy algorithm to obtain a first confidence coefficient entropy; and

and determining the first detection result as the initial obstacle detection result according to the first confidence entropy.

According to an embodiment of the present disclosure, the semantic segmentation layer is included in a second obstacle detection model, and the second obstacle detection model further includes an image countermeasure generation network layer, a degree of difference evaluation layer, and an abnormal obstacle detection layer;

wherein, the performing obstacle detection on the environmental semantic image to obtain at least one initial obstacle detection result includes:

inputting the environment semantic image into the image countermeasure generation network layer, and outputting a predicted driving environment image;

inputting the driving environment image and the predicted driving environment image into the difference evaluation layer, and outputting image difference information;

and processing the image difference degree information, the environment semantic image, the driving environment image and the predicted driving environment image according to the abnormal obstacle detection layer to obtain at least one initial obstacle detection result.

According to an embodiment of the present disclosure, the semantic segmentation layer further outputs environmental semantic dispersion information, the environmental semantic dispersion information characterizes semantic prediction probability dispersion of pixels in the environmental semantic image, and the abnormal obstacle detection layer includes a first convolution sub-layer, a second convolution sub-layer, a first fusion sub-layer, and an image decoding sub-layer;

Wherein the processing of the image difference information, the running environment image, and the predicted running environment image according to the abnormal obstacle detection layer includes:

inputting the environment semantic image, the driving environment image and the predicted driving environment image into the first convolution sublayer, and outputting environment semantic image characteristics, driving environment image characteristics and predicted driving environment image characteristics;

inputting the splicing result of the environment semantic image features, the driving environment image features and the predicted driving environment image features into the second convolution sublayer to obtain a first intermediate fusion feature;

inputting the first intermediate fusion feature, the image difference degree information and the environment semantic dispersion information into the first fusion sublayer to obtain a second intermediate fusion feature; and

and inputting the second intermediate fusion characteristic into the image decoding sublayer, and outputting at least one initial obstacle detection result.

According to an embodiment of the present disclosure, the inputting the candidate obstacle image block associated with the candidate obstacle detection result in the driving environment image to the traffic obstacle detection model, and outputting the traffic obstacle detection result includes:

Inputting the candidate obstacle image block into an obstacle recognition layer of the traffic obstacle detection model, and outputting a candidate obstacle category, wherein the traffic obstacle detection model further comprises a traffic condition prediction layer;

inputting the candidate obstacle category into the traffic condition prediction layer, and outputting predicted traffic condition parameters corresponding to the candidate obstacle detection result; and

and determining the candidate obstacle detection result as the passing obstacle detection result according to the predicted passing condition parameter.

According to an embodiment of the present disclosure, the above-described obstacle detection method further includes:

generating an obstacle detection message according to the passing obstacle detection result; and

and sending the obstacle detection message to the target client.

Another aspect of the present disclosure provides a vehicle control method including:

collecting a running environment image related to a vehicle;

according to the obstacle detection method provided by the embodiment of the disclosure, the driving environment image is processed, and a passing obstacle detection result is obtained; and

and controlling the vehicle to execute movement operation according to the passing obstacle detection result.

Another aspect of the present disclosure provides an obstacle detecting apparatus, including:

The environment semantic image acquisition module is used for inputting a running environment image related to the vehicle into the semantic segmentation layer and outputting the environment semantic image, wherein the environment semantic image comprises a passing semantic image block representing a passing area in the running environment, and the passing area is an area suitable for passing the vehicle;

the initial obstacle detection result obtaining module is used for carrying out obstacle detection on the environment semantic image to obtain at least one initial obstacle detection result;

a candidate obstacle detection result obtaining module, configured to determine a candidate obstacle detection result that at least partially coincides with the passing semantic image block from at least one of the initial obstacle detection results according to a positional relationship between the initial obstacle detection result and the passing semantic image block; and

and the passing obstacle detection result obtaining module is used for inputting the candidate obstacle image blocks associated with the candidate obstacle detection result into the passing obstacle detection model in the running environment image and outputting the passing obstacle detection result.

Another aspect of the present disclosure provides a vehicle control apparatus including:

The acquisition module is used for acquiring a running environment image related to the vehicle;

an obstacle detection module, configured to process the driving environment image according to the obstacle detection method described in any one of claims 1 to 6, to obtain a passing obstacle detection result; and

and the control module is used for controlling the vehicle to execute movement operation according to the passing obstacle detection result.

Another aspect of the present disclosure provides an electronic device, comprising:

one or more processors;

a memory for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the methods as described above.

Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed, are configured to implement a method as described above.

Another aspect of the present disclosure provides a computer program product comprising computer executable instructions which, when executed, are for implementing a method as described above.

According to the embodiment of the disclosure, the candidate obstacle detection result which is at least partially overlapped with the passing semantic image block is determined from the initial obstacle detection result, so that the initial obstacle detection result which is irrelevant to the passing area of the vehicle in the driving environment image can be removed, the data volume required to be processed by a subsequent passing obstacle detection model is reduced, meanwhile, the passing obstacle detection result is obtained by means of the image classification performance of the passing obstacle detection model, the prediction accuracy of passing conditions for rolling passing the obstacle can be improved, the technical problem of lower obstacle detection accuracy is solved, and the technical effect of improving the accuracy of subsequent vehicle control is realized.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments thereof with reference to the accompanying drawings in which:

FIG. 1 schematically illustrates an exemplary system architecture to which obstacle detection methods, apparatus may be applied, in accordance with embodiments of the present disclosure;

fig. 2 schematically illustrates a flowchart of an obstacle detection method according to an embodiment of the disclosure;

FIG. 3 schematically illustrates a schematic diagram of a second obstacle detection model according to an embodiment of the disclosure;

FIG. 4 schematically illustrates a schematic diagram of an abnormal obstacle detection layer according to an embodiment of the present disclosure;

FIG. 5 schematically illustrates a schematic diagram of a traffic obstacle detection model according to an embodiment of the disclosure;

FIG. 6 schematically illustrates a flow chart of a vehicle control method according to an embodiment of the disclosure;

FIG. 7 schematically illustrates an application scenario diagram of a vehicle control method according to an embodiment of the present disclosure;

fig. 8 schematically illustrates a block diagram of an obstacle detection device according to an embodiment of the disclosure;

fig. 9 schematically illustrates a block diagram of a vehicle control apparatus according to an embodiment of the present disclosure; and

fig. 10 schematically illustrates a block diagram of an electronic device adapted to implement the obstacle detection method, the vehicle control method, according to an embodiment of the disclosure.

Detailed Description

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is only exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the concepts of the present disclosure.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where expressions like at least one of "A, B and C, etc. are used, the expressions should generally be interpreted in accordance with the meaning as commonly understood by those skilled in the art (e.g.," a system having at least one of A, B and C "shall include, but not be limited to, a system having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

In embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc., of the data involved (including, but not limited to, user personal information) all comply with relevant legal regulations, are used for legal purposes, and do not violate well-known. In particular, necessary measures are taken for personal information of the user, illegal access to personal information data of the user is prevented, and personal information security, network security and national security of the user are maintained.

In embodiments of the present disclosure, the user's authorization or consent is obtained before the user's personal information is obtained or collected.

In application scenes such as unmanned, wisdom commodity circulation, unmanned vehicles, intelligent transfer robot etc. have vehicles of autopilot function, can carry out information acquisition to the running environment with the help of detection device such as camera, laser radar to through the environmental information that processes the acquisition, generate the testing result of barrier, and then be convenient for instruct the vehicle to plan the travel route again to the barrier that detects, and then promote the running safety and the running efficiency of vehicle. The inventor finds that in the running process of the vehicle, the accuracy of detecting the obstacle is low, and the condition of missed detection and false detection is easy to exist, so that the running efficiency and the running safety of the vehicle are negatively affected to a certain extent.

The embodiment of the disclosure provides an obstacle detection method, which comprises the following steps: inputting a running environment image related to a vehicle to a semantic segmentation layer, and outputting an environment semantic image, wherein the environment semantic image comprises a passing semantic image block representing a passing area in the running environment, and the passing area is an area suitable for passing of the vehicle; performing obstacle detection on the environment semantic image to obtain at least one initial obstacle detection result; according to the position relation between the initial obstacle detection result and the passing semantic image block, determining a candidate obstacle detection result which at least partially coincides with the passing semantic image block from at least one initial obstacle detection result; and inputting the candidate obstacle image blocks associated with the candidate obstacle detection result into the traffic obstacle detection model in the running environment image, and outputting the traffic obstacle detection result.

According to the embodiment of the disclosure, by determining the candidate obstacle detection result which is at least partially overlapped with the passing semantic image block from the initial obstacle detection results, the initial obstacle detection result which is irrelevant to the passing area of the vehicle in the driving environment image can be removed, so that the data amount required to be processed by a subsequent passing obstacle detection model is reduced, meanwhile, the passing obstacle detection result is obtained by means of the image classification performance of the passing obstacle detection model, the prediction accuracy of passing conditions for rolling passing through the obstacle can be improved, the obstacle detection precision is improved, and the accuracy of subsequent vehicle control is improved.

Fig. 1 schematically illustrates an exemplary system architecture 100 to which the obstacle detection methods, apparatuses may be applied, according to embodiments of the disclosure. It should be noted that fig. 1 is only an example of a system architecture to which embodiments of the present disclosure may be applied to assist those skilled in the art in understanding the technical content of the present disclosure, but does not mean that embodiments of the present disclosure may not be used in other devices, systems, environments, or scenarios.

As shown in fig. 1, a system architecture 100 according to this embodiment may include vehicles 101, 102, 103, a network 104, and a server 105. The network 104 is the medium used to provide communication links between the vehicles 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and/or wireless communication links, and the like.

A user may interact with the server 105 over the network 104 using the vehicles 101, 102, 103 to receive or send messages, etc. The vehicles 101, 102, 103 may have processors installed thereon and may also have various communication client applications installed, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients and/or social platform software, to name a few.

The vehicles 101, 102, 103 may be any type of vehicle having an autopilot function or an intelligent drive assist function, including, but not limited to, passenger cars, intelligent transfer robots, unmanned patrol cars, and the like.

The server 105 may be a server that provides various services, such as a background management server (by way of example only) that provides support for websites that users browse with the vehicles 101, 102, 103. The background management server may analyze and process the received data such as the user request, and may feed back the processing result (e.g., web pages, information, data, etc., acquired or generated according to the user request) to the vehicle.

It should be noted that, the obstacle detection method provided by the embodiment of the present disclosure may be generally performed by the vehicle 101, 102 or 103, or may be performed by another vehicle other than the vehicle 101, 102 or 103. Accordingly, the obstacle detecting apparatus provided by the embodiments of the present disclosure may be generally provided in the vehicle 101, 102, or 103, or in another vehicle other than the vehicle 101, 102, or 103. Alternatively, the obstacle detection method provided by the embodiment of the present disclosure may also be performed by the server 105. Accordingly, the obstacle detection device provided in the embodiment of the present disclosure may also be provided in the server 105. The obstacle detection method provided by the embodiments of the present disclosure may also be performed by a server or cluster of servers other than the server 105 and capable of communicating with the vehicles 101, 102, 103 and/or the server 105. Accordingly, the obstacle detection apparatus provided by the embodiments of the disclosure may also be provided in a server or a server cluster that is different from the server 105 and is capable of communicating with the vehicles 101, 102, 103 and/or the server 105.

It should be understood that the number of terminal devices, networks and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

Fig. 2 schematically shows a flowchart of an obstacle detection method according to an embodiment of the disclosure.

As shown in fig. 2, the obstacle detection method includes operations S210 to S240.

In operation S210, a driving environment image related to a vehicle is input to a semantic segmentation layer, and an environment semantic image is output, wherein the environment semantic image includes a traffic semantic image block representing a traffic region in the driving environment, the traffic region being a region suitable for traffic of the vehicle;

according to embodiments of the present disclosure, the vehicle may be any type of vehicle, such as a passenger car, a truck, an automated guided vehicle, an intelligent patrol robot, etc., and the embodiments of the present disclosure do not limit the type of vehicle. The driving environment image may represent the driving environment of the vehicle, for example, may represent a road, a building, a pedestrian, a traffic sign, or the like.

According to an embodiment of the disclosure, the semantic segmentation layer may be constructed based on an image semantic segmentation algorithm, and the environment semantic image may be an image obtained by performing image semantic segmentation on the driving environment image. The environment semantic image can comprise an image area which is obtained by carrying out image semantic segmentation and represents any type of driving environment information such as buildings, roads, barriers and the like.

According to an embodiment of the present disclosure, the traffic semantic image block may be a semantic image block corresponding to an area where a vehicle can travel in the environmental semantic image, and the area where the vehicle can travel may be, for example, a lane area, a vehicle operation area, or the like.

In operation S220, obstacle detection is performed on the environmental semantic image, and at least one initial obstacle detection result is obtained.

According to the embodiment of the present disclosure, the initial obstacle detection result may be any type of detection result of a detection frame, an obstacle type, a size, an image block, etc. corresponding to an obstacle in a driving environment, and the embodiment of the present disclosure does not limit a specific type of the initial obstacle detection result.

According to the embodiments of the present disclosure, the initial obstacle detection result may be obtained based on any type of neural network algorithm, for example, the environment semantic image may be processed based on a multi-layer perceptron algorithm, but not limited thereto, and the initial obstacle detection result may be obtained based on other types of neural network algorithms, which are not limited thereto.

In operation S230, a candidate obstacle detection result at least partially overlapping with the pass semantic image block is determined from at least one initial obstacle detection result according to a positional relationship between the initial obstacle detection result and the pass semantic image block.

According to an embodiment of the present disclosure, the positional relationship between the initial obstacle detection result and the traffic semantic image block may be determined by a pixel position corresponding to the initial obstacle detection result and a pixel position of the traffic semantic image block. However, the method is not limited thereto, and the positional relationship between the initial obstacle detection result and the traffic semantic image block may be determined based on other manners, for example, the positional relationship is determined based on the obstacle geographic coordinate position corresponding to the initial obstacle detection result and the geographic position of the traffic area, and the embodiment of the present disclosure does not limit the specific manner of determining the positional relationship between the initial obstacle detection result and the traffic semantic image block.

According to the embodiment of the disclosure, the candidate obstacle detection result at least partially overlapped with the passing semantic image block can represent the obstacle influencing the running of the vehicle, so that the candidate obstacle detection result is determined from at least one initial obstacle detection result, and at least the initial obstacle detection result which does not influence the running of the vehicle can be removed, thereby reducing the data processing amount of the subsequent passing obstacle detection model and saving the calculation overhead generated in the obstacle detection process.

In operation S240, a candidate obstacle image block associated with the candidate obstacle detection result in the driving environment image is input to the passing obstacle detection model, and the passing obstacle detection result is output.

According to the embodiment of the disclosure, the traffic obstacle detection model may be constructed based on any type of neural network algorithm, for example, may be constructed based on an attention network algorithm, or may also be constructed based on other types of neural network algorithms, and the specific algorithm type for constructing the traffic obstacle detection model is not limited in the embodiment of the disclosure.

According to an embodiment of the present disclosure, the passing obstacle detection result may indicate a passing condition in which the vehicle passes through the obstacle by rolling, and the passing obstacle detection result may include a detection result corresponding to an obstacle through which the vehicle can pass by rolling, such as a passing obstacle of an empty paper box, a branch, or the like. Therefore, based on the passing obstacle detection result, the vehicle can be instructed to roll and pass through the passing obstacle in the running environment, so that the running efficiency of the vehicle is improved. Or, the passing obstacle detection result can also indicate the vehicle to bypass the obstacle, so that the safety accident caused by collision of the vehicle with the obstacle such as the indication board is avoided.

According to an embodiment of the present disclosure, operation S220 of performing obstacle detection on the environmental semantic image, obtaining at least one initial obstacle detection result may include the following operations.

Inputting the environment semantic image into a first obstacle detection model, and outputting a first detection result, wherein the first detection result comprises a plurality of first obstacle classification results and first confidence degrees corresponding to the plurality of first obstacle classification results respectively; under the condition that the plurality of first confidence coefficients meet the preset condition, the plurality of first confidence coefficients are processed according to an information entropy algorithm to obtain a first confidence coefficient entropy; and determining the first detection result as an initial obstacle detection result according to the first confidence entropy.

According to embodiments of the present disclosure, the first obstacle detection model may be constructed based on a target detection algorithm, for example, may be constructed based on an attention network algorithm, or may also be constructed based on other types of neural network algorithms, for example, a residual network algorithm, or the like. The embodiment of the present disclosure does not limit the specific algorithm type for constructing the first obstacle detection model.

According to embodiments of the present disclosure, the first obstacle classification result may characterize a category of obstacles, such as a category of pedestrians, bicycles, street trees, and the like.

According to an embodiment of the present disclosure, the first confidence may characterize a predictive probability for the first obstacle classification result. The case where the preset condition is satisfied may be a case where it is difficult to determine the classification result of the obstacle by the plurality of first confidence degrees. Whether the preset condition is satisfied may be determined based on the difference values between the plurality of first confidences, for example, it may be determined that the difference value between the plurality of first confidences is set to be smaller than or equal to the preset value to satisfy the preset condition, or it may be determined that the difference value between the M first confidences of the N first confidences is smaller than or equal to the preset value to satisfy the preset condition, n+.m1. The embodiment of the present disclosure does not limit a specific setting manner of the preset condition.

According to the embodiment of the disclosure, the plurality of first confidences are processed through the information entropy algorithm, the obtained first confidence entropy is used for representing the uncertainty of the first obstacle detection model on the first obstacle classification result, so that the first detection result can be determined as the initial obstacle detection result under the condition that the first confidence entropy meets the preset entropy threshold, the problem of obstacle missed detection and false detection caused by difficulty in determining the category of the obstacle is avoided at least in the initial detection of the abnormal category of the obstacle, and the further accurate detection of the category of the obstacle in the driving environment image is realized through the subsequent passing obstacle detection model, so that the obstacle detection precision is improved.

According to an embodiment of the present disclosure, the semantic segmentation layer is included in a second obstacle detection model that further includes an image countermeasure generation network layer, a degree of difference evaluation layer, and an abnormal obstacle detection layer.

Fig. 3 schematically illustrates a schematic diagram of a second obstacle detection model according to an embodiment of the disclosure.

As shown in fig. 3, the second obstacle detection model 300 may include a semantic segmentation layer 310, an image countermeasure generation network layer 320, a degree of difference evaluation layer 330, and an abnormal obstacle detection layer 340.

According to an embodiment of the present disclosure, performing obstacle detection on the environmental semantic image, obtaining at least one initial obstacle detection result may further include the following operations.

Inputting the environment semantic image into an image countermeasure generation network layer, and outputting a predicted driving environment image; the driving environment image and the predicted driving environment image are input to a difference degree evaluation layer, and image difference degree information is output; and processing the image difference degree information, the environment semantic image, the driving environment image and the predicted driving environment image according to the abnormal obstacle detection layer to obtain at least one initial obstacle detection result.

As shown in fig. 3, a driving environment image 301 may be input to a semantic segmentation envelope 310, outputting an environment semantic image 302. The environmental semantic image 302 is input to the image countermeasure generation network layer 310, and the predicted running environment image 303 is output. The image countermeasure generation network layer 320 may be constructed based on cGAN (Conditional Generative Adversarial Network, conditional generation countermeasure network) algorithm, and may generate a predicted driving environment image with pixel-to-pixel correspondence according to a semantic mapping relationship between the semantic environment image 302 and the driving environment image 301 based on the image countermeasure generation network layer 320 obtained after training, so as to implement re-synthesis of the driving environment image.

As shown in fig. 3, the running environment image 301 and the predicted running environment image 303 may also be input to the difference evaluation layer 330, outputting the image difference information 304. The disparity estimation layer 330 may be constructed based on a learnable perceived image block similarity (Learned Perceptual Image Patch Similarity, LPIPS) algorithm. The predicted running environment image 303 may miss basic information such as color, appearance, etc. of objects in the running environment such as obstacles in the running environment image 301, and by inputting the running environment image 301 and the predicted running environment image 303 to the difference evaluation layer 330, it is possible to implement pixel value comparison of the running environment image 301 and the predicted running environment image 303, and characterize the perceived difference between the running environment image 301 and the predicted running environment image 303 by the image difference information 304, so as to promote misclassification, and omission of abnormal classification for the obstacles.

As shown in fig. 3, the image difference information 304, the environment semantic image 302, the driving environment image 301, and the predicted driving environment image 303 may be input to the abnormal obstacle detection layer 340 according to the abnormal obstacle detection layer processing image difference information, the environment semantic image, the driving environment image, and the predicted driving environment image, to obtain at least one initial obstacle detection result 305.

According to an embodiment of the present disclosure, the abnormal obstacle detection layer may be constructed based on an image encoder and an image decoder, and the image encoder may generate an image code representing a driving environment by fusing at least image difference information, an environment semantic image, a driving environment image, and a predicted driving environment image, and decode the image by a decoder, resulting in an accurate initial obstacle detection result.

According to an embodiment of the disclosure, the semantic segmentation layer further outputs environmental semantic dispersion information, the environmental semantic dispersion information characterizing semantic prediction probability dispersion of pixels in the environmental semantic image, and the abnormal obstacle detection layer comprises a first convolution sub-layer, a second convolution sub-layer, a first fusion sub-layer and an image decoding sub-layer.

According to embodiments of the present disclosure, the environmental semantic dispersion information may include pixel class prediction probability uncertainty information, as well as a difference distance between pixel class prediction probabilities. The pixel class prediction probability uncertainty information may be represented by the following equation (1), for example.

In the formula (1), hx represents pixel class prediction probability uncertainty information, p (c) represents a prediction probability corresponding to a pixel in the environmental semantic image, and c represents a prediction probability corresponding to the pixel (i.e., pixel class prediction probability).

Also, for example, the difference distance between pixel class prediction probabilities can be expressed by the following formula (2).

In the formula (2), dx represents a difference distance between pixel class prediction probabilities, and p (c') represents an average value of pixel class prediction probabilities of a plurality of pixels in the environmental semantic image.

According to the embodiment of the disclosure, the uncertainty of the prediction for the obstacle in the environment semantic image can be quantitatively represented through the environment semantic dispersion information, so that the detection precision of the abnormal obstacle which is difficult to classify can be improved by enabling the abnormal obstacle detection layer to fully learn the uncertainty information of the prediction for the obstacle, and the detection precision of the passing obstacle can be further improved.

Fig. 4 schematically illustrates a schematic diagram of an abnormal obstacle detection layer according to an embodiment of the present disclosure.

As shown in fig. 4, the abnormal obstacle detection layer 400 may include a first convolution sub-layer 410, a second convolution sub-layer 420, a first fusion sub-layer 430, and an image decoding sub-layer 440.

According to an embodiment of the present disclosure, processing the image difference degree information, the driving environment image, and the predicted driving environment image according to the abnormal obstacle detection layer may include the following operations.

Inputting the environment semantic image, the driving environment image and the predicted driving environment image into a first convolution sublayer, and outputting environment semantic image features, driving environment image features and predicted driving environment image features; inputting the splicing result of the environment semantic image features, the driving environment image features and the predicted driving environment image features into a second convolution sublayer to obtain a first intermediate fusion feature; inputting the first intermediate fusion characteristics, the image difference information and the environment semantic dispersion information into a first fusion sublayer to obtain second intermediate fusion characteristics; and inputting the second intermediate fusion feature into the image decoding sublayer, and outputting at least one initial obstacle detection result.

As shown in fig. 4, the environmental semantic image 401, the driving environment image 402, and the predicted driving environment image 403 may be input to the first convolution sub-layer 410, outputting the environmental semantic image features 401', the driving environment image features 402', and the predicted driving environment image features 403'. The first convolution sublayer 410 may be constructed based on a convolutional neural network algorithm, for example, may be constructed based on a VGG (Visual Geometry Group) network structure, so as to extract respective depth image features of images from an environment semantic image, a driving environment image and a predicted driving environment image.

As shown in fig. 4, the concatenation result of the environmental semantic image feature 401', the driving environment image feature 402', and the predicted driving environment image feature 403' may be input to the second convolution sublayer 420, and the first intermediate fusion feature 406 may be output. The second convolutional sublayer 420 may be constructed based on a convolutional neural network algorithm, for example, may be constructed based on a single convolutional neural network layer.

As shown in fig. 4, the first intermediate fusion feature 406, the environmental semantic dispersion information 404 and the image difference information 405 are input to the first fusion sub-layer 430, and the second intermediate fusion feature 407 is output.

According to the embodiment of the disclosure, the first fusion sub-layer 430 can calculate the feature correlation between the first intermediate fusion feature 406 and the environmental semantic dispersion information 404 and the feature correlation between the first intermediate fusion feature 406 and the image difference information 405, so that feature data with category prediction uncertainty in the first intermediate fusion feature 406 can be more accurately represented, abnormal segmentation part areas in the environmental semantic image can be accurately predicted through the image decoding sub-layer, the number of missed images in the image semantic segmentation stage is reduced, the number of recalls for abnormal type obstacles, such as short obstacles with long tail characteristics of data, and other unknown type obstacles, and recall accuracy are improved, and the missed detection problem for the abnormal type obstacles is reduced.

According to embodiments of the present disclosure, the first fusion sublayer may be constructed based on a correlation algorithm, for example, a correlation coefficient algorithm. But not limited thereto, the first fusion sublayer may also be constructed based on an attention network algorithm, and the specific algorithm type for constructing the first fusion sublayer is not limited in the embodiments of the present disclosure.

According to an embodiment of the present disclosure, operation S240, inputting a candidate obstacle image block associated with a candidate obstacle detection result in a driving environment image to a passing obstacle detection model, and outputting the passing obstacle detection result may include the following operations.

Inputting the candidate obstacle image blocks into an obstacle recognition layer of a traffic obstacle detection model, and outputting candidate obstacle categories, wherein the traffic obstacle detection model further comprises a traffic condition prediction layer; inputting the candidate obstacle category into a traffic condition prediction layer, and outputting predicted traffic condition parameters corresponding to the candidate obstacle detection result; and determining the candidate obstacle detection result as a traffic obstacle detection result according to the predicted traffic condition parameter.

According to the embodiment of the disclosure, the candidate obstacle image block corresponding to the candidate obstacle detection result may be segmented from the driving environment image according to the candidate obstacle detection result, and the candidate obstacle image block may characterize the candidate obstacle.

According to the embodiment of the disclosure, the obstacle recognition layer may be constructed based on an image classification algorithm, for example, may be constructed based on a multi-layer perceptron algorithm, or may also be constructed based on a convolutional neural network algorithm, and the specific algorithm type for constructing the obstacle recognition layer is not limited in the embodiment of the disclosure.

According to the embodiment of the disclosure, the traffic condition predicting layer may obtain a traffic obstacle detection result indicating whether the vehicle can roll and travel through the candidate obstacle by processing the identified candidate obstacle category. The passing obstacle detection result may be, for example, marked with 1 and 0 for pixels corresponding to an abnormal type of obstacle in the running environment image, may indicate that the vehicle is able to roll through the obstacle for the pixel marked with 1, and may indicate that the vehicle needs to bypass the obstacle for the pixel marked with 0.

According to the embodiment of the disclosure, the candidate obstacles representing the abnormal types can be further and accurately classified through the passing obstacle detection model, so that the passing obstacles such as well covers, empty cartons and the like in the passing area can be accurately identified, the accuracy of subsequent vehicle motion control is improved, and the running efficiency of the vehicle is improved.

Fig. 5 schematically illustrates a schematic diagram of a traffic obstacle detection model according to an embodiment of the disclosure.

As shown in fig. 5, the traffic obstacle detection model 500 may include an obstacle recognition layer 510 and a traffic condition prediction layer 520. The candidate obstacle image block 501 may be input to the obstacle recognition layer of the traffic obstacle detection model 500, and the candidate obstacle category 502 may be output. The candidate obstacle category 502 is input to the traffic condition prediction layer 520, and the predicted traffic condition parameter 503 corresponding to the candidate obstacle detection result is output.

According to an embodiment of the present disclosure, the obstacle detection method may further include the following operations.

Generating an obstacle detection message according to the passing obstacle detection result; and sending an obstacle detection message to the target client.

According to the embodiment of the disclosure, the passing obstacle detection result can accurately identify the passing obstacle such as the manhole cover, the empty paper box and the like in the passing area, so that the passing efficiency of a plurality of vehicles can be improved by sending the obstacle detection message containing the passing obstacle detection result to the target client or the service end related to the vehicle path planning, so that the target client or the service end can improve the accuracy of vehicle path planning, and the generated vehicle path can be issued to one or more other vehicles.

Fig. 6 schematically illustrates a flowchart of a vehicle control method according to an embodiment of the present disclosure.

As shown in fig. 6, the vehicle control method may include operations S610 to S630.

In operation S610, a driving environment image related to a vehicle is acquired.

In operation S620, the obstacle detection method provided according to the embodiment of the present disclosure processes the driving environment image to obtain a passing obstacle detection result.

In operation S630, the vehicle is controlled to perform a moving operation according to the passing obstacle detection result.

According to the embodiment of the disclosure, the vehicle is controlled to perform the movement operation according to the passing obstacle detection result, for example, a vehicle planned path may be generated according to the passing obstacle detection result, and the vehicle movement may be controlled according to the vehicle planned path.

Fig. 7 schematically illustrates an application scenario diagram of a vehicle control method according to an embodiment of the present disclosure.

As shown in fig. 7, in the application scenario 700, a vehicle 710 may perform image acquisition on a driving environment by using an image acquisition device, so as to obtain a driving environment image. Meanwhile, the vehicle 710 may further process the driving environment image based on the obstacle detection method provided in the embodiment of the present disclosure, to obtain a passing obstacle detection result. The passing obstacle detection result may indicate that the obstacle 711 is an empty paper box in the travel path. From the passing obstacle detection result, a travel path rolled through the obstacle 711 may be generated, so that it is known that the control vehicle 710 performs a moving operation in accordance with the travel path.

Fig. 8 schematically illustrates a block diagram of an obstacle detection device according to an embodiment of the disclosure.

As shown in fig. 8, the obstacle detection device 800 includes an environmental semantic image obtaining module 810, an initial obstacle detection result obtaining module 820, a candidate obstacle detection result obtaining module 830, and a passing obstacle detection result obtaining module 840.

The environment semantic image obtaining module 810 is configured to input a driving environment image related to a vehicle to the semantic segmentation layer, and output an environment semantic image, where the environment semantic image includes a traffic semantic image block representing a traffic region in the driving environment, and the traffic region is a region suitable for traffic of the vehicle.

The initial obstacle detection result obtaining module 820 is configured to perform obstacle detection on the environmental semantic image to obtain at least one initial obstacle detection result.

The candidate obstacle detection result obtaining module 830 is configured to determine, from at least one initial obstacle detection result, a candidate obstacle detection result that at least partially coincides with the traffic semantic image block according to a positional relationship between the initial obstacle detection result and the traffic semantic image block.

The traffic obstacle detection result obtaining module 840 is configured to input, in the driving environment image, a candidate obstacle image block associated with the candidate obstacle detection result to the traffic obstacle detection model, and output a traffic obstacle detection result.

According to an embodiment of the present disclosure, an initial obstacle detection result obtaining module includes: the device comprises a first detection result obtaining sub-module, a first confidence entropy obtaining sub-module and a first initial obstacle detection result obtaining sub-module.

The first detection result obtaining sub-module is used for inputting the environment semantic image into the first obstacle detection model, outputting the first detection result, wherein the first detection result comprises a plurality of first obstacle classification results and first confidence degrees corresponding to the plurality of first obstacle classification results respectively.

The first confidence coefficient entropy obtaining sub-module is used for processing the plurality of first confidence coefficients according to the information entropy algorithm to obtain first confidence coefficient entropy under the condition that the plurality of first confidence coefficients meet the preset condition.

The first initial obstacle detection result obtaining sub-module is used for determining the first detection result as an initial obstacle detection result according to the first confidence entropy.

Wherein, initial obstacle detection result obtains the module and includes: the system comprises a predicted driving environment image generation sub-module, an image difference degree information generation sub-module and a second initial obstacle detection result obtaining sub-module.

The predicted running environment image generation sub-module is used for inputting the environment semantic image into the image countermeasure generation network layer by the predicted running environment image and outputting the predicted running environment image.

The image difference degree information generation sub-module is used for inputting the running environment image and the predicted running environment image into the difference degree evaluation layer and outputting the image difference degree information;

the second initial obstacle detection result obtaining sub-module is used for obtaining at least one initial obstacle detection result according to the abnormal obstacle detection layer processing image difference degree information, the environment semantic image, the driving environment image and the predicted driving environment image.

Wherein the second initial obstacle detection result obtaining submodule includes: the device comprises a feature extraction unit, a first intermediate fusion feature obtaining unit, a second intermediate fusion feature obtaining unit and an initial obstacle detection result obtaining unit.

The feature extraction unit is used for inputting the environment semantic image, the driving environment image and the predicted driving environment image into the first convolution sublayer and outputting environment semantic image features, driving environment image features and predicted driving environment image features.

The first intermediate fusion feature obtaining unit is used for inputting the splicing result of the environment semantic image features, the driving environment image features and the predicted driving environment image features into the second convolution sublayer to obtain the first intermediate fusion features.

The second intermediate fusion feature obtaining unit is used for inputting the first intermediate fusion feature, the image difference degree information and the environment semantic dispersion degree information into the first fusion sublayer to obtain a second intermediate fusion feature.

And the initial obstacle detection result obtaining unit is used for inputting the second intermediate fusion characteristic into the image decoding sublayer and outputting at least one initial obstacle detection result.

According to an embodiment of the present disclosure, a traffic obstacle detection result obtaining module includes: the system comprises a candidate obstacle category obtaining sub-module, a predicted traffic condition parameter obtaining sub-module and a traffic obstacle detection result obtaining sub-module.

The candidate obstacle category obtaining sub-module is used for inputting the candidate obstacle image block into an obstacle recognition layer of the traffic obstacle detection model and outputting the candidate obstacle category, wherein the traffic obstacle detection model further comprises a traffic condition prediction layer.

And the predicted traffic condition parameter obtaining sub-module is used for inputting the candidate obstacle category into the traffic condition predicting layer and outputting the predicted traffic condition parameter corresponding to the candidate obstacle detection result.

And the traffic obstacle detection result obtaining submodule is used for determining the candidate obstacle detection result as a traffic obstacle detection result according to the predicted traffic condition parameter.

According to an embodiment of the present disclosure, the obstacle detection device further includes: the message generating module and the message sending module.

And the message generating module is used for generating an obstacle detection message according to the passing obstacle detection result.

And the message sending module is used for sending the obstacle detection message to the target client.

Fig. 9 schematically shows a block diagram of a vehicle control apparatus according to an embodiment of the present disclosure.

As shown in fig. 9, the vehicle control apparatus 900 includes: an acquisition module 910, an obstacle detection module 920, and a control module 930.

The acquisition module 910 is used to acquire a running environment image related to a vehicle.

The obstacle detection module 920 is configured to process the driving environment image according to the obstacle detection method provided in the embodiment of the disclosure, and obtain a passing obstacle detection result.

The control module 930 is configured to control the vehicle to perform a movement operation according to the detection result of the traffic obstacle.

Any number of the modules, sub-modules, units, or at least some of the functionality of any number of the modules, sub-modules, units, may be implemented in one module in accordance with embodiments of the present disclosure. Any one or more of the modules, sub-modules, units according to embodiments of the present disclosure may be implemented as a split into multiple modules. Any one or more of the modules, sub-modules, units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or in hardware or firmware in any other reasonable manner of integrating or packaging the circuits, or in any one of or in any suitable combination of three of software, hardware, and firmware. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be at least partially implemented as computer program modules, which when executed, may perform the corresponding functions.

For example, the environmental semantic image obtaining module 810, the initial obstacle detection result obtaining module 820, the candidate obstacle detection result obtaining module 830, and the passing obstacle detection result obtaining module 840, or any of the acquisition module 910, the obstacle detection module 920, and the control module 930 may be combined in one module/sub-module/unit, or any one of the modules/sub-modules/units may be split into a plurality of modules/sub-modules/units. Alternatively, at least some of the functionality of one or more of these modules/sub-modules/units may be combined with at least some of the functionality of other modules/sub-modules/units and implemented in one module/sub-module/unit. According to embodiments of the present disclosure, at least one of the environmental semantic image acquisition module 810, the initial obstacle detection result acquisition module 820, the candidate obstacle detection result acquisition module 830, and the traffic obstacle detection result acquisition module 840, or the acquisition module 910, the obstacle detection module 920, and the control module 930 may be implemented at least in part as hardware circuitry, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in hardware or firmware in any other reasonable manner of integrating or packaging the circuitry, or in any one of or a suitable combination of three of software, hardware, and firmware. Alternatively, at least one of the environmental semantic image obtaining module 810, the initial obstacle detection result obtaining module 820, the candidate obstacle detection result obtaining module 830, and the passing obstacle detection result obtaining module 840, or the collecting module 910, the obstacle detection module 920, and the control module 930 may be at least partially implemented as a computer program module, which may perform corresponding functions when being executed.

It should be noted that, in the embodiment of the present disclosure, the obstacle detection device portion corresponds to the obstacle detection method portion in the embodiment of the present disclosure, and the description of the obstacle detection device portion specifically refers to the obstacle detection method portion, which is not described herein.

It should be noted that, in the embodiment of the present disclosure, the vehicle control device portion corresponds to the vehicle control method portion in the embodiment of the present disclosure, and the description of the vehicle control device portion specifically refers to the vehicle control method portion and is not described herein again.

Fig. 10 schematically illustrates a block diagram of an electronic device adapted to implement the obstacle detection method, the vehicle control method, according to an embodiment of the disclosure. The electronic device shown in fig. 10 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.

As shown in fig. 10, an electronic device 1000 according to an embodiment of the present disclosure includes a processor 1001 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1002 or a program loaded from a storage section 1008 into a Random Access Memory (RAM) 1003. The processor 1001 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. The processor 1001 may also include on-board memory for caching purposes. The processor 1001 may include a single processing unit or multiple processing units for performing different actions of the method flows according to embodiments of the present disclosure.

In the RAM 1003, various programs and data necessary for the operation of the electronic apparatus 1000 are stored. The processor 1001, the ROM 1002, and the RAM 1003 are connected to each other by a bus 1004. The processor 1001 performs various operations of the method flow according to the embodiment of the present disclosure by executing programs in the ROM 1002 and/or the RAM 1003. Note that the program may be stored in one or more memories other than the ROM 1002 and the RAM 1003. The processor 1001 may also perform various operations of the method flow according to the embodiments of the present disclosure by executing programs stored in the one or more memories.

According to an embodiment of the disclosure, the electronic device 1000 may also include an input/output (I/O) interface 1005, the input/output (I/O) interface 1005 also being connected to the bus 1004. The system 1000 may also include one or more of the following components connected to an input/output (I/O) interface 1005: an input section 1006 including a keyboard, a mouse, and the like; an output portion 1007 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage portion 1008 including a hard disk or the like; and a communication section 1009 including a network interface card such as a LAN card, a modem, or the like. The communication section 1009 performs communication processing via a network such as the internet. The drive 1010 is also connected to an input/output (I/O) interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like, is installed as needed in the drive 1010, so that a computer program read out therefrom is installed as needed in the storage section 1008.

According to embodiments of the present disclosure, the method flow according to embodiments of the present disclosure may be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 1009, and/or installed from the removable medium 1011. The above-described functions defined in the system of the embodiments of the present disclosure are performed when the computer program is executed by the processor 1001. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

The present disclosure also provides a computer-readable storage medium that may be embodied in the apparatus/device/system described in the above embodiments; or may exist alone without being assembled into the apparatus/device/system. The computer-readable storage medium carries one or more programs which, when executed, implement methods in accordance with embodiments of the present disclosure.

According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. Examples may include, but are not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

For example, according to embodiments of the present disclosure, the computer-readable storage medium may include ROM 1002 and/or RAM 1003 and/or one or more memories other than ROM 1002 and RAM 1003 described above.

Embodiments of the present disclosure also include a computer program product comprising a computer program comprising program code for performing the methods provided by the embodiments of the present disclosure, when the computer program product is run on an electronic device, for causing the electronic device to carry out the methods provided by the embodiments of the present disclosure.

The above-described functions defined in the system/apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 1001. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the disclosure.

In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted in the form of signals on a network medium, distributed, and downloaded and installed via the communication section 1009, and/or installed from the removable medium 1011. The computer program may include program code that may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.

According to embodiments of the present disclosure, program code for performing computer programs provided by embodiments of the present disclosure may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. Those skilled in the art will appreciate that the features recited in the various embodiments of the disclosure and/or in the claims may be combined in various combinations and/or combinations, even if such combinations or combinations are not explicitly recited in the disclosure. In particular, the features recited in the various embodiments of the present disclosure and/or the claims may be variously combined and/or combined without departing from the spirit and teachings of the present disclosure. All such combinations and/or combinations fall within the scope of the present disclosure.

The embodiments of the present disclosure are described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the embodiments cannot be used advantageously in combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be made by those skilled in the art without departing from the scope of the disclosure, and such alternatives and modifications are intended to fall within the scope of the disclosure.

Claims

1. An obstacle detection method comprising:

inputting a running environment image related to a vehicle to a semantic segmentation layer, and outputting an environment semantic image, wherein the environment semantic image comprises a passing semantic image block representing a passing area in the running environment, and the passing area is an area suitable for passing of the vehicle;

2. The method of claim 1, wherein performing obstacle detection on the environmental semantic image to obtain at least one initial obstacle detection result comprises:

under the condition that the first confidence coefficients meet preset conditions, the first confidence coefficients are processed according to an information entropy algorithm to obtain first confidence coefficient entropy; and

3. The method of claim 1, wherein the semantic segmentation layer is included in a second obstacle detection model, the second obstacle detection model further comprising an image countermeasure generation network layer, a degree of difference evaluation layer, and an abnormal obstacle detection layer;

inputting the driving environment image and the predicted driving environment image into the difference degree evaluation layer, and outputting image difference degree information;

4. The method of claim 3, wherein the semantic segmentation layer further outputs environmental semantic dispersion information characterizing semantic predictive probability dispersions of pixels in the environmental semantic image, the abnormal obstacle detection layer comprising a first convolution sub-layer, a second convolution sub-layer, a first fusion sub-layer, and an image decoding sub-layer;

wherein the processing the image difference degree information, the running environment image, and the predicted running environment image according to the abnormal obstacle detection layer includes:

Inputting the environment semantic image, the driving environment image and the predicted driving environment image to the first convolution sublayer, and outputting environment semantic image characteristics, driving environment image characteristics and predicted driving environment image characteristics;

5. The method of claim 1, wherein the inputting the candidate obstacle image block associated with the candidate obstacle detection result in the driving environment image to a traffic obstacle detection model, outputting the traffic obstacle detection result includes:

and determining the candidate obstacle detection result as the traffic obstacle detection result according to the predicted traffic condition parameter.

6. The method of claim 1, further comprising:

and sending the obstacle detection message to the target client.

7. A vehicle control method comprising:

collecting a running environment image related to a vehicle;

the obstacle detection method according to any one of claims 1 to 6, processing the running environment image to obtain a passing obstacle detection result; and

8. An obstacle detection device comprising:

the environment semantic image acquisition module is used for inputting a running environment image related to the vehicle into the semantic segmentation layer and outputting the environment semantic image, wherein the environment semantic image comprises a passing semantic image block representing a passing area in the running environment, and the passing area is an area suitable for passing of the vehicle;

a candidate obstacle detection result obtaining module, configured to determine a candidate obstacle detection result that at least partially coincides with the passing semantic image block from at least one initial obstacle detection result according to a positional relationship between the initial obstacle detection result and the passing semantic image block; and

and the passing obstacle detection result obtaining module is used for inputting the candidate obstacle image blocks associated with the candidate obstacle detection result into a passing obstacle detection model in the running environment image and outputting the passing obstacle detection result.

9. A vehicle control apparatus comprising:

an obstacle detection module for processing the driving environment image according to the obstacle detection method of any one of claims 1 to 6 to obtain a passing obstacle detection result; and

10. An electronic device, comprising:

one or more processors;

a memory for storing one or more programs,

wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1 to 7.

11. A computer readable storage medium having stored thereon executable instructions which when executed by a processor cause the processor to implement the method of any of claims 1 to 7.

12. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.