CN113591569A

CN113591569A - Obstacle detection method, obstacle detection device, electronic apparatus, and storage medium

Info

Publication number: CN113591569A
Application number: CN202110721781.3A
Authority: CN
Inventors: 路金诚; 张伟; 谭啸; 孙昊
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-06-28
Filing date: 2021-06-28
Publication date: 2021-11-02

Abstract

The disclosure provides an obstacle detection method, an obstacle detection device, electronic equipment and a storage medium, relates to the field of artificial intelligence, in particular to computer vision and deep learning technology, and can be used in intelligent transportation and smart city scenes. The specific implementation scheme is as follows: acquiring a detection image acquired in a target scene and a background image of the target scene, wherein the background image does not contain an obstacle; generating a target image according to the detection image and the background image, wherein the color channel of each pixel point in the target image comprises the color channel of the corresponding pixel point in the detection image and the color channel of the corresponding pixel point in the background image; and inputting the target image into the trained detection model to acquire the obstacle information in the detection image. Therefore, the detection model can accurately detect the obstacles which do not appear in the training process of the detection model, and the obstacle detection accuracy is improved.

Description

Obstacle detection method, obstacle detection device, electronic apparatus, and storage medium

Technical Field

The present disclosure relates to the field of artificial intelligence, and in particular, to computer vision and deep learning technologies, which are particularly applicable to intelligent transportation and smart city scenes, and in particular, to a method and an apparatus for detecting obstacles, an electronic device, and a storage medium.

Background

With the rapid development of the automatic driving technology, the automatic driving vehicle is gradually replacing the conventional vehicle as a travel choice of the user. Automatic driving not only minimizes the risk of vehicle travel, but also reduces the burdensome driving tasks for the user. Thus, autonomous driving will be a big trend for future automobile development.

Autonomous parking is an important function in automatic driving, which means that a vehicle is automatically parked without manual operation or control. In an autonomous parking scene, a parking space or an obstacle on a passage of a parking place needs to be accurately positioned so as to determine an empty parking space and avoid a driving route of the obstacle in the passage, so that an autonomous vehicle is guided to realize autonomous parking.

Disclosure of Invention

The disclosure provides an obstacle detection method, an obstacle detection device, an electronic apparatus, and a storage medium.

According to an aspect of the present disclosure, there is provided an obstacle detection method including: acquiring a detection image acquired in a target scene and a background image of the target scene, wherein the background image does not contain an obstacle; generating a target image according to the detection image and the background image, wherein the color channels of all pixel points in the target image comprise the color channels of the corresponding pixel points in the detection image and the color channels of the corresponding pixel points in the background image; and inputting the target image into a trained detection model to acquire the obstacle information in the detection image.

According to another aspect of the present disclosure, there is provided an obstacle detection apparatus including: the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a detection image acquired in a target scene and a background image of the target scene, and the background image does not contain obstacles; a generating module, configured to generate a target image according to the detection image and the background image, where a color channel of each pixel in the target image includes a color channel of a corresponding pixel in the detection image and a color channel of a corresponding pixel in the background image; and the first processing module is used for inputting the target image into the trained detection model so as to acquire the obstacle information in the detection image.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the obstacle detection method as described above.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the obstacle detection method as described above.

According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements an obstacle detection method according to the above.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a schematic flow diagram of an obstacle detection method according to a first embodiment of the present disclosure;

fig. 2 is a schematic flow diagram of an obstacle detection method according to a second embodiment of the present disclosure;

fig. 3 is an example diagram of an obstacle image according to a second embodiment of the present disclosure;

FIG. 4 is an exemplary diagram of a sample detection image according to a second embodiment of the present disclosure;

fig. 5 is a schematic flow chart of an obstacle detection method according to a third embodiment of the present disclosure;

fig. 6 is a schematic structural view of an obstacle detecting device according to a fourth embodiment of the present disclosure;

fig. 7 is a schematic structural view of an obstacle detecting device according to a fifth embodiment of the present disclosure;

fig. 8 is a block diagram of an electronic device for implementing the obstacle detection method of the embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

It can be appreciated that with the rapid development of the automatic driving technology, the automatic driving vehicle is gradually replacing the conventional vehicle as a travel choice for the user. Automatic driving not only minimizes the risk of vehicle travel, but also reduces the burdensome driving tasks for the user. Thus, autonomous driving will be a big trend for future automobile development.

In the related art, a neural network model is usually adopted to directly detect obstacles in an image acquired by a camera so as to determine information such as an area and a type of the obstacles in the image. In this way, obstacles which do not appear in the training process of the neural network model cannot be accurately detected, and the accuracy is poor.

The present disclosure provides a method for detecting an obstacle, which includes obtaining a detection image collected in a target scene and a background image of the target scene, where the background image does not include an obstacle, generating a target image according to the detection image and the background image, where color channels of pixels in the target image include color channels of corresponding pixels in the detection image and color channels of corresponding pixels in the background image, and inputting the target image into a trained detection model to obtain obstacle information in the detection image. The detection model in the embodiment of the disclosure can learn the difference between the image containing the obstacle and the background image not containing the obstacle in the same scene, so that the obstacle which does not appear in the training process of the detection model can be accurately detected, and the obstacle detection accuracy is improved.

An obstacle detection method, an apparatus, an electronic device, a non-transitory computer-readable storage medium, and a computer program product of the embodiments of the present disclosure are described below with reference to the accompanying drawings.

First, the obstacle detection method provided by the present disclosure is described in detail with reference to fig. 1.

Fig. 1 is a schematic flow chart of an obstacle detection method according to a first embodiment of the present disclosure. It should be noted that, in the obstacle detection method provided in the embodiment of the present disclosure, the execution subject is an obstacle detection device. The obstacle detection device can be an electronic device, and can also be configured in the electronic device to improve the accuracy of obstacle detection. The embodiments of the present disclosure are described taking as an example that an obstacle detection device is disposed in an electronic apparatus.

The electronic device may be any stationary or mobile computing device capable of performing data processing, for example, a mobile computing device such as a notebook computer, a smart phone, and a wearable device, or a stationary computing device such as a desktop computer, or a server, or other types of computing devices, and the disclosure is not limited thereto.

As shown in fig. 1, the obstacle detection method may include the steps of:

step 101, acquiring a detection image acquired in a target scene and a background image of the target scene, wherein the background image does not include an obstacle.

The detection image is an image to be subjected to obstacle detection; the background image is a pre-acquired image of the target scene without obstacles.

In an exemplary embodiment, for example, obstacle detection on a parking space or a passage of a parking place is taken as an example, a camera may be fixedly installed in at least one position of the parking place, so that an image which is acquired by the camera in advance and does not include an obstacle may be taken as a background image, and an image which is acquired by the camera and is to be subjected to obstacle detection may be taken as a detection image.

It should be noted that, in the embodiments of the present disclosure, an image captured by one camera is considered to be an image of one scene, that is, images captured by different cameras installed at different positions of a parking place are images of different scenes. Correspondingly, the detection image acquired in the target scene and the background image of the target scene are images acquired by the same camera installed at the same position.

And 102, generating a target image according to the detection image and the background image, wherein the color channels of all pixel points in the target image comprise the color channels of the corresponding pixel points in the detection image and the color channels of the corresponding pixel points in the background image.

In an exemplary embodiment, the detection image and the background image may be channel-merged to generate a target image, and the color channel of each pixel in the target image includes a color channel of a corresponding pixel in the detection image and a color channel of a corresponding pixel in the background image.

For example, assume that the detected image and the background image are 512 × 3 images. 512 × 3 indicates that the image includes 512 × 512 pixels, and each pixel includes three color channels of R (Red ), G (Green ), and B (Blue ).

Then in embodiments of the present disclosure, the color channels of the detected image and the background image may be merged to generate a 512 x 6 target image. The target image includes 512 × 512 pixel points, and each pixel point includes 6 color channels.

Step 103, inputting the target image into the trained detection model to acquire the obstacle information in the detection image.

The detection model may be any neural network model capable of realizing target detection, such as YOLOv3, SSD (Single Shot Multi Box Detector), centret (central network), and the disclosure is not limited thereto.

The obstacle information may include information about the obstacle, such as a location area, a type, and a size of the obstacle.

In an exemplary embodiment, a detection model may be trained in advance, an input of the detection model is a target image generated according to a detection image and a background image acquired in the same scene, and an output of the detection model is obstacle information in the detection image, so that the target image generated according to the detection image acquired in the target scene and the background image of the target scene is input into the trained detection model, and the obstacle information in the detection image may be acquired.

The detection model takes the target image generated according to the detection image and the background image collected in the same scene as input, and the detection model can learn the difference between the image containing the obstacle and the background image not containing the obstacle in the same scene in the training process, so that the obstacle which does not appear in the training process of the detection model can be accurately detected, and the obstacle detection accuracy is improved.

The obstacle detection method provided by the embodiment of the disclosure includes acquiring a detection image acquired in a target scene and a background image of the target scene, wherein the background image does not include an obstacle, generating the target image according to the detection image and the background image, wherein color channels of each pixel point in the target image include color channels of a corresponding pixel point in the detection image and color channels of a corresponding pixel point in the background image, and then inputting the target image into a trained detection model to acquire obstacle information in the detection image. Therefore, the detection model can accurately detect the obstacles which do not appear in the training process of the detection model, and the obstacle detection accuracy is improved.

As can be seen from the above analysis, in the embodiment of the present disclosure, the target image may be generated according to the detection image collected in the target scene and the background image of the target scene, where the color channel of each pixel point in the target image includes the color channel of the corresponding pixel point in the detection image and the color channel of the corresponding pixel point in the background image, and then the target image is input into the trained detection model to obtain the obstacle information in the detection image. In the obstacle detection method provided by the present disclosure, a process of generating a target image and a training process of a detection model are further described below with reference to fig. 2.

Fig. 2 is a schematic flow chart of an obstacle detection method according to a second embodiment of the present disclosure. As shown in fig. 2, the obstacle detection method may include the steps of:

step 201, acquiring a detection image acquired in a target scene and a background image of the target scene, wherein the background image does not include an obstacle.

The specific implementation process and principle of step 201 may refer to the description of the foregoing embodiments, and are not described herein again.

Step 202, aligning each pixel point of the detection image with each pixel point of the background image.

Step 203, merging the color channels of the pixels aligned with each other to generate a target image, where the color channels of the pixels in the target image include the color channel of the corresponding pixel in the detected image and the color channel of the corresponding pixel in the background image.

In an exemplary embodiment, each pixel point of the detected image may be aligned with each pixel point of the background image, and then color channels of the aligned pixel points are combined to generate the target image, so that the color channels of the pixel points in the target image include the color channel of the corresponding pixel point in the detected image and the color channel of the corresponding pixel point in the background image.

For example, assuming that the detected image and the background image are 512 × 3 images, each pixel of the detected image and the background image may be aligned, so that for each pixel of the 512 × 512 pixels aligned with each other, the color channel of the pixel in the detected image and the color channel of the pixel in the background image may be merged, thereby generating a 512 × 6 target image, and the color channel of each pixel in the target image includes 3 color channels of the corresponding pixel in the detected image and 3 color channels of the corresponding pixel in the background image.

Through the process, the channels of the detection image collected under the target scene and the background image of the target scene are combined, so that the color channels of all the pixel points are generated to comprise the color channels of the corresponding pixel points in the detection image and the target image of the color channels of the corresponding pixel points in the background image.

Step 204, inputting the target image into the trained detection model to obtain the obstacle information in the detection image.

In an exemplary embodiment, the trained detection model may include a feature extraction layer and an obstacle detection layer, wherein the feature extraction layer is configured to perform feature extraction on a target image to obtain image features of the target image, and the obstacle detection layer is configured to identify obstacle information in a detection image according to the image features.

It should be noted that the input number of the color channels corresponding to the feature extraction layer of the detection model needs to be matched with the number of the color channels of each pixel point in the target image, so that after the target image is input into the detection model, the feature extraction layer can extract features of the target image.

For example, assuming that the number of color channels of each pixel in the target image is 6 and the detection model is the YOLOv3 model, in the embodiment of the present disclosure, the input number of the color channels corresponding to the feature extraction layer of the YOLOv3 model needs to be set to 6, so that the feature extraction layer of the YOLOv3 model can perform feature extraction on the target image with the color channel of each pixel being 6 to obtain the image features of the target image, and then identify the obstacle information in the detection image according to the image features of the target image by using the obstacle detection layer.

In an exemplary embodiment, before inputting the target image into the trained detection model to obtain the obstacle information in the detection image, the detection model needs to be trained. Specifically, the detection model may be obtained by training in the following method, that is, before step 204, the method may further include:

acquiring training data; the training data comprises a plurality of sample images, each sample image is generated according to a sample detection image collected under a training scene and a sample background image of the training scene, color channels of all pixel points in the sample images comprise color channels of corresponding pixel points in the sample detection images and color channels of corresponding pixel points in the sample background images, the sample detection images contain obstacles, the sample background images do not contain the obstacles, and the training data is labeled by sample obstacle information in the sample detection images;

obtaining an initial detection model;

inputting the sample image into an initial detection model to obtain predicted obstacle information in the sample detection image;

and adjusting the model parameters of the initial detection model according to the difference between the predicted obstacle information in the sample detection image and the sample obstacle information in the sample detection image to obtain a trained detection model.

The initial detection model may be any neural network model capable of realizing target detection, such as YOLOv3, SSD, centrnet, etc., and the disclosure is not limited thereto.

The sample detection image and the sample background image used when the same sample image is generated are images of the same training scene, namely, the sample detection image and the sample background image used when the same sample image is generated are acquired by the same camera installed at the same position. Also, the training scenario in the embodiments of the present disclosure may include a plurality of scenarios. Under the same training scene, a plurality of sample detection images and sample background images can be collected to generate a plurality of sample images corresponding to the same training scene.

For example, the training data may include 100 sample images. The 1 st to 30 th sample images can be generated according to a sample detection image collected by a camera A arranged at the position a and a sample background image; the 31 st to 60 th sample images can be generated according to a sample detection image collected by the camera B arranged at the position B and a sample background image: the 61 st to 100 th sample images may be generated according to a sample detection image collected by the camera C installed at the position C and a sample background image.

The process of generating the sample image according to the sample detection image acquired in the training scene and the sample background image of the training scene may refer to the process of generating the target image according to the detection image acquired in the target scene and the background image of the target scene, and is not described here again.

In an exemplary embodiment, when the initial detection model is trained, for example, deep learning can be performed, which performs better on a large data set than other machine learning methods.

When the initial detection model is trained in a deep learning manner, one or more sample images in training data are used as input, the initial detection model is input, the predicted obstacle information in the sample detection image used when the sample image is generated is obtained, the difference between the predicted obstacle information in the sample detection image and the sample obstacle information in the sample detection image is obtained by combining the sample obstacle information in the sample detection image, and the model parameters of the initial detection model are adjusted according to the difference to obtain the adjusted detection model. And then inputting another or more sample images in the training data as input, inputting the adjusted detection model, obtaining the predicted obstacle information in the sample detection image utilized when the sample image is generated, combining the sample obstacle information in the sample detection image to obtain the difference between the predicted obstacle information in the sample detection image and the sample obstacle information in the sample detection image, and adjusting the model parameters of the adjusted detection model according to the difference to obtain the further adjusted detection model. Therefore, iterative training is carried out on the initial detection model by continuously adjusting the model parameters of the initial detection model until the accuracy of the predicted obstacle information output by the detection model meets a preset threshold value, and the training is finished to obtain the trained detection model.

After the training of the detection model is completed, the detection model can be adopted to process the target image generated according to the detection image acquired under the target scene and the background image of the target scene so as to acquire the obstacle information in the detection image.

It should be noted that the training scene in the training data according to the embodiment of the present disclosure may be different from the target scene when the detection model is used to detect the obstacle information in the detection image collected in the target scene after the detection model is trained. Namely, the trained detection image can be used for detecting obstacles in the detection image collected in the scene which does not appear in the training process.

When the detection model is trained, the sample image input by the detection model is generated according to the sample detection image collected under the training scene and the sample background image of the training scene, the color channel of each pixel point in the sample image comprises the color channel of the corresponding pixel point in the sample detection image and the color channel of the corresponding pixel point in the sample background image, the sample detection image contains an obstacle, the sample background image does not contain the obstacle, the detection model can learn the difference between the sample detection image containing the obstacle and the sample background image not containing the obstacle, so that when the trained detection model is used for detecting the obstacle in the detection image collected under the target scene, the detection model can identify the difference between the detection image collected under the target scene and the background image of the target scene to accurately obtain the obstacle information in the detection image, and even for the obstacles which do not appear in the training process, the information of the obstacles can be accurately acquired, and the accuracy of obstacle detection is improved.

It is understood that, in a large database such as COCO (Common Objects in Context), a large number of images including obstacles and images not including obstacles collected by a camera are collected.

It can be understood that the types and the number of obstacles contained in an image normally acquired by the camera in a certain scene may be small, so that the image normally acquired by the camera is used as a sample detection image, and after the sample image is generated by combining the sample background image, when a detection model is trained, the detection model may not well learn the difference between the sample detection image and the sample background image.

In the embodiment of the disclosure, the image with the obstacle image added can be added to the image normally acquired by the camera, and the image with the obstacle image added can be used as the sample detection image, so that when the detection model utilizes the sample detection image and the sample background image of the corresponding scene to train the detection model, the detection model can better learn the difference between the sample detection image and the sample background image, and the accuracy of the detection model in detecting the obstacle is improved.

The obstacle image is an image including an obstacle.

In an exemplary embodiment, in the training process of the detection model, a barrier image may be randomly selected from a library of previously collected barrier images, and the randomly selected barrier image may be combined with an image acquired by a camera, so as to obtain a sample detection image. That is, the sample detection image in the embodiment of the present disclosure may be obtained by combining an image acquired by the camera with a randomly selected obstacle image.

It should be noted that the number of the obstacle images that need to be combined with the image collected by the camera and are randomly selected may be set according to needs, and the embodiment of the present disclosure does not limit this.

Referring to fig. 3, a plurality of obstacle images, such as a traffic cone image 301, a bucket image 302, a cart image 303, a carton image, etc., may be previously stored in the obstacle image library, so that when training of the detection model is performed, the obstacle image may be randomly selected from the obstacle image library, and the randomly selected obstacle image may be combined with the image captured by the camera, thereby obtaining a sample detection image.

Referring to fig. 4, assuming that a traffic cone image 401 and a carton image 402 are randomly selected from the obstacle image library, the traffic cone image 401 and the carton image 402 may be merged with the image captured by the camera, so as to obtain a sample detection image as shown in fig. 4.

The obstacle detection method of the embodiment of the disclosure includes acquiring a detection image acquired in a target scene and a background image of the target scene, wherein the background image does not contain obstacles, aligning each pixel point of the detection image with each pixel point of the background image, merging color channels of the aligned pixel points to generate the target image, wherein the color channels of the pixel points in the target image include color channels of corresponding pixel points in the detection image and color channels of corresponding pixel points in the background image, and inputting the target image into a trained detection model to acquire obstacle information in the detection image. Therefore, the channel combination of the detection image acquired in the target scene and the background image of the target scene is realized, the target image is generated, and then the target image is used for acquiring the obstacle information in the detection image.

The following takes a scenario in which autonomous parking is implemented in a parking lot as an example, and further describes the obstacle detection method in the embodiment of the present disclosure.

Fig. 5 is a schematic flow chart of an obstacle detection method according to a third embodiment of the present disclosure. As shown in fig. 5, the obstacle detection method may include the steps of:

step 501, acquiring a detection image acquired in a target scene and a background image of the target scene, wherein the background image does not contain an obstacle, and the detection image and the background image are acquired by a camera installed at a fixed position in a parking lot.

Step 502, generating a target image according to the detection image and the background image, wherein the color channels of the pixels in the target image comprise the color channels of the corresponding pixels in the detection image and the color channels of the corresponding pixels in the background image.

In an exemplary embodiment, a camera may be provided at a fixed position in a parking lot and a background image T1 not including an obstacle may be captured by the camera in advance, and the camera may capture a parking lot image T2 in real time or at preset time intervals and transmit the background image T1 and the parking lot image T2 to an obstacle detecting device. Wherein the obstacle detecting device may be configured in a smart vehicle.

After the obstacle detection device obtains the background image T1 and the parking lot image T2, the parking lot image T2 may be used as a detection image, each pixel point of the detection image is aligned with each pixel point of the background image T1, and color channels of the aligned pixel points are merged to generate a target image, where the color channels of the pixel points in the target image include a color channel of a corresponding pixel point in the detection image and a color channel of a corresponding pixel point in the background image T1.

Step 503, inputting the target image into the trained detection model to obtain the obstacle information in the detection image.

In an exemplary embodiment, after the obstacle detection apparatus generates the target image, the target image may be input to a trained detection model to acquire obstacle information in the detection image.

And step 504, determining the empty parking spaces in the parking lot and feasible routes according to the obstacle information.

And 505, guiding the intelligent vehicle to park autonomously according to the empty parking space and the feasible route.

The obstacle information may include information about an area, a type, and a size of the obstacle.

In an exemplary embodiment, after the obstacle detection device obtains the obstacle information, it may determine whether there is an obstacle in the parking lot, and the type, size, and position of the obstacle when there is an obstacle, according to the obstacle information, and further determine which parking space is an empty parking space, and plan a feasible route that runs from the location of the vehicle to the empty parking space and can avoid the location of the obstacle, so as to guide the smart vehicle to autonomously park according to the empty parking space and the feasible route.

The obstacle detection method of the embodiment of the disclosure includes the steps of acquiring a detection image acquired in a target scene and a background image of the target scene, wherein the background image does not contain obstacles, the detection image and the background image are acquired by a camera installed at a fixed position in a parking lot, generating the target image according to the detection image and the background image, color channels of all pixel points in the target image comprise color channels of corresponding pixel points in the detection image and color channels of corresponding pixel points in the background image, inputting the target image into a trained detection model to acquire obstacle information in the detection image, determining an empty parking space and a feasible route in the parking lot according to the obstacle information, and guiding an intelligent vehicle to conduct autonomous parking according to the empty parking space and the feasible route. Therefore, the obstacles in the parking lot can be accurately detected, the empty parking space can be accurately determined according to the information of the obstacles, the feasible route of the obstacles can be avoided, and the intelligent vehicle can be guided to realize autonomous parking.

The obstacle detection device provided by the present disclosure is explained below with reference to fig. 6.

Fig. 6 is a schematic structural diagram of an obstacle detecting device according to a fourth embodiment of the present disclosure.

As shown in fig. 6, the present disclosure provides an obstacle detection apparatus 600 including: a first obtaining module 601, a generating module 602, and a first processing module 603.

The first obtaining module 601 is configured to obtain a detection image collected in a target scene and a background image of the target scene, where the background image does not include an obstacle;

a generating module 602, configured to generate a target image according to the detected image and the background image, where a color channel of each pixel in the target image includes a color channel of a corresponding pixel in the detected image and a color channel of a corresponding pixel in the background image;

the first processing module 603 is configured to input the target image into the trained detection model to obtain obstacle information in the detection image.

It should be noted that the obstacle detection device provided in this embodiment may perform the obstacle detection method of the foregoing embodiment. The obstacle detection device may be an electronic device, or may be configured in the electronic device, so as to improve the accuracy of obstacle detection.

It should be noted that the foregoing descriptions of the embodiments of the obstacle detection method are also applicable to the obstacle detection apparatus provided in the present disclosure, and are not repeated herein.

The obstacle detection device provided by the embodiment of the present disclosure acquires a detection image acquired in a target scene and a background image of the target scene, wherein the background image does not include an obstacle, generates the target image according to the detection image and the background image, and a color channel of each pixel point in the target image includes a color channel of a corresponding pixel point in the detection image and a color channel of a corresponding pixel point in the background image, and then inputs the target image into a trained detection model to acquire obstacle information in the detection image. Therefore, the detection model can accurately detect the obstacles which do not appear in the training process of the detection model, and the obstacle detection accuracy is improved.

The obstacle detection device provided by the present disclosure is explained below with reference to fig. 7.

Fig. 7 is a schematic structural diagram of an obstacle detecting device according to a sixth embodiment of the present disclosure.

As shown in fig. 7, the obstacle detection apparatus 700 may specifically include: a first obtaining module 701, a generating module 702 and a first processing module 703. The first obtaining module 701, the generating module 702, and the first processing module 703 in fig. 7 have the same functions and structures as the first obtaining module 601, the generating module 602, and the first processing module 603 in fig. 6.

In an exemplary embodiment, the trained detection model includes a feature extraction layer and an obstacle detection layer;

the input number of color channels corresponding to the feature extraction layer is matched with the number of color channels of each pixel point in the target image, and the feature extraction layer is used for extracting features of the target image to obtain image features of the target image;

and the obstacle detection layer is used for identifying and detecting obstacle information in the image according to the image characteristics.

In an exemplary embodiment, as shown in fig. 7, the obstacle detection apparatus 700 may further include:

a second obtaining module 704, configured to obtain training data; the training data comprises a plurality of sample images, each sample image is generated according to a sample detection image collected under a training scene and a sample background image of the training scene, color channels of all pixel points in the sample images comprise color channels of corresponding pixel points in the sample detection images and color channels of corresponding pixel points in the sample background images, the sample detection images contain obstacles, the sample background images do not contain the obstacles, and the training data is labeled by sample obstacle information in the sample detection images;

a third obtaining module 705, configured to obtain an initial detection model;

a second processing module 706, configured to input the sample image into the initial detection model to obtain predicted obstacle information in the sample detection image;

and a training module 707, configured to adjust a model parameter of the initial detection model according to a difference between the predicted obstacle information in the sample detection image and the sample obstacle information in the sample detection image, so as to obtain a trained detection model.

In an exemplary embodiment, the sample detection image is a combination of the image captured by the camera and the image of the randomly selected obstacle.

In an exemplary embodiment, the generation module 702 includes:

the alignment unit is used for aligning each pixel point of the detection image with each pixel point of the background image;

and the merging unit is used for merging the color channels of the pixels which are aligned with each other to generate a target image. In an exemplary embodiment, wherein the detection image and the background image are captured by a camera mounted at a fixed position in the parking lot;

accordingly, as shown in fig. 7, the obstacle detection model 700 may further include:

a determination module 708 for determining empty parking spaces and feasible routes in the parking lot according to the obstacle information;

and the guiding module 709 is used for guiding the intelligent vehicle to park autonomously according to the empty parking space and the feasible route.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 8 illustrates a schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 8, the apparatus 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM)802 or a computer program loaded from a storage unit 808 into a Random Access Memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The calculation unit 801, the ROM 802, and the RAM 803 are connected to each other by a bus 804. An input/output (I/O) interface 805 is also connected to bus 804.

A number of components in the device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, a mouse, or the like; an output unit 807 such as various types of displays, speakers, and the like; a storage unit 808, such as a magnetic disk, optical disk, or the like; and a communication unit 809 such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

Computing unit 801 may be a variety of general and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 801 executes the respective methods and processes described above, such as the obstacle detection method. For example, in some embodiments, the obstacle detection method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and/or installed onto device 800 via ROM 802 and/or communications unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the obstacle detection method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the obstacle detection method by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.

It should be noted that the embodiment of the present disclosure relates to the field of artificial intelligence, in particular to computer vision and deep learning technologies, and is particularly applicable to intelligent transportation and smart city scenes.

The artificial intelligence is a subject for researching and enabling a computer to simulate certain thinking process and intelligent behaviors (such as learning, reasoning, thinking, planning and the like) of a human, and has a hardware level technology and a software level technology. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing, and the like; the artificial intelligence software technology mainly comprises computer vision, a voice recognition technology, a natural language processing technology, machine learning/deep learning, a big data processing technology, a knowledge map technology and the like.

Computer vision is a science for researching how to make a machine look, and further, it refers to that a camera and a computer are used to replace human eyes to perform machine vision of identifying, tracking and measuring a target, and further to perform graphic processing, so that the computer processing becomes an image more suitable for human eyes to observe or transmitted to an instrument to detect.

Deep learning, a new research direction in the field of machine learning, is introduced into machine learning to make it closer to the original target, artificial intelligence. Deep learning is the intrinsic law and expression level of the learning sample data, and the information obtained in the learning process is very helpful for the interpretation of data such as characters, images and sounds. The final aim of the method is to enable the machine to have the analysis and learning capability like a human, and to recognize data such as characters, images and sounds.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. An obstacle detection method comprising:

acquiring a detection image acquired in a target scene and a background image of the target scene, wherein the background image does not contain an obstacle;

generating a target image according to the detection image and the background image, wherein the color channels of all pixel points in the target image comprise the color channels of the corresponding pixel points in the detection image and the color channels of the corresponding pixel points in the background image;

and inputting the target image into a trained detection model to acquire the obstacle information in the detection image.

2. The method of claim 1, wherein the trained detection model comprises a feature extraction layer and an obstacle detection layer;

the input number of color channels corresponding to the feature extraction layer is matched with the number of color channels of each pixel point in the target image, and the feature extraction layer is used for performing feature extraction on the target image to obtain image features of the target image;

and the obstacle detection layer is used for identifying obstacle information in the detection image according to the image characteristics.

3. The method according to claim 1 or 2, wherein before inputting the target image into the trained detection model to obtain the obstacle information in the detection image, the method further comprises:

obtaining an initial detection model;

inputting the sample image into the initial detection model to obtain predicted obstacle information in the sample detection image;

and adjusting the model parameters of the initial detection model according to the difference between the predicted obstacle information in the sample detection image and the sample obstacle information in the sample detection image to obtain the trained detection model.

4. The method of claim 3, wherein the sample detection image is a combination of a camera acquired image and a randomly selected image of an obstacle.

5. The method of claim 1 or 2, wherein the generating a target image from the detection image and the background image comprises:

aligning each pixel point of the detection image with each pixel point of the background image;

and combining the color channels of the pixel points which are aligned with each other to generate a target image.

6. The method of claim 1 or 2, wherein the detection image and the background image are acquired by a camera mounted at a fixed position in a parking lot;

after the acquiring of the obstacle information in the detection image, the method further includes:

determining empty parking spaces and feasible routes in the parking lot according to the obstacle information;

and guiding the intelligent vehicle to park autonomously according to the empty parking space and the feasible route.

7. An obstacle detection device comprising:

the device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a detection image acquired in a target scene and a background image of the target scene, and the background image does not contain obstacles;

a generating module, configured to generate a target image according to the detection image and the background image, where a color channel of each pixel in the target image includes a color channel of a corresponding pixel in the detection image and a color channel of a corresponding pixel in the background image;

and the first processing module is used for inputting the target image into the trained detection model so as to acquire the obstacle information in the detection image.

8. The apparatus of claim 7, wherein the trained detection model comprises a feature extraction layer and an obstacle detection layer;

9. The apparatus of claim 7 or 8, further comprising:

the second acquisition module is used for acquiring training data; the training data comprises a plurality of sample images, each sample image is generated according to a sample detection image collected under a training scene and a sample background image of the training scene, color channels of all pixel points in the sample images comprise color channels of corresponding pixel points in the sample detection images and color channels of corresponding pixel points in the sample background images, the sample detection images contain obstacles, the sample background images do not contain the obstacles, and the training data is labeled by sample obstacle information in the sample detection images;

the third acquisition module is used for acquiring the initial detection model;

the second processing module is used for inputting the sample image into the initial detection model so as to obtain predicted obstacle information in the sample detection image;

and the training module is used for adjusting the model parameters of the initial detection model according to the difference between the predicted obstacle information in the sample detection image and the sample obstacle information in the sample detection image so as to obtain the trained detection model.

10. The apparatus of claim 9, wherein the sample detection image is a combination of a camera acquired image and a randomly selected image of an obstacle.

11. The apparatus of claim 7 or 8, wherein the generating means comprises:

and the merging unit is used for merging the color channels of the pixels which are aligned with each other to generate a target image.

12. The apparatus of claim 7 or 8, wherein the detection image and the background image are acquired by a camera mounted at a fixed position in a parking lot;

the device, still include:

the determining module is used for determining empty parking spaces and feasible routes in the parking lot according to the obstacle information;

and the guiding module is used for guiding the intelligent vehicle to park autonomously according to the empty parking space and the feasible route.

13. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-6.

15. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-6.