CN115713745A - Obstacle detection method, electronic device, and storage medium - Google Patents

Abstract

Embodiments of the present application provide an obstacle detection method, electronic equipment, and a storage medium. Wherein, the obstacle detection method includes: acquiring a target image, and the target image is an image outside the vehicle; in response to acquiring the target image, identifying whether there is an obstacle in the target image; in response to identifying the target image Obstacles exist in the target image, acquiring position information of the obstacles in the target image, and identifying whether the obstacles are located in the dangerous area based on the position information. In the embodiment of the present application, it is possible to identify whether there is an obstacle outside the vehicle based on the image outside the vehicle, and further identify whether the obstacle is located in a dangerous area, so as to determine whether there is a potential safety problem at present, so that the potential safety problem can be addressed accordingly. To avoid accidents and improve the safety of vehicles.

Description

Obstacle detection method, electronic device and storage medium

technical field

The present application relates to the technical field of vehicle control, in particular to an obstacle detection method, electronic equipment and a storage medium.

Background technique

With the continuous improvement of people's living standards, cars are more and more favored by people and become an important means of transportation for people to go out. While cars bring convenience to people's travel, there are also some potential safety problems. For example, when a car is running, there may be other obstacles outside the car. If these obstacles are located in a dangerous area, they may cause traffic accidents such as collisions.

Therefore, how to detect obstacles outside the vehicle so as to identify whether the obstacle is located in a dangerous area is a technical problem that needs to be solved urgently.

Contents of the invention

In view of the above problems, the embodiment of the present application proposes an obstacle detection method, an electronic device and a storage medium, which are used to detect obstacles outside the vehicle, so as to identify whether the obstacle is located in a dangerous area, and then help users discover road risks in time , to ensure the driving safety of users.

According to an aspect of an embodiment of the present application, there is provided an obstacle detection method, the method comprising: acquiring a target image, the target image being an image outside the vehicle; in response to acquiring the target image, identifying the Whether there is an obstacle in the target image; in response to recognizing that there is an obstacle in the target image, obtaining position information of the obstacle in the target image, and identifying whether the obstacle is located in a dangerous area based on the position information .

According to another aspect of the embodiments of the present application, an electronic device is provided, including: one or more processors; and one or more computer-readable storage media storing instructions thereon; when the instructions are executed by the When the one or more processors are executed, the processors are made to execute the obstacle detection method as described in any one of the above items.

According to yet another aspect of the embodiments of the present application, there is provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the processor is made to perform the The obstacle detection method described above.

In the embodiment of the present application, the target image is acquired, and the target image is an image outside the vehicle; in response to acquiring the target image, identifying whether there is an obstacle in the target image; in response to identifying that there is an obstacle in the target image Obstacles, acquiring position information of the obstacles in the target image, and identifying whether the obstacles are located in a dangerous area based on the position information. It can be seen that in the embodiment of the present application, it is possible to identify whether there is an obstacle outside the vehicle based on the image outside the vehicle, and further identify whether the obstacle is located in a dangerous area, so as to determine whether there is a potential safety problem at present, so as to target potential Safety issues should be dealt with accordingly to avoid accidents.

Description of drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments of the present application. Obviously, the accompanying drawings in the following description are only some of the accompanying drawings of the present application , for those skilled in the art, other drawings can also be obtained according to these drawings on the premise of not paying creative work.

FIG. 1 is a schematic diagram of device interaction according to an embodiment of the present application.

Fig. 2 is a schematic diagram of another device interaction according to an embodiment of the present application.

FIG. 3 is a flowchart of steps of an obstacle detection method according to an embodiment of the present application.

FIG. 4 is a flow chart of steps of another obstacle detection method according to an embodiment of the present application.

FIG. 5 is a schematic diagram of an obstacle comparison according to an embodiment of the present application.

FIG. 6 is a schematic diagram of another obstacle comparison according to the embodiment of the present application.

Fig. 7 is a schematic diagram of a vehicle left image and a vehicle right image according to an embodiment of the present application.

FIG. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the application. Apparently, the described embodiments are only part of the embodiments of the application, rather than the entirety of the application. Example. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

In an optional implementation, a video capture device and a vehicle-machine terminal can be installed on the vehicle. Referring to FIG. 1 , it shows a schematic diagram of device interaction according to an embodiment of the present application. As shown in Figure 1, two-way communication can be performed between the video capture device and the vehicle terminal. Wherein, the video capture device can be various cameras, and one or more video capture devices can be installed outside the vehicle and/or inside the vehicle. The video capture device is used for real-time or timing capture of video outside the vehicle, and the collected The video is transmitted to the vehicle terminal. The vehicle-machine terminal is a vehicle-mounted intelligent terminal installed on the vehicle. The vehicle-machine terminal can receive the video outside the vehicle transmitted by the video acquisition device, and execute the obstacle detection method in the embodiment of the present application.

In another optional implementation manner, a video capture device and a vehicle-machine terminal may be installed on the vehicle, and the vehicle-machine terminal may also have a corresponding cloud server. Referring to FIG. 2 , it shows another schematic diagram of device interaction according to the embodiment of the present application. As shown in Figure 2, two-way communication can be performed between the video capture device and the vehicle terminal, and two-way communication can be performed between the vehicle terminal and the cloud server. Among them, the video capture device can be installed outside the vehicle and/or inside the vehicle, and one or more video capture devices can be installed. The video capture device is used to collect the video outside the vehicle in real time or at regular intervals, and transmit the collected video outside the vehicle to car terminal. The vehicle-machine terminal is a vehicle-mounted intelligent terminal installed on the vehicle. The vehicle-machine terminal can receive the video outside the vehicle transmitted by the video acquisition device, and transmit the received video outside the vehicle to the cloud server. The cloud server can receive the video outside the vehicle transmitted by the vehicle-machine terminal, and execute the obstacle detection method in the embodiment of the present application.

Next, the obstacle detection method will be introduced in detail through the following embodiments.

Referring to FIG. 3 , it shows a flowchart of steps of an obstacle detection method according to an embodiment of the present application. The obstacle detection method shown in FIG. 3 can be executed by a vehicle-machine terminal or a cloud server.

As shown in Figure 3, the obstacle detection method may include the following steps:

Step 301, acquiring a target image.

The target image is an image of the exterior of the vehicle.

The target image may be an image extracted from the received video outside the vehicle. Optionally, an image can be extracted from the video outside the vehicle according to a preset fixed time interval as the target image; or, according to the driving speed of the vehicle, a variable time interval related to the driving speed of the vehicle can be used to extract the image from the video outside the vehicle The extracted image is used as the target image, which is not limited in this embodiment.

Step 302, in response to acquiring the target image, identify whether there is an obstacle in the target image.

After acquiring a frame of the target image, image recognition is performed on the target image in response to the acquisition of the target image, so as to identify whether there is an obstacle in the image. Optionally, obstacles may include, but are not limited to: various vehicles, pedestrians, other objects, roadblocks, and so on.

Step 303, in response to identifying an obstacle in the target image, acquiring position information of the obstacle in the target image, and identifying whether the obstacle is located in a dangerous area based on the position information.

If it is recognized that there is an obstacle in the target image, in response to recognizing that there is an obstacle in the target image, the position information of the obstacle in the target image is obtained, and based on the position information, it is identified whether the obstacle is located in the dangerous area. Dangerous area means that when the obstacle is located in this area, there will be potential safety problems for the vehicle, and the driver needs to pay special attention.

If it is recognized that there is no obstacle in the target image, continue to identify the target image acquired subsequently.

In the embodiment of the present application, it is possible to identify whether there is an obstacle outside the vehicle based on the image outside the vehicle, and further identify whether the obstacle is located in a dangerous area, so as to determine whether there is a potential safety problem at present, so that the potential safety problem can be addressed accordingly. treatment to avoid accidents.

Referring to FIG. 4 , it shows a flowchart of steps of another obstacle detection method according to an embodiment of the present application. The obstacle detection method shown in FIG. 4 can be executed by a vehicle-machine terminal or a cloud server.

As shown in Figure 4, the obstacle detection method may include the following steps:

Step 401, acquiring a target image.

In this embodiment, the driving speed of the vehicle can be obtained in real time or at regular intervals, and different extraction time intervals are selected to extract the target image according to the different driving speeds of the vehicle.

Therefore, the process of acquiring the target image may include the following steps A1-A3:

Step A1, acquire the video outside the vehicle and the driving speed of the vehicle.

In implementation, a speed sensor can be installed on the vehicle, and two-way communication can be performed between the speed sensor and the vehicle-machine terminal. The speed sensor can obtain the driving speed of the vehicle, and transmit the driving speed of the vehicle to the vehicle-machine terminal. If the obstacle detection is performed by the cloud server, the vehicle-machine terminal can further transmit the driving speed of the vehicle to the cloud server.

Optionally, the speed sensor in this embodiment may include but not limited to: a photoelectric vehicle speed sensor, a magnetoelectric vehicle speed sensor, a Hall-type vehicle speed sensor, a wheel speed sensor, an engine speed sensor, and so on.

Step A2, determining an extraction time interval based on the driving speed.

Considering that the faster the driving speed of the vehicle, the greater the possibility of danger, so the target image can be extracted more quickly for obstacle detection, so as to increase the frequency of obstacle detection, so as to determine whether the obstacle is more timely Located in a hazardous area. Conversely, when the vehicle is traveling at a slower speed, the possibility of danger is less, so the target image can be extracted relatively slowly for obstacle detection, so as to reduce the frequency of obstacle detection, thereby saving resources and avoiding waste. Therefore, in this embodiment, it may be set that the extraction time interval is negatively correlated with the driving speed of the vehicle. That is, the faster the driving speed of the vehicle, the shorter the extraction time interval; the slower the driving speed of the vehicle, the longer the extraction time interval.

Optionally, the corresponding relationship between the driving speed of the vehicle and the extraction time interval may be preset, and based on the acquired driving speed of the vehicle, the extraction time interval corresponding to the driving speed of the vehicle is queried from the corresponding relationship.

Optionally, in the corresponding relationship, the driving speed of the vehicle may be in the form of a speed interval, that is, one speed interval may correspond to one extraction time interval. Therefore, after the vehicle’s driving speed is obtained, the corresponding relationship can be queried based on the vehicle’s driving speed to obtain the speed interval in which the vehicle’s driving speed is located, and the extraction time interval corresponding to the speed interval can be used as the vehicle’s driving speed correspondence extraction time interval.

For the above corresponding relationship, any applicable value may be set according to actual experience, which is not limited in this embodiment. For example, when the driving speed of the vehicle is 90km/h to 120km/h (kilometers per hour), the corresponding extraction time interval is 1 second; when the driving speed of the vehicle is 60km/h to 89km/h, the corresponding extraction time interval is 1.5 seconds; when the driving speed of the vehicle is 30km/h-59km/h, the corresponding extraction time interval is 2 seconds, and so on.

Step A3, extracting an image from the video as the target image according to the extraction time interval.

After the video outside the vehicle is acquired, an image is extracted from the video outside the vehicle as the target image according to the extraction time interval determined based on the driving speed of the vehicle.

For example, if the driving speed of the vehicle is 100km/h, the corresponding extraction time interval is determined to be 1 second. Therefore, a frame of image is extracted from the video outside the vehicle every 1 second, and the extracted frame of image is used as the target image.

Step 402, in response to acquiring the target image, identify whether there is an obstacle in the target image.

Image recognition technology is used to perform target detection on the acquired target image, so as to detect whether there is an obstacle in the target image, and obtain the position information of the obstacle in the target image if there is an obstacle.

Optionally, the manner of performing target detection on the target image may include but not limited to: HOG (Histogram of Oriented Gradients, histogram of oriented gradients)+SVM (Support Vector Machine, support vector machine) mode, DPM (Deformable Part based Model, which can Deformable part model) method, R-CNN (Region Convolutional Neural Network, regional convolutional neural network) method, SPPNet (SpatialPyramid Pooling Networks, spatial pyramid pooling network) method, Fast RCNN method, Faster RCNN method, etc.

For example, in the case of using the HOG+SVM method, for a frame of target image, first extract the HOG feature vector of the target image, and then input the HOG feature vector into the SVM for classification detection. The SVM can identify which pixels belong to the obstacle category and which The pixel does not belong to the obstacle category, so as to detect whether there is an obstacle in the target image of the frame, and obtain the position information of the obstacle in the target image.

For another example, in the case of using the R-CNN method, for a frame of target image, first use the selective search (selective search) method to determine multiple (about 1000-2000) candidate frames in the target image; The image blocks in each candidate frame are scaled to the same size, and input to CNN (Convolutional Neural Network, Convolutional Neural Network) for feature extraction; then, for the features extracted in the candidate frame, use a classifier to determine whether they belong to a specific class (whether it belongs to the obstacle category); then, for the candidate frame belonging to a certain feature, use the regressor to further adjust its position, so as to detect whether there is an obstacle in the target image of the frame, and obtain the position information of the obstacle in the target image .

Wherein, there may be one or more obstacles in the target image.

Step 403, in response to identifying an obstacle in the target image, acquiring position information of the obstacle in the target image, and identifying whether the obstacle is located in a dangerous area based on the position information.

If it is recognized that there is an obstacle in the target image, in response to the presence of an obstacle in the target image, obtain the position information of the obstacle detected in the target image during the target detection process in step 402 above, and based on the position information of the obstacle Identify whether the obstacle is in the danger zone. The details will be introduced below. If it is recognized that there is no obstacle in the target image, continue to identify the target image acquired subsequently.

Step 404, in response to identifying that the obstacle is located in a dangerous area, triggering a danger reminder message.

If it is recognized that the obstacle is located in the dangerous area, then in response to the recognition that the obstacle is located in the dangerous area, a danger reminder message may be further triggered, so as to remind the user (such as the driver) of the current dangerous state and need to pay attention to safety. If it is recognized that the obstacle is located in a non-dangerous area, continue to recognize the target image acquired subsequently.

Optionally, the form of danger reminder information may include but not limited to: voice reminder (such as playing reminder voice, etc.), text reminder (such as pop-up window display reminder, etc.), sound and light reminder (such as sound and light reminder, etc.), etc. .

It should be noted that if the vehicle-machine terminal executes the obstacle detection method of this embodiment, the danger reminder information is triggered by the vehicle-machine terminal itself; if the cloud server executes the obstacle detection method of this embodiment, the cloud server triggers Danger reminder information, and send the danger reminder information to the car-machine terminal, and then the car-machine terminal sends the corresponding danger reminder information to the user.

In one application scenario, when a user (that is, a driver) is driving a vehicle, if the user's vehicle is too close to the vehicle in front and/or the vehicle behind, it may easily cause a traffic accident. In this case, the method of this embodiment can be used for obstacle detection, so as to identify whether the vehicle in front and/or the vehicle behind is located in a dangerous area (that is, whether it is too close to the vehicle), and when the vehicle in front and/or the vehicle behind is identified Or when the vehicle behind is located in a dangerous area, a danger reminder message is triggered, so as to remind the user that the vehicle ahead and/or the vehicle behind are too close, and pay attention to maintaining a distance between vehicles.

For this scenario, in an optional implementation manner, the video capture device may be installed in front of the vehicle and/or behind the vehicle, and correspondingly, the target image may include an image in front of the vehicle and/or an image behind the vehicle.

Based on the position information of the obstacle in the target image, the process of identifying whether the obstacle is located in the dangerous area may include the following steps B1-B3:

Step B1, calculating a first proportion of the width of the obstacle in the width of the target image and a second proportion of the height of the obstacle in the height of the target image based on the position information.

Generally, the farther the distance between the obstacle and the lens, the smaller the proportion of the width and/or height of the obstacle in the target image; the closer the distance between the obstacle and the lens, the smaller the width and/or height of the obstacle. Or the greater the proportion of the height in the target image. For example, in the target images taken at distances of 5 meters, 10 meters, 15 meters, 20 meters, 30 meters, and 40 meters from the lens, the proportions of the width and/or height of obstacles are different. Compared with the obstacles photographed when the distance from the lens is 10 meters, the proportion of obstacles photographed is larger. Therefore, the proportion of the width and/or height of the obstacle in the target image can reflect the distance between the obstacle and the vehicle. Therefore, whether the obstacle is located in the dangerous area can be identified based on the proportion of the width and/or height of the obstacle in the target image.

The position information of the obstacle in the target image recognized by the image recognition technology may include but not limited to: the coordinates of each pixel corresponding to the obstacle, and the coordinates take the lower left corner of the target image as the origin.

Based on the coordinates of the leftmost pixel corresponding to the obstacle (that is, the pixel with the smallest abscissa corresponding to the obstacle) and the coordinates of the rightmost pixel (that is, the pixel with the largest abscissa corresponding to the obstacle), the obstacle can be calculated width. Specifically, the difference between the coordinates of the rightmost pixel corresponding to the obstacle and the coordinates of the leftmost pixel corresponding to the obstacle is taken as the width of the obstacle.

Obstacles can be calculated based on the coordinates of the lowest pixel corresponding to the obstacle (that is, the pixel with the smallest ordinate corresponding to the obstacle) and the coordinates of the uppermost pixel (that is, the pixel with the largest ordinate corresponding to the obstacle). the height of. Specifically, the difference between the coordinates of the uppermost pixel corresponding to the obstacle and the coordinates of the lowermost pixel corresponding to the obstacle is taken as the height of the obstacle.

The width of the target image is obtained, and a first ratio between the width of the obstacle and the width of the target image is calculated, and the first ratio is used as a first ratio of the width of the obstacle to the width of the target image. The height of the target image is acquired, and a second ratio between the height of the obstacle and the height of the target image is calculated, and the second ratio is used as a second ratio of the height of the obstacle to the height of the target image.

Step B2, identifying the type of the obstacle, and obtaining a preset percentage range of the dangerous area corresponding to the type of the obstacle.

When the distance between the obstacle and the lens is the same, the type of the obstacle is different, and the proportion of the width and/or height of the obstacle in the target image is different. For example, the larger the obstacle, the greater the proportion of the width and/or height of the obstacle in the target image; the smaller the obstacle, the smaller the proportion of the width and/or height of the obstacle in the target image. Therefore, for different types of obstacles, the proportion range of the dangerous area corresponding to each type of obstacle can be set in advance.

Optionally, the types of obstacles may include but are not limited to: small obstacles (such as cars, electric vehicles, etc.), medium obstacles (such as small trucks, passenger cars, small buses, etc.), large obstacles (such as large trucks, coaches, large buses, etc.), etc.

In implementation, the distance range between the obstacle corresponding to the dangerous area and the vehicle may be preset according to actual experience. For example, the distance range corresponding to the dangerous area may be set to be 5 meters to 25 meters, and so on. Then, for each type of obstacle, in the image captured within the corresponding distance range of the dangerous area, the type of obstacle corresponds to the proportion range of the width and height of the obstacle in the image, and the proportion range The proportion range of the dangerous area corresponding to the type of this obstacle.

After identifying the obstacle in the target image, you can continue to identify the type of the obstacle, and obtain the proportion of the dangerous area corresponding to the type of obstacle from the preset range of dangerous area proportions corresponding to each obstacle type scope.

Optionally, the aspect ratio range corresponding to each obstacle type is set in advance according to actual experience. The process of identifying the type of obstacle may include the following steps: matching the aspect ratio of the obstacle with the preset aspect ratio range corresponding to each obstacle type; when the aspect ratio of the obstacle is within a certain When the obstacle type is within the aspect ratio range corresponding to the obstacle type, it is determined that the obstacle type is successfully matched; in response to the successful matching, the successfully matched type is used as the type of the obstacle. For the aspect ratio range, any applicable value may be set according to actual experience or multiple experiments, which is not limited in this embodiment.

Step B3, in response to judging that the first proportion and/or the second proportion is within the proportion range of the dangerous area, determine that the obstacle is located in the dangerous area.

Determine whether the first proportion of the width of the obstacle in the width of the target image and/or the second proportion of the height of the obstacle in the height of the target image are within the proportion of the dangerous area corresponding to the type of obstacle . In response to judging that the first proportion and/or the second proportion are within the proportion range of the dangerous area corresponding to the type of the obstacle, it is determined that the obstacle is located in the dangerous area.

FIG. 5 is a schematic diagram of an obstacle comparison according to an embodiment of the present application. As shown in Figure 5, the proportion of the width and height of the obstacle in image a in image a is smaller than the proportion of the width and height of the obstacle in image b in image b, which can explain that in image a The distance between the obstacle in image b and the vehicle currently driven by the user is larger than the distance between the obstacle in image b and the vehicle currently driven by the user. It is judged that the obstacle in the image a is located in the non-dangerous area, and the obstacle in the image b is located in the dangerous area.

In another application scenario, when the user (that is, the driver) drives the vehicle to reach the intersection, if the vehicle in front is too high, it will block the user's sight, so that the user cannot see the traffic conditions at the intersection, and cannot see clearly the traffic conditions at the intersection. Traffic lights can easily cause traffic accidents and violate traffic rules. In this case, the method of this embodiment can be used for obstacle detection, so as to identify whether the vehicle in front is located in a dangerous area (that is, whether to block the user's line of sight), and when it is recognized that the vehicle in front is located in a dangerous area, a danger reminder is triggered Information, in order to remind the user that the vehicle in front blocks the line of sight, pay attention to the safety of the intersection, and pay attention to the traffic lights.

For this scenario, in an optional implementation manner, the video capture device may be installed in front of the vehicle, and accordingly, the target image may include an image in front of the vehicle.

Based on the position information of the obstacle in the target image, the process of identifying whether the obstacle is located in the dangerous area may include the following steps C1-C3:

Step C1, judging whether the vehicle is currently at the intersection.

In an optional implementation, when the user is driving the vehicle, the vehicle-machine terminal can obtain the vehicle's navigation data in real time or periodically, and the navigation data can include intersection information and the like. If the vehicle-machine terminal performs the obstacle detection method, the vehicle-machine terminal can judge whether the vehicle is currently at an intersection based on the vehicle's navigation data; if the cloud server performs the obstacle detection method, the vehicle-machine terminal can use the vehicle's navigation data Send to the cloud server, and the cloud server judges whether the vehicle is currently at an intersection based on the vehicle's navigation data.

In another optional implementation, when the user is driving the vehicle, the vehicle-machine terminal can obtain the positioning data of the vehicle and preset road network data in real time or periodically, and the road network data can include intersection information and the like. If the vehicle-machine terminal executes the obstacle detection method, the vehicle-machine terminal can judge whether the vehicle is currently at an intersection based on the vehicle’s positioning data and road network data; if the cloud server executes the obstacle detection method, the vehicle-machine terminal can The positioning data and road network data of the vehicle are sent to the cloud server, and the cloud server judges whether the vehicle is currently located at the intersection based on the positioning data of the vehicle and the road network data.

Step C2, in response to judging that the vehicle is currently at an intersection, calculating the distance between the top of the obstacle and the top of the target image based on the location information.

In general, the higher the obstacle in the target image, the smaller the distance between the top of the obstacle and the top of the target image; the lower the obstacle, the larger the distance between the top of the obstacle and the top of the target image . Therefore, the distance between the top of the obstacle and the top of the target image can reflect the height of the obstacle, and further reflect whether the obstacle may block the user's sight of the vehicle behind. Therefore, it is possible to recognize whether an obstacle is located in a dangerous area based on the distance between the top of the obstacle and the top of the target image.

The position information of the obstacle in the target image recognized by the image recognition technology may include but not limited to: the coordinates of each pixel corresponding to the obstacle, and the coordinates take the lower left corner of the target image as the origin. Based on the coordinates of the uppermost pixel corresponding to the obstacle (that is, the pixel with the largest vertical coordinate corresponding to the obstacle) and the coordinates of the top pixel of the target image, the distance between the top of the obstacle and the top of the target image can be calculated . Specifically, the difference between the coordinates of the top pixel of the target image and the coordinates of the uppermost pixel corresponding to the obstacle is taken as the distance between the top of the obstacle and the top of the target image.

Step C3, in response to judging that the distance is less than a preset threshold, determine that the obstacle is located in a dangerous area.

It is determined whether the distance between the top of the obstacle and the top of the target image is less than a preset threshold, and in response to determining that the distance between the top of the obstacle and the top of the target image is less than the preset threshold, it is determined that the obstacle is located in the dangerous area.

For the value of the preset threshold, a large number of experiments can be carried out according to the actual situation to find the maximum value of the distance between the top of the obstacle and the top of the collected target image when it can block the user's field of view, and set the maximum The value is used as the preset threshold, and this embodiment does not limit the specific value of the preset threshold.

FIG. 6 is a schematic diagram of another obstacle comparison according to the embodiment of the present application. As shown in Figure 6, the distance between the top of the obstacle and the top of the target image in image c is larger than the distance between the top of the obstacle and the top of the target image in image d, which can explain that in image c The possibility of obstacles blocking the user's line of sight is smaller than the possibility of obstacles blocking the user's line of sight in image d. After judgment, it can be concluded that the obstacle in image c is located in a non-dangerous area (wide view), and the obstacle in image d is located in a dangerous area (obstructed view).

In another application scenario, if the user (that is, the driver) is driving the vehicle, if the user's vehicle is too close to the left obstacle and/or the right obstacle, it may easily cause a traffic accident. For example, when a user turns right while driving a vehicle, if other vehicles or pedestrians are close to the vehicle, a traffic accident may occur during the right turn. For another example, if the user opens the car door, if other vehicles or pedestrians are close to the vehicle, a traffic accident may occur during the process of opening the car door. For this situation, the method of this embodiment can be used for obstacle detection, so as to identify whether the left obstacle and/or the right obstacle are located in the dangerous area (that is, whether they are too close to the vehicle), and when the obstacle is identified When the obstacle on the left and/or the obstacle on the right is located in the dangerous area, a danger reminder message is triggered to remind the user that the obstacle on the left and/or the obstacle on the right is too close, please pay attention to safety.

For this scenario, in an optional implementation, the video capture device can be installed on the left side of the vehicle (such as at the rearview mirror on the left side of the vehicle, on the left side of the rear of the vehicle, etc.) and/or on the right side of the vehicle (such as on the right side of the vehicle rearview mirror, the right side of the rear of the vehicle, etc.), correspondingly, the target image may include an image on the left side of the vehicle and/or an image on the right side of the vehicle.

Based on the position information of the obstacle in the target image, the process of identifying whether the obstacle is located in the dangerous area may include the following steps D1-D2:

Step D1, acquiring the type of the vehicle and the preset range of dangerous areas corresponding to the type of the vehicle.

The type of the vehicle may be the model of the vehicle, etc., and the vehicle-machine terminal may store the type of the vehicle in advance.

Different vehicle types may correspond to different dangerous area ranges. Therefore, a large number of experiments (such as different video capture device angles, different distances, etc.) can be carried out according to the actual situation to find out the corresponding dangerous area ranges of different vehicle types. The range of the dangerous area may be a part of the range in the image captured by the video capture device, and the range of the dangerous area may include coordinates of pixels on the edge of the dangerous area, and the like. This embodiment does not limit the specific value of the range of the dangerous area.

Step D2, in response to judging that the location information is within the range of the dangerous area, determine that the obstacle is located in the dangerous area.

The position information of the obstacle in the target image recognized by the image recognition technology may include but not limited to: the coordinates of each pixel corresponding to the obstacle, and the coordinates take the lower left corner of the target image as the origin. Based on the coordinates of the pixel corresponding to the obstacle, it can be determined whether the position information of the obstacle in the target image is within the range of the dangerous area corresponding to the type of the vehicle. In response to judging that the location information is within the range of the dangerous area, it is determined that the obstacle is located in the dangerous area.

Fig. 7 is a schematic diagram of a vehicle left image and a vehicle right image according to an embodiment of the present application. As shown in FIG. 7 , image e is an image on the left side of the vehicle. In the image on the left side of the vehicle, a dangerous area and a non-dangerous area are divided, and the position information of obstacles is located within the range of the dangerous area. Image f is the image on the right side of the vehicle. In the image on the right side of the vehicle, a dangerous area and a non-dangerous area are divided, and the position information of the obstacle is located within the range of the dangerous area.

In the embodiment of the present application, for different application scenarios, different identification methods can be used to identify whether an obstacle is located in a dangerous area, so as to remind the user for different scenarios, thereby improving driving safety.

In the embodiment of the present application, an electronic device is also provided. The electronic device may include one or more processors and one or more computer-readable storage media having instructions, such as application programs, stored thereon. When the instructions are executed by the one or more processors, the processors are made to execute the obstacle detection method in any of the above embodiments.

FIG. 8 shows a schematic structural diagram of an electronic device 800 according to an embodiment of the present application. As shown in FIG. 8 , an electronic device 800 includes a central processing unit (Central Processing Unit, referred to as CPU) 801, which can be stored in a read-only memory (Read Only Memory, referred to as ROM) 802 according to computer program instructions or from a storage unit 808 The computer program instructions loaded into the Random Access Memory (RAM for short) 803 execute various appropriate actions and processes. In the RAM 803, various programs and data necessary for the operation of the electronic device 800 can also be stored. The CPU 801 , ROM 802 , and RAM 803 are connected to each other via a bus 804 . An input/output (Input/Output, I/O for short) interface 805 is also connected to the bus 804 .

Multiple components in the electronic device 800 are connected to the I/O interface 805, including: an input unit 806, such as a keyboard, mouse, microphone, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk , an optical disc, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 809 allows the electronic device 800 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

The various procedures and processing described above can be executed by the processing unit 801 . For example, the obstacle detection method in any of the above embodiments can be implemented as a computer software program, which is tangibly contained in a computer-readable medium, such as the storage unit 808 . In some embodiments, part or all of the computer program may be loaded and/or installed on the electronic device 800 via the ROM 802 and/or the communication unit 809 . When the computer program is loaded into RAM 803 and executed by CPU 801 , one or more actions in the obstacle detection method described above can be performed.

In an embodiment of the present application, there is also provided a computer-readable storage medium, on which a computer program is stored, and the program can be executed by a processor of an electronic device. When the computer program is executed by the processor, the The processor executes the obstacle detection method described in any one of the above embodiments.

The processors mentioned above may include, but are not limited to: CPU, Network Processor (NP for short), Digital Signal Processing (DSP for short), Application Specific Integrated Circuit (ASIC for short) , Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like.

The computer-readable storage medium mentioned above may include but not limited to: ROM, RAM, Compact Disc ReadOnly Memory (CD-ROM for short), Electronic Erasable Programmable ReadOnly Memory (Electronic Erasable Programmable ReadOnly Memory, EEPROM for short), hard disk, floppy disk, flash memory, and so on.

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or order between them. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or terminal equipment comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements identified, or also include elements inherent in such a process, method, article, or end-equipment. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising said element.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM, RAM, disk, CD) contains several instructions to enable a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in various embodiments of the present application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of this application, without departing from the purpose of this application and the scope of protection of the claims, many forms can also be made, all of which belong to the protection of this application.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed in the embodiments of the present application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

Those skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: various media capable of storing program codes such as U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk.

The above is only a specific implementation of the application, but the scope of protection of the application is not limited thereto. Anyone familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the application. Should be covered within the protection scope of this application. To sum up, the contents of this specification should not be understood as limiting the application.

Claims (9)

1. an obstacle detection method, is characterized in that, described method comprises: acquiring a target image, where the target image is an image outside the vehicle; Responsive to acquiring the target image, identifying whether an obstacle exists in the target image; In response to identifying an obstacle in the target image, acquiring position information of the obstacle in the target image, and identifying whether the obstacle is located in a dangerous area based on the position information. 2. The method according to claim 1, said acquiring target image comprises the steps of: Obtain video of the exterior of the vehicle and the speed of the vehicle; determining an extraction time interval based on the driving speed; wherein the extraction time interval is negatively correlated with the driving speed; Extracting an image from the video according to the extraction time interval as the target image. 3. The method according to claim 1, wherein the target image is an image in front of the vehicle and/or an image behind the vehicle; identifying whether the obstacle is located in a dangerous area based on the position information comprises the following steps: calculating a first ratio of the width of the obstacle to the width of the target image and a second ratio of the height of the obstacle to the height of the target image based on the position information; Identifying the type of the obstacle, and obtaining a preset proportion of the dangerous area corresponding to the type of the obstacle; In response to judging that the first proportion and/or the second proportion is within the proportion range of the dangerous area, it is determined that the obstacle is located in the dangerous area. 4. The method according to claim 3, said identifying the type of said obstacle comprises the steps of: matching the aspect ratio of the obstacle with a preset aspect ratio range corresponding to each obstacle type; In response to successful matching, the type of successful matching is used as the type of the obstacle. 5. The method according to claim 1, wherein the target image is an image in front of the vehicle; and identifying whether the obstacle is located in a dangerous area based on the position information comprises the following steps: judging whether the vehicle is currently at the intersection; calculating a distance between a top of the obstacle and a top of the target image based on the position information in response to determining that the vehicle is currently located at an intersection; In response to judging that the distance is less than a preset threshold, it is determined that the obstacle is located in a dangerous area. 6. The method according to claim 1, wherein the target image is an image on the left side of the vehicle and/or an image on the right side of the vehicle; the identifying whether the obstacle is located in a dangerous area based on the position information comprises the following steps: Obtaining the type of the vehicle and the preset dangerous area range corresponding to the type of the vehicle; In response to judging that the location information is within the range of the dangerous area, it is determined that the obstacle is located in the dangerous area. 7. The method of claim 1, further comprising: In response to recognizing that the obstacle is located in a dangerous area, triggering a danger reminder message. 8. An electronic device, characterized in that it comprises: one or more processors; and one or more computer-readable storage media having instructions stored thereon; When the instructions are executed by the one or more processors, the processors are made to execute the obstacle detection method according to any one of claims 1-7. 9. A computer-readable storage medium, characterized in that a computer program is stored thereon, and when the computer program is executed by a processor, the processor executes the method according to any one of claims 1 to 7. Obstacle detection method.

Images