WO2022228023A1

WO2022228023A1 - Blind area monitoring method and apparatus, electronic device, and storage medium

Info

Publication number: WO2022228023A1
Application number: PCT/CN2022/084399
Authority: WO
Inventors: 罗铨; 李弘扬; 蒋沁宏
Original assignee: 上海商汤智能科技有限公司
Priority date: 2021-04-28
Filing date: 2022-03-31
Publication date: 2022-11-03
Also published as: CN113103957B; CN113103957A

Abstract

Provided are a blind area monitoring method and apparatus, an electronic device, and a storage medium. The blind area monitoring method comprises: obtaining a current frame monitoring image acquired by an acquisition device (2) on a target vehicle (S101); performing object detection on the current frame monitoring image to obtain type information and positions of objects comprised in the current frame monitoring image (S102); determining, according to the positions of the objects and a field of view blind area of the target vehicle, a target object located in the field of view blind area of the target vehicle (S103); and generating a monitoring result according to the type information and the position of the target object and a driving state of the target vehicle (S104), thereby improving driving safety and blind area monitoring performance.

Description

A blind spot monitoring method, device, electronic device and storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure is based on the Chinese patent application with the application number of 202110467776.4, the application date of April 28, 2021, and the application name of "a blind spot monitoring method, device, electronic device and storage medium", and requires the priority of the above-mentioned Chinese patent application The entire contents of the above-mentioned Chinese patent application are hereby incorporated into the present disclosure by reference.

technical field

The present disclosure relates to the technical field of image processing, and in particular, to a blind spot monitoring method, device, electronic device, and storage medium.

Background technique

During the driving process of vehicles and robots, blind spots are prone to occur due to the limited range that can be observed. Due to the existence of blind spots, it is very easy to cause errors in judgment and operation of drivers or autonomous vehicles, reducing the safety of driving.

SUMMARY OF THE INVENTION

The embodiments of the present disclosure provide at least a blind spot monitoring method, an apparatus, an electronic device, and a storage medium.

In a first aspect, an embodiment of the present disclosure provides a blind spot monitoring method, including: acquiring a current frame monitoring image collected by a collection device on a target vehicle; performing object detection on the current frame monitoring image to obtain the current frame monitoring image Type information and position of the object included in the image; according to the position of the object and the blind spot of the target vehicle, determine the target object located in the blind spot of the target vehicle; according to the type information and position of the target object and the driving state of the target vehicle to generate monitoring results.

In the embodiment of the present disclosure, the current frame monitoring image acquired by the acquisition device on the target vehicle is acquired, and the object detection is performed on the current frame monitoring image to determine the type information and position of the object included in the current frame monitoring image, and then according to the position of the object and the blind spot of the target vehicle, to determine the target object located in the blind spot of the target vehicle; and then generate monitoring results according to the type information and location of the target object and the driving state of the target vehicle. In this way, different monitoring results can be generated for different types of target objects, thereby improving driving safety and blind spot monitoring performance.

In a possible implementation manner, the monitoring result includes warning information, the driving state of the target vehicle includes steering information of the target vehicle; the type information and position of the target object and the target vehicle According to the type information and position of the target object and the steering information of the target vehicle, determining the level of the alarm information; generating and prompting the alarm information of the determined level.

In a possible implementation manner, the monitoring result includes vehicle control instructions, the driving state of the target vehicle includes steering information of the target vehicle; the type information and position of the target object and the target The driving state of the vehicle, and generating a monitoring result includes: generating the vehicle control instruction according to the type information and position of the target object and the steering information of the target vehicle; the blind spot monitoring method further includes: based on the vehicle control instruction to control the target vehicle to travel.

In this way, more targeted and accurate monitoring results can be generated according to the type information and location of the target object and the driving state of the target vehicle.

In a possible implementation manner, determining the target object located in the blind spot of the target vehicle according to the position of the object and the blind spot of the target vehicle includes: monitoring the image in the current frame according to the The position of the object is determined, and the current first distance information between the target vehicle and the object in the current frame monitoring image is determined; according to the current first distance information, the target object located in the blind spot of the target vehicle is determined.

In this way, the target object located in the blind spot of the target vehicle can be accurately detected from all the objects included in the monitoring image of the current frame.

In a possible implementation manner, determining the current first distance information between the target vehicle and the object in the current frame monitoring image according to the position of the object in the current frame monitoring image includes: Based on the current frame monitoring image, determine the distance information to be adjusted between the target vehicle and the object in the current frame monitoring image; based on the object in the multi-frame historical frame monitoring image collected by the collection device, two adjacent two The scale change information between the scales in the frame monitoring image, and the historical first distance information between the object and the target vehicle in each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, for all The distance information to be adjusted is adjusted to obtain the current first distance information between the target vehicle and the object.

In a possible implementation manner, the adjusting the distance information to be adjusted to obtain the current first distance information between the target vehicle and the object includes: adjusting the distance information to be adjusted , until the error amount of the scale change information is the smallest, and the adjusted distance information is obtained; wherein, the error amount is based on the to-be-adjusted distance information, the scale change information, and each frame in the multi-frame historical frame monitoring image The historical first distance information corresponding to the monitoring image of the historical frame is determined; and the current first distance information is determined based on the adjusted distance information.

In the embodiment of the present disclosure, by continuously optimizing the scale change information between the scale of the object in the current frame monitoring image and the historical frame monitoring image adjacent to the current frame monitoring image, it is possible to reduce the amount of the acquired object in the current frame monitoring image. The error of the scale change information between the image and the scale in the monitoring image of the historical frame adjacent to the monitoring image of the current frame, thereby improving the stability of the adjusted distance information.

In a possible implementation manner, before determining the current first distance information based on the adjusted distance information, the blind spot monitoring method further includes: performing target detection on the monitoring image of the current frame, and determining The position information of the detection frame of the object contained in the current frame monitoring image; the current second distance information is determined based on the position information of the detection frame and the calibration parameters of the acquisition device; the adjustment is based on the adjustment After the distance information is obtained, determining the current first distance information includes: based on the current second distance information, the object and the target vehicle in each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image between the historical second distance information, the historical first distance information corresponding to the frame of the historical frame monitoring image, and the adjusted distance information, determine the distance offset information for the adjusted distance information; The adjusted distance information is adjusted by the distance offset information to obtain the current first distance information.

In the embodiment of the present disclosure, after the distance offset information is obtained, the adjusted distance information can be further adjusted, so as to obtain the current distance information of the target vehicle and the object with high accuracy.

In a possible implementation manner, based on the current second distance information, the historical distance between the object and the target vehicle in each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image Second distance information, the historical first distance information corresponding to the frame of the historical frame monitoring image, and the adjusted distance information, and determining distance offset information for the adjusted distance information, including: based on the current first distance information The second distance information and the historical second distance information corresponding to each frame of the historical frame monitoring image in the multi-frame historical frame monitoring images, determine the distance corresponding to each historical frame monitoring image in the multi-frame historical frame monitoring images. The first linear fitting coefficient of the first fitting curve fitted by the historical second distance information and the current second distance information; based on each frame of the historical frame monitoring image corresponding to the multi-frame historical frame monitoring image The historical first distance information and the adjusted distance information, determine the historical first distance information and the adjusted distance information corresponding to each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image. a second linear fitting coefficient of the second fitting curve fitted to the distance information; determining a distance bias for the adjusted distance information based on the first linear fitting coefficient and the second linear fitting coefficient setting information.

In a possible implementation manner, the determining the current second distance information based on the position information of the detection frame and the calibration parameters of the collection device includes: obtaining, based on the position information of the detection frame, The pixel coordinate value of the corner point is set in the detection frame; based on the pixel coordinate value of the set corner point, the calibration parameters of the acquisition device, and the vanishing point of the lane line used when determining the calibration parameters of the acquisition device The pixel coordinate value of , determines the current second distance information.

In a possible implementation manner, the calibration parameters of the acquisition device include a first height value of the acquisition device relative to the ground and a focal length of the acquisition device; the pixel coordinate value based on the set corner point , the calibration parameters of the collection device and the pixel coordinate value of the vanishing point of the lane line used when determining the calibration parameters of the collection device, and determining the current second distance information includes: based on the pixels of the vanishing point of the lane line The coordinate value and the pixel coordinate value of the set corner point in the detection frame, determine the first pixel height value of the acquisition device relative to the ground; based on the pixel coordinate value of the set corner point, determine the current frame monitoring a second pixel height value of the object in the image relative to the ground; determining a second pixel height value of the object relative to the ground based on the first pixel height value, the second pixel height value, and the first height value height value; determining the current second distance information based on the second height value, the focal length of the collecting device, and the second pixel height value.

In the embodiment of the present disclosure, under the condition that the complete detection frame of the object in the monitoring image of the current frame can be detected, the actual height of the object can be obtained quickly and accurately by introducing the pixel coordinate value of the vanishing point of the lane line and the calibration parameters of the acquisition device value, further, the current second distance information between the target vehicle and the object can be quickly and accurately determined.

In a possible implementation manner, the determining, based on the monitoring image of the current frame, the distance information to be adjusted between the target vehicle and the object in the monitoring image of the current frame includes: The scale change information between the scale in the frame monitoring image and the scale in the historical frame monitoring image adjacent to the current frame monitoring image; based on the scale change information and the scale adjacent to the current frame monitoring image The historical first distance information corresponding to the historical frame monitoring image is determined to determine the to-be-adjusted distance information.

In the embodiment of the present disclosure, the historical first distance information with high accuracy corresponding to the monitoring image of the historical frame adjacent to the monitoring image of the current frame, and the monitoring image of the object in the current frame and the monitoring image of the current frame adjacent to the monitoring image of the current frame are used. The scale change information between scales in the historical frame monitoring image can more accurately obtain the distance information to be adjusted, so that the adjustment speed can be improved when the current first distance information is determined based on the distance information to be adjusted later.

In a possible implementation manner, the scale change information between the scales of the object in two adjacent frames of monitoring images is determined in the following manner: by extracting a plurality of feature points contained in the object in the adjacent two frames respectively. In the frame monitoring image, the first position information in the previous frame monitoring image, and the second position information in the following frame monitoring image; based on the first position information and the second position information, it is determined that the object is in Scale change information between scales in two adjacent frames of monitoring images.

In a possible implementation manner, the determining, based on the first position information and the second position information, the scale change information between the scales of the object in two adjacent frames of monitoring images includes: based on For the first position information, determine the first scale value of the target line segment formed by a plurality of feature points included in the object in the monitoring image of the previous frame; based on the second position information, determine the target line segment the second scale value in the monitoring image of the next frame; based on the first scale value and the second scale value, determine the scale change information of the object between the scales in the two adjacent frames of the monitoring image .

In the embodiment of the present disclosure, by extracting the position information of the multiple feature points contained in the object in the monitoring image, the position information of the object in the monitoring image can be more accurately represented, so as to obtain more accurate scale change information, which is convenient for the When the change information adjusts the to-be-adjusted distance information, more accurate current first distance information can be obtained.

In a second aspect, an embodiment of the present disclosure provides a blind spot monitoring device, including:

an acquisition module, used to acquire the current frame monitoring image acquired by the acquisition device on the target vehicle;

a detection module, configured to perform object detection on the current frame monitoring image to obtain type information and position of the object included in the image;

a determining module, configured to determine the target object located in the blind spot of the target vehicle according to the position of the object and the blind spot of the target vehicle;

The generating module is configured to generate monitoring results according to the type information and position of the target object and the driving state of the target vehicle.

In a third aspect, embodiments of the present disclosure provide an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processing A bus communicates between the processor and the memory, and when the machine-readable instructions are executed by the processor, the steps of the blind spot monitoring method according to the first aspect are performed.

In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to execute the blind spot monitoring method according to the first aspect. step.

In order to make the above-mentioned objects, features and advantages of the present disclosure more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present disclosure more clearly, the following briefly introduces the accompanying drawings required in the embodiments, which are incorporated into the specification and constitute a part of the specification. The drawings illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. Other related figures are obtained from these figures.

1 shows a flowchart of a blind spot monitoring method provided by an embodiment of the present disclosure;

FIG. 2 shows a schematic diagram of determining a blind spot of a visual field provided by an embodiment of the present disclosure;

3 shows a flowchart of a specific method for determining current first distance information provided by an embodiment of the present disclosure;

FIG. 4 shows a flowchart of a method for determining scale change information provided by an embodiment of the present disclosure;

FIG. 5 shows a flowchart of a method for determining distance information to be adjusted provided by an embodiment of the present disclosure;

6 shows a flowchart of a method for determining current first distance information provided by an embodiment of the present disclosure;

FIG. 7 shows a flowchart of a method for determining current second distance information provided by an embodiment of the present disclosure;

FIG. 8 shows a schematic diagram of a positional relationship among a target device, a collection device, and a target object provided by an embodiment of the present disclosure;

FIG. 9 shows a schematic diagram of a detection frame of a target object provided by an embodiment of the present disclosure;

FIG. 10 shows a schematic diagram of a principle for determining current second distance information provided by an embodiment of the present disclosure;

FIG. 11 shows a schematic diagram of a scenario for determining current second distance information provided by an embodiment of the present disclosure;

FIG. 12 shows a schematic structural diagram of a blind spot monitoring device provided by an embodiment of the present disclosure;

FIG. 13 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only These are some, but not all, embodiments of the present disclosure. The components of the disclosed embodiments generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the disclosure provided in the accompanying drawings is not intended to limit the scope of the disclosure as claimed, but is merely representative of selected embodiments of the disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present disclosure.

It should be noted that like numerals and letters refer to like items in the following figures, so once an item is defined in one figure, it does not require further definition and explanation in subsequent figures.

The term "and/or" in this paper only describes an association relationship, which means that there can be three kinds of relationships, for example, A and/or B, which can mean: the existence of A alone, the existence of A and B at the same time, the existence of B alone. a situation. In addition, the term "at least one" herein refers to any combination of any one of the plurality or at least two of the plurality, for example, including at least one of A, B, and C, and may mean including from A, B, and C. Any one or more elements selected from the set of B and C.

The study found that when using radar to monitor the target object in the blind spot, since the point cloud points obtained by the radar during scanning are for the entire range of the blind spot, in addition to paying attention to vehicles, pedestrians and signboards during monitoring, it will also monitor to other non-concerned objects. When blind spot monitoring is performed in this way, once an object is detected by the radar in the blind spot of the vehicle's field of vision, an alarm will be generated. However, in fact, not all objects in the blind spot of the field of view will affect the driving safety of the vehicle, which leads to many invalid alarms, and the current blind spot monitoring method has the problem of poor monitoring performance.

Based on the above research, an embodiment of the present disclosure provides a blind spot monitoring method. The embodiment of the present disclosure determines the current frame monitoring image by acquiring the current frame monitoring image collected by the acquisition device on the target vehicle, and performing object detection on the current frame monitoring image. The type information and position of the object included in the image, and then according to the position of the object and the blind spot of the target vehicle, determine the target object located in the blind spot of the target vehicle; then according to the type information and position of the target object, and the driving of the target vehicle status and generate monitoring results. In this way, different monitoring results can be generated for different types of target objects, thereby improving blind spot monitoring performance and improving driving safety.

In order to facilitate the understanding of this embodiment, a blind spot monitoring method disclosed in the embodiment of the present disclosure is first introduced in detail. The device includes, for example, a terminal device or a server or other processing device, and the terminal device may be a computing device, a vehicle-mounted device, or the like. In some possible implementations, the blind spot monitoring method may be implemented by the processor calling computer-readable instructions stored in the memory.

Referring to FIG. 1, which is a flowchart of a blind spot monitoring method provided by an embodiment of the present disclosure, the blind spot monitoring method includes the following S101-S104:

S101, acquiring the current frame monitoring image acquired by the acquisition device on the target vehicle;

S102, performing object detection on the current frame monitoring image to obtain type information and positions of objects included in the current frame monitoring image;

S103, according to the position of the object and the blind spot of the target vehicle, determine the target object located in the blind spot of the target vehicle;

S104 , generating a monitoring result according to the type information and position of the target object and the driving state of the target vehicle.

For the above S101, in different scenarios, the corresponding target vehicles are also different.

Exemplarily, in a scenario where a driver drives a vehicle, the target vehicle may include, for example, a vehicle driven by the driver; in an autonomous driving scenario, the target vehicle may include, for example, an autonomous vehicle; in a warehousing and freight scenario, the target vehicle may include, for example, an autonomous vehicle. Can include cargo robots. The embodiments of the present disclosure are described by taking the target vehicle as a vehicle as an example.

A collection device may also be mounted on the target vehicle, and the collection device may be a monocular camera set on the target vehicle, which is used for shooting during the driving of the target vehicle. Exemplarily, if the target area includes: a blind spot of the vehicle's field of vision, the capture device may be installed on a column of the vehicle, and the camera of the capture device faces the blind spot of the vehicle's field of view.

Among them, different target vehicles may have different corresponding blind spots due to their different models. The embodiment of the present disclosure determines the blind area of the field of vision with reference to the national standard. For details, see FIG. 2 , which is a schematic diagram of determining the blind area of the field of view provided by the embodiment of the present disclosure. In FIG. 2 , the vehicle 1 is equipped with a collection device 2 , and the blind area of the visual field included in the target area collected by the collection device 2 includes the positions indicated by 3 and 4 .

Specifically, when acquiring the monitoring image of the current frame collected by the acquisition device on the target vehicle, for example, the following methods may be used: acquiring the monitoring video obtained by the acquisition device on the target vehicle performing image acquisition on the target area; determining the current frame from the monitoring video Frame monitoring images; wherein, the target area includes: an area within the shooting field of view of the acquisition device.

When using the capture device to capture the target area, the direction of the capture can be pre-set; after the capture device is mounted on the target vehicle, it can be determined that the target area to be captured is the area within the capture range of the capture device. When the vehicle is running or stopped, the acquisition device can collect images of the target area, obtain monitoring videos, and determine the monitoring images of the current frame from the monitoring videos.

Wherein, when using the monitoring video to determine the monitoring image of the current frame, the method of determining the video frame image from the monitoring video can be used, for example, the frame of the captured video frame image that is closest to the current time is used as the monitoring image of the current frame.

For the above S102, after the current frame monitoring image is acquired by the above S101, object detection may also be performed on the current frame monitoring image to obtain the type information and position of the object included in the current frame monitoring image.

In a specific implementation, when the current frame monitoring image is subjected to object detection and the type information of the object included in the image is determined, for example, a pre-trained target detection neural network can be used to perform object detection processing on the current frame monitoring image, Exemplarily, when performing object detection on the monitoring image of the current frame, at least one of the following object detection algorithms can be used: convolutional neural network (Convolutional Neural Networks, CNN), target detection network (Region-based CNN, RCNN) , Fast Neural Network (Fast RCNN), and Faster Neural Network (Faster RCNN).

When the object detection algorithm is used to perform object monitoring on the monitoring graph of the current frame, the objects that can be detected include, for example, other driving vehicles, pedestrians, road facilities, and road obstacles.

When the object detection is performed on the monitoring image of the current frame, the position of the object included in the image can also be obtained. Through the position of the detected object in the image, the actual position of the object in the actual driving process of the target vehicle can be further determined.

For the above S103, using the position of the object and the blind spot of the target vehicle, the target object located in the blind spot of the target vehicle can be determined in the following manner: According to the position of the object in the current frame monitoring image, determine the target vehicle and the current frame monitoring image The current first distance information of the object in the target vehicle; according to the current first distance information, the target object located in the blind spot of the target vehicle's field of view is determined.

At present, when using the images collected by the monocular camera to determine the distance, the monocular camera mounted on the smart car will have problems such as road bumps or obstructions due to changes in the driving road conditions during the driving of the smart car. In this case, when the distance measurement is performed based on the detection frame corresponding to the object in the monitoring image of the current frame, the accurate distance to the object may not be detected. , so when the distance to the object is continuously detected based on the detection frame, the obtained distance between the smart car and the object is not stable in time series.

In order to use the image collected by the monocular camera to detect the distance between the target vehicle and the object as accurately and stably as possible, an embodiment of the present disclosure also proposes a distance detection scheme,

Referring to FIG. 3, which is a flowchart of a specific method for determining current first distance information provided by an embodiment of the present disclosure, the method includes the following S301-S302:

S301 , based on the monitoring image of the current frame, determine the distance information to be adjusted between the target vehicle and the object in the monitoring image of the current frame.

Exemplarily, the objects may include, but are not limited to, vehicles, pedestrians, fixed obstacles, and the like. In the present disclosure, the object is a vehicle as an example for introduction.

Exemplarily, the current frame monitoring images provided by the embodiments of the present disclosure are all monitoring images that are not detected for the first time. If the current frame monitoring images are monitoring images that detect objects for the first time, it can be directly The position information, the parameter information of the acquisition device and the pixel coordinate value of the vanishing point obtained in the above calibration process determine the current second distance information between the object and the object, and the current second distance information can be directly used as the current first distance information, specifically The process of determining the current second distance information is described later in detail.

Exemplarily, when the current frame monitoring image is not the first time to collect the object, the current first distance information corresponding to the current frame monitoring image, or the historical first distance information corresponding to each frame of the historical frame monitoring image indicates that after adjustment obtained distance information.

Exemplarily, when determining the distance information to be adjusted between the target vehicle and the object based on the current frame monitoring image, it can be based on the historical first distance information corresponding to the historical frame monitoring image adjacent to the current frame monitoring image and the current frame monitoring The scale change information between the image and the scale in the historical frame monitoring image adjacent to the current frame monitoring image is determined, and the to-be-adjusted distance information is adjusted subsequently.

S302, based on the scale change information between the scales in two adjacent frames of monitoring images in the multi-frame historical frame monitoring images collected by the acquisition device, and the object in each frame of the historical frame monitoring images in the multi-frame historical frame monitoring images and the object in the historical frame monitoring images The historical first distance information between the target vehicles is adjusted, and the to-be-adjusted distance information is adjusted to obtain the current first distance information between the target vehicle and the object.

Exemplarily, the scale change information of the object in two adjacent frames of monitoring images (such as including monitoring image i and monitoring image j) in the multi-frame historical frame monitoring images collected by the acquisition device includes the size of the object in the next frame of monitoring image j. The ratio of the scale to the scale of the object in the monitoring image i in the previous frame, and the specific determination process will be described later.

Exemplarily, the embodiment of the present disclosure determines the historical first distance information between the target vehicle and the object corresponding to each frame of the historical frame monitoring image in the same manner as the current first distance information between the target vehicle and the object. Therefore, In this embodiment of the present disclosure, the process of determining the historical first distance information will not be described repeatedly.

In the embodiment of the present disclosure, the scale change information in two adjacent frames of monitoring images in the multi-frame historical frame monitoring images based on the object and the historical first distance information between the target vehicle and the object that have been adjusted in the historical process can be used. , adjust the distance information to be adjusted obtained based on the current monitoring image, so that the distance between the target vehicle and the object corresponding to two adjacent frames of monitoring images changes relatively smoothly, and can truly reflect the distance between the target vehicle and the object during the driving process. The actual distance changes can improve the stability of the predicted distance between the target vehicle and the object in the time series.

In addition, the scale change information of the object in two adjacent frames of monitoring images can also reflect the distance change between the target vehicle and the object. In the multi-frame historical frame monitoring image, each historical frame monitoring image corresponds to the history between the target vehicle and the object. The first distance information is relatively accurate distance information obtained after adjustment. Therefore, in the multi-frame historical frame monitoring images collected by the object-based acquisition device, the scale change information in two adjacent frames of monitoring images, and the multi-frame historical frame monitoring images. After adjusting the historical first distance information between the target vehicle and the object corresponding to each frame of the historical frame monitoring image in the to-be-adjusted distance information, more accurate current first distance information can be obtained.

In the embodiment of the present disclosure, when determining the current first distance information, it can be based on the scale change information between scales in two adjacent frames of monitoring images in multiple frames of historical frame monitoring images based on the object, and the information that has been adjusted in the historical process. Obtain the historical first distance information between the target vehicle and the object, and adjust the distance information to be adjusted obtained based on the current monitoring image, so that the distance between the same object and the target vehicle in two adjacent frames of monitoring images can be changed. It is relatively stable, and can truly reflect the actual distance change between the target vehicle and the object during the driving process, and can improve the stability of the predicted distance between the target vehicle and the object in the time series.

First, for the scale change information mentioned above, as shown in FIG. 4 , the scale change information between the scales of an object in two adjacent frames of monitoring images can be determined in the following manner, including the following S401 to S402:

S401, respectively extracting the first position information of the multiple feature points included in the object in the monitoring image of the previous frame of the two adjacent frames of the monitoring image, and the second position information of the monitoring image of the following frame.

Exemplarily, target detection can be performed on the monitoring image based on a pre-trained target detection model to obtain a detection frame for representing the position of the object in the monitoring image, and then a plurality of feature points constituting the object can be extracted in the detection frame, these Feature points refer to points in the object where the pixels change drastically, such as inflection points, corner points, etc.

S402, based on the first position information and the second position information, determine the scale change information of the object between scales in two adjacent frames of monitoring images.

Exemplarily, the connecting line of any two feature points in the same frame of monitoring image can form a line segment, so that the first position information of any two feature points in the previous frame of monitoring image can be obtained. The scale of the line segment formed by any two feature points in the previous frame of monitoring image, and similarly, from the second position information of any two feature points in the next frame of monitoring image, the line segment formed by any two feature points can be obtained in For the scale in the monitoring image of the next frame, in this way, the scale of the multiple line segments on the object in the monitoring image of the previous frame and the scale of the monitoring image of the subsequent frame can be obtained respectively.

Further, the scale change information of the object in two adjacent frames of monitoring images can be determined according to the respective scales of the multiple line segments in the previous frame of the monitoring image and the respective scales in the next frame of the monitoring image.

Specifically, for S402, when determining the scale change information between the scales of the object in two adjacent frames of monitoring images based on the first position information and the second position information, the following steps S4021 to S4022 are included:

S4021 , based on the first position information, determine the first scale value of the target line segment formed by multiple feature points included in the object in the monitoring image of the previous frame.

S4022, based on the second position information, determine the second scale value of the target line segment in the monitoring image of the next frame.

Exemplarily, the target line segment includes n, where n is greater than or equal to 1 and less than a set threshold, and based on the first position information of the feature points included in each target line segment, the first scale value corresponding to the target line segment can be obtained, And, based on the second position information of the feature points included in each target line segment, a second scale value corresponding to the target line segment can be obtained.

S4023 , based on the first scale value and the second scale value, determine scale change information of the object between scales in two adjacent frames of monitoring images.

Exemplarily, the ratio between the second scale value and the first scale value corresponding to any target line segment can be used to represent the scale change information corresponding to the target line segment, and further according to the scale change information corresponding to the multiple target line segments. , to determine the scale change information of the object in two adjacent frames of monitoring images. For example, the average value of the scale change information corresponding to the set number of target line segments can be used as the scale change information of the object in two adjacent frames of monitoring images.

Compared with the method of representing the scale of the object by the position information of the two corner points of the detection frame in the monitoring image, for example, the position information of the upper left corner and the lower right corner of the detection frame in the monitoring image indicates that the object is in the monitoring image. In the embodiment of the present disclosure, by selecting the position information of a plurality of feature points in two adjacent frames of monitoring images, respectively, the scale change information of the object in the two adjacent frames of monitoring images is determined. The included position information of multiple feature points in the monitoring image can more accurately represent the position information of the object in the monitoring image, thereby obtaining more accurate scale change information.

For the above S302, when determining the distance information to be adjusted between the target vehicle and the object in the current frame monitoring image based on the current frame monitoring image, as shown in FIG. 5, the following S501-S502 may be included:

S501: Acquire the scale change information between the scale of the object in the monitoring image of the current frame and the scale in the monitoring image of the historical frame adjacent to the monitoring image of the current frame.

Exemplarily, the historical frame monitoring image adjacent to the current frame monitoring image refers to the previous frame monitoring image before the current frame monitoring image at the time of acquisition, and the object is in the current frame monitoring image and the historical frame adjacent to the current frame monitoring image. The scale change information between monitoring images can be represented by the ratio of the scale of the object in the monitoring image of the current frame to the scale of the object in the monitoring image of the historical frame adjacent to the monitoring image of the current frame. The specific determination process will be carried out later. elaborate.

S502 , based on the scale change information and the historical first distance information corresponding to the monitoring image of the historical frame adjacent to the monitoring image of the current frame, determine the distance information to be adjusted.

Considering that the target vehicle and the object are in the process of approaching, the scale of the object in the collected monitoring images will gradually increase, that is, the scale of the object in two adjacent frames of monitoring images and the target vehicle and object corresponding to the two frames of monitoring images. There is a proportional relationship between the distances. Based on this, the distance information to be adjusted can be determined by the following formula (1):

d _{0_scale} =scale×D _{1_final} ; (1);

Wherein, d _{0_scale} represents the distance information to be adjusted; scale represents the ratio of the scale of the object in the monitoring image of the current frame to the scale of the object in the monitoring image of the historical frame adjacent to the monitoring image of the current frame; D _{1_final} represents the monitoring image of the current frame The historical first distance information corresponding to the adjacent historical frame monitoring images.

In the embodiment of the present disclosure, the historical first distance information corresponding to the monitoring image of the historical frame adjacent to the monitoring image of the current frame, and the scale of the object in the monitoring image of the current frame and the monitoring image of the historical frame adjacent to the monitoring image of the current frame are used. It is possible to obtain relatively accurate distance information to be adjusted, so that the adjustment speed can be increased when the current first distance information is determined based on the distance information to be adjusted later.

Exemplarily, there may be errors in the scale change information of the obtained object between the current frame monitoring image and the historical frame monitoring image adjacent to the current frame monitoring image, such as jittering when the current monitoring image is captured, or detecting object error, the scale change information obtained based on this will have a sudden change compared with the scale change information in the adjacent two frames of monitoring images in the multi-frame historical frame monitoring image, so the distance information to be adjusted obtained based on this is compared with the adjacent ones. There will also be a sudden change in the historical first distance information. At this time, the scale change information in two adjacent frames of monitoring images in the multi-frame historical frame monitoring image collected by the object, and each frame in the multi-frame historical frame monitoring image can be used. The historical first distance information corresponding to the frame historical frame monitoring image adjusts the to-be-adjusted distance information.

Specifically, when the distance information to be adjusted is adjusted to obtain the current first distance information between the target vehicle and the object, as shown in FIG. 6 , the following steps S601 to S603 may be included:

S601, adjust the distance information to be adjusted until the error amount of the scale change information is the smallest, and obtain the adjusted distance information; wherein, the error amount is based on the distance information to be adjusted, the scale change information, and each historical frame in the multi-frame historical frame monitoring image The historical first distance information corresponding to the monitoring image is determined.

Exemplarily, the amount of error used to represent the scale change information of the object between the current frame monitoring image and the historical frame monitoring image adjacent to the current frame monitoring image can be predicted based on the following formula (2):

Among them, E represents the error amount of the scale change information of the object between the monitoring image of the current frame and the monitoring image of the historical frame adjacent to the monitoring image of the current frame; T contains the frame number of the monitoring image of the object, and T is less than or equal to the preset frame Number; t is used to indicate the historical frame monitoring image, representing the t-th frame historical frame monitoring image starting from the current frame monitoring image, for example, t=1 represents the first frame historical frame monitoring image starting from the current frame monitoring image; L _t represents The preset weight of the t-th frame historical frame monitoring image starting from the current frame monitoring image when determining the error amount E, D _{t_final} represents the historical first distance information corresponding to the t-th frame historical frame monitoring image starting from the current frame monitoring image; scale _i represents the scale change information between the i-th historical frame monitoring image and the i+1-th historical frame monitoring image starting from the current frame monitoring image.

Exemplarily, the above formula (2) can be optimized through a variety of optimization methods, for example, the d _{0_scale} in the above formula (2) can be adjusted in a manner including but not limited to the Newton gradient descent method. When E is the smallest, it is obtained: The adjusted distance information D _{0_scale} .

By continuously optimizing the scale change information of the object in the monitoring image of the current frame and the monitoring image of the historical frame adjacent to the monitoring image of the current frame, the obtained object in the monitoring image of the current frame and the difference between the monitoring image of the current frame and the current frame can be reduced. The error of the scale change information between the adjacent historical frames of the monitoring images is monitored, thereby improving the stability of the determined adjusted distance information.

S602, based on the adjusted distance information, determine the current first distance information.

Exemplarily, after the adjusted distance information is obtained, in order to further improve the accuracy of the adjusted distance information, the adjusted distance information may be further adjusted to obtain the current first distance information between the target vehicle and the object. .

Specifically, before the current first distance information is determined based on the adjusted distance information, as shown in FIG. 7 , the blind spot monitoring method provided by the embodiment of the present disclosure further includes the following S701 to S702:

S701 , perform target detection on the monitoring image of the current frame, and determine the position information of the detection frame of the object included in the monitoring image of the current frame.

S702: Determine the current second distance information based on the position information of the detection frame and the calibration parameters of the collection device.

Exemplarily, before the target vehicle travels, the acquisition device provided on the target vehicle can be calibrated, for example, the acquisition device is installed on the top of the target vehicle, as shown in FIG. The optical axis of the collecting device is parallel to the horizontal ground and the advancing direction of the target vehicle. In this way, the focal length (f _x , f _y ) of the collecting device and the height H _c of the collecting device relative to the ground can be obtained.

Exemplarily, target detection can be performed on the monitoring image of the current frame by using a pre-trained target detection model to obtain the object contained in the monitoring image of the current frame, and the detection frame corresponding to the object. As shown in FIG. 8 , the position information of the detection frame is obtained. The position information of the corner points of the detection frame in the monitoring image of the current frame may be included, for example, the pixel coordinate values of the corner points A, B, C and D in the monitoring image of the current frame may be included.

Further, according to the principle of pinhole imaging, the following formulas (3) and (4) can be obtained:

Among them, H _x represents the actual width of the object; H _y represents the actual height of the object relative to the ground; w _b represents the pixel width of the object in the monitoring image of the current frame, which can be determined by the pixel width of the detection frame ABCD of the object; h _b represents The pixel height of the object relative to the ground can be determined by the pixel height of the detection frame ABCD of the object; D ₀ represents the current second distance information between the target vehicle and the object.

Exemplarily, in one embodiment, H _x and _Hy can be determined based on the type of the detected object, for example, when the object is a vehicle, it can be determined based on the detected type of the target vehicle and the pre-stored vehicle type and The corresponding height of the vehicle and the corresponding relationship between the widths determine the actual width and actual height of the target vehicle.

Exemplarily, the width w _b of the object in the monitoring image of the current frame can be determined by the pixel coordinate value of the corner point AB in the monitoring image of the current frame in the detection frame ABCD as shown in Figure 9, or by the corner point CD in the current frame. The pixel coordinate value in the monitoring image is determined; the height h _b of the object in the monitoring image of the current frame can be determined by the pixel coordinate value of the corner point BC in the monitoring image of the current frame, or the pixel coordinate value of the corner point AD in the monitoring image of the current frame can be determined. The coordinate value is determined, which will not be repeated here.

Considering that there is a situation in which the type of the object cannot be identified, the actual height or actual width of the object may not be directly obtained, the embodiment of the present disclosure is described as an example of determining the actual height of the object. For the above S702, in the detection frame-based detection The location information and the calibration parameters of the acquisition device, when determining the current second distance information, include the following S7021-S7022:

S7021 , based on the position information of the detection frame, obtain the pixel coordinate value of the set corner in the detection frame.

S7022: Determine the current second distance information based on the pixel coordinate value of the set corner point, the calibration parameters of the collection device, and the pixel coordinate value of the vanishing point of the lane line used in determining the calibration parameter of the collection device.

10, the current second distance is determined based on the pixel coordinate value of the set corner point, the calibration parameters of the acquisition device, and the pixel coordinate value of the vanishing point of the lane line used when determining the calibration parameters of the acquisition device. The principle of information is explained:

Exemplarily, in the initial calibration process of the acquisition device, the target vehicle can be parked between parallel lane lines, and the parallel lane lines in the distance intersect at a point when the phase plane of the acquisition device is projected, which can be called the disappearance of the lane lines. The vanishing point of the lane line approximately coincides with point V in Figure 9. The vanishing point of the lane line can be used to represent the projection position of the acquisition device in the monitoring image, and the pixel coordinate value of the vanishing point of the lane line can indicate that the acquisition device is in the current frame. Monitor pixel coordinate values in an image.

As shown in Figure 10, the distance between the two points EG can represent the actual height H _c of the acquisition device relative to the ground; the distance between the two points FG can represent the actual height _Hy of the object relative to the ground; the distance between the two points MN The distance of MV can represent the pixel height h _b of the object relative to the ground; the distance between the two points of MV can represent the pixel height of the acquisition device relative to the ground.

Further, as shown in FIG. 10 , according to the principle of pinhole imaging, it is determined that the ratio between the actual height H _c of the acquisition device relative to the ground and the actual height H _y of the object relative to the ground when the acquisition device captures the monitoring image of the current frame is equal to The ratio of the pixel height of the acquisition device relative to the ground and the pixel height h _b of the object relative to the ground. In this way, after determining the pixel coordinate values of points M, V and N, it is possible to further determine the distance of the marked object relative to the ground. The pixel height h _b and the pixel height of the acquisition device relative to the ground are used to predict the actual height H _y of the object relative to the ground.

Further, after the actual height of the object relative to the ground is predicted, the current second distance information can be determined in combination with the above formula (3).

The principle of determining the current second distance information has been introduced above in conjunction with FIG. 10 , and the following describes the specific process of how to determine the current second distance information in conjunction with FIG. 11 :

As shown in FIG. 11 , after the current frame monitoring image is subjected to de-distortion processing, an image coordinate system is established for the current frame monitoring image, and the pixel coordinate values (x _v , y _v ) of the vanishing point V of the road line are marked in the image coordinate system ); the pixel coordinate value of the upper left point A of the detection frame of the object (x _tl , y _tl ), the pixel coordinate value of the lower right point C (x _br , y _br ), and further, you can pass the corner point AC along the y-axis The pixel coordinate value in the direction determines the distance between the two points MN as shown in Figure 9; the distance between the two points MV as shown in Figure 9 can be determined by the pixel coordinate value of the corner point CV along the y-axis direction.

Specifically, the calibration parameters of the collection device include the first height value of the collection device relative to the ground and the focal length of the collection device; for the above S7022, in the pixel coordinate value based on the set corner point, the calibration parameters of the collection device, and the determination of the collection device The pixel coordinate value of the vanishing point of the lane line used in the calibration parameters of the

S70221, based on the pixel coordinate value of the vanishing point of the lane line and the pixel coordinate value of the corner point set in the detection frame, determine a first pixel height value of the collecting device relative to the ground.

Combined with the above figure 11, the first pixel height value can be obtained: y _br -y _v .

S70222: Determine, based on the pixel coordinate value of the set corner point, a second pixel height value of the object in the monitoring image of the current frame relative to the ground.

For example, the difference between the pixel coordinate values along the y-axis of the two corners of AC in the above-mentioned FIG. 11 may be used as the second pixel height value here, which may be represented by h _b .

S70223: Determine a second height value of the object relative to the ground based on the first pixel height value, the second pixel height value, and the first height value.

Among them, H _c represents the first height value, which is used to represent the actual height of the acquisition device relative to the ground, which can be obtained when the acquisition device is calibrated; H _y represents the second height value, which is used to represent the actual height of the object relative to the ground. .

S70224: Determine current second distance information based on the second height value, the focal length of the acquisition device, and the second pixel height value.

Exemplarily, the current second distance information may be determined by the above formula (3).

In the embodiment of the present disclosure, under the condition that the complete detection frame corresponding to the object in the monitoring image of the current frame can be detected, the actual detection frame of the object can be obtained quickly and accurately by introducing the pixel coordinate value of the vanishing point of the lane line and the calibration parameters of the acquisition device. The height value can further quickly and accurately determine the current second distance information between the target vehicle and the object.

After obtaining the current second distance information between the target vehicle and the object, for the above S603, when determining the current first distance information based on the adjusted distance information, the following steps S6031 to S6032 are included:

S6031, based on the current second distance information, the historical second distance information between the object and the target vehicle in each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, and the historical first distance information corresponding to the frame of the historical frame monitoring image and the adjusted distance information to determine distance offset information for the adjusted distance information.

Exemplarily, the current second distance information and each historical second distance information are the distance information between the target vehicle and the object determined based on the single-frame monitoring image. The accurate and complete detection frame of the object can be based on the position information of the detection frame to obtain the second distance information with high accuracy between the target vehicle and the object. On the contrary, if the accurate detection frame of the object cannot be detected, or the detected object is The detection frame is incomplete, and the accuracy of the obtained second distance information is low. Therefore, the accuracy of the plurality of second distance information determined based on this method is high, but the fluctuation is large.

Exemplarily, each historical first distance information is distance information determined based on multiple frames of monitoring images, and the adjusted distance information is also distance information adjusted based on multiple historical first distance information. Therefore, based on this method The fluctuation between the obtained multiple historical first distance information and the adjusted distance information is small, but because the scale changes corresponding to two adjacent frames of monitoring images are used when determining the historical first distance information and the adjusted distance information The process of determining the scale change information depends on the position information of the feature points of the recognized object in the monitoring image. When there is an error, the error will be accumulated. Therefore, the determined multiple historical first distance information and adjusted distance information The accuracy is compared to the accuracy of the second distance information determined based on the complete detection frame.

Considering that the accuracy of the current second distance information and the historical second distance information determined based on the detection frame is high, and the stability between the historical first distance information determined based on the scale change information and the adjusted distance information is high, in order to obtain For the current first distance information with high accuracy and stability, the adjusted distance information can be further adjusted for the distance information between the target vehicle and the object corresponding to the determined multi-frame monitoring images respectively in two ways.

S6032: Adjust the adjusted distance information based on the distance offset information to obtain current first distance information.

Exemplarily, after the distance offset information is obtained, the adjusted distance information may be further adjusted based on the distance offset information, so that the current first distance information is more accurate.

In one embodiment, based on the current second distance information, the historical second distance information between the object and the target vehicle in each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, the historical frame monitoring image corresponding to the frame When determining the distance offset information for the adjusted distance information, the following steps S60311 to S60313 may be included:

S60311, based on the current second distance information and the historical second distance information corresponding to each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, determine the history corresponding to each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image The first linear fitting coefficient of the first fitting curve fitted by the second distance information and the current second distance information.

Exemplarily, the current second distance information can be represented by D ₀ , and a plurality of historical second distance information can be represented by D ₁ , D ₂ , D ₃ . . . respectively, which can be performed by D ₀ and D ₁ , D ₂ , D ₃ . Linear fitting is performed to obtain a first fitting curve composed of multiple historical second distance information and current second distance information, and the first fitting curve can be represented by the following formula (6):

y ₁ =ax+bx ² +c (6);

In the fitting process, the frame numbers 0, 1, 2, 3 of the monitoring images used in determining the plurality of second distance information can be used as the x value, and the second distance information D ₀ , D corresponding to the frame numbers respectively ₁ , D ₂ , D ₃ , .

S60312, based on the historical first distance information and the adjusted distance information corresponding to each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, determine the history corresponding to each historical frame monitoring image in the multi-frame historical frame monitoring image A second linear fitting coefficient of the second fitting curve fitted by the first distance information and the adjusted distance information.

Exemplarily, the adjusted distance information can be represented by D _{0_scale} , and a plurality of historical first distance information can be represented by D _{1_final} , D _{2_final} , D _{3_final} . . respectively, which can be performed by D _{0_scale} and D _{1_final} , D _{2_final} , D _{3_final} . Linear fitting is performed to obtain a second fitting curve composed of multiple historical first distance information and adjusted distance information, and the second fitting curve can be expressed by the following formula (7):

y ₂ =a'x+b'x ² +c'(7);

During the fitting process, the frame numbers 0, 1, 2, 3 of the monitoring images used in determining the multiple historical first distance information and the adjusted distance information can be used as the x value, and the adjustment corresponding to the frame number respectively The latter distance information and a plurality of historical first distance information D _{0_scale} , D _{1_final} , D _{2_final} , D _{3_final} . .

S60313: Determine distance offset information for the adjusted distance information based on the first linear fitting coefficient and the second linear fitting coefficient.

Exemplarily, the distance offset information can be determined by the following formula (8):

L=(a/a′+b/b′+c/c′)/3 (8);

The distance offset information determined in this way can be adjusted according to the following formula (9) to obtain the current first distance information D _{0_final} :

D _{0_final} =D _{0_scale} ×L (9);

In another embodiment, based on the current second distance information, the historical second distance information between the object and the target vehicle in each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, the historical frame monitoring image of the frame The corresponding historical first distance information and the adjusted distance information, when determining the distance offset information for the adjusted distance information, can also be determined by the Kalman filtering algorithm, and further determine the current first distance based on the Kalman filtering algorithm. information.

When the current first distance information is determined based on the Kalman filter algorithm, it can be determined by the following formula (10):

D _{0_final} = kal(D _{0_scale} , D ₀ , R, Q) (10);

Among them, R represents the variance of D _{0_scale} and D _{1_final} , D _{2_final} , D _{3_final} . . . Q represents the variance of D ₀ and D ₁ , D ₂ , D ₃ _. , and further correct the adjusted distance information based on the distance offset information to obtain the current first distance information with higher accuracy.

After the current first distance information is determined, the target object located in the blind spot of the target vehicle can be determined according to each object in the current frame image and the current first distance information of the target vehicle.

Exemplarily, the method of determining the blind spot of the target vehicle may be shown in FIG. 2 above, and the blind spot of the vision corresponding to the target vehicle may be determined. In addition, by using the current first distance information and the determined blind spot of the target vehicle, it is possible to determine the object that falls into the blind spot of the visual field among the detected objects, that is, the target object.

Here, the target object may include, for example, other driving vehicles, pedestrians, road facilities, and parts of road obstacles among the above-mentioned objects.

For the above S104, the monitoring result can be generated according to the type information and position of the target object and the driving state of the target vehicle.

Here, since the generated monitoring result is determined according to the image, when using the monitoring result to guide the automatic driving vehicle or assist the driver to drive, it is usually necessary to continuously generate the monitoring result for a continuous period of time. During this continuous time, multiple frames of images will be acquired, but the multiple frames of images are discrete relative to this continuous time. In order to obtain more accurate monitoring of the target object, tracking and smoothing processing can also be performed when the continuous frame image is monitored for the object, for example, interpolation is used to further improve the accuracy.

In addition, since the tracking and smoothing method can use multiple frames of monitoring images corresponding to discrete time to determine the distance between the target object and the target vehicle in continuous time, this method can also alleviate the need for the acquisition device to acquire multiple frames of monitoring images in rapid succession. pressure and reduce equipment wear.

Exemplarily, in order to make the obtained monitoring results less discrete, so as to ensure the safety of the target vehicle while driving, it is necessary to acquire a frame of monitoring images in 0.1 seconds; if a frame of monitoring images is acquired in 0.5 seconds, In a fast-speed scenario, a sudden impact may occur within 0.5 seconds, which means safety cannot be guaranteed. However, the frequency of acquiring a frame of monitoring images in 0.1 seconds requires more power consumption for the acquisition device than the frequency of acquiring a frame of monitoring images in 0.2 seconds. Within 0.1 seconds, a relatively accurate frame of predictive monitoring images can ensure safety.

In addition, after the target object is determined, different target objects in different positions have different effects on the target vehicle. Therefore, it is also possible to combine the type information of the target object, the location, and the driving state of the target vehicle to determine the different target objects. corresponding monitoring results.

Specifically, the monitoring result may include alarm information, for example. The driving state of the target vehicle may include, for example, steering information of the target vehicle.

In a specific implementation, when generating the monitoring result according to the type information and position of the target object and the driving state of the target vehicle, for example, the following method may be used: determine the warning information according to the type information and position of the target object and the steering information of the target vehicle level; generate alarm information of a certain level and prompt.

The type information of the target object may include pedestrians, for example. Since the target object is located in the blind spot of the target vehicle, that is, the target vehicle may affect the driving safety because the target object is located in the blind spot of the target vehicle, the monitoring result including the warning information can be generated.

In a possible implementation, the steering information of the target vehicle indicates that the target vehicle turns left, and the position of the target object indicates that the target object is in the blind spot on the left side of the target vehicle; or, the steering information of the target vehicle indicates that the target vehicle turns right, And the position of the target object indicates that when the target object is in the blind spot on the right side of the target vehicle, it is considered that the target vehicle has a greater impact on the safety of the target object when driving. For example, the target vehicle may collide with pedestrians while driving, and the monitoring results can include the highest level monitoring results. Here, for example, the monitoring results can also be divided into multiple levels, such as level 1, level 2, level 3, and level 4; the higher the level, the greater the impact on the driving safety of the target vehicle, and the corresponding alarm information is also The corresponding representation has a great influence on the driving safety of the target vehicle.

Taking the first monitoring result as an example, the first monitoring result corresponds to, for example, level 1, and includes a "beep" sound at a higher frequency, or a voice prompt message "Currently too close to the vehicle, please drive carefully".

In addition, the first monitoring result may be further refined according to the position of the target object. Take the driving state of the target device as driving to the left, and the position of the target object indicates that there is a target object on the left side of the target vehicle. If the first monitoring result indicates that the target device is constantly approaching the target object, gradually increase the "di ” sound, or generate more accurate warning information such as “currently 1 meter away from the pedestrian on the left” and “currently 0.5 meters away from the pedestrian on the left”.

In another possible implementation, the steering information of the target vehicle indicates that the target vehicle is turning left, and the position of the target object indicates that the target object is in the blind spot on the right side of the target vehicle; or, the steering information of the target vehicle indicates that the target vehicle is turning right , and the position of the target object indicates that when the target object is in the blind spot on the left side of the target vehicle, it is believed that the target vehicle has a considerable probability of having a certain impact on safety when driving. For example, when pedestrians approach the target vehicle, collision may occur, then monitor The results can correspond to Level 2, including a “beep” sound at a lower frequency than the monitoring results of the corresponding Level 1, or a voice prompt message “Currently close to pedestrians, please drive carefully”.

In addition, the type information of the target object may also include, for example, a vehicle; here, the vehicle is a vehicle other than the target vehicle.

Similar to the above-mentioned method for determining the monitoring result when the type information indicates that the target object is a pedestrian, in a possible implementation, the steering information of the target vehicle indicates that the target vehicle turns left, and the current position of the target object indicates that the target object is in The blind spot on the left side of the target vehicle; or, when the steering information of the target vehicle indicates that the target vehicle is turning right, and the position of the current target object indicates that the target object is in the blind spot on the right side of the target vehicle, it is considered that the target vehicle has a greater impact on safety when driving. For example, the target vehicle may collide with other vehicles when turning, and the monitoring results may include monitoring results corresponding to three levels.

In addition, the monitoring results can be further refined according to the driving state. Take the driving state of the target vehicle as an example of the target vehicle turning left, and the monitoring result representing the target object on the left side of the target vehicle. frequency, or generate alarm information with more accurate prompt information, such as "currently 1 meter away from the left vehicle" and "currently 0.5 meters away from the left vehicle".

In another possible implementation, the steering information of the target vehicle indicates that the target vehicle is turning left, and the position of the target object indicates that the target object is in the blind spot on the right side of the target vehicle; or, the steering information of the target vehicle indicates that the target vehicle is turning right , and the position of the target object indicates that when the target object is in the blind spot on the left side of the target vehicle, it is believed that the target object has a considerable probability of affecting the safety of the target vehicle when driving. For example, when the target vehicle is driving, other vehicles may If the vehicle collides, the monitoring result can correspond to Level 4, including a “beep” sound at a lower frequency than the monitoring result corresponding to Level 3, or a voice prompt message “Currently close to the vehicle, please drive carefully”.

In this way, by generating alarm information more accurately and quickly, the driver who controls the target vehicle can be guided to drive more safely.

In addition, the monitoring results may also include vehicle control instructions, for example. Correspondingly, the driving state of the target vehicle may include, for example, steering information of the target vehicle.

Here, since the type information and position of the target object can be acquired efficiently and accurately, monitoring results including vehicle control instructions can also be generated to control safe driving of the traveling device and the like. The traveling device is, for example, but not limited to, any of the following: an autonomous vehicle, a vehicle equipped with an advanced driving assistance system (Advanced Driving Assistance System, ADAS), or a robot.

In a specific implementation, when generating the monitoring result according to the type information and position of the target object and the driving state of the target vehicle, for example, the vehicle control instruction can be generated according to the type information and position of the target object and the steering information of the target vehicle.

Here, when generating the vehicle control command according to the type information and position of the target object and the steering information of the target vehicle, for example, the target vehicle can determine the generated vehicle control command according to the type information and position of the target object, so as to ensure that the target vehicle can avoid Collision with the target object to ensure safe driving.

In this way, the monitoring results are more conducive to deployment in the intelligent driving device, improve the safety of the intelligent driving device during the automatic driving control process, that is, can better meet the needs of the automatic driving field.

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.

Based on the same technical concept, the embodiment of the present disclosure also provides a blind spot monitoring device corresponding to the blind spot monitoring method. Reference may be made to the implementation of the method, and repeated descriptions will not be repeated.

Referring to FIG. 12 , which is a schematic diagram of a blind spot monitoring device provided by an embodiment of the present disclosure, the blind spot monitoring device includes: an acquisition module 121 , a detection module 122 , a determination module 123 , and a generation module 124 ; wherein,

an acquisition module 121, configured to acquire the current frame monitoring image acquired by the acquisition device on the target vehicle;

A detection module 122, configured to perform object detection on the current frame monitoring image to obtain type information and position of the object included in the image;

A determination module 123, configured to determine the target object located in the blind spot of the target vehicle according to the position of the object and the blind spot of the target vehicle;

The generating module 124 is configured to generate monitoring results according to the type information and position of the target object and the driving state of the target vehicle.

In a possible implementation manner, the monitoring result includes warning information, and the driving state of the target vehicle includes steering information of the target vehicle; the generating module 124 is based on the type information and location of the target object and The driving state of the target vehicle, when generating the monitoring result, is used to: determine the level of the alarm information according to the type information and position of the target object and the steering information of the target vehicle; generate the alarm information of the determined level and prompt .

In a possible implementation manner, the monitoring result includes vehicle control instructions, and the driving state of the target vehicle includes steering information of the target vehicle; the generation module 124 is based on the type information and position of the target object. and the driving state of the target vehicle, when generating the monitoring result, it is used to: generate the vehicle control instruction according to the type information and position of the target object and the steering information of the target vehicle; the blind spot monitoring device further includes: The control module 125 is configured to: control the target vehicle to travel based on the vehicle control instruction.

In a possible implementation manner, when determining the target object located in the blind spot of the target vehicle according to the position of the object and the blind spot of the target vehicle, the determining module 123 is configured to: The position of the object in the current frame monitoring image is determined, and the current first distance information between the target vehicle and the object in the current frame monitoring image is determined; according to the current first distance information, it is determined that the target vehicle is located in the target vehicle. target object in the blind spot of the field of view.

In a possible implementation manner, the determining module 123 determines the current first distance between the target vehicle and the object in the monitoring image of the current frame according to the position of the object in the monitoring image of the current frame information is used to: determine the distance information to be adjusted between the target vehicle and the object in the current frame monitoring image based on the current frame monitoring image; based on the multiple frames collected by the object in the collecting device The scale change information between the scales in the two adjacent frames of monitoring images in the historical frame monitoring images, and the relationship between the object and the target vehicle in each frame of the historical frame monitoring images in the multi-frame historical frame monitoring images. The historical first distance information is adjusted, and the to-be-adjusted distance information is adjusted to obtain the current first distance information between the target vehicle and the object.

In a possible implementation manner, when the determining module 123 adjusts the distance information to be adjusted to obtain the current first distance information between the target vehicle and the object, the determining module 123 is configured to: The distance information to be adjusted is adjusted until the error amount of the scale change information is the smallest, and the adjusted distance information is obtained; wherein the error amount is based on the distance information to be adjusted, the scale change information and the multi-frame history The historical first distance information corresponding to each frame of the historical frame monitoring image in the frame monitoring image is determined; and the current first distance information is determined based on the adjusted distance information.

In a possible implementation manner, before determining the current first distance information based on the adjusted distance information, the determining module 123 is further configured to: monitor the distance of the object in the image based on the current frame position, and the calibration parameters of the collection device, to determine the current second distance information; the determining module 123, when determining the current first distance information based on the adjusted distance information, is used for: based on the current second distance information, historical second distance information between the object and the target vehicle in each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, and the history corresponding to the historical frame monitoring image of the frame The first distance information and the adjusted distance information determine distance offset information for the adjusted distance information; adjust the adjusted distance information based on the distance offset information to obtain the current First distance information.

In a possible implementation manner, the determining module 123 determines the relationship between the object and the target vehicle in each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image based on the current second distance information. When determining the distance offset information for the adjusted distance information, it is used for : Based on the current second distance information and the historical second distance information corresponding to each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, determine that each frame in the multi-frame historical frame monitoring image is determined by The first linear fitting coefficient of the first fitting curve fitted by the historical second distance information corresponding to the historical frame monitoring image and the current second distance information; The historical first distance information and the adjusted distance information corresponding to the frame historical frame monitoring images, determine the historical first distance information corresponding to each frame of the historical frame monitoring images in the multi-frame historical frame monitoring images a second linear fitting coefficient of the second fitting curve fitted with the adjusted distance information; based on the first linear fitting coefficient and the second linear fitting coefficient, determine The distance offset information of the distance information.

In a possible implementation manner, when the determining module 123 determines the current second distance information based on the position of the object in the monitoring image of the current frame and the calibration parameters of the acquisition device, the determining module 123 is configured to: : based on the position information of the detection frame of the object in the monitoring image of the current frame, obtain the pixel coordinate value of the set corner point in the detection frame; based on the pixel coordinate value of the set corner point, the acquisition The current second distance information is determined by the calibration parameters of the device and the pixel coordinate value of the vanishing point of the lane line used when determining the calibration parameters of the acquisition device.

In a possible implementation manner, the calibration parameters of the acquisition device include a first height value of the acquisition device relative to the ground and a focal length of the acquisition device; the determining module 123 is based on the set corner point The pixel coordinate value of the collection device, the calibration parameter of the collection device, and the pixel coordinate value of the vanishing point of the lane line used when determining the calibration parameter of the collection device, when determining the current second distance information, for: based on the The pixel coordinate value of the vanishing point of the lane line and the pixel coordinate value of the corner point set in the detection frame, determine the first pixel height value of the collection device relative to the ground; based on the pixel coordinate value of the set corner point, determining a second pixel height value of the object in the monitoring image of the current frame relative to the ground; determining the object based on the first pixel height value, the second pixel height value and the first height value a second height value relative to the ground; determining the current second distance information based on the second height value, the focal length of the acquisition device, and the second pixel height value.

In a possible implementation manner, when determining the distance information to be adjusted between the target vehicle and the object in the current frame monitoring image based on the current frame monitoring image, the determining module 123 is configured to: Obtain the scale change information between the scale of the object in the current frame monitoring image and the scale in the historical frame monitoring image adjacent to the current frame monitoring image; The historical first distance information corresponding to the historical frame monitoring image adjacent to the current frame monitoring image is determined, and the to-be-adjusted distance information is determined.

In a possible implementation manner, the determining module 123 determines the scale change information between the scales of the object in two adjacent frames of monitoring images in the following manner: extracting a plurality of feature points contained in the object respectively in The first position information in the previous frame of monitoring images in the two adjacent frames of monitoring images, and the second position information in the next frame of monitoring images; based on the first position information and the second position information, The scale change information between the scales of the object in two adjacent frames of monitoring images is determined.

In a possible implementation manner, the determining module 123 determines, based on the first position information and the second position information, when determining the scale change information of the object between scales in two adjacent frames of monitoring images , for: determining, based on the first position information, the first scale value of the target line segment composed of multiple feature points contained in the object in the monitoring image of the previous frame; based on the second position information, Determine the second scale value of the target line segment in the next frame of monitoring images; based on the first scale value and the second scale value, determine the difference between the scales of the object in two adjacent frames of monitoring images Scale change information between.

Corresponding to the blind spot monitoring method in FIG. 1 , an embodiment of the present disclosure further provides an electronic device 1300 . As shown in FIG. 13 , a schematic structural diagram of the electronic device 1300 provided by an embodiment of the present disclosure includes:

The processor 10, the memory 20, and the bus 30; the memory 20 is used to store the execution instructions, including the memory 210 and the external memory 220; the memory 210 here is also called the internal memory, which is used to temporarily store the operation data in the processor 10, and For data exchanged by an external memory 220 such as a hard disk, the processor 10 exchanges data with the external memory 220 through the memory 210. When the electronic device 1300 is running, the processor 10 and the memory 20 communicate through the bus 30, so that the processor 10 executes the following instructions : obtain the current frame monitoring image collected by the acquisition device on the target vehicle; perform object detection on the current frame monitoring image to obtain the type information and position of the object included in the current frame monitoring image; according to the position of the object and the blind spot of the target vehicle, determine the target object located in the blind spot of the target vehicle; generate monitoring results according to the type information and position of the target object and the driving state of the target vehicle.

Embodiments of the present disclosure further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the blind spot monitoring method described in the above method embodiments are executed. Wherein, the storage medium may be a volatile or non-volatile computer-readable storage medium.

Embodiments of the present disclosure further provide a computer program product, where the computer program product carries program codes, and the instructions included in the program codes can be used to execute the steps of the blind spot monitoring method described in the foregoing method embodiments. For details, please refer to the foregoing method. The embodiments are not repeated here.

Wherein, the above-mentioned computer program product can be specifically implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK), etc. Wait.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can be referred to the corresponding process in the foregoing method embodiments, which will not be repeated here. In the several embodiments provided by the present disclosure, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces, indirect coupling or communication connection of devices or units, which may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a processor-executable non-volatile computer-readable storage medium. Based on such understanding, the technical solutions of the present disclosure can be embodied in the form of software products in essence, or the parts that contribute to the prior art or the parts of the technical solutions. The computer software products are stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present disclosure, and are used to illustrate the technical solutions of the present disclosure, but not to limit them. The protection scope of the present disclosure is not limited to this, although the aforementioned The embodiments describe the present disclosure in detail, and those skilled in the art should understand that: any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure. Or can easily think of changes, or equivalently replace some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered in the present disclosure. within the scope of protection. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

A blind spot monitoring method, comprising:

Obtain the current frame monitoring image collected by the collection device on the target vehicle;

Perform object detection on the current frame monitoring image to obtain the type information and position of the object included in the current frame monitoring image;

According to the position of the object and the blind spot of the target vehicle, determine the target object located in the blind spot of the target vehicle;

The monitoring result is generated according to the type information and position of the target object and the driving state of the target vehicle.
The blind spot monitoring method according to claim 1, wherein the monitoring result includes warning information, and the driving state of the target vehicle includes steering information of the target vehicle;

The generating monitoring results according to the type information and position of the target object and the driving state of the target vehicle, including:

Determine the level of the warning information according to the type information and position of the target object and the steering information of the target vehicle;

Generate alarm information of a certain level and prompt it.
The blind spot monitoring method according to claim 1, wherein the monitoring result includes a vehicle control command, and the driving state of the target vehicle includes steering information of the target vehicle;

The generating monitoring results according to the type information and position of the target object and the driving state of the target vehicle, including:

generating the vehicle control instruction according to the type information and position of the target object and the steering information of the target vehicle;

The blind spot monitoring method further includes: controlling the target vehicle to drive based on the vehicle control instruction.
The blind spot monitoring method according to any one of claims 1-3, wherein determining the target object located in the blind spot of the target vehicle according to the position of the object and the blind spot of the target vehicle, comprising:

Determine the current first distance information between the target vehicle and the object in the current frame monitoring image according to the position of the object in the current frame monitoring image;

According to the current first distance information, the target object located in the blind spot of the target vehicle is determined.
The blind spot monitoring method according to any one of claims 1-4, wherein the target vehicle and the object in the current frame monitoring image are determined according to the position of the object in the current frame monitoring image. The current first distance information of the object, including:

Based on the current frame monitoring image, determining the distance information to be adjusted between the target vehicle and the object in the current frame monitoring image;

Based on the scale change information between the scales in two adjacent frames of monitoring images in the multi-frame historical frame monitoring images collected by the acquisition device, and the monitoring image of each historical frame in the multi-frame historical frame monitoring images The historical first distance information between the object and the target vehicle in , adjust the to-be-adjusted distance information to obtain the current first distance information between the target vehicle and the object.
The blind spot monitoring method according to claim 5, wherein the adjusting the distance information to be adjusted to obtain the current first distance information between the target vehicle and the object, comprising:

Adjust the distance information to be adjusted until the error amount of the scale change information is the smallest, and obtain the adjusted distance information; wherein, the error amount is based on the distance information to be adjusted, the scale change information and the Determine the historical first distance information corresponding to each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image;

The current first distance information is determined based on the adjusted distance information.
The blind spot monitoring method according to claim 6, wherein before the current first distance information is determined based on the adjusted distance information, the blind spot monitoring method further comprises:

determining the current second distance information based on the position of the object in the monitoring image of the current frame and the calibration parameters of the acquisition device;

The determining the current first distance information based on the adjusted distance information includes:

Based on the current second distance information, the historical second distance information between the object and the target vehicle in each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, the historical frame monitoring image corresponding to the frame The historical first distance information and the adjusted distance information of , determine the distance offset information for the adjusted distance information;

The adjusted distance information is adjusted based on the distance offset information to obtain the current first distance information.
The blind spot monitoring method according to claim 7, wherein the object and the target in each frame of the historical frame monitoring image based on the current second distance information and the multi-frame historical frame monitoring image The historical second distance information between vehicles, the historical first distance information corresponding to the frame of historical frame monitoring images, and the adjusted distance information, and the distance offset information for the adjusted distance information is determined, including :

Based on the current second distance information and the historical second distance information corresponding to each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, determine the historical data of each frame in the multi-frame historical frame monitoring image the first linear fitting coefficient of the first fitting curve fitted by the historical second distance information and the current second distance information corresponding to the frame monitoring image;

Based on the historical first distance information and the adjusted distance information corresponding to each frame of the historical frame monitoring image in the multi-frame historical frame monitoring image, determine the historical first distance information of each frame in the multi-frame historical frame monitoring image a second linear fitting coefficient of a second fitting curve fitted by the historical first distance information corresponding to the frame monitoring image and the adjusted distance information;

Based on the first linear fitting coefficient and the second linear fitting coefficient, distance bias information for the adjusted distance information is determined.
The blind spot monitoring method according to claim 7 or 8, wherein the current second distance information is determined based on the position of the object in the current frame monitoring image and the calibration parameters of the acquisition device ,include:

Based on the position information of the detection frame of the object in the current frame monitoring image, obtain the pixel coordinate value of the set corner in the detection frame;

The current second distance information is determined based on the pixel coordinate value of the set corner point, the calibration parameter of the collection device, and the pixel coordinate value of the vanishing point of the lane line used in determining the calibration parameter of the collection device.
The blind spot monitoring method according to claim 9, wherein the calibration parameters of the acquisition device include a first height value of the acquisition device relative to the ground and a focal length of the acquisition device;

The current second distance is determined based on the pixel coordinate value of the set corner point, the calibration parameter of the acquisition device, and the pixel coordinate value of the vanishing point of the lane line used when determining the calibration parameter of the acquisition device. information, including:

Determine the first pixel height value of the collection device relative to the ground based on the pixel coordinate value of the vanishing point of the lane line and the pixel coordinate value of the corner point set in the detection frame;

determining a second pixel height value of the object in the current frame monitoring image relative to the ground based on the pixel coordinate value of the set corner;

determining a second height value of the object relative to the ground based on the first pixel height value, the second pixel height value, and the first height value;

The current second distance information is determined based on the second height value, the focal length of the acquisition device, and the second pixel height value.
The blind spot monitoring method according to any one of claims 5 to 10, wherein the to-be-adjusted distance between the target vehicle and the object in the current frame monitoring image is determined based on the current frame monitoring image Distance information, including:

Obtaining the scale change information between the scale of the object in the current frame monitoring image and the scale in the historical frame monitoring image adjacent to the current frame monitoring image;

The to-be-adjusted distance information is determined based on the scale change information and the historical first distance information corresponding to historical frame monitoring images adjacent to the current frame monitoring image.
The blind spot monitoring method according to any one of claims 5 to 11, wherein the scale change information between the scales of the object in two adjacent frames of monitoring images is determined in the following manner:

Respectively extract the first position information of the plurality of feature points contained in the object in the previous frame of the monitoring image in the two adjacent frames of the monitoring image, and the second position information in the next frame of the monitoring image;

Based on the first position information and the second position information, the scale change information between the scales of the object in two adjacent frames of monitoring images is determined.
The blind spot monitoring method according to claim 12, wherein the scale between the scales of the object in two adjacent frames of monitoring images is determined based on the first position information and the second position information Change information, including:

determining, based on the first position information, the first scale value of the target line segment formed by a plurality of feature points included in the object in the monitoring image of the previous frame;

Based on the second position information, determine the second scale value of the target line segment in the monitoring image of the next frame;

Based on the first scale value and the second scale value, the scale change information between the scales of the object in two adjacent frames of monitoring images is determined.
A blind spot monitoring device, comprising:

an acquisition module, used to acquire the current frame monitoring image acquired by the acquisition device on the target vehicle;

a detection module, configured to perform object detection on the current frame monitoring image to obtain type information and position of the object included in the image;

a determining module, configured to determine the target object located in the blind spot of the target vehicle according to the position of the object and the blind spot of the target vehicle;

The generating module is configured to generate monitoring results according to the type information and position of the target object and the driving state of the target vehicle.
An electronic device, characterized in that it includes: a processor, a memory, and a bus, the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate with each other. The machine-readable instructions execute the steps of the blind spot monitoring method according to any one of claims 1 to 13 when the machine-readable instructions are executed by the processor.
A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, the steps of the blind spot monitoring method according to any one of claims 1 to 13 are executed .
A computer program product, characterized in that, the computer program product includes a computer program, and the computer program executes the steps of the blind spot monitoring method according to any one of claims 1 to 13 when the computer program is run by a processor.