CN114255452A

CN114255452A - Target ranging method and device

Info

Publication number: CN114255452A
Application number: CN202011003926.8A
Authority: CN
Inventors: 程光亮; 石建萍
Original assignee: Shanghai Sensetime Lingang Intelligent Technology Co Ltd
Current assignee: Shanghai Sensetime Lingang Intelligent Technology Co Ltd
Priority date: 2020-09-22
Filing date: 2020-09-22
Publication date: 2022-03-29

Abstract

The disclosure relates to the technical field of computer vision, and particularly provides a target ranging method and device. The target ranging method comprises the following steps: acquiring a target image acquired by acquisition equipment; detecting the target image to obtain the category of each target object in the target image and the position of each target object in the target image; determining the actual height of each target object according to the category of each target object in the target image; and determining the distance between each target object and the acquisition equipment according to the actual height of each target object and the position of each target object in the target image. The distance measurement method disclosed by the invention improves the accuracy and robustness of distance measurement.

Description

Target ranging method and device

Technical Field

The disclosure relates to the technical field of computer vision, in particular to a target ranging method, a target ranging device, electronic equipment and a storage medium.

Background

In recent years, autonomous driving techniques mainly based on deep learning have been greatly developed, and target ranging is one of the most important research directions in the field of autonomous driving. The automatic driving system as a whole generally comprises a sensing module at the front end and a subsequent distance measurement module, wherein the sensing module is used for collecting road condition information and identifying a target, and the distance measurement module is used for measuring the distance of the identified target. However, in the related art, the sensing module only focuses on the identification of the target, and provides less effective information for the subsequent ranging, so that the ranging accuracy is difficult to meet the actual requirement.

Disclosure of Invention

In order to improve the accuracy of target ranging, the embodiments of the present disclosure provide a target ranging method, an apparatus, an electronic device, and a storage medium.

In a first aspect, an embodiment of the present disclosure provides a target ranging method, including:

acquiring a target image acquired by acquisition equipment;

detecting the target image to obtain the category of each target object in the target image and the position of each target object in the target image;

determining the actual height of each target object according to the category of each target object in the target image;

and determining the distance between each target object and the acquisition equipment according to the actual height of each target object and the position of each target object in the target image.

In some embodiments, the determining a distance between each target object and the acquisition device according to the actual height of each target object and the position of each target object in the target image includes:

determining the pixel height of each target object in the target image according to the position of each target object in the target image;

and determining the distance between each target object and the acquisition equipment according to the actual height of each target object, the pixel height of each target object and the focal length of the acquisition equipment.

In some embodiments, the acquisition device is disposed on a smart driving device, and after the determining the distance between each target object and the acquisition device, the method further comprises:

determining the distance between each target object and the intelligent driving equipment according to the distance between each target object and the acquisition equipment;

and when the distance between each target object and the intelligent driving equipment is smaller than a preset distance threshold value, controlling the intelligent driving equipment.

and when the distance between each target object and the intelligent driving equipment is smaller than a preset distance threshold value, the intelligent driving equipment sends out alarm information.

In some embodiments, the determining the actual height of each target object according to the category of each target object in the target image includes:

and determining the actual height of each target object according to the corresponding relation between different types of the actual heights and the type of each target object.

In some embodiments, the detecting the target image to obtain the category of each target object in the target image includes:

detecting the target images, and determining a first category to which each target object in the target images belongs and a second category of the vehicles in the target images, wherein the first category comprises vehicles and non-vehicles, and the second category is a sub-category of the first category.

In some embodiments, the detecting the target image, obtaining the category of each target object in the target image, and the position of each target object in the target image are performed by a neural network;

the main network in the neural network is used for extracting the features of the target image to obtain image features;

a first classification branch in the neural network, configured to determine, according to the image features, the first class to which each target object in the target image belongs;

a fine classification branch in the neural network for determining the second class of vehicles in the target image from the image features;

and the regression branch in the neural network is used for determining the position of each target object in the target image according to the image characteristics.

In a second aspect, the present disclosure provides a target ranging apparatus, including:

the acquisition module is used for acquiring a target image acquired by the acquisition equipment;

the detection module is used for detecting the target image to obtain the category of each target object in the target image and the position of each target object in the target image;

the height determining module is used for determining the actual height of each target object according to the category of each target object in the target image;

and the first distance determining module is used for determining the distance between each target object and the acquisition equipment according to the actual height of each target object and the position of each target object in the target image.

In some embodiments, the first distance determining module is specifically configured to:

In some embodiments, the collecting device is disposed on the intelligent driving device, and the apparatus further includes:

the second distance determining module is used for determining the distance between each target object and the intelligent driving equipment according to the distance between each target object and the acquisition equipment;

and the control module is used for controlling the intelligent driving equipment when the distance between each target object and the intelligent driving equipment is smaller than a preset distance threshold value.

the third distance determining module is used for determining the distance between each target object and the intelligent driving equipment according to the distance between each target object and the acquisition equipment;

and the warning module is used for sending warning information by the intelligent driving equipment when the distance between each target object and the intelligent driving equipment is smaller than a preset distance threshold value.

In some embodiments, the height determination module is specifically configured to:

In some embodiments, the target image is a road image, and the detection module is specifically configured to:

In some embodiments, the detection module is a neural network, and a backbone network in the neural network is configured to perform feature extraction on the target image to obtain an image feature;

In a third aspect, the disclosed embodiments provide an electronic device, including:

a processor; and

a memory communicatively coupled to the processor and storing computer instructions readable by the processor, the computer instructions, when read, causing the processor to perform the method according to any of the embodiments of the first aspect.

In a fourth aspect, the disclosed embodiments provide a storage medium storing computer-readable instructions for causing a computer to perform the method according to any of the embodiments of the first aspect.

The target ranging method comprises the steps of obtaining a target image collected through a collecting device, detecting the target image to obtain a category corresponding to each target object in the target image and the position of each target object in the target image, determining the actual height of each target object according to the category, and further determining the distance between each target object and the collecting device according to the actual height and the position of each target object in the image. The target objects are classified based on the height, and then ranging is carried out according to the actual height and the position corresponding to the classification type, so that the accuracy and the robustness of a ranging network are improved.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a flow chart of a target ranging method according to some embodiments of the present disclosure.

Fig. 2 is a flow chart of a method for ranging a target according to further embodiments of the present disclosure.

Fig. 3 is a schematic diagram of a network structure of a neural network in some embodiments according to the present disclosure.

Fig. 4 is a flow chart of a target ranging method in some embodiments according to the present disclosure.

FIG. 5 is a network training flow diagram of a neural network in some embodiments according to the present disclosure.

Fig. 6 is a flowchart of a target ranging method according to an embodiment of the present disclosure.

Fig. 7 is a flowchart of a target ranging method according to another embodiment of the present disclosure.

Fig. 8 is a flowchart of a target ranging method according to still another embodiment of the present disclosure.

Fig. 9 is a block diagram of a target ranging device according to some embodiments of the present disclosure.

Fig. 10 is a block diagram of a target ranging device according to further embodiments of the present disclosure.

FIG. 11 is a block diagram of a target ranging device in accordance with further embodiments of the present disclosure.

FIG. 12 is a block diagram of a computer system suitable for use in implementing the disclosed target ranging method.

Detailed Description

The technical solutions of the present disclosure will be described clearly and completely with reference to the accompanying drawings, and it is to be understood that the described embodiments are only some embodiments of the present disclosure, but not all embodiments. All other embodiments, which can be derived by one of ordinary skill in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure. In addition, technical features involved in different embodiments of the present disclosure described below may be combined with each other as long as they do not conflict with each other.

The target ranging method provided by the embodiment of the disclosure is applicable to ranging scenes in the field of computer vision, so as to measure the distance between a target object and image acquisition equipment, for example, scenes such as automatic driving, auxiliary driving and monitoring ranging.

Taking an automatic driving scenario as an example, an automatic driving system in the related art generally includes a front-end sensing module and a subsequent distance measuring module. The sensing module is used for collecting road condition information and identifying a target, for example, by collecting a road image, identifying a target (such as a pedestrian, a non-motor vehicle, a motor vehicle, and the like) on the image and then identifying the target by using a detection frame, wherein the common detection frame is a rectangular detection frame. And the ranging module performs ranging on the target identified by the detection frame according to the identification result of the sensing module, for example, ranging is realized by an H matrix ranging method, which is not described in detail in this disclosure.

Therefore, in the ranging method of the related art, a gap exists between the sensing module and the ranging module, that is, the sensing module only focuses on the existence of the target, and cannot provide more auxiliary information for subsequent ranging, which results in poor accuracy and robustness of ranging. For example, when the target vehicle is partially blocked, or the image capturing vehicle is in a bumpy state or an uphill or downhill scene, a larger error exists between the sensing result of the target vehicle on the image and the actual situation, so that a larger error occurs in the subsequent distance measurement result, and even fluctuation of about 30 meters occurs, which obviously fails to meet the distance measurement requirement of the automatic driving system.

Based on the defects in the related art, in a first aspect, the embodiments of the present disclosure provide a target ranging method to improve the accuracy and robustness of target ranging. Some embodiments of the disclosed method are shown in fig. 1.

As shown in fig. 1, in some embodiments, the disclosed target ranging method includes:

and S110, acquiring a target image acquired by the acquisition equipment.

Specifically, the target image may be acquired by an image acquisition device, for example, in an automatic driving scene, the target image may be acquired by a camera provided at a center console of the vehicle; for another example, in a monitoring ranging scene, a target image can be acquired through a monitoring camera; and the like. Those skilled in the art will appreciate that different acquisition devices may be selected for implementation according to the requirements of the ranging scenario, the environment, and other factors, which is not limited in this disclosure.

The target image may be an image of a set direction captured by the above-described capturing device, such as an image of a road directly in front of the vehicle captured by a camera at a center console of the vehicle. The target image may include a single frame image captured by the capturing device, and may also include a frame image in a video stream captured by the capturing device, which is not limited in this disclosure.

The target image comprises each target object, and the target object is a target needing distance measurement. The target object of the present disclosure refers to an object having a height difference, which may be at least one target object of one object type, or at least one target object of a plurality of object types. For example, in the case where only the vehicle is taken as the target object, the target object may be at least one vehicle; for example, in the case where a vehicle, a road block, a non-motor vehicle, or the like is collectively used as a target object, the target object may be at least one vehicle, a road block, or a non-motor vehicle. The present disclosure is not so limited.

In one example, taking an automatic driving scene as an example, a single-frame image shot by a camera of a center console of a current vehicle is taken as a target image, the target image is a road image right in front of the current vehicle, and other vehicles included in the target image are taken as target objects, and the distance measurement between the target vehicle and the current vehicle is realized through the following steps.

And S120, detecting the target image to obtain the category of each target object in the target image and the position of each target object in the target image.

In one example, the target image may be detected by a pre-trained classification network, so as to obtain a class corresponding to each target object output by the classification network. This is explained in detail below, and is not shown here.

It should be noted that "category" as used herein may refer to a category classified for different object types, or may refer to a sub-category classified for the same object type. For example, taking a road image as an example, the target object includes three object categories of "pedestrian", "vehicle", and "obstacle", where the "category" may be a classification category for the three object categories, or may be a sub-category classified under the "vehicle" category, which is not limited by the present disclosure.

In one example, taking an automatic driving scenario as an example, when the target object is a vehicle, since there are many vehicle models and the height difference between vehicles of different models is relatively large, the vehicles can be classified finely according to the models of different vehicles. For example, after the target image is detected, the "car" included in the image corresponds to "category 1", and the "bus" corresponds to "category 6", and the like.

In another example, in some scenarios, when a vehicle and an obstacle are collectively the target object, the category of the target object may represent a coarse category of "vehicle" and "obstacle", and may also represent a sub-category that sub-classifies the vehicle according to the actual height of different vehicles, and a sub-category that sub-classifies the obstacle according to the height of different obstacles.

When the target images are detected and classified, the positions of all target objects in the target images are obtained at the same time. In one example, the positions of the target objects in the target image may be regression predicted through a regression network, so as to obtain the positions of the target objects on the image.

And S130, determining the actual height of each target object according to the category of each target object in the target image.

Specifically, each category of the target objects is preset with a set height, which is used to represent the actual height of the target objects belonging to the category. Still by way of example, if "car" included in the image obtained in step S120 belongs to "category 1", and the target height corresponding to "category 1" is "1.5 m", the actual height corresponding to the "car" is "1.5 m". Taking this as an example, the actual height of each target object can be determined according to the corresponding relationship between the category of each target object on the target image and the actual height.

In one example, a table of correspondence between different types of target objects and actual heights may be preset, and the actual height corresponding to each target object may be obtained through a table lookup method.

And S140, determining the distance between each target object and the acquisition equipment according to the actual height of each target object and the position of each target object in the target image.

And obtaining the distance between each target object and the current acquisition equipment based on the obtained actual height corresponding to each target object and the position in the image. The specific process is described below, and will not be described in detail here.

As can be seen from the above description, the target distance measuring method according to the embodiment of the present disclosure obtains the category of each target object and the position of each target object in the image by detecting the target image, determines the actual height of each target object according to the category, and obtains the distance between each target object according to the actual height and the position. Therefore, follow-up ranging is well determined by utilizing the category obtained by front-end sensing, the actual height is determined based on the classification result to realize ranging, and the robustness to the severe conditions that a target object is partially shielded and the like is good, so that the ranging accuracy is improved.

In some embodiments, in step S140, after obtaining the actual height of each target object, it is necessary to calculate the distance between each target object and the current acquisition device according to the actual height. Fig. 2 shows a specific process of ranging in some embodiments, and as shown in fig. 2, the step S140 specifically includes:

s210, determining the pixel height of each target object in the target image according to the position of each target object in the target image.

S220, determining the distance between each target object and the acquisition equipment according to the actual height of each target object, the pixel height of each target object and the focal length of the acquisition equipment.

Specifically, after the image position of the target object is determined, the pixel height of the target object at the position on the image can be obtained according to the image position, and further, based on the distance measurement principle, the distance between the target object and the acquisition device can be calculated according to the actual height, the pixel height and the focal length of the acquisition device.

In one example, a target object "vehicle a" is included in the target image, and the distance between the current capture device and the vehicle a needs to be determined. After the position of the vehicle a on the target image is obtained, the pixel height H of the vehicle a on the target image can be obtained according to the image position, and meanwhile, the actual height of the vehicle a is obtained as H and the focal length of the acquisition device is obtained as f through the step S130. Thus, the distance Z between the vehicle a and the current acquisition device is represented as:

Z＝fH/h (1)

the distance between the vehicle A and the acquisition equipment can be calculated through the formula (1), and the distance between each target object on the image can be obtained by sequentially calculating each target object through the formula (1).

In some embodiments, on the basis of the above embodiments, the ranging method of the present disclosure includes hierarchical classification ranging, where "hierarchical" refers to not only performing coarse classification on target objects of different object types, but also performing fine classification on target objects of the same object type, thereby achieving more accurate ranging.

For example, in an unmanned scene (including an autonomous vehicle, a robot that can travel on a road, etc.), a capture device on the autonomous vehicle captures an image of the road directly in front of the vehicle as a target image. In detecting the target image, not only rough classification is performed on, for example, "pedestrian", "vehicle", and "non-motor vehicle" included on the target image, but also fine classification is performed on, for example, "car", "van", "bus", and the like included under the category of "vehicle". In brief, the fine classification category is a sub-classification category under the rough classification category, and hereinafter, the rough classification category is defined as a "first category" and the fine classification category is defined as a "second category".

In some embodiments, the hierarchical classification may be implemented by using a neural network, and fig. 3 and 4 illustrate a ranging method and a ranging neural network in some embodiments of the present disclosure, which are described in detail below with reference to fig. 3 and 4.

As shown in fig. 3, the ranging neural network of the present embodiment will be described first. The ranging neural network includes a backbone network 310, which is used to perform feature extraction on an input target image to obtain a feature map of the target image. In one example, backbone network 310 can employ a residual network, a VGG network, GoogleNet, or the like. It will be understood by those skilled in the art that the backbone network 310 can be implemented by using general purpose detectors, such as a two-stage detector, a one-stage detector, etc., and the present disclosure is not limited thereto.

In the present embodiment, a plurality of branch networks, i.e., a first branch 321, a plurality of second branches 322, and a regression branch 323, are connected to the main network 310. The first classification branch 321 is used for performing coarse classification on a plurality of target objects of different object types, so as to obtain a first classification of each target object. The second classification branch 322 is used for performing a fine classification on a plurality of target objects of the same object type to obtain a second class of each target object, that is, the second class is a sub-class of the first class, and the number of the second sub-classification networks 322 may be set according to the number of the object types of the target objects.

In one example, still taking an automatic driving scenario as an example, in a specific scenario, the ranging neural network is utilized to realize ranging for vehicles with different heights. As shown in fig. 4, in this example, the ranging method of the present disclosure includes:

and S410, performing feature extraction on the target image to obtain the image features of the target image.

And S420, predicting to obtain a first category and a second category of each target object in the target image and the position of the target object in the target image based on the image characteristics, wherein the first category comprises a vehicle and a non-vehicle.

Specifically, a neural network shown in fig. 3 is used to perform feature extraction on an input target image through the backbone network 310, so as to obtain a feature map of the target image.

In this example, the target object includes two object types of "vehicle" and "road block", that is, the "vehicle" and the "road block" on the target image need to be subjected to ranging. The first category needs to include at least two categories of "vehicle" and "roadblock", and in each first category, the classification into a plurality of second categories may be preset based on the actual height of the target object, for example, the "vehicle" category includes three second categories of "small vehicle", "medium vehicle", "large vehicle"; the 'roadblock' category comprises three second categories of 'guardrail', 'road rod' and 'green belt'. In the ranging network design, the second classification branch 322 may be set to two, one of which implements the "vehicle" sub-classification and the other implements the "roadblock" sub-classification.

The position of each target object on the image is obtained by the regression branch 323, the first class of each target object is obtained by the first classification branch 321, and the second class of each target object is obtained by the second branch 322.

It should be noted that at least one inventive concept of the ranging method of the present embodiment is: and carrying out hierarchical classification based on the actual height of the target object, thereby realizing distance measurement according to the actual height corresponding to the category of the target object on the image. Thus, in some preferred embodiments, for each subdivided first category, the individual target objects belonging to that first category preferably have a significant difference in height, suitable for classification by actual height. For example, in the above-described embodiments, the "vehicles" may be classified into several second categories according to actual heights of different types of vehicles. Of course, it can be understood by those skilled in the art that, based on the above inventive concept, the first category and the second category may also be any other categories suitable for implementation, and are not limited to the above examples, and the disclosure is not limited thereto.

Still taking the automatic driving scenario as an example, in one embodiment, the target image is a road image, and the target objects on the target image include three categories of "vehicle", "pedestrian", and "non-motor vehicle", and for these three categories of targets, it is obvious that "vehicle" is a category more suitable for being subdivided in height, and "pedestrian" and "non-motor vehicle" are more difficult to be differentiated and classified in height. Thus, in the context of the present embodiment, it may be determined that the first category includes "vehicles", "pedestrians", and "non-motor vehicles", while the second category is classified into the second category only for "vehicles".

When the distance measurement is carried out, a plurality of pedestrians, non-motor vehicles and vehicles are included on the target image acquired by the current acquisition equipment. After the ranging neural network is input, the first classification branch 321 classifies pedestrians, non-motor vehicles and vehicles into three first categories, i.e., "pedestrians", "non-motor vehicles" and "vehicles", and the second classification branch 322 classifies only vehicles into a second category, i.e., "small vehicles", "medium vehicles" and "large vehicles". Further, the distance measurement method can be adopted for realizing distance measurement aiming at each target object under the category of vehicles, and similar technologies can be adopted for realizing distance measurement for pedestrians and non-motor vehicles. It will be understood by those skilled in the art that the details are not repeated herein.

Continuing to refer to fig. 3, regression branch 323 performs regression prediction on the position of each target object on the target image on the image based on the feature map of the target image. In this embodiment, the first classification branch 321 and the plurality of second classification branches 322 share one regression branch 323, that is, for the same target object, the first classification branch 321 and the second classification branch 322 respectively give prediction results, that is, the first class and the second class.

In some embodiments, the disclosed target ranging method further comprises training the network of second classification branches 322. As shown in fig. 5, the method of the present disclosure further includes:

and S510, acquiring a training sample set.

Specifically, each sample in the training sample set includes a sample image and a second class label of a respective target object on the sample image.

In one example, the sample images are road sample images, each road sample image includes at least one vehicle thereon, i.e. the target object is a vehicle, and each vehicle corresponds to a second category tag, e.g. a "car" corresponds to a "car" tag.

S520, inputting the sample set into the backbone network to obtain the image characteristics of the sample image output by the backbone network, and obtaining a second category output by a second classification branch based on the image characteristics of the sample image.

Specifically, each sample image in the sample set is input to, for example, the backbone network 310 shown in fig. 3, and a feature map of each sample image output by the backbone network 310 is obtained. The second classification branch 322 classifies each target object on the sample image according to the image features extracted by the backbone network 310, and obtains a second class of each target object output by the second classification branch 322.

S530, obtaining the loss between the second category label and the second category label, and adjusting the network parameters of the second classification branch according to the loss until the loss meets the convergence condition.

Specifically, after obtaining the second class of each target object output by the second classification branch 322, obtaining the loss according to the difference between the second class and the second class label of the target object, and adjusting the network parameter of the second classification branch 322 according to the loss until the loss satisfies the convergence condition, the training of the second classification branch 322 is completed.

In one example, the second classification branch 322 training is complete when the loss between the second class of the target object and the second class label is less than a preset threshold. For training other networks in the ranging neural network, those skilled in the art can implement the training by referring to the related art, which is not described in detail herein.

The following describes the target ranging method in detail by taking a single visual distance scene in automatic driving as an example. In practical applications, the inventor of the present invention finds, through research, that the height of a vehicle is a relatively accurate index that can be used for implementing monocular distance measurement, and different types of vehicles have obvious differences in height, and can implement hierarchical classified distance measurement.

The target ranging method of the present embodiment includes two stages: the first stage is to preset a scheme for classifying a first class of vehicles; the second phase is ranging based on hierarchical classification.

First, in the first stage, based on vehicle distance measurement, considering that the subsequent distance measurement is based on the vehicle height, vehicles with similar actual heights can be classified into one type. Specifically, the vehicle models currently in common use are classified into the following categories:

class 1: a small car. Such as cars, sports cars, pickup trucks, etc.

Class 2: SUVs, business vehicles, etc.

Class 3: microbus, IVECO, minibus, etc.

Class 4: a small truck.

Class 5: a small box wagon.

Category 6: large box wagons, large trucks, trailers, buses, school buses, and the like.

Class 7: a level tricycle, etc.

Class 8: tricycle with shed, express delivery car, etc.

The above categories 1 to 8 are a plurality of second categories of the vehicle. Of course, it should be understood by those skilled in the art that the above classification categories are merely exemplary, and do not limit the method of the present disclosure, and in other implementation scenarios, other classification categories based on height may also be used, and the present disclosure is not repeated herein.

Based on the classification result, the ranging neural network is trained, and the network training process is as described with reference to the embodiment of fig. 5, which is not described herein again.

After the preset second categories are obtained, a set height needs to be preset for each second category, the set height represents the actual height of the vehicle belonging to the second category, and the actual height can be reasonably set according to actual conditions. In one example, the correspondence of the second category of vehicle to the actual height is shown in table 1:

TABLE 1

Second class	Actual height h
		Class 1	1.5 m
Class 2	1.7 m
		Class 3	1.8 m
Class 4	2.0 m
		Class 5	2.2 m
Category 6	2.8 m
		Class 7	1.0 m
Class 8	1.3 m

Next, in the second stage, as shown in fig. 6, the target ranging method of the present embodiment includes:

s610, acquiring a road image acquired by acquisition equipment on the automatic driving vehicle.

Specifically, a single-frame road image in front of the automatic driving vehicle is acquired through a camera at a center console, and the road image comprises a plurality of target objects such as vehicles, pedestrians and non-motor vehicles.

And S620, extracting the features of the road image to obtain the image features of the road image.

Specifically, the feature of the road image can be obtained by extracting features of the collected road image through the backbone network 310 shown in fig. 3, for example.

S630, predicting the first category of each target object in the road image, the second category of each vehicle in the road image and the position of each target object in the road image based on the image characteristics.

Specifically, in the present embodiment, referring to fig. 3, the structure of the ranging neural network, the first classification branch 321 can predict a first classification result for pedestrians, non-motor vehicles and vehicles on the road image, and since the target ranging in the present embodiment only focuses on vehicles, the ranging for pedestrians and non-motor vehicles can be implemented by using a correlation technique according to the classification result of the first classification branch 321, which is not described herein again. For vehicles on the road image, the second classification branch 322 predicts a second class of each vehicle, i.e., class 1 to class 8 in table 1 above. Meanwhile, the regression branch 323 performs regression prediction on the position of each target object on the image according to the image characteristics.

And S640, respectively determining the actual height of each vehicle according to the corresponding relation between each second type and the actual height.

S650, acquiring the focal length of the acquisition equipment for acquiring the road image, and obtaining the pixel height of each vehicle in the road image according to the position of each vehicle on the road image.

And S660, obtaining the distance between each vehicle and the acquisition equipment of the automatic driving vehicle according to the actual height, the pixel height and the focal length of each vehicle.

Specifically, the actual height H corresponding to each vehicle can be determined through the table 1, the focal length f of the current acquisition device of the autonomous vehicle is obtained, and the pixel height H of each vehicle in the image can be determined according to the position of each vehicle on the road image. The distance Z between the target vehicle and the current autonomous vehicle's collection device can be calculated by equation (1) above.

It should be noted that, for an unmanned driving scene or a vehicle equipped with an Advanced Driver Assistance System (ADAS), after obtaining a distance between a target vehicle and a current vehicle, corresponding control needs to be implemented according to different distances. For example, when the target vehicle is in a short distance, deceleration or avoidance is required to avoid an accident. Thus, two embodiments of the method of the present disclosure are shown in fig. 7 and 8, which are described below in conjunction with fig. 7 and 8, respectively.

As shown in fig. 7, in the unmanned driving scenario, after step S660, the disclosed embodiment further includes:

and S671, determining the distance between each target object and the current intelligent driving equipment according to the distance between each target object and the acquisition equipment. The intelligent driving device comprises an automatic driving vehicle, a robot capable of driving on a road and the like. The following description will take an example in which the intelligent travel device is an autonomous vehicle.

It will be appreciated that for autonomous vehicles, the image acquisition devices are typically located at the center console of the vehicle, and the center console of the vehicle often has a non-negligible distance from the vehicle head. Therefore, after the distance between the target object and the acquisition device is obtained, the distance between each target object and the head of the current automatic driving vehicle needs to be calculated according to the distance. It will be appreciated by those skilled in the art that the present disclosure is not described in detail herein.

And S672, when the distance between each target object and the current intelligent driving device is smaller than a preset distance threshold value, controlling the current intelligent driving device.

Specifically, in the present embodiment, each target object may include only the "vehicle" described above, and may also include some or all of the "vehicle", "pedestrian", and "non-motor vehicle", which is not limited by the present disclosure.

The preset distance threshold value refers to a safe distance traveled by the intelligent traveling device at the current speed, when the distance between a certain target object and the current intelligent traveling device is smaller than the preset distance threshold value, it is indicated that the distance between the target object and the current intelligent traveling device is smaller than the safe distance, and the intelligent traveling device can take corresponding control, such as avoiding measures of speed reduction, braking, lane changing and the like.

In some embodiments, the preset distance threshold may also be a plurality of levels of distance thresholds according to the distance, for example, a first level distance threshold is set at a distance of 50m to 100m or more, a second level distance threshold is set at a distance of 20m to 50m, and a third level distance threshold is set at a distance of 20m or less. When the distance between the target object and the current intelligent driving equipment is in a first-level distance threshold value, the intelligent driving equipment performs deceleration processing; and when the distance is at the third-level distance threshold value, the intelligent driving device brakes. Of course, those skilled in the art will understand that this embodiment is only an example, and in other embodiments, any other control method suitable for implementation may be implemented according to a specific scenario.

As shown in fig. 8, in a scenario of a vehicle with an ADAS installed therein, after step S660, the embodiment of the present disclosure further includes:

and S681, determining the distance between each target object and the vehicle provided with the ADAS according to the distance between each target object and the acquisition equipment.

And S682, when the distance between each target object and the vehicle provided with the ADAS is smaller than a preset distance threshold value, the vehicle provided with the ADAS sends out warning information to prompt a driver to control the vehicle so as to avoid the target object.

Each target object may include only the "vehicle" described above, and may also include several or all of the "vehicle", "pedestrian", and "non-motor vehicle", which the present disclosure does not limit.

The preset distance threshold value is a safe distance for the vehicle to travel at the current speed, when the distance between a certain target object and the vehicle provided with the ADAS is smaller than the preset distance threshold value, the fact that the distance between the target object and the vehicle provided with the ADAS is smaller than the safe distance is indicated, and the vehicle provided with the ADAS can give out warning information such as sound and light to prompt a driver.

In some embodiments, the preset distance threshold may also be a plurality of levels of distance thresholds according to the distance, for example, a first level distance threshold is set at a distance of 50m to 100m or more, a second level distance threshold is set at a distance of 20m to 50m, and a third level distance threshold is set at a distance of 20m or less. When the distance between the target object and the vehicle provided with the ADAS is in the first-level distance threshold value, only warning light reminding is carried out on the driver; when the distance is in the second-level distance threshold value, warning lights and low-frequency sound alarm reminding are simultaneously carried out on the driver; when the distance is in the third-level distance threshold value, the high-frequency warning lamp and the sound alarm are used for reminding a driver at the same time, or a central control system of the vehicle can be directly taken over, so that the vehicle is controlled to avoid a target object. Of course, those skilled in the art will understand that this embodiment is only an example, and in other embodiments, any other control method suitable for implementation may be implemented according to a specific scenario.

According to the ranging method, vehicles are classified in a layering mode, target ranging is achieved according to actual heights corresponding to categories, the fact that whether target objects exist or not is not only paid attention to, accurate auxiliary information is provided for follow-up ranging according to the categories of the target objects, errors can be controlled within a range of 10% through experimental comparison, and accuracy is greatly improved compared with a traditional ranging method. In addition, the method disclosed by the invention realizes distance measurement based on vehicle types, is insensitive to errors caused by vehicle bump, up and down slopes and partial shielding of the vehicle to be measured, has good distance measurement robustness, and meets the distance measurement requirement of an automatic driving system. In the present embodiment, the autonomous driving safety is improved by performing appropriate control and warning on the autonomous vehicle according to the distance to the target object.

In a second aspect, the disclosed embodiments provide a target ranging device. As shown in fig. 9, in some embodiments, the disclosed target ranging apparatus includes:

an obtaining module 110, configured to obtain a target image collected by a collecting device;

a detection module 120, configured to detect the target image, so as to obtain a category of each target object in the target image and a position of each target object in the target image;

a height determining module 130, configured to determine an actual height of each target object according to a category of each target object in the target image;

a first distance determining module 140, configured to determine a distance between each target object and the acquisition device according to the actual height of each target object and the position of each target object in the target image.

In some embodiments, the first distance determining module 140 is specifically configured to:

In some embodiments, the collecting device is disposed on the intelligent driving device, as shown in fig. 10, in this embodiment, the target distance measuring device further includes:

a second distance determining module 150, configured to determine, according to a distance between each target object and the acquisition device, a distance between each target object and the intelligent driving device;

the control module 160 is configured to control the intelligent driving device when a distance between each target object and the intelligent driving device is smaller than a preset distance threshold.

In some embodiments, the collecting device is disposed on the intelligent driving device, as shown in fig. 11, in this embodiment, the target distance measuring device further includes:

a third distance determining module 170, configured to determine a distance between each target object and the intelligent driving device according to a distance between each target object and the collecting device;

and the warning module 180 is configured to send warning information by the intelligent driving device when the distance between each target object and the intelligent driving device is smaller than a preset distance threshold.

In some embodiments, the height determination module 130 is specifically configured to:

In some embodiments, the target image is a road image, and the detection module 120 is specifically configured to:

In some embodiments, the detection module 120 is a neural network, and a backbone network in the neural network is configured to perform feature extraction on the target image to obtain an image feature;

a processor; and

Specifically, fig. 12 shows a schematic structural diagram of a computer system 600 suitable for implementing the method of the present disclosure, and the corresponding functions of the processor and the storage medium can be implemented by the system shown in fig. 12.

As shown in fig. 12, the computer system 600 includes a processor (CPU)601 which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)602 or a program loaded from a storage section 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

The following components are connected to the I/O interface 605: an input portion 606 including a keyboard, a mouse, and the like; an output portion 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 608 including a hard disk and the like; and a communication section 609 including a network interface card such as a LAN card, a modem, or the like. The communication section 609 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 605 as needed. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted in the storage section 608 as necessary.

In particular, the above method processes may be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the above-described method. In such embodiments, the computer program may be downloaded and installed from a network through the communication section 609, and/or installed from the removable medium 611.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

It should be understood that the above embodiments are only examples for clearly illustrating the present invention, and are not intended to limit the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications of the present disclosure may be made without departing from the scope of the present disclosure.

Claims

1. A method for ranging a target, comprising:

acquiring a target image acquired by acquisition equipment;

2. The method of claim 1, wherein determining the distance between each target object and the acquisition device according to the actual height of each target object and the position of each target object in the target image comprises:

3. The method according to claim 1 or 2, wherein the acquisition device is provided on a smart driving device, and after the determining the distance between each target object and the acquisition device, the method further comprises:

4. The method according to claim 1 or 2, wherein the acquisition device is provided on a smart driving device, and after the determining the distance between each target object and the acquisition device, the method further comprises:

5. The method of claim 1, wherein determining the actual height of each target object according to the category of each target object in the target image comprises:

6. The method according to claim 1, wherein the target image is a road image, and the detecting the target image to obtain the category of each target object in the target image comprises:

7. The method of claim 6, wherein the detecting the target image, obtaining the category of each target object in the target image, and the position of each target object in the target image are performed by a neural network;

8. An object ranging device, comprising:

9. The apparatus of claim 8, wherein the first distance determining module is specifically configured to:

10. The apparatus according to claim 8 or 9, wherein the collecting device is provided on an intelligent driving device, the apparatus further comprising:

11. The apparatus according to claim 8 or 9, wherein the collecting device is provided on an intelligent driving device, the apparatus further comprising:

12. The apparatus of claim 8, wherein the height determination module is specifically configured to:

13. The apparatus of claim 8, wherein the target image is a road image, and the detection module is specifically configured to:

14. The device according to claim 13, wherein the detection module is a neural network, and a backbone network in the neural network is configured to perform feature extraction on the target image to obtain image features;

a second classification branch in the neural network for determining the second class of vehicles in the target image according to the image features;

15. An electronic device, comprising:

a processor; and

a memory, communicatively coupled to the processor, storing computer instructions readable by the processor, the processor performing the method of any of claims 1 to 7 when the computer instructions are read.

16. A storage medium having stored thereon computer-readable instructions for causing a computer to perform the method of any one of claims 1 to 7.