CN111968376B

CN111968376B - Road condition prompting method and device, electronic equipment and storage medium

Info

Publication number: CN111968376B
Application number: CN202010887576.XA
Authority: CN
Inventors: 殷卫华; 邵巾芮; 蔺颖
Original assignee: Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing Sensetime Technology Development Co Ltd
Priority date: 2020-08-28
Filing date: 2020-08-28
Publication date: 2022-06-28
Anticipated expiration: 2040-08-28
Also published as: WO2022041869A1; JP2022550895A; CN111968376A; KR20220057577A

Abstract

The present disclosure provides a road condition prompting method, apparatus, electronic device and storage medium, wherein the road condition prompting method includes: acquiring object attribute information corresponding to a target object within a distance setting range corresponding to obstacle avoidance equipment and distance information corresponding to the target object; determining risk information of the target object based on the object attribute information and the distance information corresponding to the target object; and prompting road conditions through the obstacle avoidance equipment based on the determined risk information.

Description

Road condition prompting method and device, electronic equipment and storage medium

Technical Field

The disclosure relates to the technical field of blind guiding, and in particular to a road condition prompting method, a road condition prompting device, electronic equipment and a storage medium.

Background

According to the statistics of the world health organization, about 2.85 hundred million visually impaired people exist in the world, the visually impaired people lose normal vision and have difficulty in understanding color, shape, distance and movement, so that the convenience of living, going and the like of the visually impaired people is greatly influenced, especially in strange outdoor going environments.

The traditional visually impaired people mainly depend on modes such as a blind person walking stick and a guide of a blind dog for going out, the detection range is limited in a mode of assisting going out through the blind person walking stick, and the use scene of the mode of assisting going out through the guide of the blind dog is limited.

Therefore, it is highly desirable to provide an effective blind guiding method to assist the visually impaired.

Disclosure of Invention

The embodiment of the disclosure provides at least one road condition prompting scheme.

In a first aspect, an embodiment of the present disclosure provides a road condition prompting method, including:

acquiring object attribute information corresponding to a target object within a distance setting range corresponding to obstacle avoidance equipment and distance information corresponding to the target object;

determining risk information of the target object based on the object attribute information and the distance information corresponding to the target object;

and prompting road conditions through the obstacle avoidance equipment based on the determined risk information.

In the embodiment of the disclosure, the object attribute information corresponding to the target object within the distance setting range corresponding to the obstacle avoidance device and the distance information corresponding to the target object can be acquired, and the risk information of the target object can be determined based on the object attribute information and the distance information corresponding to the target object, so that the risk information of the target object can be accurately determined.

In a second aspect, an embodiment of the present disclosure provides a road condition prompting device, including:

the device comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring object attribute information corresponding to a target object within a distance setting range corresponding to obstacle avoidance equipment and distance information corresponding to the target object;

a determining module, configured to determine risk information of the target object based on the object attribute information and the distance information corresponding to the target object;

and the prompting module is used for prompting the road condition through the obstacle avoidance equipment based on the determined risk information.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: the road condition prompting method comprises a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, when the electronic equipment runs, the processor and the memory are communicated through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the road condition prompting method according to the first aspect.

In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the road condition prompting method according to the first aspect are executed.

In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.

Fig. 1 shows a flowchart of a road condition prompting method provided by the embodiment of the disclosure;

FIG. 2 is a flowchart illustrating a method for risk information of a target object according to an embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating a method for language prompting of a first kind of target object according to an embodiment of the present disclosure;

FIG. 4 is a flowchart illustrating a method for language prompting of a second type target object according to an embodiment of the present disclosure;

Fig. 5 is a flowchart illustrating a first method for determining a road scene type corresponding to a location of an obstacle avoidance device according to an embodiment of the present disclosure;

fig. 6 is a flowchart illustrating a second method for determining a road scene type corresponding to a location of an obstacle avoidance device according to an embodiment of the present disclosure;

FIG. 7 is a flowchart illustrating a method for voice playback of a first type of voice prompt information and a second type of voice prompt information provided by an embodiment of the present disclosure;

fig. 8 is a schematic diagram illustrating a concrete road condition prompting scheme provided by the embodiment of the disclosure;

fig. 9 is a schematic structural diagram illustrating a road condition prompting device according to an embodiment of the disclosure;

fig. 10 shows a schematic structural diagram of an electronic device provided in an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

The term "and/or" herein merely describes an associative relationship, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

To the trip field, the accurate suggestion of road conditions information has great significance in the traffic safety field, and traditional looking the trip of barrier personage mainly relies on modes such as blind person's stick, guide dog guide, and the mode detection range of assisting the trip through blind person's stick is limited, and the use scene of the mode of assisting the trip through guide dog guide is limited, consequently, needs urgently to provide an effective guide's mode and assists looking barrier personage's trip.

The disclosure provides a road condition prompting method which can effectively assist visually impaired people in traveling. According to the method, the object attribute information corresponding to the target object in the distance setting range corresponding to the obstacle avoidance device and the distance information corresponding to the target object can be obtained, the risk information of the target object is determined based on the object attribute information and the distance information corresponding to the target object, further, road condition prompt can be carried out based on the risk information, and the traveling safety of the visually impaired people is improved.

To facilitate understanding of the present embodiment, first, a detailed description is given to a road condition prompting method disclosed in the embodiments of the present disclosure, where an execution main body of the road condition prompting method provided in the embodiments of the present disclosure is generally a computer device with certain computing capability, and the computer device includes, for example: a terminal device, which may be a User Equipment (UE), a mobile device, a User terminal, a handheld device, a computing device, a vehicle-mounted device, a wearable device, or a server or other processing device. The execution main body and the obstacle avoidance device of the road condition prompting method provided by the embodiment of the disclosure may be the same or different, and in some possible implementation manners, the road condition prompting method may be implemented by a way that a processor calls a computer readable instruction stored in a memory.

Referring to fig. 1, a flowchart of a road condition prompting method provided in the embodiment of the present disclosure specifically includes the following steps S101 to S103:

s101, acquiring object attribute information corresponding to a target object in a distance setting range corresponding to the obstacle avoidance device and distance information corresponding to the target object.

The execution main body of the road condition prompting method provided by the embodiment of the disclosure may be an obstacle avoidance device, or may also be a server capable of communicating with the obstacle avoidance device, and in a normal case, the server may maintain a communication connection with the obstacle avoidance device or establish a communication connection in a case where data transmission is required, which is not limited herein.

Illustratively, when the road condition prompting method provided by the embodiment of the present disclosure is applied to the blind guiding field, the obstacle avoidance device may include obstacle avoidance glasses and a mobile device, or obstacle avoidance glasses and a server/cloud device. Of course, wearable devices such as obstacle avoidance glasses can also be referred to separately.

For example, for different application scenes, the target object may include a preset object which is matched with the application scene and can be used as an obstacle or a road condition indication, or a pedestrian, for example, taking a road scene as an example, the target object may include an object or an indication signal and the like for indicating traffic of a walking mode of a user in the road scene, such as a traffic light and a pedestrian crossing, and may also include an obstacle which may cause a collision during a forward process of the user, such as a vehicle, a pedestrian, a step, a roadblock, a telegraph pole, a tree and the like.

Correspondingly, the object attribute information of the target object may be information for representing the attribute characteristics of the target object, which is obtained by classifying the attributes of different target objects included in the application scene in advance, for example, the object attribute information corresponding to the target object included in the road scene may include a dynamic motor vehicle, a dynamic non-motor vehicle and a static object, and the risk coefficients corresponding to different object attribute information are different and will be described in detail later.

For example, a target detection method may be used to perform target detection on a scene image captured by the obstacle avoidance device, and determine whether a preset target object matched with an application scene is included, for example, for a road scene, after the obstacle avoidance device captures the scene image, whether the scene image includes a vehicle, a pedestrian, a rail, a tree, a step, a roadblock, a telegraph pole, and the like may be detected based on a pre-trained target detection network.

Illustratively, keep away and can be provided with image acquisition part on the barrier equipment, this image acquisition part can be including installing one or more cameras on keeping away the barrier equipment, and wherein, the camera can be the panorama camera, or can gather the camera that the scope is wider, when keeping away the barrier equipment for keeping away barrier glasses, every camera can distribute on keeping away the different positions of barrier glasses for gather the scene image of the road scene that the user is located.

The target object within the distance setting range corresponding to the obstacle avoidance device is the target object within the distance setting range from the obstacle avoidance device, and after a scene image is acquired through an image acquisition component on the obstacle avoidance device, the target object within the distance setting range from the obstacle avoidance device can be obtained through the scene image. The distance between the boundary of the set range and the obstacle avoidance device is within a preset range, the preset range is related to internal parameters of an image acquisition component on the obstacle avoidance device, and after the internal parameters of the image acquisition component are determined, the farthest distance which can be shot by the image acquisition component can represent the distance corresponding to the set range.

For example, the distance information corresponding to the target object may represent a distance between the target object and the obstacle avoidance device, and the distance may be determined in various manners, such as a laser ranging manner, and may also be determined by detecting a time of flight of a light pulse, which will be described in detail later.

S102, determining risk information of the target object based on the object attribute information and the distance information corresponding to the target object.

In the blind guiding field, the closer the distance between the target object and the user is, the greater the risk to the user is, first, taking the obstacle as an example, when the distance between the obstacle and the user is close, the higher the possibility of collision occurs, regarding an object or an indication signal indicated by traffic, such as a traffic light, and when the user does not walk according to the indication of the traffic light, the higher the probability of danger occurrence, so the distance information corresponding to the target object can be used as an index for measuring the risk information of the target object, and in addition, the risks to the user of the target objects with different object attribute information are different, for example, the target object is a motor vehicle with a faster driving speed, and the target object is a static object, considering that the risk degree of the motor vehicle when colliding with the user is greater than the risk degree of the static object when colliding with the user, so the object attribute information corresponding to the target object here can also be used as an index for measuring the risk information of the target object, that is, in the case where the object attribute information of the target object is a motor vehicle and a static object, respectively, the degree of risk of the target object with respect to the user can be determined by the attribute information.

And S103, prompting the road condition through the obstacle avoidance device based on the determined risk information of the target object.

After the risk information of the target object is obtained, a road condition prompt may be performed for the target object, for example, in the case that a plurality of target objects exist, considering that the risk information of each target object is different, at this time, a user may be selectively prompted according to the risk information of each target object, for example, a target object with a higher risk degree is selected to perform the road condition prompt for the user, so as to ensure the travel safety of the user.

Specifically, when the road condition is prompted by the obstacle avoidance device, the method may include:

and prompting in at least one of a voice form and a vibration form.

For example, the user is prompted by voice "there is an obstacle ahead, please pay attention to deceleration and avoidance", or the user is prompted by vibration that there is an obstacle ahead.

The following will describe the above S101 to S103 in detail with reference to specific embodiments:

for the above S101, when acquiring the object attribute information corresponding to the target object in the distance setting direction corresponding to the obstacle avoidance device, the following S1011 to S1012 may be included:

s1011, obtaining a scene image obtained by shooting a target road condition scene by obstacle avoidance equipment;

s1012, by performing target detection on the scene image, a target object included in the scene image and object attribute information corresponding to the target object are determined.

For example, the image acquisition component of the obstacle avoidance device may include a camera for acquiring an RGB image, where the camera may acquire an RGB scene image of a target road condition scene, and for the RGB scene image, target detection may be performed on a target object included in the RGB scene image according to a pre-trained target detection neural network, so as to determine the target object included in the RGB scene image and object attribute information corresponding to the target object.

For example, a neural network capable of detecting a set target object may be trained in advance based on a large number of sample images corresponding to a road condition scene, for example, a neural network capable of detecting obstacles such as motor vehicles, bicycles, pedestrians, traffic indication type roadblocks (traffic signal indicator lamps), traffic markings (pedestrian crossings), trees, telegraph poles, steps, and the like may be trained.

In the embodiment of the disclosure, the scene image can be directly shot through the obstacle avoidance device, when the obstacle avoidance device is worn by a user, the scene image corresponding to the trip scene of the user can be directly obtained, after the scene image is obtained, the target object and the corresponding object attribute information can be determined according to the scene image, illustratively, the target object can be obtained by directly carrying out target detection on the scene image based on a pre-trained target detection neural network, and the process can be satisfied only by configuring and installing a lens capable of obtaining the scene image by the obstacle avoidance device, so that the volume of the obstacle avoidance device can be saved, and the user can conveniently carry the obstacle avoidance device.

In one embodiment, for the above S101, when obtaining the distance information corresponding to the target object, the following S1013 to S1014 may be included:

s1013, acquiring a scene image and a depth image obtained by shooting a target road condition scene by the obstacle avoidance device;

and S1014, determining distance information corresponding to the target object based on the scene image and the depth image.

Illustratively, the obstacle avoidance device may further include a camera for acquiring a depth image, and may include a binocular stereoscopic camera or a Time Of Flight (TOF) camera, in addition to the RGB camera for acquiring the RGB scene image corresponding to the target road condition scene.

Exemplarily, when the obstacle avoidance device includes a TOF camera, a time-of-flight ranging method may be used to obtain a depth image corresponding to the target road condition scene, and specifically, the depth image corresponding to the target road condition scene may be generated according to a time interval from transmitting a light pulse to receiving a light pulse and a transmission speed of the light pulse.

After the depth image corresponding to the target road condition scene is obtained, the depth image corresponding to the target road condition scene and the scene image can be aligned, so that the RGB camera and the TOF camera are in the same coordinate system, and further, the distance information between each target object and the obstacle avoidance device can be determined based on the aligned depth image and scene image.

In the embodiment of the disclosure, the depth image corresponding to the target road condition scene is obtained through the obstacle avoidance device, and then the distance information corresponding to each target object can be quickly determined by combining the scene image corresponding to the target road condition scene.

The sequence of the process of identifying the target object and the object attribute information corresponding to the target object, that is, the processes of S1011 to S1012, and the process of identifying the distance information corresponding to the target object, that is, the processes of S1013 to S1014, is not limited, and the distance information corresponding to each object included in the scene image may be determined first, and then which objects are the target objects may be determined, or the target object may be determined first, and then the distance information corresponding to the target object may be determined.

Specifically, with respect to S1014 described above, when determining distance information corresponding to a target object based on a scene image and a depth image, the following S10141 to S10142 are included:

s10141, respectively extracting calibration feature points contained in the scene image and the depth image;

s10142, aligning the scene image and the depth image based on the calibration feature points of the scene image and the calibration feature points of the depth image, and extracting the depth information of the target object contained in the scene image from the depth image to obtain the distance information corresponding to the target object.

For example, edge position points of the same building included in both the scene image and the depth image may be selected as calibration feature points, for example, the scene image includes a building a, the depth image also includes a building a, and feature points forming a contour of the building a may be used as calibration feature points, so that the scene image and the depth image are aligned, so that depth information corresponding to a target object included in the scene image may be determined in the depth image, and the depth information may be used as distance information corresponding to the target object.

In relation to S102, when determining the risk information of the target object based on the object attribute information and the distance information corresponding to the target object, as shown in fig. 2, the following S201 to S203 may be included:

S201, determining a first risk coefficient corresponding to a target object based on object attribute information corresponding to the target object;

s202, determining a second risk coefficient corresponding to the target object according to the distance between the target object and the obstacle avoidance device indicated by the distance information corresponding to the target object;

and S203, determining the risk information of the target object based on the distance information, the first risk coefficient and the second risk coefficient.

For example, a corresponding first risk coefficient may be set in advance for each type of object attribute information according to the risk degree of each type of object attribute information, and a corresponding second risk coefficient may be set in advance for each distance section according to the risk degree corresponding to a different distance section.

Specifically, the danger degree of the dynamic motor vehicle for the user is greater than that of the dynamic non-motor vehicle, and the danger degree of the dynamic non-motor vehicle is greater than that of the static object for the user, so that the first danger coefficient of the dynamic motor vehicle such as a car and a motorcycle is greater than that of the dynamic non-motor vehicle such as a person and a bicycle; the first risk factor of the dynamic non-motor vehicle is greater than the first risk factor of static objects such as railings, trees.

For example, the second risk coefficients corresponding to different distance intervals may be preset, for example, the distance intervals may include an interval greater than 0.5 m and less than or equal to 1 m, an interval greater than 1 m and less than or equal to 3 m, and an interval greater than 3 m, where each distance interval corresponds to a different second risk coefficient.

Considering that the risk degrees corresponding to different object attribute information are different, and the risk degrees corresponding to different distance intervals are different, for each target object, the risk information of the target object can be determined jointly according to the distance between the target object and the obstacle avoidance device, the first risk coefficient and the second risk coefficient corresponding to the target object.

In the embodiment of the disclosure, when determining the risk information corresponding to the target object, the risk information of the target object for the user can be determined by comprehensively considering the object attribute information corresponding to the target object and the distance information between the target object and the obstacle avoidance device, so that the accuracy of the determined risk information of the target object is improved.

Specifically, with respect to S203 described above, when determining the risk information of the target object based on the distance information, the first risk coefficient, and the second risk coefficient, the following S2031 to S2032 may be included:

S2031, determining a target distance difference value based on the maximum distance and distance information corresponding to the shooting range of the obstacle avoidance device;

and S2032, taking the product of the target distance difference, the first risk coefficient and the second risk coefficient as the risk information of the target object.

The farthest distance corresponding to the shooting range of the camera on the obstacle avoidance device is related to internal parameters of the camera, the farthest distance is a fixed parameter, and a target distance difference value representing the distance between the target object and the farthest distance can be determined according to the farthest distance and the distance indicated by the distance information corresponding to the target object.

Specifically, the risk value representing the risk information of the target object may be determined by the target distance difference value, the first risk coefficient, and the second risk coefficient corresponding to the target object, and for a case where a plurality of target objects are included, the risk value corresponding to each target object may be determined in the same manner, and specifically, the risk value R of the target object i may be determined according to the following formula (1)_i：

R_i＝(H-d_i)*w_i1*w_i2 (1)；

H represents the maximum distance corresponding to the shooting range of the obstacle avoidance device; d_iRepresenting the distance between the target object i and the obstacle avoidance device; w is a_i1Representing a first danger coefficient corresponding to a target object i; w is a _i2And representing a second risk factor corresponding to the target object i.

In an implementation manner, the road condition prompting method provided by the embodiment of the present disclosure further includes:

determining category information corresponding to the target object based on object attribute information corresponding to the target object; the category information includes a traffic indication category and a non-traffic indication category.

For example, in order to improve travel safety and effectiveness, the embodiments of the present disclosure propose that target objects may be classified into target objects of traffic indication category, such as including traffic signal indicator lights and traffic sign lights, such as traffic lights and pedestrian crossings, and target objects of non-traffic indication category according to the object attribute information; target objects of the non-traffic indicating category may include dynamic motor vehicles such as cars, motorcycles, dynamic non-motor vehicles such as pedestrians, bicycles, and static non-traffic indicating type objects such as railings, trees, steps, and the like.

Based on the risk information confirmed, road condition prompt is carried out through obstacle avoidance equipment, and the method comprises the following steps:

and prompting the road condition through the obstacle avoidance equipment based on the category information and the risk information corresponding to the target object.

In the embodiment of the disclosure, when the road condition is prompted to the user, the road condition prompting information can be determined based on the target category information and the risk information corresponding to the target object, and then the user is prompted according to the road condition prompting information, so that the travel safety of the user is improved.

In one embodiment, according to the obtained object attribute information corresponding to the target object, the target object may be divided into a first type target object belonging to a traffic indication category and a second type target object belonging to a non-traffic indication category, and the meaning of the target object of the traffic indication category in the road scene, that is, the target object is used for indicating traffic rules, so that in a prompting sequence of the road condition prompt, the priority of the first type target object belonging to the determined traffic indication category is not lower than the priority of the second type target object belonging to the non-traffic indication category.

In order to distinguish the prompt sequence of the first type target object from the prompt sequence of the second type target object, when the risk value of the target object is determined based on the formula (1), the first type risk coefficient corresponding to the first type target object may be a negative number, and the first type risk coefficient corresponding to the second type target object may be a positive number.

For example, considering the significance of the target object of the traffic indication class to the disabled user in the travel environment, such as when the user crosses the road, the user may be prompted preferentially for the information indicated by the traffic signal indicator light, and the user may determine whether to cross the road based on the information and then prompt the target object of the non-traffic indication class existing on the road, which may help the user to travel safely in the road scene.

Specifically, when the road condition is prompted by the obstacle avoidance device based on the category information and the risk information corresponding to the target object, as shown in fig. 3, the following steps S301 to S302 may be included:

s301, under the condition that the target objects comprise first-class target objects belonging to traffic indication categories, determining a prompting sequence of the first-class target objects when road condition prompting is carried out through obstacle avoidance equipment on the basis of risk information of the first-class target objects;

and S302, according to the determined prompt sequence, road condition prompt is carried out on the first type of target objects through the obstacle avoidance device.

First, how to prompt the road condition for the first type of target object will be described.

Specifically, under the condition that the road condition prompting device comprises a plurality of first-class target objects, the risk information of each first-class target object can be determined according to the formula (1), and then the first-class target objects are sequenced in a risk descending manner to obtain a prompting sequence of each first-class target object when the road condition prompting is performed through the obstacle avoidance device.

Specifically, for the above S301, when determining a prompting sequence of the first type target object when the road condition is prompted by the obstacle avoidance device based on the risk information of the first type target object, the method includes the following S3011 to S3014:

S3011, determining first direction information of the obstacle avoidance device pointing to the first type of target object based on a scene image shot by the obstacle avoidance device, distance information corresponding to the first type of target object and parameter information corresponding to the obstacle avoidance device.

For example, the parameter information corresponding to the obstacle avoidance device may include internal parameters for converting the image coordinate system to the camera coordinate system, and external parameters for converting the camera coordinate system to the world coordinate system.

For example, the first direction information of the obstacle avoidance device pointing to the first type target object can be used to indicate a direction relationship between the first type target object and the obstacle avoidance device in the world coordinate system, for example, the first type target object is located in a positive direction of the obstacle avoidance device along an X axis in the world coordinate system, for example, if the positive direction of the X axis points to an east direction, the first type target object is located in the east direction of the obstacle avoidance device.

Specifically, when determining first direction information in which the obstacle avoidance device points at the first type of target object based on a scene image shot by the obstacle avoidance device, distance information corresponding to the first type of target object, and parameter information corresponding to the obstacle avoidance device, the method may include:

(1) and determining the pose information of the obstacle avoidance equipment under a world coordinate system corresponding to the real scene based on the scene image shot by the obstacle avoidance equipment.

Illustratively, the pose information of the obstacle avoidance device in the world coordinate system corresponding to the real scene may be determined based on a scene image captured by the obstacle avoidance device and a pre-established three-dimensional scene map representing the real scene, and in the world coordinate system, the three-dimensional scene map representing the real scene may completely coincide with the real scene, so that the pose information of the obstacle avoidance device in the world coordinate system corresponding to the real scene may be determined based on the scene image captured by the obstacle avoidance device and the three-dimensional scene map.

Specifically, the pose information of the obstacle avoidance device in the world coordinate system corresponding to the real scene may include a position coordinate value and/or a direction angle of the obstacle avoidance device in the world coordinate system, where the direction angle may be represented by an included angle with a coordinate axis of the world coordinate system.

(2) And determining two-dimensional detection information of the first class target object under an image coordinate system corresponding to the scene image by performing target detection on the scene image.

Illustratively, the target object included in the scene image and the pixel coordinates of the detection frame corresponding to the target object in the image coordinate system may be detected through a pre-trained target detection neural network, where the pixel coordinates corresponding to the center point of the detection frame may be used as two-dimensional detection information of the first type target object in the image coordinate system corresponding to the scene image.

(3) And determining the pose information of the first class target object under a world coordinate system based on the two-dimensional detection information, the distance information corresponding to the first class target object and the parameter information corresponding to the obstacle avoidance device.

Specifically, coordinate values of the first type target object along an X axis and a Y axis of a camera coordinate system corresponding to the obstacle avoidance device in the camera coordinate system corresponding to the obstacle avoidance device may be determined based on two-dimensional detection information of the first type target object in an image coordinate system corresponding to the scene image and internal parameters (pre-stored internal parameters) corresponding to the obstacle avoidance device, then, coordinate values of the first type target object along a Z axis of the camera coordinate system corresponding to the obstacle avoidance device in the camera coordinate system corresponding to the obstacle avoidance device may be determined based on distance information corresponding to the first type target object, and further, pose information of the first type target object in the world coordinate system may be determined in combination with external parameters (which may be determined by pose information of the obstacle avoidance device in a world coordinate system corresponding to the real scene).

For example, the pose information of the first type target object in the world coordinate system may include a position coordinate value of the first type target object in the world coordinate system.

(4) And determining first direction information of the obstacle avoidance device pointing to the first type of target object based on the position and pose information corresponding to the obstacle avoidance device and the position and pose information corresponding to the first type of target object.

Specifically, when determining the first direction information in which the obstacle avoidance device points to the first type of target object, the determination may be based on position coordinate values of the obstacle avoidance device and the first type of target object in the world coordinate system.

S3012, determining the movement direction of the obstacle avoidance device based on multiple scene images shot by the obstacle avoidance device.

For example, the change of the position coordinate value of the obstacle avoidance device in the world coordinate system may be determined based on a plurality of scene images, and then the movement direction of the obstacle avoidance device may be obtained.

S3013, when the first-class target objects comprise a plurality of first-class target objects, determining a first-class target object to be prompted from the first-class target objects based on first direction information corresponding to the first-class target objects and the movement direction of the obstacle avoidance device, wherein a first angle between a direction indicated by the first direction information corresponding to the first-class target object to be prompted and the movement direction of the obstacle avoidance device is smaller than a set angle threshold.

When the first-class target objects include a plurality of objects, the first-class target objects, which are set to be close to the obstacle avoidance device, may be regarded as first-class target objects to be prompted, for example, three first-class target objects are detected and are respectively marked as a first-class target object a, a first-class target object B, and a first-class target object C, and if the movement direction of the obstacle avoidance device is toward the first-class target object a and is far from the first-class target object B and the first-class target object C, the first-class target object a may be regarded as the first-class target objects to be prompted here.

S3014, determining a prompting sequence of the first type of target objects to be prompted when road condition prompting is carried out through the obstacle avoidance device based on risk information of the first type of target objects to be prompted.

After the first type target objects to be prompted are screened out from the plurality of first type target objects, a prompting sequence of the first type target objects to be prompted when road condition prompting is carried out through the obstacle avoidance device can be further determined based on risk information of the first type target objects to be prompted.

Exemplarily, in a case that it is determined that the first type target object includes a traffic light and a crosswalk, and a distance between the traffic light and the obstacle avoidance device is smaller than a distance between the crosswalk and the obstacle avoidance device, it may be determined according to the above formula (1) that a risk value corresponding to the traffic light is larger than a risk value corresponding to the crosswalk, and it may be determined that a prompt order of the traffic light is prior to a prompt order of the crosswalk.

Further, when the road condition prompt is performed on the first type of target object through the obstacle avoidance device according to the determined prompt sequence, the method includes:

and according to the determined prompt sequence corresponding to the first type of target object to be prompted, under the condition that the distance between the first type of target object to be prompted and the obstacle avoidance device, which is indicated by the distance information corresponding to the first type of target object to be prompted, is smaller than a first preset distance threshold, road condition prompt is carried out on the first type of target object to be prompted through the obstacle avoidance device.

For example, considering that a route is changed in advance at a position far away from a first type target object to be prompted or stops not to advance any more in a driving process of a user, at this time, the first type target object to be prompted does not have danger to the user any more, and prompting may not be performed any more.

In the embodiment of the disclosure, when a plurality of first-class target objects are included, the first-class target objects close to the obstacle avoidance device can be preferentially selected as target objects to be prompted, and prompting is performed when it is determined that the distance between the obstacle avoidance device and the first-class target objects is smaller than a first preset distance threshold, so that the condition of invalid prompting is reduced, and the accuracy of road condition prompting is improved in the process of prompting the road condition for a user.

In another embodiment, when the road condition is prompted by the obstacle avoidance device based on the category information and the risk information corresponding to the target object, as shown in fig. 4, the method may include the following steps S401 to S402:

s401, under the condition that the target object comprises a second type target object belonging to a non-traffic indication category, determining a second type target object meeting preset conditions from the second type target object based on risk information of the second type target object.

In order to effectively prompt the user according to the road condition environment, the second type target object which has a high probability of causing danger to the user when the user goes out can be screened out according to the risk information of the second type target object and serves as the second type target object to be prompted.

S402, when the distance between the second type target object to be prompted and the obstacle avoidance device, indicated by the distance information corresponding to the second type target object to be prompted, is smaller than a second preset distance threshold, road condition prompting is conducted on the second type target object to be prompted through the obstacle avoidance device.

For example, the second preset distance threshold may be the same as or different from the first preset distance threshold, and is similar to the prompting process for the first type of target object to be prompted, and when the road condition prompting is performed on the second type of target object to be prompted, the road condition prompting can be performed on the second type of target object to be prompted through the obstacle avoidance device under the condition that it is determined that the distance between the second type of target object to be prompted and the obstacle avoidance device is smaller than the second preset distance threshold, and the condition of invalid prompting can also be reduced, so that the accuracy of the road condition prompting is improved in the road condition prompting process.

Specifically, for the above S401, when determining, from the second type target objects, a second type target object to be prompted, which meets the preset condition, based on the risk information of the second type target object, S4011 to S4014 may be included:

s4011, second direction information of the obstacle avoidance device pointing to the second type target object is determined based on the scene image shot by the obstacle avoidance device, the distance information corresponding to the second type target object and the parameter information corresponding to the obstacle avoidance device.

Here, a manner of determining the second direction information of the obstacle avoidance device pointing to the second type of target object is similar to the above-mentioned manner of determining the first direction information of the obstacle avoidance device pointing to the first type of target object, and is not described herein again.

And S4012, determining the movement direction of the obstacle avoidance equipment based on a plurality of scene images shot by the obstacle avoidance equipment.

The details of determining the moving direction of the obstacle avoidance device are described above, and are not described herein again.

And S4013, determining a second angle between a second direction indicated by the second direction information and the movement direction of the obstacle avoidance device based on the second direction information corresponding to the second type target object and the movement direction of the obstacle avoidance device.

For example, whether the obstacle avoidance device moves towards the second type target object may be determined by the second angle, for example, the smaller the second angle, the higher the probability that the obstacle avoidance device moves towards the second type target object.

S4014, when a plurality of second type target objects are included, at least one second type target object with a second angle smaller than a preset angle threshold is determined from the second type target objects and is used as the second type target object to be prompted, and the risk degree indicated by the risk information corresponding to the at least one second type target object is higher than the risk degree indicated by the risk information corresponding to other target objects except the second type target object to be prompted in the second type target object.

When the second type of target object includes multiple second type of target objects, the second type of target object to be prompted may be selected based on the second angle and the risk information corresponding to the second type of target object, for example, the second type of target object, which is gradually approached by the obstacle avoidance device and has a higher risk indicated by the corresponding risk information, may be used as the second type of target object to be prompted.

In the embodiment of the disclosure, the second type target object to be prompted is screened by combining the movement direction of the obstacle avoidance device and the distance between the second type target object and the obstacle avoidance device, so that the second type target object which is dangerous to the user can be prompted by selecting a high probability, and the travel safety of the user is improved.

In an embodiment, in addition to considering a target object encountered by a user in a road scene, a road scene type encountered by the user in the road scene may also be considered, for example, a pedestrian road scene type, a crossroad scene type, an overpass and underpass scene type, a station scene type, and the like, in order to help a visually impaired user to safely go out, the target object encountered by the user is prompted, and a road scene type where the user is currently located may also be prompted, so as to help the user to better determine a road scene type where the user is located in a road for going out, thereby improving the safety for going out, for this reason, the road condition prompting method provided by the embodiment of the present disclosure further includes:

And acquiring the road scene type corresponding to the position of the obstacle avoidance device.

For example, the road scene type may be obtained in different manners, for example, when the obstacle avoidance device includes a navigation component, the road scene type in which the user is currently located may be determined based on the navigation component and a preselected navigation route, or the road scene type in which the user is currently located may be determined by a scene mark shot by the obstacle avoidance device, which is described in detail later.

Aiming at the situation that the road condition is prompted through the obstacle avoidance device based on the determined risk information, the method comprises the following steps:

and prompting the road condition through obstacle avoidance equipment based on the determined risk information and the road scene type.

For example, when the road condition is prompted, the target object may be prompted first and then the road scene type is prompted, or the road scene type may be prompted first and then the target object is prompted.

In a possible implementation manner, when acquiring a road scene type corresponding to a position where an obstacle avoidance device is located, as shown in fig. 5, the following steps S501 to S502 may be included:

s501, acquiring position information of obstacle avoidance equipment;

and S502, determining a road scene type corresponding to the position information of the obstacle avoidance device from a preset navigation path of the obstacle avoidance device.

For example, the position information of the obstacle avoidance device may be obtained through a navigation component on the obstacle avoidance device, or the position information of the obstacle avoidance device may be determined based on a scene image captured by the obstacle avoidance device, which is detailed above and is not described herein again.

Illustratively, the obstacle avoidance device may include a mobile device, such as a mobile phone, a tablet, and the navigation component is a navigation component on the mobile phone or the tablet.

For example, before departure, a user may select a navigation path to a destination through an obstacle avoidance device, where the navigation path may include multiple road scenes and location information corresponding to each road scene, and when it is determined that the location information of the obstacle avoidance device matches the location information corresponding to any road scene, a road scene type corresponding to a location where the obstacle avoidance device is located may be determined.

In another embodiment, when acquiring a road scene type corresponding to a position where an obstacle avoidance device is located, as shown in fig. 6, the following steps S601 to S602 may be included:

s601, acquiring a stop board image shot by obstacle avoidance equipment;

and S602, carrying out scene mark identification on the stop board image, and determining the road scene type corresponding to the area where the obstacle avoidance equipment is located.

Illustratively, the stop board image can be acquired through an image acquisition component on the obstacle avoidance device, and the stop board image can be identified based on a pre-trained marker identification neural network to determine the road scene type corresponding to the area where the obstacle avoidance device is located.

For example, by performing scene mark recognition on the stop board image and determining that the scene mark of the stop board image is a 'middle-customs station', it can be determined that the road scene type corresponding to the area where the obstacle avoidance device is located is the station scene type.

Specifically, when the road condition is prompted by the obstacle avoidance device based on the determined risk information and the road scene type, as shown in fig. 7, the following steps S701 to S703 may be included:

s701, generating first-class prompt information based on the determined risk information of the target object;

s702, generating second prompting information based on the road scene type;

and S703, carrying out road condition prompting on the first type of prompting information and the second type of prompting information through the obstacle avoidance device based on a preset prompting sequence.

For example, the first type of prompt information for the obstacle avoidance device may be generated for target object category information, object attribute information, and a distance from the obstacle avoidance device of the target object.

For example, the second type of prompt information for the obstacle avoidance device may be used to prompt the user of the road scene type where the user is located, so that the user may determine the road scene type where the user is located, and thus determine whether to adjust the driving speed to ensure safety of travel.

Illustratively, when the preset prompting sequence between the first type of prompt information and the second type of prompt information is that the second type of prompt information is prior to the first type of prompt information, the second type of prompt information is broadcasted first, and then the first type of prompt information is broadcasted, so that a user can determine the type of a road scene where the user is located first, and then determine obstacles encountered under the type of the road scene, thereby adjusting the walking speed and direction in time and ensuring the safety of travel.

Referring to fig. 8, a road condition prompting scheme provided in the embodiment of the present disclosure is described.

As shown in fig. 8, for example, when a user is a visually impaired user, and an obstacle avoidance device includes obstacle avoidance glasses and a terminal device, the user wears the obstacle avoidance glasses to acquire a scene image corresponding to a road scene through an RGB camera disposed on the obstacle avoidance glasses when walking in the road scene, and then the scene image may be subjected to target detection through the terminal device, so as to determine a target object included in the scene image, for example, the target object may be identified by a pedestrian road/blind road, a motor vehicle/non-motor vehicle, a subway station/bus station, a pedestrian/animal, a street bridge/underpass, a ramp/step/elevator, a station yard guideboard, a telegraph pole/tree, a side slope/water, and the like, determine a target object included in the scene image, and at the same time, a depth image corresponding to the road scene may be acquired based on the camera, and the scene image and the depth image are aligned, in addition, the terminal device may determine a road scene type where the user is located based on the position information of the user and a preset navigation route, for example, may recognize a walking scene type of a sidewalk, a crossing scene type, a pedestrian bridge and subway scene type, and the like, or may determine the road scene type where the user is located based on a stop sign, and after determining the road scene type where the user is located, may generate voice prompt information for the user by combining the road scene type, object attribute information corresponding to the target object, and distance information between the target object and the obstacle avoidance device, and then voice prompt is carried out on the user through the obstacle avoidance glasses based on the language prompt information, so that the safety of the user in travel is ensured.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

Based on the same technical concept, a road condition prompting device corresponding to the road condition prompting method is further provided in the embodiment of the present disclosure, and as the principle of solving the problem of the device in the embodiment of the present disclosure is similar to the road condition prompting method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and repeated parts are not described again.

Referring to fig. 9, which is a schematic view of a road condition prompting device 800 provided in the embodiment of the present disclosure, the road condition prompting device 800 includes:

an obtaining module 801, configured to obtain object attribute information corresponding to a target object within a distance setting range corresponding to an obstacle avoidance device, and distance information corresponding to the target object;

a determining module 802, configured to determine risk information of the target object based on object attribute information and distance information corresponding to the target object;

and the prompting module 803 is configured to prompt the road condition through the obstacle avoidance device based on the determined risk information.

In a possible implementation manner, when the obtaining module 801 is configured to obtain object attribute information corresponding to a target object in a set distance range corresponding to an obstacle avoidance device, the obtaining module includes:

acquiring a scene image obtained by shooting a target road condition scene by obstacle avoidance equipment;

and determining a target object contained in the scene image and object attribute information corresponding to the target object by performing target detection on the scene image.

In a possible implementation, the obtaining module 801, when configured to obtain distance information corresponding to a target object, includes:

acquiring a scene image and a depth image which are obtained by shooting a target road condition scene by obstacle avoidance equipment;

and determining distance information corresponding to the target object based on the scene image and the depth image, wherein the distance information comprises the distance between the target object and the obstacle avoidance device.

In a possible implementation, the obtaining module 801, when configured to determine distance information corresponding to a target object based on a scene image and a depth image, includes:

respectively extracting calibration characteristic points contained in the scene image and the depth image;

aligning the scene image and the depth image based on the calibration characteristic points of the scene image and the calibration characteristic points of the depth image, and extracting the depth information of the target object contained in the scene image from the depth image to obtain the distance information corresponding to the target object.

In one possible implementation, the determining module 802, when configured to determine the risk information of the target object based on the object attribute information and the distance information corresponding to the target object, includes:

determining a first risk coefficient corresponding to the target object based on object attribute information corresponding to the target object;

determining a second risk coefficient corresponding to the target object according to the distance between the target object and the obstacle avoidance device indicated by the distance information corresponding to the target object;

based on the distance information, the first risk coefficient and the second risk coefficient, risk information of the target object is determined.

In one possible implementation, the determining module 802, when configured to determine the risk information of the target object based on the distance information, the first risk coefficient, and the second risk coefficient, includes:

determining a target distance difference value based on the maximum distance and the distance information corresponding to the shooting range of the obstacle avoidance equipment;

and taking the product of the target distance difference value, the first risk coefficient and the second risk coefficient as the risk information of the target object.

In one possible implementation, the determining module 802 is further configured to:

determining category information corresponding to the target object based on object attribute information corresponding to the target object; the category information comprises a traffic indication category and a non-traffic indication category;

The prompt module includes when being used for carrying out road conditions prompt through obstacle avoidance equipment based on the determined risk information:

and prompting the road condition through obstacle avoidance equipment based on the category information and the risk information corresponding to the target object.

In a possible implementation manner, in the prompting sequence of the road condition prompt, the priority of the first type target object belonging to the traffic indication category is not lower than the priority of the second type target object belonging to the non-traffic indication category.

In a possible implementation manner, the prompting module 803, when configured to prompt the road condition through the obstacle avoidance device based on the category information and the risk information corresponding to the target object, includes:

determining a prompting sequence of the first type target objects when road condition prompting is carried out through obstacle avoidance equipment on the basis of risk information of the first type target objects under the condition that the target objects comprise the first type target objects belonging to the traffic indication category;

and according to the determined prompt sequence, road condition prompt is carried out on the first type of target objects through the obstacle avoidance equipment.

In a possible implementation manner, the prompting module 803, when configured to determine, based on the risk information of the first type target object, a prompting order of the first type target object when the road condition is prompted through the obstacle avoidance device, includes:

Determining first direction information of the obstacle avoidance device pointing to the first type of target object based on a scene image shot by the obstacle avoidance device, distance information corresponding to the first type of target object and parameter information corresponding to the obstacle avoidance device;

determining the movement direction of the obstacle avoidance equipment based on a plurality of scene images shot by the obstacle avoidance equipment;

under the condition that the first type of target objects comprise a plurality of target objects, determining a first type of target object to be prompted from the first type of target objects on the basis of first direction information corresponding to the first type of target objects and the movement direction of the obstacle avoidance device, wherein a first angle between the direction indicated by the first direction information corresponding to the first type of target objects to be prompted and the movement direction of the obstacle avoidance device is smaller than a set angle threshold value;

determining a prompting sequence of the first type of target objects to be prompted when road condition prompting is carried out through obstacle avoidance equipment based on risk information of the first type of target objects to be prompted;

the prompting module 803, when configured to perform road condition prompting on the first type of target object through the obstacle avoidance device according to the determined prompting sequence, includes:

according to the determined prompting sequence corresponding to the first type of target object to be prompted, under the condition that the distance between the first type of target object to be prompted and the obstacle avoidance device, indicated by the distance information corresponding to the first type of target object to be prompted, is smaller than a first preset distance threshold, road condition prompting is carried out on the first type of target object to be prompted through the obstacle avoidance device.

In a possible implementation manner, the prompting module 803, when configured to determine, based on a scene image shot by an obstacle avoidance device, distance information corresponding to a first type of target object, and parameter information corresponding to the obstacle avoidance device, first direction information in which the obstacle avoidance device points at the first type of target object, includes:

determining pose information of the obstacle avoidance device under a world coordinate system corresponding to a real scene based on a scene image shot by the obstacle avoidance device;

determining two-dimensional detection information of a first type of target object under an image coordinate system corresponding to the scene image by performing target detection on the scene image;

determining pose information of the first type target object in a world coordinate system based on the two-dimensional detection information, the distance information corresponding to the first type target object and the parameter information corresponding to the obstacle avoidance device;

and determining first direction information of the obstacle avoidance device pointing to the first type of target object based on the position and pose information corresponding to the obstacle avoidance device and the position and pose information corresponding to the first type of target object.

Under the condition that the target object comprises a second type target object belonging to a non-traffic indication category, determining a second type target object to be prompted, which meets a preset condition, from the second type target object based on risk information of the second type target object;

and under the condition that the distance between the second type target object to be prompted and the obstacle avoidance device, which is indicated by the distance information corresponding to the second type target object to be prompted, is smaller than a second preset distance threshold, road condition prompting is carried out on the second type target object to be prompted through the obstacle avoidance device.

In one possible implementation, the prompt module 803

When the method is used for determining a second type target object to be prompted, which meets a preset condition, from the second type target object based on the risk information of the second type target object, the method comprises the following steps:

determining second direction information of the obstacle avoidance device pointing to the second type of target object based on the scene image shot by the obstacle avoidance device, the distance information corresponding to the second type of target object and the parameter information corresponding to the obstacle avoidance device;

determining a second angle between a second direction indicated by the second direction information and the movement direction of the obstacle avoidance device based on the second direction information corresponding to the second type target object and the movement direction of the obstacle avoidance device;

Under the condition that the number of the second type target objects is multiple, at least one second type target object with a second angle smaller than a preset angle threshold value is determined from the second type target objects and serves as the second type target object to be prompted, and the risk degree indicated by the risk information corresponding to the at least one second type target object is higher than the risk degree indicated by the risk information corresponding to other target objects except the second type target object to be prompted in the second type target object.

In a possible implementation, the obtaining module 801 is further configured to:

acquiring a road scene type corresponding to the position of the obstacle avoidance equipment;

the prompting module 803 is used for prompting the road condition through the obstacle avoidance device based on the determined risk information, and includes:

In a possible implementation manner, the obtaining module 801, when configured to obtain a road scene type corresponding to a location of an obstacle avoidance device, includes:

acquiring position information of obstacle avoidance equipment;

and determining the road scene type corresponding to the position information of the obstacle avoidance device from a preset navigation path of the obstacle avoidance device.

Acquiring a stop board image shot by obstacle avoidance equipment;

and carrying out scene mark identification on the stop board image, and determining the road scene type corresponding to the position of the obstacle avoidance device.

In a possible implementation, the prompting module 803, when configured to prompt the road condition through the obstacle avoidance device based on the determined risk information and the road scene type, includes:

generating first type prompt information based on the determined risk information of the target object;

generating second type prompt information based on the road scene type;

and road condition prompting is carried out on the first prompting information and the second prompting information through the obstacle avoidance equipment based on a preset prompting sequence.

The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.

Corresponding to the road condition prompting method in fig. 1, an embodiment of the present disclosure further provides an electronic device 900, as shown in fig. 10, which is a schematic structural diagram of the electronic device 900 provided in the embodiment of the present disclosure, and includes:

a processor 91, a memory 92, and a bus 93; the memory 92 is used for storing execution instructions and includes a memory 921 and an external memory 922; here, the memory 921 is also referred to as an internal memory, and temporarily stores operation data in the processor 91 and data exchanged with an external memory 922 such as a hard disk, and the processor 91 exchanges data with the external memory 922 through the memory 921, and when the electronic apparatus 900 is operated, the processor 91 communicates with the memory 92 through the bus 93, so that the processor 91 executes the following instructions: acquiring object attribute information corresponding to a target object within a distance setting range corresponding to the obstacle avoidance device and distance information corresponding to the target object; determining risk information of the target object based on object attribute information and distance information corresponding to the target object; and prompting the road condition through the obstacle avoidance equipment based on the determined risk information.

The embodiment of the present disclosure further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the road condition prompting method in the above method embodiments are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

The computer program product of the road condition prompting method provided in the embodiment of the present disclosure includes a computer readable storage medium storing a program code, where instructions included in the program code may be used to execute the steps of the road condition prompting method in the above method embodiment, which may be referred to specifically in the above method embodiment, and are not described herein again.

The embodiments of the present disclosure also provide a computer program, which when executed by a processor implements any one of the methods of the foregoing embodiments. The computer program product may be embodied in hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.

It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working process of the system and the apparatus described above may refer to the corresponding process in the foregoing method embodiment, and details are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, the division of the units into only one type of logical function may be implemented in other ways, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be through some communication interfaces, indirect coupling or communication connection between devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in software functional units and sold or used as a stand-alone product, may be stored in a non-transitory computer-readable storage medium executable by a processor. Based on such understanding, the technical solutions of the present disclosure, which are essential or part of the technical solutions contributing to the prior art, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A road condition prompting method is characterized by comprising the following steps:

acquiring object attribute information corresponding to a target object within a distance setting range corresponding to obstacle avoidance equipment and distance information corresponding to the target object, wherein the object attribute information is information used for representing target object attribute characteristics and obtained by classifying attributes of different target objects contained in an application scene, and danger coefficients corresponding to different object attribute information are different;

determining category information corresponding to the target object based on the object attribute information corresponding to the target object; the category information comprises a traffic indication category and a non-traffic indication category;

based on the category information and the risk information, sequentially prompting road conditions through the obstacle avoidance device, wherein in the prompting sequence of the road condition prompting, the priority of a first type target object belonging to the traffic indication category is not lower than the priority of a second type target object belonging to the non-traffic indication category;

the road condition prompting is performed through the obstacle avoidance device in sequence based on the category information and the risk information, and the method comprises the following steps:

When the target objects comprise first-class target objects belonging to the traffic indication category, determining a prompt sequence of the first-class target objects when road condition prompt is carried out through the obstacle avoidance equipment on the basis of risk information of the first-class target objects;

2. The road condition prompting method according to claim 1, wherein the obtaining of the object attribute information corresponding to the target object within the set distance range corresponding to the obstacle avoidance device comprises:

acquiring a scene image obtained by shooting a target road condition scene by the obstacle avoidance equipment;

and determining the target object contained in the scene image and object attribute information corresponding to the target object by performing target detection on the scene image.

3. The road condition prompting method according to claim 1, wherein obtaining distance information corresponding to the target object comprises:

acquiring a scene image and a depth image which are obtained by shooting the target road condition scene by the obstacle avoidance equipment;

determining distance information corresponding to the target object based on the scene image and the depth image, wherein the distance information comprises a distance between the target object and the obstacle avoidance device.

4. The road condition prompting method according to claim 3, wherein the determining distance information corresponding to the target object based on the scene image and the depth image comprises:

respectively extracting calibration feature points contained in the scene image and the depth image;

aligning the scene image and the depth image based on the calibration feature point of the scene image and the calibration feature point of the depth image, and extracting the depth information of the target object contained in the scene image from the depth image to obtain the distance information corresponding to the target object.

5. The road condition prompting method according to any one of claims 1 to 4, wherein the determining risk information of the target object based on the object attribute information and the distance information corresponding to the target object comprises:

determining a first risk coefficient corresponding to the target object based on the object attribute information corresponding to the target object;

Determining risk information for the target object based on the distance information, the first risk factor, and the second risk factor.

6. The road condition prompting method according to claim 5, wherein the determining risk information of the target object based on the distance information, the first risk coefficient and the second risk coefficient comprises:

determining a target distance difference value based on the maximum distance corresponding to the shooting range of the obstacle avoidance device and the distance information;

7. The road condition prompting method according to claim 1, wherein the determining a prompting sequence of the first type target object when the road condition prompting is performed through the obstacle avoidance device based on the risk information of the first type target object comprises:

under the condition that the first type of target objects comprise a plurality of target objects, determining a first type of target object to be prompted from the first type of target objects on the basis of first direction information corresponding to the first type of target objects and the movement direction of the obstacle avoidance device, wherein a first angle between a direction indicated by the first direction information corresponding to the first type of target object to be prompted and the movement direction of the obstacle avoidance device is smaller than a set angle threshold value;

determining a prompting sequence of the first class target objects to be prompted when road condition prompting is carried out through the obstacle avoidance equipment based on the risk information of the first class target objects to be prompted;

according to the determined prompt sequence, the road condition prompt is carried out on the first type of target object through the obstacle avoidance device, and the road condition prompt comprises the following steps:

and according to the determined prompting sequence corresponding to the first type of target object to be prompted, under the condition that the distance between the first type of target object to be prompted and the obstacle avoidance device, which is indicated by the distance information corresponding to the first type of target object to be prompted, is smaller than a first preset distance threshold, road condition prompting is performed on the first type of target object to be prompted through the obstacle avoidance device.

8. The road condition prompting method according to claim 7, wherein the determining first direction information that the obstacle avoidance device points to the first type of target object based on the scene image captured by the obstacle avoidance device, the distance information corresponding to the first type of target object, and the parameter information corresponding to the obstacle avoidance device comprises:

determining the position and attitude information of the obstacle avoidance device under a world coordinate system corresponding to a real scene based on the scene image shot by the obstacle avoidance device;

determining two-dimensional detection information of the first class of target objects under an image coordinate system corresponding to the scene image by performing target detection on the scene image;

determining pose information of the first type target object under the world coordinate system based on the two-dimensional detection information, the distance information corresponding to the first type target object and the parameter information corresponding to the obstacle avoidance device;

9. A road condition prompting method according to claim 1, wherein the road condition prompting is performed by the obstacle avoidance device in sequence based on the category information and the risk information, further comprising:

Under the condition that the target objects comprise second type target objects belonging to the non-traffic indication category, determining second type target objects to be prompted, which meet preset conditions, from the second type target objects on the basis of risk information of the second type target objects;

and under the condition that the distance between the second type target object to be prompted and the obstacle avoidance device, which is indicated by the distance information corresponding to the second type target object to be prompted, is smaller than a second preset distance threshold, road condition prompting is performed on the second type target object to be prompted through the obstacle avoidance device in sequence.

10. The road condition prompting method according to claim 9, wherein the determining a second type target object to be prompted which meets a preset condition from the second type target object based on the risk information of the second type target object comprises:

determining second direction information of the obstacle avoidance device pointing to the second type of target object based on a scene image shot by the obstacle avoidance device, distance information corresponding to the second type of target object and parameter information corresponding to the obstacle avoidance device;

Determining a second angle between a second direction indicated by the second direction information and the motion direction of the obstacle avoidance device based on second direction information corresponding to the second type target object and the motion direction of the obstacle avoidance device;

when the second type target objects comprise a plurality of second type target objects, at least one second type target object with a second angle smaller than a preset angle threshold value is determined from the second type target objects and is used as the second type target object to be prompted, and the risk degree indicated by the risk information corresponding to the at least one second type target object is higher than the risk degree indicated by the risk information corresponding to other target objects except the second type target object to be prompted in the second type target object.

11. A road condition prompting method as claimed in any one of claims 1 to 4 or 6 to 10, further comprising:

acquiring a road scene type corresponding to the position of the obstacle avoidance device;

the determining a prompting sequence of the first type target object when road condition prompting is performed through the obstacle avoidance device based on the risk information of the first type target object includes:

And determining a prompting sequence of the first type target objects when road condition prompting is carried out through the obstacle avoidance equipment based on the risk information of the first type target objects and the road scene type.

12. A road condition prompting method according to claim 11, wherein the obtaining of the road scene type corresponding to the position of the obstacle avoidance device comprises:

acquiring position information of the obstacle avoidance equipment;

13. A road condition prompting method according to claim 11, wherein the obtaining of the road scene type corresponding to the position of the obstacle avoidance device comprises:

acquiring a stop board image shot by the obstacle avoidance equipment;

and carrying out scene mark identification on the stop board image, and determining the road scene type corresponding to the area where the obstacle avoidance equipment is located.

14. A road condition prompting method according to claim 11, wherein the prompting the road condition for the first type target object through the obstacle avoidance device according to the determined prompting sequence comprises:

generating first-class prompt information based on the risk information of the first-class target object;

Generating second type prompt information based on the road scene type;

and road condition prompting is carried out on the first type of prompt information and the second type of prompt information through the obstacle avoidance equipment based on a preset prompt sequence.

15. A road condition prompting device is characterized by comprising:

the system comprises an acquisition module and a processing module, wherein the acquisition module is used for acquiring object attribute information corresponding to a target object within a distance setting range corresponding to obstacle avoidance equipment and distance information corresponding to the target object, the object attribute information is information which is obtained by classifying attributes of different target objects contained in an application scene and is used for representing attribute characteristics of the target object, and danger coefficients corresponding to different object attribute information are different;

the determining module is further configured to determine category information corresponding to the target object based on the object attribute information corresponding to the target object; the category information comprises a traffic indication category and a non-traffic indication category;

a prompting module, configured to perform road condition prompting through the obstacle avoidance device in sequence based on the category information and the risk information, where in a prompting sequence of the road condition prompting, a priority of a first category target object belonging to the traffic indication category is not lower than a priority of a second category target object belonging to the non-traffic indication category;

The prompting module is used for prompting the road condition through the obstacle avoidance device based on the category information and the risk information corresponding to the target object, and comprises the following steps:

16. An electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, when the electronic device runs, the processor and the memory communicate through the bus, and when the processor executes the machine-readable instructions, the processor performs the steps of the road condition prompting method according to any one of claims 1 to 14.

17. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the road condition prompting method according to any one of claims 1 to 14 are executed.