CN113591673A - Method and device for recognizing traffic signs - Google Patents

Method and device for recognizing traffic signs Download PDF

Info

Publication number
CN113591673A
CN113591673A CN202110856025.1A CN202110856025A CN113591673A CN 113591673 A CN113591673 A CN 113591673A CN 202110856025 A CN202110856025 A CN 202110856025A CN 113591673 A CN113591673 A CN 113591673A
Authority
CN
China
Prior art keywords
traffic sign
vehicle
information
traffic
validity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110856025.1A
Other languages
Chinese (zh)
Inventor
禹尧
支蓉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mercedes Benz Group AG
Original Assignee
Daimler AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Daimler AG filed Critical Daimler AG
Priority to CN202110856025.1A priority Critical patent/CN113591673A/en
Publication of CN113591673A publication Critical patent/CN113591673A/en
Priority to DE102022002710.0A priority patent/DE102022002710A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/02—Services making use of location information
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
    • H04W4/44—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention relates to the field of target object detection and identification of vehicles. The invention provides a method for identifying traffic signs, comprising the following steps: s1: acquiring motion information of a vehicle and change information in an image sequence of traffic signs in the surrounding environment of the vehicle, which are shot in time succession; s2: determining the validity of the traffic sign based on a comparison of the motion information of the vehicle and the change information of the traffic sign; s3: detecting the position relation between the traffic sign and other vehicles in the surrounding environment of the vehicle; s4: and carrying out credibility check on the validity of the determined traffic sign based on the position relation. The invention also provides a device for identifying traffic signs. In the invention, the traffic sign validity information is rechecked before being provided to the vehicle decision unit, so that only the most valuable information input is considered or considered with higher weight when the vehicle driving assistance/automatic driving function is executed, and the interference of error information to the normal running of the vehicle is reduced.

Description

Method and device for recognizing traffic signs
Technical Field
The invention relates to a method for identifying a traffic sign and to a device for identifying a traffic sign.
Background
With the progress of vehicle intelligence, intelligent sensors equipped with vehicles and/or intelligent computing units receiving and processing sensor data have been able to support accurate recognition of various road traffic signs, thereby enabling recognition results to be used as input for vehicle driving assistance/automatic driving functions. However, in addition to fixed traffic signs located at the roadside, there are many dynamic signs affixed to the rear of large vehicles in an actual traffic environment.
For example, in response to a call by the ministry of public security, many provinces and cities carry out work of pasting the highest speed limit sign at the tail of a large bus, and the purpose of the work is to prompt passengers and masses to supervise the large bus to comply with corresponding speed limit regulations, but the work is not applicable to speed limit of a traveling environment, so that after the vehicle recognizes the speed limit sign, the speed limit sign is easily interpreted as a speed limit standard for the whole road section, and thus unsuitable information is input to a calculation and control unit of the vehicle, which indirectly causes accuracy reduction of driving assistance/automatic driving functions (e.g., Traffic Sign Recognition (TSR), Intelligent Speed Limit Indication (ISLI), Intelligent Speed Limit Control (ISLC), High speed intelligent driving assistance (HWA), and the like) of the vehicle. In addition, there are many situations where large vehicles misuse traffic warning signs, which also may cause certain difficulties for the intelligent sensors of the vehicles and/or the intelligent computing units that receive and process the sensor data.
At present, methods for improving the recognition of traffic signs are proposed in the prior art, in which a series of images of the space in front of the vehicle are first recorded with a camera and the changes in the size and/or position of the traffic sign are checked in these images, respectively, whereby the speed of the traffic sign is deduced and is then compared with the speed of the vehicle to determine whether the traffic sign is stationary or attached to another vehicle.
Furthermore, a method for classifying traffic signs in an environmental area of a motor vehicle is known, in which the geometric dimensions of the traffic signs detected by means of an object detection algorithm are compared with reference dimensions of fixed traffic signs, so that the position of the traffic signs is estimated and the traffic signs are classified as stickers or fixed traffic signs depending on the position.
However, the above solution still has a number of disadvantages, in particular because the vehicle itself is in motion, and there may be errors in the validity of the discrimination of the traffic sign by means of a single recognition result. Furthermore, in some cases, the mere movement information of a traffic sign is not sufficient to distinguish it from a fixed traffic sign precisely.
In this context, it is desirable to provide an improved traffic sign recognition scheme to more reliably discriminate the plausibility and application range of traffic signs in traffic environments.
Disclosure of Invention
It is an object of the present invention to provide a method for identifying a traffic sign and an apparatus for identifying a traffic sign that solves at least some of the problems of the prior art.
According to a first aspect of the invention, a method for identifying a traffic sign is proposed, the method comprising the steps of:
s1: acquiring motion information of a vehicle and change information of a traffic sign in the surrounding environment of the vehicle in an image sequence shot in time succession;
s2: determining the validity of the traffic sign based on a comparison of the motion information of the vehicle and the change information of the traffic sign; and
s3: detecting a positional relationship of the traffic sign to at least one other vehicle in a vehicle surroundings; and
s4: and carrying out credibility check on the validity of the determined traffic sign based on the position relation.
The invention comprises in particular the following technical concepts: first, by considering information about the changes of the traffic sign in the image sequence and the movement information of the vehicle itself, a preliminary conclusion about the validity of the traffic sign can be drawn. The reliability of this preliminary conclusion is then verified by detecting the positional relationship of the traffic sign to other traffic objects. Thereby, it is achieved in an advantageous manner: the validity information about the traffic sign is reviewed before it is provided to the vehicle decision unit, thereby ensuring that only the most valuable information input is considered or considered with a higher weight when generating the vehicle driving assistance/automatic driving function, thus reducing to some extent the interference of erroneous information with the normal driving of the vehicle.
Optionally, the method further comprises the steps of:
s3': receiving external input information about the vehicle surroundings by means of the V2X technique; and
s4': a plausibility check of the validity of the determined traffic sign is carried out on the basis of the external input information.
Thereby, the following technical advantages are achieved: and the data source for discriminating the effectiveness of the traffic sign is expanded. Under the condition that the reasonable application range of the traffic sign cannot be accurately judged by the vehicle self-sensor (for example, due to function limitation, view obstruction and the like), the deficiency of the vehicle self-sensing result can be made up by combining with multi-source data outside the vehicle, and the validity of the traffic sign is favorably verified from multiple aspects.
Optionally, the motion information of the vehicle includes a displacement variation amount of the vehicle within a certain period of time, and the variation information of the traffic sign includes a 3D position variation amount of the traffic sign recognized within the certain period of time by means of the vehicle-mounted camera.
Thereby, the following technical advantages are achieved: in the driving process of the vehicle, the identification of the traffic signs or other scenes is realized in a mode of combining with visual perception anyway, and the existing 3D target detection algorithms based on the visual perception are various, and no specific algorithm is required.
Optionally, the method further comprises, at least before step S3, the steps of:
determining the speed and/or position of the traffic sign relative to the vehicle using a plurality of onboard sensors in a fused manner; and
additionally determining the validity of the traffic sign based on the speed and/or position.
Thereby, the following technical advantages are achieved: the sensors based on different principles usually have different task emphasis and applicable conditions, and the identification results can be verified among different sensors in a multi-sensor fusion mode, so that multi-space information complementation and combination processing are realized, and a more accurate identification result is obtained.
Optionally, the step S2 includes:
classifying the traffic sign as a static traffic sign or a dynamic traffic sign based on a result of the comparison; and
the traffic sign is determined to be a valid traffic sign if the traffic sign is classified as a static traffic sign.
Thereby, the following technical advantages are achieved: by means of the differentiation based on the static characteristic and the dynamic characteristic, the interference item can be eliminated as much as possible in the initial detection stage, so that even if the subsequent rechecking process is not implemented or can be implemented only in a limited way, the error recognition result can be reduced to the greatest extent, and the acceptance of the driving assistance function is improved.
Alternatively, in the step S4, when the position of the traffic sign overlaps with the position of at least one other vehicle in the vehicle surroundings, the determined valid traffic sign is determined as invalid.
Thereby, the following technical advantages are achieved: for a traffic sign suspended from another vehicle, if the other vehicle is stationary (e.g. parked at the roadside), it is not sufficient to distinguish it from a fixed traffic sign originally placed at the roadside simply by the static/dynamic nature of the traffic sign. In this case, if it is sufficiently known whether or not a positional overlap occurs between the traffic sign and other vehicles in the vicinity thereof, it is possible to reliably verify whether or not the traffic sign is actually attached to the other vehicles.
Optionally, the external input information comprises presence information about a fixed traffic sign in the surroundings of the vehicle, and in said step S4', the determined valid traffic sign is only confirmed as valid if said presence information indicates the presence of a fixed traffic sign in the surroundings of the vehicle.
Thereby, the following technical advantages are achieved: by knowing whether a fixed traffic sign exists near the vehicle, the identified effective traffic sign can be compared with the reference information, so that the correctness of the initial inspection result is verified.
Optionally, the external input information comprises category information about at least one traffic object in the vehicle surroundings, and in step S4', the determined invalid traffic sign is determined to be valid when the category information of the at least one traffic object indicates at least one predefined category.
Thereby, the following technical advantages are achieved: if it is known that there are traffic objects of a particular class in the surroundings of the vehicle, this may indicate that the vehicle is in an extreme application scenario, where the criteria for determining the validity of the traffic sign may change. For insurance purposes, traffic signs that have been determined to be invalid should be reclassified as valid to maximize safety.
Optionally, the method further comprises the steps of: the plausibility-checked information about the validity of the traffic sign is transmitted to at least one further vehicle, infrastructure and/or road supervision platform.
Thereby, the following technical advantages are achieved: by this sharing of information, other vehicles in the vehicle surroundings can be informed in time, so that they are less disturbed by invalid traffic signs. In addition, the validity of the identification result can be further verified by means of other platforms.
According to a second aspect of the invention, a device for identifying a traffic sign for performing the method of the first aspect of the invention is proposed, the device comprising:
an acquisition module configured to be able to acquire motion information of a vehicle and change information in a sequence of images taken temporally consecutively of a traffic sign in a vehicle surroundings, the acquisition module further configured to be able to detect a positional relationship of the traffic sign with at least one other vehicle in the vehicle surroundings;
a determination module configured to be able to determine the validity of the traffic sign based on a comparison of the movement information of the vehicle and the change information of the traffic sign; and
a confidence check module configured to enable a confidence check on the validity of the determined traffic sign based on the positional relationship.
Drawings
The principles, features and advantages of the present invention may be better understood by describing the invention in more detail below with reference to the accompanying drawings. The drawings comprise:
fig. 1 shows a block diagram of an apparatus for recognizing traffic signs according to an exemplary embodiment of the present invention;
FIG. 2 shows a flow diagram of a method for identifying traffic signs according to an exemplary embodiment of the present invention;
FIG. 3 shows a flow chart of two steps of a method for identifying traffic signs according to an exemplary embodiment of the present invention; and
fig. 4a and 4b show schematic diagrams of the use of the method according to the invention in an exemplary application scenario.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects of the present invention more apparent, the present invention will be described in further detail with reference to the accompanying drawings and exemplary embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of the invention.
Fig. 1 shows a block diagram of a device 1 for recognizing traffic signs according to an exemplary embodiment of the present invention.
As shown in fig. 1, the apparatus 1 is used, for example, for a vehicle 2 having a driving assistance/automatic driving function. The vehicle 2 may, for example, travel partially autonomously or may also travel completely autonomously. In order to achieve improved traffic sign recognition, the device 1 comprises an acquisition module, a determination module 20 and a plausibility check module 30. Here, the acquisition module is not shown in the form of a single integrated module, but is shown, for example, as including a plurality of types of vehicle-mounted sensors (e.g., the camera 11, the radar sensor 12, and the vehicle wheel speed sensor 14). Furthermore, the acquisition module for example further comprises a data fusion unit 13.
The camera 11 comprises, for example, a monocular, binocular or stereo camera configured to take at least one image of the vehicle surroundings at different times and to recognize traffic signs and other types of traffic objects according to a corresponding 3D object detection algorithm. The radar sensor 12 includes, for example, a laser radar, a millimeter wave radar, or the like, which is configured to detect distance and speed information of a traffic object in the vehicle surroundings. The vehicle wheel speed sensor 14 is configured to detect a wheel rotation speed, for example, whereby the speed of the vehicle can be calculated.
Furthermore, the acquisition module may also include other types of sensors, such as GNSS sensors, inertial sensors, ultrasonic sensors, infrared sensors, etc., whose detection results may be jointly analyzed and processed in the fusion unit 13 to obtain information describing the movement characteristics or position characteristics of the traffic sign and the vehicle.
After the acquisition of the corresponding sensor information by means of the acquisition module, this information is transmitted to the determination module 20, in order to compare the movement information of the vehicle there with the time-dependent change information of the traffic sign, in order to determine the validity of the traffic sign.
According to an exemplary embodiment of the present invention, this validity statement is not provided directly for the driving assistance function of the vehicle, before which this validity information of the traffic sign can also be checked again by means of the plausibility check module 30. To perform a review, the plausibility check module 30 may, for example, receive from an acquisition module (e.g., the data fusion unit 13) a positional relationship of the traffic sign with other vehicles in the vehicle surroundings and then verify whether the validity conclusion is reliable on the basis of this positional relationship.
The device 1 further comprises, for example, a communication interface 15 based on car networking technology configured to receive verified traffic sign validity information from the trustworthiness verification module 30 and to share such information to other traffic participants, infrastructure and/or road supervision platforms. Furthermore, the communication interface 15 may also be part of the acquisition module and thus also be configured to receive information about external inputs from the vehicle surroundings and to provide it to the plausibility check module 30 in order to be able to assist in verifying the validity of the traffic sign on the basis of the internet of vehicles data.
Fig. 2 shows a flow chart of a method for identifying traffic signs according to an exemplary embodiment of the present invention. The method can be carried out, for example, using the device 1 according to fig. 1.
The method comprises a step S1 in which movement information of the vehicle and information of changes in a sequence of images taken temporally one after the other of a traffic sign in the surroundings of the vehicle are acquired. The motion information of the vehicle includes, for example, the speed and displacement variation amount of the vehicle. The change information of the traffic sign may be, for example, a change in the geometric outline and/or size of the traffic sign, or a 3D displacement change amount of the traffic sign detected in different images. The purpose of acquiring the change information of the traffic sign is to find out the displacement change quantity of the traffic sign relative to the vehicle in a determined time period. Optionally, in step S1, the speed and/or position of the traffic sign relative to the vehicle may additionally be determined in a fused manner by means of a plurality of vehicle-mounted sensors.
Next, it is determined whether the identified traffic sign is valid based on a comparison of the motion information of the vehicle and the amount of change in the 3D position of the traffic sign in step S2. Optionally, the validity may be determined in this step additionally on the basis of the relative speed and/or the relative position of the traffic sign resulting from the fusion of a plurality of sensors with respect to the own vehicle. This has the advantage that the movement characteristics of the traffic sign can be analyzed in an integrated manner on the basis of the recognition results of the plurality of sensors, thereby increasing the accuracy of the validity determination. For example, if it is recognized that the traffic sign may be located in the center of the road based on the relative positions of the traffic sign and the own vehicle, it may be excluded from the category of valid traffic signs at the beginning or the result of 3D object detection may also be verified using such information. Here, the reference point of the relative speed and/or the relative position may be decided by the developer based on the design and algorithm, which may be, for example, the front axle center, the front bumper midpoint, the rear axle center, etc. of the own vehicle. In the sense of the present invention, a valid traffic sign indicates, for example, that the range of application of the traffic sign is suitable for the traffic environment in which the vehicle is currently located, and therefore the recognition result about the traffic sign can be used for information presentation or vehicle control. The invalid traffic indication then indicates, for example, that the traffic sign is of abuse or is only used for special purposes (e.g. is only intended to give warning effect to other vehicles or persons), in which case the recognition result of such a traffic sign should not be provided for the driving assistance function of the own vehicle.
According to one embodiment, if the traffic sign is determined to be valid, the positional relationship of the traffic sign with at least one other vehicle in the vehicle surroundings may also be acquired in step S3. In this case, for example, the 3D position of objects in the surroundings of the vehicle can be detected by means of a vehicle-mounted camera (for example, based on a corresponding 3D object detection algorithm) and it can be determined therefrom whether the 3D position of the traffic sign overlaps other vehicles.
In step S4, the information on the validity of the traffic sign previously obtained in step S2 is checked for reliability based on the positional relationship. For example, the degree of coincidence of the 3D position information of the object around the vehicle and the 3D position information of the traffic sign may be compared, and it is determined that there is position overlap if the degree of coincidence reaches a predefined threshold value. Whether the traffic sign and at least one target (such as other vehicles) in the surrounding limited range have the same 3D position information can also be judged, and if the traffic sign and the at least one target have the same 3D position information, whether the traffic sign is overlapped in position or not can be further judged.
If a traffic sign is found to overlap at least one vehicle, this means in particular that the traffic sign is not present in the form of a separate individual but is attached to another vehicle. This means, for example, that the traffic sign identified is not a standard traffic sign fixed on the roadside, but a dynamic label affixed to the rear of another vehicle. In this case, such traffic signs should be considered invalid, regardless of the validity conclusions drawn in the preceding steps. In this case, the already determined valid traffic sign may be confirmed as invalid in step S5, for example.
In contrast, if the occurrence of the position overlap is not found in step S4, the previous validity determination result, i.e., the valid traffic sign that has been determined, continues to be held valid in step S6.
According to another embodiment, if it is determined in step S2 that the traffic sign is valid, external input information about the vehicle surroundings may also be received by means of the V2X technique in step S3'. As an example, the external input information may include category information about at least one traffic object in the vehicle surroundings. As another example, the external input information includes presence information about a fixed traffic sign in the vehicle surroundings. In turn, input information including a (dynamic) traffic sign of the vehicle surroundings is received, for example, in a safe, reliable and real-time manner by means of the V2X technology.
In step S4', it may be determined whether the traffic object belongs to a predefined category. As an example, it may be determined by means of external input information whether a police car with a speed limit sign attached thereto passes through the vehicle surroundings. For another example, it is possible to determine whether a transport vehicle that transports dangerous goods or is facing a safety problem is present in the vehicle surroundings by means of external input information.
If at least one traffic object is determined to belong to the predefined category. The determined invalid traffic sign is confirmed as valid in step S7. This is done in order that if there is a special category of vehicle (police car, hazardous material transport vehicle, etc.) located in the current driving environment of the vehicle, the original validity judgment criterion may no longer apply, so that for caution, even if an invalid traffic sign has been identified, the content of the traffic sign should be properly considered or certain prompts given to the persons in the vehicle when performing the driving assistance function.
In contrast, if the external input information does not indicate that there is a traffic object of a predefined category in the vehicle surroundings, the previous validity determination result may be continued to be maintained in step S8.
Alternatively, it may additionally be determined in step S4' whether the external input information includes presence information about a fixed traffic sign in the vehicle surroundings. The determined valid traffic sign is only confirmed as valid if the presence information indicates that a fixed traffic sign is indeed present in the vehicle surroundings.
Fig. 3 shows a flow chart of two steps of a method for identifying traffic signs according to an exemplary embodiment of the present invention. In the exemplary embodiment, method step S1 in FIG. 2 includes sub-steps S11-S12, and method step S2 includes sub-steps S21-S24.
In step S11, the 3D position variation amount of the traffic sign in the time period between the first time and the second time is calculated. In this case, for example, images of the surroundings of the vehicle can be recorded at a first and a second time using the onboard camera and the traffic sign can be recognized in the respective images. According to the 3D target detection algorithm, the 3D positions D of the traffic sign at the first moment and the second moment can be respectively obtained_sign_time_1、D_sign_time_2. In the sense of the present invention, such a 3D position represents the position of the traffic sign relative to the own vehicle. Then, the 3D position change amount of the traffic sign can be found by, for example, the following equation: d_sign＝D_sign_time_1-D_sign_time_2。
In step S12, the amount of change in displacement of the vehicle over the period between the first time and the second time is calculated. In this case, the speed v of the vehicle at a certain point in time is sensed, for example, by means of a vehicle wheel speed sensor_currentThe displacement variation over a period of time is:
Figure BDA0003183983770000091
alternatively or additionally, the position P of the vehicle at the first and second time can also be determined by means of a positioning module of the vehicle itself_vehicle_time_1、P_vehicle_time_2The displacement variation, P, can also be calculated by comparing the positions at different times_vehicle＝P_vehicle_time_1–P_vehicle_time_2。
In step S21, the 3D position variation amount of the traffic sign is compared with the displacement variation amount of the vehicle. Here, for example, a deviation relationship between the amount of change in the 3D position of the traffic sign and the amount of change in the vehicle displacement may be obtained.
In step S22, it is determined whether the traffic sign is dynamic or not based on the result of the comparison. As an example, the amount of change D when the 3D position of a traffic sign changes_signDisplacement variation D of approaching vehicle_vehicleWhen, i.e. for example, satisfyRelation D_vehicle*80％≤D_sign≤D_vehicleAnd when the speed is 120%, judging that the traffic sign is static. As an example, the amount of change D when the 3D position of a traffic sign changes_signAnd the vehicle displacement variation D_vehicleWhen the difference is large, i.e. for example when D_sign＜D _vehicle80% or D_sign＞D_vehicleAnd when the speed is 120%, judging that the traffic sign is dynamic.
If the traffic sign is determined to be dynamic, it indicates that the traffic sign is likely to hang on and travel with other vehicles, in which case it is determined in step S24 that an invalid traffic sign is recognized.
If the traffic sign is determined to be static, it indicates that the traffic sign is a stationary object fixed on the roadside, and thus it may be determined in step S23 that a valid traffic sign is recognized.
Fig. 4a and 4b show schematic diagrams of the use of the method according to the invention in an exemplary application scenario.
In the illustrated scenario, images of the road surface ahead of the vehicle are captured at different times during the driving of the vehicle, for example by means of a camera arranged on the vehicle. Fig. 4a shows an image taken at a first moment in time, and fig. 4b shows an image taken at a second moment in time after the first moment in time.
As can be seen from fig. 4a, a large bus 41 is present in the traveling state at a short distance in front of the host vehicle, and a first speed limit sign 401 is attached to the rear of the large bus 41. In addition, a second speed limit sign 402 attached to the rear of the large truck 42 and a third speed limit sign 403 fixed to the roadside are provided at the front right of the vehicle. In this exemplary scenario, the large truck 42 is parked over the curb and is therefore stationary. Three speed limit signs 401, 402, 403 can be captured in the image, for example, by a corresponding 3D object detection algorithm.
In order to check the validity of the three speed limit signs, an image of the road surface ahead is again recorded at a second point in time, which is shown by way of example in fig. 4 b. As can be seen from observing fig. 4b, since the large bus 41 travels faster than the host vehicle, the distance of the large bus 41 from the host vehicle becomes larger as time passes. Further, as the vehicle advances, the distance between the speed limit sign 403 fixed to the roadside and the large truck 42 parked at the roadside becomes smaller with respect to the vehicle. With the method according to the invention, it can be determined that the first speed limit sign 401 belongs to a dynamic sign and the second and third speed limit signs 402, 403 belong to static signs on the basis of a comparison of the displacement change of the vehicle itself with the recognized 3D position change of the respective speed limit signs 401, 402, 403, so that in the preliminary validity determination process the first speed limit sign 401 is classified as an invalid traffic sign and the second and third speed limit signs 402, 403 are classified as valid traffic signs.
In this exemplary scenario, the second speed limit sign 402 is essentially only intended to let other traffic participants supervise the driving behavior of the large bus 42, which is not applicable to the speed regulation of the current road segment. However, since the second speed limit sign 402 is suspended from the stationary vehicle 42, it is also recognized as a static object in the preliminary validity determination according to the movement characteristics and is therefore erroneously classified as valid. It follows that simply relying on the speed information of a traffic sign is not sufficient for such a posted identifier to be reliably distinguished from a normal speed limit sign.
In this case, it is necessary to further verify the result of the validity determination based on the 3D positional relationship of the traffic sign and the other vehicle. For example, the 3D position information of the second speed limit sign 402 and the 3D position information of the vehicle 42 in the vicinity thereof may be acquired based on a 3D object detection algorithm, and the two 3D position information may be absolute positions in a world coordinate system or relative positions with respect to a specific coordinate system, and it is only necessary to ensure that the 3D position information of the two is calculated based on the same reference coordinate system. Here, for example, it is judged that the position of the second speed limit sign 402 overlaps with the 3D position of the large truck 42, and it is inferred that the second speed limit sign is attached to the large truck 42, and therefore the previous validity conclusion thereof needs to be corrected.
It can also be seen in fig. 4a-4b that in the vehicle surroundings there is also another vehicle 43 with an internet of vehicles communication interface 50, which vehicle 43 is equipped with and knows from, for example, the latest version of a high-precision map: there is only one fixed speed limit sign in the current vehicle environment, and the fixed speed limit sign indicates a speed limit requirement of "80 km/h". The vehicle 43 shares such information to the own vehicle, for example. When such information is received, the own vehicle can be used for checking the reliability of the validity of the traffic sign.
Although specific embodiments of the invention have been described herein in detail, they have been presented for purposes of illustration only and are not to be construed as limiting the scope of the invention. Various substitutions, alterations, and modifications may be devised without departing from the spirit and scope of the present invention.

Claims (10)

1. A method for identifying a traffic sign (401, 402, 403), the method comprising the steps of:
s1: acquiring motion information of a vehicle (2) and change information in a sequence of images taken in time succession of traffic signs (401, 402, 403) in the vehicle's surroundings;
s2: determining the validity of a traffic sign (401, 402, 403) based on a comparison of the movement information of the vehicle (2) and the change information of the traffic sign (401, 402, 403);
s3: detecting a positional relationship of the traffic sign (401, 402, 403) to at least one other vehicle (42) in a vehicle surroundings; and
s4: the validity of the determined traffic sign (401, 402, 403) is checked for plausibility on the basis of the positional relationship.
2. The method of claim 1, wherein the method further comprises the steps of:
s3': receiving external input information about the vehicle surroundings by means of the V2X technique; and
s4': the validity of the determined traffic sign (401, 402, 403) is checked for plausibility on the basis of the external input information.
3. The method according to claim 1 or 2, wherein the movement information of the vehicle (2) comprises an amount of change of displacement of the vehicle (2) over a determined period of time, and the change information of the traffic sign (401, 402, 403) comprises an amount of change of 3D position of the traffic sign (401, 402, 403) identified over the determined period of time by means of the vehicle-mounted camera (11).
4. The method according to any one of claims 1 to 3, wherein the method further comprises, at least before step S3, the steps of:
determining the speed and/or position of the traffic sign (401, 402, 403) relative to the vehicle (2) using a plurality of onboard sensors in a fused manner; and
additionally determining the validity of the traffic sign (401, 402, 403) based on the speed and/or the position.
5. The method according to any one of claims 1 to 4, wherein the step S2 includes:
classifying the traffic sign (401, 402, 403) as a static traffic sign or a dynamic traffic sign based on a result of the comparison; and
determining a traffic sign (401, 402, 403) as a valid traffic sign if the traffic sign (401, 402, 403) is classified as a static traffic sign.
6. The method according to any one of claims 1 to 5, wherein in step S4, an already determined valid traffic sign is confirmed as invalid when the position of the traffic sign (401, 402, 403) overlaps with the position of at least one other vehicle (42) in the vehicle surroundings.
7. The method according to claim 2, wherein the external input information comprises presence information about a fixed traffic sign in the vehicle surroundings, and in step S4', an already determined valid traffic sign is only confirmed as valid if the presence information indicates the presence of a fixed traffic sign in the vehicle surroundings.
8. The method according to claim 2, wherein the external input information comprises category information about at least one traffic object in the vehicle surroundings, and in step S4', the determined invalid traffic sign is confirmed as valid when the category information of the at least one traffic object indicates at least one predefined category.
9. The method according to any one of claims 1 to 8, wherein the method further comprises the steps of:
the plausibility-checked information about the validity of the traffic sign (401, 402, 403) is transmitted to at least one further vehicle (43), infrastructure and/or road supervision platform.
10. A device (1) for identifying a traffic sign (401, 402, 403), the device being adapted to perform the method according to any one of claims 1 to 9, the device (1) comprising:
an acquisition module configured to be able to acquire motion information of a vehicle (2) and change information in a sequence of images taken in temporal succession of a traffic sign (401, 402, 403) in a vehicle surroundings, the acquisition module being further configured to be able to detect a positional relationship of the traffic sign (401, 402, 403) with at least one other vehicle (42) in the vehicle surroundings;
a determination module configured to be able to determine the validity of a traffic sign (401, 402, 403) based on a comparison of the movement information of the vehicle (2) and the change information of the traffic sign (401, 402, 403); and
a plausibility check module configured to enable plausibility checking of the validity of the determined traffic sign (401, 402, 403) on the basis of the positional relationship.
CN202110856025.1A 2021-07-28 2021-07-28 Method and device for recognizing traffic signs Pending CN113591673A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202110856025.1A CN113591673A (en) 2021-07-28 2021-07-28 Method and device for recognizing traffic signs
DE102022002710.0A DE102022002710A1 (en) 2021-07-28 2022-07-26 Method and device for traffic sign recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110856025.1A CN113591673A (en) 2021-07-28 2021-07-28 Method and device for recognizing traffic signs

Publications (1)

Publication Number Publication Date
CN113591673A true CN113591673A (en) 2021-11-02

Family

ID=78250887

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110856025.1A Pending CN113591673A (en) 2021-07-28 2021-07-28 Method and device for recognizing traffic signs

Country Status (2)

Country Link
CN (1) CN113591673A (en)
DE (1) DE102022002710A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115394077A (en) * 2022-08-18 2022-11-25 中国第一汽车股份有限公司 Speed limit information determining method and device and nonvolatile storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115394077A (en) * 2022-08-18 2022-11-25 中国第一汽车股份有限公司 Speed limit information determining method and device and nonvolatile storage medium
CN115394077B (en) * 2022-08-18 2023-10-27 中国第一汽车股份有限公司 Speed limit information determining method and device and nonvolatile storage medium

Also Published As

Publication number Publication date
DE102022002710A1 (en) 2023-02-02

Similar Documents

Publication Publication Date Title
JP4420011B2 (en) Object detection device
US10395527B2 (en) Method and control and detection device for a plausibility check of a wrong-way driving incident of a motor vehicle
US10480949B2 (en) Apparatus for identifying position of own vehicle and method for identifying position of own vehicle
US10691125B2 (en) Blinker judgment device and autonomous driving system
CN105128836A (en) Autonomous emergency braking system and method for recognizing pedestrian therein
EP2884476A1 (en) Object detection apparatus and driving assistance apparatus
JP6654923B2 (en) Map information output device
CN106537180A (en) Method for mitigating radar sensor limitations with video camera input for active braking for pedestrians
CN106157664A (en) A kind of road speed limit identification recognition device
CN104742901B (en) Method and control and detection device for detecting the entry of a motor vehicle into a driving lane of a road counter to the direction of travel
US20100052944A1 (en) Vehicle and Method for Identifying Vehicles Located in the Surroundings of the Vehicle
JP2014241036A (en) Vehicle driving assist system
WO2019216386A1 (en) Vehicle control device and vehicle
JP2020106915A (en) Information presentation device
JP2022172444A (en) Method and assist device for assisting traveling operation of motor vehicle, and motor vehicle
CN111352414B (en) Decoy removing device and method for vehicle and vehicle comprising the device
JP2024072009A (en) Drive support device
CN109195849B (en) Image pickup apparatus
CN112352268A (en) Drive assist system for vehicle, vehicle with drive assist system, and drive assist method for vehicle
CN113591673A (en) Method and device for recognizing traffic signs
KR20160131196A (en) Device for detecting an obstacle
CN112466139A (en) Speed limit fusion method and system based on multiple sensors, vehicle and storage medium
US20240127694A1 (en) Method for collision warning, electronic device, and storage medium
CN115017967A (en) Detecting and collecting accident-related driving experience event data
CN114333414A (en) Parking yield detection device, parking yield detection system, and recording medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination