Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
  • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
  • G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/20—Image preprocessing
  • G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/20—Image preprocessing
  • G06V10/30—Noise filtering
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/60—Type of objects
  • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
  • G06V20/63—Scene text, e.g. street names

Definitions

• the invention relates to a method and a device for detecting the content of traffic signs according to the preamble of the independent claims.
• Modern driver assistance systems are increasingly equipped with an electronic traffic sign recognition system, for example, to warn the driver when exceeding the maximum speed limit.
• an electronic traffic sign recognition system typically takes a camera on the environment in front of the vehicle and delivers corresponding image data to a
• Image evaluation device that uses an algorithm to analyze and classify the image data to identify a traffic sign. Such a method is e.g. known from DE 198 52 631 AI.
• the information from such detected traffic signs can then be incorporated into a driver assistance function, eg. B. as an indication of the currently permissible maximum speed in the instrument cluster of the vehicle.
• WO 2009/135460 Al shows a method for
• Exposure control for a camera in a motor vehicle which makes it possible to realize two driver assistance functions with different exposure requirements, such as a lane detection and traffic character recognition at night, with a common camera.
• a sufficiently short exposure time can be used for traffic sign recognition image data, in particular at high driving speeds, in order to avoid or minimize "motion-blur", ie motion blur, so that the statement of the traffic sign can be recognized.
• the traffic signs or signs erected in the road are provided with one or more additional symbols which concretize the meaning of the main character or limit it situationally.
• WO 2013/017125 A1 shows a method for recognizing traffic signs in which additional characters are classified as far as possible by means of a pattern recognition method, additional character text is read and interpreted by means of a text recognition method, if the classification was not or not completely possible and a relevance assessment is made the recognized text is compared with situation data.
• a fundamental difficulty in reading textual content and, in particular, variable text elements on road signs, is that it requires images that have sufficient resolution and sharpness.
• the invention has for its object to meet this difficulty and to provide a method for fast and reliable detection of the content of traffic signs with variable text component.
• a starting point of the invention are the following considerations: Although cameras today can offer basically a good resolution and sharpness, this is when using cameras in the moving vehicle, with different
• the approach to determine the motion blur of a traffic sign from an image or from an image sequence Since it will not be possible for economic reasons in the foreseeable future to increase the image quality, ie the optics and the resolution and sensitivity of the imager, in vehicle cameras so that a reading of traffic signs or signs with a variable text component will be immediately possible the present invention, the approach to determine the motion blur of a traffic sign from an image or from an image sequence.
• An inventive method for detecting a traffic sign by means of a camera from a moving vehicle has the following steps:
• the camera is in particular a forward-facing monocamera, preferably with color-resolving
• CMOS complementary metal-oxide-semiconductor
• CCD Image recording sensor
• a stereo camera has the advantage that it provides spatial information about the vehicle environment.
• further camera-based driver assistance functions can preferably be implemented in addition to traffic sign recognition, such as e.g. a lane departure warning, a lane keeping assistance, an automatic high beam control, a collision warning, a precipitation detection, an automatic longitudinal control, a parking assistance, automatic emergency braking and / or emergency steering systems.
• traffic sign recognition such as e.g. a lane departure warning, a lane keeping assistance, an automatic high beam control, a collision warning, a precipitation detection, an automatic longitudinal control, a parking assistance, automatic emergency braking and / or emergency steering systems.
• the determination of the motion blur comprises the estimation of the motion blur, but also the calculation of the motion blur, in particular one
• the influence of the motion blur on the recorded image can be calculated out by equalizing the blurred image or a blurred image area (deconvolution). This leads to a sharpened image (area).
• the recognition of the traffic sign taking into account the sharpened image area can take place via pattern recognition, classification or text recognition (OCR).
• a method according to the invention has the advantage that the influence of the motion blur on surrounding images of a camera of a moving vehicle, which contain traffic signs, can be attenuated or even eliminated, whereby a reliable and complete detection of traffic signs is possible.
• the step of "recognizing the traffic sign taking into account the sharpened image area" comprises a text recognition in the sharpened
• Image area This allows complete recognition of variable text components in traffic signs.
• data are evaluated which characterize the proper motion of the vehicle.
• the motion blur of traffic signs recorded while driving stems only from the camera movement, so typically in a fixed camera in the vehicle only from the vehicle's own movement ago. This cause can be taken into account by knowing about the current proper motion of the vehicle optionally in combination with imaging properties of the camera system and the geometry between the camera and traffic signs.
• Such data may, for example, provide vehicle motion sensors.
• Image data evaluations can also provide indications of the vehicle's own motion, in particular in the case of a stereo camera. According to a preferred embodiment, edges in this image area are determined and evaluated in order to determine the motion blur in the image area in which the existing traffic signs are located.
• Traffic sign used from the recorded image A predefined template traffic sign corresponding to the partially recognized traffic sign is taken into account in determining the motion blur in the image area. Taking into account the sharpened image area, the variable content of the traffic sign is recognized. As a result, the traffic sign, in particular a main character, already partially or even completely recognized at the time ofsecuritysun2020rfeCt tion, so its appearance without motion blur (template traffic sign) is known and can be taken into account in determining the motion blur.
• the static content comprises the form and outline of a main traffic sign.
• the variable content is then the content included in the image area of the border of the main traffic sign.
• the image area is formed in such a way that it contains at least one main traffic sign and all auxiliary signs assigned to this main traffic sign.
• this can only be a single additional character if only one additional character is assigned to the main traffic sign.
• the at least one main traffic sign can then be completely recognized and the contents of the additional sign (s) identified as variable content.
• a separate image area is formed for each individual traffic sign (main character or additional sign) in the image. It is then checked whether at least two traffic signs are assigned to each other, in particular based on their geometric arrangement to each other.
• the motion blur is determined only for the first image area and then adopted for the second and the further image areas.
• the first image area can advantageously contain a main traffic sign, in particular a
• Main traffic sign which can be completely recognized without text recognition.
• a synthetic high-definition image area is created from a plurality of successive ones with the camera Recorded images extracted sharpened image areas in each of which is the same traffic sign.
• the traffic sign is then recognized taking into account the synthetic high-definition image area, which increases the reliability and availability of the
• the invention further relates to a device for detecting traffic signs comprising a camera for recording at least one image of a vehicle environment from a moving vehicle and an image evaluation unit, wherein the image evaluation unit is designed such that it detects the presence of one or more traffic signs (Hz, Zz) from the detected motion picture data detects a motion blur in the image area in which the existing traffic sign (Hz, Zz) is located, which calculates motion blur in this image area, resulting in a sharpened image area, the traffic sign (Hz, Zz) recognizes taking into account the sharpened image area and can generate a corresponding output signal.
• the image evaluation unit is designed such that it detects the presence of one or more traffic signs (Hz, Zz) from the detected motion picture data detects a motion blur in the image area in which the existing traffic sign (Hz, Zz) is located, which calculates motion blur in this image area, resulting in a sharpened image area
• the traffic sign (Hz, Zz) recognizes taking into account the sharpened image area and can generate a corresponding output
• Fig. 1 Examples of traffic signs with variable
• Fig. 2 a main character with variable text content
• Fig. 3 a speed-limiting
• Fig. 4 a main character with associated
• FIG. 1 shows traffic signs with variable contents, in particular variable text contents:
• FIG. 1a shows main characters with variable text content, namely a prohibition for vehicles (starting from the left) over 5.5 tons actual mass, over 2 meters actual width (including outside mirrors) resp over 8 tons of actual axle load.
• Fig. Lb shows additional characters with variable text content, which limit the validity of the associated main character (not shown in Fig. 1), namely to a distance from the main character of 600 meters, a period of 8-11 clock and 16-18 clock or for vehicles with an actual mass of 7.5 tonnes or more. 1, in particular, the numerical values are "variable text contents.”
• Complete recognition of such traffic signs, in particular by means of pattern recognition, is difficult, and requires text recognition (eg, OCR) Images of these traffic signs, which have a sufficient resolution and sharpness.
• images taken with a vehicle camera from a moving car have a blurring that results from the movement of the vehicle and thus the camera relative to the traffic sign. In English, this motion blur is referred to as "motion blur.”
• This motion blur is referred to as "motion blur.” The shorter the exposure time when taking a picture of a camera, the lower the motion blur is, but in the dark a minimum exposure time is required to see anything from the captured image data.
• Hz, Zz traffic signs
• an image of the vehicle environment recorded by the camera is searched for existing traffic signs (Hz, Zz). If traffic signs (Hz, Zz) are determined in an image area, the motion blur in this image area is determined (calculated or at least estimated). This calculation or estimation is the subject of current research, especially in hand-held cell phone / photo cameras that often provide shaky images when triggered.
• the motion blur is typically determined as a "blur kernel" (motion blur core) or as a "point spread function”.
• Fig. 2 shows the first traffic sign of Fig. La: a ban on vehicles with an actual mass of over 5, 5 tons.
• This traffic sign can be subdivided into a static component (S), namely its shape and outline, ie essentially the red ring which is indicative of prohibition signs, and a variable component (V), here the text "5,5t"
• S static component
• V variable component
• this traffic sign (Hz) is a prohibition sign
• Image data with motion blur are not immediately recognized because they are too noisy than that a text recognition would be successful. Therefore, the motion blur for the image area in which the traffic sign (Hz) is included is estimated. This is possible because the circular shape and the red border of the prohibition sign are very well detectable, and especially at the time ofuploadssun2020rfeush can already be detected, whereby their appearance is known without motion blur.
• This motion blur estimate is then used to sharpen the image area within the circular red border (V) by excluding motion blur. Subsequently, a recognition or text recognition of this inner image area (V) is possible, which results in the fact that the characters "5.5 t" are contained there, thereby completely detecting the regulatory content of this traffic sign, namely a prohibition for vehicles with an actual mass of over
• This can now be communicated to the driver of a vehicle (eg display of a corresponding symbol in the instrument cluster) or with vehicle-specific data such as the current or maximum actual mass of the vehicle adjusted and discarded if not relevant (eg for a car) or even be output from the traffic sign recognition to a vehicle navigation device or an autonomous vehicle control device.
• FIG. 3 shows a main character (Hz) which restricts the maximum permissible speed to 50 km / h with an additional character (Zz) with the French text "par temps de man dioxide", which means that the cruise control of the associated main traffic sign can only be used in the event of rain or rain
• this text is not variable in France, but for a traffic sign recognition, the In as many different countries or languages as possible the content of additional characters can be recognized and interpreted, it would be desirable that such texts can be read as single letters without prior knowledge of possible textual content. Therefore, within the scope of the invention, this is the variable text component (V) which it is to be recognized. Upon recognizing this variable text component, it is nevertheless possible to display to the driver this text as additional information about the speed limitation in the instrument cluster, or to search in vehicle databases or in the cloud for the meaning or a translation of the recognized words.
• V variable text component
• the image area is designed such that it comprises the main traffic sign (Hz) and the additional sign (Zz) assigned here individually. Then, the motion blur for this entire image area can be determined from the motion blur for the main character (Hz). The entire main character can be assumed as static content (S), because the possible values for
• V variable content
• Zz additional character
• Fig. 4 shows a main traffic sign (Hz), namely the danger sign "game change” with an associated additional character (Zz), which limits the period of validity of the danger symbol "to 3 km”.
• Hz main traffic sign
• Zz additional character
• the main character (Hz) and the additional character (Zz) are recognized as two separate traffic signs and correspondingly formed two separate image areas. Subsequently, it is checked whether the two traffic signs (Hz, Zz) are assigned to each other. Based on the present arrangement, in which the main character is located directly above the additional character, their assignment is detected. The motion blur is now determined only for the first image area in which the main character (Hz) is located. This can be fully recognized without text recognition: possibly already from the recorded (blurred) image, we assume, however, that the recognition succeeds only after the estimation of the motion blur and subsequent deconvolution. In any case, the calculation / estimation of the
• Motion blur for the image area of the main character (Hz) much easier and more accurate.
• the main character is larger and due to its strong red border, here triangular, very well detectable. Since the main character (Hz) and the additional character (Zz) are placed directly above or next to each other and captured in the same image, the motion blur parameters are almost identical for both characters and can thus be transferred from the main character (Hz) to the auxiliary character (Zz) become. The deconvolution of the image area with the additional character (Zz) is thus directly feasible and then the content of the additional character (Zz) can be recognized or read in the sharpened image detail.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• Traffic Control Systems (AREA)
• Image Analysis (AREA)
• Character Input (AREA)
• Image Processing (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=51627910

Family Applications (1)

Country Status (5)

Families Citing this family (22)

Family Cites Families (75)

• 2013
  • 2013-10-01 DE DE102013219909.0A patent/DE102013219909A1/de not_active Withdrawn
• 2014
  • 2014-09-04 DE DE112014002543.7T patent/DE112014002543A5/de active Pending
  • 2014-09-04 JP JP2016546142A patent/JP6445024B2/ja active Active
  • 2014-09-04 EP EP14777257.8A patent/EP3053097A1/de not_active Withdrawn
  • 2014-09-04 US US14/902,246 patent/US9697430B2/en active Active
  • 2014-09-04 WO PCT/DE2014/200444 patent/WO2015048954A1/de active Application Filing

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events