EP2316003A1

EP2316003A1 - Mobile navigation device and corresponding method

Info

Publication number: EP2316003A1
Application number: EP09780926A
Authority: EP
Inventors: Achim Rafalczik; Anes Hodzic; Rainer Garnitz; Wolfgang Niehsen; Mathias Garrelts; Jochen Wingbermuehle; Thomas Wiechers
Original assignee: Robert Bosch GmbH
Current assignee: Robert Bosch GmbH
Priority date: 2008-08-18
Filing date: 2009-07-22
Publication date: 2011-05-04
Also published as: DE102008041295A1; WO2010020512A1

Abstract

In order to further develop a mobile navigation device (10) and a method for providing at least one additional navigation-related functionality and/or at least one additional driver assistance function in such a mobile navigation device (10) in such a way that additional driver and/or pedestrian functionalities can be realized, at least one combined, particularly integrated, image recording device or camera (30) is proposed.

Description

Mobi l it N av i o n s g e r e a ss a n d th e r es t a t of this procedure

Technical area

The present invention relates to a mobile navigation device according to the preamble of claim 1 and to a related method according to the preamble of claim 6.

State of the art

Mobile navigation devices have gained increasing acceptance in recent years because such devices also provide pedestrian navigation functionality based on conventional vehicle navigation based on G [lobal] positioning [ystem] data, gyro data and / or map data GPS signals directs the user to the desired destination.

The mobile navigation devices available today are characterized by a graphic or linguistic output function for map data, which can be loaded onto the navigation device by means of different storage media.

Furthermore, mobile navigation devices can be equipped with additional, non-navigation-relevant functionalities, for example with an MP3 player. As a rule, no purely mobile navigation device (except P ersonal D [igital] A [ssitant] s or so-called SmartPhones) is equipped with an image recording unit or camera (unit). On the other hand, modern vehicles contain a multitude of different assistance functions, for example relating to Adaptive Cruise Control (ACC) and / or Lane Departure Warning (LDW), which assist the driver when driving and in warn safety-critical situations. Such systems are capable of detecting a critical situation and sending a warning to the driver with the aid of the appropriate display systems, be it visually, acoustically and / or haptically.

Furthermore, it should be noted in the prior art that for video-based monitoring of the environment of a motor vehicle and for the realization of driver assistance functions that require additional automatic image analysis (lane assistance, traffic sign recognition and the like), in addition to permanently installed in the motor vehicle cameras already remote , subsequently to be mounted camera modules have been proposed.

By way of example, mention may be made of the driver assistance system FAS 100 provided by ALAN Electronics GmbH for passenger cars. This video analysis system, which the manufacturer calls intelligent, analyzes the images of the roadway captured with a digital camera in fractions of a second, what is the purpose of this

A [pplication] S [pecific] l [inte- grated] C [ircuit] video processor has a camera module and a control unit. The camera module is mounted in the center of the windscreen (below the rearview mirror) and the control unit can be placed on the dashboard or in a clamp mount.

For such camera modules, in the case of purely indicative functions, in particular, the wireless image transmission to a for the

Driver's viewable display and the power supply from a battery (mulatoren) unit proposed. The display was made under which also proposed mobile computer.

Furthermore, in the prior art, for example in the publication WO 2004/074046 A1, the use of mobile computers, including mobile phones, mobile telephones and personal [...] telephones, has been integrated

Camera for video-based driver assistance functions described. It has been proposed here that the image processing module and the display unit are formed by a microcomputer, in particular by a mobile telephone or by a palmtop or by a personal digital assistant (PDA).

Here, the integration of the image processing module in the vehicle interior ensures that this image processing module is not exposed to strong temperature fluctuations or extreme temperatures. Because the image processing module and the image processing device display are formed by the processor module and the display of the microcomputer, the requirements for the e-mechanical stability and the longevity of this known system can be considerably reduced.

Furthermore, attention should be drawn to the state of the art - to document WO 2006/132522 A1, in which a combined display of live image and navigation instructions, for example directional arrows, for a mobile navigation system is proposed, as well as

to the document WO 2007/101755 A1, in which a method and an arrangement for displaying at least one navigation instruction of a navigation system in a vehicle is proposed, wherein a section of the vehicle surroundings is taken by a camera and displayed as an environmental image sequence by means of a display unit, wherein the Navigation hint is also displayed by the display unit, and wherein the movement of the vehicle, the position of the at least one navigation instruction is changed within the environmental image sequence as a function of the movement of the environment relative to the vehicle.

Presentation of the present invention: object, solution, advantages

Based on the disadvantages and inadequacies set out above, and in appreciation of the outlined prior art, the present invention seeks to further develop a mobile navigation device of the type mentioned in the technical field and a method of the type mentioned in the technical field so that additional driver and / or pedestrian functionalities can be realized.

The object of the present invention is achieved by a mobile navigation device having the features specified in claim 1 and by a method having the features specified in claim 6. Advantageous embodiments and expedient developments of the present invention are characterized in the respective subclaims.

The present invention is based on the equipment or equipment of a mobile navigation device with at least one image acquisition unit or with at least one camera (unit), that is, on the implementation of at least one image acquisition unit or at least one camera in a mobile navigation device.

The equipment or equipment of a mobile navigation device with at least one image acquisition unit or with at least one camera allows additional functionality. The image acquisition unit or

Camera allows recording of pedestrian routes that are directly can be stored and / or can be passed as route guidance to acquaintances, and saves the cumbersome take away of additional recording devices, such as digital cameras.

If the image recording unit or camera for the aforementioned

Functions are not or not only used, it (also) can be used for the realization of driver assistance functions.

In this case, the present invention uses in its various forms

- Which built in a mobile navigation device anyway or

- At least one connected to a mobile navigation device external

Camera for the realization of conventionally present on these devices displaying and / or warning driver assistance functions.

Consequently, by means of the present invention, the components already present in a mobile navigation device, such as processor or computer, display or display unit and optionally image acquisition unit or camera, are used to provide additional customer-value, in particular driver assistance, functions in addition to today's customary applications realize. In this way, even with pure software components, the customer benefit of mobile navigation devices can be increased in a cost-effective manner compared to conventional systems.

Finally, the present invention relates to the use of at least one mobile navigation device according to the type set out above and / or a method according to the above-mentioned kind for video-based driver assistance functions and / or for video-based pedestrian assistance functions. Brief description of the drawings

As already discussed above, there are several ways that

To design and develop the teaching of the present invention in an advantageous manner. For this purpose, reference is made on the one hand to the claims subordinate to claim 1 and claim 6, on the other hand further embodiments, features and advantages of the present invention will be explained below with reference to the embodiment illustrated by Fig. 1.

It shows:

Fig. 1 is a schematic representation of an embodiment of a mobile navigation device according to the present invention, which operates according to the method according to the present invention.

Best way to carry out the present invention

At least one camera module 30 is connected to the mobile navigation device 10 illustrated by way of example in FIG. 1 via at least one image processing module 20, wherein the camera module 30 can be implemented or integrated in the mobile navigation device 10 or, as described in connection with FIG

Fig. 1 illustrates, wired or wirelessly connected to the mobile navigation device 10 may be.

The image taken by at least one such camera 30 is transmitted to the (optional) image processing module 20, wherein the image processing module 20 may be integrated in the mobile navigation device 10 or, as illustrated with reference to FIG. 1, be implemented separately may or may be integrated in the camera module 30, whether in a realization as hardware and / or in a realization as software, for example in the form of an image processing software as a component of at least one software kernel or software kernel.

The image transmitted to the image processing module 20 is processed there and, after one or more such processing steps, either completely or partially, be it unchanged or modified, displayed by the mobile navigation device 10, be it on the display 12 of the mobile navigation device 10 itself or on an external display 12 connected to the mobile navigation device 10.

This display documents that the information gained in image processing can be used for driver assistance, for example in form

- at least one warning for the driver and / or at least one passenger who may be present and / or

- at least one projection of at least one guide line, for example in video sequences.

Alternatively, the image is not displayed, but merely a situation-dependent generated by an image processing warning and / or message from the mobile navigation device 10 further processed and / or stored and / or output.

Assuming that mobile navigation devices in their conventional design are portable devices without image acquisition functionality and / or without telephone functionality, the present invention thus offers the advantage that additional driver assistance functionalities can be realized with the camera 30. When the camera 30 is not used by other applications, for example when the mobile navigation device 10 is connected to the vehicle infrastructure, the camera 30 may be used to monitor the traffic situation. The recorded video sequences or digital photos are processed with the image processing software 20, for example, to measure the distance to preceding vehicles or to detect an exit from the selected lane.

Based on this information, which is obtained in the image processing 20, the driver can be warned by different warning mechanisms against a dangerous situation in a timely manner, which contributes to accident prevention.

Furthermore, the image taken by the camera 30 can

A / ideosequenzen be used as a basis for further navigation applications (for example, so-called Videonavi) in which direction arrows of the route guidance can be projected directly into the image sequence.

The present invention thus offers the possibility of optimally utilizing the existing resources, for example camera 30 and computing power of mobile navigation device 10, and to realize additional driver assistance functionalities.

With regard to such additional driver assistance functionalities, the realization of different driver assistance functions listed below and briefly described on a mobile navigation device 10 is possible, wherein the mobile navigation device 10 has either an internal camera or wired or by radio transmission, ie wirelessly to at least one external Camera 30 is connected (see Fig. 1). The camera 30 monitors this

- the interior (see below attention monitoring, child monitoring / cargo space monitoring, video-supported communication) and / or

- the environment (see rear view camera, side cameras, blind spot monitoring or dead center monitoring, lane departure warning, traffic sign recognition, video navigation, processed night image, P [oint] O [f] l [nterest] from driver's perspective / georeferenced videos) of the vehicle ,

In the monitoring of attention in the interior of a vehicle, the images of the inwardly directed to the driver camera 30 are analyzed with methods of digital image processing such that due to the eyelid activity, the line of sight and / or posture conclusions on the attention and / or fatigue of the driver can be. If the driver shows signs of fatigue or if the driver is inattentive in critical situations, a warning is issued from the mobile navigation device 10.

In the case of child monitoring / loading space monitoring in the interior of a vehicle, the images of the rear area and / or the loading space of a passenger or heavy goods vehicle are recorded

Camera 30 displayed in the display area or on the display 12 of the mobile navigation device 10.

In video-assisted communication in the interior of a vehicle, a system similar to video telephony is used to facilitate communication between the driver and the passenger and the passengers in the rear. nation partners without the driver or front passenger having to turn backwards.

For this purpose, the faces of the passengers are detected by the camera 30. The image display is carried out on the display 12 of the mobile device 10. Optionally, a return channel from the driver or the passenger to the passenger / passengers can be realized if the rear seats are equipped with display units or with displays.

In the reversing function in the environment of a vehicle, the image of the

Rear view camera 30 displayed on the display 12 of the mobile navigation device 10. In the displayed image or next to the displayed image, guides or other symbols or numbers may optionally be displayed to assist the driver in parking the vehicle and / or maneuvering the vehicle. This auxiliary information can be obtained from the images of the camera 30 using methods of automatic and / or digital image processing 20.

In the side camera function in the vicinity of a vehicle, the images of one or more cameras 30, which observe the lateral or lateral surroundings of the vehicle and / or the sides of the vehicle, are displayed on the display 12 of the mobile navigation device 10. In the displayed image or next to the displayed image, guide lines or other symbols or numbers may optionally be displayed to assist the driver in driving through narrow gates or performing similar maneuvers. This auxiliary information can be obtained from the images of the camera (s) 30 using methods of automatic and / or digital image processing 20.

In the case of blind spot monitoring or dead center monitoring in the vicinity of a vehicle, the images of one or more cameras 30 become the laterally rearward or laterally rearward surroundings of the vehicle monitor, displayed on the display 12 of the mobile navigation device 10. With methods of automatic and / or digital image processing 20 approaching passenger cars are detected in this area. If there is a risk of collision, for example when overtaking a lane change, a warning is issued.

In lane departure warning in the vicinity of a vehicle, lane markings to the left of the vehicle and / or to the right of the vehicle are observed by one or more suitably positioned cameras 30. Inattentive driving over the lines, a warning via the mobile navigation device 10.

In traffic sign recognition in the vicinity of a vehicle, the images of a roughly aligned in the direction of travel camera 30 with methods of automatic and / or digital image processing 20 with the

Target of detection of traffic signs evaluated.

One possible character class to be recognized here are speed limits and their cancellation. If such characters are automatically detected, then this information compared to a stored map is much more current and can be used to warn the driver, for example, when exceeding the currently permitted maximum speed.

Another possible character class are overtaking bans and their

Repeal. Again, the driver can be made aware of currently valid restrictions. A display of the recognized character also has a memory function, that is, the driver can check whether he has passed during an inattentive phase at a corresponding sign over.

An improved stage of the aforementioned variances exists in the Additional recognition of restrictive additional characters, such as the restriction "In the wet". Here, a meaningful expression already exist in the mere detection of the presence of such additional characters. The driver then receives only the additional indication that a sign is only valid to a limited extent. Information about the type of

Restriction can be given if the sign is not only detected but recognized. However, the reliable recognition of the additional characters is very demanding due to their great diversity and in view of the comparatively small dimensions.

Another possible character class are the triangular danger signs, such as deer, rockfall and the like. If these characters are recognized, then a corresponding indication can be output via the mobile navigation device 10. Furthermore, it is also possible here to display the recognized and / or detected characters only in order to realize a reminder function.

A character type that can be executed in the form of a special function is the sign "Entry forbidden" (= red circle with white crossbar) at driveways. Will this character be on the right side of the

Detected vehicle, the driver of the mobile navigation device 10 is warned against a collision against the direction of travel (so-called "ghost driver").

Other possible character classes are signboards and city signs.

In the simplest case, these can only be detected and displayed as a section of the input image on the mobile navigation device 10 in the sense of a reminder function. In a further expansion stage, local entrance signs and local exit signs can be recognized as such and the maximum permissible maximum speed can be modified on the basis of this information. The basic idea of a video navigation in the environment of a vehicle is the simultaneous display of a oriented in the direction of travel live image of the camera 30 with navigation instructions, in particular with directional arrows on the display 12 of the mobile navigation device 10. An important advantage of this display form is the possibility of the relevant environment of

Keep an eye on the vehicle during the reading of the mobile navigation device 10 in view.

In addition to an adjacent display of live image and navigation information, the navigation instructions can be used in more advanced

Systems can also be displayed partially transparent or overlapping in the live image. This can be done in particular taking into account the perspective of the image, so that, for example, the impression arises that a directional arrow is located directly on the roadway in front of the vehicle.

In a first stage of such video navigation, the directional arrows are statically adapted to a medium perspective. Allow later expansion stages - by methods of automatic and / or digital image processing

20 and / or - by additional sensors, the tracking of the arrows displayed when yawing or pitching of the vehicle and the consideration of the course of the lane in front of the vehicle.

In order to display a live image on the display 12 of the mobile navigation device 10, for example for video navigation, an image processed by means of image processing can be used in addition to the unmodified image of the camera 30. A treatment can be used in particular at night in order to draw the attention of the driver to certain image areas and / or the visibility of objects in front of the vehicle. to improve. It is also possible here to switch from daytime image to nighttime image, which is analogous to switching between daytime display and nighttime display of the map.

In conjunction with the mobile navigation system 10, a camera 30 oriented in the direction of travel can also be used to take individual pictures of a location of interest (so-called POI = point of interest) from the perspective of the vehicle driver and to store these pictures with the position information. If a location stored in this way is to be approached again later, this can be done by selecting the corresponding picture from a list. The driver sees the image of the place he wants to drive from his perspective.

For complex sections, for example for roundabouts with multiple exits, a video recording to document the correct path from the perspective of the driver can be recorded and optionally provided to other (subsequent) drivers.

For motorcyclists can be created in a similar way, the possibility of a video recording of a particularly appealing section of track. The recorded video can then be provided together with the associated geographic coordinates, for example on a web server, as information for like-minded people.

Combinations of the aforementioned functionalities, for example, a warning only in case of inattention of the driver, are also possible.

Claims

At least one associated, in particular integrated, image recording unit or camera (30) for realizing at least one additional navigation-related functionality and / or at least one additional driver assistance function is provided in the mobile navigation device (10), e e e c e.

2. Mobile navigation device according to claim 1, characterized in that the image recording unit or camera (30) is wired or wirelessly connected to the mobile navigation device (10).

3. Mobile navigation device according to claim 1 or 2, characterized in that the image recording unit or camera (30) is designed as a rear camera for observing the rear region of the vehicle and / or as a side camera for observing at least one lateral or lateral region of the vehicle and that the images of the rear camera or the side camera additional linformationen by means of image processing (20) are obtainable.

4. Mobile navigation device according to at least one of claims 1 to 3, characterized by at least one of the image recording unit or camera (30) downstream image processing (20), in particular image processing module or Bildverarbei- tion software.

5. Mobile navigation device according to claim 4, characterized in that the image processing (20) in the mobile navigation device (10) is integrated or designed as a separate unit or as a separate module or - in the image recording unit or camera (30) is integrated.

6. A method for providing at least one additional navigation-related functionality and / or at least one additional driver assistance function in a mobile navigation device (10), characterized by the evaluation of the at least one image associated with the mobile navigation device (10), in particular in the mobile navigation device (10 ) integrated, image capture unit or camera (30).

7. The method according to claim 6, characterized in that in the attention monitoring in the interior of a vehicle, the image of the directed to the driver Bildaufnahmeein- unit or camera (30) by image processing (20), in particular by means of at least one image processing module or by means of image processing software is analyzed ,

8. The method according to claim 6 or 7, characterized in that in the monitoring, in particular in the child monitoring and / or loading space monitoring, in the interior of a vehicle, the image of the rear area, in particular rear seat, and / or the loading space receiving image pickup unit or camera (30) is displayed in the display area or on the display (12) of the mobile navigation device (10).

9. Method according to at least one of claims 6 to 8, characterized characterized in that in the video-assisted communication in the interior of a vehicle, the faces of the vehicle passengers with the image recording unit or camera (30) detected and displayed in the display area or on the display (12) of the mobile navigation device (10).

10. The method according to any one of claims 6 to 9, characterized in that the blind spot monitoring or dead center monitoring in the environment of a vehicle, the laterally rearward or laterally rearward environment of the vehicle by means of the image recording unit or camera (30) is monitored, in particular by means of Image processing (20) in the lateral rearward or laterally rearward environment approaching vehicles are detected.

11. The method according to any one of claims 6 to 10, characterized in that the lane departure warning in the vicinity of a vehicle lane markings left of the vehicle and / or right of the vehicle by the camera (30) are observed and that inattentive driving over at least one line at least a warning is issued via the mobile navigation device (10).

12. The method according to any one of claims 6 to 11, characterized in that in the traffic sign recognition in the environment of a vehicle, the image of at least roughly aligned in the direction of travel camera (30) by means of image processing (20) with respect to the detection of at least one traffic sign is valued.

13. The method according to any one of claims 6 to 12, da- characterized in that during video navigation in the vicinity of a vehicle at least one oriented in the direction of travel live image of the camera (30) displayed simultaneously with at least one navigation hint, in particular with at least one directional arrow, in the display area or on the display (12) of the mobile navigation device (10) becomes.

14. The method according to any one of claims 6 to 13, characterized in that the additional navigation-related functionality and / or the additional driver assistance function is activated only when detected by means of the camera (30) inattention of the driver.

15. Use of a mobile navigation device (10) according to at least one of claims 1 to 5 and / or a method according to at least one of claims 6 to 14 for video-based vehicle driver assistance functions and / or for video-based pedestrian assistance functions.