DE102017215956A1 - A method of outputting information about an object in an environment of a vehicle, system and automobile - Google Patents

Abstract

The invention relates to a method for outputting information about an object in an environment of a means of locomotion (10), comprising the steps of: determining (100) a viewing direction (8) of a user (3); recognizing (200) that the object in the viewing direction (8) is located; and • outputting (300) information about the object by means of an output device of the means of locomotion (10).

Description

The invention relates to a method for outputting information about an object in an environment of a means of locomotion, a system adapted to carry out this method and an automobile with the system.

Means of transport are nowadays equipped with screen operating systems on which a user navigation cards, menus and many other content can be output. Furthermore, there are devices for measuring the head position and viewing direction of the driver, these devices mostly include cameras.

WO 2013/104337 A1 for example, discloses a "dual-view display" that fades out content on a screen for the driver that a passenger can see. However, here the driver is protected only from a distraction. Furthermore, such an arrangement prevents the driver can satisfy his information needs while driving.

However, the display area in a transportation means is generally limited, whereby in many display states, a prioritization of the output contents is necessary. On a navigation map, e.g. No or only in a very limited way additional information on objects in the vicinity of the means of transport, also called points-of-interest (POI), issued, otherwise obscured navigation instructions. Thus, if a user encounters a POI in the vicinity of the vehicle, e.g. If a point of interest becomes attentive, information can only be output by additional manual effort, which distracts the user from the traffic in a risky manner.

Based on the aforementioned prior art, it is therefore an object of the present invention to provide a method by which a user can easily, time efficient and demand information about objects in the environment can be output. It is another object of the present invention to provide a system configured to carry out this method and to provide an automobile comprising the system.

Disclosure of the invention

The above object is achieved by the subject of claim 1. According to a first aspect, the present invention relates to a method for outputting information about an object in an environment of a means of locomotion.

As a means of transport within the meaning of the invention, for example, automobiles, in particular cars and / or trucks, and / or aircraft and / or ships and / or motorcycles come into question.

The method according to the invention comprises determining a viewing direction of a user. In this case, the viewing direction can be determined, for example, based on a tilt angle of a user's head and / or on the basis of a rotation angle of the user's head and / or on the basis of an eye position of the user. This is possible, for example, via a stereo camera and / or an infrared camera, in particular with two infrared diodes, and / or via an RGB webcam image. In the latter case, for example, determining a line of sight via 2D images of an evaluation unit, for example a microcontroller or a CPU, can be trained via an e-learning algorithm. The respective trained investigations based on the 2D images can then be stored in a memory of the means of locomotion and retrieved by the evaluation unit. For example, an object in the vicinity of the means of locomotion may be a point of interest for which the user, e.g. the driver of the means of transport, interested.

In a further step of the method according to the invention, a recognition is made that the object lies in the viewing direction. This can be done, for example, using cartographic data stored in the means of transportation in conjunction with the determined viewing direction of the user. This avoids that accidental detection of the object by the line of sight of the user triggers an output of information. The detection may comprise a predefined period of time, in particular 2 to 6 seconds, in which the user views the object and / or a recognition with respect to a workload of the user. For example, in a congestion situation where the user does not have to focus too much on the traffic and can pay more attention to the environment, the predefined amount of time can be reduced. Furthermore, multiple focusing of the object by the user may also trigger recognition. The abovementioned possibilities of recognition can take place, for example, on the basis of a predefined reference. In other words, depending on how much attention the user is paying to an object, it can be recognized by the means of locomotion whether or not it wants to obtain information about the object.

In a next step, information about the object which has been recognized as being in the viewing direction of the user is output by means of an output device of the means of locomotion. The output device may include, for example, a central information display. Furthermore, an output device may also include an acoustic output device. On the basis of this information, the user receives in a simple way demand information about the object that is of interest to him. Such information may include, for example, the name of the object and / or reservation possibilities with respect to the object and / or opening times related to an object and / or other object-specific characteristics, such as a construction period, if the object is a Romanesque style church, and / or digital Button to contact the operator of the object include. The method according to the invention eliminates the need for elaborate dialing by, for example, the menu of a GUI in order to view information about a POI and a distraction associated therewith. Thus, the user has the environment, in particular the traffic, better in view than when he has to look at a display to work through the operation of a menu tree information to work out. This time, the user wins in contrast to the described conventional interactions with the means of transport. Such a gain in time, for example, helps the user to be able to adequately respond to a traffic situation, as a result of which the method is also time-efficient. Accordingly, the present invention provides a simple, on-demand and time efficient method of outputting information about objects in the user's environment.

The dependent claims have advantageous developments to the content.

In an advantageous embodiment, the method according to the invention moreover comprises fading in of an interaction menu with respect to the object and / or buttons for establishing contact with an operator of the object for the user. For example, an interaction menu may include a menu for obtaining further information about the object, such as a construction period when the object is e.g. is a church in the Romanesque architectural style and / or service content and / or visitor numbers and / or entry or usage prices. Also, a lexical entry to the object can be called directly, if it is e.g. is a tourist attraction. In the event that the object is a museum, for example, information about current exhibitions can be obtained from the interaction menu for the interested user. For example, the information menu can also be left through the interaction menu. The contact buttons include, for example, a call setup button for a telephone call and / or a call for a reservation form, which may be e.g. can be completed by voice. For example, when the user contacts the operator of the "parking lot" object via a corresponding button, e.g. By phone, the user can conveniently reserve a parking space. If required, the interaction menu can be displayed by entry or already preset. Thus, the user can conveniently obtain further needs-based information about the object or make reservations quickly and easily.

In a further advantageous embodiment, the method according to the invention comprises a step of opening a speech input mode in response to the recognition of a viewing direction of an object. Thus, the above-mentioned functions with respect to the information and / or the interaction possibilities with respect to the object can be operated simply by voice control, which further reduces the distraction for the user.

In another advantageous development, the information is output on the basis of cartographic data which are stored, for example, in the means of locomotion (for example in a memory). Furthermore, the deposited information may also be obtained from lexical or cartographic online databases and / or a cloud via wireless communication, e.g. is made possible via an antenna of the means of locomotion, to supplement the cartographically stored in the means of transport with these.

In another advantageous development, the method according to the invention further comprises the step of masking out the information about the object after a predefined period of time and / or masking out the information by recognizing that another object is in the line of sight of the user. By the first-mentioned variant of the blanking after a predefined period of time, it is possible to output information which is of interest to the user only for a short time. This can be detected, for example, when the user directs his line of sight back to the road and no longer looks at the output device for a predefined time. Fading out, if another object is in the line of sight of the user, is useful if the user wants to quickly and easily information about the other object. One possible scenario is, for example, if the user has the Information of a fully occupied parking lot in the area is spent and the user detects another parking nearby with his gaze to see if it is empty.

In an advantageous embodiment, the outputting may take place after a predefined period of time and / or as a function of a workload and / or a number of focusing processes on the basis of a predefined reference. The time span may for example comprise 1 to 10 seconds, preferably 1 to 3 seconds or particularly preferably 1 to 2 seconds. This avoids information appearing on a randomly detected by the user's view object, which the user has not intended, on the output device. Furthermore, the output can occur depending on a workload. If, for example, the user is currently on the phone or has a particularly hectic traffic situation, for example because of an accident, the output can be prevented based on a predefined reference. Furthermore, for a predefined reference, a number of focusing operations of the object may be specified, in the presence of which the information is output. The outputting of the information can take place after three times, preferably after twice, recognition of the user's line of sight directed at the object. The predefined reference can be stored, for example, in a memory of the means of transportation.

In a further embodiment, the object includes a landmark. These include castles and / or castles and / or churches and / or statues and / or rivers and / or lakes and / or mountains and / or neighborhoods and / or museums and / or public spaces and / or landscape sections and / or others as attractions commonly known places into consideration. Further, the object may include a parking lot. In addition, it is also possible that the object comprises a service location. For this purpose, in particular restaurants and / or hairdressers and / or medical facilities and / or workshops and / or other known as service locations objects into consideration. As an object also a building in question. This includes, for example, sports facilities and / or office buildings and / or residential buildings and / or building parts. The variety of possible objects also gives users a quick sense of certainty as to whether a certain object is what they are looking for when, for example, they are on their way to a job interview. In other words, outputting information serves as a guide in close proximity. In particular, the user can place a focus on certain POI or objects defined by him, which are then to be output.

In addition, the method according to the invention comprises as output device a central information display (CID) and / or a head-up display (HUD) and / or an instrument cluster and / or an acoustic dispensing device. As acoustic output device is in particular a speaker and / or headphones into consideration. The acoustic output device can in this case refer to speech patterns which are stored on the memory. Such speech patterns contain, for example, recorded information texts about the respective objects. Furthermore, speech patterns can be obtained via wireless communication.

The following aspects of the invention include the advantageous embodiments and developments as well as the general advantages of the device according to the invention and the technical effects associated therewith, as it is pointed out to avoid repetition.

According to a second aspect, the present invention comprises a system for outputting information about an object in an environment of a means of locomotion arranged to carry out a method according to the first aspect of the invention. Such a system may comprise, for example, a data input, for example a stereo camera and / or an infrared camera and / or an RGB webcam, by which recognition of a viewing direction with respect to the object is detected. Furthermore, the system can comprise an evaluation unit, in particular a microcontroller and / or a CPU, via which data arriving through the data input are processed. If the evaluation unit detects an object in the field of view of the user, information can be passed from a memory to the data output. For example, a data output is a HUD through which the information is visibly output to the user.

According to a third aspect, the present invention comprises an automobile incorporating a system according to the second aspect.

list of figures

• 1 ein Flussdiagramm einer Ausführungsform des erfindungsgemäßen Verfahrens;
• 2 eine Ausführungsform eines erfindungsgemäßen Automobils;
• 3a eine Verbildlichung des Schrittes eines Ermittelns einer Blickrichtung gemäß einer Ausführungsform des erfindungsgemäßen Verfahrens;
• 3b eine Verbildlichung des Schrittes eines Erkennens einer Ausführungsform des erfindungsgemäßen Verfahrens; und
• 3c eine Verbildlichung des Schrittes eines Ausgebens einer Ausführungsform des erfindungsgemäßen Verfahrens.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings:

• 1 a flowchart of an embodiment of the method according to the invention;
• 2 an embodiment of an automobile according to the invention;
• 3a an illustration of the step of determining a viewing direction according to a Embodiment of the method according to the invention;
• 3b a representation of the step of recognizing an embodiment of the method according to the invention; and
• 3c an illustration of the step of outputting an embodiment of the method according to the invention.

The inventive method is based on an embodiment according to 1 explained. In a first step 100 becomes the viewing direction 8th of the user 3 a means of transport 10 determines what's about a stereo camera 2 he follows. Having an object, especially a landmark 4 , the user's attention 3 has attracted his attention, which depends on the inclination angle and the rotation angle and the eye position of the user 3 is determined on the sight 4 , After the user's gaze for 2 seconds the sight 4 this is done in the second step 200 recognized. Thereupon, in a third step 300 causally to an information about the attraction 4 , which uses cartographic data in a memory 6 of the means of transport 10 is deposited on a central information display 1 of the means of transport 10 output. After the user 3 the information has read through, his thirst for knowledge is satisfied. Then in the fourth step 400 the information disappears after 10 seconds, because the user 3 neither the information nor the sight 4 at this time has captured with his gaze, which is based on the stereo camera 2 is determined.

2 illustrates an embodiment of an automobile according to the invention. The user 3 , which is located in the car, this looks like a landmark 4 , which are in his line of sight 8th located. A stereo camera 2 determines the viewing direction 8th of the user 3 , Based on an evaluation unit 5 which is arranged to access a predefined reference stored on a memory 6 can be deposited, for example, by the evaluation 5 after 2 seconds, be recognized that the user 3 the attraction 4 consciously focused. The evaluation unit 5 is set up, issuing information about the sight 4 on the central information display 1 to illustrate. This is possible either on the basis of cartographic data stored on the memory 6 deposited or based on cartographic data generated by the antenna 7 are receivable. Receiving via the antenna 7 can be done for example via an Internet database and / or via a server of a service provider.

3a shows a representation of an embodiment of the step of determining 100 the method according to the invention. Here, a look through the windshield from the user or driver's perspective is shown. A sight 4 has the interest of the user 3 awakened and the look of the user 3 then turns away from the street 11 from. The look of the user 3 depends on a point of interest 4 , this process being indicated by a dashed arrow. The change of perspective 8th is based on the tilt and the rotation angle of the head using a stereo camera 2 determined. While investigating 100 shows the central information display 1 of the means of transport 10 only a navigation map.

3b illustrates an embodiment of the step of recognizing 200 the method according to the invention. The evaluation unit recognizes this 5 (S. 2 ) using the stereo camera 2 as described above, that the look of the user 3 the attraction 4 recorded for 2 seconds and the user 3 is therefore interested in information about it.

3c illustrates a depiction of an embodiment of the outputting 300 an information, here a name of the place of interest 12 and an opening time, on a central information display 1 in response to the step of knowing 200 , The central information display contains a button 13 to contact the owner of the object. The button 13 can wear the caption "book guide" and, for example, enable direct telephone contact with the operator.

LIST OF REFERENCE NUMBERS

1

Central information display

2

stereo camera

3

user

4

point of interest

5

evaluation

6

Storage

7

antenna

8th

line of sight

10

Means

11

Street

12

Name of a place of interest

13

button

100-400

steps

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• WO 2013/104337 A1 [0003]

Claims (10)

A method of outputting information about an object in an environment of a means of locomotion (10), comprising the steps of: • determining (100) a viewing direction (8) of a user (3); • recognizing (200) that the object lies in the viewing direction (8); and • outputting (300) information about the object by means of an output device of the means of locomotion (10). Method according to Claim 1 further comprising the step of fading in an interaction menu with respect to the object and / or fading in buttons (13) to contact an operator of the object for the user (3). Method according to one of Claims 1 or 2 further comprising the step of opening a voice input mode in response to detecting (200) a line of sight (8) with respect to the object. Method according to one of the preceding claims, wherein the outputting (300) of the information about the object takes place on the basis of cartographic data stored in the means of transportation (10). The method of any one of the preceding claims, further comprising the step Fading out (400) the information about the object after a predefined period of time and / or • masking out the information by recognizing that another object is in the user's line of sight. Method according to one of the preceding claims, wherein the dispensing (300) • after a predefined period of time and / or Depending on a workload of the user (3) and / or • after a number of focusing operations based on a predefined reference. Method according to one of the preceding claims, wherein the object comprises a point of interest (4) and / or a parking space and / or a service location and / or a building. Method according to one of the preceding claims, wherein the output device comprises a central information display (1) and / or a head-up display and / or a combination instrument and / or an acoustic output device. A system for outputting information about an object in an environment of a means of locomotion (10), which is set up, a method according to one of Claims 1 to 8th perform. Automobile comprising a system according to Claim 9 ,

Images