WO2017032549A1

WO2017032549A1 - Method for triggering at least one processing step by assigning an information element to a device

Info

Publication number: WO2017032549A1
Application number: PCT/EP2016/068154
Authority: WO
Inventors: Bernd Weymann; Karl-Heinz Knobl
Original assignee: Continental Automotive Gmbh
Priority date: 2015-08-24
Filing date: 2016-07-29
Publication date: 2017-03-02
Also published as: DE102015216108A1

Abstract

The invention relates to a method for triggering at least one processing step, which should be applied to an information element and which goes beyond the displaying of information associated with the information element on a display, in a system having a plurality of displays and having at least one computer connected to the displays, which method comprises presenting a first information element or an indication representing the presence of a first information element. The method also comprises detecting a gesture performed by the user in relation to the presented first information element or the presented indication and, in accordance with the detected gesture, performing one or more processing steps concerning the first information element.

Description

description

Method for triggering at least one processing step by assigning an information element to a device

The invention relates to a method and apparatus for processing information elements, in particular to a method of triggering at least one on an information ^¬ element applicable processing step in a system with multiple arranged at different spatial positions displays, for example, in a motor vehicle cockpit.

In the context of this description, an information element called ^¬ an executable computer program or a computer program executable associated content. For example, content associated with an executable computer program includes one or more text, image, audio, or video elements, singly or in combination. For example, text items are messages or status information that can be captured and processed by a user or computer.

In the context of this description, a processing step to be applied to an information element includes, among other things, the extraction of information and / or computer-executable program instructions from the information element, the forwarding of information extracted from the information element and / or program instructions to another process step or to another, the information verar ^¬ beitendes device, or a display of information associated with the information element after previous execution of processing steps.

In the context of this description, the term is synonymous display ver ^¬ applies for a screen for displaying information or an information processing device with a screen. In the case of an information processing device with a screen can be applied to parts of an information element processing steps abgear ^¬ processed by this device. A pure display can represent a processing function to be applied to parts of an information element, the actual processing taking place in a computer connected to the display.

In the course of increasing information technology networking of automobiles, a driver is becoming an ever larger number of

Information is supplied, and an ever greater number of comfort and safety features are provided to relieve or assist the driver. Operating these functions and processing this information often requires a series of complex sequential operations. This complex operation can greatly distract the driver from driving, so that he may be able to respond to dangerous situations only late or inadequately.

In particular, information and controls are increasingly displayed on displays. These displays are often not arranged in the driver's field of vision in such a way that they neither restrict the driver's field of vision nor avert the gaze from driving. The arrangement of the representation of information about different functions of a motor vehicle is often individually adjustable by the user.

1 shows an exemplary representation of a known from the prior art, configurable by a user interface on a screen 100. On the screen 100 a plurality of windows 102, 104, 106 are arranged, representing the status of ^¬ different, parallel programs or functions ,

Window 102 is associated with a radio function and displays information about a tuned radio station. The in window 102 information includes the name of the radio station, 1021, the frequency at which the radio station broadcasts, 1022, a transmitter logo, 1023, and a QR code, by means of which further information is made accessible using a smartphone or other suitable network device.

Window 104 is associated with a navigation system of a motor vehicle and displays in an electronic compass 1041 the current orientation of the vehicle, as well as a destination address 1042.

Window 106 is associated with a weather application and displays weather data 1061 and temperature data 1062 to a selected location 1063. In addition, current date 1064 and current time 1065 are displayed in window 106.

In FIG. 1, a user wishes to move the window 106 to another location on the screen 100. Arrow 108 in FIG. 1 indicates a displacement direction for window 106. In FIG. 1, a user moves the window 106 from the upper right quadrant to the lower left quadrant. The original position of the window 106 is indicated by the weak-light gray ^¬ rendered image of the window 106. The shift occurs, for example, by touching the window 106 on a touch screen, e.g. With a finger, and moving the touch point across the screen to the new desired position without lifting the finger off the screen. When the desired new position is reached, the user can lift his finger off the screen and the window shift is completed. In some cases, an approximate positioning of a window to be moved is sufficient, with final positioning being automatic with respect to other windows. Figure 2 shows the screen 100 after the shift. The positions of the windows 102 and 104 are unchanged. Window 106 is now at the new position in the lower left Quadrant, the original position of the window 106 is now empty. The information displayed in the windows is essentially unchanged. A similar shift function for information windows of computer programs from a display arranged at a first location to a display arranged at a second location is known from EP 2 018 992. Moving information windows from one display to another is merely a matter of continuing to play or play on another display.

However, the shifting of information windows on a display does not yet cause any further operation of the underlying function or processing of information presented in an information window.

The invention defined in the claims simplifies the operation of functions provided in a motor vehicle having a plurality of displays and reduces the complexity of the operating steps required for their operation. The inventive method enables a user Informa tion elements ^¬ devices and / or functions in a simple and intuitive way to allocate for the processing, wherein a deflection is prevented from driving process or at least reduced.

The inventive method for triggering at least one to be applied to an item of information about the pure Dar ^¬ position associated with the information element information on a display beyond machining step is carried out in a system comprising a plurality of displays and min ^¬ least a processor coupled to the display computer to execute of the at least one processing step. The method comprises the presentation of a first information element or a hint representing the presence of a first information element. Such an indication can be made visually, acoustically or haptically. One an optical indication is, for example, a light signal or symbol appearing on the screen in the field of vision of a user. The type of light signal, z. As color, steady light or flashing, or a combination thereof can provide information about the information element. An indication by a symbol on a screen can give information about the information element by the design of the symbol. An audible indication may be given by one or more user-audible beeps, or by voice information, such as synthesized voice output. A haptic indication can be made, for example, by vibration of a control element touched by the user. The control element does not have to be connected with the processing of the information element. For example, in a motor vehicle, a part of the steering wheel encompassed by a driver's hand can vibrate, or the seat surface or seat back.

The method further comprises detecting a gesture executed by the user with respect to the presented first information item or hint. The gesture may be performed on a touch-sensitive surface, such as on a touchpad or a touch-sensitive screen. Alternatively, the gesture can be performed freely in the room, for example, by hand, arm, Kopfbe ^¬ movements or the like. The detection of gestures executed freely in space can be done, for example, by means of one or more cameras with subsequent image processing. To improve the recognition of gestures carried out freely in space, an object to be moved by the user in the room may be provided, for example a ring or a bracelet. The object can be designed so that it recognizes changes in its position in the room itself and detects gestures. A gesture detected by the object can then be transmitted to a control unit by means of wireless transmission, for example by radio or light signal, and used in the sense of the method according to the invention. In this case, a detection of gestures by means of a camera is not absolutely necessary. A however, optional combination of both detection methods may improve the accuracy of detection.

Gestures can be, for example, pointing, gripping or swiping gestures.

The method further comprises executing one or more processing steps concerning the first information item, wherein the one or more processing steps to be performed depend on the detected gesture and the type or type of the information item.

The execution of the one or more processing steps may include assigning the information element or outputting a result of the processing steps to a display, wherein a computer connected to the display executes the processing steps. The display stands for a function or a software program which executes the processing steps relating to the information element. In any case, the processing steps applied to the Informati ^¬ onselement more than the mere representation of the information element itself on another display are. Conveniently, the gesture describes picking up the information element at a first location and moving the information element to a second location, or more generally a displacement of the information element from a first location to a second location. A location preferably corresponds at least approximately to the position of a display. However, a place can also be a virtual place, which is clearly described by the direction of a gesture, for example. For example, such a virtual location is a virtual recycle bin. A move to this virtual recycle bin then causes the deletion of an information item or the abort of a current item related to the item of information.

The gesture may also be a shift of the Informati ^¬ onselement representing the object on a display in a be from several spatial directions. It is not absolutely necessary to touch the display.

The gesture may also be an independent hand, eye or head movement from an object displayed on a display.

Appropriately, such a hand, eye or head movement occurs in close temporal relation with a signaling to a user that a new information element is ready. The close temporal relationship is determined, for example, by a first time period following the signaling. The first period may be preset for certain types of information elements and / or individually adjustable or modifiable by a user.

The related to an information element executed Be ^¬ processing steps are different depending on the kind or type of Informati ^¬ onselements, even if different kinds or types of information elements are assigned the same screen or the same gestures are used.

The at least one processing step to be executed on the information element is linked to the information element. The link may be imp ^¬ lementiert in different ways. For example, one or more computer readable program instructions may be associated with the information element, for example, in a file header. However, it is also conceivable to assign one or more processing steps to certain types of information elements and to store the assignment in the at least one computer arranged in the system, so that only the type or type of information element must be determined by the one or more to determine a specific gesture toward executed Bear ^¬ beitungsschritte.

Several different gestures can be connected to an information ^¬ element with alternative processing steps. In the following the invention will be explained by way of example with reference to figures of a drawing.

In the drawing shows:

Fig. 1 shows a first exemplary illustration of a user interface known from the prior art, configurable by a user on an image screen ^¬;

2 shows a second exemplary representation of the known from the prior art, configurable by a user interface on a screen ^¬ screen;

3 shows an exemplary schematic representation of an arrangement of a plurality of devices functionally linked to each other according to the invention; and Fig. 4 shows an illustrating the method from ^¬ strahierte schematic arrangement according to the invention interconnected devices of Figure 3.

Figures 1 and 2 have already been described above and will not be discussed further below.

FIG. 3 shows an exemplary schematic representation of an arrangement of a plurality of devices functionally linked together according to the invention, for example in a motor vehicle cockpit. The motor vehicle cockpit 300 has an instrument panel 302 in front of which a steering wheel 304 is arranged. In the area of the steering wheel 304, a combination ^¬ instrument 306 is arranged on the dashboard 302 having a first display. To the right of the position indicated by the steering wheel 304 of the driver is a center console 308, on which a second display 310 is arranged. Above the dashboard is a windshield 312, on which a Pro ektionsfläche for a head-up display 314 is indicated. In the lower area of the center console, a virtual recycle bin 316 is indicated. The combination instrument 306 with the first display, the second display 310, the head-up display 314 and the virtual wastebasket 316 represent devices which, in addition to being able to display information, can perform or control certain functions.

In FIG. 4, the devices described in FIG. 3 are arranged along a horizontal or vertical axis in order to clarify the functional principle of the method according to the invention. The axes are parallel to corresponding gestures that trigger processing steps to be applied to information items. The exemplary arrangement of the devices in Figure 4 may be considered to be spatially approximate as a typical arrangement of corresponding devices in a motor vehicle.

The first instrument, instrument cluster 306, is located in the left half of the figure on the x-axis (not shown). The second device, second display 310, is arranged in the right half of the figure on the x-axis. The third device, Head-up

Display 314 is located in the upper half of the figure on the y-axis (not shown), and the fourth device, the virtual wastebasket 316, is located in the lower half of the figure on the y-axis.

In Figure 4, possible directions for simple gestures are indicated by arrows 320-323 pointing up, down, left, or right from the origin of the coordinate system spanned by the x-axis and the y-axis. The origin of the coordinate system is represented by a circle 324 in the center re ^¬ .

In a first example for explaining the method according to the invention, the information element to which one or more processing steps are applied is an in-depth one

Text message (SMS). A message, eg. For example, an icon or a text hint that an SMS has been received is for example displayed on the display of the instrument cluster 306. Alternatively or additionally, it is also possible to report a received new SMS by means of a specific audio signal. The reading of text messages is usually not possible for drivers of motor vehicles while driving, without taking their eyes off the road and possibly endanger themselves or other road users. Therefore, reading text messages while driving is prohibited by law in many countries.

In application of the method according to the invention, a driver can activate a read-only function by means of a simple gesture, here by means of a directed swipe gesture to the right, which reads out the SMS. Corresponding devices that convert text to speech are known from the prior art and will not be explained in detail here. The swipe gesture from left to right, from the instrument cluster 306 to the second display 310 in the center console 308, causes the assignment of the text message whose arrival has been signaled on the display of the instrument cluster 306, to another device and simultaneously starting the text-to-speech Function and its application to the text message. In one embodiment of the method according to the invention, after the message has been read a speech dialog program is started which offers the driver a choice of reactions to the text message, for example the dictation of a response, or the establishment of a telephone connection to the sender of the SMS.

Alternatively, the driver can perform a swipe gesture from top to bottom, ie from the instrument cluster 306 in the direction of virtual wastebasket 316, to clear the notification in the display of the instrument 306. Additionally, the SMS may be deleted, e.g. For example, if the driver does not want to receive messages from a particular sender, or if the content recognizable to the driver is unimportant. This assumes that the sender of a message and / or content summary are made visible in the notification.

Text messages are not limited to SMS. Any text messages can be processed according to the method explained above, for example from social networks, news services, emails, etc.

The executed in response to a gesture editing ^¬ steps and the available gestures may be dependent and may be stored in a control unit, which performs the gesture recognition or which first receives the information element or be riding ^¬ represents the type of information element. The devices exchange communication-compatible attributes for the correct control of the function.

However, it is also possible to summarize processing steps to summarize with the information element, for example in a header or header of the information element bein ^¬ holding file or message. The processing steps can be transmitted in the form of executable by a computer program commands, for example in a script ^¬ language.

In a second example for explaining the method according to the invention, the information element to which one or more processing steps are applied is a traffic announcement or traffic information, e.g. B. concerning a currently traveled or planned route. A corresponding information is displayed on the display of the instrument cluster 306 or the Ver ^¬ traffic announcement is received by a radio receiver of a media and communication unit and reproduced. The media and communication unit is in motor vehicles usually arranged in the center console second display 310 assigned ^¬ , and the control of the relevant functions is at least displayed in the second display 310. Also becomes an input a navigation target usually via the arranged in the middle ^¬ console second display 310 done.

Swiping from left to right, toward the second display 310, the acquisition of traffic information causes the display in the instrument cluster 306 in a navigation system and recalculate the route and calculating an alternative route based transmitted by the information element over ^¬ data. If no route is planned, a probable route can also be estimated and information about an expected time delay can be output on the probable route. Accordingly, a swipe gesture from top to bottom, that is to say to the virtual wastebasket 316, causes discarding of the traffic information without recalculation of the route or other actions.

The swipe gestures in the foregoing examples may be supplemented by a corresponding animation of the item of information represents ^¬ alternate forming image or icon, Example ^¬, by a displacement of the image or icon on the display of the device or, in the case of the swipe gesture from the top is located in the wiping direction downwards only the pushing out of the picture or pictogram from the display in wiping direction.

In a third example for explaining the method according to the invention, the information element to which one or more processing steps are applied is an address or a destination to which route guidance by means of a navigation device is to take place. As mentioned above, the input of an address or destination to which a route towards ^¬ guide is to be made via the arranged in the center console second display 310 is usually. The second display 310 also provides a selection of available functions for selection as a rule. When, after calling up the navigation function, the destination information is entered and displayed on the second display 310 in a window associated with the navigation function, a swipe gesture causes from bottom to top, So from the second display 310 to the head-up display 314, the acquisition of the destination information in a corresponding Naviga ^¬ tion program and the calculation of the route. The route guidance information is then displayed at any given time in the head-up display 314, possibly supplemented by corresponding voice instructions.

In a further development of the method according to the invention, gestures available for an information element additionally depend on operating parameters of a motor vehicle. In the example of the text message, a representation of the message is not provided, because reading texts while driving can lead to dangerous situations. When the motor vehicle is stationary, however, such as abge ^¬ the engine being switched, it is possible to expand the available for a text message information element representing gestures. A swipe gesture from bottom to top, ie from the display in the instrument cluster 306 or from the second display 310 to the head-up display 314, could then cause the representation of the text in the head-up display.

It can be seen from the examples described above that identical gestures, depending on the information element, cause the execution of different processing steps. Thereby, the number of gestures can be kept small, whereby a user can learn the operation quickly.

It should be noted that the home position for gesture recognition is essentially irrelevant as long as a gesture can be clearly recognized. It is therefore not necessary to start a gesture in spatial proximity to a first device or its display and to close it in spatial proximity to a second device or its display in order to effect an assignment of the information element to the second device for processing.

Instead of using a hand or arm on a touch-sensitive surface or in the air Gestures also head ewegungen or movements of the eyes can be detected and used for control in the context of the method according to the invention.

The invention has been described above with reference to a motor vehicle. However, it is also possible to apply the method according to the invention to other systems which can process different information elements in different ways and in which gestures allow a clear assignment of information elements to devices or subsystems.

Such a system is for example a crosslinked Compu ^¬ tersystem with one or more computers or subsystems and corresponding displays. The computers do not have to be identical. Rather, for example, a tablet computer represent a central unit that receives information elements or references to it, but limited in their ability to process certain information elements. On a touch screen of the tablet computer being led ^¬ detected or via a built-in camera tablet computer gestures cause allocation of the information elements to another suitable computer for further processing. It is also conceivable that a gesture which is identical in itself causes an allocation to other devices, computers or subsystems through a changing orientation of the tablet computer in a room.

In another system suitable for the method according to the invention, a computer for the execution of processing steps is provided and connected to a plurality of displays. The computer is further connected to at least one means for recognizing gestures carried out by a user. The computer provides the function of a control unit provided, which applies a detected gesture on an information element and as a function of the gesture and the Informati ^¬ onselement the execution of one or more of information, the on processing relevant processing steps triggers by an ent ^¬ speaking program running on the computer.

Claims

claims

A method for triggering at least one processing step to be applied to an information element, via a display of information associated with the information element on a display, in a system having a plurality of displays and at least one computer connected to the displays, comprising:

Presenting a first information element or a hint representing the presence of a first information element;

- detecting a single gesture executed by the user with respect to the presented first information item or hint;

- in accordance with the detected single gesture, performing one or more first information element concerned processing steps without further Be ^¬ user typed done in terms of the information element or the presented hint.

2. The method of claim 1, wherein the presentation of the first information element or the hint representing this precedes the reception of data associated with the first information ^¬ element.

3. The method of claim 1, wherein the presenting takes place on a first display.

4. The method of claim 1, wherein the gesture speaks ent ^¬ a assigning the information item to a second display.

5. The method of claim 1, wherein the gesture describes an indicated displacement of an information element or the hint representing object from a first display to a second display in the spatial direction. The method of claim 1, wherein the at least one processing step is type-dependent linked to the information ^¬ element.

The method of claim 1, wherein alternative processing steps to be performed for an information element are associated with different gestures.

The method of claim 1, wherein the gesture is not directly related to an information element or hint ^¬ indicative object on a display, and wherein the gesture recognition only within a first period of time after signaling that a new Informationsob ekt is available executed.

The method of claim 8, wherein the first time period is preset for particular types or types of information items or individually adjustable by the user.

A system comprising a plurality of displays and at least one computer connected to the displays, characterized in that the system comprises at least one means for detecting gestures made by a user, the system further comprising a control unit adapted to apply a recognized gesture to an information ^¬ element and depending on the gesture and the information element to trigger the execution of one or more information elements of the relevant processing steps by the at least one computer.