EP2209694B1 - Vehicle communication system and method - Google Patents

Vehicle communication system and method Download PDF

Info

Publication number
EP2209694B1
EP2209694B1 EP08849994.2A EP08849994A EP2209694B1 EP 2209694 B1 EP2209694 B1 EP 2209694B1 EP 08849994 A EP08849994 A EP 08849994A EP 2209694 B1 EP2209694 B1 EP 2209694B1
Authority
EP
European Patent Office
Prior art keywords
steering wheel
communication
display
vehicle
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP08849994.2A
Other languages
German (de)
French (fr)
Other versions
EP2209694A2 (en
EP2209694A4 (en
Inventor
Leonard Cech
Michael Edmund O'boyle
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
TK Holdings Inc
Original Assignee
TK Holdings Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US12/213,591 external-priority patent/US9520061B2/en
Application filed by TK Holdings Inc filed Critical TK Holdings Inc
Publication of EP2209694A2 publication Critical patent/EP2209694A2/en
Publication of EP2209694A4 publication Critical patent/EP2209694A4/en
Application granted granted Critical
Publication of EP2209694B1 publication Critical patent/EP2209694B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B62LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62DMOTOR VEHICLES; TRAILERS
    • B62D1/00Steering controls, i.e. means for initiating a change of direction of the vehicle
    • B62D1/02Steering controls, i.e. means for initiating a change of direction of the vehicle vehicle-mounted
    • B62D1/04Hand wheels
    • B62D1/046Adaptations on rotatable parts of the steering wheel for accommodation of switches
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R16/00Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for
    • B60R16/02Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements
    • B60R16/037Electric or fluid circuits specially adapted for vehicles and not otherwise provided for; Arrangement of elements of electric or fluid circuits specially adapted for vehicles and not otherwise provided for electric constitutive elements for occupant comfort, e.g. for automatic adjustment of appliances according to personal settings, e.g. seats, mirrors, steering wheel
    • B60R16/0373Voice control
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/02Arrangements for holding or mounting articles, not otherwise provided for for radio sets, television sets, telephones, or the like; Arrangement of controls thereof
    • B60R11/0247Arrangements for holding or mounting articles, not otherwise provided for for radio sets, television sets, telephones, or the like; Arrangement of controls thereof for microphones or earphones
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00Arrangements for holding or mounting articles, not otherwise provided for
    • B60R2011/0001Arrangements for holding or mounting articles, not otherwise provided for characterised by position
    • B60R2011/0003Arrangements for holding or mounting articles, not otherwise provided for characterised by position inside the vehicle
    • B60R2011/001Vehicle control means, e.g. steering-wheel or column
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02165Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise

Definitions

  • the present application relates generally to a system for providing a message to a vehicle driver. More specifically, the application relates to a system for facilitating communication and providing information to a vehicle driver.
  • Tum-by-Tum icons are conventionally presented in a small display on the driver instrument panel (IP) or in the steering wheel rim. The icons are typically only updated when a driver completes a required turn or when a new maneuver event is within a designated range.
  • a communication steering wheel device has been developed in which a variety of informational messages can be displayed to the driver primarily through icons. These messages include a variety of warning messages, informational messages about upcoming landmarks, and navigation commands.
  • the communication steering wheel can be configured to present driver warning, assistance, and diagnostic information to the driver within his or her peripheral vision when looking at the approaching road.
  • this information is sent via a wired connection between the steering wheel and the vehicle wired network (CANbus, other any other vehicle bus).
  • the existing connection is generally not redundant, as there are a limited number of physical connections that can exist between the steering column and the steering wheel. Accordingly, a system is needed that can facilitate communication via the communication steering wheel with other vehicle systems not connected to a wired vehicle network to provide useful information and services to the user of the vehicle.
  • DE 10 2004 042 331 shows a communication steering wheel for a vehicle, comprising:
  • a vehicle communication system for a vehicle includes the steering wheel.
  • a vehicle driver may operate a vehicle from a cockpit 1.
  • the cockpit 1 includes a communication steering wheel 10.
  • the driver may rotate the communication steering wheel 10 to turn the vehicle wheels and steer the vehicle in a desired direction.
  • the communication steering wheel 10 may also be configured to provide a message (e.g., a visual message, an audible message, etc.) to the driver related to a current vehicle state, the road geometry or conditions, a predicted vehicle state, or any combination thereof.
  • the message may be generated by a controller (not shown) based on data related to the vehicle and road.
  • the message can be a warning message of excessive or insufficient speed, acceleration, or braking or under-steer or over-steer.
  • the message may also include instructions on actions to take, for example how to negotiate or maneuver a corner, obstacle, or road condition.
  • the communication steering wheel 10 may be any steering wheel usable by a driver to steer the vehicle and capable of providing a message to the driver.
  • Fig. 2 is a block diagram of a vehicle communication system, according to one embodiment.
  • the communication steering wheel 10 includes a display 20, a wireless transceiver 30, a steering sensor 40 and a controller 50.
  • the communication steering wheel 10 also includes a navigation device 60 (e.g., electric compass, GPS, etc.).
  • a navigation device 60 e.g., electric compass, GPS, etc.
  • an external navigation device 70 is operatively connected to the communication steering wheel 10.
  • a wireless device 80 and a vehicle sensor 90 are also operatively connected to the communication steering wheel 10.
  • the communication steering wheel 10 is configured to output and receive signals (wirelessly and via wires). According to one embodiment, the communication steering wheel 10 is configured to output signals to a wireless device 80 and receive signals from the wireless device 80 via a wireless transceiver 30.
  • the signals can convey information of any type, including but not limited to telephony commands, caller ID information, audio, text, video, images, navigation information, turn-by-turn directions, point of interest information, vehicle information or financial information.
  • the communication steering wheel 10 can include other input and output devices that allow a user to interact with the communication steering wheel.
  • the communication steering wheel 10 may include any one of a speaker, a microphone, a light, buttons or switches.
  • the display 20 is configured to output information to a driver.
  • the display 20 is incorporated into the communication steering wheel as shown in FIG. 3 .
  • the display can be anyone of an inorganic LED, an organic LED, an LCD display, a TFT display, an incandescent bulb, a plasma display and a CRT display.
  • the display can be a touch screen that allows a user to input information by touching the display.
  • the communication steering wheel 10 includes a wireless transceiver 30 that is configured to operably connect to a wireless device 80.
  • the wireless transceiver 30 can communicate with the wireless device 80 using a short-range, low power communication protocol.
  • the wireless transceiver 30 communicates with the wireless device 80 using the well known Bluetooth® protocol. Bluetooth is routinely supported in automotive applications for "hands-free" telephony.
  • the wireless transceiver 30 can communicate with the wireless device 80 using a wireless fidelity protocol.
  • the communication steering wheel 10 includes a navigation device 60.
  • the navigation device 60 is housed within the communication steering wheel module.
  • a navigation device 70 may be an external device and operably (preferably via a wireless connection) connected to the communication steering wheel 10.
  • the navigation device may be any one of a electronic compass, GPS or well-know navigational system.
  • the navigation device 60 may be used to locate a desired destination of a user.
  • the address and/or location of the desired destination may be transmitted to the navigation device 60, 70 using the wireless device 80.
  • the navigation system 60 is used to provide turn-by-turn directions to a user.
  • the navigation device 60, 70 can conduct a dead reckoning type analysis of vehicle position using vehicle sensor data obtained by the steering wheel sensor 40 and/or vehicle sensors 90 (described in further detail below).
  • the communication steering wheel 10 is configured to communicate with a wireless device 80.
  • the wireless device 80 is configured to input data to the controller 50.
  • the wireless device 80 is any one of a cell phone, PDA, laptop or other portable electronic device.
  • the wireless device 80 can be built into the vehicle.
  • the wireless device 80 is an external third party device.
  • the wireless device 80 and external navigation device 70 can be embodied in the same physical module.
  • the wireless device 80 also includes a transceiver (not shown) that facilitates communication with the transceiver 30 of the communication steering wheel 10.
  • the wireless device 10 can be used to by a user to transmit destination information to the navigation device 60, 70 via wireless transmission (Bluetooth®), which according to one embodiment, eliminates the need for the communication steering wheel 10 to include a cumbersome input device (e.g., keypad, trackball, switches etc.)
  • Navigation information may be entered in a number of ways including but not limited to direct text entry of a street address, selection of a point of interest from a preprogrammed list: such as restaurants, parks, entertainment facilities, etc. or direct entry of GPS coordinates.
  • information may also be provided to the navigation device 60, 70 from various other sensors including a steering sensor 40 and/or vehicle sensors 90.
  • the steering wheel sensors 40, vehicle sensors 90 and the navigation device 60,70 may be connected to a CANbus so that information can pass both to and from the navigation system via the data bus.
  • the vehicle sensor data could be passed to the communication steering wheel 10 via the car network (e.g., a data bus) and transmitted/exchanged with the navigation system using a wireless protocol (e.g., Bluetooth).
  • a wireless protocol e.g., Bluetooth
  • the controller 50 receives navigation information from a wireless device 80 via the transceiver 30 (Step 310). As shown in step 320, the controller 50 receives data from one or more sensors 40, 90. Next, the controller 50 determines whether a current heading of the vehicle is accurate based on the received navigation information and received data (Step 330). Then, the controller 50 transmits a signal to the display 20 that causes the display to present a message to the driver based on an accuracy of the current heading (Step 340).
  • the message is configured to warn the driver of excessive speed, excessive acceleration, excessive braking, insufficient braking, under-steering or oversteering. According to another embodiment the message is configured to warn the driver of a incorrect heading or incorrect turn.
  • the steering sensors 40 and vehicle sensors 90 will be described in greater detail below.
  • the steering sensor(s) 40 are configured to obtain directional steering information based in a user's operation of the communication steering wheel 10 (e.g., turning) during navigation of a vehicle.
  • the steering sensor(s) 40 are configured to measure at least one of steering wheel angle, steering wheel angle rate, tilt and inertia.
  • using the steering sensor(s) 40 the communication steering wheel 10 provides for dimming the communication steering wheel 10 display 20 when the steering wheel is turned away from the normal or 12 o'clock upright position.
  • the display 20 can be dimmed when turning the communication steering wheel 10 to reduce driver distractions, and raise the display intensity when the steering wheel is returned to the straight or normal position.
  • the brightness of the display 20 may be adjusted or changed to provide a cue to the driver regarding an upcoming turn or other issues that need to be brought to the driver's attention.
  • the driver could be alerted or coached (via the display 20 (e.g., brightness)) to achieve the appropriate steering angle to achieve a navigation segment based on information provided by steering sensors 40 communication steering wheel 10 includes display 20 wherein the intensity increases when the driver is turning at the ideal angle and dims when the driver moves the steering wheel away from the ideal angle.
  • the display 20 color is be used in such a way (e.g., color turns green when the correct steering angle is achieved, and yellow then red when turning at the wrong angle).
  • Data from the navigation device 60, 70 is also input into the controller 50 so that the display 20 can alert the driver to appropriate or inappropriate movement of the steering wheel.
  • the vehicle communication system includes at least one vehicle sensor 90, configured to collect and provide sensor information about the vehicle to the communication steering wheel 10 and/or an external navigation device 70.
  • the sensor 90 can obtain information including but not limited to a directional heading, acceleration, braking, velocity, yaw, and pitch of the vehicle.
  • the controller 50 is configured to receive input from the wireless transceiver 30 and the steering sensor 40 and output a signal to the display 20 based on the received input.
  • the controller 50 can use input from the navigation devices 60, 70, and the vehicle sensor 90.
  • the controller 50 may be an analog controller, a digital controller, a software controller, or any combination thereof.
  • the controller is configured to initiate communication with third party devices using the wireless device 80.
  • the controller 50 is configured to execute software that allows any external device with wireless communication functionality to operate in conjunction with and/or exchange information with the communication steering wheel 10.
  • the software interfaces with the Bluetooth API and can be used for any one of hands-free telephony, audio (e.g., mp3, mp4, etc.) and text data exchange, video and image (e.g., mpeg, jpeg, etc.) data exchange.
  • the controller's 50 functionality can be applied to navigation information, point of interest information, vehicle operational status, transaction or financial data and information queries.
  • a wireless device 80 such as a cell phone or PDA
  • the driver can interact with the cell phone or PDA/Blackberry to receive calls/make calls using the communication steering wheel 10, without direct tactile interaction with the phone. That is, the driver can initiate calls made with the wireless device 80 using an input device of the communication steering wheel 10 (e.g., touch screen, microphone via voice recognition software, physical buttons or switches). Similarly, the driver may accept incoming calls to the wireless device 80 also using an input device of the communication steering wheel 10 (e.g., touch screen, microphone via voice recognition software, physical buttons or switches). For example, caller ID information may be displayed via display 20 to enable a user to decide whether to answer an incoming call to wireless device 80.
  • an input device of the communication steering wheel 10 e.g., touch screen, microphone via voice recognition software, physical buttons or switches.
  • caller ID information may be displayed via display 20 to enable a user to decide whether to answer an incoming call to wireless device 80.

Description

    BACKGROUND
  • The present application relates generally to a system for providing a message to a vehicle driver. More specifically, the application relates to a system for facilitating communication and providing information to a vehicle driver.
  • Conventional vehicle navigation systems employ various displays to inform the driver of the next navigation event (e.g., maneuver or turn) and the distance to the event. Driver navigation based on simple icon information is often termed "Turn-by-Turn" navigation. Tum-by-Tum icons are conventionally presented in a small display on the driver instrument panel (IP) or in the steering wheel rim. The icons are typically only updated when a driver completes a required turn or when a new maneuver event is within a designated range.
  • The quality, reliability, and prevalence of vehicle navigation systems as well as map resolution and road attributes continue to rapidly improve. The National Highway Traffic Safety Administration (NHTSA) has initiated a new regulation mandating the use of electronic stability control (ESC) systems on all U.S. vehicles by the year 2013. Conventional ESC systems use a steering angle sensor to track the real-time status of the driver steering input. Vehicle inertial and wheel speed sensors can provide real-time information about vehicle dynamics. Predictive sensors such as radar, Light Detection and Ranging (LIDAR), and vision sensors can provide anticipatory information of what is in the vehicle path. Map and GPS based navigation systems can provide information on the current and pending road geometry, features and attributes. Intelligent transportation systems such as vehicle to vehicle and vehicle to infrastructure communications can provide additional information on the road and traffic situation. Each of these sensors and sources of data can provide a wide range of information that may be useful in enhancing safety or driver assistance.
  • A communication steering wheel device has been developed in which a variety of informational messages can be displayed to the driver primarily through icons. These messages include a variety of warning messages, informational messages about upcoming landmarks, and navigation commands. The communication steering wheel can be configured to present driver warning, assistance, and diagnostic information to the driver within his or her peripheral vision when looking at the approaching road. Currently this information is sent via a wired connection between the steering wheel and the vehicle wired network (CANbus, other any other vehicle bus). The existing connection is generally not redundant, as there are a limited number of physical connections that can exist between the steering column and the steering wheel. Accordingly, a system is needed that can facilitate communication via the communication steering wheel with other vehicle systems not connected to a wired vehicle network to provide useful information and services to the user of the vehicle.
  • DE 10 2004 042 331 shows a communication steering wheel for a vehicle, comprising:
    • a display configured to output information to a driver of the vehicle; a wireless transceiver configured to operably connect to a wireless device; and a controller configured to receive input from the wireless transceiver; wherein the controller is configured to output a signal to the display.
    US 2002/0135163 describes a vehicle steering device comprising a steering wheel which has a gas bag module with a gas bag and a covering cap on a front side. The covering cap closes an outlet opening for the gas bag up to its unfolding. At least one of a display- and operating device is arrange in the covering cap. The outlet opening runs around at least one of the display- and operating device. KR 2000 0017980 describes a motor vehicle wheel alignment marking device provided to help a safe driving while traveling and parking by visually indicating the alignment state of the wheels to a driver sitting on a driver seat. A motor vehicle wheel alignment marking device comprises: a location detecting/transferring unit and a location receiving/marking unit for receiving the signal outputted from the location detecting/transferring unit for indicating the steering wheel state of the motor vehicle wheel visually. Herein, the location detecting/transferring unit is composed of: a hall sensor detecting unit being attached to the steering wheel of the motor vehicle for detecting the moving state of the steering wheel; a controlling unit for receiving the signal being outputted from the hall sensor detecting unit for outputting after a code transformation; and an ultra violet transferring unit for transferring the transformed signal being outputted from the controlling unit as an ultra violet signal. SUMMARY
  • According to the present invention, there is provided a communication steering wheel as defined in appended claim 1.
  • According to another embodiment, a vehicle communication system for a vehicle includes the steering wheel.
  • It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only, and are not restrictive of the claimed subject matter.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • Features, aspects and advantages of the present invention will become apparent from the following description, appended claims, and the accompanying exemplary embodiments shown in the drawings, which are briefly described below.
    • FIG. 1 is a perspective view of a vehicle cockpit, according to an exemplary embodiment.
    • FIG. 2 is a block diagram of a vehicle communication system, according to an exemplary embodiment.
    • FIG. 3 is a schematic view of a steering wheel including a display, according to an exemplary embodiment.
    • FIG. 4 is process flow diagram of a method for providing driver guidance, according to an exemplary embodiment.
    DETAILED DESCRIPTION
  • Embodiments of the disclosure will be described below with reference to the accompanying drawings. It should be understood that the following description is intended to describe exemplary embodiments, and not to limit the claimed subject matter.
  • Referring to FIG. 1, a vehicle driver may operate a vehicle from a cockpit 1. The cockpit 1 includes a communication steering wheel 10. The driver may rotate the communication steering wheel 10 to turn the vehicle wheels and steer the vehicle in a desired direction. The communication steering wheel 10 may also be configured to provide a message (e.g., a visual message, an audible message, etc.) to the driver related to a current vehicle state, the road geometry or conditions, a predicted vehicle state, or any combination thereof. The message may be generated by a controller (not shown) based on data related to the vehicle and road. For example, the message can be a warning message of excessive or insufficient speed, acceleration, or braking or under-steer or over-steer. The message may also include instructions on actions to take, for example how to negotiate or maneuver a corner, obstacle, or road condition. According to various exemplary embodiments, the communication steering wheel 10 may be any steering wheel usable by a driver to steer the vehicle and capable of providing a message to the driver.
  • Fig. 2 is a block diagram of a vehicle communication system, according to one embodiment. The communication steering wheel 10 includes a display 20, a wireless transceiver 30, a steering sensor 40 and a controller 50. Preferably, the communication steering wheel 10 also includes a navigation device 60 (e.g., electric compass, GPS, etc.). Alternatively, or in addition, an external navigation device 70 is operatively connected to the communication steering wheel 10. In addition, a wireless device 80 and a vehicle sensor 90 are also operatively connected to the communication steering wheel 10.
  • The communication steering wheel 10 is configured to output and receive signals (wirelessly and via wires). According to one embodiment, the communication steering wheel 10 is configured to output signals to a wireless device 80 and receive signals from the wireless device 80 via a wireless transceiver 30. The signals can convey information of any type, including but not limited to telephony commands, caller ID information, audio, text, video, images, navigation information, turn-by-turn directions, point of interest information, vehicle information or financial information.
  • In addition to a display 20 (described in further detail below) the communication steering wheel 10 can include other input and output devices that allow a user to interact with the communication steering wheel. According to one embodiment, the communication steering wheel 10 may include any one of a speaker, a microphone, a light, buttons or switches.
  • The display 20 is configured to output information to a driver. The display 20 is incorporated into the communication steering wheel as shown in FIG. 3. According to one embodiment, the display can be anyone of an inorganic LED, an organic LED, an LCD display, a TFT display, an incandescent bulb, a plasma display and a CRT display. In addition, the display can be a touch screen that allows a user to input information by touching the display.
  • According to one embodiment, the communication steering wheel 10 includes a wireless transceiver 30 that is configured to operably connect to a wireless device 80. The wireless transceiver 30 can communicate with the wireless device 80 using a short-range, low power communication protocol. Preferably, the wireless transceiver 30 communicates with the wireless device 80 using the well known Bluetooth® protocol. Bluetooth is routinely supported in automotive applications for "hands-free" telephony. In the alternative, the wireless transceiver 30 can communicate with the wireless device 80 using a wireless fidelity protocol.
  • According to one embodiment, the communication steering wheel 10 includes a navigation device 60. Preferably, the navigation device 60 is housed within the communication steering wheel module. In the alternative, or in addition to, a navigation device 70 may be an external device and operably (preferably via a wireless connection) connected to the communication steering wheel 10. The navigation device may be any one of a electronic compass, GPS or well-know navigational system. The navigation device 60 may be used to locate a desired destination of a user. According to one embodiment, the address and/or location of the desired destination may be transmitted to the navigation device 60, 70 using the wireless device 80. Via the display, the navigation system 60 is used to provide turn-by-turn directions to a user. According to another embodiment, the navigation device 60, 70 can conduct a dead reckoning type analysis of vehicle position using vehicle sensor data obtained by the steering wheel sensor 40 and/or vehicle sensors 90 (described in further detail below).
  • According to one embodiment, the communication steering wheel 10 is configured to communicate with a wireless device 80. Specifically, the wireless device 80 is configured to input data to the controller 50. The wireless device 80 is any one of a cell phone, PDA, laptop or other portable electronic device. According to one embodiment, the wireless device 80 can be built into the vehicle. In the alternative, the wireless device 80 is an external third party device. According to still another embodiment, the wireless device 80 and external navigation device 70 can be embodied in the same physical module. The wireless device 80 also includes a transceiver (not shown) that facilitates communication with the transceiver 30 of the communication steering wheel 10. According to one embodiment, the wireless device 10 can be used to by a user to transmit destination information to the navigation device 60, 70 via wireless transmission (Bluetooth®), which according to one embodiment, eliminates the need for the communication steering wheel 10 to include a cumbersome input device (e.g., keypad, trackball, switches etc.) Navigation information may be entered in a number of ways including but not limited to direct text entry of a street address, selection of a point of interest from a preprogrammed list: such as restaurants, parks, entertainment facilities, etc. or direct entry of GPS coordinates.
  • According to one embodiment, information may also be provided to the navigation device 60, 70 from various other sensors including a steering sensor 40 and/or vehicle sensors 90. According to one embodiment, the steering wheel sensors 40, vehicle sensors 90 and the navigation device 60,70 may be connected to a CANbus so that information can pass both to and from the navigation system via the data bus. Alternatively, the vehicle sensor data could be passed to the communication steering wheel 10 via the car network (e.g., a data bus) and transmitted/exchanged with the navigation system using a wireless protocol (e.g., Bluetooth).
  • As shown in Fig. 4, information gathered from the sensors 40, 90 can be used in conjunction with navigation data to display a message to a driver. According to one embodiment, the controller 50 receives navigation information from a wireless device 80 via the transceiver 30 (Step 310). As shown in step 320, the controller 50 receives data from one or more sensors 40, 90. Next, the controller 50 determines whether a current heading of the vehicle is accurate based on the received navigation information and received data (Step 330). Then, the controller 50 transmits a signal to the display 20 that causes the display to present a message to the driver based on an accuracy of the current heading (Step 340). According to one embodiment, the message is configured to warn the driver of excessive speed, excessive acceleration, excessive braking, insufficient braking, under-steering or oversteering. According to another embodiment the message is configured to warn the driver of a incorrect heading or incorrect turn. The steering sensors 40 and vehicle sensors 90 will be described in greater detail below.
  • According to one embodiment, the steering sensor(s) 40 are configured to obtain directional steering information based in a user's operation of the communication steering wheel 10 (e.g., turning) during navigation of a vehicle. The steering sensor(s) 40 are configured to measure at least one of steering wheel angle, steering wheel angle rate, tilt and inertia. According to one embodiment, using the steering sensor(s) 40 the communication steering wheel 10 provides for dimming the communication steering wheel 10 display 20 when the steering wheel is turned away from the normal or 12 o'clock upright position.
  • For example, the display 20 can be dimmed when turning the communication steering wheel 10 to reduce driver distractions, and raise the display intensity when the steering wheel is returned to the straight or normal position. In addition, using input from the navigation device 60, 70, the brightness of the display 20 may be adjusted or changed to provide a cue to the driver regarding an upcoming turn or other issues that need to be brought to the driver's attention.
  • In another embodiment, the driver could be alerted or coached (via the display 20 (e.g., brightness)) to achieve the appropriate steering angle to achieve a navigation segment based on information provided by steering sensors 40 communication steering wheel 10 includes display 20 wherein the intensity increases when the driver is turning at the ideal angle and dims when the driver moves the steering wheel away from the ideal angle. Similarly, the display 20 color is be used in such a way (e.g., color turns green when the correct steering angle is achieved, and yellow then red when turning at the wrong angle). Data from the navigation device 60, 70 is also input into the controller 50 so that the display 20 can alert the driver to appropriate or inappropriate movement of the steering wheel.
  • According to one embodiment, the vehicle communication system includes at least one vehicle sensor 90, configured to collect and provide sensor information about the vehicle to the communication steering wheel 10 and/or an external navigation device 70. The sensor 90 can obtain information including but not limited to a directional heading, acceleration, braking, velocity, yaw, and pitch of the vehicle.
  • As shown in FIG. 2, the controller 50 is configured to receive input from the wireless transceiver 30 and the steering sensor 40 and output a signal to the display 20 based on the received input. In addition, the controller 50 can use input from the navigation devices 60, 70, and the vehicle sensor 90. The controller 50 may be an analog controller, a digital controller, a software controller, or any combination thereof. According to another embodiment, the controller is configured to initiate communication with third party devices using the wireless device 80.
  • Preferably, the controller 50 is configured to execute software that allows any external device with wireless communication functionality to operate in conjunction with and/or exchange information with the communication steering wheel 10. Preferably, the software interfaces with the Bluetooth API and can be used for any one of hands-free telephony, audio (e.g., mp3, mp4, etc.) and text data exchange, video and image (e.g., mpeg, jpeg, etc.) data exchange. In addition, in conjunction with the steering sensor 40, the vehicle sensor 90 and the navigation devices 60, 70 the controller's 50 functionality can be applied to navigation information, point of interest information, vehicle operational status, transaction or financial data and information queries.
  • For example, once a wireless device 80 such as a cell phone or PDA has been enabled, the driver can interact with the cell phone or PDA/Blackberry to receive calls/make calls using the communication steering wheel 10, without direct tactile interaction with the phone. That is, the driver can initiate calls made with the wireless device 80 using an input device of the communication steering wheel 10 (e.g., touch screen, microphone via voice recognition software, physical buttons or switches). Similarly, the driver may accept incoming calls to the wireless device 80 also using an input device of the communication steering wheel 10 (e.g., touch screen, microphone via voice recognition software, physical buttons or switches). For example, caller ID information may be displayed via display 20 to enable a user to decide whether to answer an incoming call to wireless device 80.
  • The foregoing description has been presented for purposes of illustration and description. It is not intended to be exhaustive or to be limited to the precise form disclosed, and modifications and variations are possible in light of the above teaching or may be acquired from practice of the disclosure. The above-referenced embodiments were chosen and described in order to explain the principles of the disclosure and as a practical application to enable one skilled in the art to utilize the disclosure in various embodiments, and with various modifications, are suited to the particular use contemplated. It should be understood that the following description is intended to describe exemplary embodiments, and not to limit the claimed subject matter.

Claims (12)

  1. A communication steering wheel (10) for a vehicle, comprising:
    a display (20) configured to output information to a driver of the vehicle;
    a wireless transceiver (30) configured to operably connect to a wireless device (80);
    a sensor (40) configured to obtain directional steering information; and
    a controller (50) configured to receive input from the wireless transceiver (30) and the sensor (40); wherein:
    the controller (50) is configured to output a signal to the display (20) for alerting or coaching a driver to achieve an appropriate steering angle to achieve a navigation segment, the signal being based on the received input and data input from a navigation device (60, 70), to change either: 1) a brightness of the display (20) wherein the intensity of the display increases when the driver is turning at the ideal angle and dims when the driver moves the steering wheel away from the ideal angle, or 2) a color of the display (20) such that the display (20) is a first color when the driver is turning at the ideal angle, and the display (20) is another color, different from the first color, when the driver moves the steering wheel away from the ideal angle.
  2. A communication steering wheel, as claimed in claim 1, wherein the display (20) comprises one or more of an inorganic LED, an organic LED, an LCD display, a TFT display, an incandescent bulb, a plasma display and a CRT display.
  3. A communication steering wheel, as claimed in claim 1, wherein the wireless transceiver (30) is configured to communicate with the wireless device (80) using one of: a short-range, low power communication protocol,
    the Bluetooth® protocol and
    a wireless fidelity protocol.
  4. A communication steering wheel, as claimed in claim 1, wherein the wireless transceiver (30) is configured to operably connect to a wireless device (80) being any one of a cell phone, PDA, laptop or other portable electronic device.
  5. A communication steering wheel, as claimed in claim 1, wherein the sensor (40) is configured to measure at least one of steering wheel angle, steering wheel angle rate, tilt and inertia.
  6. A communication steering wheel, as claimed in claim 1, wherein the signal can convey information of any one of telephony commands, caller ID information, audio, text, video, images, navigation information, turn-by-turn directions, point of interest information, vehicle information or financial information.
  7. A communication steering wheel, as claimed in claim 1, further comprising:
    a speaker, configured to receive a signal from the controller (50) and output an audio signal.
  8. A communication steering wheel, as claimed in claim 1, wherein the controller (50) is configured to initiate communication with third party devices using the wireless device or to input data to the controller (50).
  9. A communication steering wheel, as claimed in claim 1, further comprising:
    the navigation system (60), the navigation system (60) being configured to receive input from the wireless device (80) via the wireless transceiver (30).
  10. A vehicle communication system, comprising:
    a steering wheel (10) according to any of claims 1 to 8.
  11. A vehicle communication system, as claimed in claim 10, further comprising:
    the navigation system (70), the navigation system (70) being operably connected to the controller (50) and configured to receive input from the wireless device (80); and
    at least one vehicle sensor (90), configured to collect and provide sensor information about the vehicle to the navigation system (70).
  12. A vehicle communication system, as claimed in claim 10, wherein the sensor information is at least one of directional heading, acceleration, braking, velocity, yaw, and pitch.
EP08849994.2A 2007-11-13 2008-11-13 Vehicle communication system and method Not-in-force EP2209694B1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US99635107P 2007-11-13 2007-11-13
US629808P 2008-01-04 2008-01-04
US12/213,591 US9520061B2 (en) 2008-06-20 2008-06-20 Vehicle driver messaging system and method
PCT/US2008/083404 WO2009064886A2 (en) 2007-11-13 2008-11-13 Vehicle communication system and method

Publications (3)

Publication Number Publication Date
EP2209694A2 EP2209694A2 (en) 2010-07-28
EP2209694A4 EP2209694A4 (en) 2011-11-30
EP2209694B1 true EP2209694B1 (en) 2015-01-07

Family

ID=40639120

Family Applications (2)

Application Number Title Priority Date Filing Date
EP08849392A Not-in-force EP2209693B1 (en) 2007-11-13 2008-11-13 System and method for receiving audible input in a vehicle
EP08849994.2A Not-in-force EP2209694B1 (en) 2007-11-13 2008-11-13 Vehicle communication system and method

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP08849392A Not-in-force EP2209693B1 (en) 2007-11-13 2008-11-13 System and method for receiving audible input in a vehicle

Country Status (4)

Country Link
EP (2) EP2209693B1 (en)
JP (3) JP2011502885A (en)
CN (2) CN101855120B (en)
WO (2) WO2009064877A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016070950A1 (en) 2014-11-06 2016-05-12 Brückner Maschinenbau GmbH & Co. KG Plasma exhaust purification

Families Citing this family (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE112010004222B4 (en) 2009-10-29 2022-05-05 Joyson Safety Systems Acquisition Llc Steering wheel system with audio input, steering wheel for a vehicle and audio input system for a vehicle steering wheel
KR101307708B1 (en) 2011-05-18 2013-09-11 (주)엘지하우시스 Steering wheel having intelligent remo- con
JP5844178B2 (en) * 2012-02-29 2016-01-13 株式会社日本自動車部品総合研究所 In-vehicle device
US20140371987A1 (en) * 2013-03-15 2014-12-18 Ford Global Technologies Llc Reconfigurable steering wheel with visual feedback related to vehicle systems
EP3950433A1 (en) * 2013-05-23 2022-02-09 NEC Corporation Speech processing system, speech processing method, speech processing program and vehicle including speech processing system on board
CN203439096U (en) * 2013-07-16 2014-02-19 江门市锐速电子科技有限公司 Automobile steering wheel with independent wireless hand-free phone system
US9704049B2 (en) * 2014-08-04 2017-07-11 Gentex Corporation Driver assist system utilizing an inertial sensor
KR101684107B1 (en) * 2015-05-07 2016-12-20 현대자동차주식회사 Electronic device and driver determining method thereof
US20160375924A1 (en) * 2015-06-25 2016-12-29 Steering Solutions Ip Holding Corporation Steering wheel with integrated keyboard assembly
US20160375770A1 (en) * 2015-06-25 2016-12-29 Steering Solutions Ip Holding Corporation Steering assembly and method of control
DE102016225019B4 (en) * 2015-12-29 2020-12-10 Ford Global Technologies, Llc Method for improving speech recognition in a vehicle
US10322682B2 (en) 2016-03-03 2019-06-18 Steering Solutions Ip Holding Corporation Steering wheel with keyboard
EP3447460A4 (en) * 2016-04-21 2019-11-27 NSK Ltd. Method of detecting abnormal sound in steering system, and steering system evaluating device
WO2018000111A1 (en) * 2016-06-26 2018-01-04 李仁涛 Automobile steering wheel with microphones
CN106427830B (en) * 2016-08-26 2018-12-28 东风汽车公司 Voice picking up system and its method
US10144383B2 (en) 2016-09-29 2018-12-04 Steering Solutions Ip Holding Corporation Steering wheel with video screen and airbag
CN106515600B (en) * 2016-10-27 2018-11-27 杭州广安汽车电器有限公司 A kind of separable screen and vehicle mounted guidance on the steering wheel can be hung
JP2018181229A (en) * 2017-04-21 2018-11-15 三菱電機株式会社 Driving support apparatus and message transmitting method
JP6881211B2 (en) * 2017-10-12 2021-06-02 トヨタ自動車株式会社 Vehicle display device
CN108791462A (en) * 2018-08-02 2018-11-13 广东工业大学 A kind of steering wheel and vehicle
CN110450739B (en) * 2019-08-29 2021-02-12 北京梧桐车联科技有限责任公司 Vehicle with a steering wheel
CN112542164B (en) * 2019-09-19 2024-01-26 中车时代电动汽车股份有限公司 Voice control method, device and equipment for vehicle steering wheel and storage medium
JP7258262B2 (en) * 2021-02-17 2023-04-14 三菱電機株式会社 Adjustment device, adjustment system, display device, occupant monitoring device, and adjustment method

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4850015A (en) 1987-10-27 1989-07-18 Gary Martin Vehicular steering wheel telephone apparatus
US5666102A (en) * 1996-07-24 1997-09-09 United Technologies Automotive Systems, Inc. Vehicle signals incorporated into steering wheel rim
CN1175517A (en) * 1997-08-18 1998-03-11 李维华 Mid-set steering wheel type automobile function monitoring display device controlled by modular computer
JP3498582B2 (en) * 1998-08-26 2004-02-16 トヨタ自動車株式会社 Parking assistance device
JP2001010429A (en) * 1999-06-29 2001-01-16 Fujitsu Ten Ltd Vehicle operation assist device
KR100395478B1 (en) * 1999-12-28 2003-08-25 이수송 Wheel position detector
JP2001253219A (en) * 2000-03-14 2001-09-18 Toyota Motor Corp Vehicle characteristic adjusting device
DE10013432C1 (en) 2000-03-17 2001-09-06 Daimler Chrysler Ag Microphone device installed in automobile positioned in front of driver facing towards front windscreen for reception of speech signals reflected from windscreen surface
US20020068605A1 (en) * 2000-08-03 2002-06-06 Stanley Winfield Scott Manual mobile-communication interface
CN2444364Y (en) * 2000-09-22 2001-08-22 航欣工业股份有限公司 Display apparatus for communication data
JP3888572B2 (en) * 2000-11-22 2007-03-07 マツダ株式会社 Vehicle surrounding information display device
DE20105002U1 (en) 2001-03-22 2001-07-26 Trw Automotive Safety Sys Gmbh Vehicle steering device
JP2003066986A (en) * 2001-08-23 2003-03-05 Sharp Corp Voice recognizing robot
US6983170B2 (en) * 2001-10-03 2006-01-03 Jerry Stulberger Mobile cellular telephone
JP2003195890A (en) * 2001-12-25 2003-07-09 Nippon Seiki Co Ltd Speech operating device
JP3946067B2 (en) * 2002-03-25 2007-07-18 Tcm株式会社 Industrial vehicle meter display device
JP2003300468A (en) * 2002-04-09 2003-10-21 Nissan Motor Co Ltd Vehicular driving display device
JP4019839B2 (en) * 2002-07-23 2007-12-12 ソニー株式会社 NAVIGATION DEVICE, NAVIGATION METHOD, NAVIGATION DEVICE PROGRAM, AND RECORDING MEDIUM CONTAINING NAVIGATION DEVICE PROGRAM
JP2004132897A (en) * 2002-10-11 2004-04-30 Mitsubishi Electric Corp On-vehicle navigation system
US6882906B2 (en) * 2002-10-31 2005-04-19 General Motors Corporation Vehicle information and interaction management
JP4082579B2 (en) * 2002-10-31 2008-04-30 独立行政法人情報通信研究機構 In-vehicle terminal
JP4352790B2 (en) * 2002-10-31 2009-10-28 セイコーエプソン株式会社 Acoustic model creation method, speech recognition device, and vehicle having speech recognition device
JP4003675B2 (en) * 2003-03-20 2007-11-07 株式会社デンソー Driving support system and in-vehicle driving support device
DE10352733A1 (en) * 2003-04-04 2004-10-28 Takata-Petri Ag Motor vehicle steering wheel has a driver communications interface permitting display of information to the driver and input of commands by the driver with minimal distraction from the driving situation
US7076072B2 (en) * 2003-04-09 2006-07-11 Board Of Trustees For The University Of Illinois Systems and methods for interference-suppression with directional sensing patterns
JP4162604B2 (en) * 2004-01-08 2008-10-08 株式会社東芝 Noise suppression device and noise suppression method
KR101022159B1 (en) * 2004-03-04 2011-03-17 엘지전자 주식회사 navigation apparatus for cars and working method thereof
DE102004042331A1 (en) * 2004-05-07 2005-12-01 Walter Becker Steering wheel for motor vehicle with integral monitor in middle of steering wheel
KR20060026157A (en) * 2004-09-20 2006-03-23 주식회사 대우일렉트로닉스 A steering wheel having a display device and input device
JP2006226840A (en) * 2005-02-17 2006-08-31 Denso It Laboratory Inc Navigation device and navigation method
JP2006264615A (en) * 2005-03-25 2006-10-05 Toyota Motor Corp Display device for vehicle
JP2006327421A (en) * 2005-05-26 2006-12-07 Tokai Rika Co Ltd Steering control device and steering system of by-wire type
JP2007001417A (en) * 2005-06-23 2007-01-11 Nissan Motor Co Ltd Visual drive support device
JP2007026271A (en) * 2005-07-20 2007-02-01 Isuzu Motors Ltd Driving concentration determination method and device therefor
DE112005003669T5 (en) * 2005-08-10 2008-06-19 Autoliv ASP, Inc., Ogden Improved control device
US7605693B2 (en) * 2006-08-30 2009-10-20 Kulas Charles J Visual indicators on vehicle steering wheel displayed in response to hand position
JP5495355B2 (en) * 2006-12-15 2014-05-21 現代自動車株式会社 Display system for automotive steering wheel

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016070950A1 (en) 2014-11-06 2016-05-12 Brückner Maschinenbau GmbH & Co. KG Plasma exhaust purification
DE102014016380A1 (en) 2014-11-06 2016-05-12 Brückner Maschinenbau GmbH & Co. KG Plasma exhaust gas purification

Also Published As

Publication number Publication date
JP2011502885A (en) 2011-01-27
EP2209694A2 (en) 2010-07-28
EP2209694A4 (en) 2011-11-30
EP2209693A4 (en) 2011-11-30
CN101855120A (en) 2010-10-06
JP2011502884A (en) 2011-01-27
EP2209693A1 (en) 2010-07-28
EP2209693B1 (en) 2013-04-03
CN101855121B (en) 2014-01-29
WO2009064886A3 (en) 2009-08-06
WO2009064886A2 (en) 2009-05-22
CN101855120B (en) 2012-07-04
JP2015098320A (en) 2015-05-28
JP5432169B2 (en) 2014-03-05
CN101855121A (en) 2010-10-06
JP6141251B2 (en) 2017-06-07
WO2009064877A1 (en) 2009-05-22

Similar Documents

Publication Publication Date Title
EP2209694B1 (en) Vehicle communication system and method
US8296012B2 (en) Vehicle communication system and method
US9855957B2 (en) Method for using a communication terminal in a motor vehicle while autopilot is activated and motor vehicle
JP5278292B2 (en) Information presentation device
JP4847178B2 (en) Vehicle driving support device
WO2014054151A1 (en) Vehicle-mounted information processing device
KR20190007286A (en) Driving system for vehicle and Vehicle
US10160319B2 (en) Vehicle display processing apparatus
JP2008021181A (en) Driving support information notifying device
CN102066181A (en) Vehicle driver messaging system and method
US7647170B2 (en) Navigation apparatus displaying visual guidance
JP6614573B2 (en) Automatic operation control device and automatic operation control method
JP5735657B2 (en) Display device and display method
US20080055056A1 (en) Intersection information provision system and driving assist system
WO2020230308A1 (en) Driving assistance method and driving assistance device
US11151871B2 (en) Autonomous driving vehicle information presentation apparatus
WO2015045112A1 (en) Driving-assistance apparatus, control method, program, and storage medium
WO2013088557A1 (en) Display device and display method
US20210197863A1 (en) Vehicle control device, method, and program
JP6620775B2 (en) Display device and display method
JP2009265806A (en) Vehicular fuel-saving driving support device
WO2013046428A1 (en) Head-up display, display device and display method
WO2014192135A1 (en) Display device, display method and display program
JP2002150484A (en) Automobile drive support system
JP2014235054A (en) Display device, method, and program

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20100428

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

A4 Supplementary search report drawn up and despatched

Effective date: 20111027

RIC1 Information provided on ipc code assigned before grant

Ipc: B60R 16/037 20060101ALI20111021BHEP

Ipc: B62D 1/04 20060101AFI20111021BHEP

Ipc: G10L 21/02 20060101ALI20111021BHEP

Ipc: B60R 11/02 20060101ALI20111021BHEP

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20121108

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602008036286

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: B62D0001040000

Ipc: G10L0021021600

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: B60R 11/02 20060101ALI20140613BHEP

Ipc: B62D 1/04 20060101ALI20140613BHEP

Ipc: B60R 11/00 20060101ALI20140613BHEP

Ipc: G10L 21/0216 20130101AFI20140613BHEP

Ipc: B60R 16/037 20060101ALI20140613BHEP

Ipc: G10L 21/0208 20130101ALI20140613BHEP

INTG Intention to grant announced

Effective date: 20140723

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 706212

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150215

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602008036286

Country of ref document: DE

Effective date: 20150219

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20150107

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 706212

Country of ref document: AT

Kind code of ref document: T

Effective date: 20150107

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150407

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150407

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150408

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150507

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602008036286

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20151008

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151113

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20151113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151130

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151130

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20160729

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151113

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20151130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20081113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150107

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20201119

Year of fee payment: 13

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602008036286

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220601