US20190156109A1 - Forward-looking radar signals alert via driver's phone - Google Patents

Forward-looking radar signals alert via driver's phone Download PDF

Info

Publication number
US20190156109A1
US20190156109A1 US16/255,767 US201916255767A US2019156109A1 US 20190156109 A1 US20190156109 A1 US 20190156109A1 US 201916255767 A US201916255767 A US 201916255767A US 2019156109 A1 US2019156109 A1 US 2019156109A1
Authority
US
United States
Prior art keywords
vehicle
teleproctor
driver
phone
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US16/255,767
Inventor
Jeffrey T Haley
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=52480001&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=US20190156109(A1) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Individual filed Critical Individual
Priority to US16/255,767 priority Critical patent/US20190156109A1/en
Publication of US20190156109A1 publication Critical patent/US20190156109A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W40/09Driving style or behaviour
    • G06K9/00288
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60KARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K28/00Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions
    • B60K28/02Safety devices for propulsion-unit control, specially adapted for, or arranged in, vehicles, e.g. preventing fuel supply or ignition in the event of potentially dangerous conditions responsive to conditions relating to the driver
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/18Propelling the vehicle
    • B60W30/18009Propelling the vehicle related to particular drive situations
    • B60W30/18163Lane change; Overtaking manoeuvres
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08Interaction between the driver and the control system
    • B60W50/12Limiting control by the driver depending on vehicle state, e.g. interlocking means for the control input for preventing unsafe operation
    • G06K9/00597
    • G06K9/00838
    • G06K9/00845
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q40/00Finance; Insurance; Tax strategies; Processing of corporate or income taxes
    • G06Q40/08Insurance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/59Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/593Recognising seat occupancy
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/59Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18Eye characteristics, e.g. of the iris
    • GPHYSICS
    • G07CHECKING-DEVICES
    • G07CTIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C5/00Registering or indicating the working of vehicles
    • G07C5/008Registering or indicating the working of vehicles communicating information to a remotely located station
    • GPHYSICS
    • G07CHECKING-DEVICES
    • G07CTIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
    • G07C5/00Registering or indicating the working of vehicles
    • G07C5/02Registering or indicating driving, working, idle, or waiting time only
    • G07C5/06Registering or indicating driving, working, idle, or waiting time only in graphical form
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04MTELEPHONIC COMMUNICATION
    • H04M1/00Substation equipment, e.g. for use by subscribers
    • H04M1/60Substation equipment, e.g. for use by subscribers including speech amplifiers
    • H04M1/6033Substation equipment, e.g. for use by subscribers including speech amplifiers for providing handsfree use or a loudspeaker mode in telephone sets
    • H04M1/6041Portable telephones adapted for handsfree use
    • H04M1/6075Portable telephones adapted for handsfree use adapted for handsfree use in a vehicle
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/11Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/56Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • H04N23/74Circuitry for compensating brightness variation in the scene by influencing the scene brightness using illuminating means
    • H04N5/2256
    • H04N5/2354
    • H04N5/332
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04WWIRELESS COMMUNICATION NETWORKS
    • H04W4/00Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/80Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8006Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0809Driver authorisation; Driver identity check
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818Inactivity or incapacity of driver
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/22Strain gauge
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40Photo, light or radio wave sensitive means, e.g. infrared sensors
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403Image sensing, e.g. optical camera
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2510/00Input parameters relating to a particular sub-units
    • B60W2510/08Electric propulsion units
    • B60W2510/085Power
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00Input parameters relating to overall vehicle dynamics
    • B60W2520/10Longitudinal speed
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00Input parameters relating to overall vehicle dynamics
    • B60W2520/10Longitudinal speed
    • B60W2520/105Longitudinal acceleration
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00Input parameters relating to overall vehicle dynamics
    • B60W2520/12Lateral speed
    • B60W2520/125Lateral acceleration
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00Input parameters relating to occupants
    • B60W2540/043Identity of occupants
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2555/00Input parameters relating to exterior conditions, not covered by groups B60W2552/00, B60W2554/00
    • B60W2555/60Traffic rules, e.g. speed limits or right of way
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00Input parameters relating to data
    • B60W2556/45External transmission of data to or from the vehicle
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60WCONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00Input parameters relating to data
    • B60W2556/45External transmission of data to or from the vehicle
    • B60W2556/50External transmission of data to or from the vehicle of positioning data, e.g. GPS [Global Positioning System] data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/10Recognition assisted with metadata

Definitions

  • Auto insurance companies, auto rental companies, parents, and employers would like to know how much a vehicle is driven, by whom, when, where, at what speed compared to the speed limit, carrying what load, and whether any of the drivers take their eyes off the road for too long or engage in other risky driving behaviors.
  • Parents and employers would like reports on some of these items promptly upon occurrence. All four want assurance that the monitoring cannot be avoided by subterfuge.
  • the teleproctor is installed in the vehicle by the insured or built into a new car. Installation for retrofit requires no skills. It is quick and easy and gives immediate feedback whether successful or unsuccessful.
  • the teleproctor merely needs to be adhered to the dashboard or the windshield or the rear-view-mirror at a spot where the driver can see it and either a power wire is then plugged into a power source, such as a USB port or a cigarette lighter receptacle, or the teleproctor includes a photovoltaic panel that charges its battery. To verify correct set up, the teleproctor sends a message via a radio network to a server and, when it receives an acknowledgment, beeps and flashes a green light.
  • the teleproctor collects data and wirelessly sends it to a central repository from which data is provided to the insurance company and/or the car owner (rental company, parents, or employer).
  • the teleproctor can be set to give auditory reports and/or visual reports to the driver whenever it reports to the insurance company that the driver was exceeding the speed limit by more than a threshold or had eyes looking away from the road for too long while the car is moving or other detected behaviors, bad or good.
  • the teleproctor can include wireless network communications circuitry and a network account can be maintained for transmission, like with the General Motors Onstar system.
  • the teleproctor can send the data to pre-identified mobile phone via Bluetooth or WiFi when that phone is in the vehicle and the phone runs an app that forwards the data in SMS messages or via internet to the data repository.
  • a vehicle owner or insurance company can instruct the repository to forward selected data events immediately upon receipt.
  • the teleproctor provides data to answer these questions:
  • the most important part of the subterfuge detection system detecting removal of the retrofit teleproctor from the vehicle—requires no extra hardware, merely sophisticated software.
  • This component of the teleproctor can be a valuable component to add to any device with radio circuits where detection of removal is important.
  • FIG. 1 shows a passenger vehicle with a teleproctor adhered to the windshield.
  • FIG. 2 shows the circuit components of a teleproctor.
  • the teleproctor 21 includes a camera 29 mounted on a base that includes electronic circuitry.
  • the base need not be larger than a small mobile phone.
  • the base may be installed in new vehicles built into the dashboard.
  • the camera housing includes an infra-red light emitter that shines light not visible to humans toward the camera's field of view.
  • the camera pixels detect both visible and infra-red light.
  • the base may be adhered to the windshield 15 near the rear view mirror.
  • it may be adhered to the dashboard or affixed to the mirror.
  • it has a wire that leads to a plug for a USB port (5 volts) or to a 12 volt outlet (cigarette lighter socket) or it includes a solar photovoltaic panel that charges a battery.
  • the teleproctor includes an image processor circuit 28 which is programmed to use image recognition to determine:
  • the teleproctor 21 is programmed to learn the necessary recognition details automatically after the teleproctor is installed. Facial image and eye position recognition methods are well known. If the camera aim is changed so that it is not pointed at a human face and the car is moving faster than a threshold, this fact is reported to the driver and to the owner and/or insurance company.
  • a first listed driver sits in the driver's seat and adjusts the camera angle to point at his or her eyes.
  • the user interface for the teleproctor can be via a mobile phone or laptop or tablet computer with a Bluetooth or WiFi or similar connection. Data to recognize each other pre-listed driver—number 2, number 3, etc.—is also stored in this way the first time each of them drives. The teleproctor then determines when it is looking at one of these drivers and can report the driver's identity with little data transmitted.
  • the teleproctor reports this fact to the driver.
  • the driver will be reported to the owner and/or insurance company as a not-pre-listed driver unless the system is commanded to store the new facial data as the image of a newly listed driver.
  • the image of each face used to train the system is transmitted to the owner and/or insurance company and stored in their records for human review if and when necessary. If a person changes their facial appearance, by changing their glasses or other features, they simply reprogram the teleproctor to store their new facial data characteristics.
  • the teleproctor can keep two or more stored sets of facial data for each driver, such as with dark glasses or clear glasses or with a hat, etc.
  • the entire original images may be uploaded to a central server where the facial recognition processing is done. Then, each time the vehicle starts moving from a stop long enough to change drivers, an image is taken and uploaded for facial recognition.
  • the teleproctor includes components for eye direction determination, such as by identifying the pupils as contrasted against whites of the eyes and computing metrics of the pupils and the whites on each side of the pupils. Any method for making these computations may be used.
  • the image processor may update the looking-at-the-road eye position data set for the driver by assuming that an eye position that is steady while the car is moving with only short glances in other directions should be the basis for updating the data set.
  • the system concludes that the eyes were not looking at the road. If the dark pupils move to the side and the eye metrics change a large amount, the head has turned, such as to check the blind spot.
  • the brightness of the captured data in the visible and infra-red spectra may be adjusted independently.
  • the intensity of emitted infra-red light may be adjusted independently for each data set, giving a total of three possible brightness adjustments for each of eye direction recognition and facial recognition purposes. For example, for eye direction determination when the driver is wearing dark glasses, the three brightnesses will be adjusted for optimal distinction between the pupils and whites of the eyes as seen through the dark glasses. These brightness adjustments will typically be too bright for recognition of other facial features.
  • the teleproctor includes a determiner of location, speed, and speed limit 27 which places into the data store 24 location data and speed relative to the local speed limit.
  • the location, speed, and speed limit determiner 27 may get this data from a GPS with map circuit in the vehicle.
  • the location, speed, and speed limit determiner 27 includes a GPS circuit and a stored digital map with speed limits for each location. The map is periodically updated by download through the radio transceiver 30 .
  • the teleproctor saves in the data store 24 the location, speed, and speed limit data for reporting.
  • the teleproctor includes at least two accelerometers.
  • An acceleration data processor can determine when lane changes are made and the image recognition processor can determine whether the driver first looked in the blind spot. If the teleproctor is built into the vehicle, it can be coupled to data sources from the vehicle such as whether a turn signal was activated before changing lanes and braking or turning data.
  • the acceleration data processor can determine when the driver causes hard braking or cornering or speeding up. A summary of these determinations can be uploaded to the data repository for review by the insurance company or owner.
  • Rental companies would like to know how much load their vehicle is carrying on each trip as a gauge of wear. This information can be used to set lower rates for people who carry light loads. Data from the accelerometers on hard acceleration, hard cornering and hard braking is also of interest to these companies because it increases tire wear. Parents and employers would like to know when the vehicle is being used to carry extra passengers.
  • a load computation can be implemented with data showing power consumption provided by engine sensors.
  • the built-in teleproctor receives input from the vehicle engine computer. When the engine power output is high, either the vehicle is going up a hill or it is accelerating. Using the speed determining circuits, these two factors can be computationally separated. By comparing the power output to change in speed and factoring out any hill incline, the load the vehicle is carrying can be computed.
  • Another method to distinguish between climbing a hill and speeding up uses a direction of gravity+acceleration sensor. By comparing the output of this sensor to change in speed, these two factors can be separated.
  • a more direct method for obtaining load data to report is to add strain gauges to one or more suspension springs of the vehicle. It may be a tiny strain gauge that directly senses strain in the metal of the spring or it may be a distance sensor that senses the distance that the spring is collapsed overall. Placing the gauge on only one spring, a rear spring, may be sufficiently accurate. A wire is run from the strain gauge to the teleproctor or to the vehicle's computer which passes the data on to the teleproctor. Of course, the teleproctor circuits may be built into the vehicle's computer, with only the camera and other sensors being outside the vehicle computer.
  • a radio control processor 23 commands the radio transceiver 30 to send all the new data in the data store 24 to a central repository 31 where it is repackaged and supplied to the insurance companies and/or car owner.
  • the transmission route may go directly from a wireless network transmitter built into the teleproctor or the vehicle, in which case an account for the teleproctor is maintained in the radio network, or it may pass via Bluetooth or WiFi to a phone in the car which acts as the first link in a radio network 32 .
  • Any programmable “smart” phone can be programmed with an “app” to perform this function automatically without the phone owner taking any action other than leaving the phone and the app turned on whenever the phone is in the vehicle.
  • the phone automatically detects the Bluetooth or WiFi signal from the teleproctor and connects to it.
  • the teleproctor and phone app together then determine how often a data bundle is uploaded through the phone. For example, if the phone is in an area with poor reception, the uploads may be delayed.
  • This uploading of the data can be programmed to happen the first occasion after a designated time each day that one of the designated phones with Bluetooth or WiFi is in the car and the car is turned on, or other occasions. It can be programmed to happen immediately when certain events are detected or upon request from the system server 31 .
  • the teleproctor can be programmed so that, when a specified number of events of not looking at the road for longer than a threshold occur within a specified window of time, a report is immediately sent via the radio network. Similarly, it can be programmed to immediately send a report when speed over the speed limit is maintained for longer than a threshold amount of time or any other event. If the teleproctor detects extreme deceleration suggestive of an accident, or upon request from the system server, it can transmit images from the teleproctor and sounds collected by a microphone in the teleproctor or the phone for a period of time.
  • the actions of the image processor and the radio control processor can be updated, they are each controlled by programs stored in a rewritable program store 25 which can be rewritten by download through the radio network 32 and loaded into the program store 25 .
  • the teleproctor can upload data showing:
  • the teleproctor can be set to play an auditory message to the driver through a speaker in the teleproctor or the driver's phone and/or present a visual message to the driver in a display for the driver whenever speeding or eyes off the road or other events are reported to the insurance company. With each message to the driver, the teleproctor can tell the driver how much money the driver lost on insurance costs due to speeding or eyes off the road, etc.
  • Data elements 1-7 above can be reported without adding a teleproctor camera into the vehicle.
  • Some vehicles and mobile telephones already have the required hardware. All that is required is software to be downloaded into them to work as follows.
  • a processor receives vehicle speed from the vehicle, from either a speedometer or a GPS circuit in the vehicle.
  • the processor receives a speed limit from the GPS and a stored map.
  • the processor may be in the on-board vehicle electronics associated with the GPS or in a mobile telephone that communicates with the on-board vehicle electronics via Bluetooth or WiFi.
  • the phone provides acceleration data.
  • the mobile telephone runs an app that receives the data and forwards it to the remote parties.
  • each phone in the vehicle running an appropriate app can automatically connect to the teleproctor for bi-directional communication when the phone is in the vehicle.
  • the teleproctor can know which listed driver is driving the vehicle.
  • the teleproctor can advise the driver's phone that it should enter a restricted mode when the vehicle is moving.
  • the usual drivers' phones are each identified to the teleproctor when they first connect to it. Then, with the phone owner's acceptance by user input to load the app, the app will put the phone in a restricted mode whenever the teleproctor advises the phone that its owner is in the driver's seat and the vehicle is moving. If the driver's usual phone is not running the app to allow restriction, this suspicious circumstance can be reported to the vehicle owner or insurance company which can use this information to set rates or rules.
  • An optimal form of functionality restriction might block all text message alerts or display and all reading, writing, or other user interactions with a display while moving, but, to accommodate emergencies, it should not block all voice communications while moving.
  • restrictions that can be implemented in the phone include:
  • Level of attention required for driving can be approximated with the accelerometers in the teleproctor.
  • the teleproctor can advise the driver's phone to enter a more restricted state when the vehicle is speeding up or slowing down or turning or quickly changing lanes. If it is built into the vehicle, the teleproctor can receive inputs from various vehicle sensors that suggest when more attention is required for driving. At these times, the outgoing voice signal or the incoming voice signal or both can be interrupted or suspended (and perhaps replaced with a sound or recorded words) to reduce distraction for the driver and inform the other party to the conversation that the driver needs to pay attention to driving at this time.
  • restriction There can be multiple levels of restriction based on time of day or location on a map in memory based on GPS location or other factors.
  • a restriction might be to stop the handset speaker from working and require use of a headset speaker.
  • Conversations with a passenger in a car present less of a distraction than telephone conversations because the remote conversant does not know what else the driver is doing simultaneously and what the driver is seeing, while the passenger does.
  • the passenger understands what is happening when the driver pauses to concentrate on driving.
  • the concentration demands for a telephone conversation can be reduced and made more like the demands for conversation with a passenger by periodically playing a beep or other signal to other party to remind them that the person they are talking to is also driving.
  • the app running in the driver's phone can be programmed so that, when a caller calls while the phone is in a restricted state, before the phone rings, a message is played to the caller saying: “The person you are calling is driving. If this call is important enough to interrupt that person while driving, press 1 or say yes.” The called phone would only ring if the caller presses 1 or says yes. Otherwise, the call is rerouted to voice mail.
  • the teleproctor includes a rechargeable battery that keeps it working for a length of time after it is depowered.
  • the teleproctor sends a radio ping out the power wire and measures the radio echo signature by a process described below.
  • the teleproctor takes no action other than going into its normal sleep mode. If the echo signature shows that the teleproctor is unplugged or a fuse is removed, whether accidently or otherwise, it plays an auditory alert warning to plug it back in. If it is not soon plugged in to the same vehicle as determined by the echo signature methods described below, the teleproctor reports this fact to the insurance company and/or owner.
  • the teleproctor To determine the cause of depowering, the teleproctor includes a circuit that captures and records a radio-echo signature of the power source and its connections. Using its radio transmitter, which it has for Bluetooth or WiFi communication with the phone or for wide area data network communication more directly to a server, the teleproctor emits a ping or series of pings into the power wire. Using its radio receiver, the teleproctor records a signature of the echo to obtain a radio echo signature of the vehicle electronics.
  • the power wire echo will be different if the power source is a 12 volt to 5 volt converter such as for USB receptacles rather than a 12 volt source such as from a cigarette lighter. It may also be affected by plugging other electronic devices into the vehicle. These changes can be distinguished as not material changes because the echo will be vastly different if the teleproctor is left unplugged or a fuse is removed or blown.
  • the echo data set may be reduced to a small but distinctive data set known as a signature hash.
  • the algorithm for computing the hash is developed by collecting full signature data sets for many different vehicles and power supplies and developing the algorithm to make the hash data set optimally small while still distinguishing between the plugged and unplugged or no fuse conditions in all cases, no matter what else may be plugged in.
  • a total vehicle radio echo signature is obtained.
  • the radio echo signature is taken using the teleproctor's antenna for both broadcast and reception to obtain a signature of the entire vehicle, with a major factor being the shape of and relationship to all metal in the vehicle and a very minor factor being a change in the configuration of what's plugged into the vehicle's power and where. This allows users to unplug the retrofit teleproctor, make changes to what's plugged in and where, and then plug in the retrofit teleproctor again without a problem. If the total vehicle radio echo signature hash is the same as before, no alert is sent to the insurance company. For the built in teleproctor, no total vehicle radio echo signature capability is required.
  • the algorithm for computing the hash for the vehicle radio echo signature is developed by collecting full signature data sets for many different vehicles and developing the algorithm to make the hash data set optimally small while still distinguishing between as many vehicle models as possible and never computing two different hashes for the same vehicle.
  • a driver might, while the vehicle power is off, unplug the retrofit teleproctor, leave it unplugged for some trips, and then plug it back into the same vehicle.
  • the back-up battery has enough power to last several days and, while depowered in sleep mode, the teleproctor wakes up periodically, about every 3 hours, and queries the GPS to determine its location. If the location is not the same as where the teleproctor went to sleep, the insurance company and/or owner is notified.
  • a driver might, while the vehicle power is off, unplug the retrofit teleproctor and remove it from the vehicle, leaving it in the same location as where the teleproctor went to sleep while the vehicle drives away.
  • the back-up battery has enough power to last several days and, while depowered in sleep mode, the teleproctor wakes up periodically, about every 3 hours, and takes a total vehicle radio echo to determine the radio reflection characteristics of its surroundings and computes a hash. If the computed hash is not the same as for the vehicle in which the teleproctor was installed, the insurance company and/or owner is notified.
  • the total vehicle echo signature never needs to be taken because the teleproctor cannot easily be removed.
  • the original total vehicle echo signature hashes are computed when the teleproctor is initially set up.
  • the teleproctor is programmed to report that the data is likely wrong if the eye direction data does not change with glances away from the road as much as is minimally human.
  • the angle of looking at the passenger will be too oblique and will be automatically detectable by the image processor. If the camera is moved from its proper location, the echo signature of the total vehicle will be detectably different.
  • Crypto-authenticating chips are well known. They are designed to receive as input a first data string and, in response, output a second data string. When both data strings are processed together with a secret key, the result will show the second data string to have been produced by the crypto-authenticating chip or not.
  • the second string cannot be inexpensively determined from knowing the first string without having the key.
  • the contents of the authenticating chip cannot be read and duplicated because a fusible link was burned after manufacture.
  • the data compilation server 31 sends a first data string to the teleproctor mounted in the vehicle which replies with a second data string from a crypto-authentication circuit 22 included in the teleproctor. If the data compilation server determines that the strings do not match with the key, the data is so annotated.
  • Each teleproctor sends to the data compilation server 31 an identifier for use to determine which key should be used.
  • the identifier may be the teleproctor's mobile number in the network or it may be a separate identifier stored in and sent by the authentication circuit 22 (e.g., a serial number).
  • An alternative design for an authentication circuit uses a clock in the teleproctor that is synchronized with a clock in the central server to periodically change a data string that it sends to the server. Each new data string must match what is expected to achieve authentication.
  • Other authentication circuits are known and may be used.
  • teleproctors require approved authentication circuits 22 to be built in by licensed manufacturers to avoid bootlegging of devices that report false information, the authentic teleproctors can be sold freely though all channels of trade and installed by anyone.
  • the server that checks for the authentication chip is the only component that is permitted to load an updated program into the teleproctor.
  • the authentication chip is programmed to also check the credentials of the server and insure that no other source can supply an updated program.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Finance (AREA)
  • Accounting & Taxation (AREA)
  • Human Computer Interaction (AREA)
  • Technology Law (AREA)
  • Economics (AREA)
  • General Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Marketing (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Development Economics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Ophthalmology & Optometry (AREA)
  • Traffic Control Systems (AREA)

Abstract

When an automobile's risk alerting forward looking radar senses a risk, or the GPS circuit with map indicates that a zone of high risk (such as certain intersections) is being entered, this system instructs the phone to play an alert sound into any on-going telephone conversation or suspend the call so that both parties know the driver needs to pay attention.

Description

  • This application is a continuation of Ser. No. 15/094,672 filed Apr. 8, 2016, which was a continuation of Ser. No. 13/975,246 filed Aug. 23, 2013, which issued as U.S. Pat. No. 9,311,544, and claims priority from 61/693,079 filed Aug. 24, 2012 as well as from 61/728 filed Nov. 20, 2012.
  • BACKGROUND
  • Auto insurance companies, auto rental companies, parents, and employers would like to know how much a vehicle is driven, by whom, when, where, at what speed compared to the speed limit, carrying what load, and whether any of the drivers take their eyes off the road for too long or engage in other risky driving behaviors. Parents and employers would like reports on some of these items promptly upon occurrence. All four want assurance that the monitoring cannot be avoided by subterfuge.
  • Insurance companies and auto rental companies would offer lower rates or trip-by-trip discounts where they are assured that technology is in place to report the above information without possibility of avoidance. To be sure they qualify for lower rates, drivers would like immediate feedback on actions that might preclude lower rates.
  • To reduce annoyance, avoid temptation, reduce risk, and show the vehicle owner and/or insurance company that they are a safe driver, some drivers would like their phones to automatically detect when they are driving and, at those times, not ring with an incoming call unless the caller believes the call is important enough to be put though while the driver is driving, delay the ring of an incoming text until the vehicle is stopped, and display only a simple user interface that can be understood with quick glances. For drivers who do not own a cell phone, they would be happy to receive lower insurance rates by accepting a system that proves to the insurance company that they never hold a cell phone while driving,
  • SUMMARY
  • In the prior art, insurance companies incentivize customers to install in their autos cameras that take images of drivers while they are driving and send the images to the insurance companies for human review. An improved device to do this and more, called a teleproctor, is described. Detection and reporting of when eyes are not on the road for longer than a glance is automated with image processing. The device also reports how much the car is driven, by whom, when, where, carrying what load, at what speed compared to the speed limit, with what pattern of hard breaking or hard cornering, whether the driver looked into the blind spot before changing lanes, and whether the driver allowed their phone to enter a restricted state while driving. Reports of selected events can be sent to parents or employers promptly upon occurrence. Importantly, the removable retrofit device can detect if it is depowered or removed from the vehicle and report this to the insurance company, rental company, parents, and employers.
  • Systems to provide the monitoring and reporting and screening of calls can be retrofit to existing vehicles at low cost. If there is sufficient demand for the desired features, auto manufacturers will pre-install the hardware components to run software loaded by insurance companies, rental companies, parents, and employers.
  • The teleproctor is installed in the vehicle by the insured or built into a new car. Installation for retrofit requires no skills. It is quick and easy and gives immediate feedback whether successful or unsuccessful. The teleproctor merely needs to be adhered to the dashboard or the windshield or the rear-view-mirror at a spot where the driver can see it and either a power wire is then plugged into a power source, such as a USB port or a cigarette lighter receptacle, or the teleproctor includes a photovoltaic panel that charges its battery. To verify correct set up, the teleproctor sends a message via a radio network to a server and, when it receives an acknowledgment, beeps and flashes a green light.
  • The teleproctor collects data and wirelessly sends it to a central repository from which data is provided to the insurance company and/or the car owner (rental company, parents, or employer). The teleproctor can be set to give auditory reports and/or visual reports to the driver whenever it reports to the insurance company that the driver was exceeding the speed limit by more than a threshold or had eyes looking away from the road for too long while the car is moving or other detected behaviors, bad or good.
  • To report its data, the teleproctor can include wireless network communications circuitry and a network account can be maintained for transmission, like with the General Motors Onstar system. Alternatively, the teleproctor can send the data to pre-identified mobile phone via Bluetooth or WiFi when that phone is in the vehicle and the phone runs an app that forwards the data in SMS messages or via internet to the data repository. A vehicle owner or insurance company can instruct the repository to forward selected data events immediately upon receipt.
  • The teleproctor provides data to answer these questions:
      • 1. How many minutes per week is the car driven? on which risky roads at what times?
      • 2. In each minute, what was the vehicle speed and what is the speed limit recorded in a map for that location?
      • 3. In each minute, was it driven by the primary listed driver? The number 2 listed driver? Number 3, etc.? A non-listed driver?
      • 4. In which minutes and for how long each time did the driver take his or her eyes off the road?
      • 5. Did the driver brake hard? corner hard? look in the blind spot before changing lanes?
      • 6. Did the driver prevent their phone from automatically entering a restricted mode while driving to reduce distractions?
      • 7. For each trip, what was the load added to the vehicle?
      • 8. Where is the car now, what are current images of the driver or driver's seat, and what are current sounds?
      • 9. Was the teleproctor removed from the vehicle or depowered?
  • Although software in the teleproctor is sophisticated, the hardware cost of each teleproctor is low. Most of the hardware is already present in many new vehicles, such as the GM Onstar system. The labor cost of retrofit installation is insignificant because each vehicle owner can do it themselves without expert guidance or review. The eye direction recognition software need not be so effective that it detects every too long glance away from the road. It only needs to detect enough of the too long glances away that the driver is deterred from risky behavior and there is no way the driver can consistently avoid detection.
  • Similarly, the most important part of the subterfuge detection system—detecting removal of the retrofit teleproctor from the vehicle—requires no extra hardware, merely sophisticated software. This component of the teleproctor can be a valuable component to add to any device with radio circuits where detection of removal is important.
  • DESCRIPTION OF FIGURES
  • FIG. 1 shows a passenger vehicle with a teleproctor adhered to the windshield.
  • FIG. 2 shows the circuit components of a teleproctor.
  • A. EYES AND FACIAL RECOGNITION
  • As shown in FIG. 1, the teleproctor 21 includes a camera 29 mounted on a base that includes electronic circuitry. The base need not be larger than a small mobile phone. The base may be installed in new vehicles built into the dashboard. The camera housing includes an infra-red light emitter that shines light not visible to humans toward the camera's field of view. The camera pixels detect both visible and infra-red light.
  • For retrofit, as shown in FIG. 1, the base may be adhered to the windshield 15 near the rear view mirror. Alternatively, it may be adhered to the dashboard or affixed to the mirror. For retrofit, it has a wire that leads to a plug for a USB port (5 volts) or to a 12 volt outlet (cigarette lighter socket) or it includes a solar photovoltaic panel that charges a battery.
  • The teleproctor includes an image processor circuit 28 which is programmed to use image recognition to determine:
      • (a) Is the camera pointed at a face recognized as a pre-listed driver?
      • (b) While the car is moving faster than a threshold, are the eyes looking away from the road too long for a glance in another direction?
      • (c) Did the head turn in a manner typical of checking the blind spot?
  • The teleproctor 21 is programmed to learn the necessary recognition details automatically after the teleproctor is installed. Facial image and eye position recognition methods are well known. If the camera aim is changed so that it is not pointed at a human face and the car is moving faster than a threshold, this fact is reported to the driver and to the owner and/or insurance company.
  • When the teleproctor is installed, a first listed driver sits in the driver's seat and adjusts the camera angle to point at his or her eyes. The driver clicks a button, then looks straight ahead at the road as if driving. Three seconds after the button was clicked, the teleproctor captures an image to save data characteristics of this face and these eyes as the first listed driver in correct driving position. The user interface for the teleproctor can be via a mobile phone or laptop or tablet computer with a Bluetooth or WiFi or similar connection. Data to recognize each other pre-listed driver—number 2, number 3, etc.—is also stored in this way the first time each of them drives. The teleproctor then determines when it is looking at one of these drivers and can report the driver's identity with little data transmitted.
  • If a face is not recognized when the vehicle is moving, the teleproctor reports this fact to the driver. The driver will be reported to the owner and/or insurance company as a not-pre-listed driver unless the system is commanded to store the new facial data as the image of a newly listed driver. The image of each face used to train the system is transmitted to the owner and/or insurance company and stored in their records for human review if and when necessary. If a person changes their facial appearance, by changing their glasses or other features, they simply reprogram the teleproctor to store their new facial data characteristics. The teleproctor can keep two or more stored sets of facial data for each driver, such as with dark glasses or clear glasses or with a hat, etc.
  • Instead of storing facial recognition data in the teleproctor, the entire original images may be uploaded to a central server where the facial recognition processing is done. Then, each time the vehicle starts moving from a stop long enough to change drivers, an image is taken and uploaded for facial recognition.
  • In addition to a facial recognition system for determining who is driving, the teleproctor includes components for eye direction determination, such as by identifying the pupils as contrasted against whites of the eyes and computing metrics of the pupils and the whites on each side of the pupils. Any method for making these computations may be used. When the driver slightly changes position while the vehicle is moving, the image processor may update the looking-at-the-road eye position data set for the driver by assuming that an eye position that is steady while the car is moving with only short glances in other directions should be the basis for updating the data set.
  • When the eye direction data changes significantly from the looking-at-the-road data set and then returns to the looking-at-the-road data set within a window of time, the system concludes that the eyes were not looking at the road. If the dark pupils move to the side and the eye metrics change a large amount, the head has turned, such as to check the blind spot.
  • For optimal processing for each of eye direction recognition and facial recognition, the brightness of the captured data in the visible and infra-red spectra may be adjusted independently. In addition, the intensity of emitted infra-red light may be adjusted independently for each data set, giving a total of three possible brightness adjustments for each of eye direction recognition and facial recognition purposes. For example, for eye direction determination when the driver is wearing dark glasses, the three brightnesses will be adjusted for optimal distinction between the pupils and whites of the eyes as seen through the dark glasses. These brightness adjustments will typically be too bright for recognition of other facial features.
  • B. COLLECTING ADDITIONAL DATA
  • Location, Speed, and Speed Limit.
  • The teleproctor includes a determiner of location, speed, and speed limit 27 which places into the data store 24 location data and speed relative to the local speed limit. For a built-in teleproctor, the location, speed, and speed limit determiner 27 may get this data from a GPS with map circuit in the vehicle. In the retrofit teleproctor, the location, speed, and speed limit determiner 27 includes a GPS circuit and a stored digital map with speed limits for each location. The map is periodically updated by download through the radio transceiver 30. The teleproctor saves in the data store 24 the location, speed, and speed limit data for reporting.
  • Acceleration.
  • The teleproctor includes at least two accelerometers. An acceleration data processor can determine when lane changes are made and the image recognition processor can determine whether the driver first looked in the blind spot. If the teleproctor is built into the vehicle, it can be coupled to data sources from the vehicle such as whether a turn signal was activated before changing lanes and braking or turning data. The acceleration data processor can determine when the driver causes hard braking or cornering or speeding up. A summary of these determinations can be uploaded to the data repository for review by the insurance company or owner.
  • Load.
  • Rental companies would like to know how much load their vehicle is carrying on each trip as a gauge of wear. This information can be used to set lower rates for people who carry light loads. Data from the accelerometers on hard acceleration, hard cornering and hard braking is also of interest to these companies because it increases tire wear. Parents and employers would like to know when the vehicle is being used to carry extra passengers.
  • A load computation can be implemented with data showing power consumption provided by engine sensors. The built-in teleproctor receives input from the vehicle engine computer. When the engine power output is high, either the vehicle is going up a hill or it is accelerating. Using the speed determining circuits, these two factors can be computationally separated. By comparing the power output to change in speed and factoring out any hill incline, the load the vehicle is carrying can be computed.
  • Another method to distinguish between climbing a hill and speeding up uses a direction of gravity+acceleration sensor. By comparing the output of this sensor to change in speed, these two factors can be separated.
  • A more direct method for obtaining load data to report is to add strain gauges to one or more suspension springs of the vehicle. It may be a tiny strain gauge that directly senses strain in the metal of the spring or it may be a distance sensor that senses the distance that the spring is collapsed overall. Placing the gauge on only one spring, a rear spring, may be sufficiently accurate. A wire is run from the strain gauge to the teleproctor or to the vehicle's computer which passes the data on to the teleproctor. Of course, the teleproctor circuits may be built into the vehicle's computer, with only the camera and other sensors being outside the vehicle computer.
  • C. REPORTING DRIVING BEHAVIOR TO THE DRIVER AND THE INSURANCE COMPANY AND/OR VEHICLE OWNER
  • Collected data for each trip is stored in a data store 24 shown in FIG. 1. Periodically, a radio control processor 23 commands the radio transceiver 30 to send all the new data in the data store 24 to a central repository 31 where it is repackaged and supplied to the insurance companies and/or car owner. The transmission route may go directly from a wireless network transmitter built into the teleproctor or the vehicle, in which case an account for the teleproctor is maintained in the radio network, or it may pass via Bluetooth or WiFi to a phone in the car which acts as the first link in a radio network 32.
  • Any programmable “smart” phone can be programmed with an “app” to perform this function automatically without the phone owner taking any action other than leaving the phone and the app turned on whenever the phone is in the vehicle. The phone automatically detects the Bluetooth or WiFi signal from the teleproctor and connects to it. The teleproctor and phone app together then determine how often a data bundle is uploaded through the phone. For example, if the phone is in an area with poor reception, the uploads may be delayed. This uploading of the data can be programmed to happen the first occasion after a designated time each day that one of the designated phones with Bluetooth or WiFi is in the car and the car is turned on, or other occasions. It can be programmed to happen immediately when certain events are detected or upon request from the system server 31.
  • The teleproctor can be programmed so that, when a specified number of events of not looking at the road for longer than a threshold occur within a specified window of time, a report is immediately sent via the radio network. Similarly, it can be programmed to immediately send a report when speed over the speed limit is maintained for longer than a threshold amount of time or any other event. If the teleproctor detects extreme deceleration suggestive of an accident, or upon request from the system server, it can transmit images from the teleproctor and sounds collected by a microphone in the teleproctor or the phone for a period of time.
  • So that the actions of the image processor and the radio control processor can be updated, they are each controlled by programs stored in a rewritable program store 25 which can be rewritten by download through the radio network 32 and loaded into the program store 25.
  • The teleproctor can upload data showing:
      • 1. the start and stop (longer than a threshold such as 4 minutes) times of all vehicle movement,
      • 2. coordinates at each start or stop time,
      • 3. coordinates at each turn relative to mapped roads to show each leg of the route taken on each trip,
      • 4. number of minutes in each leg,
      • 5. number of minutes in each leg that the vehicle speed exceeded the speed limit by more than a threshold,
      • 6. any hard breaking or cornering or extreme deceleration,
      • 7. amount of load (or estimated number of passengers) carried on each trip,
      • 8. number of times in each leg that the driver turned their eyes away from the “looking at the road” direction for more than a threshold duration and by how many tenths of seconds the threshold was exceeded, both a maximum and an average, and
      • 9. whether the driver prevented their phone from automatically entering restricted mode controlled by the teleproctor.
  • The teleproctor can be set to play an auditory message to the driver through a speaker in the teleproctor or the driver's phone and/or present a visual message to the driver in a display for the driver whenever speeding or eyes off the road or other events are reported to the insurance company. With each message to the driver, the teleproctor can tell the driver how much money the driver lost on insurance costs due to speeding or eyes off the road, etc.
  • Data elements 1-7 above can be reported without adding a teleproctor camera into the vehicle. Some vehicles and mobile telephones already have the required hardware. All that is required is software to be downloaded into them to work as follows. A processor receives vehicle speed from the vehicle, from either a speedometer or a GPS circuit in the vehicle. The processor receives a speed limit from the GPS and a stored map. The processor may be in the on-board vehicle electronics associated with the GPS or in a mobile telephone that communicates with the on-board vehicle electronics via Bluetooth or WiFi. The phone provides acceleration data. The mobile telephone runs an app that receives the data and forwards it to the remote parties.
  • D. LEVELS OF RESTRICTED FUNCTIONALITY OF DRIVERS' PHONES WHILE DRIVING
  • As described above, each phone in the vehicle running an appropriate app can automatically connect to the teleproctor for bi-directional communication when the phone is in the vehicle. By facial recognition, the teleproctor can know which listed driver is driving the vehicle. With this information and prior cooperation by the usual drivers of the vehicle to load apps in their phones and list a phone identifier with the teleproctor, the teleproctor can advise the driver's phone that it should enter a restricted mode when the vehicle is moving. The usual drivers' phones are each identified to the teleproctor when they first connect to it. Then, with the phone owner's acceptance by user input to load the app, the app will put the phone in a restricted mode whenever the teleproctor advises the phone that its owner is in the driver's seat and the vehicle is moving. If the driver's usual phone is not running the app to allow restriction, this suspicious circumstance can be reported to the vehicle owner or insurance company which can use this information to set rates or rules.
  • An optimal form of functionality restriction might block all text message alerts or display and all reading, writing, or other user interactions with a display while moving, but, to accommodate emergencies, it should not block all voice communications while moving. Examples of restrictions that can be implemented in the phone include:
      • 1. Defer until the vehicle stops the playing of text ring sounds;
      • 2. Block displays to read text or to receive text input at the keypad;
      • 3. Block display of typical web pages which require focused attention to glean information, allowing only simple displays;
      • 4. For incoming calls, play a message to the caller that the callee is driving and ask whether the call is important enough to put through; the called phone would ring only if the caller says “yes” or presses “1’.
      • 5. When the vehicle takes certain actions, such as acceleration or breaking or sharp turns, suspend voice calls and play an explanatory tone or message to the remote party.
      • 6. When the automobile's risk alerting forward looking radar senses a risk or the GPS circuit with map indicates that a zone of high risk (such as certain intersections) is being entered, the teleproctor instructs the phone to play an alert sound into any on-going telephone conversation or suspend the call so that both parties know the driver needs to pay attention.
  • In restricted mode, all screen displays are so simple that a viewer can glean all available information with just a glance—no text beyond a few words to read, no large sets of items to choose from.
  • Level of attention required for driving can be approximated with the accelerometers in the teleproctor. The teleproctor can advise the driver's phone to enter a more restricted state when the vehicle is speeding up or slowing down or turning or quickly changing lanes. If it is built into the vehicle, the teleproctor can receive inputs from various vehicle sensors that suggest when more attention is required for driving. At these times, the outgoing voice signal or the incoming voice signal or both can be interrupted or suspended (and perhaps replaced with a sound or recorded words) to reduce distraction for the driver and inform the other party to the conversation that the driver needs to pay attention to driving at this time.
  • There can be multiple levels of restriction based on time of day or location on a map in memory based on GPS location or other factors. A restriction might be to stop the handset speaker from working and require use of a headset speaker.
  • Conversations with a passenger in a car present less of a distraction than telephone conversations because the remote conversant does not know what else the driver is doing simultaneously and what the driver is seeing, while the passenger does. The passenger understands what is happening when the driver pauses to concentrate on driving. The concentration demands for a telephone conversation can be reduced and made more like the demands for conversation with a passenger by periodically playing a beep or other signal to other party to remind them that the person they are talking to is also driving.
  • The app running in the driver's phone can be programmed so that, when a caller calls while the phone is in a restricted state, before the phone rings, a message is played to the caller saying: “The person you are calling is driving. If this call is important enough to interrupt that person while driving, press 1 or say yes.” The called phone would only ring if the caller presses 1 or says yes. Otherwise, the call is rerouted to voice mail.
  • As described above, for independent adult drivers, restricting phones that are moving to block texting and display interactions but only temporarily block voice communications while also accelerating may be optimal. However, for children, or children and certain employees, the parent or employer may prefer a greater level of restriction. In this system, the phone owner can have a password that allows greater restriction to be placed on the phone. Here are examples of possible voice call restrictions that may be preferred in these situations for outgoing calls.
      • (1) Only calls to 911 or any other listed emergency number are allowed.
      • (2) Phone numbers of family members are listed with the app and only calls to these numbers are put through.
      • (3) Speed dial calls are allowed but not calls requiring pressing more than 3 keys.
      • (4) Voice calls to other than listed emergency numbers are cut off after 3 minutes.
  • Here are examples of possible restrictions for incoming calls.
      • (5) The likely source phone numbers of family members are listed with the app and only calls from these numbers are put through.
      • (6) Voice calls from those listed family member numbers are cut off after 3 minutes.
    E. AVOIDING POSSIBLE SUBTERFUGE
  • For the system to be valuable to insurance companies and vehicle rental companies, there must be no way to defeat it that would keep low insurance rates or rental rates in place when they should not be kept in place. Parents and employers have similar concerns. Subterfuges that might be attempted include:
  • 1. Unplug or remove the teleproctor for some vehicle trips. The teleproctor includes a rechargeable battery that keeps it working for a length of time after it is depowered. In versions that do not include photovoltaic charging of the battery, when power to the teleproctor is lost, to determine whether the loss is due to a proper switch off of vehicle power or improper unplugging or removal of a power fuse, the teleproctor sends a radio ping out the power wire and measures the radio echo signature by a process described below.
  • If the echo signature shows that power was lost due to vehicle power switch off, the teleproctor takes no action other than going into its normal sleep mode. If the echo signature shows that the teleproctor is unplugged or a fuse is removed, whether accidently or otherwise, it plays an auditory alert warning to plug it back in. If it is not soon plugged in to the same vehicle as determined by the echo signature methods described below, the teleproctor reports this fact to the insurance company and/or owner.
  • To determine the cause of depowering, the teleproctor includes a circuit that captures and records a radio-echo signature of the power source and its connections. Using its radio transmitter, which it has for Bluetooth or WiFi communication with the phone or for wide area data network communication more directly to a server, the teleproctor emits a ping or series of pings into the power wire. Using its radio receiver, the teleproctor records a signature of the echo to obtain a radio echo signature of the vehicle electronics.
  • The power wire echo will be different if the power source is a 12 volt to 5 volt converter such as for USB receptacles rather than a 12 volt source such as from a cigarette lighter. It may also be affected by plugging other electronic devices into the vehicle. These changes can be distinguished as not material changes because the echo will be vastly different if the teleproctor is left unplugged or a fuse is removed or blown.
  • With known signature processing techniques, the echo data set may be reduced to a small but distinctive data set known as a signature hash. The algorithm for computing the hash is developed by collecting full signature data sets for many different vehicles and power supplies and developing the algorithm to make the hash data set optimally small while still distinguishing between the plugged and unplugged or no fuse conditions in all cases, no matter what else may be plugged in.
  • Every time the retrofit teleproctor is powered, whether from turning on vehicle power or from unplugging and then plugging back in, a total vehicle radio echo signature is obtained. For this, the radio echo signature is taken using the teleproctor's antenna for both broadcast and reception to obtain a signature of the entire vehicle, with a major factor being the shape of and relationship to all metal in the vehicle and a very minor factor being a change in the configuration of what's plugged into the vehicle's power and where. This allows users to unplug the retrofit teleproctor, make changes to what's plugged in and where, and then plug in the retrofit teleproctor again without a problem. If the total vehicle radio echo signature hash is the same as before, no alert is sent to the insurance company. For the built in teleproctor, no total vehicle radio echo signature capability is required.
  • The algorithm for computing the hash for the vehicle radio echo signature is developed by collecting full signature data sets for many different vehicles and developing the algorithm to make the hash data set optimally small while still distinguishing between as many vehicle models as possible and never computing two different hashes for the same vehicle.
  • A driver might, while the vehicle power is off, unplug the retrofit teleproctor, leave it unplugged for some trips, and then plug it back into the same vehicle. To prevent this subterfuge, the back-up battery has enough power to last several days and, while depowered in sleep mode, the teleproctor wakes up periodically, about every 3 hours, and queries the GPS to determine its location. If the location is not the same as where the teleproctor went to sleep, the insurance company and/or owner is notified.
  • A driver might, while the vehicle power is off, unplug the retrofit teleproctor and remove it from the vehicle, leaving it in the same location as where the teleproctor went to sleep while the vehicle drives away. To prevent this subterfuge, the back-up battery has enough power to last several days and, while depowered in sleep mode, the teleproctor wakes up periodically, about every 3 hours, and takes a total vehicle radio echo to determine the radio reflection characteristics of its surroundings and computes a hash. If the computed hash is not the same as for the vehicle in which the teleproctor was installed, the insurance company and/or owner is notified.
  • For new cars with a built in teleproctor, the total vehicle echo signature never needs to be taken because the teleproctor cannot easily be removed. For retrofit teleproctors, the original total vehicle echo signature hashes are computed when the teleproctor is initially set up.
  • 2. Point the camera at a manikin or mask or image. The teleproctor is programmed to report that the data is likely wrong if the eye direction data does not change with glances away from the road as much as is minimally human.
  • 3. Point the camera at a passenger. Unless the passenger is looking at the road as much as the driver should be, this will not achieve the driver's objective—the glances away from the road by a passenger will be too long. If the driver points the camera at a passenger who is directed to stare intently at the road ahead, the face will not be recognized as a covered driver unless the driver attempted to defraud the insurance company by claiming that the face on which the system was trained is the driver's face, but this would be easily caught by a human looking at the photo of the face that was uploaded when the system was trained on that face.
  • Also, if the camera is not moved from its location in front of the driver, the angle of looking at the passenger will be too oblique and will be automatically detectable by the image processor. If the camera is moved from its proper location, the echo signature of the total vehicle will be detectably different.
  • 4. Install a bootlegged teleproctor. It is necessary to include a method for the data compiling system 31 to authenticate each teleproctor. Otherwise, sociopathic entrepreneurs could make and sell bootlegged teleproctors that will give false and prettied-up data on use of the vehicle. The authentication method need not be strong. It only needs to be strong enough to make it expensive to make and sell an unauthorized teleproctor that will work adequately to go undetected more than about half of the time.
  • Crypto-authenticating chips are well known. They are designed to receive as input a first data string and, in response, output a second data string. When both data strings are processed together with a secret key, the result will show the second data string to have been produced by the crypto-authenticating chip or not. The second string cannot be inexpensively determined from knowing the first string without having the key. The contents of the authenticating chip cannot be read and duplicated because a fusible link was burned after manufacture.
  • The data compilation server 31 sends a first data string to the teleproctor mounted in the vehicle which replies with a second data string from a crypto-authentication circuit 22 included in the teleproctor. If the data compilation server determines that the strings do not match with the key, the data is so annotated.
  • Each teleproctor sends to the data compilation server 31 an identifier for use to determine which key should be used. The identifier may be the teleproctor's mobile number in the network or it may be a separate identifier stored in and sent by the authentication circuit 22 (e.g., a serial number).
  • An alternative design for an authentication circuit uses a clock in the teleproctor that is synchronized with a clock in the central server to periodically change a data string that it sends to the server. Each new data string must match what is expected to achieve authentication. Other authentication circuits are known and may be used.
  • Although teleproctors require approved authentication circuits 22 to be built in by licensed manufacturers to avoid bootlegging of devices that report false information, the authentic teleproctors can be sold freely though all channels of trade and installed by anyone.
  • 5. Load a bootlegged program into the teleproctor. The server that checks for the authentication chip is the only component that is permitted to load an updated program into the teleproctor. The authentication chip is programmed to also check the credentials of the server and insure that no other source can supply an updated program.
  • 6. Block the driver's phone from connecting to the teleproctor. The image recognition system identifies who is driving. If that person's listed phone is turned off or not running the app such that the teleproctor cannot connect to it, this is reported, with adverse consequences for the driver.
  • 7. Carry a non-listed phone as well as a listed phone. The image recognition system identifies who is driving. If that person's listed phone is in the vehicle and running the app, the phone will enter a restricted state as appropriate. If the driver uses a second, non-listed phone, it will not be in a restricted state, but the eye direction monitoring system will still be watching the eyes and reporting too much eyes-off-the-road behavior. Whenever the eyes are off the road too long, the system can upload an image of the driver's face and upper body for possible human or automated review.

Claims (11)

1. A system for use in a vehicle to reduce traffic accidents, comprising:
(a) a forward-looking radar detector of objects in front of the vehicle; coupled to
(b) a processor that, when an object that presents a risk of accident is detected by the radar detector, sends an alert signal to a telephone of a driver of the vehicle.
2. The system of claim 1 wherein, when the alert signal is received by the telephone of the driver, the telephone plays an alert sound.
3. The system of claim 1 wherein, when the alert signal is received by the telephone of the driver, the telephone suspends on-going voice communications.
4. The system of claim 2 wherein the alert signal is sent to the telephone via Bluetooth radio communications.
5. The system of claim 1 further comprising a speed determining circuit, coupled to the processor, which speed determining circuit provides input to the processor which the processor uses to assess risk.
6. The system of claim 5 wherein the speed determining circuit receives speed data input from one or more sensors in the vehicle that detect speed and report detected speed to the system.
7. The system of claim 1 further comprising a radio communications circuit coupled to the processor which radio communications circuit sends information to a remote server.
8. The system of claim 7 further comprising an authenticating circuit coupled to the radio communications circuit which authenticating circuit authenticates the system via a radio network to a remote server coupled to the radio network.
9. The system of claim 1 where the components are assembled into a device adapted for retrofit into a pre-existing automobile.
10. The system of claim 1 where the components are built into a new vehicle as the vehicle is constructed.
11. The system of claim 1 further comprising a rechargeable battery power supply coupled to a solar cell that charges the battery.
US16/255,767 2012-08-24 2019-01-23 Forward-looking radar signals alert via driver's phone Abandoned US20190156109A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/255,767 US20190156109A1 (en) 2012-08-24 2019-01-23 Forward-looking radar signals alert via driver's phone

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201261693079P 2012-08-24 2012-08-24
US201261728763P 2012-11-20 2012-11-20
US13/975,246 US9311544B2 (en) 2012-08-24 2013-08-23 Teleproctor reports use of a vehicle and restricts functions of drivers phone
US16/255,767 US20190156109A1 (en) 2012-08-24 2019-01-23 Forward-looking radar signals alert via driver's phone

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US13/975,246 Continuation US9311544B2 (en) 2012-08-24 2013-08-23 Teleproctor reports use of a vehicle and restricts functions of drivers phone

Publications (1)

Publication Number Publication Date
US20190156109A1 true US20190156109A1 (en) 2019-05-23

Family

ID=52480001

Family Applications (3)

Application Number Title Priority Date Filing Date
US13/975,246 Active 2033-11-21 US9311544B2 (en) 2012-08-24 2013-08-23 Teleproctor reports use of a vehicle and restricts functions of drivers phone
US15/094,672 Active 2034-04-02 US10204261B2 (en) 2012-08-24 2016-04-08 Camera in vehicle reports identity of driver
US16/255,767 Abandoned US20190156109A1 (en) 2012-08-24 2019-01-23 Forward-looking radar signals alert via driver's phone

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US13/975,246 Active 2033-11-21 US9311544B2 (en) 2012-08-24 2013-08-23 Teleproctor reports use of a vehicle and restricts functions of drivers phone
US15/094,672 Active 2034-04-02 US10204261B2 (en) 2012-08-24 2016-04-08 Camera in vehicle reports identity of driver

Country Status (1)

Country Link
US (3) US9311544B2 (en)

Families Citing this family (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9311544B2 (en) * 2012-08-24 2016-04-12 Jeffrey T Haley Teleproctor reports use of a vehicle and restricts functions of drivers phone
US20150077548A1 (en) * 2013-09-13 2015-03-19 At&T Mobility Ii Llc Mobile device intermediary for content analysis
US9596357B1 (en) * 2014-10-01 2017-03-14 Netzer Ruperto Phone activity tracking device
GB201420988D0 (en) * 2014-11-26 2015-01-07 Tomtom Telematics Bv Apparatus and method for vehicle economy improvement
CN107341866A (en) * 2015-04-17 2017-11-10 吴小再 Face, the drive recorder of eye recognition are suitable to based on Internet of Things
US10257344B2 (en) 2016-04-20 2019-04-09 Stephen Rhyne System, device, and method for tracking and monitoring mobile phone usage while operating a vehicle in order to deter and prevent such usage
US9996756B2 (en) * 2015-08-31 2018-06-12 Lytx, Inc. Detecting risky driving with machine vision
US9940530B2 (en) * 2015-12-29 2018-04-10 Thunder Power New Energy Vehicle Development Company Limited Platform for acquiring driver behavior data
US10009455B2 (en) 2016-04-20 2018-06-26 Stephen Rhyne System, device, and method for tracking and monitoring mobile phone usage to deter and prevent such usage and for generating an audible alarm and/or visual alarm to maintain compliance
CN107306384B (en) * 2016-04-22 2020-11-10 宇龙计算机通信科技(深圳)有限公司 Vehicle loss processing method and device and server
CN106004735B (en) * 2016-06-27 2019-03-15 京东方科技集团股份有限公司 The method of adjustment of onboard system and vehicle service
JP2019530061A (en) * 2016-08-10 2019-10-17 ジーボ インコーポレーテッドXevo Inc. Method and apparatus for providing information through metadata collected and stored using a speculative attention model
US10351143B2 (en) * 2016-09-13 2019-07-16 Ford Global Technologies, Llc Vehicle-based mobile device usage monitoring with a cell phone usage sensor
KR102414041B1 (en) * 2016-10-06 2022-06-29 젠텍스 코포레이션 Rearview mirror assembly with occupant detection
US20180259958A1 (en) * 2017-03-09 2018-09-13 Uber Technologies, Inc. Personalized content creation for autonomous vehicle rides
US10893136B2 (en) * 2017-08-02 2021-01-12 Boris Stricevic System and method for controlling in-vehicle electronic devices
DE102017215559A1 (en) * 2017-09-05 2019-03-07 Robert Bosch Gmbh Method and device for risk assessment of a journey by road vehicle
US10572745B2 (en) * 2017-11-11 2020-02-25 Bendix Commercial Vehicle Systems Llc System and methods of monitoring driver behavior for vehicular fleet management in a fleet of vehicles using driver-facing imaging device
US10339401B2 (en) 2017-11-11 2019-07-02 Bendix Commercial Vehicle Systems Llc System and methods of monitoring driver behavior for vehicular fleet management in a fleet of vehicles using driver-facing imaging device
US10600234B2 (en) 2017-12-18 2020-03-24 Ford Global Technologies, Llc Inter-vehicle cooperation for vehicle self imaging
US10417911B2 (en) * 2017-12-18 2019-09-17 Ford Global Technologies, Llc Inter-vehicle cooperation for physical exterior damage detection
US10745005B2 (en) 2018-01-24 2020-08-18 Ford Global Technologies, Llc Inter-vehicle cooperation for vehicle self height estimation
EP3540710A1 (en) * 2018-03-14 2019-09-18 Honda Research Institute Europe GmbH Method for assisting operation of an ego-vehicle, method for assisting other traffic participants and corresponding assistance systems and vehicles
CN108657186B (en) * 2018-05-08 2020-09-04 奇瑞汽车股份有限公司 Intelligent cockpit interaction method and device
US10628690B2 (en) 2018-05-09 2020-04-21 Ford Global Technologies, Llc Systems and methods for automated detection of trailer properties
WO2019244083A1 (en) * 2018-06-20 2019-12-26 Gentex Corporation Driver identification and identification systems and methods
CN109050246A (en) * 2018-07-12 2018-12-21 武汉仁山智水科技服务有限公司 A kind of method and apparatus of giving fatigue pre-warning
CN108891416B (en) * 2018-07-12 2020-05-01 浙江黄岩黎明实业有限公司 Vehicle safety driving system
CN109727337A (en) * 2018-12-05 2019-05-07 上海博泰悦臻电子设备制造有限公司 Vehicle device, accident record method and system
CN109783657B (en) * 2019-01-07 2022-12-30 北京大学深圳研究生院 Multi-step self-attention cross-media retrieval method and system based on limited text space
US11351917B2 (en) 2019-02-13 2022-06-07 Ford Global Technologies, Llc Vehicle-rendering generation for vehicle display based on short-range communication
US11377114B2 (en) * 2019-03-14 2022-07-05 GM Global Technology Operations LLC Configuration of in-vehicle entertainment based on driver attention
CN110329268B (en) * 2019-03-22 2021-04-06 中国人民财产保险股份有限公司 Driving behavior data processing method, device, storage medium and system
JP7286368B2 (en) * 2019-03-27 2023-06-05 本田技研工業株式会社 VEHICLE DEVICE CONTROL DEVICE, VEHICLE DEVICE CONTROL METHOD, AND PROGRAM
CN112622916A (en) * 2019-10-08 2021-04-09 株式会社斯巴鲁 Driving assistance system for vehicle
CN110861645B (en) * 2019-12-05 2021-02-26 中国汽车工程研究院股份有限公司 Intelligent network-connected automobile driver lane change system and working method thereof
JP7372230B2 (en) * 2020-12-21 2023-10-31 トヨタ自動車株式会社 Display system and display device
CN114475243B (en) * 2022-01-05 2023-06-09 东风柳州汽车有限公司 Control method for preventing vehicle from mistakenly stepping on accelerator pedal and vehicle

Citations (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110077028A1 (en) * 2009-09-29 2011-03-31 Wilkes Iii Samuel M System and Method for Integrating Smartphone Technology Into a Safety Management Platform to Improve Driver Safety
US20110105082A1 (en) * 2009-11-04 2011-05-05 Jeff Haley Exempt from automatic restriction of functionality moving phones accompanied by an override transmitter
US20110169626A1 (en) * 2010-01-13 2011-07-14 Denso International America, Inc. Hand-held device integration for automobile safety
US20110195699A1 (en) * 2009-10-31 2011-08-11 Saied Tadayon Controlling Mobile Device Functions
US20120176235A1 (en) * 2011-01-11 2012-07-12 International Business Machines Corporation Mobile computing device emergency warning system and method
US20130321179A1 (en) * 2010-12-27 2013-12-05 Mario Donato Santucci Real-Time Drive Assistance System And Method
US20130342337A1 (en) * 2012-06-22 2013-12-26 GM Global Technology Operations LLC Alert systems and methods for a vehicle
US20140019167A1 (en) * 2012-07-16 2014-01-16 Shuli Cheng Method and Apparatus for Determining Insurance Risk Based on Monitoring Driver's Eyes and Head
US20140032094A1 (en) * 2010-11-12 2014-01-30 Lucas Automotive Gmbh Method for Detecting Critical Driving Situations of Lorries or Passenger Vehicles and Method for Avoiding Collisions
US20140172467A1 (en) * 2012-12-17 2014-06-19 State Farm Mutual Automobile Insurance Company System and method to adjust insurance rate based on real-time data about potential vehicle operator impairment
US20140180563A1 (en) * 2012-12-21 2014-06-26 Sascha Simon System and method for smartphone communication during vehicle mode
US20140203926A1 (en) * 2011-10-07 2014-07-24 Ford Global Technologies, Llc A system and method to mask incoming calls for a communication device connected to an automotive telematics system
US20150019266A1 (en) * 2013-07-15 2015-01-15 Advanced Insurance Products & Services, Inc. Risk assessment using portable devices
US8941510B2 (en) * 2010-11-24 2015-01-27 Bcs Business Consulting Services Pte Ltd Hazard warning system for vehicles
US20150042491A1 (en) * 2010-11-24 2015-02-12 Bcs Business Consulting Services Pte Ltd Hazard warning system for vehicles
US20150054934A1 (en) * 2012-08-24 2015-02-26 Jeffrey T. Haley Teleproctor reports use of a vehicle and restricts functions of drivers phone
US20150151689A1 (en) * 2012-07-20 2015-06-04 Denso Corporation Vehicular video control apparatus
US9233642B2 (en) * 2014-01-10 2016-01-12 Industrial Technology Research Institute Vehicle collision warning apparatus and method
US9676328B2 (en) * 2013-05-29 2017-06-13 Robert Bosch Gmbh Method and device for detecting a collision between a vehicle and an object using a mobile user terminal which is able to be coupled to the vehicle
US20170332347A1 (en) * 2016-05-11 2017-11-16 International Business Machines Corporation Preventing driver distraction from incoming notifications
US20190005816A1 (en) * 2013-12-20 2019-01-03 Sfara, Inc. System and method for smartphone communication during vehicle mode

Family Cites Families (48)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3934845B2 (en) * 2000-03-02 2007-06-20 株式会社東芝 Mobile communication terminal
US7110570B1 (en) * 2000-07-21 2006-09-19 Trw Inc. Application of human facial features recognition to automobile security and convenience
US7439847B2 (en) * 2002-08-23 2008-10-21 John C. Pederson Intelligent observation and identification database system
JP4209303B2 (en) * 2003-10-17 2009-01-14 アルパイン株式会社 Vehicle face image processing apparatus
JPWO2005069675A1 (en) * 2004-01-20 2007-09-06 オムロン株式会社 Telephone countermeasure device and telephone countermeasure method when using telephone while driving
DE102004044771B4 (en) * 2004-09-16 2006-08-24 Bayerische Motoren Werke Ag Method for image-based driver identification in a motor vehicle
CA2587740A1 (en) * 2004-11-03 2006-05-11 Thomas Dewaal Method system, and apparatus for monitoring vehicle operation
WO2006131926A2 (en) * 2005-06-09 2006-12-14 Drive Diagnostics Ltd. System and method for displaying a driving profile
EP2428413B1 (en) * 2005-07-11 2013-03-27 Volvo Technology Corporation Methods and arrangement for performing driver identity verification
US8549318B2 (en) * 2006-02-13 2013-10-01 Affirmed Technologies, Llc Method and system for preventing unauthorized use of a vehicle by an operator of the vehicle
KR100778059B1 (en) * 2007-05-22 2007-11-21 (주)텔릭스타 Apparatus and system blowing out dozing off motorways using facial recognition technology
KR100778060B1 (en) * 2007-06-01 2007-11-21 (주)텔릭스타 Vehicle emergency preventive terminal device and internet system using facial recognition technology
JP5030063B2 (en) * 2007-10-05 2012-09-19 本田技研工業株式会社 Navigation device and navigation system
KR100921092B1 (en) * 2008-07-04 2009-10-08 현대자동차주식회사 Driver state monitorring system using a camera on a steering wheel
US8854199B2 (en) * 2009-01-26 2014-10-07 Lytx, Inc. Driver risk assessment system and method employing automated driver log
US8532574B2 (en) * 2009-08-05 2013-09-10 Honda Motor Co., Ltd. Destination information sharing for the automobile environment
TWI405134B (en) * 2009-10-21 2013-08-11 Automotive Res & Testing Ct Driver face image recognition system
US20140207309A1 (en) * 2010-05-11 2014-07-24 Cartasite, Inc. Vehicle driver determination and behavior monitoring
US11070661B2 (en) * 2010-09-21 2021-07-20 Cellepathy Inc. Restricting mobile device usage
KR101172227B1 (en) * 2010-11-18 2012-08-07 현대자동차주식회사 System of management for entrance and exit using driver's face certification within vehicle and thereof method
US20120215403A1 (en) * 2011-02-20 2012-08-23 General Motors Llc Method of monitoring a vehicle driver
US8606492B1 (en) * 2011-08-31 2013-12-10 Drivecam, Inc. Driver log generation
US9235750B1 (en) * 2011-09-16 2016-01-12 Lytx, Inc. Using passive driver identification and other input for providing real-time alerts or actions
US8744642B2 (en) * 2011-09-16 2014-06-03 Lytx, Inc. Driver identification based on face data
US20140309813A1 (en) * 2013-04-15 2014-10-16 Flextronics Ap, Llc Guest vehicle user reporting
CN103324904A (en) * 2012-03-20 2013-09-25 凹凸电子(武汉)有限公司 Face recognition system and method thereof
DE102012106522A1 (en) * 2012-07-18 2014-01-23 Huf Hülsbeck & Fürst Gmbh & Co. Kg Method for authenticating a driver in a motor vehicle
US9471838B2 (en) * 2012-09-05 2016-10-18 Motorola Solutions, Inc. Method, apparatus and system for performing facial recognition
JP6110396B2 (en) * 2012-10-05 2017-04-05 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカPanasonic Intellectual Property Corporation of America Sleepiness estimation device, sleepiness estimation method, computer-readable non-transitory recording medium
EP2908726B1 (en) * 2012-10-19 2022-12-21 Veoneer Sweden AB Method and device for detecting decreased attentiveness of vehicle driver
WO2014128273A1 (en) * 2013-02-21 2014-08-28 Iee International Electronics & Engineering S.A. Imaging device based occupant monitoring system supporting multiple functions
US20140347458A1 (en) * 2013-05-23 2014-11-27 Ford Global Technologies, Llc Cellular phone camera for driver state estimation
US20140379385A1 (en) * 2013-06-24 2014-12-25 Elwha, Llc System and method for monitoring driving to determine an insurance property
US20150009010A1 (en) * 2013-07-03 2015-01-08 Magna Electronics Inc. Vehicle vision system with driver detection
EP2892020A1 (en) * 2014-01-06 2015-07-08 Harman International Industries, Incorporated Continuous identity monitoring for classifying driving data for driving performance analysis
US9428054B2 (en) * 2014-04-04 2016-08-30 Here Global B.V. Method and apparatus for identifying a driver based on sensor information
US9135803B1 (en) * 2014-04-17 2015-09-15 State Farm Mutual Automobile Insurance Company Advanced vehicle operator intelligence system
US9283847B2 (en) * 2014-05-05 2016-03-15 State Farm Mutual Automobile Insurance Company System and method to monitor and alert vehicle operator of impairment
US9158962B1 (en) * 2014-05-07 2015-10-13 Lytx, Inc. Passive driver identification
US9440603B2 (en) * 2014-06-09 2016-09-13 Atieva, Inc. Location sensitive learning interface
US9428195B1 (en) * 2014-07-24 2016-08-30 Lytx, Inc. Back-end event risk assessment with historical coaching profiles
US9533687B2 (en) * 2014-12-30 2017-01-03 Tk Holdings Inc. Occupant monitoring systems and methods
US9646433B1 (en) * 2015-09-23 2017-05-09 State Farm Mutual Automobile Insurance Company Systems and methods for using image data to generate vehicle operation logs
WO2017123226A1 (en) * 2016-01-14 2017-07-20 Ford Global Technologies, Llc Driver-identification system and method
US9707911B1 (en) * 2016-03-21 2017-07-18 Ford Global Technologies, Llc Identifying a driver of a vehicle
WO2018009567A1 (en) * 2016-07-05 2018-01-11 Nauto Global Limited System and method for automatic driver identification
DE102016215297A1 (en) * 2016-08-17 2018-02-22 Robert Bosch Gmbh Method for identifying a driver change in a motor vehicle
US20180068392A1 (en) * 2016-09-02 2018-03-08 Allstate Insurance Company Generating an Output Based on Processed Sensor Data

Patent Citations (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110077028A1 (en) * 2009-09-29 2011-03-31 Wilkes Iii Samuel M System and Method for Integrating Smartphone Technology Into a Safety Management Platform to Improve Driver Safety
US20110195699A1 (en) * 2009-10-31 2011-08-11 Saied Tadayon Controlling Mobile Device Functions
US9769663B2 (en) * 2009-11-04 2017-09-19 Jeffrey Haley Before completing a call to a moving driver, query the caller
US20180007554A1 (en) * 2009-11-04 2018-01-04 Jeffrey Haley Modify voice calls with a moving driver to reduce distractions
US20150341493A1 (en) * 2009-11-04 2015-11-26 Jeffrey Haley Before completing a call to a moving driver, query the caller
US8442490B2 (en) * 2009-11-04 2013-05-14 Jeffrey T. Haley Modify function of driver's phone during acceleration or braking
US20130250083A1 (en) * 2009-11-04 2013-09-26 Jeffrey T. Haley System to monitor vehicle drivers with a camera
US9100824B2 (en) * 2009-11-04 2015-08-04 Jeffrey T Haley System to monitor vehicle drivers with a camera
US20110105082A1 (en) * 2009-11-04 2011-05-05 Jeff Haley Exempt from automatic restriction of functionality moving phones accompanied by an override transmitter
US20110169626A1 (en) * 2010-01-13 2011-07-14 Denso International America, Inc. Hand-held device integration for automobile safety
US8669857B2 (en) * 2010-01-13 2014-03-11 Denso International America, Inc. Hand-held device integration for automobile safety
US20140032094A1 (en) * 2010-11-12 2014-01-30 Lucas Automotive Gmbh Method for Detecting Critical Driving Situations of Lorries or Passenger Vehicles and Method for Avoiding Collisions
US8941510B2 (en) * 2010-11-24 2015-01-27 Bcs Business Consulting Services Pte Ltd Hazard warning system for vehicles
US20150042491A1 (en) * 2010-11-24 2015-02-12 Bcs Business Consulting Services Pte Ltd Hazard warning system for vehicles
US20130321179A1 (en) * 2010-12-27 2013-12-05 Mario Donato Santucci Real-Time Drive Assistance System And Method
US20120176235A1 (en) * 2011-01-11 2012-07-12 International Business Machines Corporation Mobile computing device emergency warning system and method
US20140203926A1 (en) * 2011-10-07 2014-07-24 Ford Global Technologies, Llc A system and method to mask incoming calls for a communication device connected to an automotive telematics system
US9150154B2 (en) * 2011-10-07 2015-10-06 Ford Global Technologies, Llc System and method to mask incoming calls for a communication device connected to an automotive telematics system
US20130342337A1 (en) * 2012-06-22 2013-12-26 GM Global Technology Operations LLC Alert systems and methods for a vehicle
US20140019167A1 (en) * 2012-07-16 2014-01-16 Shuli Cheng Method and Apparatus for Determining Insurance Risk Based on Monitoring Driver's Eyes and Head
US20150151689A1 (en) * 2012-07-20 2015-06-04 Denso Corporation Vehicular video control apparatus
US20150054934A1 (en) * 2012-08-24 2015-02-26 Jeffrey T. Haley Teleproctor reports use of a vehicle and restricts functions of drivers phone
US10204261B2 (en) * 2012-08-24 2019-02-12 Jeffrey T Haley Camera in vehicle reports identity of driver
US9311544B2 (en) * 2012-08-24 2016-04-12 Jeffrey T Haley Teleproctor reports use of a vehicle and restricts functions of drivers phone
US20160224827A1 (en) * 2012-08-24 2016-08-04 Jeffrey T. Haley Camera in vehicle reports identity of driver
US20140172467A1 (en) * 2012-12-17 2014-06-19 State Farm Mutual Automobile Insurance Company System and method to adjust insurance rate based on real-time data about potential vehicle operator impairment
US20140180563A1 (en) * 2012-12-21 2014-06-26 Sascha Simon System and method for smartphone communication during vehicle mode
US9676328B2 (en) * 2013-05-29 2017-06-13 Robert Bosch Gmbh Method and device for detecting a collision between a vehicle and an object using a mobile user terminal which is able to be coupled to the vehicle
US20150019266A1 (en) * 2013-07-15 2015-01-15 Advanced Insurance Products & Services, Inc. Risk assessment using portable devices
US20190005816A1 (en) * 2013-12-20 2019-01-03 Sfara, Inc. System and method for smartphone communication during vehicle mode
US9233642B2 (en) * 2014-01-10 2016-01-12 Industrial Technology Research Institute Vehicle collision warning apparatus and method
US20170332347A1 (en) * 2016-05-11 2017-11-16 International Business Machines Corporation Preventing driver distraction from incoming notifications

Also Published As

Publication number Publication date
US20150054934A1 (en) 2015-02-26
US9311544B2 (en) 2016-04-12
US10204261B2 (en) 2019-02-12
US20160224827A1 (en) 2016-08-04

Similar Documents

Publication Publication Date Title
US20190156109A1 (en) Forward-looking radar signals alert via driver's phone
US11366708B2 (en) Managing functions on an iOS mobile device using ANCS notifications
US8914014B2 (en) Phone that prevents concurrent texting and driving
US11074769B2 (en) Safety for vehicle users
US8538402B2 (en) Phone that prevents texting while driving
US20190349470A1 (en) Mobile device context aware determinations
US9638537B2 (en) Interface selection in navigation guidance systems
US20190082047A1 (en) Device context determination
US20170302785A1 (en) Device context determination in transportation and other scenarios
US9175967B2 (en) Navigation instructions
US9100824B2 (en) System to monitor vehicle drivers with a camera
US20170279957A1 (en) Transportation-related mobile device context inferences
AU2014377550B2 (en) Vehicle control system
US20180026669A1 (en) Phone docking station for enhanced driving safety
CN107623773B (en) Mobile terminal, game program and game method executed on mobile terminal
US10708700B1 (en) Vehicle external speaker system
US11044566B2 (en) Vehicle external speaker system
US10348885B2 (en) Method and apparatus for limiting portable device functionality
CN108090992A (en) Alarming device

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION