WO2016183241A1 - Detecting objects within a vehicle in connection with a service - Google Patents

Detecting objects within a vehicle in connection with a service Download PDF

Info

Publication number
WO2016183241A1
WO2016183241A1 PCT/US2016/031929 US2016031929W WO2016183241A1 WO 2016183241 A1 WO2016183241 A1 WO 2016183241A1 US 2016031929 W US2016031929 W US 2016031929W WO 2016183241 A1 WO2016183241 A1 WO 2016183241A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
service
location
sensors
user
Prior art date
Application number
PCT/US2016/031929
Other languages
French (fr)
Inventor
David McAllister BRADLEY
Jean-Sebastien VALOIS
Original Assignee
Uber Technologies, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Uber Technologies, Inc. filed Critical Uber Technologies, Inc.
Priority to CA2985619A priority Critical patent/CA2985619A1/en
Priority to EP16793475.1A priority patent/EP3295421A4/en
Priority to KR1020177035479A priority patent/KR102395766B1/en
Priority to CN201680040538.2A priority patent/CN107873098A/en
Priority to AU2016262526A priority patent/AU2016262526B9/en
Priority to RU2017142642A priority patent/RU2725681C9/en
Publication of WO2016183241A1 publication Critical patent/WO2016183241A1/en
Priority to IL255562A priority patent/IL255562B/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/08Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/40Business processes related to the transportation industry
    • EFIXED CONSTRUCTIONS
    • E05LOCKS; KEYS; WINDOW OR DOOR FITTINGS; SAFES
    • E05FDEVICES FOR MOVING WINGS INTO OPEN OR CLOSED POSITION; CHECKS FOR WINGS; WING FITTINGS NOT OTHERWISE PROVIDED FOR, CONCERNED WITH THE FUNCTIONING OF THE WING
    • E05F15/00Power-operated mechanisms for wings
    • E05F15/70Power-operated mechanisms for wings with automatic actuation
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60NSEATS SPECIALLY ADAPTED FOR VEHICLES; VEHICLE PASSENGER ACCOMMODATION NOT OTHERWISE PROVIDED FOR
    • B60N2/00Seats specially adapted for vehicles; Arrangement or mounting of seats in vehicles
    • B60N2/002Seats provided with an occupancy detection means mounted therein or thereon
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/04Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0088Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02Control of position or course in two dimensions
    • G05D1/021Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0276Control of position or course in two dimensions specially adapted to land vehicles using signals provided by a source external to the vehicle
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/08Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • G06Q10/083Shipping
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/08Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
    • G06Q10/083Shipping
    • G06Q10/0832Special goods or special handling procedures, e.g. handling of hazardous or fragile goods
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18Status alarms
    • G08B21/24Reminder alarms, e.g. anti-loss alarms
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60NSEATS SPECIALLY ADAPTED FOR VEHICLES; VEHICLE PASSENGER ACCOMMODATION NOT OTHERWISE PROVIDED FOR
    • B60N2210/00Sensor types, e.g. for passenger detection systems or for controlling seats
    • B60N2210/10Field detection presence sensors
    • B60N2210/16Electromagnetic waves
    • B60N2210/22Optical; Photoelectric; Lidar [Light Detection and Ranging]
    • B60N2210/24Cameras
    • EFIXED CONSTRUCTIONS
    • E05LOCKS; KEYS; WINDOW OR DOOR FITTINGS; SAFES
    • E05YINDEXING SCHEME ASSOCIATED WITH SUBCLASSES E05D AND E05F, RELATING TO CONSTRUCTION ELEMENTS, ELECTRIC CONTROL, POWER SUPPLY, POWER SIGNAL OR TRANSMISSION, USER INTERFACES, MOUNTING OR COUPLING, DETAILS, ACCESSORIES, AUXILIARY OPERATIONS NOT OTHERWISE PROVIDED FOR, APPLICATION THEREOF
    • E05Y2400/00Electronic control; Electrical power; Power supply; Power or signal transmission; User interfaces
    • E05Y2400/10Electronic control
    • E05Y2400/44Sensors not directly associated with the wing movement
    • EFIXED CONSTRUCTIONS
    • E05LOCKS; KEYS; WINDOW OR DOOR FITTINGS; SAFES
    • E05YINDEXING SCHEME ASSOCIATED WITH SUBCLASSES E05D AND E05F, RELATING TO CONSTRUCTION ELEMENTS, ELECTRIC CONTROL, POWER SUPPLY, POWER SIGNAL OR TRANSMISSION, USER INTERFACES, MOUNTING OR COUPLING, DETAILS, ACCESSORIES, AUXILIARY OPERATIONS NOT OTHERWISE PROVIDED FOR, APPLICATION THEREOF
    • E05Y2400/00Electronic control; Electrical power; Power supply; Power or signal transmission; User interfaces
    • E05Y2400/10Electronic control
    • E05Y2400/45Control modes
    • EFIXED CONSTRUCTIONS
    • E05LOCKS; KEYS; WINDOW OR DOOR FITTINGS; SAFES
    • E05YINDEXING SCHEME ASSOCIATED WITH SUBCLASSES E05D AND E05F, RELATING TO CONSTRUCTION ELEMENTS, ELECTRIC CONTROL, POWER SUPPLY, POWER SIGNAL OR TRANSMISSION, USER INTERFACES, MOUNTING OR COUPLING, DETAILS, ACCESSORIES, AUXILIARY OPERATIONS NOT OTHERWISE PROVIDED FOR, APPLICATION THEREOF
    • E05Y2900/00Application of doors, windows, wings or fittings thereof
    • E05Y2900/50Application of doors, windows, wings or fittings thereof for vehicles
    • E05Y2900/53Type of wing
    • E05Y2900/548Trunk lids

Definitions

  • a service arrangement system can arrange a service to be provided for a requesting user through the use of mobile computing devices. For example, a user can request a service, such as a transport or delivery service, by operating the user's mobile computing device, and the service arrangement system can process the request to select the appropriate provider that can provide the service. The service arrangement system can also monitor the service, such as to determine the current location of the vehicle, before and during the progress of the service.
  • a service such as a transport or delivery service
  • FIG. 1 illustrates an example system to detect objects within a vehicle in connection with a transport service.
  • FIG. 2 illustrates an example method for detecting objects within a vehicle in connection with a transport service.
  • FIG. 3 illustrates another example method for detecting objects within a vehicle in connection with a transport service.
  • FIGS. 4A through 4D illustrate examples of user interfaces displayed on a mobile computing device in various use case examples.
  • FIG. 5 is a block diagram that illustrates a computing device upon which embodiments described herein may be implemented.
  • FIG. 6 is a block diagram that illustrates a computer system upon which examples described herein may be implemented.
  • a service arrangement system which implement a network service, can receive requests for location- based services and arrange those services to be provided by service providers.
  • service providers can include human drivers (or human-driven vehicles) that receive and/or accept invitations for services, and in some instances, autonomous (or self-driving) vehicles that can be used by the network service to transport goods or people.
  • AV autonomous vehicle
  • the network service can use information from a set of sensors of that vehicle to perform processes, when necessary, to supplement the assigned service.
  • a computing system associated with a vehicle can determine, through use of sensor data, whether the service has been properly completed or whether an object has inadvertently been left behind in the vehicle.
  • a computing system can determine that a vehicle has been arranged to provide a location-based service.
  • the computing system can correspond to a computing system associated with a vehicle (e.g., an on-board computing system of the vehicle, or a driver device operated by the driver of the vehicle) or a computing system(s) that implements the network service (e.g., a backend server system remote from the vehicle).
  • a vehicle e.g., an on-board computing system of the vehicle, or a driver device operated by the driver of the vehicle
  • a computing system(s) that implements the network service e.g., a backend server system remote from the vehicle.
  • a sensor is a device that can detect some characteristic of the surrounding environment or detect an event or change in quantity of an amount, and provide a corresponding output (e.g., sensor data).
  • the computing system can detect, at the time the service is being completed or after the service has been completed, whether the object has been removed from the vehicle based on a second set of information from the one or more sensors.
  • a vehicle can include or be equipped with one or more sensors, such as a set of cameras, a set of weighing devices, a set of suspension sensors, a set of light detection and ranging (LIDAR) sensors, a set of ultrasound sensors, a set of location-determination sensors, a set of radio-frequency sensors (such as Bluetooth or Wi-Fi transceivers), as well as other sensors used for various vehicle operations (e.g., engine sensors, tire pressure sensors, door open or close sensors, seat belt sensors, external stereo cameras or LIDAR sensors for AVs, etc.).
  • sensors such as a set of cameras, a set of weighing devices, a set of suspension sensors, a set of light detection and ranging (LIDAR) sensors, a set of ultrasound sensors, a set of location-determination sensors, a set of radio-frequency sensors (such as Bluetooth or Wi-Fi transceivers), as well as other sensors used for various vehicle operations (e.g., engine sensors, tire pressure sensors, door open or close sensors, seat belt sensors,
  • a set of sensors can correspond to one or more individual sensors or one or more arrays of sensors.
  • the computing system can use information (sensor data) from one or more sets of sensors (e.g., an individual sensor or a combination of a plurality of different sensors) to detect whether an object has been placed within the vehicle and/or has been removed from the vehicle.
  • the computing system can detect a position of an object inside the vehicle based on image information from a set of cameras that are positioned within the vehicle with the respective lens of the set of cameras facing towards the interior of the vehicle.
  • the computing system can detect that an object has been placed in or removed from a compartment or the interior of the vehicle based on weight information from a set of weighing devices that are positioned within the compartment or the interior of the vehicle.
  • the set of weighing devices can be positioned in the vehicle such that the respective measuring platform of the set of weighing devices is positioned on, positioned below, or included with a bottom surface of the respective compartment or interior of the vehicle.
  • the computing system can detect that a mobile computing device with wireless communications has been placed in or removed from the vehicle based on if that mobile computing device is communicating with antennas in the vehicle that use wireless protocols such as Bluetooth or Wi-Fi.
  • the computing system can compare the output of those location sensors to the output of location sensors on the vehicle to determine if the mobile computing device is moving with the vehicle, and is therefore likely to be inside the vehicle.
  • the computing system can detect that an object has been placed in or removed from the compartment or the interior of the vehicle based on compression information from a set of suspension sensors that are positioned proximate to or included in a suspension system of the vehicle.
  • the computing system can automatically perform a remedial action, such as communicating a message to a computing device associated with the user that requested the service and/or notifying the service arrangement system that the object is still present in the vehicle.
  • a remedial action such as communicating a message to a computing device associated with the user that requested the service and/or notifying the service arrangement system that the object is still present in the vehicle.
  • the computing system can also instruct the driver or cause the vehicle to remain stationary at a location of the vehicle when the object was detected to still be present in the vehicle for at least a predetermined duration of time.
  • the computing system can instruct the driver or cause the vehicle to travel to a predetermined or specific location where the object can be removed from the vehicle.
  • the computing system can also use sensor data to identify what the object is or what type of object has been placed in the vehicle. Still further, in some examples, the computing system can also use the detection of an object being placed in the vehicle and/or the detection of the object being removed from the vehicle to determine (or as part of determining) when the service has started and/or ended.
  • a client device, a driver device, and/or a mobile computing device refer to devices corresponding to desktop computers, cellular devices or smartphones, personal digital assistants (PDAs), laptop computers, tablet devices, etc., that can provide network connectivity and processing resources for communicating with a remote computing system(s) over one or more networks, such as a service arrangement system .
  • a remote computing system can refer to one or more computing systems or servers that is remote from the client device, the driver device, and/or the vehicle (e.g., corresponds to the back-end server system of the network service).
  • a driver device or a computing system associated with a vehicle can also correspond to custom hardware of the vehicle, such as an in-vehicle or on-board computing system, that has network connectivity and location-determination capabilities.
  • examples described herein relate to a variety of location-based (and/or on-demand) services, such as a transport service, a food truck service, a delivery service, an entertainment service, etc. to be arranged between users and service providers.
  • a service arrangement system can be implemented by any entity that provides goods or services for purchase through the use of computing devices and network(s).
  • the service can be implemented by any entity that provides goods or services for purchase through the use of computing devices and network(s).
  • arrangement system can correspond to a transport arrangement system that arranges transport services to be provided for riders/users by drivers or vehicles (e.g., transporting objects or people).
  • drivers or vehicles e.g., transporting objects or people.
  • One or more examples described herein provide that methods, techniques, and actions performed by a computing device are performed programmatically, or as a computer-implemented method.
  • Programmatically means through the use of code or computer-executable instructions. These instructions can be stored in one or more memory resources of the computing device.
  • a programmatically performed step may or may not be automatic.
  • a programmatic module, engine, or component can include a program, a sub-routine, a portion of a program, or a software component or a hardware component capable of performing one or more stated tasks or functions.
  • a module or component can exist on a hardware component independently of other modules or components.
  • a module or component can be a shared element or process of other modules, programs or machines.
  • Some examples described herein can generally require the use of computing devices, including processing and memory resources.
  • computing devices including processing and memory resources.
  • one or more examples described herein may be implemented, in whole or in part, on computing devices such as servers, desktop computers, cellular or smartphones, personal digital assistants (e.g., PDAs), laptop computers, printers, digital picture frames, network equipment (e.g., routers) and tablet devices.
  • PDAs personal digital assistants
  • Memory, processing, and network resources may all be used in connection with the establishment, use, or performance of any example described herein (including with the performance of any method or with the implementation of any system).
  • one or more examples described herein may be implemented through the use of instructions that are executable by one or more processors. These instructions may be carried on a computer-readable medium .
  • Machines shown or described with figures below provide examples of processing resources and computer-readable mediums on which instructions for implementing examples described herein can be carried and/or executed.
  • the numerous machines shown with examples described herein include processor(s) and various forms of memory for holding data and instructions.
  • Examples of computer-readable mediums include permanent memory storage devices, such as hard drives on personal computers or servers.
  • Other examples of computer storage mediums include portable storage units, such as CD or DVD units, flash memory (such as carried on smartphones, multifunctional devices or tablets), and magnetic memory.
  • Computers, terminals, network enabled devices are all examples of machines and devices that utilize processors, memory, and instructions stored on computer-readable mediums. Additionally, examples may be implemented in the form of computer-programs, or a computer usable carrier medium capable of carrying such a program .
  • FIG. 1 illustrates an example system to detect objects within a vehicle in connection with a transport service.
  • a system can be implemented by and operated on a mobile computing device of a driver, an in-vehicle computing system of a vehicle, or a service arrangement system that provides a network service.
  • the system 100 is described as being implemented by and operated on an in-vehicle computing system of a vehicle, such as an AV, through execution of instructions stored in one or more memory resources of the in-vehicle computing system.
  • the system 100 can include, be a part of, or correspond to an application or an operating system of the in-vehicle computing system that communicates with a service arrangement system 190 over one or more networks.
  • the system 100 can include a system manage 110, a location determination 120, an object detection 130, a location database 140, a trip database 150, and a service interface 160.
  • other components of the system 100 such as a user interface component, other databases, device interfaces to communicate with device components, etc., are not illustrated in FIG. 1.
  • the components of the system 100 can combine to detect objects that are placed within and/or removed from a vehicle in connection with transport services.
  • Logic can be implemented with various applications (e.g., software) and/or with hardware of the computing system that implements the system 100.
  • the system 100 can communicate with the service arrangement system 190 via the service interface 160.
  • the service arrangement system 190 can communicate with a plurality of service providers (e.g., driver devices and/or in-vehicle computing systems) and a plurality of client (or user) devices in order to arrange transport services between users and service providers.
  • the service arrangement system 190 can be implemented on network side resources, such as on one or more servers or data centers, or implemented through other computer systems in alternative architectures (e.g., peer-to-peer networks, etc.).
  • a user can operate a client or user device (not shown in FIG.
  • the service arrangement system 190 can communicate with the service arrangement system 190 over a network(s) and make a request for a transport service at a specified location (e.g., a pickup location of the package or a person to be transported).
  • a specified location e.g., a pickup location of the package or a person to be transported.
  • the user device can run a designated service application that communicates with the service arrangement system 190.
  • the service arrangement system 190 can receive the request and programmatically select a service provider to provide the transport service.
  • the service arrangement system 190 can transmit an invitation to the selected driver's device, thereby providing the driver with an option to accept or reject the invitation to provide the transport service for the requesting rider.
  • the service arrangement system 190 selects an AV from a plurality of available AVs (and/or human drivers) to provide the service, such as described in FIG. 1, the service arrangement system 190 can transmit information about the service (referred to as service instructions 191) to the system 100 via the service interface 160.
  • the service interface 160 enables the system 100 to exchange data between the system 100 and the service arrangement system 190.
  • the service interface 160 can use one or more network resources of the in-vehicle computing system to exchange communications over one or more wireless networks (e.g., a cellular
  • the service interface 160 can include or use an application programming interface (API), such as an externally facing API, to communicate data with the service arrangement system 190.
  • API application programming interface
  • the externally facing API can provide access to the service arrangement system 190 via secure access channels over the network through any number of methods, such as web-based forms, programmatic access via restful APIs, Simple Object Access Protocol (SOAP), remote procedure call (RPC), scripting access, etc.
  • the service arrangement system 190 has processed a request for a transport service by a user and has selected the AV that implements the system 100 to provide the transport service for the requesting user.
  • the service arrangement system 190 may have determined, based on the pickup location specified by the requesting user and the current locations (or most recently determined locations) of available service providers nearby the pickup location (e.g., within a predetermined distance of the pickup location), that the AV implementing the system 100 is to provide the transport service.
  • the service arrangement system 190 can transmit service instructions 191 about the transport service to instruct the vehicle about where to go and/or what to do.
  • the service instructions 191 can include user information 192 (e.g., a user identifier, contact information of the user, a user's rating, etc.), pickup location 193 specified by the user (e.g., an address, a landmark, a location data point corresponding to a latitude and longitude coordinate), destination location 194, and/or other information, such as the type of transport service (e.g., transport a person, deliver a package, etc.) or details about the number of people or number of packages.
  • the service instructions 191 can, in one example, cause the vehicle controlling
  • the vehicle control component(s) of the AV can use some or all of the service instructions 191 (in addition to other
  • the system manage 110 can receive the service instructions 191 and in response, can perform a plurality of operations on behalf of the system 100. For example, the system manage 110 can process the service
  • instructions 191 by determining information about the transport service it is to perform, and can initiate the performance of the transport service by determining the route(s) to travel to the pickup location and/or the destination location and controlling the AV to travel to the pickup location and/or the destination location.
  • a trip manage component of the system manage 110 can create a trip entry 151 for the transport service using some or all of the information from the service instructions 191 and store the trip entry 151 in a trip database 150 of the system 100.
  • the trip manage component can then update the trip entry 151 with information about the vehicle's location as the vehicle performs the transport service (e.g., travels to the pickup location from the current location, picks up the user or an object, travels to the drop off location, etc.).
  • the trip manage component can also record, in the trip entry 151, the time and the location of the vehicle when the service
  • the trip entry 151 can be used by the system 100 and/or the service arrangement system 190 to determine the cost for the transport service for the requesting user (e.g., after completion) and for maintaining a historical record of the AV and/or the requesting user.
  • the location determination 120 can provide the system manage 110 with information about the vehicle's location by receiving location data points 121 (e.g., periodically) from one or more geo-aware resources of the in-vehicle computing system. For example, the location determination 120 can periodically determine the current location of the vehicle by communicating with, e.g., via a device interface, the GPS receiver of the driver device, one or more other sensors, and/or one or more transceivers (e.g., cellular
  • the location determination 120 can make a call to the GPS receiver periodically, such as every two seconds or four seconds, etc., to periodically receive a current location data point 121 of the driver device.
  • the location determination 120 can also determine the associated timestamp of each determined location data point 121 of the vehicle.
  • the location determination 120 can receive a timestamp (and/or an error value) along with the location data point 121 of the vehicle from the GPS receiver or can determine the timestamp of the location data point 121 using a separate clock (e.g., an internal clock of the driver device).
  • the system manage 110 can store the location data points 121 in the location database 140 and/or the trip entry 151 to maintain data about where the vehicle traveled in connection with transport services.
  • the location determination 120 can store the location data points 121 of the vehicle in the location database 140.
  • the trip manage component can also use the location data points 121 from the location determination 120 to monitor the vehicle.
  • the trip manage component can determine the position and the state of the vehicle or system 100 in connection with the performance of the transport service (e.g., what stage of the transport service the vehicle is at). For example, the state of the system 100 may previously have been in an available state, which corresponds to a state in which the vehicle can be assigned the transport service from the service arrangement system 190.
  • the trip manage component can place the system 100 in a different state, e.g., on route state, which corresponds to the vehicle traveling to the pickup location.
  • the trip manage component can place the system 100 in a respective state (e.g., change from the on route state to the on trip state, or change from the on trip state to completed or available state, respectively).
  • the trip manage component can store information about the state and/or when the state change occurred along with the corresponding location data points 121 in the trip entry 151 of the transport service.
  • the service arrangement system 190 can locally generate and store its own trip entry that corresponds to or is similar to the trip entry 151 stored in the system 100.
  • the trip manage component can periodically provide the above-described information about the transport service to the service arrangement system 190, e.g., via the state information 111 and/or the location data points 113.
  • the service arrangement system 190 can use the received information from the system 100 to provide updated information about the transport service to the requesting user (e.g., the location of the AV and/or the current state of the AV, etc.).
  • the service arrangement system 190 can determine, from the state information 111 and/or the location data points 113, when the transport service has initiated or completed and/or when the AV is available to provide another transport service.
  • the system manage 110 can initiate the performance of the transport service based, at least in part, on the service instructions 191.
  • a vehicle control component of the system manage 110 can control the AV to perform the transport service.
  • the vehicle control component can interface with the AV control system or can correspond to the AV control system.
  • an AV includes (i) computing resources that implement the system 100 and/or the AV control system, (ii) other sub-components or sub-subsystems to control various other operations or parts of the AV, and (iii) a plurality of vehicle parts. For purpose of simplicity, not all parts of an AV are discussed herein.
  • various parts of an AV can include, but are not limited to, a frame or a chassis, a body (e.g., including doors, bumpers, windows, a cab, seats, a trunk space, etc.), tires, a suspension system, a braking system, a steering system, an electrical system (e.g., including batteries, wires, cables, alarm sub-system, etc.), a transmission system (e.g., including a drive train, engine, etc.), a fuel system, an exhaust system, a cooling system, and an AV control system .
  • the AV can also include electronic control units (ECUs), which include controllers and/or
  • the AV control system can be programmed, through execution of instructions stored in memory by one or more processors, to control the AV by sending control signals and by receiving data (e.g., feedback or sensor data) from the various systems and vehicle parts.
  • data e.g., feedback or sensor data
  • the AV control system can be instructed to control the AV so that it can safely navigate roads and streets to move from one location to another in connection with transport services.
  • the AV can include a location determination module or resources, such as a GPS receiver and/or wireless transceivers, to
  • the AV can also include a plurality of sensors, such as a set of cameras, a set of stereo cameras, a set of radars, a set of LIDAR sensors, etc. to enable the AV to detect the real-time environment of the AV.
  • sensors such as a set of cameras, a set of stereo cameras, a set of radars, a set of LIDAR sensors, etc. to enable the AV to detect the real-time environment of the AV.
  • One or more (or all) of these sensors can be positioned on the exterior of the vehicle (e.g., on the front, rear, roof of the vehicle, etc.) to detect information about the environment around the AV, such as the position of the vehicle relative to roads, building, other vehicles, etc., or the position of inanimate objects.
  • the AV control system can access a stored detailed spatial map(s) of the region it is in (e.g., including map data 112) and determine the route(s) to travel to the specified location. Based on the detected environment surrounding the AV (from data periodically and continuously detected by the set of sensors), the AV control system can periodically perform decision making operations that instruct the AV to take a certain action(s) to move to the specified location.
  • the AV control system can determine that it is to travel in a certain direction at a certain speed at one instance in time, for example, and at a next instance in time, can determine that it is to decelerate to a certain speed, change bearings/directions slightly, or change lanes, etc.
  • the AV control system can provide control signals to the various vehicle components/systems accordingly.
  • the system 100 can periodically determine whether the AV has reached the specified pickup location (e.g., start location). In other words, the system 100 can periodically check when the AV arrives at or is within a predetermined distance (or is within a predetermined estimated travel time) from the pickup location. For example, the trip manage
  • the trip manage component can periodically compare the current location (e.g., the latest location data point 121 provided by the location determination 120) with the pickup location 193 and determine whether the AV is within the predetermined distance or estimated travel time.
  • the current location e.g., the latest location data point 121 provided by the location determination 120
  • the pickup location 193 determines whether the AV is within the predetermined distance or estimated travel time.
  • the trip manage component can change the state of the system 100 from the on route state to the arriving/has arrived state.
  • the state information 111 about this change can be transmitted to the service
  • the service arrangement system 190 so that the service arrangement system 190 can provide updated information about the AV to the requesting user's device.
  • the system 100 can also determine whether an object(s) has been placed in the vehicle in connection with the transport service.
  • a driver or an AV control system may need to determine if a requesting user has placed an object in the vehicle while being transported (e.g., luggage, boxes, groceries) or determine if the object has been placed in the vehicle for a delivery transport service.
  • an object can correspond to an item(s) or a person(s) that is placed within or enters a vehicle.
  • the object detection 130 can communicate with one or more sets of sensors or one or more combinations of sensors of the AV, via one or more vehicle or sensor interfaces, to receive sensor data 131, and based on the sensor data 131, can determine whether an object has been placed in (or removed from) the AV (referred to herein as an object detection process or operation).
  • the object detection 130 can transmit a set of control signals to the sensors to cause (or trigger) the sensors to provide the sensor data 131.
  • Such sensors can include a set of cameras, a set of weighing devices, a set of suspension sensors, a set of light detection and ranging (LIDAR) sensors, a set of radio-frequency antennas and/or transceivers, and/or a set of ultrasound sensors that are positioned in or provided with the vehicle to detect the presence of, the position of, and/or the weight of one or more objects.
  • LIDAR light detection and ranging
  • the object detection 130 can perform an object detection process by (i) receiving sensor data 131 from one or more sensors, (ii) processing the received sensor data 131 using other information (e.g., models or previously stored data), and (iii) determining whether an object has been placed in the AV (when it was previously not in the AV) or determining whether an object has been removed from the AV (when it was previously in the AV).
  • other information e.g., models or previously stored data
  • the object detection 130 can receive image information from one or more cameras positioned within a trunk of the AV (e.g., with the lens facing the inside of the trunk), and process the image information by using image processing or image recognition processes and/or by comparing the image information to previously stored baseline image information about the trunk (e.g., showing the image of the trunk when it was empty or the image of the trunk before receiving the service instructions 191).
  • the stored baseline image information can be a predefined image or can be an image captured by the one or more cameras at a time before the AV arrived at or near the pickup location 193.
  • the object detection 130 can use image processing (as well as other stored or received data from the service arrangement system 190) to identify what the object is based on visual characteristics of the object in the image.
  • the object detection 130 can use other baseline or model information as a basis for comparison with received sensor data 131.
  • the object detection 130 can receive sensor data 131 from one or more weighing devices or sensors that are positioned in a compartment of the AV such that its respective measuring platform is on, positioned below, or included with a bottom surface of such compartment.
  • the object detection 130 can determine a change in the measurements from the sensor data 131 at a time before the AV arrived at or near the pickup location 193 and after the service has started.
  • the object detection 130 can perform the object detection process at various times. For example, the object detection 130 can concurrently perform the object detection process as the AV travels to the pickup location 193 and/or as the trip manage component determines whether the AV is within the predetermined distance or
  • the object detection 130 can make calls to the set of sensors to receive sensor data 131 periodically. In another example, the object detection 130 can perform the object detection process at various instances when instructed or triggered by the system manage 110 (via trigger 115). For example, when the trip manage component determines that the AV is within the predetermined distance or predetermined estimated travel time of the pickup location 193, the trip manage component can transmit a trigger 115 to cause the object detection 130 to perform the object detection process.
  • the trip manage component can transmit the trigger 115 when it determines that the AV is stationary for a predetermined duration of time and/or when it determines that the AV is stationary for a predetermined duration of time while it the AV is within the predetermined distance or predetermined estimated travel time of the pickup location 193.
  • the object detection 130 can be triggered to perform the object detection process when the trip manage component determines that the transport service has started. By causing the object detection 130 to perform the object detection process at select times, the system 100 can reduce excessive consumption of energy and computing resources by causing the sensor(s) to wake up (from a suspended or sleep state) and detect or perform a measurement only when necessary.
  • the object detection 130 can provide object information 133 to the system manage 110.
  • the object information 133 can indicate to the system manage 110 that an object has been placed in the AV (e.g., a message, or a binary bit can be toggled) and/or can include information about that object based on the types of sensor(s) that is used by the AV to perform the detection.
  • the additional information can include (i) what the object is (e.g., a box, a bag, a skateboard, etc.), (ii) one or more images of the object captured by the set of cameras, if cameras are used, (iii)
  • the trip manage component can store some or all of the object information 133 with the trip entry 151 associated with the transport service, so that information about the object can be logged and associated with the particular user. [0042] As described, concurrently (or before or after) the object detection 130 detects an object, the trip manage component can determine that the transport service has begun.
  • the system manage 110 can determine that the transport service has started, based, at least in part, on detecting that an object has been placed in the vehicle.
  • the transport service can correspond to transporting a person (and/or the person's things) from one location to another, or transporting an object for delivery from one location to another.
  • the object detection 130 can detect, from one or more sensors, such as a camera or weighing devices, if a person and/or objects have entered the vehicle or if an object (without the person) has been placed in the vehicle.
  • the set of cameras can be positioned inside the cab/interior of the vehicle to enable the object detection 130 to detect that the person has entered the vehicle using image information from the camera(s) and processing the image(s).
  • a set of weighing devices or a set of suspension sensors that is positioned proximate to or included in a suspension system of the AV can determine that an amount of mass has been added to the vehicle.
  • the trip manage component can store information about the performance of the transport service (e.g., the time and/or the location of travel of the AV), including the start time and start location of the transport service, in the trip entry 151.
  • the trip manage component can also use the location data points 121 to determine when the AV approaches and has approached the destination location 194, for purpose of determined, at least in part, that the transport service has completed.
  • the trip manage component can cause the object detection 130 to also detect whether the object has been removed from the vehicle.
  • the object detection 130 can detect whether the object has been removed from the vehicle at different times, depending on variations.
  • the system manage 110 can cause the object detection 130 to concurrently perform the object detection process as the AV travels to the destination location 194 (e.g., periodically), or perform the object detection process when the trip management component
  • the trip manage component can cause the object detection 130 to perform the object detection process when the trip manage component determines that the AV is stationary for a predetermined duration of time, when the trip manage component determines that the AV is stationary for a predetermined duration of time while it the AV is within the predetermined distance or predetermined estimated travel time of the destination location 194, or in response to determining that the transport service has been completed.
  • the object detection 130 can perform the object detection process to determine if the object has been removed using sensor data 131 received from the set of sensors (e.g., a second set of sensor data 131). If the object detection 130 detects that the object is still present, it can provide information to the system manage 110 indicating that the object is still present in the vehicle. In response, the system manage 110 can automatically perform a remedial action. On the other hand, if the object detection 130 detects that the object has been removed, no remedial process or action is taken. In one variation, if the system 100 detects that the object has been removed (e.g., in the case of a delivery transport service), the system 100 can determine that the delivery transport service has been completed.
  • sensor data 131 received from the set of sensors (e.g., a second set of sensor data 131). If the object detection 130 detects that the object is still present, it can provide information to the system manage 110 indicating that the object is still present in the vehicle. In response, the system manage 110 can automatically perform a remedial action. On the other hand
  • a remedial action can correspond to an operation (or set of operations) the system 100 performs to return whatever object is accidentally left behind to the user or to make it easier for the user to retrieve the object.
  • the remedial action(s) can be performed by the system 100 and/or performed by the system 100 in connection with the service arrangement system 190. In the latter example, the remedial action(s) can be performed as a result of the exchange of data between the system 100 and the service arrangement system 190.
  • the system manage 110 can receive information that the object is still remaining in the vehicle and/or information about the object itself (e.g., based on recently determined information about the object, or from object information 133 previously received when the object was previously detected) and can generate an object message 117 based on such information.
  • the object message 117 can include, for example, a time and/or a location when the transport service was completed, an identifier of the AV, an identifier of the user, an identifier of the transport service, and information about the object (e.g., what the object is, one or more images of the object, measurements pertaining to the object, where the object is located).
  • the system manage 110 can determine a communication identifier for the user (e.g., a phone number or email address of the user or temporarily associated with the user's account), and transmit the object message 117 to the user's computing device.
  • a communication identifier for the user e.g., a phone number or email address of the user or temporarily associated with the user's account
  • Such an object message 117 can also include notification information to cause the user's computing device to display content to inform the user that the object has been left behind in the AV and/or to contact the entity that provides the service arrangement system 190 (e.g., via the designated application running on the user's computing device).
  • an object message 117 can be transmitted to the service arrangement system 190, which can then transmit a message, including notification information, to the user's computing device.
  • the notification information displayed to the user can provide information about the object and/or how the user can retrieve the object. Examples of different notification information is illustrated in FIGS. 4A through 4D.
  • the system manage 110 can also perform one or more remedial actions by positioning the AV at a particular location (e.g., by causing the AV to move to a particular location or keeping stationary at a particular location). For example, when a remedial action is to be performed, the system manage 110 can access stored positioning instructions that instruct the vehicle control component to control, via the vehicle components, the AV to remain stationary at (i) the destination location 194, or (ii) the location of the vehicle when the object detection 130
  • the object message 117 can inform the user that the user's object has been left behind in the AV, and that the AV will remain at the specified location for a predetermined duration of time (e.g., two minutes).
  • the stored positioning instructions can instruct the vehicle control component to cause the AV to travel to a specified location (e.g., a storage location or office building where lost objects are stored), and the object message 117 can inform the user that the object can be retrieved at the specified location during certain times.
  • the AV can use automatically-controlled actuators to remove the object from the AV for storage or disposal.
  • the AV itself can include one or more actuators coupled to one or more mechanical hands in the vehicle's interior or trunk that can grab an object and remove it.
  • the AV can move to a specified lane or spot at the storage or disposal location, open the vehicle doors and/or trunk, and an automatically-controlled mechanical arm and hand or claw can remove the object from the vehicle.
  • the system manage 110 can provide information to the service arrangement system 190 indicating that an object has been left behind, and in response, the service arrangement system 190 can transmit instructions to the system 100 to cause the AV to perform one or more tasks.
  • the service arrangement system 190 can transmit vehicle instructions to the system manage 110 that causes the vehicle control component to control the movement and operation of the AV. The vehicle instructions can cause the AV to move to a specified location or remain stationary at a specified location .
  • the AV may be arranged/selected to provide a transport service to another requesting user.
  • the system 100 may perform the remedial action, such as transmitting the object message 117, but also cause the AV to travel to the pickup location specified by the subsequent requesting user.
  • the service arrangement system 190 (or the system 100) can transmit a message to the user's computing device, which queries the user for an response input to (i) instruct the disposal of the object, (ii) ask the entity that operates the service arrangement system 190 to hold on to the object on behalf of the user for a duration of time, (iii) request a transport service to a specified location where the object is held, or (iv) request a delivery of the object to the user's current location (or specified location).
  • the service arrangement system 190 can receive the user input, via the
  • the service arrangement system 190 can process the response as a delivery transport service request, identify the previous AV that the object is located in, and schedule that particular AV to travel to the user's current location (e.g., before, while, or after completing the subsequent transport service for the subsequent requesting user).
  • the user can be charged for the delivery as a separate transport service in its entirety or as part of the user's previous transport service (e.g., as a surcharge).
  • the system 100 can be implemented on a mobile computing device operated by a driver of a vehicle (e.g., a driver device).
  • the system 100 can be implemented as an application or program (or as part of an application) that runs on the driver device, and that communicates data with the service arrangement system 190 over one or more networks.
  • the system 100 can include other components, such as a user interface component that generates and presents content (e.g., graphic user interfaces) on the display of the driver device.
  • the system 100 can receive and display an invitation to provide the transport service via the user interface component, and the driver can either accept or reject the invitation.
  • the invitation can include the user information 192, the pickup location 193, and/or the destination location 194.
  • the object detection 130 can receive sensor data 131 via a communication interface of the driver device, such as a short-range wireless communication transceiver (e.g., Bluetooth or Wi-Fi) or a physical connector port.
  • the driver can fix or attach, in the trunk or in the backseat of the vehicle, etc., one or more sensors which can communicate sensor data 131 to the object detection 130 via the communication interface.
  • one or more components of the system 100 can be implemented as part of the service arrangement system 190 or as part of another remote system (e.g., remote from the autonomous vehicle or the driver device) that communicates with the service arrangement system 190.
  • the system 100 can transmit sensor data 131 to an object detection 130 that is implemented in the service arrangement system 190, which then detects when an object has been placed in a vehicle and/or has been taken out of the vehicle.
  • the service arrangement system 190 can then perform the remedial actions, such as transmitting notifications to the user, based on the detection.
  • sensors can be used by the system 100 to detect a presence and/or removal of an object from a vehicle.
  • electromagnetic sensors can be used to detect electronic products, such as cell phones, tablets, laptop computers, etc., or other advanced sensors, such as chemosensors or gas chromatography sensors can be used to detect objects using smell (e.g., food, groceries, etc.).
  • using electromagnetic sensors or antennas/wireless transceivers can be helpful to detect small electronic devices that may be hard to detect in the vehicle, e.g., may have fallen or slipped between the seats or under the seat.
  • the system 100 can also use the sensor data to detect whether there has been damage to the interior of the vehicle as a result of the requesting user riding in the vehicle and/or transporting an object of the user.
  • the system 100 can detect the state of the vehicle's interior at a time before the transport service is initiated (e.g., in response to when the service instructions are received or at a predetermined distance or estimated time away from the pickup location of the user) using sensors that are positioned in the interior of the vehicle.
  • the system 100 can again detect the state of the vehicle's interior at this time, and compare the information with the previous information about the state of the vehicle's interior. Using this comparison, the system 100 can detect whether something has spilled on the seats or the floor, whether the seats have been ripped or scratched, etc., after the transport service. If damage is detected, the system 100 can provide information indicating the damage to the service arrangement system 190 and/or to the user device.
  • the service arrangement system 190 can charge the user and/or instruct the driver or cause the vehicle to travel to a specified location for clean-up and/or repair.
  • FIGS. 2 and 3 illustrate example methods for detecting objects within a vehicle in connection with a transport service. Methods such as described by examples of FIGS. 2 and 3 can be implemented using, for example, components described with an example of FIG. 1. Accordingly, references made to elements of FIG. 1 are for purposes of illustrating a suitable element or component for performing a step or sub-step being described. For purpose of simplicity, the example method of FIG. 2 is described as being performed by the service arrangement system 190 and/or the system 100 operating on a driver device.
  • the service arrangement system 190 can determine that a driver of a vehicle or an AV been selected to provide a service for a requesting user (200). For example, the service arrangement system 190 can perform a vehicle selection process to select the vehicle based on the location of the vehicle and the service location(s) specified by the requesting user.
  • the service can correspond to an object or item delivery transport service from a pickup location to a destination location, or a transport service of a person (e.g., the user) from a pickup location to a destination location.
  • the service arrangement system 190 can transmit, to the respective system 100 of the selected vehicle, service instructions
  • the system 100 can process the service instructions to control the AV to travel from the current location to the pickup location, while in the example of a human driver, the system 100 can provide route or navigation information that the driver can follow to move to the pickup location.
  • the system 100 can perform an object detection operation using data from one or more sensors of the vehicle.
  • the service arrangement system 190 can receive a first set of sensor information from the system 100 (e.g., at a first time) and can detect that an object has been placed within the vehicle based on the first set of information (210).
  • the system 100 can receive the sensor information from a set of sensors of the vehicle.
  • the system 100 can detect that the object has been placed within the vehicle and provide information indicating such object to the service arrangement system 190.
  • the system 100 and/or the service arrangement system 190 can receive image information from a set of cameras (or radars, LIDARs, or ultrasound sensors) that are positioned within the vehicle, where the set of cameras are positioned such that the respective lens of the set of cameras faces towards an interior of the vehicle (e.g., the back seat, the front seat, the trunk, or other compartment).
  • a set of cameras or radars, LIDARs, or ultrasound sensors
  • the set of cameras are positioned such that the respective lens of the set of cameras faces towards an interior of the vehicle (e.g., the back seat, the front seat, the trunk, or other compartment).
  • the system 100 and/or the service arrangement system 190 can receive weight information from a set of weighing devices that are positioned within a compartment or the interior of the vehicle, where the set of weighing devices are positioned such that the respective measuring platform of the set of weighing devices is positioned on, positioned below, or included with a bottom surface of the compartment or the interior of the vehicle (e.g., the platform is at the surface of the trunk to weigh objects that are placed in the trunk or at or in a seat to weigh a person that sits in the back set or front seat).
  • system 100 and/or the service arrangement system 190 can receive compression information from a set of suspension sensors that is positioned proximate to or included in a suspension system of the vehicle (e.g., near the rear and/or front suspension system or frame of the vehicle). Additionally, a sensor can also provide information whether a compartment or trunk has been opened and/or closed, which can be used in connection with other sensor information.
  • the system 100 and/or the service arrangement system 190 can use information from one or more of these different sources to perform the object detection operation. In addition, based on the first set of received sensor input/information, the system 100 and/or the service arrangement system 190 can detect that the object has been placed in the vehicle, what the object is (or type of object), the color or size or dimension or weight of the object, what compartment the object is in, and/or the position of the object. According to an example, the service arrangement system 190 can also associate the object information with the trip entry of the transport service (and/or with the user's profile or account).
  • the system 100 can trigger the sensors to determine sensor data (e.g., perform detection or measurement) based on location information of the AV. For example, in order to conserve energy or power, the system 100 can trigger the sensors to determine sensor data when the AV arrives at or close to a service location, e.g., arrives at or within a predetermined distance or estimated travel time of the pickup location, and/or when the AV is stationary for a duration of time while at or close to the service location. Alternatively, in another example, the system 100 can trigger the sensors to determine sensor data in response to determining that the service has begun.
  • sensor data e.g., perform detection or measurement
  • the service arrangement system 190 can also determine that the service has begun (220). Depending on
  • the service arrangement system 190 can determine that the service has begun (i) in response to detecting that the user has entered the vehicle (e.g., in response to detecting the object in the vehicle in step 210, or based on location data points of the user's computing device and the location data points of the vehicle, and determining that they are within a
  • a predetermined distance of each other in response to detecting that a predetermined duration of time has elapsed (e.g., ten seconds) since detecting that the user has entered the vehicle, (iii) in response to receiving, from the user's computing device, information indicating that the service can begin as a result of an input being provided at the computing device associated with the user (e.g., the user selects a feature corresponding to starting the trip), (iv) in response to receiving, from driver device (or the on-board computing system of the AV), information indicating that the service can begin as a result of an input being provided at the driver device or the on-board computing system from the driver or the user, and/or (v) in response to determining that the vehicle has been positioned within a predetermined distance of the start location for a predetermined duration of time based on location information of the vehicle.
  • a predetermined duration of time e.g., ten seconds
  • the system 100 detects that the service has begun based on any of the examples (i) through (v).
  • the service arrangement system 190 also monitors the service, for example, as the driver or the AV performs the service (230). As described herein, monitoring the service can include periodically determining the location (and corresponding timestamp) and/or state of the vehicle as it travels from the pickup location to the destination location, and storing such information in the trip entry associated with the service. The system 100 can provide such information to the service arrangement system 190. In some examples, the service arrangement system 190 can also provide the information about the progress of the service to the user's computing device (or to another device).
  • the user may have requested a delivery service for a package to be picked up at the user's location and dropped off at a destination location for a friend.
  • the application on the user's device can receive (periodically) information about the progress of the service, e.g., the AV's location and movement on a map, from the service arrangement system 190.
  • the user may have provided, to the service arrangement system 190 via the application, the contact information (e.g., phone number or email address) of the friend who is to receive the package.
  • the service arrangement system 190 can monitor the service and provide updated information about the service to the friend, so that the friend can track the package as the AV travels to the destination location.
  • the service arrangement system 190 can determine that the service has been completed, e.g., as a result of monitoring the service (240). Depending on implementation, the service arrangement system 190 can determine that the service has completed (i) in response to detecting that the user has left the vehicle (e.g., based on location data points of the user's computing device and the location data points of the vehicle, and determining that they are separated by more than a predetermined distance), (ii) in response to detecting that a predetermined duration of time has elapsed (e.g., twenty seconds) since detecting that the user has left the vehicle, (iii) in response to receiving, from the user's computing device, information
  • the service arrangement system 190 can detect whether the object (that was previously detected in the vehicle) has been removed from the vehicle based on a second set of sensor information from one or more sensors of the vehicle (e.g., at a second time) (250). For example, the service arrangement system 190 can receive the second set of sensor information from the system 100 and can compare the first and second sets of sensor information to determine if there is a difference, e.g., difference in imagery, difference in weight, difference in compression information, etc. As an alternative, the system 100 can perform the comparison operation and provide an indication of whether the object has been removed to the service arrangement system 190.
  • a difference e.g., difference in imagery, difference in weight, difference in compression information, etc.
  • the system 100 can trigger the sensors to determine sensor data (e.g., perform detection or measurement) at a particular instance in time, depending on examples, such as in response to detecting that the service has been completed, or after a predetermined duration of time since detecting that the service has been completed.
  • sensor data e.g., perform detection or measurement
  • the service arrangement system 190 can determine whether the object has been removed from the vehicle before the service is determined to be completed. For example, the service arrangement system 190 (or the system 100) can perform the object detection process in response to determining that the AV has reached or is close to the destination location and/or has been stationary for a duration of time while at or close to the destination location. In another example, the service arrangement system 190 can determine that the service has been completed in response to detecting that the object has been removed from the vehicle (e.g., after performing step 250).
  • the service arrangement system 190 determines that the object has been removed from the vehicle, it can determine that no remedial action needs to be performed, as the user in the vehicle has taken the object with him or her, or another user has removed the object (e.g., at the delivery destination location) (270). On the other hand, if the service arrangement system 190 determines that the object has not been removed, the service arrangement system 190 can automatically perform one or more remedial actions (260). For example, the service arrangement system 190 can message the driver and/or the user, via the respective applications on the driver device and/or the user device, to inform them about the misplaced object. The message can include instructions (e.g., including navigation instructions) for enabling the driver and the user to meet up at a location, e.g., the destination location, to enable the user to retrieve the object.
  • instructions e.g., including navigation instructions
  • the service arrangement system 190 can transmit a message or notification to the user device and enable the user to request a delivery of the object to a location specified by the user. Still further, in one example, if the vehicle is an AV, the service arrangement system 190 can transmit vehicle control instructions to the AV to cause the AV to travel to a specified location. Such a specified location can correspond to a new meeting location (delivery drop off location) for the user specified in the user's subsequent delivery transport service request, or a designated storage or warehouse location to house misplaced objects. The service arrangement system 190 can also message the user device to provide such location information on where the user can retrieve the object.
  • FIG. 3 illustrates another example method for detecting objects within a vehicle in connection with a transport service.
  • one or more steps of the method described in FIG. 3 can be performed concurrently with one or more steps of the method described in FIG. 2.
  • the example method of FIG. 3 is described as being performed by the system 100 operating on an on-board computing system of an
  • the system 100 can receive service instructions from the service arrangement system 190 for the AV to provide a service for a requesting user (300).
  • a service can have a start location and a destination location, and can correspond to a delivery of an object or a person (e.g., the requesting user).
  • the system 100 can process the service instructions in order to determine what service to provide, where to travel to (e.g., from the current location, to the start/pickup location, then to the destination location), and how to travel there (e.g., the route to take).
  • the system 100 can control the AV to travel to the start location, and concurrently, monitor the movement of the AV (310).
  • the system 100 can correspond to, include, or be in communication with the AV control system to control the operation of the AV by sending control signals to vehicle control modules of various sub-systems in response to receiving sensor data from the sensors and feedback signals from various vehicle components.
  • the system 100 can also determine the location of the AV from GPS data, for example, and monitor the current position as compared to the route and/or the start location.
  • the system 100 can determine when the AV arrives at or is within a predetermined distance of the start location (e.g., the AV is close to the start location). In one example, when the system 100 determines that the AV is close to the start location, the system 100 can change the state of the AV (from the on route state to the arriving now or has arrived state).
  • the system 100 can perform an object detection process to detect that an object has been placed within the AV (320). Depending on
  • the system 100 can periodically perform the object detection process, as described in FIGS. 1 and 2, from the time after determining that the AV is close to the start location to a time after the AV begins traveling to the destination location.
  • the system 100 can perform the object detection process in response to determining that the service has started or can perform the object detection process before or concurrently while determining that the service has started.
  • the system 100 can detect that that an object (e.g., a box) has been placed in the back seat of the AV from sensor information received from the AV, as described in FIGS. 1 and 2.
  • the system 100 can control the AV to travel to the destination location, and also monitor the vehicle and/or the progress of the service (330). Similarly as described above, the system 100 can determine the location of the AV from GPS data and monitor the current position as compared to the route and/or the destination location. The system 100 can determine when the AV arrives at or is within a predetermined distance of the destination location (e.g., the AV is close to the destination location) (340). At a time after determining that the AV is close to the destination location, the system 100 can perform an object detection process to detect whether the object has been removed from within the AV (350). Depending on implementation, step 350 can be performed before, after, or while determining that the service has been completed.
  • the system 100 determines that the object has been removed from the AV, the system 100 can determine that no remedial action needs to be performed (370). On the other hand, if the system 100 determines that the object is still present in the vehicle, the system 100 can perform one or more remedial actions and/or cause the service arrangement system 190 to perform one or more remedial actions, such as described in FIGS. 1 and 2 (360).
  • FIGS. 4A through 4D illustrate examples of user interfaces displayed on a mobile computing device in various use case examples.
  • an individual user interface can be displayed on a user device of a user who has left an object in a vehicle in connection with the user's transport service.
  • the user interface can be displayed as part of a designated application associated with the service arrangement system 190 or as part of another application operating on the user device (e.g., email application, text message application, a web browser application, etc.).
  • a notification can be displayed on the user device, and selection of a feature or notification can cause the user interface to be displayed on the user device.
  • the user device can receive a notification or a message from the service arrangement system 190 or the system 100 after (or in some instances, just before) the service has ended.
  • the application running on the user device can display the user interface 400, which includes image content 405 of the object and textual content 410 reminding the user and/or notifying the user that the object was left in the vehicle.
  • the image content 405 can be generated from image information from one or more sensors (e.g., cameras) of the vehicle. Alternatively, if a camera is not used in the vehicle, other content representing the object can be displayed in place of the image content 405, such as additional textual content or other graphics or placeholder images representing the object.
  • the textual content 410 in this example, also provides a contact information for the user to message in order to determine what to do in order to retrieve the object
  • a user interface 420 can be displayed on the user device.
  • the user interface 420 can also include image content 425, such as described in FIG. 4A, and textual content 430 notifying the user that the object was left in the vehicle and informing the user of a specific location in which to pick up the object.
  • the service arrangement system 190 can instruct the AV or the driver to travel to a specific location (or remain at the destination location or where the user was dropped off).
  • the service arrangement system 190 can generate textual content 430 to include corresponding content about the location.
  • the textual content 430 informs the user that the vehicle with the object will be waiting for 2 more minutes (or some other predetermined duration of time) where the user was dropped off.
  • a user interface 440 of FIG. 4C can be displayed on the user device as an interactive notification prompt.
  • a notification prompt can be in-app (e.g., displayed as part of the application user interface) or out-of-app (e.g., displayed when the application is not currently open and using native user interface notification features of an operating system of a mobile device).
  • the user interface 440 includes textual content 445 that includes a specific location at which the user's object will be stored or held.
  • the textual content 445 can include location information (e.g., address or street intersection), a time or duration of time when the user can pick up the object (e.g., "normal business hours” or "8am to 6pm"), and a selectable feature 450 that the user can select in order to request a transport service to the specified location.
  • location information e.g., address or street intersection
  • time or duration of time when the user can pick up the object e.g., "normal business hours” or "8am to 6pm”
  • a selectable feature 450 that the user can select in order to request a transport service to the specified location.
  • the designated service application can be opened or launched on the user device, so that the user can confirm the transport service to the specified location (e.g., from the user's current location or from a specified pickup location).
  • the designated service application can automatically generate and transmit a request for the transport service to the service arrangement system 190.
  • the request can include the type of service (e.g., transport the user), the
  • a user interface 460 can include textual content 465 to inform the user that the user can request a delivery of the user's object.
  • the user interface 460 can enable the user to select a selectable feature 470 to request a delivery transport service of the user's object to the user's current location .
  • the designated service application can be opened or launched on the user device, so that the user can confirm the delivery transport service of the user's object to the specified location (e.g., from the user's current location or from a specified pickup location) or the designated service application can
  • the request can include the type of service (e.g., delivery of the object in the specific vehicle that has the user's object), and the drop off or destination location (e.g., the current location of the user device determined via the GPS receiver of the user device).
  • the service arrangement system 190 can identify the particular vehicle that the user had just previously used or the vehicle that is associated with the trip entry having the identified object, and select that vehicle to go to the user's current location.
  • a user interface described in any of FIGS. 4A through 4D can also include detailed information about the object, such as what the object is (including color or shape), the object type, the dimensions and/or weight of the object, where the object was placed, etc. While the examples of FIGS. 4A through 4D are described with respect to the user device, in some variations, such user interfaces can also be displayed on the driver device to inform the driver of an object that has been left behind in his or her vehicle. Such a user interface can instruct the driver where to go, where to wait (and/or for how long), and/or who to meet with, so that the user can retrieve the object.
  • HARDWARE DIAGRAMS can instruct the driver where to go, where to wait (and/or for how long), and/or who to meet with, so that the user can retrieve the object.
  • FIG. 5 is a block diagram that illustrates a computing device upon which embodiments described herein may be implemented.
  • a computing device 500 may correspond to a mobile computing device, such as a cellular device that is capable of telephony, messaging, and data services.
  • the computing device 500 can correspond to a user device or a driver device. Examples of such devices include smartphones, handsets or tablet devices for cellular carriers.
  • the computing device 500 includes a processor 510, memory resources 520, a display device 530 (e.g., such as a touch-sensitive display device), one or more communication sub-systems 540 (including wireless communication sub-systems), input mechanisms 550 (e.g., an input
  • a display device 530 e.g., such as a touch-sensitive display device
  • communication sub-systems 540 including wireless communication sub-systems
  • input mechanisms 550 e.g., an input
  • the mechanism can include or be part of the touch-sensitive display device), and one or more sensors 560, including a location detection mechanism (e.g., GPS receiver).
  • a location detection mechanism e.g., GPS receiver
  • at least one of the communication sub-systems 540 sends and receives cellular data over data channels and voice channels.
  • the communications sub-systems 540 can include a cellular transceiver and one or more short-range wireless transceivers.
  • the processing resources 510 can receive communications from the service arrangement system, for example, via the communications sub-systems 540.
  • the computing device 500 can correspond to a driver device that is operated by a driver of a vehicle.
  • the processor 510 can provide a variety of content to the display 530 by executing instructions stored in the memory resources 520.
  • the memory resources 520 can store instructions corresponding to the driver application 525, for example, and other data, such as one or more trip entries 527, one or more parameters, or databases of location data points.
  • the processor 510 is configured with software and/or other logic to perform one or more processes, steps, and other functions described with implementations, such as described by FIGS. 1 through 4D, and elsewhere in the application .
  • the processor 510 can execute instructions and data stored in the memory resources 520 in order to be notified about a transport service the driver is to provide, and in connection with the transport service, can use sensor data 565 to perform object detection processes.
  • the sensors 560 can include a sensor included with the computing device 500 (e.g., a GPS receiver) as well as sensor interfaces that communicate with one or more sensors of the vehicle (e.g., ultrasound sensors, weighing devices, etc.).
  • the processor 510 can also execute instructions corresponding to the driver application 525 to cause various user interfaces 515 to be displayed on the display 530, such as an invitation to provide a transport service, navigation information, or notifications about a user having left an object in the driver's vehicle.
  • the user interfaces 515 can include selectable features that the driver can select, by providing user input 555 via the input mechanisms 550 (e.g., in combination with a touch-sensitive display 530), to interact with the driver application 525, such as to indicate when the transport service has started or completed, in some examples.
  • FIG. 5 is illustrated for a mobile computing device, one or more examples may be implemented on other types of devices, including full- functional computers, such as laptops and desktops (e.g., PC).
  • a variation of FIG. 5 can illustrate (at least a portion of) an onboard computing system of an AV, such has described in FIG. 1.
  • FIG. 6 is a block diagram that illustrates a computer system upon which examples described herein may be implemented.
  • the service arrangement system 190 may be implemented using a computer system such as described by FIG. 6.
  • arrangement system 190 may also be implemented using a combination of multiple computer systems as described by FIG. 6.
  • the computer system 600 includes processing resources, such as one or more processors 610, a main memory 620, a read-only memory (ROM) 630, a storage device 640, and a
  • the computer system 600 includes at least one processor 610 for processing information and the main memory 620, such as a random access memory (RAM) or other dynamic storage device, for storing information and instructions to be executed by the processor 610.
  • the main memory 620 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by the processor 610.
  • the computer system 600 may also include the ROM 630 or other static storage device for storing static information and instructions for the processor 610.
  • the storage device 640 such as a magnetic disk or optical disk, is provided for storing information and instructions.
  • the storage device 640 can correspond to a computer-readable medium that stores location determination instructions 642 and object detection
  • the computer system 600 can determine the locations of the vehicles, determine the service locations of the transport services, and perform object detection processes based on sensor data received from the vehicle systems or driver devices.
  • the storage device 640 can include other instructions, such as instructions to implement a dispatch sub-system, which receives requests for transport services from requesting users and selects vehicles to provide those transport services.
  • the communication interface 650 can enable the computer system 600 to communicate with one or more networks 680 (e.g., cellular network) through use of the network link (wirelessly or using a wire). Using the network link, the computer system 600 can communicate with a plurality of devices, such as the mobile computing devices of the riders and drivers. According to some examples, using the communication interface 650, the computer system 600 can receive location information 652 from the driver devices, AVs, and/or user devices, and can transmit notification messages 654 to user devices to inform users if users' belongings have been left in a vehicle, such as described by some examples of FIGS. 1 through 5.
  • networks 680 e.g., cellular network
  • the computer system 600 can communicate with a plurality of devices, such as the mobile computing devices of the riders and drivers.
  • the computer system 600 can receive location information 652 from the driver devices, AVs, and/or user devices, and can transmit notification messages 654 to user devices to inform users if users' belongings have been left in a vehicle, such as described by
  • the computer system 600 can also include a display device 660, such as a cathode ray tube (CRT), an LCD monitor, or a television set, for example, for displaying graphics and information to a user.
  • a display device 660 such as a cathode ray tube (CRT), an LCD monitor, or a television set, for example, for displaying graphics and information to a user.
  • the input mechanisms 670 can be coupled to the computer system 600 for communicating information and command selections to the processor 610.
  • Other non-limiting, illustrative examples of the input mechanisms 670 include a mouse, a trackball, touch-sensitive screen, or cursor direction keys for communicating direction information and command selections to the processor 610 and for controlling cursor movement on the display 660.
  • Examples described herein are related to the use of the computer system 600 for implementing the techniques described herein. According to one example, those techniques are performed by the computer system 600 in response to the processor 610 executing one or more sequences of one or more instructions contained in the main memory 620. Such instructions may be read into the main memory 620 from another machine-readable medium, such as the storage device 640. Execution of the sequences of instructions contained in the main memory 620 causes the processor 610 to perform the process steps described herein. In alternative implementations, hard-wired circuitry may be used in place of or in combination with software instructions to implement examples described herein. Thus, the examples described are not limited to any specific combination of hardware circuitry and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Economics (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Theoretical Computer Science (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Operations Research (AREA)
  • Development Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Quality & Reliability (AREA)
  • Mechanical Engineering (AREA)
  • Health & Medical Sciences (AREA)
  • Automation & Control Theory (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Transportation (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Evolutionary Computation (AREA)
  • Game Theory and Decision Science (AREA)
  • Medical Informatics (AREA)
  • Artificial Intelligence (AREA)
  • Emergency Management (AREA)
  • Traffic Control Systems (AREA)
  • Navigation (AREA)

Abstract

A system for monitoring a vehicle used in providing a service is disclosed. The system detects when the service is complete, and upon detecting that the service is complete, the system determines, from one or more sensors of the vehicle, that an object which should have been removed from the vehicle after completion of the transport service, remains in the vehicle. The system then automatically initiates performance of a remedial action.

Description

DETECTING OBJECTS WITHIN A VEHICLE IN CONN ECTION WITH
A SERVICE
RELATED APPLICATION
[0001] This application claims the benefit of U.S. Patent Application No. 14/708,611, filed May 11, 2015, titled DETECTING OBJECTS WITHIN A
VEHICLE IN CONN ECTION WITH A SERVICE; the aforementioned application is being hereby incorporated by reference in its entirety.
BACKGROUND
[0002] A service arrangement system can arrange a service to be provided for a requesting user through the use of mobile computing devices. For example, a user can request a service, such as a transport or delivery service, by operating the user's mobile computing device, and the service arrangement system can process the request to select the appropriate provider that can provide the service. The service arrangement system can also monitor the service, such as to determine the current location of the vehicle, before and during the progress of the service.
BRIEF DESCRIPTION OF THE DRAWINGS
[0003] FIG. 1 illustrates an example system to detect objects within a vehicle in connection with a transport service.
[0004] FIG. 2 illustrates an example method for detecting objects within a vehicle in connection with a transport service.
[0005] FIG. 3 illustrates another example method for detecting objects within a vehicle in connection with a transport service.
[0006] FIGS. 4A through 4D illustrate examples of user interfaces displayed on a mobile computing device in various use case examples.
[0007] FIG. 5 is a block diagram that illustrates a computing device upon which embodiments described herein may be implemented.
[0008] FIG. 6 is a block diagram that illustrates a computer system upon which examples described herein may be implemented. DETAILED DESCRIPTION
[0009] According to examples described herein, a service arrangement system, which implement a network service, can receive requests for location- based services and arrange those services to be provided by service providers. As used herein, service providers can include human drivers (or human-driven vehicles) that receive and/or accept invitations for services, and in some instances, autonomous (or self-driving) vehicles that can be used by the network service to transport goods or people. Whether a human-driven vehicle or an autonomous vehicle (AV) is assigned to provide a service, the network service can use information from a set of sensors of that vehicle to perform processes, when necessary, to supplement the assigned service. For example, a computing system associated with a vehicle can determine, through use of sensor data, whether the service has been properly completed or whether an object has inadvertently been left behind in the vehicle.
[0010] In some examples, a computing system can determine that a vehicle has been arranged to provide a location-based service. Depending on implementation, the computing system can correspond to a computing system associated with a vehicle (e.g., an on-board computing system of the vehicle, or a driver device operated by the driver of the vehicle) or a computing system(s) that implements the network service (e.g., a backend server system remote from the vehicle). During the performance of the service, the
computing system can detect that an object has been placed within the vehicle based on a first set of information from one or more sensors of the vehicle at a first time. As described herein, a sensor is a device that can detect some characteristic of the surrounding environment or detect an event or change in quantity of an amount, and provide a corresponding output (e.g., sensor data). The computing system can detect, at the time the service is being completed or after the service has been completed, whether the object has been removed from the vehicle based on a second set of information from the one or more sensors. In response to detecting that the object is still present in the vehicle (e.g., has not been removed from the vehicle), the computing system can automatically perform a remedial action, such as causing the driver or the vehicle to perform an additional operation that would otherwise not have been performed. [0011] In various examples, a vehicle can include or be equipped with one or more sensors, such as a set of cameras, a set of weighing devices, a set of suspension sensors, a set of light detection and ranging (LIDAR) sensors, a set of ultrasound sensors, a set of location-determination sensors, a set of radio-frequency sensors (such as Bluetooth or Wi-Fi transceivers), as well as other sensors used for various vehicle operations (e.g., engine sensors, tire pressure sensors, door open or close sensors, seat belt sensors, external stereo cameras or LIDAR sensors for AVs, etc.). A set of sensors, as described herein, can correspond to one or more individual sensors or one or more arrays of sensors. Depending on implementation, the computing system can use information (sensor data) from one or more sets of sensors (e.g., an individual sensor or a combination of a plurality of different sensors) to detect whether an object has been placed within the vehicle and/or has been removed from the vehicle.
[0012] For example, the computing system can detect a position of an object inside the vehicle based on image information from a set of cameras that are positioned within the vehicle with the respective lens of the set of cameras facing towards the interior of the vehicle. In another example, the computing system can detect that an object has been placed in or removed from a compartment or the interior of the vehicle based on weight information from a set of weighing devices that are positioned within the compartment or the interior of the vehicle. The set of weighing devices can be positioned in the vehicle such that the respective measuring platform of the set of weighing devices is positioned on, positioned below, or included with a bottom surface of the respective compartment or interior of the vehicle.
[0013] In another example, the computing system can detect that a mobile computing device with wireless communications has been placed in or removed from the vehicle based on if that mobile computing device is communicating with antennas in the vehicle that use wireless protocols such as Bluetooth or Wi-Fi. In another example, if the mobile computing device has its own location sensors, the computing system can compare the output of those location sensors to the output of location sensors on the vehicle to determine if the mobile computing device is moving with the vehicle, and is therefore likely to be inside the vehicle. Still further, in another example, the computing system can detect that an object has been placed in or removed from the compartment or the interior of the vehicle based on compression information from a set of suspension sensors that are positioned proximate to or included in a suspension system of the vehicle.
[0014] If the computing system detects that an object has been placed within the vehicle as part of or during the performance of the on-demand service, and subsequently detects, at the time the service is being completed or after the service has been completed, that the object is still present in the vehicle, the computing system can automatically perform a remedial action, such as communicating a message to a computing device associated with the user that requested the service and/or notifying the service arrangement system that the object is still present in the vehicle. According to some examples, the computing system can also instruct the driver or cause the vehicle to remain stationary at a location of the vehicle when the object was detected to still be present in the vehicle for at least a predetermined duration of time. In another example, the computing system can instruct the driver or cause the vehicle to travel to a predetermined or specific location where the object can be removed from the vehicle.
[0015] According to an example, the computing system can also use sensor data to identify what the object is or what type of object has been placed in the vehicle. Still further, in some examples, the computing system can also use the detection of an object being placed in the vehicle and/or the detection of the object being removed from the vehicle to determine (or as part of determining) when the service has started and/or ended.
[0016] As used herein, a client device, a driver device, and/or a mobile computing device refer to devices corresponding to desktop computers, cellular devices or smartphones, personal digital assistants (PDAs), laptop computers, tablet devices, etc., that can provide network connectivity and processing resources for communicating with a remote computing system(s) over one or more networks, such as a service arrangement system . A remote computing system can refer to one or more computing systems or servers that is remote from the client device, the driver device, and/or the vehicle (e.g., corresponds to the back-end server system of the network service). Still further, in examples described herein, a driver device or a computing system associated with a vehicle can also correspond to custom hardware of the vehicle, such as an in-vehicle or on-board computing system, that has network connectivity and location-determination capabilities.
[0017] Still further, examples described herein relate to a variety of location-based (and/or on-demand) services, such as a transport service, a food truck service, a delivery service, an entertainment service, etc. to be arranged between users and service providers. In other examples, a service arrangement system can be implemented by any entity that provides goods or services for purchase through the use of computing devices and network(s). For purpose of simplicity, in examples described herein, the service
arrangement system can correspond to a transport arrangement system that arranges transport services to be provided for riders/users by drivers or vehicles (e.g., transporting objects or people).
[0018] One or more examples described herein provide that methods, techniques, and actions performed by a computing device are performed programmatically, or as a computer-implemented method. Programmatically, as used herein, means through the use of code or computer-executable instructions. These instructions can be stored in one or more memory resources of the computing device. A programmatically performed step may or may not be automatic.
[0019] One or more examples described herein can be implemented using programmatic modules, engines, or components. A programmatic module, engine, or component can include a program, a sub-routine, a portion of a program, or a software component or a hardware component capable of performing one or more stated tasks or functions. As used herein, a module or component can exist on a hardware component independently of other modules or components. Alternatively, a module or component can be a shared element or process of other modules, programs or machines.
[0020] Some examples described herein can generally require the use of computing devices, including processing and memory resources. For example, one or more examples described herein may be implemented, in whole or in part, on computing devices such as servers, desktop computers, cellular or smartphones, personal digital assistants (e.g., PDAs), laptop computers, printers, digital picture frames, network equipment (e.g., routers) and tablet devices. Memory, processing, and network resources may all be used in connection with the establishment, use, or performance of any example described herein (including with the performance of any method or with the implementation of any system).
[0021] Furthermore, one or more examples described herein may be implemented through the use of instructions that are executable by one or more processors. These instructions may be carried on a computer-readable medium . Machines shown or described with figures below provide examples of processing resources and computer-readable mediums on which instructions for implementing examples described herein can be carried and/or executed. In particular, the numerous machines shown with examples described herein include processor(s) and various forms of memory for holding data and instructions. Examples of computer-readable mediums include permanent memory storage devices, such as hard drives on personal computers or servers. Other examples of computer storage mediums include portable storage units, such as CD or DVD units, flash memory (such as carried on smartphones, multifunctional devices or tablets), and magnetic memory.
Computers, terminals, network enabled devices (e.g., mobile devices, such as cell phones) are all examples of machines and devices that utilize processors, memory, and instructions stored on computer-readable mediums. Additionally, examples may be implemented in the form of computer-programs, or a computer usable carrier medium capable of carrying such a program .
SYSTEM DESCRIPTION
[0022] FIG. 1 illustrates an example system to detect objects within a vehicle in connection with a transport service. Depending on implementation, such a system can be implemented by and operated on a mobile computing device of a driver, an in-vehicle computing system of a vehicle, or a service arrangement system that provides a network service. For purpose of simplicity, in the example of FIG. 1, the system 100 is described as being implemented by and operated on an in-vehicle computing system of a vehicle, such as an AV, through execution of instructions stored in one or more memory resources of the in-vehicle computing system. As described herein, the system 100 can include, be a part of, or correspond to an application or an operating system of the in-vehicle computing system that communicates with a service arrangement system 190 over one or more networks. [0023] According to examples, the system 100 can include a system manage 110, a location determination 120, an object detection 130, a location database 140, a trip database 150, and a service interface 160. For purpose of simplicity, other components of the system 100, such as a user interface component, other databases, device interfaces to communicate with device components, etc., are not illustrated in FIG. 1. The components of the system 100 can combine to detect objects that are placed within and/or removed from a vehicle in connection with transport services. Logic can be implemented with various applications (e.g., software) and/or with hardware of the computing system that implements the system 100.
[0024] In one example, the system 100 can communicate with the service arrangement system 190 via the service interface 160. As described herein, the service arrangement system 190 can communicate with a plurality of service providers (e.g., driver devices and/or in-vehicle computing systems) and a plurality of client (or user) devices in order to arrange transport services between users and service providers. The service arrangement system 190 can be implemented on network side resources, such as on one or more servers or data centers, or implemented through other computer systems in alternative architectures (e.g., peer-to-peer networks, etc.). Typically, a user can operate a client or user device (not shown in FIG. 1 for simplicity) to communicate with the service arrangement system 190 over a network(s) and make a request for a transport service at a specified location (e.g., a pickup location of the package or a person to be transported). In some examples, the user device can run a designated service application that communicates with the service arrangement system 190. The service arrangement system 190 can receive the request and programmatically select a service provider to provide the transport service.
[0025] Depending on implementation, if the service arrangement system 190 selects a human driver to provide the service, the service arrangement system 190 can transmit an invitation to the selected driver's device, thereby providing the driver with an option to accept or reject the invitation to provide the transport service for the requesting rider. In another example, if the service arrangement system 190 selects an AV from a plurality of available AVs (and/or human drivers) to provide the service, such as described in FIG. 1, the service arrangement system 190 can transmit information about the service (referred to as service instructions 191) to the system 100 via the service interface 160.
[0026] According to examples, the service interface 160 enables the system 100 to exchange data between the system 100 and the service arrangement system 190. For example, the service interface 160 can use one or more network resources of the in-vehicle computing system to exchange communications over one or more wireless networks (e.g., a cellular
transceiver, a WLAN transceiver, etc.). The service interface 160 can include or use an application programming interface (API), such as an externally facing API, to communicate data with the service arrangement system 190. The externally facing API can provide access to the service arrangement system 190 via secure access channels over the network through any number of methods, such as web-based forms, programmatic access via restful APIs, Simple Object Access Protocol (SOAP), remote procedure call (RPC), scripting access, etc.
[0027] In the example of FIG. 1, the service arrangement system 190 has processed a request for a transport service by a user and has selected the AV that implements the system 100 to provide the transport service for the requesting user. The service arrangement system 190 may have determined, based on the pickup location specified by the requesting user and the current locations (or most recently determined locations) of available service providers nearby the pickup location (e.g., within a predetermined distance of the pickup location), that the AV implementing the system 100 is to provide the transport service. The service arrangement system 190 can transmit service instructions 191 about the transport service to instruct the vehicle about where to go and/or what to do. In some examples, the service instructions 191 can include user information 192 (e.g., a user identifier, contact information of the user, a user's rating, etc.), pickup location 193 specified by the user (e.g., an address, a landmark, a location data point corresponding to a latitude and longitude coordinate), destination location 194, and/or other information, such as the type of transport service (e.g., transport a person, deliver a package, etc.) or details about the number of people or number of packages. The service instructions 191 can, in one example, cause the vehicle controlling
component(s) of the AV to control, at least in part, the AV to travel to the appropriate locations. In other words, the vehicle control component(s) can use some or all of the service instructions 191 (in addition to other
information) to control the vehicle to move to the pickup location and/or the destination or drop off location.
[0028] The system manage 110 can receive the service instructions 191 and in response, can perform a plurality of operations on behalf of the system 100. For example, the system manage 110 can process the service
instructions 191 by determining information about the transport service it is to perform, and can initiate the performance of the transport service by determining the route(s) to travel to the pickup location and/or the destination location and controlling the AV to travel to the pickup location and/or the destination location.
[0029] In one example, a trip manage component of the system manage 110 can create a trip entry 151 for the transport service using some or all of the information from the service instructions 191 and store the trip entry 151 in a trip database 150 of the system 100. The trip manage component can then update the trip entry 151 with information about the vehicle's location as the vehicle performs the transport service (e.g., travels to the pickup location from the current location, picks up the user or an object, travels to the drop off location, etc.). The trip manage component can also record, in the trip entry 151, the time and the location of the vehicle when the service
instructions 191 were received, when the service started, and when the service ended. The trip entry 151 can be used by the system 100 and/or the service arrangement system 190 to determine the cost for the transport service for the requesting user (e.g., after completion) and for maintaining a historical record of the AV and/or the requesting user.
[0030] The location determination 120 can provide the system manage 110 with information about the vehicle's location by receiving location data points 121 (e.g., periodically) from one or more geo-aware resources of the in-vehicle computing system. For example, the location determination 120 can periodically determine the current location of the vehicle by communicating with, e.g., via a device interface, the GPS receiver of the driver device, one or more other sensors, and/or one or more transceivers (e.g., cellular
transceiver, Wi-Fi transceiver) for purpose of determining the current location data point 121 of the vehicle. In one example, the location determination 120 can make a call to the GPS receiver periodically, such as every two seconds or four seconds, etc., to periodically receive a current location data point 121 of the driver device. The location determination 120 can also determine the associated timestamp of each determined location data point 121 of the vehicle. Depending on implementation, the location determination 120 can receive a timestamp (and/or an error value) along with the location data point 121 of the vehicle from the GPS receiver or can determine the timestamp of the location data point 121 using a separate clock (e.g., an internal clock of the driver device). The system manage 110 can store the location data points 121 in the location database 140 and/or the trip entry 151 to maintain data about where the vehicle traveled in connection with transport services. As an addition or an alternative, the location determination 120 can store the location data points 121 of the vehicle in the location database 140.
[0031] Referring back to the system manage 110, the trip manage component can also use the location data points 121 from the location determination 120 to monitor the vehicle. The trip manage component can determine the position and the state of the vehicle or system 100 in connection with the performance of the transport service (e.g., what stage of the transport service the vehicle is at). For example, the state of the system 100 may previously have been in an available state, which corresponds to a state in which the vehicle can be assigned the transport service from the service arrangement system 190. When the system 100 receives the service instructions 191, the trip manage component can place the system 100 in a different state, e.g., on route state, which corresponds to the vehicle traveling to the pickup location. Similarly, each time the trip manage component determines that the transport service has initiated or ended, the trip manage component can place the system 100 in a respective state (e.g., change from the on route state to the on trip state, or change from the on trip state to completed or available state, respectively). The trip manage component can store information about the state and/or when the state change occurred along with the corresponding location data points 121 in the trip entry 151 of the transport service.
[0032] Still further, as an addition or an alternative, the service arrangement system 190 can locally generate and store its own trip entry that corresponds to or is similar to the trip entry 151 stored in the system 100. The trip manage component can periodically provide the above-described information about the transport service to the service arrangement system 190, e.g., via the state information 111 and/or the location data points 113. The service arrangement system 190 can use the received information from the system 100 to provide updated information about the transport service to the requesting user (e.g., the location of the AV and/or the current state of the AV, etc.). In addition, the service arrangement system 190 can determine, from the state information 111 and/or the location data points 113, when the transport service has initiated or completed and/or when the AV is available to provide another transport service.
[0033] In addition to creating and updating a trip entry 151 for the transport service, the system manage 110 can initiate the performance of the transport service based, at least in part, on the service instructions 191.
According to some examples, a vehicle control component of the system manage 110 can control the AV to perform the transport service. Depending on implementation, the vehicle control component can interface with the AV control system or can correspond to the AV control system. As described herein, an AV includes (i) computing resources that implement the system 100 and/or the AV control system, (ii) other sub-components or sub-subsystems to control various other operations or parts of the AV, and (iii) a plurality of vehicle parts. For purpose of simplicity, not all parts of an AV are discussed herein.
[0034] According to examples, various parts of an AV can include, but are not limited to, a frame or a chassis, a body (e.g., including doors, bumpers, windows, a cab, seats, a trunk space, etc.), tires, a suspension system, a braking system, a steering system, an electrical system (e.g., including batteries, wires, cables, alarm sub-system, etc.), a transmission system (e.g., including a drive train, engine, etc.), a fuel system, an exhaust system, a cooling system, and an AV control system . The AV can also include electronic control units (ECUs), which include controllers and/or
microcontrollers, and other electronic components that can control the various systems or sub-systems of the AV, such as a door control unit, an engine control unit, a speed control unit, a transmission control unit, etc. The AV control system can be programmed, through execution of instructions stored in memory by one or more processors, to control the AV by sending control signals and by receiving data (e.g., feedback or sensor data) from the various systems and vehicle parts. For purposes of this application, the AV control system can be instructed to control the AV so that it can safely navigate roads and streets to move from one location to another in connection with transport services.
[0035] Still further, the AV can include a location determination module or resources, such as a GPS receiver and/or wireless transceivers, to
determine the location of the AV relative to other objects or in reference to a coordinate system . The AV can also include a plurality of sensors, such as a set of cameras, a set of stereo cameras, a set of radars, a set of LIDAR sensors, etc. to enable the AV to detect the real-time environment of the AV. One or more (or all) of these sensors can be positioned on the exterior of the vehicle (e.g., on the front, rear, roof of the vehicle, etc.) to detect information about the environment around the AV, such as the position of the vehicle relative to roads, building, other vehicles, etc., or the position of inanimate objects. For example, in order for the AV to travel from a current position or location (e.g., determined from the GPS receiver as a location data point 121) to a specified location (e.g., the pickup location 193 and/or destination location 194 indicated in the service instructions 191), the AV control system can access a stored detailed spatial map(s) of the region it is in (e.g., including map data 112) and determine the route(s) to travel to the specified location. Based on the detected environment surrounding the AV (from data periodically and continuously detected by the set of sensors), the AV control system can periodically perform decision making operations that instruct the AV to take a certain action(s) to move to the specified location. As a result of each decision making operation, the AV control system can determine that it is to travel in a certain direction at a certain speed at one instance in time, for example, and at a next instance in time, can determine that it is to decelerate to a certain speed, change bearings/directions slightly, or change lanes, etc. The AV control system can provide control signals to the various vehicle components/systems accordingly.
[0036] In one example, when the AV travels to the pickup location to initiate the transport service, the system 100 can periodically determine whether the AV has reached the specified pickup location (e.g., start location). In other words, the system 100 can periodically check when the AV arrives at or is within a predetermined distance (or is within a predetermined estimated travel time) from the pickup location. For example, the trip manage
component can periodically compare the current location (e.g., the latest location data point 121 provided by the location determination 120) with the pickup location 193 and determine whether the AV is within the predetermined distance or estimated travel time. When the trip manage component
determines that the AV is within the predetermined distance or estimated travel time, the trip manage component can change the state of the system 100 from the on route state to the arriving/has arrived state. The state information 111 about this change can be transmitted to the service
arrangement system 190 so that the service arrangement system 190 can provide updated information about the AV to the requesting user's device.
[0037] In addition, according to an example of FIG. 1, the system 100 can also determine whether an object(s) has been placed in the vehicle in connection with the transport service. In some instances, a driver or an AV control system may need to determine if a requesting user has placed an object in the vehicle while being transported (e.g., luggage, boxes, groceries) or determine if the object has been placed in the vehicle for a delivery transport service. As described herein, an object can correspond to an item(s) or a person(s) that is placed within or enters a vehicle. The object detection 130 can communicate with one or more sets of sensors or one or more combinations of sensors of the AV, via one or more vehicle or sensor interfaces, to receive sensor data 131, and based on the sensor data 131, can determine whether an object has been placed in (or removed from) the AV (referred to herein as an object detection process or operation). In one example, the object detection 130 can transmit a set of control signals to the sensors to cause (or trigger) the sensors to provide the sensor data 131. Such sensors can include a set of cameras, a set of weighing devices, a set of suspension sensors, a set of light detection and ranging (LIDAR) sensors, a set of radio-frequency antennas and/or transceivers, and/or a set of ultrasound sensors that are positioned in or provided with the vehicle to detect the presence of, the position of, and/or the weight of one or more objects.
[0038] In some examples, the object detection 130 can perform an object detection process by (i) receiving sensor data 131 from one or more sensors, (ii) processing the received sensor data 131 using other information (e.g., models or previously stored data), and (iii) determining whether an object has been placed in the AV (when it was previously not in the AV) or determining whether an object has been removed from the AV (when it was previously in the AV). For example, the object detection 130 can receive image information from one or more cameras positioned within a trunk of the AV (e.g., with the lens facing the inside of the trunk), and process the image information by using image processing or image recognition processes and/or by comparing the image information to previously stored baseline image information about the trunk (e.g., showing the image of the trunk when it was empty or the image of the trunk before receiving the service instructions 191). The stored baseline image information can be a predefined image or can be an image captured by the one or more cameras at a time before the AV arrived at or near the pickup location 193. Still further, in one example, the object detection 130 can use image processing (as well as other stored or received data from the service arrangement system 190) to identify what the object is based on visual characteristics of the object in the image.
[0039] In other examples, when one or more other sensors are used for detecting the presence or position of an object, the object detection 130 can use other baseline or model information as a basis for comparison with received sensor data 131. For example, the object detection 130 can receive sensor data 131 from one or more weighing devices or sensors that are positioned in a compartment of the AV such that its respective measuring platform is on, positioned below, or included with a bottom surface of such compartment. The object detection 130 can determine a change in the measurements from the sensor data 131 at a time before the AV arrived at or near the pickup location 193 and after the service has started.
[0040] Depending on implementation, the object detection 130 can perform the object detection process at various times. For example, the object detection 130 can concurrently perform the object detection process as the AV travels to the pickup location 193 and/or as the trip manage component determines whether the AV is within the predetermined distance or
predetermined estimated travel time of the pickup location 193. The object detection 130 can make calls to the set of sensors to receive sensor data 131 periodically. In another example, the object detection 130 can perform the object detection process at various instances when instructed or triggered by the system manage 110 (via trigger 115). For example, when the trip manage component determines that the AV is within the predetermined distance or predetermined estimated travel time of the pickup location 193, the trip manage component can transmit a trigger 115 to cause the object detection 130 to perform the object detection process. Still further, as another variation, the trip manage component can transmit the trigger 115 when it determines that the AV is stationary for a predetermined duration of time and/or when it determines that the AV is stationary for a predetermined duration of time while it the AV is within the predetermined distance or predetermined estimated travel time of the pickup location 193. As another example, the object detection 130 can be triggered to perform the object detection process when the trip manage component determines that the transport service has started. By causing the object detection 130 to perform the object detection process at select times, the system 100 can reduce excessive consumption of energy and computing resources by causing the sensor(s) to wake up (from a suspended or sleep state) and detect or perform a measurement only when necessary.
[0041] If the object detection 130 detects the position of an object or detects that an object has been placed within the vehicle based on a first set of sensor data 131, the object detection 130 can provide object information 133 to the system manage 110. The object information 133 can indicate to the system manage 110 that an object has been placed in the AV (e.g., a message, or a binary bit can be toggled) and/or can include information about that object based on the types of sensor(s) that is used by the AV to perform the detection. For example, the additional information can include (i) what the object is (e.g., a box, a bag, a skateboard, etc.), (ii) one or more images of the object captured by the set of cameras, if cameras are used, (iii)
measurements pertaining to the object (e.g., a weight, if weighing devices or suspension sensors are used, or dimensions of the object, if cameras, LIDAR sensors or ultrasound sensors are used, etc.), and/or (iv) where the object is located (e.g., what compartment, or the position of the object). The trip manage component can store some or all of the object information 133 with the trip entry 151 associated with the transport service, so that information about the object can be logged and associated with the particular user. [0042] As described, concurrently (or before or after) the object detection 130 detects an object, the trip manage component can determine that the transport service has begun. In some examples, the system manage 110 can determine that the transport service has started, based, at least in part, on detecting that an object has been placed in the vehicle. The transport service can correspond to transporting a person (and/or the person's things) from one location to another, or transporting an object for delivery from one location to another. The object detection 130 can detect, from one or more sensors, such as a camera or weighing devices, if a person and/or objects have entered the vehicle or if an object (without the person) has been placed in the vehicle. In this example, the set of cameras (or LIDAR sensors or ultrasound sensors) can be positioned inside the cab/interior of the vehicle to enable the object detection 130 to detect that the person has entered the vehicle using image information from the camera(s) and processing the image(s). As an addition or an alternative, a set of weighing devices or a set of suspension sensors that is positioned proximate to or included in a suspension system of the AV can determine that an amount of mass has been added to the vehicle.
[0043] Once the transport service has started, the trip manage component can store information about the performance of the transport service (e.g., the time and/or the location of travel of the AV), including the start time and start location of the transport service, in the trip entry 151. As the AV travels to the destination location 194 of the transport service, the trip manage component can also use the location data points 121 to determine when the AV approaches and has approached the destination location 194, for purpose of determined, at least in part, that the transport service has completed.
[0044] In one example, in instances when the object detection 130 previously detected an object in connection with the transport service (e.g., the rider placed a bag in the trunk of the AV), the trip manage component can cause the object detection 130 to also detect whether the object has been removed from the vehicle. The object detection 130 can detect whether the object has been removed from the vehicle at different times, depending on variations. According to one example, the system manage 110 can cause the object detection 130 to concurrently perform the object detection process as the AV travels to the destination location 194 (e.g., periodically), or perform the object detection process when the trip management component
determines that the AV is within a predetermined distance or predetermined estimated travel time of the destination location 194. As an addition or an alternative, the trip manage component can cause the object detection 130 to perform the object detection process when the trip manage component determines that the AV is stationary for a predetermined duration of time, when the trip manage component determines that the AV is stationary for a predetermined duration of time while it the AV is within the predetermined distance or predetermined estimated travel time of the destination location 194, or in response to determining that the transport service has been completed.
[0045] When the trip manage component determines that the transport service has been completed, for example, the object detection 130 can perform the object detection process to determine if the object has been removed using sensor data 131 received from the set of sensors (e.g., a second set of sensor data 131). If the object detection 130 detects that the object is still present, it can provide information to the system manage 110 indicating that the object is still present in the vehicle. In response, the system manage 110 can automatically perform a remedial action. On the other hand, if the object detection 130 detects that the object has been removed, no remedial process or action is taken. In one variation, if the system 100 detects that the object has been removed (e.g., in the case of a delivery transport service), the system 100 can determine that the delivery transport service has been completed.
[0046] As described herein, a remedial action can correspond to an operation (or set of operations) the system 100 performs to return whatever object is accidentally left behind to the user or to make it easier for the user to retrieve the object. In various implementations, the remedial action(s) can be performed by the system 100 and/or performed by the system 100 in connection with the service arrangement system 190. In the latter example, the remedial action(s) can be performed as a result of the exchange of data between the system 100 and the service arrangement system 190.
[0047] In one example, the system manage 110 can receive information that the object is still remaining in the vehicle and/or information about the object itself (e.g., based on recently determined information about the object, or from object information 133 previously received when the object was previously detected) and can generate an object message 117 based on such information. The object message 117 can include, for example, a time and/or a location when the transport service was completed, an identifier of the AV, an identifier of the user, an identifier of the transport service, and information about the object (e.g., what the object is, one or more images of the object, measurements pertaining to the object, where the object is located). The system manage 110 can determine a communication identifier for the user (e.g., a phone number or email address of the user or temporarily associated with the user's account), and transmit the object message 117 to the user's computing device. Such an object message 117 can also include notification information to cause the user's computing device to display content to inform the user that the object has been left behind in the AV and/or to contact the entity that provides the service arrangement system 190 (e.g., via the designated application running on the user's computing device). As an addition or an alternative, an object message 117 can be transmitted to the service arrangement system 190, which can then transmit a message, including notification information, to the user's computing device. The notification information displayed to the user can provide information about the object and/or how the user can retrieve the object. Examples of different notification information is illustrated in FIGS. 4A through 4D.
[0048] As an addition or an alternative, the system manage 110 can also perform one or more remedial actions by positioning the AV at a particular location (e.g., by causing the AV to move to a particular location or keeping stationary at a particular location). For example, when a remedial action is to be performed, the system manage 110 can access stored positioning instructions that instruct the vehicle control component to control, via the vehicle components, the AV to remain stationary at (i) the destination location 194, or (ii) the location of the vehicle when the object detection 130
determined that the object was still present in the vehicle. By causing the AV to remain stationary at a specified location, the object message 117 can inform the user that the user's object has been left behind in the AV, and that the AV will remain at the specified location for a predetermined duration of time (e.g., two minutes). Alternatively, the stored positioning instructions can instruct the vehicle control component to cause the AV to travel to a specified location (e.g., a storage location or office building where lost objects are stored), and the object message 117 can inform the user that the object can be retrieved at the specified location during certain times.
[0049] In some examples where the AV travels to a specified location, such as a storage location or a disposal location, the AV can use automatically- controlled actuators to remove the object from the AV for storage or disposal. For example, the AV itself can include one or more actuators coupled to one or more mechanical hands in the vehicle's interior or trunk that can grab an object and remove it. In another example, the AV can move to a specified lane or spot at the storage or disposal location, open the vehicle doors and/or trunk, and an automatically-controlled mechanical arm and hand or claw can remove the object from the vehicle.
[0050] Still further, in another example, the system manage 110 can provide information to the service arrangement system 190 indicating that an object has been left behind, and in response, the service arrangement system 190 can transmit instructions to the system 100 to cause the AV to perform one or more tasks. In one implementation, the service arrangement system 190 can transmit vehicle instructions to the system manage 110 that causes the vehicle control component to control the movement and operation of the AV. The vehicle instructions can cause the AV to move to a specified location or remain stationary at a specified location .
[0051] In some instances, after dropping off the user at the destination location 194, the AV may be arranged/selected to provide a transport service to another requesting user. As a result, the system 100 may perform the remedial action, such as transmitting the object message 117, but also cause the AV to travel to the pickup location specified by the subsequent requesting user. In some examples, the service arrangement system 190 (or the system 100) can transmit a message to the user's computing device, which queries the user for an response input to (i) instruct the disposal of the object, (ii) ask the entity that operates the service arrangement system 190 to hold on to the object on behalf of the user for a duration of time, (iii) request a transport service to a specified location where the object is held, or (iv) request a delivery of the object to the user's current location (or specified location). One or more of these options can be associated with an extra cost for the user. The service arrangement system 190 can receive the user input, via the
application on the user's computing device, and can process the response accordingly. For example, if the user provides an input to receive the delivery of the object at the user's current location, the service arrangement system 190 can process the response as a delivery transport service request, identify the previous AV that the object is located in, and schedule that particular AV to travel to the user's current location (e.g., before, while, or after completing the subsequent transport service for the subsequent requesting user). In this example, the user can be charged for the delivery as a separate transport service in its entirety or as part of the user's previous transport service (e.g., as a surcharge).
[0052] While the system 100 of FIG. 1 is described as being
implemented on computing and memory resources of an autonomous vehicle, in some examples, the system 100 can be implemented on a mobile computing device operated by a driver of a vehicle (e.g., a driver device). For example, the system 100 can be implemented as an application or program (or as part of an application) that runs on the driver device, and that communicates data with the service arrangement system 190 over one or more networks. In such examples, the system 100 can include other components, such as a user interface component that generates and presents content (e.g., graphic user interfaces) on the display of the driver device. When the driver is selected by the service arrangement system 190 to provide a transport service for a requesting user, as opposed to receiving service instructions 191, the system 100 can receive and display an invitation to provide the transport service via the user interface component, and the driver can either accept or reject the invitation. The invitation can include the user information 192, the pickup location 193, and/or the destination location 194.
[0053] Still further, in examples where the system 100 is implemented by the driver device, the object detection 130 can receive sensor data 131 via a communication interface of the driver device, such as a short-range wireless communication transceiver (e.g., Bluetooth or Wi-Fi) or a physical connector port. The driver can fix or attach, in the trunk or in the backseat of the vehicle, etc., one or more sensors which can communicate sensor data 131 to the object detection 130 via the communication interface. [0054] As an alternative, one or more components of the system 100 can be implemented as part of the service arrangement system 190 or as part of another remote system (e.g., remote from the autonomous vehicle or the driver device) that communicates with the service arrangement system 190. For example, the system 100 can transmit sensor data 131 to an object detection 130 that is implemented in the service arrangement system 190, which then detects when an object has been placed in a vehicle and/or has been taken out of the vehicle. The service arrangement system 190 can then perform the remedial actions, such as transmitting notifications to the user, based on the detection.
[0055] Still further, in some examples, other sensors can be used by the system 100 to detect a presence and/or removal of an object from a vehicle. For example, electromagnetic sensors can be used to detect electronic products, such as cell phones, tablets, laptop computers, etc., or other advanced sensors, such as chemosensors or gas chromatography sensors can be used to detect objects using smell (e.g., food, groceries, etc.). In one example, using electromagnetic sensors or antennas/wireless transceivers can be helpful to detect small electronic devices that may be hard to detect in the vehicle, e.g., may have fallen or slipped between the seats or under the seat.
[0056] In a different application, the system 100 can also use the sensor data to detect whether there has been damage to the interior of the vehicle as a result of the requesting user riding in the vehicle and/or transporting an object of the user. The system 100 can detect the state of the vehicle's interior at a time before the transport service is initiated (e.g., in response to when the service instructions are received or at a predetermined distance or estimated time away from the pickup location of the user) using sensors that are positioned in the interior of the vehicle. At a time the transport service is being completed (e.g., the vehicle is close to or has arrived at the destination location) or has completed, the system 100 can again detect the state of the vehicle's interior at this time, and compare the information with the previous information about the state of the vehicle's interior. Using this comparison, the system 100 can detect whether something has spilled on the seats or the floor, whether the seats have been ripped or scratched, etc., after the transport service. If damage is detected, the system 100 can provide information indicating the damage to the service arrangement system 190 and/or to the user device. The service arrangement system 190 can charge the user and/or instruct the driver or cause the vehicle to travel to a specified location for clean-up and/or repair.
M ETHODOLOGY
[0057] FIGS. 2 and 3 illustrate example methods for detecting objects within a vehicle in connection with a transport service. Methods such as described by examples of FIGS. 2 and 3 can be implemented using, for example, components described with an example of FIG. 1. Accordingly, references made to elements of FIG. 1 are for purposes of illustrating a suitable element or component for performing a step or sub-step being described. For purpose of simplicity, the example method of FIG. 2 is described as being performed by the service arrangement system 190 and/or the system 100 operating on a driver device.
[0058] Referring to FIG. 2, the service arrangement system 190 can determine that a driver of a vehicle or an AV been selected to provide a service for a requesting user (200). For example, the service arrangement system 190 can perform a vehicle selection process to select the vehicle based on the location of the vehicle and the service location(s) specified by the requesting user. The service can correspond to an object or item delivery transport service from a pickup location to a destination location, or a transport service of a person (e.g., the user) from a pickup location to a destination location. The service arrangement system 190 can transmit, to the respective system 100 of the selected vehicle, service instructions
corresponding to the service that the driver or the AV is to perform . In the example of an AV, the system 100 can process the service instructions to control the AV to travel from the current location to the pickup location, while in the example of a human driver, the system 100 can provide route or navigation information that the driver can follow to move to the pickup location.
[0059] At a time after the system 100 determines that the driver or the AV is to provide the service, the system 100 can perform an object detection operation using data from one or more sensors of the vehicle. In one example, the service arrangement system 190 can receive a first set of sensor information from the system 100 (e.g., at a first time) and can detect that an object has been placed within the vehicle based on the first set of information (210). The system 100 can receive the sensor information from a set of sensors of the vehicle. Alternatively, the system 100 can detect that the object has been placed within the vehicle and provide information indicating such object to the service arrangement system 190.
[0060] Depending on implementation, one or more sensors or
combination of sensors can be used by the system 100 and/or the service arrangement system 190 to detect that an object has been placed in the vehicle. For example, the system 100 and/or the service arrangement system 190 can receive image information from a set of cameras (or radars, LIDARs, or ultrasound sensors) that are positioned within the vehicle, where the set of cameras are positioned such that the respective lens of the set of cameras faces towards an interior of the vehicle (e.g., the back seat, the front seat, the trunk, or other compartment). In another example, the system 100 and/or the service arrangement system 190 can receive weight information from a set of weighing devices that are positioned within a compartment or the interior of the vehicle, where the set of weighing devices are positioned such that the respective measuring platform of the set of weighing devices is positioned on, positioned below, or included with a bottom surface of the compartment or the interior of the vehicle (e.g., the platform is at the surface of the trunk to weigh objects that are placed in the trunk or at or in a seat to weigh a person that sits in the back set or front seat). Still further, in another example, the system 100 and/or the service arrangement system 190 can receive compression information from a set of suspension sensors that is positioned proximate to or included in a suspension system of the vehicle (e.g., near the rear and/or front suspension system or frame of the vehicle). Additionally, a sensor can also provide information whether a compartment or trunk has been opened and/or closed, which can be used in connection with other sensor information.
[0061] The system 100 and/or the service arrangement system 190 can use information from one or more of these different sources to perform the object detection operation. In addition, based on the first set of received sensor input/information, the system 100 and/or the service arrangement system 190 can detect that the object has been placed in the vehicle, what the object is (or type of object), the color or size or dimension or weight of the object, what compartment the object is in, and/or the position of the object. According to an example, the service arrangement system 190 can also associate the object information with the trip entry of the transport service (and/or with the user's profile or account).
[0062] In various examples, the system 100 can trigger the sensors to determine sensor data (e.g., perform detection or measurement) based on location information of the AV. For example, in order to conserve energy or power, the system 100 can trigger the sensors to determine sensor data when the AV arrives at or close to a service location, e.g., arrives at or within a predetermined distance or estimated travel time of the pickup location, and/or when the AV is stationary for a duration of time while at or close to the service location. Alternatively, in another example, the system 100 can trigger the sensors to determine sensor data in response to determining that the service has begun.
[0063] Referring back to FIG. 2, the service arrangement system 190 can also determine that the service has begun (220). Depending on
implementation, the service arrangement system 190 can determine that the service has begun (i) in response to detecting that the user has entered the vehicle (e.g., in response to detecting the object in the vehicle in step 210, or based on location data points of the user's computing device and the location data points of the vehicle, and determining that they are within a
predetermined distance of each other), (ii) in response to detecting that a predetermined duration of time has elapsed (e.g., ten seconds) since detecting that the user has entered the vehicle, (iii) in response to receiving, from the user's computing device, information indicating that the service can begin as a result of an input being provided at the computing device associated with the user (e.g., the user selects a feature corresponding to starting the trip), (iv) in response to receiving, from driver device (or the on-board computing system of the AV), information indicating that the service can begin as a result of an input being provided at the driver device or the on-board computing system from the driver or the user, and/or (v) in response to determining that the vehicle has been positioned within a predetermined distance of the start location for a predetermined duration of time based on location information of the vehicle. In other examples, the system 100 detects that the service has begun based on any of the examples (i) through (v). [0064] The service arrangement system 190 also monitors the service, for example, as the driver or the AV performs the service (230). As described herein, monitoring the service can include periodically determining the location (and corresponding timestamp) and/or state of the vehicle as it travels from the pickup location to the destination location, and storing such information in the trip entry associated with the service. The system 100 can provide such information to the service arrangement system 190. In some examples, the service arrangement system 190 can also provide the information about the progress of the service to the user's computing device (or to another device). For example, the user may have requested a delivery service for a package to be picked up at the user's location and dropped off at a destination location for a friend. The application on the user's device can receive (periodically) information about the progress of the service, e.g., the AV's location and movement on a map, from the service arrangement system 190. As an addition or an alternative, the user may have provided, to the service arrangement system 190 via the application, the contact information (e.g., phone number or email address) of the friend who is to receive the package. The service arrangement system 190 can monitor the service and provide updated information about the service to the friend, so that the friend can track the package as the AV travels to the destination location.
[0065] The service arrangement system 190 can determine that the service has been completed, e.g., as a result of monitoring the service (240). Depending on implementation, the service arrangement system 190 can determine that the service has completed (i) in response to detecting that the user has left the vehicle (e.g., based on location data points of the user's computing device and the location data points of the vehicle, and determining that they are separated by more than a predetermined distance), (ii) in response to detecting that a predetermined duration of time has elapsed (e.g., twenty seconds) since detecting that the user has left the vehicle, (iii) in response to receiving, from the user's computing device, information
indicating that the service has completed as a result of an input being provided at the computing device associated with the user (e.g., the user selects a feature corresponding to ending the trip), (iv) in response to receiving, from driver device (or the on-board computing system of the AV), information indicating that the service has completed as a result of an input being provided at the driver device or the on-board computing system from the driver or the user, and/or (v) in response to determining that the vehicle has been positioned within a predetermined distance of the destination location for a predetermined duration of time based on location information of the vehicle.
[0066] After determining that the service has been completed, the service arrangement system 190 can detect whether the object (that was previously detected in the vehicle) has been removed from the vehicle based on a second set of sensor information from one or more sensors of the vehicle (e.g., at a second time) (250). For example, the service arrangement system 190 can receive the second set of sensor information from the system 100 and can compare the first and second sets of sensor information to determine if there is a difference, e.g., difference in imagery, difference in weight, difference in compression information, etc. As an alternative, the system 100 can perform the comparison operation and provide an indication of whether the object has been removed to the service arrangement system 190. The system 100 can trigger the sensors to determine sensor data (e.g., perform detection or measurement) at a particular instance in time, depending on examples, such as in response to detecting that the service has been completed, or after a predetermined duration of time since detecting that the service has been completed.
[0067] Alternatively, the service arrangement system 190 can determine whether the object has been removed from the vehicle before the service is determined to be completed. For example, the service arrangement system 190 (or the system 100) can perform the object detection process in response to determining that the AV has reached or is close to the destination location and/or has been stationary for a duration of time while at or close to the destination location. In another example, the service arrangement system 190 can determine that the service has been completed in response to detecting that the object has been removed from the vehicle (e.g., after performing step 250).
[0068] If the service arrangement system 190 determines that the object has been removed from the vehicle, it can determine that no remedial action needs to be performed, as the user in the vehicle has taken the object with him or her, or another user has removed the object (e.g., at the delivery destination location) (270). On the other hand, if the service arrangement system 190 determines that the object has not been removed, the service arrangement system 190 can automatically perform one or more remedial actions (260). For example, the service arrangement system 190 can message the driver and/or the user, via the respective applications on the driver device and/or the user device, to inform them about the misplaced object. The message can include instructions (e.g., including navigation instructions) for enabling the driver and the user to meet up at a location, e.g., the destination location, to enable the user to retrieve the object.
[0069] In another example, the service arrangement system 190 can transmit a message or notification to the user device and enable the user to request a delivery of the object to a location specified by the user. Still further, in one example, if the vehicle is an AV, the service arrangement system 190 can transmit vehicle control instructions to the AV to cause the AV to travel to a specified location. Such a specified location can correspond to a new meeting location (delivery drop off location) for the user specified in the user's subsequent delivery transport service request, or a designated storage or warehouse location to house misplaced objects. The service arrangement system 190 can also message the user device to provide such location information on where the user can retrieve the object.
[0070] FIG. 3 illustrates another example method for detecting objects within a vehicle in connection with a transport service. In one example, one or more steps of the method described in FIG. 3 can be performed concurrently with one or more steps of the method described in FIG. 2. For purpose of simplicity, the example method of FIG. 3 is described as being performed by the system 100 operating on an on-board computing system of an
autonomous vehicle.
[0071] In FIG. 3, the system 100 (of the AV) can receive service instructions from the service arrangement system 190 for the AV to provide a service for a requesting user (300). As described herein, a service can have a start location and a destination location, and can correspond to a delivery of an object or a person (e.g., the requesting user). The system 100 can process the service instructions in order to determine what service to provide, where to travel to (e.g., from the current location, to the start/pickup location, then to the destination location), and how to travel there (e.g., the route to take).
[0072] Based on the service instructions, map data, and/or sensor data used for operating the AV (e.g., data from sensors mounted on the body of the AV), the system 100 can control the AV to travel to the start location, and concurrently, monitor the movement of the AV (310). The system 100 can correspond to, include, or be in communication with the AV control system to control the operation of the AV by sending control signals to vehicle control modules of various sub-systems in response to receiving sensor data from the sensors and feedback signals from various vehicle components. The system 100 can also determine the location of the AV from GPS data, for example, and monitor the current position as compared to the route and/or the start location. Using the periodically determined current location of the AV, the system 100 can determine when the AV arrives at or is within a predetermined distance of the start location (e.g., the AV is close to the start location). In one example, when the system 100 determines that the AV is close to the start location, the system 100 can change the state of the AV (from the on route state to the arriving now or has arrived state).
[0073] At a time after determining that the AV is close to the start location, the system 100 can perform an object detection process to detect that an object has been placed within the AV (320). Depending on
implementation, the system 100 can periodically perform the object detection process, as described in FIGS. 1 and 2, from the time after determining that the AV is close to the start location to a time after the AV begins traveling to the destination location. In one example, the system 100 can perform the object detection process in response to determining that the service has started or can perform the object detection process before or concurrently while determining that the service has started. In the example of FIG. 3, the system 100 can detect that that an object (e.g., a box) has been placed in the back seat of the AV from sensor information received from the AV, as described in FIGS. 1 and 2.
[0074] Once the service has started, the system 100 can control the AV to travel to the destination location, and also monitor the vehicle and/or the progress of the service (330). Similarly as described above, the system 100 can determine the location of the AV from GPS data and monitor the current position as compared to the route and/or the destination location. The system 100 can determine when the AV arrives at or is within a predetermined distance of the destination location (e.g., the AV is close to the destination location) (340). At a time after determining that the AV is close to the destination location, the system 100 can perform an object detection process to detect whether the object has been removed from within the AV (350). Depending on implementation, step 350 can be performed before, after, or while determining that the service has been completed.
[0075] If the system 100 determines that the object has been removed from the AV, the system 100 can determine that no remedial action needs to be performed (370). On the other hand, if the system 100 determines that the object is still present in the vehicle, the system 100 can perform one or more remedial actions and/or cause the service arrangement system 190 to perform one or more remedial actions, such as described in FIGS. 1 and 2 (360).
[0076] FIGS. 4A through 4D illustrate examples of user interfaces displayed on a mobile computing device in various use case examples. For each of the examples illustrated in FIGS. 4A through 4D, an individual user interface can be displayed on a user device of a user who has left an object in a vehicle in connection with the user's transport service. Depending on implementation, the user interface can be displayed as part of a designated application associated with the service arrangement system 190 or as part of another application operating on the user device (e.g., email application, text message application, a web browser application, etc.). In one example, a notification can be displayed on the user device, and selection of a feature or notification can cause the user interface to be displayed on the user device.
[0077] Referring to FIG. 4A, in one example, the user device can receive a notification or a message from the service arrangement system 190 or the system 100 after (or in some instances, just before) the service has ended. The application running on the user device can display the user interface 400, which includes image content 405 of the object and textual content 410 reminding the user and/or notifying the user that the object was left in the vehicle. The image content 405 can be generated from image information from one or more sensors (e.g., cameras) of the vehicle. Alternatively, if a camera is not used in the vehicle, other content representing the object can be displayed in place of the image content 405, such as additional textual content or other graphics or placeholder images representing the object. The textual content 410, in this example, also provides a contact information for the user to message in order to determine what to do in order to retrieve the object
[0078] In the example of FIG. 4B, a user interface 420 can be displayed on the user device. The user interface 420 can also include image content 425, such as described in FIG. 4A, and textual content 430 notifying the user that the object was left in the vehicle and informing the user of a specific location in which to pick up the object. For example, the service arrangement system 190 can instruct the AV or the driver to travel to a specific location (or remain at the destination location or where the user was dropped off). The service arrangement system 190 can generate textual content 430 to include corresponding content about the location. In FIG. 4B, the textual content 430 informs the user that the vehicle with the object will be waiting for 2 more minutes (or some other predetermined duration of time) where the user was dropped off.
[0079] According to another example, a user interface 440 of FIG. 4C can be displayed on the user device as an interactive notification prompt. Such a notification prompt can be in-app (e.g., displayed as part of the application user interface) or out-of-app (e.g., displayed when the application is not currently open and using native user interface notification features of an operating system of a mobile device). In this example, the user interface 440 includes textual content 445 that includes a specific location at which the user's object will be stored or held. The textual content 445 can include location information (e.g., address or street intersection), a time or duration of time when the user can pick up the object (e.g., "normal business hours" or "8am to 6pm"), and a selectable feature 450 that the user can select in order to request a transport service to the specified location. If the user selects the selectable feature ("Request" or "Request trip") 450, the designated service application can be opened or launched on the user device, so that the user can confirm the transport service to the specified location (e.g., from the user's current location or from a specified pickup location). Alternatively, if the user selects the selectable feature 450, the designated service application can automatically generate and transmit a request for the transport service to the service arrangement system 190. The request can include the type of service (e.g., transport the user), the pickup location (e.g., the current location of the user device determined via the GPS receiver of the user device), and the destination location (e.g., 123 Main Street).
[0080] In another example, such as illustrated in FIG. 4D, a user interface 460 can include textual content 465 to inform the user that the user can request a delivery of the user's object. The user interface 460 can enable the user to select a selectable feature 470 to request a delivery transport service of the user's object to the user's current location . Similar to the example of FIG. 4C, if the user selects the selectable feature 470, the designated service application can be opened or launched on the user device, so that the user can confirm the delivery transport service of the user's object to the specified location (e.g., from the user's current location or from a specified pickup location) or the designated service application can
automatically generate and transmit a request for the delivery transport service to the service arrangement system 190. In the latter example, the request can include the type of service (e.g., delivery of the object in the specific vehicle that has the user's object), and the drop off or destination location (e.g., the current location of the user device determined via the GPS receiver of the user device). As described in FIG. 1, for example, the service arrangement system 190 can identify the particular vehicle that the user had just previously used or the vehicle that is associated with the trip entry having the identified object, and select that vehicle to go to the user's current location.
[0081] According to some examples, a user interface described in any of FIGS. 4A through 4D can also include detailed information about the object, such as what the object is (including color or shape), the object type, the dimensions and/or weight of the object, where the object was placed, etc. While the examples of FIGS. 4A through 4D are described with respect to the user device, in some variations, such user interfaces can also be displayed on the driver device to inform the driver of an object that has been left behind in his or her vehicle. Such a user interface can instruct the driver where to go, where to wait (and/or for how long), and/or who to meet with, so that the user can retrieve the object. HARDWARE DIAGRAMS
[0082] FIG. 5 is a block diagram that illustrates a computing device upon which embodiments described herein may be implemented. In one example, a computing device 500 may correspond to a mobile computing device, such as a cellular device that is capable of telephony, messaging, and data services. The computing device 500 can correspond to a user device or a driver device. Examples of such devices include smartphones, handsets or tablet devices for cellular carriers. The computing device 500 includes a processor 510, memory resources 520, a display device 530 (e.g., such as a touch-sensitive display device), one or more communication sub-systems 540 (including wireless communication sub-systems), input mechanisms 550 (e.g., an input
mechanism can include or be part of the touch-sensitive display device), and one or more sensors 560, including a location detection mechanism (e.g., GPS receiver). In one example, at least one of the communication sub-systems 540 sends and receives cellular data over data channels and voice channels. The communications sub-systems 540 can include a cellular transceiver and one or more short-range wireless transceivers. The processing resources 510 can receive communications from the service arrangement system, for example, via the communications sub-systems 540.
[0083] In the example of FIG. 5, the computing device 500 can correspond to a driver device that is operated by a driver of a vehicle. The processor 510 can provide a variety of content to the display 530 by executing instructions stored in the memory resources 520. The memory resources 520 can store instructions corresponding to the driver application 525, for example, and other data, such as one or more trip entries 527, one or more parameters, or databases of location data points. For example, the processor 510 is configured with software and/or other logic to perform one or more processes, steps, and other functions described with implementations, such as described by FIGS. 1 through 4D, and elsewhere in the application . In particular, the processor 510 can execute instructions and data stored in the memory resources 520 in order to be notified about a transport service the driver is to provide, and in connection with the transport service, can use sensor data 565 to perform object detection processes. The sensors 560, as represented herein, can include a sensor included with the computing device 500 (e.g., a GPS receiver) as well as sensor interfaces that communicate with one or more sensors of the vehicle (e.g., ultrasound sensors, weighing devices, etc.). The processor 510 can also execute instructions corresponding to the driver application 525 to cause various user interfaces 515 to be displayed on the display 530, such as an invitation to provide a transport service, navigation information, or notifications about a user having left an object in the driver's vehicle. In addition, the user interfaces 515 can include selectable features that the driver can select, by providing user input 555 via the input mechanisms 550 (e.g., in combination with a touch-sensitive display 530), to interact with the driver application 525, such as to indicate when the transport service has started or completed, in some examples.
[0084] While FIG. 5 is illustrated for a mobile computing device, one or more examples may be implemented on other types of devices, including full- functional computers, such as laptops and desktops (e.g., PC). In another example, a variation of FIG. 5 can illustrate (at least a portion of) an onboard computing system of an AV, such has described in FIG. 1.
[0085] FIG. 6 is a block diagram that illustrates a computer system upon which examples described herein may be implemented. For example, in the context of FIG. 1, the service arrangement system 190 may be implemented using a computer system such as described by FIG. 6. The service
arrangement system 190 may also be implemented using a combination of multiple computer systems as described by FIG. 6.
[0086] In one implementation, the computer system 600 includes processing resources, such as one or more processors 610, a main memory 620, a read-only memory (ROM) 630, a storage device 640, and a
communication interface 650. The computer system 600 includes at least one processor 610 for processing information and the main memory 620, such as a random access memory (RAM) or other dynamic storage device, for storing information and instructions to be executed by the processor 610. The main memory 620 also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by the processor 610. The computer system 600 may also include the ROM 630 or other static storage device for storing static information and instructions for the processor 610. The storage device 640, such as a magnetic disk or optical disk, is provided for storing information and instructions. For example, the storage device 640 can correspond to a computer-readable medium that stores location determination instructions 642 and object detection
instructions 644 for performing operations discussed with respect to FIGS. 1 through 5. In such examples, the computer system 600 can determine the locations of the vehicles, determine the service locations of the transport services, and perform object detection processes based on sensor data received from the vehicle systems or driver devices. In addition, the storage device 640 can include other instructions, such as instructions to implement a dispatch sub-system, which receives requests for transport services from requesting users and selects vehicles to provide those transport services.
[0087] The communication interface 650 can enable the computer system 600 to communicate with one or more networks 680 (e.g., cellular network) through use of the network link (wirelessly or using a wire). Using the network link, the computer system 600 can communicate with a plurality of devices, such as the mobile computing devices of the riders and drivers. According to some examples, using the communication interface 650, the computer system 600 can receive location information 652 from the driver devices, AVs, and/or user devices, and can transmit notification messages 654 to user devices to inform users if users' belongings have been left in a vehicle, such as described by some examples of FIGS. 1 through 5.
[0088] The computer system 600 can also include a display device 660, such as a cathode ray tube (CRT), an LCD monitor, or a television set, for example, for displaying graphics and information to a user. An input
mechanism 670, such as a keyboard that includes alphanumeric keys and other keys, can be coupled to the computer system 600 for communicating information and command selections to the processor 610. Other non-limiting, illustrative examples of the input mechanisms 670 include a mouse, a trackball, touch-sensitive screen, or cursor direction keys for communicating direction information and command selections to the processor 610 and for controlling cursor movement on the display 660.
[0089] Examples described herein are related to the use of the computer system 600 for implementing the techniques described herein. According to one example, those techniques are performed by the computer system 600 in response to the processor 610 executing one or more sequences of one or more instructions contained in the main memory 620. Such instructions may be read into the main memory 620 from another machine-readable medium, such as the storage device 640. Execution of the sequences of instructions contained in the main memory 620 causes the processor 610 to perform the process steps described herein. In alternative implementations, hard-wired circuitry may be used in place of or in combination with software instructions to implement examples described herein. Thus, the examples described are not limited to any specific combination of hardware circuitry and software.
[0090] It is contemplated for examples described herein to extend to individual elements and concepts described herein, independently of other concepts, ideas or system, as well as for examples to include combinations of elements recited anywhere in this application. Although examples are described in detail herein with reference to the accompanying drawings, it is to be understood that the concepts are not limited to those precise examples. Accordingly, it is intended that the scope of the concepts be defined by the following claims and their equivalents. Furthermore, it is contemplated that a particular feature described either individually or as part of an example can be combined with other individually described features, or parts of other examples, even if the other features and examples make no mentioned of the particular feature. Thus, the absence of describing combinations should not preclude having rights to such combinations.

Claims

What is being claimed is:
1. A method for monitoring a vehicle used in providing a transport service, the method being implemented by one or more processors and comprising : detecting when the transport service is complete;
upon detecting the transport service is complete, determining, from one or more sensors of the vehicle, that an object which should have been removed from the vehicle after completion of the transport service, remains in the vehicle; and
in response to determining that the object remains in the vehicle, automatically initiating performance of a remedial action.
2. The method of claim 1, wherein the one or more sensors of the vehicle includes one or more of (i) a set of cameras, (ii) a set of weighing devices, (iii) a set of suspension sensors, (iv) a set of light detection and ranging (LIDAR) sensors, (v) a set of radio-frequency transceivers, or (vi) a set of ultrasound sensors.
3. The method of claim 1, wherein determining that the object remains in the vehicle includes detecting a position of the object inside the vehicle based on image information from a set of cameras that is positioned within the vehicle.
4. The method of claim 1, wherein determining that the object remains in the vehicle includes detecting that the object has been placed in a
compartment or an interior of the vehicle based on weight information from a set of weighing devices that is positioned within the compartment or the interior of the vehicle to sense weight of objects placed thereon.
5. The method of claim 1, wherein determining that the object remains in the vehicle includes detecting that the object has been placed in a
compartment or an interior of the vehicle based on compression information from a set of sensors that is positioned to detect compression of a suspension system of the vehicle.
6. The method of claim 2, wherein determining that the object remains in the vehicle includes measuring radio-frequency signals emitted by the object, and determining that the object has been placed in a compartment or an interior of the vehicle from the measured radio-frequency signals.
7. The method of claim 1, wherein determining that the object remains in the vehicle includes determining a first set of information from a first sensor set of the vehicle when the transport service is being provided, and
determining a second set of information from the first sensor set of the vehicle after the transport service is complete, and comparing the first set of sensor information to the second set of information .
8. The method of claim 1, wherein automatically initiating performance of the remedial action includes communicating, over one or more networks, a message to a computing device associated with a user that requested the service, the message notifying the user that an item has been left in the vehicle.
9. The method of claim 8, wherein automatically initiating performance of the remedial action includes causing the vehicle to remain stationary at a location of the vehicle when the object was detected to still be present in the vehicle.
10. The method of claim 8, wherein automatically initiating performance of the remedial action includes causing the vehicle to travel to a predetermined location, the predetermined location corresponding to a location where the object can be removed from the vehicle.
11. The method of claim 3, wherein determining that the object remains in the vehicle includes capturing one or more images of a portion of the vehicle using one or more cameras of the vehicle, and performing image analysis to detect the object from the one or more images.
12. The method of claim 1, further comprising :
determining, from one or more sensors of the vehicle, when the object is first placed in the vehicle by a passenger who enters the vehicle at a pickup location.
13. The method of claim 12, wherein detecting that the object is first placed in the vehicle is performed (i) in response to detecting that the passenger has entered the vehicle, (ii) in response to detecting that a predetermined duration of time has elapsed since detecting that the passenger has entered the vehicle, (iii) in response to receiving, from a computing device associated with the passenger, a message indicating that the service can begin as a result of an input being provided at the computing device associated with the
passenger, (iv) in response to receiving, from a computing device associated with the vehicle, a message indicating that the service can begin as a result of an input being provided at the computing device associated with the vehicle from the passenger within the vehicle, or (v) in response to determining that the vehicle has been positioned within a predetermined distance of the pickup location for a predetermined duration of time based on location information of the vehicle.
14. A method for monitoring a vehicle used in providing a service, comprising :
receiving, at a computing system from a remote system, instructions for a vehicle to provide a service, the service corresponding to transporting one or more objects or individuals from one location to another, the service having a start location and a destination location;
controlling, by the computing system, the vehicle to travel to the start location, the computing system corresponding to an on-board computing system of the vehicle and controlling components of the vehicle;
after the vehicle arrives at or is within a first predetermined distance of the start location, detecting, by the computing system, that an object has been placed within the vehicle based on a first set of information from one or more sensors of the vehicle at a first time;
subsequently, controlling, by the computing system, the vehicle to travel to the destination location; after the vehicle arrives at or is within a second predetermined distance of the destination location, detecting whether the object has been removed from the vehicle based on a second set of information from the one or more sensors at a second time; and
in response to detecting that the object is still present in the vehicle, automatically initiating performance of a remedial action.
15. The method of claim 14, wherein the one or more sensors of the vehicle includes one or more of (i) a set of cameras, (ii) a set of weighing devices, (iii) a set of suspension sensors, (iv) a set of light detection and ranging (LIDAR) sensors, (v) a set of radio-frequency transceivers, or (vi) a set of ultrasound sensors.
16. The method of claim 15, wherein detecting that the object has been placed within the vehicle includes detecting a position of the object inside the vehicle based on image information from the set of cameras that are
positioned within the vehicle, wherein the set of cameras are positioned such that the respective lens of the set of cameras faces towards an interior of the vehicle, and wherein automatically initiating performance of a remedial action includes (i) capturing one or more images of the object using one or more of the set of cameras, (ii) generating a message that includes the one or more images, and (iii) communicating, over one or more networks, the message to the remote system to notify the remote system that an item has been left in the vehicle, or communicating, over the one or more networks, the message to a computing device associated with a user that requested the service, the message notifying the user that an item has been left in the vehicle.
17. The method of claim 15, wherein detecting that the object has been placed within the vehicle includes detecting that the object has been placed in a compartment or an interior of the vehicle based on weight information from the set of weighing devices that are positioned within the compartment or the interior of the vehicle to sense weight of objects placed thereon.
18. The method of claim 15, wherein detecting that the object has been placed within the vehicle includes detecting that the object has been placed in a compartment or an interior of the vehicle based on compression information from the set of suspension sensors that is positioned to detect compression of a suspension system of the vehicle.
19. The method of claim 14, wherein automatically initiating performance of a remedial action includes communicating, over one or more networks, a message to the remote system to notify the remote system that an item has been left in the vehicle, or communicating, over the one or more networks, a message to a computing device associated with a user that requested the service, the message notifying the user that an item has been left in the vehicle.
20. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a computing system, cause the computing system to:
receive, from a remote system, instructions for a vehicle to provide a service, the service corresponding to transporting one or more objects or individuals from one location to another, the service having a start location and a destination location;
control the vehicle to travel to the start location;
after the vehicle arrives at or is within a first predetermined distance of the start location, detect that an object has been placed within the vehicle based on a first set of information from one or more sensors of the vehicle at a first time;
subsequently, control the vehicle to travel to the destination location; after the vehicle arrives at or is within a second predetermined distance of the destination location, detect whether the object has been removed from the vehicle based on a second set of information from the one or more sensors at a second time; and
in response to detecting that the object is still present in the vehicle, automatically perform a remedial action.
PCT/US2016/031929 2015-05-11 2016-05-11 Detecting objects within a vehicle in connection with a service WO2016183241A1 (en)

Priority Applications (7)

Application Number Priority Date Filing Date Title
CA2985619A CA2985619A1 (en) 2015-05-11 2016-05-11 Detecting objects within a vehicle in connection with a service
EP16793475.1A EP3295421A4 (en) 2015-05-11 2016-05-11 Detecting objects within a vehicle in connection with a service
KR1020177035479A KR102395766B1 (en) 2015-05-11 2016-05-11 Detection of objects in the vehicle in relation to the service
CN201680040538.2A CN107873098A (en) 2015-05-11 2016-05-11 Object in the detection vehicle relevant with service
AU2016262526A AU2016262526B9 (en) 2015-05-11 2016-05-11 Detecting objects within a vehicle in connection with a service
RU2017142642A RU2725681C9 (en) 2015-05-11 2016-05-11 Detection of objects inside a vehicle in connection with maintenance
IL255562A IL255562B (en) 2015-05-11 2017-11-09 Autonomous vehicle operated with guide assistance

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US14/708,611 2015-05-11
US14/708,611 US9616773B2 (en) 2015-05-11 2015-05-11 Detecting objects within a vehicle in connection with a service

Publications (1)

Publication Number Publication Date
WO2016183241A1 true WO2016183241A1 (en) 2016-11-17

Family

ID=57249481

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2016/031929 WO2016183241A1 (en) 2015-05-11 2016-05-11 Detecting objects within a vehicle in connection with a service

Country Status (9)

Country Link
US (7) US9616773B2 (en)
EP (1) EP3295421A4 (en)
KR (1) KR102395766B1 (en)
CN (1) CN107873098A (en)
AU (1) AU2016262526B9 (en)
CA (1) CA2985619A1 (en)
IL (1) IL255562B (en)
RU (1) RU2725681C9 (en)
WO (1) WO2016183241A1 (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2549615A (en) * 2016-04-18 2017-10-25 Ford Global Tech Llc Vehicle security system
EP3355254A1 (en) * 2017-01-30 2018-08-01 Panasonic Intellectual Property Corporation of America Method for online delivery system, management apparatus, and recording medium having program stored therein
EP3471035A1 (en) * 2017-10-16 2019-04-17 Flex Ltd. Method and system for tracking and optimizing cargo utilization and volume measurement and imaging sensing using lidars and video cameras
WO2020000396A1 (en) 2018-06-29 2020-01-02 Baidu.Com Times Technology (Beijing) Co., Ltd. Theft proof techniques for autonomous driving vehicles used for transporting goods
WO2020088892A1 (en) * 2018-10-31 2020-05-07 Valeo Comfort And Driving Assistance Unit, system and method for detecting an object in or damage to a passenger compartment of a motor vehicle
EP3899894A4 (en) * 2018-12-20 2022-01-26 Gentex Corporation Systems and methods for ride share monitoring
FR3121101A1 (en) * 2021-03-29 2022-09-30 Psa Automobiles Sa Method for detecting and identifying an object placed in a storage compartment of a motor vehicle and associated storage compartment
US11505984B2 (en) 2015-05-11 2022-11-22 Uber Technologies, Inc. Detecting objects within a vehicle in connection with a service
US11625926B2 (en) * 2018-03-22 2023-04-11 Kioxia Corporation Information processing device, information processing method, and information processing program product

Families Citing this family (104)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9747740B2 (en) 2015-03-02 2017-08-29 Ford Global Technologies, Llc Simultaneous button press secure keypad code entry
US20160257198A1 (en) 2015-03-02 2016-09-08 Ford Global Technologies, Inc. In-vehicle component user interface
US9622159B2 (en) * 2015-09-01 2017-04-11 Ford Global Technologies, Llc Plug-and-play interactive vehicle interior component architecture
US9967717B2 (en) 2015-09-01 2018-05-08 Ford Global Technologies, Llc Efficient tracking of personal device locations
US9914418B2 (en) 2015-09-01 2018-03-13 Ford Global Technologies, Llc In-vehicle control location
US9860710B2 (en) 2015-09-08 2018-01-02 Ford Global Technologies, Llc Symmetrical reference personal device location tracking
US9744852B2 (en) 2015-09-10 2017-08-29 Ford Global Technologies, Llc Integration of add-on interior modules into driver user interface
KR101841948B1 (en) * 2015-10-02 2018-03-26 엘지전자 주식회사 Apparatus, Method and Mobile Terminal for providing service to prevent lost article on inner Vehicle
US10635994B2 (en) 2015-12-11 2020-04-28 Lyft, Inc. System for navigating driver to passenger for ride authorized by another user of transportation service
US10046637B2 (en) 2015-12-11 2018-08-14 Ford Global Technologies, Llc In-vehicle component control user interface
US9766344B2 (en) * 2015-12-22 2017-09-19 Honda Motor Co., Ltd. Multipath error correction
US11508026B2 (en) 2015-12-31 2022-11-22 Lyft, Inc. System for navigating transportation service providers to fulfill transportation requests authorized by an organization
US10082877B2 (en) 2016-03-15 2018-09-25 Ford Global Technologies, Llc Orientation-independent air gesture detection service for in-vehicle environments
US20170277182A1 (en) * 2016-03-24 2017-09-28 Magna Electronics Inc. Control system for selective autonomous vehicle control
US20170291539A1 (en) * 2016-04-07 2017-10-12 General Motors Llc Systems and methods for detecting objects within a vehicle
CN105809902B (en) * 2016-04-13 2017-03-29 京东方科技集团股份有限公司 A kind of taxi monitoring method, apparatus and system
US9914415B2 (en) 2016-04-25 2018-03-13 Ford Global Technologies, Llc Connectionless communication with interior vehicle components
US9741225B1 (en) * 2016-04-28 2017-08-22 Abu Dhabi University Safety detection system and method
US10037033B2 (en) * 2016-06-15 2018-07-31 Ford Global Technologies, Llc Vehicle exterior surface object detection
US10163219B2 (en) * 2016-06-24 2018-12-25 Fca Us Llc Machine vision cargo monitoring in a vehicle
US10621515B2 (en) * 2016-06-30 2020-04-14 At&T Intellectual Property I, L.P. Optimized traffic management via an electronic driving pass
US11599833B2 (en) * 2016-08-03 2023-03-07 Ford Global Technologies, Llc Vehicle ride sharing system and method using smart modules
US10414344B1 (en) * 2016-09-01 2019-09-17 Apple Inc. Securable storage compartments
DE102016222006B4 (en) * 2016-11-09 2024-08-01 Audi Ag Method for operating a motor vehicle in a fully autonomous driving mode
US10311314B2 (en) 2016-11-23 2019-06-04 Ford Global Technologies, Llc Detection of lane-splitting motorcycles
US10753754B2 (en) 2017-01-19 2020-08-25 Andrew DeLizio Managing autonomous vehicles
US10290158B2 (en) 2017-02-03 2019-05-14 Ford Global Technologies, Llc System and method for assessing the interior of an autonomous vehicle
US10768630B2 (en) * 2017-02-09 2020-09-08 International Business Machines Corporation Human imperceptible signals
US10053088B1 (en) * 2017-02-21 2018-08-21 Zoox, Inc. Occupant aware braking system
US10220857B2 (en) * 2017-02-23 2019-03-05 Uber Technologies, Inc. Vehicle control system
JP6769383B2 (en) * 2017-04-13 2020-10-14 トヨタ自動車株式会社 Automatic driving device and notification method
US10509974B2 (en) 2017-04-21 2019-12-17 Ford Global Technologies, Llc Stain and trash detection systems and methods
US10679312B2 (en) * 2017-04-25 2020-06-09 Lyft Inc. Dynamic autonomous vehicle servicing and management
DE112017007445T5 (en) 2017-05-12 2020-02-27 Ford Global Technologies, Llc object detection
US10304165B2 (en) * 2017-05-12 2019-05-28 Ford Global Technologies, Llc Vehicle stain and trash detection systems and methods
US11009886B2 (en) * 2017-05-12 2021-05-18 Autonomy Squared Llc Robot pickup method
US11493348B2 (en) 2017-06-23 2022-11-08 Direct Current Capital LLC Methods for executing autonomous rideshare requests
US20200156663A1 (en) * 2017-06-30 2020-05-21 Intel Corporation Accessibility method and apparatus for autonomous/semi-autonomous driving
US10416671B2 (en) * 2017-07-11 2019-09-17 Waymo Llc Methods and systems for vehicle occupancy confirmation
US11200530B2 (en) * 2017-08-21 2021-12-14 Ford Global Technologies, Llc Object detection
US11151482B2 (en) * 2017-08-31 2021-10-19 Waymo Llc Identifying unassigned passengers for autonomous vehicles
JP6958255B2 (en) 2017-11-07 2021-11-02 トヨタ自動車株式会社 Information processing device and information processing method
US11106927B2 (en) * 2017-12-27 2021-08-31 Direct Current Capital LLC Method for monitoring an interior state of an autonomous vehicle
US20190220016A1 (en) 2018-01-15 2019-07-18 Uber Technologies, Inc. Discrete Decision Architecture for Motion Planning System of an Autonomous Vehicle
EP3517397A1 (en) 2018-01-30 2019-07-31 Bombardier Transportation GmbH Method for interior state monitoring, as well as a vehicle having interior state monitoring device
US10853629B2 (en) 2018-02-20 2020-12-01 Direct Current Capital LLC Method for identifying a user entering an autonomous vehicle
US10838425B2 (en) * 2018-02-21 2020-11-17 Waymo Llc Determining and responding to an internal status of a vehicle
JP6996359B2 (en) * 2018-03-09 2022-01-17 トヨタ自動車株式会社 Forgotten prevention support method, terminal device and forgotten prevention support system
DE102018204383A1 (en) 2018-03-22 2019-09-26 Audi Ag Transport means for transporting a cargo and / or a person, method for operating a means of transport and transport system
US10311704B1 (en) * 2018-03-29 2019-06-04 Futurewei Technologies, Inc. Passenger-related item loss mitigation
CN108422938B (en) * 2018-03-30 2024-04-05 斑马网络技术有限公司 Vehicle with sensing device and application
JP2020004252A (en) * 2018-06-29 2020-01-09 株式会社東芝 Residue detection system
US11354692B2 (en) * 2018-08-01 2022-06-07 Motional Ad Llc System and method for keeping an automated-taxi clean
CN109254581B (en) * 2018-08-10 2021-07-02 合肥哈工库讯智能科技有限公司 AGV dolly intelligent operation regulation and control system based on running state analysis
US10573162B1 (en) * 2018-08-21 2020-02-25 Ford Global Technologies, Llc Tracking smart devices in vehicles technical field
US11494707B2 (en) * 2018-08-22 2022-11-08 Denso International America, Inc. System and method of determining real-time location
US10515535B1 (en) * 2018-08-24 2019-12-24 General Motors Llc System and method to provide a misplacement notification
US11599963B2 (en) * 2018-09-25 2023-03-07 Uber Technologies, Inc. Pairing transport service users and drivers using animation
DE102018216761A1 (en) * 2018-09-28 2020-04-02 Zf Friedrichshafen Ag Device and method for recognizing an actual state of an interior of a people mover
JP7119871B2 (en) * 2018-10-05 2022-08-17 トヨタ自動車株式会社 Lost-and-found delivery support device, lost-and-found delivery support system, lost-and-found delivery support method, and computer program for lost-and-found delivery support
JP7047697B2 (en) 2018-10-05 2022-04-05 トヨタ自動車株式会社 Driving support device, driving support system, driving support method, and computer program for driving support
US10687045B2 (en) * 2018-10-23 2020-06-16 Zebra Technologies Corporation Systems and methods for idle time in commercial trailer loading
US11423340B2 (en) * 2018-10-29 2022-08-23 Uber Technologies, Inc. On-demand transport selection process facilitating third-party autonomous vehicles
JP7040409B2 (en) * 2018-11-07 2022-03-23 トヨタ自動車株式会社 Mobility service support devices, mobility systems, mobility service support methods, and computer programs for mobility service support
EP3893190A4 (en) * 2018-12-07 2021-12-29 Panasonic Intellectual Property Corporation of America Information processing method, information processing device, and program
US10796174B2 (en) 2018-12-21 2020-10-06 Nissan North America, Inc. Distance and object based external notification system for automated hailing service
US11520331B2 (en) * 2018-12-28 2022-12-06 Intel Corporation Methods and apparatus to update autonomous vehicle perspectives
US11353870B2 (en) * 2018-12-31 2022-06-07 Baidu Usa Llc Autonomous driving computing and storage expansion device with flexible host and client configuration
US10872253B2 (en) * 2019-02-08 2020-12-22 Ford Global Technologies, Llc Method and apparatus for vehicle interior evaluation and situational servicing
KR102155668B1 (en) * 2019-02-13 2020-09-14 충남대학교산학협력단 Lost-and-found notification device that provides information on the occurrence of lost items in public transportation
CN109857114A (en) * 2019-02-26 2019-06-07 温州市职业中等专业学校 Campus Mobile Intelligent Robot and campus intelligent management system
JP7230173B2 (en) * 2019-03-01 2023-02-28 株式会社日立製作所 Abandoned object detection device and abandoned object detection method
US11501401B2 (en) * 2019-03-02 2022-11-15 ANI Technologies Private Limited Allocation of vehicles using fitness information
JP2020142570A (en) * 2019-03-04 2020-09-10 テイ・エス テック株式会社 Notification system and vehicle mounted with the same
US11100347B2 (en) 2019-03-12 2021-08-24 Ford Global Technologies, Llc Photometric stereo object detection for articles left in an autonomous vehicle
US10887724B2 (en) 2019-05-01 2021-01-05 International Business Machines Corporation Locating a mobile device and notifying a user of the mobile device location
JP7180535B2 (en) * 2019-05-23 2022-11-30 トヨタ自動車株式会社 Information processing device, information processing method and program
DE102019208871A1 (en) * 2019-06-18 2020-12-24 Thyssenkrupp Ag Device and method for determining the interior state of a vehicle
KR102267192B1 (en) * 2019-08-14 2021-06-18 엘지전자 주식회사 Apparatus and method for detecting falling object
US10854055B1 (en) * 2019-10-17 2020-12-01 The Travelers Indemnity Company Systems and methods for artificial intelligence (AI) theft prevention and recovery
CN113147588A (en) * 2020-01-22 2021-07-23 奥迪股份公司 Reminder system for a vehicle, vehicle comprising same, and corresponding method and medium
US11738713B2 (en) * 2020-01-23 2023-08-29 Ford Global Technologies, Llc Vehicle security system
EP3859113B1 (en) * 2020-01-30 2023-08-09 Ningbo Geely Automobile Research & Development Co. Ltd. A system for automatically closing a trunk lid of a vehicle
US10834538B1 (en) * 2020-02-12 2020-11-10 International Business Machines Corporation Locating a mobile device and notifying a user of the mobile device location
US11062416B1 (en) * 2020-02-28 2021-07-13 Sos Lab Co., Ltd. Shared vehicle service providing method performed by server communicating with user device of passenger and autonomous vehicle
US11594046B2 (en) * 2020-03-26 2023-02-28 Pony Ai Inc. Vehicle cargo cameras for sensing vehicle characteristics
DE102020204291A1 (en) * 2020-04-02 2021-10-07 Volkswagen Aktiengesellschaft Method for manufacturing a vehicle and seat belt assembly
KR20220003320A (en) * 2020-07-01 2022-01-10 현대자동차주식회사 Vehicle and method of managing in-car cleanliness for the same
US11928406B2 (en) * 2020-07-02 2024-03-12 Ford Global Technologies, Llc Systems and methods for creating infrastructure models
US11756145B2 (en) * 2020-07-20 2023-09-12 Ford Global Technologies, Llc Systems and methods for providing a unified transportation service
USD960901S1 (en) 2020-10-30 2022-08-16 Zoox, Inc. Display screen or portion thereof having a graphical user interface
US11760281B2 (en) * 2020-11-17 2023-09-19 Ford Global Technologies, Llc Battery-powered vehicle sensors
US11953586B2 (en) 2020-11-17 2024-04-09 Ford Global Technologies, Llc Battery-powered vehicle sensors
US11675349B2 (en) 2020-11-30 2023-06-13 Zoox, Inc. User interface for remote vehicle monitoring
CN112590814A (en) * 2020-12-17 2021-04-02 上海悦充网络科技有限公司 Vehicle automatic driving control method and system based on Internet of vehicles
US11475755B2 (en) 2021-03-02 2022-10-18 Gm Cruise Holdings Llc Forgotten mobile device detection and management
US11780349B2 (en) 2021-03-10 2023-10-10 Toyota Motor Engineering & Manufacturing North America, Inc. System and method for detecting objects left behind by using heated resistive filament in a vehicle
US11916420B2 (en) 2021-03-12 2024-02-27 Ford Global Technologies, Llc Vehicle sensor operation
US11912235B2 (en) 2021-03-12 2024-02-27 Ford Global Technologies, Llc Vehicle object detection
US11951937B2 (en) 2021-03-12 2024-04-09 Ford Global Technologies, Llc Vehicle power management
US11904784B2 (en) * 2021-08-16 2024-02-20 Motional Ad Llc Detecting objects within a vehicle
US12106586B2 (en) * 2022-03-28 2024-10-01 Gm Cruise Holdings Llc Lost object tracking system
US11836760B1 (en) * 2022-07-01 2023-12-05 Gm Cruise Holdings Llc Vehicle advertising platform
CN116153037A (en) * 2023-03-20 2023-05-23 重庆长安汽车股份有限公司 Equipment forgetting reminding method and device, vehicle and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060002586A1 (en) * 2004-06-15 2006-01-05 Manoj Aggarwal Method and apparatus for detecting left objects
JP2006159939A (en) * 2004-12-02 2006-06-22 Xanavi Informatics Corp Notice device for in-cabin object to be informed
US20090146813A1 (en) * 2007-12-10 2009-06-11 Acropolis Engineering Automobile forgotten passenger alarm and notification
US20090216600A1 (en) * 2008-02-27 2009-08-27 Montiss Llc Systems and methods for arranging a transport transaction
US20120242492A1 (en) * 2011-03-25 2012-09-27 Tov 1 LLC Seat occupancy detection and display system

Family Cites Families (255)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5144685A (en) 1989-03-31 1992-09-01 Honeywell Inc. Landmark recognition for autonomous mobile robots
US7983817B2 (en) * 1995-06-07 2011-07-19 Automotive Technologies Internatinoal, Inc. Method and arrangement for obtaining information about vehicle occupants
US7783403B2 (en) 1994-05-23 2010-08-24 Automotive Technologies International, Inc. System and method for preventing vehicular accidents
US5654615A (en) * 1994-12-15 1997-08-05 Asc Incorporated Storage compartment sensor system for use with an automotive vehicle convertible roof
US7313467B2 (en) * 2000-09-08 2007-12-25 Automotive Technologies International Inc. System and method for in-vehicle communications
JP3634006B2 (en) 1995-06-09 2005-03-30 株式会社ザナヴィ・インフォマティクス Current position calculation device
US5751852A (en) 1996-04-29 1998-05-12 Xerox Corporation Image structure map data structure for spatially indexing an imgage
US5809179A (en) 1996-05-31 1998-09-15 Xerox Corporation Producing a rendered image version of an original image using an image structure map representation of the image
WO1998003021A1 (en) 1996-06-28 1998-01-22 Sri International Small vision module for real-time stereo and motion analysis
KR100339763B1 (en) 1996-12-16 2002-06-05 하기와라 가즈토시 Map database device
US8209120B2 (en) 1997-10-22 2012-06-26 American Vehicular Sciences Llc Vehicular map database management techniques
US7840342B1 (en) 1997-10-22 2010-11-23 Intelligent Technologies International, Inc. Road physical condition monitoring techniques
US6922622B2 (en) * 1999-06-03 2005-07-26 Robert Bosch Corporation Hot vehicle safety system and methods of preventing passenger entrapment and heat suffocation
US6385539B1 (en) 1999-08-13 2002-05-07 Daimlerchrysler Ag Method and system for autonomously developing or augmenting geographical databases by mining uncoordinated probe data
MXPA02008486A (en) * 2000-02-29 2002-12-13 United Parcel Service Inc Delivery system and method for vehicles and the like.
US7352454B2 (en) 2000-11-09 2008-04-01 Canesta, Inc. Methods and devices for improved charge management for three-dimensional and color sensing
US6559776B2 (en) 2001-02-15 2003-05-06 Yoram Katz Parking status control system and method
DE10110373C2 (en) 2001-03-03 2003-03-06 Wolfgang Daum Method and device for cleaning the interior of automobiles
FR2826727B1 (en) 2001-06-29 2004-05-28 Valeo Vision METHOD AND DEVICE FOR DETECTING THE HUMIDITY OF A ROAD ON WHICH A VEHICLE IS MOVING
US6542111B1 (en) 2001-08-13 2003-04-01 Yazaki North America, Inc. Path prediction for vehicular collision warning system
US20030040944A1 (en) * 2001-08-22 2003-02-27 Hileman Ryan M. On-demand transportation system
US6856282B2 (en) 2002-02-08 2005-02-15 Qualcomm Incorporated Directly acquiring precision code GPS signals
FR2836418A1 (en) 2002-02-22 2003-08-29 Michelin Soc Tech Vehicle tire tread, for measuring road surface adhesion, comprises tread block(s) consisting of central zone, surrounding zone and detector applied to external radial summit of tread
US7124027B1 (en) 2002-07-11 2006-10-17 Yazaki North America, Inc. Vehicular collision avoidance system
US6795031B1 (en) 2002-07-12 2004-09-21 Yazaki North America, Inc. Mechanically scanned parabolic reflector antenna
US7102496B1 (en) 2002-07-30 2006-09-05 Yazaki North America, Inc. Multi-sensor integration for a vehicle
JP3860518B2 (en) 2002-08-12 2006-12-20 株式会社豊田中央研究所 Road friction estimation device
JP4416996B2 (en) 2002-11-01 2010-02-17 三菱電機株式会社 Map information processing apparatus and map information providing apparatus
US6807473B1 (en) 2003-04-09 2004-10-19 Continental Teves, Inc. Road recognition system
JP2005028887A (en) 2003-07-07 2005-02-03 Fuji Heavy Ind Ltd Method and device for estimating road surface friction coefficient
DE602004004014T2 (en) 2003-08-26 2007-04-26 Fuji Jukogyo K.K. Apparatus and method for assessing tire type and road surface conditions for vehicles
WO2005023614A2 (en) 2003-09-03 2005-03-17 Nsk Ltd. Stability control apparatus and load mesuring instrument for wheel supporting rolling bearing unit
JP4419672B2 (en) * 2003-09-16 2010-02-24 株式会社デンソー Vehicle left behind prevention device
US6970777B2 (en) 2003-11-26 2005-11-29 Nissan Motor Co., Ltd. Automotive lane deviation prevention apparatus
US20160267720A1 (en) 2004-01-30 2016-09-15 Electronic Scripting Products, Inc. Pleasant and Realistic Virtual/Augmented/Mixed Reality Experience
JP2005271824A (en) 2004-03-25 2005-10-06 Mitsubishi Fuso Truck & Bus Corp Vehicular behavior control device
US20100013615A1 (en) 2004-03-31 2010-01-21 Carnegie Mellon University Obstacle detection having enhanced classification
DE102004016288B3 (en) 2004-04-02 2005-08-18 Daimlerchrysler Ag Determining friction value between vehicle tire, road involves evaluating tire vibration characteristic(s), especially frequency spectrum and/or time domain spectrum, by evaluating data using physical and/or phenomenological model approach
WO2005121707A2 (en) 2004-06-03 2005-12-22 Making Virtual Solid, L.L.C. En-route navigation display method and apparatus using head-up display
EP1779295A4 (en) 2004-07-26 2012-07-04 Automotive Systems Lab Vulnerable road user protection system
US7587899B2 (en) 2004-08-27 2009-09-15 University Of Houston Shape memory alloy actuated adaptive exhaust nozzle for jet engine
US20060126975A1 (en) 2004-12-10 2006-06-15 Mckellar Randy R Easy opening rear seam bags and packaging methods
US7250869B2 (en) * 2005-03-21 2007-07-31 Davis Le Roychelle Child carseat alert system
US7539574B2 (en) 2005-03-22 2009-05-26 Denso Corporation Vehicular navigation system
US20060223637A1 (en) 2005-03-31 2006-10-05 Outland Research, Llc Video game system combining gaming simulation with remote robot control and remote robot feedback
WO2007012198A1 (en) 2005-07-26 2007-02-01 Macdonald, Dettwiler & Associates Inc. Guidance, navigation, and control system for a vehicle
WO2007037186A1 (en) * 2005-09-28 2007-04-05 Pioneer Corporation Waypoint setting device, information presentation device, waypoint setting method, waypoint setting program, and recording medium
US8164628B2 (en) 2006-01-04 2012-04-24 Mobileye Technologies Ltd. Estimating distance to an object using a sequence of images recorded by a monocular camera
US7994465B1 (en) 2006-02-06 2011-08-09 Microsoft Corporation Methods and devices for improved charge management for three-dimensional and color sensing
US20090070034A1 (en) 2006-03-17 2009-03-12 Christopher L Oesterling Method for recording an annotation and making it available for later playback
US9373149B2 (en) 2006-03-17 2016-06-21 Fatdoor, Inc. Autonomous neighborhood vehicle commerce network and community
US8014796B2 (en) 2006-03-31 2011-09-06 Research In Motion Limited Map version control methods and apparatus for updating the use of network-maintained map data sets for mobile communication devices
DE102006018029A1 (en) 2006-04-19 2007-10-25 Robert Bosch Gmbh Device and method for controlling personal protection devices
US7999849B2 (en) 2006-05-17 2011-08-16 The Boeing Company Moving object detection
DE102006027121A1 (en) 2006-06-12 2007-12-13 Robert Bosch Gmbh Image acquisition system and method for determining distance with an image acquisition system
JP2008027239A (en) * 2006-07-21 2008-02-07 Denso Corp Apparatus for preventing belongings from being left behind
US20080033645A1 (en) 2006-08-03 2008-02-07 Jesse Sol Levinson Pobabilistic methods for mapping and localization in arbitrary outdoor environments
JP4848994B2 (en) 2006-12-22 2011-12-28 株式会社エクォス・リサーチ Vehicle control device
JP4915268B2 (en) 2006-12-22 2012-04-11 株式会社エクォス・リサーチ Vehicle control device
JP4897516B2 (en) 2007-02-28 2012-03-14 アイシン・エィ・ダブリュ株式会社 Navigation device and data update system
JP5075443B2 (en) 2007-03-27 2012-11-21 アイシン・エィ・ダブリュ株式会社 Road map data generation device, navigation device, and road map data generation program
JP4903616B2 (en) 2007-03-29 2012-03-28 アイシン・エィ・ダブリュ株式会社 Map update data supply device, map data update system, and map update data supply method
JP4576445B2 (en) 2007-04-12 2010-11-10 パナソニック株式会社 Autonomous mobile device and program for autonomous mobile device
JP2008262459A (en) * 2007-04-13 2008-10-30 Fujitsu Ten Ltd Mislaying announcing device
CN101687455B (en) 2007-07-02 2012-04-25 爱考斯研究株式会社 Camber angle controlling device
JP2009012635A (en) 2007-07-05 2009-01-22 Denso Corp Vehicular cleaning device
US8385658B2 (en) 2007-07-27 2013-02-26 Sportvision, Inc. Detecting an object in an image using multiple templates
JP4858381B2 (en) * 2007-09-19 2012-01-18 株式会社デンソー Car left behind prevention device
US8145402B2 (en) 2007-12-05 2012-03-27 Lockheed Martin Corporation GPS-based traction control system and method using data transmitted between vehicles
CN101952688A (en) 2008-02-04 2011-01-19 电子地图北美公司 Method for map matching with sensor detected objects
JP4613970B2 (en) 2008-03-12 2011-01-19 トヨタ自動車株式会社 Vehicle lighting device
KR100862561B1 (en) 2008-03-13 2008-10-09 (주) 서돌 전자통신 A system for sensing a traffic accident
EP2135783A1 (en) 2008-06-18 2009-12-23 GM Global Technology Operations, Inc. Motor vehicle driver assisting method
US9519921B2 (en) * 2008-06-27 2016-12-13 E-Lantis Corporation GPS and wireless integrated fleet management system and method
US20100020170A1 (en) 2008-07-24 2010-01-28 Higgins-Luthman Michael J Vehicle Imaging System
US8126642B2 (en) 2008-10-24 2012-02-28 Gray & Company, Inc. Control and systems for autonomously driven vehicles
US8364334B2 (en) 2008-10-30 2013-01-29 Honeywell International Inc. System and method for navigating an autonomous vehicle using laser detection and ranging
US8417414B2 (en) * 2008-12-11 2013-04-09 Infosys Technologies Limited Method and system for managing passenger and vehicle safety
DE102008055159A1 (en) 2008-12-29 2010-07-01 Robert Bosch Gmbh Adaptive angle and power adjustment for 3D micromirror lidar
US8677145B2 (en) 2009-02-27 2014-03-18 Atmel Corporation Single pin communication mechanism
US9278693B2 (en) 2009-03-24 2016-03-08 Ford Global Technologies, Llc System and method for improving vehicle performance on grade
JP2010231415A (en) * 2009-03-26 2010-10-14 Denso Corp Monitoring device for vehicle
US8180547B2 (en) 2009-03-27 2012-05-15 Ford Global Technologies, Llc Telematics system and method for traction reporting and control in a vehicle
US8352112B2 (en) 2009-04-06 2013-01-08 GM Global Technology Operations LLC Autonomous vehicle management
US8150620B2 (en) 2009-04-14 2012-04-03 Alpine Electronics, Inc. Route search method and apparatus for navigation system utilizing map data of XML format
WO2010135575A2 (en) 2009-05-20 2010-11-25 Express Imaging Systems, Llc Long-range motion detection for illumination control
JP5461065B2 (en) 2009-05-21 2014-04-02 クラリオン株式会社 Current position specifying device and current position specifying method
RU2400592C1 (en) * 2009-08-17 2010-09-27 Александр Макарович Салдаев Method of transport network creation
US20110046784A1 (en) 2009-08-18 2011-02-24 Noel Wayne Anderson Asymmetric stereo vision system
US9552726B2 (en) 2009-08-24 2017-01-24 Here Global B.V. Providing driving condition alerts using road attribute data
US8301374B2 (en) 2009-08-25 2012-10-30 Southwest Research Institute Position estimation for ground vehicle navigation based on landmark identification/yaw rate and perception of landmarks
US8306672B2 (en) 2009-09-09 2012-11-06 GM Global Technology Operations LLC Vehicular terrain detection system and method
WO2011038018A1 (en) 2009-09-23 2011-03-31 Earthsearch Communications International, Inc. Device and process for monitoring secure storage and delivery of items
KR101662595B1 (en) 2009-11-03 2016-10-06 삼성전자주식회사 User terminal, route guide system and route guide method thereof
US8559673B2 (en) 2010-01-22 2013-10-15 Google Inc. Traffic signal mapping and detection
US9377528B2 (en) 2010-03-19 2016-06-28 Northeastern University Roaming mobile sensor platform for collecting geo-referenced data and creating thematic maps
WO2011162227A1 (en) 2010-06-24 2011-12-29 富士フイルム株式会社 Stereoscopic panoramic image synthesis device, image capturing device, stereoscopic panoramic image synthesis method, recording medium, and computer program
US8336664B2 (en) * 2010-07-09 2012-12-25 Telecommunication Systems, Inc. Telematics basic mobile device safety interlock
US8532336B2 (en) 2010-08-17 2013-09-10 International Business Machines Corporation Multi-mode video event indexing
CN107093201B (en) 2010-09-13 2021-02-12 巴里·林恩·詹金斯 Streaming interactive media including rendered geometry, texture and lighting data for transmission and control
JP5745067B2 (en) 2010-09-24 2015-07-08 アイロボット・コーポレーション System and method for VSLAM optimization
US8509982B2 (en) 2010-10-05 2013-08-13 Google Inc. Zone driving
GB201018815D0 (en) 2010-11-08 2010-12-22 Tomtom Int Bv High-definition weather for improved routing and navigation systems
JP2012127861A (en) 2010-12-16 2012-07-05 Sumitomo Electric Ind Ltd Vehicle travel support system, computer program, and vehicle travel support method
US8626451B2 (en) 2010-12-21 2014-01-07 Delphi Technologies, Inc. Method and device for characterization and sensing of exhaust gas and control of engines and components for aftertreatment of exhaust gases
DE102011010205A1 (en) 2011-02-03 2011-10-06 Daimler Ag Device for cleaning inner space of motor vehicle i.e. car, has locomotion element e.g. autonomous movable robot, through which device is autonomously moved to predetermined area which is to-be-cleaned at inner space of vehicle
WO2012129425A2 (en) 2011-03-23 2012-09-27 Tk Holdings Inc. Driver assistance system
US20120283912A1 (en) 2011-05-05 2012-11-08 GM Global Technology Operations LLC System and method of steering override end detection for automated lane centering
US9140792B2 (en) 2011-06-01 2015-09-22 GM Global Technology Operations LLC System and method for sensor based environmental model construction
US20120327410A1 (en) 2011-06-23 2012-12-27 Cvg Management Corporation Non-contact media detection system using reflection/absoption spectroscopy
JP5565385B2 (en) 2011-07-16 2014-08-06 株式会社デンソー VEHICLE WIRELESS COMMUNICATION DEVICE AND COMMUNICATION SYSTEM
DE102011111592A1 (en) 2011-08-25 2013-02-28 Wabco Gmbh Method and control device for controlling or regulating a vehicle brake system
US9612797B2 (en) * 2011-08-25 2017-04-04 Ford Global Technologies, Llc Method and apparatus for a near field communication system to exchange occupant information
US20130060461A1 (en) 2011-09-07 2013-03-07 INRO Technologies Limited Method and apparatus for using pre-positioned objects to localize an industrial vehicle
US8798840B2 (en) 2011-09-30 2014-08-05 Irobot Corporation Adaptive mapping with spatial summaries of sensor data
CN102390370B (en) 2011-10-25 2013-07-03 河海大学 Stereoscopic vision based emergency treatment device and method for running vehicles
EP2589932B1 (en) 2011-11-07 2017-02-08 Elektrobit Automotive GmbH Technique for structuring a navigation database
JP5846485B2 (en) 2011-11-30 2016-01-20 株式会社リコー Adhering matter detection apparatus and adhering matter detection method
CN202394343U (en) * 2011-12-12 2012-08-22 浙江金刚汽车有限公司 Voice prompt system for preventing forgetting articles on taxi
US20140310379A1 (en) 2013-04-15 2014-10-16 Flextronics Ap, Llc Vehicle initiated communications with third parties via virtual personality
US8825265B1 (en) 2012-03-16 2014-09-02 Google Inc. Approach for consolidating observed vehicle trajectories into a single representative trajectory
US9145140B2 (en) 2012-03-26 2015-09-29 Google Inc. Robust method for detecting traffic signals and their associated states
WO2013155661A1 (en) * 2012-04-16 2013-10-24 Harman International (Shanghai) Management Co., Ltd. A monitoring system in a vehicle and a method performed therein
US8521352B1 (en) 2012-05-07 2013-08-27 Google Inc. Controlling a vehicle having inadequate map data
US10089537B2 (en) 2012-05-18 2018-10-02 Magna Electronics Inc. Vehicle vision system with front and rear camera integration
DE102012209172A1 (en) 2012-05-31 2013-12-05 Osram Gmbh Lens with internal reflecting reflection layer
JP5422702B2 (en) 2012-06-04 2014-02-19 本田技研工業株式会社 Vehicle display device
US9958272B2 (en) 2012-08-10 2018-05-01 Telogis, Inc. Real-time computation of vehicle service routes
US9056395B1 (en) 2012-09-05 2015-06-16 Google Inc. Construction zone sign detection using light detection and ranging
US9036865B2 (en) 2012-09-12 2015-05-19 International Business Machines Corporation Location determination for an object using visual data
US9158980B1 (en) 2012-09-19 2015-10-13 Google Inc. Use of relationship between activities of different traffic signals in a network to improve traffic signal state estimation
US9110196B2 (en) 2012-09-20 2015-08-18 Google, Inc. Detecting road weather conditions
US9383753B1 (en) 2012-09-26 2016-07-05 Google Inc. Wide-view LIDAR with areas of special attention
JP2014074699A (en) 2012-10-05 2014-04-24 Denso Corp Map information processor, and computer program
US9097800B1 (en) 2012-10-11 2015-08-04 Google Inc. Solid object detection system using laser and radar sensor fusion
US8941676B2 (en) 2012-10-26 2015-01-27 Nvidia Corporation On-chip anti-alias resolve in a cache tiling architecture
WO2014072972A1 (en) * 2012-11-08 2014-05-15 Maimoni Iris System and method for determining whether items have been left in vehicles
US9237084B2 (en) 2012-11-09 2016-01-12 Cisco Technology, Inc. Apparatus, system, and method for providing energy management, profiles, and message blocks in a cable service environment
DE102012112725A1 (en) 2012-12-20 2014-06-26 Continental Teves Ag & Co. Ohg Friction estimation from camera and wheel speed data
JP6006113B2 (en) 2012-12-28 2016-10-12 株式会社日立製作所 Map distribution server for car navigation device, map data distribution system, and road difference data generation method
DE102013200430A1 (en) * 2013-01-14 2014-07-17 Continental Automotive Gmbh System for the transport of luggage
US8917190B1 (en) 2013-01-23 2014-12-23 Stephen Waller Melvin Method of restricting turns at vehicle intersections
US20140222298A1 (en) 2013-02-03 2014-08-07 Michael H. Gurin Systems For a Shared Vehicle
CN104919279A (en) 2013-02-22 2015-09-16 爱信艾达株式会社 Navigation system, navigation system control method and program
US9045041B2 (en) 2013-02-27 2015-06-02 Here Global B.V. Driver behavior from probe data for augmenting a data model
DE102013101990A1 (en) * 2013-02-28 2014-08-28 Eberhard Göbel GmbH & Co. KG Rental car and method for carrying out a rental of a rental car
US9702349B2 (en) 2013-03-15 2017-07-11 ClearMotion, Inc. Active vehicle suspension system
GB201305067D0 (en) 2013-03-19 2013-05-01 Massive Analytic Ltd Apparatus for controlling a land vehicle which is self-driving or partially self-driving
US9863928B1 (en) 2013-03-20 2018-01-09 United Parcel Service Of America, Inc. Road condition detection system
RU132393U1 (en) 2013-03-21 2013-09-20 Федеральное государственное бюджетное образовательное учреждение высшего профессионального образования "Санкт-Петербургский государственный аграрный университет" DEVICE FOR AUTOMATIC CONTROL OF SAFETY OF MOTION OF VEHICLE
US20140302774A1 (en) 2013-04-04 2014-10-09 General Motors Llc Methods systems and apparatus for sharing information among a group of vehicles
US9207323B2 (en) 2013-04-11 2015-12-08 Google Inc. Methods and systems for detecting weather conditions including wet surfaces using vehicle onboard sensors
US9632210B2 (en) 2013-05-07 2017-04-25 Google Inc. Methods and systems for detecting weather conditions using vehicle onboard sensors
KR101519199B1 (en) 2013-05-09 2015-05-11 현대자동차주식회사 System and Method for Detecting of Road
DE202014010839U1 (en) 2013-05-15 2016-11-16 Google Inc. Efficient retrieval of map data during animation
US9488483B2 (en) 2013-05-17 2016-11-08 Honda Motor Co., Ltd. Localization using road markings
US9683928B2 (en) 2013-06-23 2017-06-20 Eric Swanson Integrated optical system and components utilizing tunable optical sources and coherent detection and phased array for imaging, ranging, sensing, communications and other applications
JP6217172B2 (en) * 2013-06-25 2017-10-25 日産自動車株式会社 Vehicle interior monitoring device
US9058524B2 (en) 2013-06-28 2015-06-16 Bendix Commercial Vehicle Systems Llc Measuring the range to an object, in an image, using size categorization
US9719801B1 (en) 2013-07-23 2017-08-01 Waymo Llc Methods and systems for calibrating sensors using road map data
DE102013012810A1 (en) 2013-08-01 2015-02-05 Connaught Electronics Ltd. Method for activating and deactivating an image correction function, camera system and motor vehicle
US9327645B2 (en) * 2013-08-28 2016-05-03 Harman International Industries, Incorporated Providing alerts for objects left in a vehicle
US20150106204A1 (en) * 2013-10-11 2015-04-16 General Motors Llc Methods for providing a vehicle with fuel purchasing options
US9187099B2 (en) 2013-10-17 2015-11-17 Richard M. Powers Systems and methods for predicting weather performance for a vehicle
US20150112585A1 (en) 2013-10-22 2015-04-23 Quicken Loans, Inc. Communication System
US9767655B2 (en) * 2013-10-31 2017-09-19 GM Global Technology Operations LLC Methods, systems and apparatus for providing notification that a wireless communication device has been left inside a vehicle
KR20150058679A (en) 2013-11-20 2015-05-29 한국전자통신연구원 Apparatus and method for localization of autonomous vehicle in a complex
US9915950B2 (en) 2013-12-31 2018-03-13 Polysync Technologies, Inc. Autonomous vehicle interface system
US20150203107A1 (en) 2014-01-17 2015-07-23 Ford Global Technologies, Llc Autonomous vehicle precipitation detection
US20150202770A1 (en) 2014-01-17 2015-07-23 Anthony Patron Sidewalk messaging of an autonomous robot
US9365214B2 (en) 2014-01-30 2016-06-14 Mobileye Vision Technologies Ltd. Systems and methods for determining the status of a turn lane traffic light
DE102014201822A1 (en) 2014-02-03 2015-08-06 Robert Bosch Gmbh Method for operating a vehicle
EP3114574A4 (en) 2014-03-03 2018-03-07 Inrix, Inc. Traffic obstruction detection
JP6427908B2 (en) 2014-03-24 2018-11-28 アイシン・エィ・ダブリュ株式会社 Map information generation system, method and program
US20150278759A1 (en) * 2014-03-26 2015-10-01 Go Taxi Truck, LLC System and Method for Vehicle Delivery Tracking Service
US9898759B2 (en) 2014-03-28 2018-02-20 Joseph Khoury Methods and systems for collecting driving information and classifying drivers and self-driving systems
DE102014208009A1 (en) 2014-04-29 2015-10-29 Bayerische Motoren Werke Aktiengesellschaft Capture static and dynamic objects
US9631933B1 (en) 2014-05-23 2017-04-25 Google Inc. Specifying unavailable locations for autonomous vehicles
US10759442B2 (en) 2014-05-30 2020-09-01 Here Global B.V. Dangerous driving event reporting
US9139204B1 (en) 2014-06-12 2015-09-22 GM Global Technology Operations LLC Road surface condition detection with recursive adaptive learning and validation
WO2015195529A1 (en) 2014-06-20 2015-12-23 Google Inc. Integrating online navigation data with cached navigation data during active navigation
JP6032248B2 (en) 2014-07-09 2016-11-24 トヨタ自動車株式会社 Vehicle lighting device
US9409644B2 (en) 2014-07-16 2016-08-09 Ford Global Technologies, Llc Automotive drone deployment system
US9442487B1 (en) 2014-08-15 2016-09-13 Google Inc. Classifier hierarchies for traffic light and traffic indicator detection
US20160055744A1 (en) 2014-08-19 2016-02-25 Qualcomm Incorporated Systems and methods for traffic efficiency and flow control
US9513132B2 (en) 2014-08-21 2016-12-06 Here Global B.V. Measuring quality in optimal navigation routes by navigation systems
US10036639B1 (en) 2014-09-02 2018-07-31 Metromile, Inc. Systems and methods for determining and displaying a route using information determined from a vehicle, user feedback, and a mobile electronic device
EP3018448B1 (en) 2014-11-04 2021-01-06 Volvo Car Corporation Methods and systems for enabling improved positioning of a vehicle
US9475500B2 (en) 2014-11-12 2016-10-25 GM Global Technology Operations LLC Use of participative sensing systems to enable enhanced road friction estimation
KR101641490B1 (en) 2014-12-10 2016-07-21 엘지전자 주식회사 Driver assistance apparatus and Vehicle including the same
US10262213B2 (en) 2014-12-16 2019-04-16 Here Global B.V. Learning lanes from vehicle probes
US9576200B2 (en) 2014-12-17 2017-02-21 Toyota Motor Engineering & Manufacturing North America, Inc. Background map format for autonomous driving
US9453941B2 (en) 2014-12-22 2016-09-27 GM Global Technology Operations LLC Road surface reflectivity detection by lidar sensor
US9815476B2 (en) 2014-12-22 2017-11-14 Here Global B.V. Method and apparatus for providing road surface friction data for a response action
JP6237656B2 (en) 2015-01-19 2017-11-29 トヨタ自動車株式会社 Vehicle system
DE102015216483A1 (en) 2015-01-29 2016-08-04 Robert Bosch Gmbh Method for operating a coefficient of friction database and coefficient of friction database
KR102534792B1 (en) 2015-02-10 2023-05-19 모빌아이 비젼 테크놀로지스 엘티디. Sparse map for autonomous vehicle navigation
GB2535718A (en) * 2015-02-24 2016-08-31 Addison Lee Ltd Resource management
US10002156B2 (en) 2015-03-16 2018-06-19 Here Global B.V. Version management for incrementally compiled map data
EP3096342B1 (en) 2015-03-18 2017-09-20 IMS Nanofabrication AG Bi-directional double-pass multi-beam writing
US9552564B1 (en) * 2015-03-19 2017-01-24 Amazon Technologies, Inc. Autonomous delivery transportation network
US20170023659A1 (en) 2015-05-08 2017-01-26 5D Robotics, Inc. Adaptive positioning system
US9616773B2 (en) 2015-05-11 2017-04-11 Uber Technologies, Inc. Detecting objects within a vehicle in connection with a service
US9494439B1 (en) 2015-05-13 2016-11-15 Uber Technologies, Inc. Autonomous vehicle operated with guide assistance of human driven vehicles
US10077987B2 (en) 2015-05-28 2018-09-18 Google Llc Dynamically integrating offline and online data in a geographic application
US9836056B2 (en) 2015-06-05 2017-12-05 Bao Tran Smart vehicle
US9386230B1 (en) 2015-06-12 2016-07-05 Google Inc. Day and night detection based on one or more of illuminant detection, lux level detection, and tiling
DE102015211482A1 (en) 2015-06-22 2016-12-22 Robert Bosch Gmbh Method and device for determining a minimum value for the coefficient of friction of a road segment
JP6641614B2 (en) 2015-06-22 2020-02-05 本田技研工業株式会社 Map information updating device and map information updating system
JP6376059B2 (en) 2015-07-06 2018-08-22 トヨタ自動車株式会社 Control device for autonomous driving vehicle
US9884623B2 (en) 2015-07-13 2018-02-06 GM Global Technology Operations LLC Method for image-based vehicle localization
DE102015214338A1 (en) 2015-07-29 2017-02-02 Volkswagen Aktiengesellschaft Determining an arrangement information for a vehicle
US10023231B2 (en) 2015-08-12 2018-07-17 Madhusoodhan Ramanujam Parking autonomous vehicles
TWI611279B (en) 2015-08-31 2018-01-11 國立臺北科技大學 Dispatch system for autonomous vehicle
US10002471B2 (en) 2015-09-30 2018-06-19 Ants Technology (Hk) Limited Systems and methods for autonomous vehicle navigation
US9958864B2 (en) 2015-11-04 2018-05-01 Zoox, Inc. Coordination of dispatching and maintaining fleet of autonomous vehicles
US9916703B2 (en) 2015-11-04 2018-03-13 Zoox, Inc. Calibration for autonomous vehicle operation
US9612123B1 (en) 2015-11-04 2017-04-04 Zoox, Inc. Adaptive mapping to navigate autonomous vehicles responsive to physical environment changes
US9910441B2 (en) 2015-11-04 2018-03-06 Zoox, Inc. Adaptive autonomous vehicle planner logic
US9494940B1 (en) 2015-11-04 2016-11-15 Zoox, Inc. Quadrant configuration of robotic vehicles
US9878664B2 (en) 2015-11-04 2018-01-30 Zoox, Inc. Method for robotic vehicle communication with an external environment via acoustic beam forming
US9507346B1 (en) 2015-11-04 2016-11-29 Zoox, Inc. Teleoperation system and method for trajectory modification of autonomous vehicles
US9720415B2 (en) 2015-11-04 2017-08-01 Zoox, Inc. Sensor-based object-detection optimization for autonomous vehicles
WO2017079341A2 (en) 2015-11-04 2017-05-11 Zoox, Inc. Automated extraction of semantic information to enhance incremental mapping modifications for robotic vehicles
US9952591B2 (en) 2015-11-24 2018-04-24 Northrop Grumman Systems Corporation Spatial-temporal forecasting for predictive situational awareness
US10018472B2 (en) 2015-12-10 2018-07-10 Uber Technologies, Inc. System and method to determine traction of discrete locations of a road segment
US9840256B1 (en) 2015-12-16 2017-12-12 Uber Technologies, Inc. Predictive sensor array configuration system for an autonomous vehicle
US9841763B1 (en) 2015-12-16 2017-12-12 Uber Technologies, Inc. Predictive sensor array configuration system for an autonomous vehicle
US9901165B2 (en) 2015-12-21 2018-02-27 Casabella Holdings, L.L.C. Dispensing brush
US10347134B2 (en) 2015-12-31 2019-07-09 Gt Gettaxi Limited System for navigating drivers to selected locations to reduce passenger wait time
US20170227470A1 (en) 2016-02-04 2017-08-10 Proxy Technologies, Inc. Autonomous vehicle, system and method for structural object assessment and manufacture thereof
KR102373926B1 (en) 2016-02-05 2022-03-14 삼성전자주식회사 Vehicle and recognizing method of vehicle's position based on map
US10032085B2 (en) 2016-02-24 2018-07-24 Audi Ag Method and system to identify traffic lights by an autonomous vehicle
US9902403B2 (en) 2016-03-03 2018-02-27 Uber Technologies, Inc. Sensory stimulation for an autonomous vehicle
US9990548B2 (en) 2016-03-09 2018-06-05 Uber Technologies, Inc. Traffic signal analysis system
CN116659526A (en) 2016-03-15 2023-08-29 康多尔收购第二分公司 System and method for providing vehicle awareness
US9739881B1 (en) 2016-03-24 2017-08-22 RFNAV, Inc. Low cost 3D radar imaging and 3D association method from low count linear arrays for all weather autonomous vehicle navigation
US9672734B1 (en) 2016-04-08 2017-06-06 Sivalogeswaran Ratnasingam Traffic aware lane determination for human driver and autonomous vehicle driving system
US9953523B2 (en) 2016-04-22 2018-04-24 Here Global B.V. Node-centric navigation optimization
US10459087B2 (en) 2016-04-26 2019-10-29 Uber Technologies, Inc. Road registration differential GPS
JP6418199B2 (en) 2016-04-27 2018-11-07 トヨタ自動車株式会社 Automatic vehicle driving system
US10255816B2 (en) 2016-04-27 2019-04-09 Uber Technologies, Inc. Transport vehicle configuration for impaired riders
US9672446B1 (en) 2016-05-06 2017-06-06 Uber Technologies, Inc. Object detection for an autonomous vehicle
CN105975083B (en) 2016-05-27 2019-01-18 北京小鸟看看科技有限公司 A kind of vision correction methods under reality environment
CA3026105A1 (en) 2016-06-02 2017-12-07 Ge Oil & Gas Esp, Inc. System and method for well lifecycle planning visualization
US11721205B2 (en) 2016-06-13 2023-08-08 Here Global B.V. Method, apparatus, and computer program product for processing sensor data
US9953535B1 (en) 2016-06-27 2018-04-24 Amazon Technologies, Inc. Annotated virtual track to inform autonomous vehicle control
US20180003511A1 (en) 2016-07-01 2018-01-04 Uber Technologies, Inc. Autonomous vehicle localization using submaps
US20180164119A1 (en) 2016-07-29 2018-06-14 Faraday&Future Inc. System and method for generating an environmental condition database using automotive sensors
EP3497405B1 (en) 2016-08-09 2022-06-15 Nauto, Inc. System and method for precision localization and mapping
US10438493B2 (en) 2016-08-24 2019-10-08 Uber Technologies, Inc. Hybrid trip planning for autonomous vehicles
US20180060778A1 (en) 2016-08-31 2018-03-01 Uber Technologies, Inc. Driver location prediction for a transportation service
US10515390B2 (en) 2016-11-21 2019-12-24 Nio Usa, Inc. Method and system for data optimization
CN111542860B (en) 2016-12-30 2024-08-27 辉达公司 Sign and lane creation for high definition maps of autonomous vehicles
US9904375B1 (en) 2017-04-10 2018-02-27 Uber Technologies, Inc. LIDAR display systems and methods

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060002586A1 (en) * 2004-06-15 2006-01-05 Manoj Aggarwal Method and apparatus for detecting left objects
JP2006159939A (en) * 2004-12-02 2006-06-22 Xanavi Informatics Corp Notice device for in-cabin object to be informed
US20090146813A1 (en) * 2007-12-10 2009-06-11 Acropolis Engineering Automobile forgotten passenger alarm and notification
US20090216600A1 (en) * 2008-02-27 2009-08-27 Montiss Llc Systems and methods for arranging a transport transaction
US20120242492A1 (en) * 2011-03-25 2012-09-27 Tov 1 LLC Seat occupancy detection and display system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3295421A4 *

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11505984B2 (en) 2015-05-11 2022-11-22 Uber Technologies, Inc. Detecting objects within a vehicle in connection with a service
GB2549615A (en) * 2016-04-18 2017-10-25 Ford Global Tech Llc Vehicle security system
EP3355254A1 (en) * 2017-01-30 2018-08-01 Panasonic Intellectual Property Corporation of America Method for online delivery system, management apparatus, and recording medium having program stored therein
EP3471035A1 (en) * 2017-10-16 2019-04-17 Flex Ltd. Method and system for tracking and optimizing cargo utilization and volume measurement and imaging sensing using lidars and video cameras
US11151506B2 (en) 2017-10-16 2021-10-19 Flex Ltd. Method and system for tracking and optimizing cargo utilization and volume measurement and imaging sensing using lidars and video camera
US11625926B2 (en) * 2018-03-22 2023-04-11 Kioxia Corporation Information processing device, information processing method, and information processing program product
CN111052193B (en) * 2018-06-29 2022-08-19 百度时代网络技术(北京)有限公司 Anti-theft technique for autonomous vehicles for transporting goods
WO2020000396A1 (en) 2018-06-29 2020-01-02 Baidu.Com Times Technology (Beijing) Co., Ltd. Theft proof techniques for autonomous driving vehicles used for transporting goods
CN111052193A (en) * 2018-06-29 2020-04-21 百度时代网络技术(北京)有限公司 Anti-theft technique for autonomous vehicles for transporting goods
EP3815061A4 (en) * 2018-06-29 2022-01-26 Baidu.com Times Technology (Beijing) Co., Ltd. Theft proof techniques for autonomous driving vehicles used for transporting goods
FR3091379A1 (en) * 2018-10-31 2020-07-03 Valeo Comfort And Driving Assistance UNIT, SYSTEM AND METHOD FOR DETECTION OF AN OBJECT OR DEGRADATION IN A COCKPIT OF A MOTOR VEHICLE
WO2020088892A1 (en) * 2018-10-31 2020-05-07 Valeo Comfort And Driving Assistance Unit, system and method for detecting an object in or damage to a passenger compartment of a motor vehicle
EP3899894A4 (en) * 2018-12-20 2022-01-26 Gentex Corporation Systems and methods for ride share monitoring
FR3121101A1 (en) * 2021-03-29 2022-09-30 Psa Automobiles Sa Method for detecting and identifying an object placed in a storage compartment of a motor vehicle and associated storage compartment

Also Published As

Publication number Publication date
US9790729B2 (en) 2017-10-17
RU2017142642A3 (en) 2019-11-20
US20170284148A1 (en) 2017-10-05
IL255562B (en) 2021-10-31
RU2725681C9 (en) 2020-09-01
US20160332535A1 (en) 2016-11-17
US20170175431A1 (en) 2017-06-22
US10662696B2 (en) 2020-05-26
US10329827B2 (en) 2019-06-25
CN107873098A (en) 2018-04-03
US20170284147A1 (en) 2017-10-05
CA2985619A1 (en) 2016-11-17
AU2016262526B9 (en) 2021-06-03
US11505984B2 (en) 2022-11-22
AU2016262526A1 (en) 2017-11-30
RU2017142642A (en) 2019-06-11
EP3295421A1 (en) 2018-03-21
RU2725681C2 (en) 2020-07-03
AU2016262526B2 (en) 2021-05-20
US20200270931A1 (en) 2020-08-27
US9616773B2 (en) 2017-04-11
EP3295421A4 (en) 2018-10-24
US9909349B2 (en) 2018-03-06
IL255562A (en) 2018-01-31
US9963926B2 (en) 2018-05-08
KR102395766B1 (en) 2022-05-10
KR20180005693A (en) 2018-01-16
US20190271184A1 (en) 2019-09-05
US20180223584A1 (en) 2018-08-09

Similar Documents

Publication Publication Date Title
US11505984B2 (en) Detecting objects within a vehicle in connection with a service
US11605246B2 (en) Programmatically determining location information in connection with a transport service
US11047700B2 (en) Navigation and routing based on image data
JP7115482B2 (en) Control system and control method
US20190244167A1 (en) Information processing apparatus, pick-up and delivery system, pick-up and delivery method, and non-transitory computer readable recording medium
US11082819B2 (en) Mobility service supporting device, mobility system, mobility service supporting method, and computer program for supporting mobility service
US12002126B2 (en) Vehicle information processing apparatus, vehicle information processing system, and method of processing vehicle information
US20210278224A1 (en) Coordinating trips by vehicles in an on-demand environment
US11989795B1 (en) Partner trip application programming interface
US20230351313A1 (en) Logistics management server, information processing method, and information processing system
JP2023079922A (en) Control device, system, program, and notification method
JP2024066265A (en) Management method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16793475

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 11201709188R

Country of ref document: SG

ENP Entry into the national phase

Ref document number: 2985619

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 255562

Country of ref document: IL

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2016262526

Country of ref document: AU

Date of ref document: 20160511

Kind code of ref document: A

ENP Entry into the national phase

Ref document number: 20177035479

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2016793475

Country of ref document: EP