KR20180005693A - Detecting objects in a vehicle in relation to services - Google Patents

Abstract

A system for monitoring a vehicle used when providing a service is disclosed. The system detects when the service is complete and when the service detects that the service has been completed, the system determines from the one or more sensors of the vehicle that an object that had to be transferred from the vehicle after the completion of the transportation service remains in the vehicle. Thereafter, the system automatically initiates the execution of the remedial action.

Description

Detecting objects in a vehicle in relation to services

This application claims priority from United States Patent Application Serial No. 14 / 708,611, filed May 11, 2015, entitled " DETECTING OBJECTS WITHIN A VEHICLE IN CONNECTION WITH A SERVICE, " It is incorporated by reference.

The service provisioning system can prepare services to be provided to the requesting user through the use of mobile computing devices. For example, a user may request a service, such as a shipping or delivery service, by operating the user's mobile computing device, and the service preparation system may process the request to select a suitable provider capable of providing the service. The service preparation system may also monitor the service to determine the current location of the vehicle, such as before or during the service progress.

Figure 1 illustrates an exemplary system for detecting objects in a vehicle in connection with a transportation service.
Figure 2 illustrates an exemplary method for detecting objects in a vehicle in connection with a transportation service.
Figure 3 illustrates another exemplary method for detecting objects in a vehicle in connection with a transportation service.
Figures 4A-4D illustrate examples of user interfaces displayed on a mobile computing device in various use case examples.
5 is a block diagram illustrating a computing device in which the embodiments described herein may be implemented.
Figure 6 is a block diagram illustrating a computer system in which the examples described herein may be implemented.

According to the examples described herein, a service provisioning system that implements network services can receive requests for location-based services and prepare those services to be provided by service providers. As used herein, service providers may be used by human operators (or human-driven vehicles) to receive and / or accept invitations for services and, in some instances, by network services And may include autonomous (or auto-driving) vehicles that may be used. Whether a human-driven vehicle is assigned to provide the service or an autonomous vehicle (AV) is assigned, the network service is responsible for performing the processes and, if necessary, Information from the set can be used. For example, a computing system associated with a vehicle may determine, through use of sensor data, whether the service has been properly completed or whether the object was inadvertently left in the vehicle.

In some instances, the computing system may determine that the vehicle is ready to provide location-based services. According to an implementation, the computing system may be a computing system (e.g., an on-board computing system associated with a vehicle or a driver device operated by a vehicle driver), or a computing system (s) (E.g., a backend server system remote from the vehicle). During the performance of the service, the computing system may detect that an object is in the vehicle based on a first set of information from one or more sensors of the vehicle at a first time. As described herein, a sensor is a device that can detect some characteristic of the environment or detect a change in an event or a total amount and provide a corresponding output (e.g., sensor data). The computing system may detect whether an object has been transferred from a vehicle based on a second set of information from one or more sensors at a time when the service is being completed or after the service has been completed. In response to detection that the object is still in the vehicle (e.g., not transferred from the vehicle), the computing system may automatically perform a remedial action (e.g., the calibration action may not be performed In which case the driver or vehicle may perform additional actions that would not have been performed.

In various examples, one or more sensors, such as a set of cameras, a set of metering devices, a set of suspension sensor sensors, a set of light detection and ranging (LIDAR) sensors, a set of ultrasonic sensors, A set of position-determination sensors, a set of wireless-frequency sensors (such as Bluetooth or Wi-Fi transceivers) as well as other sensors (e.g., engine sensors, tire pressure sensors, Or shut-off sensors, seat belt sensors, external stereo cameras for AVs or LIDAR sensors, etc.). The set of sensors used herein may correspond to one or more individual sensors or one or more arrays of sensors. According to an implementation, the computing system may receive information from one or more sets of sensors (e.g., an individual sensor or a combination of a plurality of different sensors) to detect whether the object is in the vehicle and / Data) can be used.

For example, the computing system may detect the position of an object inside the vehicle based on image information from a set of cameras disposed in the vehicle, and the individual lenses of the set of cameras are directed toward the interior of the vehicle. In another example, the computing system may be configured to detect that an object is in or out of a compartment or interior of a vehicle based on weight information from a compartment of the vehicle or a set of metering devices disposed in the interior can do. The set of metering devices may be disposed within the vehicle such that the individual measurement platform of the set of metering devices is disposed on or below the lowermost surface of an individual compartment or interior of the vehicle.

In another example, a computing system may determine that a mobile computing device is in a vehicle based on whether a mobile computing device with wireless communications is communicating with antennas in the vehicle, using wireless protocols such as Bluetooth or Wi-Fi, Can be detected. In another example, if the mobile computing device has its own position sensors, the computing system may determine that there is a possibility that the mobile computing device is in motion with the vehicle, The outputs of the position sensors can be compared. Still further, in yet another example, the computing system is configured to determine whether or not a computing system is in a compartment or interior of a vehicle based on compression information from a set of suspension sensors disposed proximate to, or contained within, the suspension system of the vehicle It can be detected that it has been transferred.

It is to be appreciated that the object is still in the vehicle as part of or during the performance of the on-demand service and that the object is still in the vehicle at the time the service is completed, The computing system may automatically perform a calibration action such as an action communicating a message to a computing device associated with the user requesting the service and / or an action notifying the service preparation system that the object still exists in the vehicle . According to some examples, the computing system may also be configured to instruct the driver to stay at the location of the vehicle when the object is detected to be still present in the vehicle for at least a predetermined period of time, . In another example, the computing system may instruct the driver to move to a predetermined or specific location where the object may be relocated from the vehicle, or may cause the vehicle to move to that location.

According to an example, a computing system may also use sensor data to identify what an object is or what type of object is in the vehicle. Still further, in some instances, the computing system may also be able to detect objects lying in the vehicle and / or from the vehicle to determine (or as part of determining this time) the time at which the service is started and / You can use detection of the object being migrated.

As used herein, a client device, a driver device, and / or a mobile computing device may be a desktop computer, a cellular device or smart phone, a personal digital assistant (PDA), a laptop computer, a tablet device And the like, which may provide network connectivity and processing resources for communicating over one or more networks with a remote computing system (s), such as a service provisioning system. The remote computing system may refer to one or more computing systems or servers (e.g., corresponding to a back-end server system of network services) remote from the client device, the driver device, and / or the vehicle. Still further, in the examples described herein, the driver device or computing system associated with the vehicle may also correspond to on-demand hardware of the vehicle, such as in-vehicle or on-board computing systems with network connectivity and location-determining capabilities.

(And / or on-demand) services such as transportation services, food trucking services, delivery services, entertainment services, etc., which will be arranged between users and service providers, )) Services. In other instances, the service provisioning system may be implemented by any entity that provides merchandise or services for purchase through the use of computing devices and network (s). For simplicity, in the examples described herein, the service provisioning system corresponds to a shipping preparation system that prepares transportation services (e.g., objects or people transportation) to be provided to passengers / users by drivers or vehicles .

One or more of the examples described herein provide that the methods, techniques, and actions performed by a computing device are performed as a program or as a computer-implemented method. As used herein, "according to program" means "through use of code or computer-executable instructions. &Quot; These instructions may be stored in one or more memory resources of the computing device. The steps performed according to the program may not be automatic or automatic.

One or more of the examples described herein may be implemented using program modules, engines or components. A program module, engine or component may comprise a program, sub-routine, part of a program, or a software or hardware component capable of performing one or more of the mentioned operations or functions. As used herein, a module or component may reside on a hardware component separate from other modules or components. Alternatively, the module or component may be a shared element or process of other modules, programs, or machines.

Some examples described herein may in general require the use of computing devices, including processing and memory resources. For example, one or more of the examples described herein may be implemented in computing devices, such as servers, desktop computers, cellular or smart phones, personal digital assistants (e.g., PDAs), laptop computers, printers, , Network devices (e.g., routers), and tablet devices. Memory, processing, and network resources may all be used in connection with the setting, use, or performance of any of the examples described herein (including the implementation of any method or the implementation of any system).

Moreover, one or more of the examples described herein may be implemented through use of instructions executable by one or more processors. These instructions may be loaded onto a computer-readable medium. The machines illustrated in or described with reference to the figures below provide examples of processing resources and computer-readable media on which instructions for implementing the examples described herein may be loaded and / or executed . In particular, the plurality of machines illustrated with the examples described herein include various types of memory and processor (s) for holding data and instructions. Examples of computer-readable media include permanent memory storage devices such as hard drives on personal computers or servers. Other examples of computer storage media include portable storage units such as CD or DVD units, flash memory (e.g., mounted on smart phones, multifunction devices or tablets), and magnetic memory. Computers, terminals, network enabled devices (e.g., mobile devices such as cell phones) all include processors and memory, and machines and devices that utilize instructions stored on computer- . In addition, examples may be implemented in the form of computer-programs, or computer-usable carrier media capable of mounting such programs.

System Description

Figure 1 illustrates an exemplary system for detecting objects in a vehicle in connection with a transportation service. According to an implementation, such a system may be implemented and operated on by a mobile computing device of a driver, a vehicle in-vehicle computing system, or a service provisioning system that provides network services. For simplicity, in the example of FIG. 1, the system 100 is implemented by an in-vehicle computing system of a vehicle, such as an AV, via execution of instructions stored in one or more memory resources of an in-vehicle computing system, . As described herein, the system 100 may include or be part of or correspond to an application or operating system of an in-vehicle computing system in communication with the service provisioning system 190 via one or more networks.

According to the examples, the system 100 includes a system management unit 110, a positioning unit 120, an object detection unit 130, a location database 140, a trip database 150, and a service interface 160 . For simplicity, other components of the system 100, such as user interface components, device databases, device interfaces for communicating with other components, etc., are not illustrated in FIG. The components of the system 100 may be combined to detect objects that are placed in and / or transferred from the vehicle in connection with transportation services. The logic may be implemented in hardware and / or in a variety of applications (e.g., software) of a computing system that implements the system 100.

In one example, the system 100 may communicate with the service provisioning system 190 via the service interface 160. As described herein, the service provisioning system 190 may include a plurality of service providers (e.g., driver devices and / or in-vehicle computing systems) and a plurality of service providers And may communicate with client (or user) devices. The service provisioning system 190 may be implemented on network side resources, such as one or more servers or data centers, or may be implemented on alternative architectures (e.g., peer-to-peer networks, etc.) ) May be implemented through other computer systems. Typically, a user may operate a client or user device (not shown in FIG. 1 for simplicity) to communicate with the service provisioning system 190 via the network (s) A pick-up location of the package). In some instances, the user device may operate a designated service application that communicates with the service provisioning system 190. The service preparation system 190 can receive the request and select the service provider that provides the transportation service according to the program.

According to the implementation, if the service preparation system 190 selects a human operator to provide the service, the service preparation system 190 sends an invitation to the selected driver's device to invite the requesting passenger The user may be given the option of accepting or declining. In another example, if the service preparation system 190 selects an AV to provide service from a plurality of available AVs (and / or human drivers) as shown in FIG. 1, the service preparation system 190 (Referred to as service instructions 191) for the service to the system 100 via the service interface 160.

In accordance with the examples, the service interface 160 allows the system 100 to exchange data between the system 100 and the service provisioning system 190. For example, the service interface 160 may use one or more network resources of the in-vehicle computing system to exchange communications via one or more wireless networks (e.g., cellular transceiver, WLAN transceiver, etc.). The service interface 160 may include or use an API, such as an externally facing application programming interface (API), for communicating data with the service provisioning system 190. The outbound oriented API may be implemented using any number of methods, such as web-based forms, programmatic access via restful APIs, Simple Object Access Protocol (SOAP) May provide access to the service provisioning system 190 via secure access channels across the network, such as by remote procedure calls (RPCs), scripting accesses, and the like.

In the example of FIG. 1, the service preparation system 190 has processed a request for a transportation service by a user and selected an AV implementing the system 100 to provide transportation services to the requesting user. The service preparation system 190 may determine the current position (or most recently determined positions) of the available service providers that are close to the pick-up position and pickup position specified by the requesting user (e.g., within a determined distance of the pick- , It may be determined that the AV implementing the system 100 will provide transportation services. The service preparation system 190 may send service commands 191 for the transportation service, which commands the vehicle to the location and / or the to-do location to go. In some instances, service commands 191 may include user information 192 (e.g., a user identifier, user contact information, user rating, etc.), a pick-up location 193 (e.g., address, landmark, (E.g., location data points corresponding to latitude and longitude coordinates), destination location 194 and / or other information such as the type of transportation service (e.g., human transport, package delivery, etc.) And may include details of the number. In one example, the service instructions 191 may cause the vehicle control component (s) of the AV to at least partially control the AV to move to appropriate locations. In other words, the vehicle control component (s) may use some or all of the service instructions 191 (in addition to other information) to control the vehicle to move to the pick up position and / or the destination or departure position.

The system management unit 110 may receive service instructions 191 and may perform a plurality of operations on behalf of the system 100 in response. For example, the system management unit 110 may process the service commands 191 by determining information about the transportation service to be performed by the system management unit 110, determine the route (s) to be moved to the pickup position and / And / or control the AV to move to the destination location.

In one example, the trip management component of system manager 110 generates a trip entry 151 for the transportation service using some or all of the commands from service commands 191, In the trip database 150 of the server 100. Thereafter, the trip management component sends information about the location of the vehicle (e.g., when the vehicle performs a transportation service (e.g., moving from a current location to a pick up location, picking up a user or object, It is possible to update the trip entry 151. [ The trip management component may also record the time and location of the vehicle in the trip entry 151 when the service commands 191 are received, when the service is started, and when the service is terminated. The trip entry 151 may be used by the system 100 and / or the service preparation system (e. G., The system 100) to determine the cost of the shipping service for the requesting user (e. 190). ≪ / RTI >

The positioning unit 120 provides information on the position of the vehicle to the system management unit 110 by receiving location data points 121 (e.g., periodically) from one or more area-aware resources of the in-vehicle computing system can do. For example, the positioning unit 120 can be configured to determine a current location data point 121 of the vehicle, for example, via a device interface, a GPS receiver of the driver device, one or more other sensors and / or one or more transceivers Cellular transceiver, Wi-Fi transceiver) to periodically determine the current location of the vehicle. In one example, the positioning unit 120 may perform a call to the GPS receiver periodically, such as every two seconds or four seconds, in order to periodically receive the current position data point 121 of the driver device . The positioning unit 120 may also determine an associated timestamp of each determined location data point 121 of the vehicle. According to an implementation, the position determination unit 120 may receive a time stamp (and / or an error value) from a GPS receiver with a location data point 121 of the vehicle, or may receive a separate clock (e.g., Clock) may be used to determine the time stamp of the location data point 121. The system management unit 110 may store the location data points 121 in the trip entry 151 and / or the location database 140 to maintain data about the location where the vehicle has moved in relation to the transportation services. Additionally or alternatively, the position determination unit 120 may store the position data points 121 of the vehicle in the position database 140.

Referring back to the system management section 110, the trip management component may also use the location data points 121 from the location determination section 120 to monitor the vehicle. The trip management component may determine the location and state of the system 100 or vehicle in connection with the performance of the transportation service (e.g., the performance of the vehicle at which stage of the transportation service). For example, the state of the system 100 may have already been available, and this state corresponds to a state in which the vehicle can be assigned a transportation service from the service management system 190. When the system 100 receives the service commands 191, the trip management component may cause the system 100 to be in a different state, e.g., a root state, which corresponds to the vehicle moving to the pickup position. Similarly, each time the trip management component determines that the transportation service has been initiated or terminated, the trip management component may cause the system 100 to be in a separate state (e.g., each transition from an on route state to a trip state) Or may change from an on-trip state to a completed or available state). The trip management component may store information about the state and / or store it in the trip entry 151 of the transportation service together with the corresponding location data points 121 when a state change occurs.

Still further, additionally or alternatively, the service preparation system 190 may locally generate and store its own trip entry corresponding to or similar to the trip entry 151 stored in the system 100 have. The trip management component may periodically provide the service described above for the transportation service to the service provisioning system 190, e.g., via the status information 111 and / or location data points 113. [ The service preparation system 190 may use the information received from the system 100 to provide updated information to the shipping service (e. G., The location of the AV and / or the current state of the AV) to the requesting user. In addition, the service provisioning system 190 may be configured to receive from the state information 111 and / or location data points 113 the availability of an AV when the transportation service is initiated or completed and / or to provide other transportation services You can decide when.

In addition to creating and updating the trip entry 151 for the transportation service, the system management section 110 may initiate the execution of the transportation service based at least in part on the service instructions 191. According to some examples, the vehicle control component of the system management unit 110 may control the AV to perform the transportation service. According to an implementation, the vehicle control component may interface with the AV control system or may correspond to the AV control system. As described herein, an AV may be configured to (i) include computing resources implementing system 100 and / or AV control system, (ii) other sub-components for controlling various other operations or portions of AV, or Sub-systems and (iii) a plurality of vehicle portions. For the sake of simplicity, not all parts of AV are discussed herein.

In accordance with the examples, various portions of the AV may include a frame or chassis, a body (e.g., doors, bumpers, windows, cockpit, seats, trunk space, etc.), tires, suspension system, braking system, steering system , Electrical systems (e.g., including batteries, wires, cables, alarm sub-systems, etc.), transmission systems But are not limited to, systems. The AV may also include electronic control units (ECUs) including controllers and / or microcontrollers, and other electronic components capable of controlling various systems or sub-systems of the AV, such as a door control unit, an engine control unit, A speed control unit, a transmission control unit, and the like. The AV control system may be programmed via execution of instructions stored in memory by one or more processors to control AV by transmitting control signals and receiving data (e.g., feedback or sensor data) from various systems and vehicle parts . For this application, the AV control system can be commanded to control the AV, and thus the AV can safely navigate the roads and distances to move from one location to another with respect to transportation services.

Still further, the AV may include a location module or resources, such as a GPS receiver and / or wireless transceivers, to determine the location of the AV relative to other objects or the location of the AV relative to the coordinate system. The AV may also include a plurality of sensors, such as a set of cameras, a set of stereo cameras, a set of radars, a set of LIDAR sensors, etc., to enable the AV to detect the real-time environment of the AV. One or more (or all) of these sensors may be located outside the vehicle (e. G., On a vehicle < / RTI > to detect information about the environment around the AV, such as the location of the vehicle relative to roads, Front, rear, roof, etc.). For example, if the AV is located at a particular location (e.g., pickup location 193 and / or destination location 194 displayed in service instructions 191) from a current location or location (e.g., determined from a GPS receiver as location data point 121) The AV control system may access the stored detailed spatial map (s) (e.g., including map data 112) of the area in which the AV is located and determine the root (s) to move to a particular location . Based on the environment detected around the AV (from the data periodically and continuously detected by the set of sensors), the AV control system determines whether the AV decides to take a specific action (s) Operations can be performed periodically. As a result of each decision action, the AV control system may determine, for example, that the AV will move in a particular direction at a particular instant in an hour instance, and at a next instance of time, the AV may decelerate at a certain rate, A slight change, or a change of lane. Thus, the AV control system can provide control signals to various vehicle components / systems.

In one example, when the AV moves to a pickup position to initiate a transportation service, the system 100 may periodically determine that the AV has reached a particular pickup position (e.g., a starting position). In other words, the system 100 may periodically check when the AV reaches a predetermined distance from the pick-up position or when it is within this predetermined distance (or within a predetermined estimated travel time). For example, the trip management component periodically compares the pickup position 193 with the current position (e.g., the most recent position data point 121 provided by the positioning portion 120) It can be determined whether or not it is within the movement time. When the trip management component determines that the AV is within the predetermined distance or estimated travel time, the trip management component may change the state of the system 100 from the on-route state to the reached / reached state. The status information 111 for this change can be sent to the service preparation system 190 so that the service preparation system 190 can provide updated information about the AV to the requesting user's device.

Moreover, according to the example of FIG. 1, the system 100 may also determine whether the object (s) are in the vehicle in connection with the transportation service. In some cases, the driver or AV control system determines whether the requesting user needs to determine whether the vehicle has placed an object (bag, boxes, groceries) in the vehicle during transportation, or whether the object was placed in the vehicle during the delivery transportation service You need to decide. As described herein, an object may correspond to the person (s) or article (s) that are in or out of the vehicle. The object detection unit 130 may communicate with one or more sets of AV sensors or one or more combinations of AV sensors via one or more vehicle or sensor interfaces to receive the sensor data 131, (Or referred to herein as an object detection process or action) whether the object has been placed (or migrated from the AV). In one example, the object detection unit 130 may provide a set of control signals to the sensors (e.g., to trigger the sensors to provide the sensor data 131) or to trigger the sensors to provide the sensor data 131 Can be transmitted. Such sensors include a set of cameras disposed in a vehicle or provided to a vehicle to detect the presence, position and / or weight of one or more objects, a set of metering devices, a set of suspension sensors, a light detection and ranging (LIDAR) sensor A set of radio-frequency antennas and / or transceivers, and / or a set of ultrasonic sensors.

In some examples, the object detection unit 130 may be configured to detect (i) sensor data 131 from one or more sensors, (ii) use other information (e.g., models or previously stored data) (Iii) whether the object was placed in the AV (when the object was not in the AV), or whether the object has been transferred from the AV The object detection process can be performed. For example, the object detection unit 130 may receive image information from one or more cameras disposed in the trunk of the AV (lens towards the inside of the trunk), and may use image processing or image recognition processes and / (E.g., representing the image of the trunk before receiving the image or service instructions 191 of the trunk when the trunk is empty) with the previously stored baseline image information (e.g., . The stored baseline image information can be a predefined image or an image captured by one or more cameras at a time before the AV reaches the pickup position 193 or before the AV reaches the pickup position 193 Lt; / RTI > Still further, in one example, the object detection unit 130 may be configured to detect image processing (and other stored or received from the service preparation system 190) to identify what the object is based on the visual characteristics of the object in the image Data) can be used.

In other instances, when one or more other sensors are used to detect the presence or location of an object, the object detection unit 130 may use different baseline or model information as a basis for comparison with the received sensor data 131 . For example, the object detection unit 130 may receive sensor data 131 from one or more of the metering devices or sensors disposed in the compartment of the AV, and thus the individual measurement platform of the AV may be on such compartment bottom surface On the lower surface, or on the lower surface. The object detection unit 130 may determine a change in measurements from the sensor data 131 at a time before the AV reaches the pickup position 193 or near the pickup position 193 or after the service is started.

According to the implementation, the object detection unit 130 can perform the object detection process at various times. For example, the object detection unit 130 may determine when the trip management component determines whether the AV moves to the pickup position 193 and / or whether AV is within a predetermined distance or a predetermined estimated travel time of the pickup position 193 The object detection process can be performed simultaneously. The object detection unit 130 may perform calls to the set of sensors to periodically receive the sensor data 131. In another example, the object detection unit 130 may perform an object detection process at various instances when commanded or triggered by the system management unit 110 (via the trigger 115). For example, when the trip management component determines that the AV is within a predetermined distance or a predetermined estimated travel time of the pick-up position 193, the trip management component may cause the object detection unit 130 to trigger (e.g., 115). Still further, in another variation, the trip management component determines that the AV is stationary for a predetermined period of time and / or when the AV is within a predetermined distance or predetermined estimated travel time of the pickup position 193, The trigger 115 may be transmitted when it is determined that it is stationary for a predetermined period of time. As another example, the object detection unit 130 may be triggered to perform the object detection process when the trip management component determines that the transportation service has been started. The system 100 may cause the sensor (s) to wake up (from an intermediate state or a sleep state) by causing the object detection unit 130 to perform an object detection process at selected times, Lt; RTI ID = 0.0 > and / or < / RTI > only when necessary.

If the object detection unit 130 detects the position of the object or if the object detection unit 130 detects that the object lies in the vehicle based on the first set of the sensor data 131, Information 131 to the system management unit 110. [ The object information 133 may indicate to the system manager 110 that an object is in the AV (e.g., the message or binary bit may be toggled) and / or used by the AV to perform detection And may include information about the object based on the types of sensor (s). For example, the additional information may include (i) what the object is (e.g., box, bag, skateboard, etc.), (ii) one or more images of the object captured by the set of cameras if cameras are used, (E.g., the dimensions of the object when weighing devices or suspension sensors are used, such as weight, cameras, LIDAR sensors or ultrasonic sensors), and / or (iv) , Any compartment or location of the object). The trip management component may store some or all of the trip entries 151 and object information 133 associated with the transportation service so that information about the objects may be logged and associated with a particular user.

As described, the trip management component may determine that the transportation service has been started at the same time (or before or after detection) that the object detection section 130 has detected the object. In some instances, the system management unit 110 may determine that the transportation service has been started, based at least in part upon detecting that the object is in the vehicle. A transportation service may correspond to transporting a person (and / or a person's things) from one location to another location, or from a location to another location. The object detection unit 130 can detect whether a person and / or objects have entered a vehicle or only an object (without a person) is located in the vehicle, from one or more sensors, such as cameras or metering devices. In this example, the cameras (or LIDAR sensors or sensors) are used to detect the object detection unit 130 to use the image information from the camera (s) and process the image (s) Ultrasonic sensors) may be disposed inside the cab / interior of the vehicle. Additionally or alternatively, a set of metering devices or a set of suspension sensors disposed adjacent to or included in the suspension system of the AV may determine that an amount of mass has been added to the vehicle.

Once the transportation service has been started, the trip management component can store in the trip entry 151 information about the performance of the transportation service (e.g., the travel time and / or the moving location of the AV), including the start time and start location of the transportation service have. When the AV moves to the destination location 194 of the transport service, the trip management component also uses the location data points 121 to determine whether the AV is in the destination location 194 (at least in part to determine that the transport service is complete) To arrive at the destination location 194.

In one example, in cases where the destination detection unit 130 has previously detected an object in connection with the transportation service (e.g., when the passenger placed the bag in the trunk of the AV), the trip management component determines whether the object has been transferred from the vehicle The object detection unit 130 can also detect the object. The object detection unit 130 can detect whether the object has been transferred from the vehicle at different times according to the changes. According to one example, the system management unit 110 may cause the object detection unit 130 to perform the object detection process at the same time when the AV moves to the destination position 194 (e.g., periodically) When the trip management component determines that it is within a predetermined distance or a predetermined estimated movement time of the object. Additionally or alternatively, when the trip management component determines that the AV is stationary for a predetermined time duration, the trip management component determines whether the AV is within a predetermined distance of the destination location 194 or within a predetermined estimated travel time The object detection unit 130 may cause the object detection unit 130 to perform the object detection process when the trip management component determines that the AV is stopped for a predetermined period of time while it is in operation or in response to the determination that the transportation service is completed.

When the trip management component determines that the transportation service has been completed, for example, the object detection unit 130 may use the sensor data 131 (e.g., the second set of sensor data 131) received from the set of sensors Lt; RTI ID = 0.0 > a < / RTI > object detection process. If the object detection unit 130 detects that the object still exists, the object detection unit 130 may provide the system management unit 110 with information indicating that the object still exists in the vehicle. In response to this, the system management unit 110 can automatically perform the calibration action. On the other hand, if the object detection unit 130 detects that the object has been transferred, no calibration process or action is taken. In one variation, if the system 100 detects that an object has been transferred (e.g., in the case of a delivery transport service), the system 100 may determine that the delivery transport service is complete.

As described herein, a calibration action may be performed on behaviors (or sets of actions) performed by the system 100 to facilitate returning, regardless of whether the user has accidentally left an object, Can respond. In various implementations, the calibration action (s) may be performed by the system 100 and / or performed by the system 100 in conjunction with the service preparation system 190. In the latter example, the calibration action (s) may be performed as a result of data exchange between the system 100 and the service preparation system 190.

In one example, the system management unit 110 may receive information that an object is still in the vehicle and / or information about the object itself (e.g., based on recently determined information about the object, or in advance (From object information 133), and may generate an object message 117 based on this information. The object message 117 may include, for example, the time and / or location at which the transport service was completed, the identifier of the AV, the identifier of the user, the identifier of the transport service, and information about the object (e.g., The measurements associated with the object, the location at which the object is located). The system manager 110 may determine the communication identifier for the user (e.g., the user's telephone number or e-mail address or a communication identifier temporarily associated with the user's account) and send the object message 117 to the user's computing device have. This object message 117 may also cause the user's computing device to display content for informing the user that the object is left in the AV and / or display the service provisioning system (e.g., via a designated application operating on the user's computing device) Lt; RTI ID = 0.0 > 190 < / RTI > Additionally or alternatively, the object message 117 may be sent to the service provisioning system 190 and the service provisioning system 190 may send a message containing the notification information to the user's computing device. The notification information displayed to the user may provide information about the object and / or how the user can retrieve the object. Examples of different notification information are illustrated in Figures 4A-4D.

Additionally or alternatively, the system manager 110 may also perform one or more calibration actions (e. G., By allowing the AV to move to a particular location or to stop at a particular location) by placing the AV at a particular location have. For example, when the calibration action is performed, the system management unit 110 determines whether the object detection unit 130 determines that the object exists in the vehicle (i) at the destination position 194 or (ii) May access stored stored positioning commands that instruct the vehicle control component to control through the vehicle components that the vehicle is stationary. By allowing the AV to remain stationary at a particular location, the object message 117 indicates that the user's object is left in the AV and that the AV will stay in a particular location for a predetermined time duration (e.g., 2 minutes) . Alternatively, the stored location commands may instruct the vehicle control component to cause the AV to move to a particular location (e.g., a storage location where the lost object is stored) or an office building, It is possible to inform the user that the object can be retrieved at a specific location during the time period.

In some instances where the AV moves to a particular location, such as a storage location or a disposal location, the AV may use auto-control actuators to transfer the object from the AV for storage or disposal. For example, the AV itself may include one or more actuators coupled to one or more machine hands of the vehicle's interior or trunk, which can hold the object and relocate the object. In another example, the AV can be moved to a particular lane or place in the storage or disposal location to open the vehicle door and / or trunk, and the auto-controlled machine arm and hand or nib can transfer objects from the vehicle.

Still further, in another example, the system management unit 110 may provide information to the service preparation system 190 indicating that the object is left, and in response, the service preparation system 190 may determine that the AV is one And send commands to the system 100 to perform the above tasks. In one implementation, the service preparation system 190 may send vehicle commands to the system manager 110 that cause the vehicle control component to control the movement and operation of the AV. Vehicle commands may cause the AV to move to a specific location or to be stationary at a particular location.

In some cases, after dropping the user at destination location 194, the AV may be prepared / selected to provide the transport service to other requesting users. As a result, the system 100 can perform a corrective action, such as sending an object message 117, but can also cause the AV to move to a pick-up position specified by the requesting user. In some instances, the service provisioning system 190 (or system 100) may be configured to (i) command the disposition of an object, or (ii) provide service provisioning system 190 (Iii) request a transportation service to a specific location where the object is maintained, or (iv) query the user with a response input to request delivery of the object to the user's current location Message to the user ' s computing device. One or more of these options may be associated with additional costs for the user. The service provisioning system 190 may receive user input via an application on the user's computing device and may process the response accordingly. For example, if the user provides an input to receive the delivery of the object at the user's current location, the service preparation system 190 processes the response as a delivery transport service request, identifies the previous AV where the object is located, and For example, it may schedule the particular AV to move to the user's current location before completing the subsequent shipping service for the subsequently requesting user, or during or after completion. In this example, the user may be billed for a separate shipping service as a whole or for delivery as part of the user's previous shipping service (e.g., at an additional charge).

While system 100 of FIG. 1 is described as being implemented for computing and memory resources of an autonomous vehicle, in some instances system 100 is a mobile computing device (e.g., a driver device) operated by a driver of a vehicle, / RTI > For example, the system 100 may be implemented as an application or program (or as part of an application) that operates on an operator device or communicates data with the service provisioning system 190 via one or more networks. In these instances, the system 100 may include other components, such as user interface components, that create content (e.g., graphical user interfaces) and present it on the display of the driver device. In contrast to receiving service commands 191 when a driver is selected by the service preparation system 190 to provide transportation services to the requesting user, the system 100 provides transportation services through user interface components And the driver can accept or decline the invitation. The invitation may include user information 192, a pickup location 193, and / or a destination location 194.

Still further, in the examples in which the system 100 is implemented by an operator device, the object detection unit 130 may be a communication interface of a driver device, such as a short-range wireless communication transceiver (e.g., Bluetooth or Wi-Fi) Lt; RTI ID = 0.0 > 131 < / RTI > The driver can fix or attach one or more sensors, such as the trunk or back seat of the vehicle, that can communicate the sensor data 131 to the object detection unit 130 via the communication interface.

Alternatively, one or more components of the system 100 may be used as part of the service provisioning system 190 or as part of another remote system (e.g., remote from an autonomous vehicle or an operator device) in communication with the service provisioning system 190 Lt; / RTI > For example, the system 100 may send the sensor data 131 to the detection detector 130 implemented in the service preparation system 190, and the object detector 130 may then determine that the object is within the vehicle and / And detects when it has moved out. The service preparation system 190 may then perform calibration actions, such as sending notifications to the user based on detection.

Still further, in some instances, other sensors may be used by the system 100 to detect the presence of an object and / or the transfer of an object from a vehicle. For example, electromagnetic sensors can be used to detect electronics, such as cell phones, tablets, laptop computers, or other advanced sensors, such as chemical sensors or gas chromatography sensors, Food items, etc.) to detect objects. In one example, the use of electromagnetic sensors or antennas / wireless transceivers may help to detect small electronic devices that may be difficult to detect in a vehicle, such as falling between seats or under a seat or slipping away .

In a different application, the system 100 may also use sensor data to detect whether there has been damage to the interior of the vehicle as a result of the requesting user riding the vehicle or carrying the user's object. The system 100 may use sensors located in the interior of the vehicle to determine whether a response is received at a time before the transport service is initiated (e.g., at a predetermined distance or estimated time away from the user ' s pick- To detect the indoor state of the vehicle. At a time when the transportation service is being completed (e.g., when the vehicle is near the destination location or when it reaches the destination location) or at the time it is completed, the system 100 again detects the state of the interior of the vehicle at this time, This information can be compared with previous information on the indoor condition of the room. Using these comparisons, the system 100 may, after the transportation service, detect whether something has fallen into the seats or floor, whether the seats have been torn or scratched, and so on. If a corruption is detected, the system 100 may provide information indicative of a corruption to the service provisioning system 190 and / or the user device. The service preparation system 190 may charge the user for cleaning and / or repair, and / or may instruct the driver to move the vehicle to a specific location for cleaning and / or repairing, or may cause the vehicle to move to a particular location have.

Way

Figures 2 and 3 illustrate an exemplary method for detecting objects in a vehicle in connection with a transportation service. Methods such as those described by the examples of FIGS. 2 and 3 may be implemented, for example, using the components described in conjunction with the example of FIG. Accordingly, references to elements of FIG. 1 are intended to illustrate the elements or components suitable for carrying out the steps or sub-steps being described. For simplicity, the exemplary method of FIG. 2 is described as being performed by system 100 and / or service preparation system 190 operating on a driver device.

Referring to FIG. 2, the service preparation system 190 may determine 200 that an AV or vehicle driver has been selected to provide service to the requesting user. For example, the service preparation system 190 may perform a vehicle selection process for selecting a vehicle based on the location of the vehicle and the service location (s) specified by the requesting user. The service may correspond to an object from a pick-up location to a destination location or a goods delivery transport service or a transportation service of a person (e.g., a user) from a pick-up location to a destination location. The service preparation system 190 may transmit service commands corresponding to the service to be performed by the driver or the AV to the individual system 100 of the selected vehicle. In the example of AV, the system 100 may process service commands to control the AV to move from the current position to the pick-up position, while in the example of the human driver, It can provide route or navigation information that the driver can follow.

At a time after the system 100 determines that the driver or AV will provide the service, the system 100 may perform object detection operations using data from one or more sensors of the vehicle. In one example, the service provisioning system 190 can receive a first set of sensor information from the system 100 (e.g., at a first time), and based on the first set of information, (210). The system 100 may receive sensor information from a set of sensors of the vehicle. Alternatively, the system 100 may detect that an object is in the vehicle and provide information to the service preparation system 190 indicating such an object.

According to an implementation, one or more sensors or a combination of sensors may be used by system 100 and / or service preparation system 190 to detect that an object is in the vehicle. For example, the system 100 and / or the service preparation system 190 may receive image information from a set of cameras (e.g., radar, LIDARs, or ultrasonic sensors) Is arranged so that the individual lenses of the set of cameras are directed towards the interior of the vehicle (e.g., the rear seat, front seat, trunk or other compartment). In another example, the system 100 and / or the service preparation system 190 may receive weight information from a set of metering devices disposed in a compartment or interior of a vehicle, The individual measurement platforms of the set are arranged to be disposed on, below, or included in a floor or compartment of a vehicle (e. G., The platform may be configured to weigh objects placed in or in a seat or in a seat, Sitting on the back seat or on the front of the trunk to weigh the person sitting in the front seat). Still further, in another example, the system 100 and / or the service provisioning system 190 may be located near the vehicle's suspension system or may be located within the suspension system (e. G., A frame or rear and / And may receive compression information from a set of suspension sensors (located near the suspension system). Additionally, the sensor may also provide information regarding whether the compartment or trunk is open and / or closed, and this information may be used in conjunction with other sensor information.

The system 100 and / or the service preparation system 190 may use information from one or more of these different sources to perform object detection operations. In addition, based on the first set of received sensor inputs / information, the system 100 and / or the service provisioning system 190 may determine the fact that the object is in the vehicle, what the object is (or the type of object) The size or size or weight of the object, the compartment in which the object is located, and / or the location of the object. According to an example, the service provisioning system 190 may also associate object information with the trip entry of the transportation service (and / or with the user's profile or account).

In various examples, the system 100 may trigger sensors (e.g., to perform detection or measurement) to determine sensor data based on location information of the AV. For example, in order to conserve energy or power, the system 100 may determine when the AV reaches a service location or near a service location, for example when the AV reaches or within a predetermined distance or estimated travel time of the pick- , And / or trigger sensors to determine sensor data when the AV is stationary for a period of time while in or near the service location. Alternatively, in another example, the system 100 may trigger the sensors to determine the sensor data in response to the determination that the service has been started.

Referring again to FIG. 2, the service preparation system 190 may also determine 220 that the service has been started. According to an implementation, the service provisioning system 190 may be configured to (i) respond in response to detecting a user entering the vehicle (e.g., in response to detecting an object in the vehicle (step 210) (I. E., In response to a determination that the location data points of the device and the location data points of the vehicle are within a predetermined distance from each other), (ii) a predetermined time duration (E. G., 10 seconds) has elapsed, and (iii) in response to detecting that the service has been started as a result of an input provided to a computing device associated with the user, (E.g., the user selects a feature corresponding to starting the trip), (iv) the driver or In response to receiving information from the user device (or the on-board computing system of the AV) indicating that the service can be started as a result of an input provided to the driver device or the on-board computing system from the user, and / (v) determine that the service has been started in response to a determination that the vehicle is located within a determined distance of the starting position for a predetermined time duration based on the vehicle's location information. In other instances, the system 100 detects that services have been started based on any of the examples (i) through (v).

The service preparation system 190 also monitors the service (e.g., 230) when, for example, the driver or the AV performs the service. As described herein, monitoring a service includes determining the location (and corresponding timestamp) and / or state of the vehicle when the vehicle moves from the pick-up location to the destination location, and determining such information in the trip entry associated with the service Storage periodically. The system 100 may provide this information to the service provisioning system 190. In some instances, the service provisioning system 190 may also provide information about the progress of the service to the user's computing device (or other device). For example, the user may be picked up at the user's location and requested a delivery service for the package to be dropped to the friend's destination location. An application on the user's device may receive (periodically) information about the progress of the service, such as the location and movement of the AV on the map, from the service provisioning system 190. Additionally or alternatively, the user may have provided the service provisioning system 190 with the contact information (e.g., telephone number or e-mail address) of the friend to receive the package through the application. The service provisioning system 190 may monitor the service and provide updated information about the service to a friend so that the friend can track the package as the AV moves to the destination location.

The service provisioning system 190 may determine 240 that the service has been completed, e.g., as a result of monitoring the service. According to an implementation, the service provisioning system 190 may be configured to (i) determine in response to detection that the user has left the vehicle (e.g., based on the location data points of the user's computing device and the location data points of the vehicle, (I. E., 20 seconds) has elapsed since detecting that the user left the vehicle, in response to the determination that the user has left the vehicle, (iii) in response to receiving information from the user's computing device indicating that the service is terminated as a result of input provided to the computing device associated with the user (e.g., the user selects a feature corresponding to terminating the trip ), (Iv) from the driver or user to the driver device or to the on-board computing system In response to receiving information from the driver device (or the on-board computing system of the AV) indicating that the service has been terminated as a result of the input being provided, and / or (v) It may determine that the service has been terminated in response to the determination that the vehicle is located within the determined distance of the destination location during the time duration.

After determining that the service has been completed, the service preparation system 190 may determine that the object (previously detected in the vehicle) based on the second set of sensor information from one or more sensors of the vehicle (e.g., at a second time) (250) if it has been transferred from the vehicle. For example, the service preparation system 190 may receive a second set of sensor information from the system 100 and may include sensor information (e. G., Image information) to determine whether there are differences, such as differences in images, differences in weight, Lt; / RTI > can be compared. Alternatively, the system 100 may perform a comparison operation to provide an indication to the service preparation system 190 as to whether or not the object has been transferred. The system 100 may be configured to determine (e.g., detect) a sensor date at a particular time instance, such as in response to detecting that the service has been completed, or after a predetermined time duration since detecting that the service has been completed Or to perform measurements).

Alternatively, the service provisioning system 190 may determine whether the object has been transferred from the vehicle before it is determined that the service is to be completed. For example, the service provisioning system 190 (or system 100) may be configured such that the AV has arrived at the destination location, or is near the destination location and / or is in the destination location or near the destination location, The object detection process can be performed in response to the determination. In another example, the service provisioning system 190 may determine that the service has been completed (e.g., after performing step 250) in response to detecting that the object has been transferred from the vehicle.

If the service preparation system 190 determines that the object has been transferred from the vehicle, the service preparation system 190 may determine that the user is in possession of the object, or if another user has moved the object (e.g., to the delivery destination location) , It can be determined 270 that the calibration action need not be performed. On the other hand, if the service preparation system 190 determines that the object has not been transferred, the service preparation system 190 may automatically perform one or more calibration actions (260). For example, the service provisioning system 190 may be able to mess up the operator and / or the user via individual applications on the driver device and / or the user device to notify the driver and user of the faulty object. The message may include commands (e.g., including navigation commands) that allow the user and the user to interact with the location, e.g., the destination location, so that the user can retrieve the object.

In another example, the service provisioning system 190 may send a message or notification to the user device to enable the user to request delivery of the object to a location specified by the user. Still further, in one example, if the vehicle is an AV, the service preparation system 190 may send vehicle control commands to the AV to cause the AV to move to a specific location. This particular location could correspond to a new contact location for a user specified in a user's subsequent forwarded shipping service request (delivery dropout location) or a designated storage or warehouse location that stores the wrong objects. The service provisioning system 190 may also be capable of messaging to the user device in order to provide such location information in which the user can retrieve the object.

Figure 3 illustrates another exemplary method for detecting objects in a vehicle in connection with a transportation service. In one example, one or more steps of the method described in FIG. 3 may be performed concurrently with one or more steps of the method described in FIG. For simplicity, the exemplary method of FIG. 3 is described as being performed by the system 100 operating on an on-board computing system of autonomous vehicles.

In FIG. 3, the system 100 of (AV) may receive 300 service instructions from the service preparation system 190 for AV to provide services to the requesting user. As described herein, a service may have a start location and a destination location, and may correspond to the delivery of an object or a person (e.g., a requesting user). The system 100 determines which service to provide, where to move (e.g., from the current location to the start / pickup location, to the destination location), and how to move to that location Lt; RTI ID = 0.0 > service commands. ≪ / RTI >

Based on service instructions, map data and / or sensor data (e.g., data from sensors mounted on the body of the AV) used to operate the AV, the system 100 may be configured to move the AV And at the same time monitor the movement of the AV (310). The system 100 may be configured to transmit control signals to the vehicle control modules of various sub-systems in response to receiving sensor data from the sensors and feedback signals from various vehicle components, And may correspond to or include or communicate with the control system. The system 100 may also monitor the current location, e.g., by determining the location of the AV from the GPS data and comparing it to the route and / or the starting location. Using the periodically determined current position of the AV, the system 100 can determine when the AV reaches or is within a predetermined distance of the starting position (e.g., when the AV approaches the starting position). In one example, when the system 100 determines that the AV is close to the starting location, the system 100 may change the state of the AV (from the on-route state to the just arrived state or the arrived complete state).

At a time after determining that the AV is close to the starting position, the system 100 may perform an object detection process 320 to detect that the object is in the AV. According to an implementation, the system 100 may be configured to perform an object detection process, such as described in Figures 1 and 2, until the time after the AV begins to move to the destination location from the time after determining that the AV is close to the start location Can be performed periodically. In one example, system 100 may perform an object detection process in response to determining that a service has been started, or may perform an object detection process at the same time as or prior to determining that a service has been started. In the example of Figure 3, the system 100 may detect that an object (e.g., a box) is placed in the back seat of the AV from the sensor information received from the AV, as described in Figures 1 and 2.

Once the service is started, the system 100 may control the AV to move from the destination location, and may also monitor the progress of the service or the vehicle (330). Similarly, as described above, the system 100 can determine the location of the AV from the GPS data and monitor the current location by comparing it with the route and / or destination location. The system 100 may determine 340 when the AV reaches or is within a predetermined distance of the destination location (e.g., when the AV approaches the destination location). At a time after determining that the AV is close to the destination location, the system 100 may perform an object detection process 350 to detect that the object is in the AV. According to an implementation, step 350 may be performed before or after the determination that the service is complete, or to be performed during its determination.

If the system 100 determines that an object has been transferred from the AV, the system 100 may determine 370 that a calibration action need not be performed. On the other hand, if the system 100 determines that an object is still present in the vehicle, the system 100 may perform one or more calibration actions and / ) May perform one or more calibration actions (360).

Figures 4A-4D illustrate examples of user interfaces displayed on a mobile computing device in various use case examples. For each of the examples illustrated in Figures 4A and 4D, an individual user interface may be displayed on the user's device of a user who has left an object in the vehicle with respect to the user's transportation service. According to an implementation, the user interface may be displayed as part of a designed application associated with the service provisioning system 190 or as part of another application (e.g., an email application, a text messaging application, a web browser application, etc.) . In one example, a notification may be displayed on the user device, and the selection of the feature or notification may cause the user interface to be displayed on the user device.

4A, in one example, the user device can receive notifications or messages from the service preparation system 190 or system 100 after the service is terminated (or, in some cases, just before the service is terminated) have. An application running on the user device may display the user interface 400 and the user interface 400 may be used to remind the user that the image content 405 of the object and the object are left in the vehicle and / Text content 410. Image content 405 may be generated from image information from one or more sensors (e.g., cameras) of the vehicle. Alternatively, if the camera is not used in a vehicle, other content representing the object may be displayed in place of the image content 405, such as additional textual content representing the object, or other graphics or placeholder images. In this example, the textual content 410 also provides contact information for the user to messaging to determine what to do to retrieve the object.

In the example of FIG. 4B, the user interface 420 may be displayed on the user device. The user interface 420 also includes image content 425, such as illustrated in FIG. 4A, and text content 430, which notifies the user that the object remains in the vehicle and informs the user of the specific location to pick up the object . For example, the service preparation system 190 may instruct the AV or driver to move to a particular location (or to stay at the destination location or the location where the user came down). The service provisioning system 190 may generate the textual content 430 containing the corresponding content for the location. In FIG. 4B, the text content 430 informs the user that the vehicle with the object at the location where the user has left will wait two more minutes (or any other predetermined time duration).

According to another example, the user interface 440 of Figure 4C may be displayed on the user device as an interactive notification prompt. This notification prompt may be an in-app (e.g., may be displayed as part of an application user interface) or may be an out-of-app (e.g., And may be displayed using the native user interface notification features of the mobile device's operating system when it is not open. In this example, the user interface 440 includes textual content 445 that contains specific locations where the user's objects will be archived or retained. The textual content 445 may include location information (e.g., address or street intersection), a time or duration that the user can pick up an object (e.g., "normal business hours" or "8am to 6pm" A selectable feature 450 may be selected by the user to request the transportation service of the user. If the user selects a selectable feature ("request" or "request trip") 450, the designated service application may be opened or launched on the user device, From a specific pickup location) to a specific location. Alternatively, if the user selects the selectable feature 450, the designated service application can automatically generate a request for the transportation service and send it to the service preparation system 190. [ The request may include a type of service (e.g., user transport), a pickup location (e.g., the current location of the user device determined via the GPS receiver of the user device), and a destination location (e.g., 123 Main Street).

In another example, such as illustrated in Figure 4D, the user interface 460 may include textual content 465 that informs the user that the user may request delivery of the user object. The user interface 460 may enable the user to select a user-selectable feature 470 to request delivery of the user object to the user's current location. Similar to the example of FIG. 4C, if the user selects a selectable feature 470, the designated service application may be opened or launched on the user device so that the user can select the feature (e.g., Location) of the user object to a particular location, or the designated service application can automatically generate a request for a delivery transport service and send it to the service provisioning system 190. [ In the latter example, the request includes a type of service (e.g., delivery of an object to a particular vehicle with a user object) and a getting-off location or destination location (e.g., the current location of the user device determined via the GPS receiver of the user device) can do. 1, for example, the service provisioning system 190 may identify a vehicle associated with a trip entry having a particular vehicle or identified object that the user has just used, You can choose that vehicle.

In some instances, the user interface described in any of Figs. 4A-4D may also include detailed information about the object, such as what the object is (including color or shape), the object type, the dimensions and / The location where the object was placed, and so on. While the examples of FIGS. 4A-4D have been described with respect to a user device, in some variations such user interfaces may also be displayed on the driver device to notify the driver of the object left in the driver's vehicle. This user interface can inform the driver of where to go, where to wait (and / or where to wait for some time), and / or who should be contacted so that the user can retrieve the object.

Hardware diagrams

5 is a block diagram illustrating a computing device in which the embodiments described herein may be implemented. In one example, the computing device 500 may correspond to a mobile computing device, such as a cellular device, which may enable telephony, messaging, and data services. The computing device 500 may correspond to a user device or a driver device. Examples of such devices include smart phones, handsets or tablet devices for cellular carriers. The computing device 500 may include a processor 510, memory resources 520, a display device 530 (e.g., a touch-sensitive display device), one or more communication sub-systems 540 One or more sensors (e. G., A GPS receiver), input mechanisms 550 (e.g., the input mechanism may include or be part of a touch-sensitive display device), and a position detection mechanism (560). In one example, at least one of the communication sub-systems 540 transmits and receives cellular data over a data channel and voice channels. The communication sub-systems 540 may include a cellular transceiver and one or more short-range wireless transceivers. The processing resources 510 may receive communications from the service preparation system, for example, via the communication sub-systems 540.

In the example of FIG. 5, the computing device 500 may correspond to a driver device operated by a driver of the vehicle. Processor 510 may provide various content to display 530 by executing instructions stored in memory resources 520. [ The memory resources 520 may store, for example, databases corresponding to the driver application 525 and other data, such as one or more trip entries 527, one or more parameters, or location data points. For example, the processor 510 may include software and / or other software to perform one or more processes, steps, and other functions described in the embodiments described with reference to FIGS. 1 through 4D and described elsewhere herein Logic. In particular, the processor 510 may execute commands and data stored in the memory resources 520 for notification of a transportation service to be provided by the driver, 565) can be used. Sensors 560 represented herein may be used to sense sensors included in a computing device 500 (e.g., a GPS receiver) as well as sensor interfaces 530 that communicate with one or more sensors (e.g., ultrasonic sensors, metering devices, Lt; / RTI > Processor 510 may also be configured to display various user interfaces 515, such as invitations to provide transportation services, navigation information, or notifications to the user that an object remains in the vehicle ' s vehicle, And execute commands corresponding to the application 525. [ In addition, in some instances, the user interfaces 515 may be used to interact with the driver application 525, such as to indicate when the transportation service has been started or completed (e.g., with the touch-sensitive display 530) ) Input mechanisms 550 to provide user input 555, which may include selectable features that the operator can select.

While FIG. 5 is illustrated for a mobile computing device, one or more examples may be implemented on other types of devices, including full-functional computers, such as laptops and desktops (e.g., a PC) have. In another example, the variation of FIG. 5 may illustrate an on-board computing system (at least a portion of the system) of AV, such as that described in FIG.

6 is a block diagram illustrating a computer system in which the examples described herein may be implemented. For example, in the context of FIG. 1, service provisioning system 190 may be implemented using a computer system such as that illustrated by FIG. The service provisioning system 190 may also be implemented using a combination of a plurality of computer systems as described by FIG.

In one implementation, computer system 600 includes processing resources such as one or more processors 610, main memory 620, read-only memory (ROM) 630, storage device 640, and communication interface 650 ). The computer system 600 includes at least one processor 610 for processing information and other information for storing information and instructions to be executed by the main memory 620, such as random access memory (RAM) Dynamic storage device. Main memory 620 may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by processor 610. [ The computer system 600 may also include ROM 630, or other static storage device for storing static information and instructions for the processor 610. A storage device 640, such as a magnetic disk or optical disk, is provided for storing information and instructions. For example, the storage device 640 may correspond to a computer-readable medium storing location determination instructions 642 and object detection instructions 644 for performing the operations discussed with respect to Figures 1-5 . In these instances, the computer system 600 may determine the location of vehicles, determine service locations of transportation services, and perform object detection processes based on sensor data received from vehicle systems or driver devices . In addition, the storage device 640 may be configured to receive requests for transportation services from the requesting users, such as commands for implementing a dispatch sub-system to select vehicles to provide these transportation services Other instructions may be included.

Communications interface 650 may enable computer system 600 to communicate with one or more networks 680 (e.g., a cellular network) through the use of a network link (wirelessly or using a wireline). Using the network link, computer system 600 may communicate with a plurality of devices, such as mobile computing devices of passengers and drivers. According to some examples, using the communication interface 650, as described by some examples of FIGS. 1-5, the computer system 600 may receive location information from the driver devices, AVs and / (652), and may send notification messages (654) to the user device to inform users that their possessions remain in the vehicle.

Computer system 600 may also include a display device 660, such as a cathode ray tube (CRT), LCD monitor, or television set, for example, to display graphics and information to a user. An input mechanism 670, such as a keyboard including alphanumeric keys and other keys, may be coupled to the computer system 600 for communicating information and command selections to the processor 610. [ Other non-limiting illustrative examples of input mechanisms 670 include directional information and command selections to a processor 610 to communicate with a mouse, trackball, touch-sensitive screen, or cover orientation to control cursor movement on the display 660 Keys.

The examples described herein relate to the use of computer system 600 to implement the techniques described herein. According to one example, these techniques are performed by the computer system 600 in response to the processor 610 executing one or more sequences of one or more instructions contained in the main memory 620. These instructions may be read from another machine-readable medium, such as from the storage device 640, into the main memory 620. [ Execution of sequences of instructions contained in main memory 620 causes processor 610 to perform the process steps described herein. In alternative implementations, hard wired circuitry may be used in place of or in combination with software instructions to implement the examples described herein. Thus, the described examples are not limited to any specific combination of hardware circuitry and software.

It is contemplated that the examples described herein may be extended to include other elements, concepts, or systems independent of other concepts, ideas, or systems, as well as combinations of the elements recited in any position of the present application do. Although the examples are described in detail herein with reference to the accompanying drawings, it should be understood that the concepts are not limited to these examples. Accordingly, it is intended that the scope of the concepts be defined by the following claims and their equivalents. Moreover, although other features and examples may make reference to particular features, they may be combined, as part of the examples, or with the parts of other features or other examples, . Thus, even if the combinations are not described, the rights to such combinations should not be excluded.

Claims (20)

A method for monitoring a vehicle used when providing a transportation service, the method being implemented by one or more processors,
The method comprises:
Detecting when the transportation service is completed;
Determining from the one or more sensors of the vehicle that an object that should have been transferred from the vehicle after completion of the transportation service remains in the vehicle when it is detected that the transportation service has been completed; And
And automatically initiating performance of a remedial action in response to a determination that the object remains in the vehicle. ≪ Desc / Clms Page number 20 > The method according to claim 1,
The one or more sensors of the vehicle include (i) a set of cameras, (ii) a set of metering devices, (iii) a set of suspension sensors, (iv) a light detection and ranging (LIDAR) A set of sensors, (v) a set of radio frequency transceivers, or (iv) a set of ultrasound sensors. The method according to claim 1,
Wherein the step of determining that the object remains in the vehicle comprises the step of detecting the position of an object inside the vehicle based on image information from a set of cameras disposed in the vehicle. Lt; / RTI > The method according to claim 1,
Wherein the step of determining that the object remains in the vehicle comprises the steps of determining whether the object remains in the vehicle from a set of metering devices disposed in the compartment or interior of the vehicle for sensing the weight of the compartment of the vehicle, And detecting that the object is located in a compartment or an interior of the vehicle based on weight information of the vehicle. The method according to claim 1,
Wherein the step of determining that the object remains in the vehicle is based on compression information from a set of sensors disposed to detect compression of the vehicle's suspension system such that the object lies in a compartment or interior of the vehicle The method comprising the steps of: detecting a presence of a vehicle; 3. The method of claim 2,
Determining that the object remains in the vehicle comprises measuring radio frequency signals emitted by the object and determining from the measured radio frequency signals that the object is in a compartment or interior of the vehicle Wherein the method further comprises the steps of: The method according to claim 1,
Wherein the step of determining that the object remains in the vehicle comprises determining a first set of information from the first set of sensors of the vehicle when the transportation service is being provided, Comprising: determining a second set of information from a first set of sensors; and comparing the first set of sensor information with a second set of the information. Way. The method according to claim 1,
Wherein automatically initiating the performance of the calibration action comprises communicating, via one or more networks, to a computing device associated with the user, a message informing the user requesting the service that the item remains in the vehicle. A method for monitoring a vehicle used when providing a service. 9. The method of claim 8,
Wherein automatically initiating the performance of the calibration action comprises causing the vehicle to remain stationary at a location of the vehicle when the object is detected to still be present in the vehicle A method for monitoring a used vehicle. 9. The method of claim 8,
Wherein automatically starting the performance of the calibration action comprises causing the vehicle to move to a predetermined position, wherein the predetermined position corresponds to a position at which the object can be transferred from the vehicle, A method for monitoring a vehicle used when providing. The method of claim 3,
Wherein the step of determining that the object remains in the vehicle comprises capturing one or more images of a portion of the vehicle using one or more cameras of the vehicle and performing image analysis to detect the object from the one or more images The method comprising the steps of: providing a transportation service; The method according to claim 1,
Further comprising determining from the one or more sensors of the vehicle when the object is initially placed in the vehicle by a passenger entering the vehicle at a pick-up position. Way. 13. The method of claim 12,
Wherein detecting the first placement of the object in the vehicle comprises the steps of: (i) in response to detecting that the passenger has entered the vehicle, (ii) in response to detecting that the passenger has entered the vehicle, (Iii) a response to receiving from the computing device associated with the passenger a message indicating that the service can be started as a result of an input provided to the computing device associated with the passenger In response to receiving from the computing device associated with the vehicle a message indicating that the service can be started as a result of input provided to the computing device associated with the vehicle from a passenger in the vehicle, (v) based on the location information of the vehicle, Wherein the method is performed in response to a determination that the vehicle is located within a predetermined distance of the pick-up position for a determined time duration. CLAIMS 1. A method for monitoring a vehicle when providing a service,
Receiving instructions from a remote system for a vehicle to provide a service having a start location and a destination location corresponding to transporting one or more objects and individuals from one location to another location;
Controlling the vehicle to move to the start position by the computing system that controls the components of the vehicle, corresponding to an on-board computing system of the vehicle;
Based on a first set of information from the one or more sensors of the vehicle at a first time after the vehicle has reached or is within a first predetermined distance of the starting position, Detecting by the computing device that it lies within the vehicle;
Subsequently controlling the vehicle by the computing system to move to the destination location;
Determining whether the object is in a second predetermined time interval based on a second set of information from the one or more sensors at a second time after the vehicle has reached or is within a second predetermined distance of the destination location; Detecting whether or not the vehicle has been transferred from the vehicle; And
And automatically initiating the performance of the calibration action in response to detecting that the object is still present in the vehicle. 15. The method of claim 14,
The one or more sensors of the vehicle may be (i) a set of cameras, (ii) a set of metering devices, (iii) a set of suspension sensors, (iv) a set of light detection and ranging (LIDAR) (v) a set of radio frequency transceivers, or (iv) a set of ultrasonic sensors. 16. The method of claim 15,
The step of detecting that the object is in the vehicle comprises detecting the position of the object inside the vehicle based on image information from the set of cameras disposed in the vehicle, Wherein individual lenses of the set of cameras are oriented towards the interior of the vehicle and the step of automatically initiating the performance of the calibration action comprises: (i) using one or more cameras in the set of cameras, (Ii) generating a message comprising the one or more images, and (iii) sending a message to the remote system to notify the remote system that the item is left in the vehicle, A step of communicating to the system, And communicating a message to the computing device associated with the user via the one or more networks to notify a user requesting the service. 16. The method of claim 15,
Wherein the step of detecting that the object is in the vehicle comprises detecting a weight from the set of metering devices disposed in the compartment or the interior of the vehicle to sense the weight of the objects in the compartment or the interior of the vehicle, And detecting that the object is located in a compartment or an interior of the vehicle based on the information. 16. The method of claim 15,
Wherein the step of detecting that the object is in the vehicle further comprises the step of determining that the object is located in a compartment or an interior of the vehicle based on compression information from the set of suspension sensors disposed to detect compression of the suspension system of the vehicle The method comprising the steps of: detecting that the vehicle is in a vehicle; 15. The method of claim 14,
Wherein automatically initiating the performance of the calibration action comprises communicating a message to the remote system via one or more networks informing the remote system that an item has remained in the vehicle, And communicating a message to the computing device associated with the user via the one or more networks to notify a user requesting the service. 17. A non-transitory computer-readable medium having stored thereon instructions,
Wherein the instructions, when executed by one or more processors of a computing system, cause the computing system to:
Receive commands from a remote system for a vehicle to provide a service having a start location and a destination location corresponding to transporting one or more objects or individuals from one location to another location;
Control the vehicle to move to the start position;
Based on a first set of information from the one or more sensors of the vehicle at a first time after the vehicle has reached or is within a first predetermined distance of the starting position, Detect that the vehicle is in the vehicle;
Subsequently controlling the vehicle to move to the destination location;
Determining whether the object is in a second predetermined time interval based on a second set of information from the one or more sensors at a second time after the vehicle has reached or is within a second predetermined distance of the destination location; Detect whether the vehicle has been transferred from the vehicle; And
Causing the computer to perform a calibration action automatically in response to detecting that the object is still present in the vehicle.

Images