WO2021029822A1

WO2021029822A1 - System and method for dispatching vehicles technical field

Info

Publication number: WO2021029822A1
Application number: PCT/SG2019/050398
Authority: WO
Inventors: Su Hnin Wut YI; Wujuan Lin
Original assignee: Hitachi, Ltd.
Priority date: 2019-08-13
Filing date: 2019-08-13
Publication date: 2021-02-18
Also published as: WO2021029822A8

Abstract

According to various embodiments, there is provided a method for dispatching vehicles including: identifying current attributes of a user based on real-time data; detecting user behavioral concepts based on the identified current attributes; generating a semantic trajectory of the user based on the detected user behavioral concepts; predicting time and location of a future request for a vehicle based on the generated semantic trajectory; and dispatching the vehicle based on the predicted time and location.

Description

SYSTEM AND METHOD FOR DISPATCHING VEHICLES

TECHNICAL FIELD

[0001] Various embodiments relate to methods for dispatching vehicles and methods for dispatching vehicles, including being applied to Autonomous Mobility on Demand (AMOD) services along with self-driving cars.

BACKGROUND

[0002] With rapid urbanization, the car population in the urban areas is expected to surge as the human population density increases. Consequently, road congestion, lack of parking spaces and air pollution may follow, severely reducing the quality of life in the urban areas. As such, there may be a need for a shared commuting service that may provide commuters a viable alternative to car ownership while reducing the overall quantity of vehicles on the road. One potential solution may be an on-demand mobility service, where vehicles are dispatched to fetch commuters. However, the on-demand mobility service may reduce road congestion only if the vehicles are efficiently utilized, i.e. minimal travelling without passengers. If a large quantity of the vehicles travel across the city without any passengers, i.e. make empty trips, the waiting time for commuters may be long and the costs for the service operator of the on-demand vehicle service may escalate. Moreover, the road congestion may worsen. As such, there is a need to accurately predict the user demand and dispatch the vehicles to meet the user demand accordingly.

SUMMARY

[0003] According to various embodiments, a method for dispatching vehicles may be provided. The method may include: identifying current attributes of a user based on real-time data; detecting user behavioral concepts based on the identified current attributes; generating a semantic trajectory of the user based on the detected user behavioral concepts; predicting time and location of a future request for a vehicle based on the generated semantic trajectory; and dispatching the vehicle based on the predicted time and location. [0004] According to various embodiments, a system for dispatching vehicles may be provided. The system may include: an attribute detector configured to identify current attributes of a user based on real-time data; a behavioral model processor configured to detect user behavioral concepts based on the identified current attributes; a semantic trajectory generator configured to generate a semantic trajectory of the user based on the detected user behavioral concepts; a future request predictor configured to predict time and location of a future request for a vehicle based on the generated semantic trajectory; and a dispatch processor configured to dispatch vehicles based on the predicted time and location.

[0005] According to various embodiments, a non-transitory computer readable medium may be provided. The non-transitory computer readable medium may include instructions, which when executed by a computer, causes the computer to perform a method for dispatching vehicles. The method may include: identifying current attributes of a user based on real-time data; detecting user behavioral concepts based on the identified current attributes; generating a semantic trajectory of the user based on the detected user behavioral concepts; predicting time and location of a future request for a vehicle based on the generated semantic trajectory; and dispatching the vehicle based on the predicted time and location.

BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In the drawings, like reference characters generally refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments are described with reference to the following drawings, in which:

[0007] FIG. 1 shows a conceptual diagram of an on-demand mobility service system according to various embodiments.

[0008] FIG. 2 shows a flow diagram of a method of predicting future requests for autonomous vehicles according to various embodiments.

[0009] FIG. 3 shows a scenario diagram illustrating an example of the method of predicting future requests for autonomous vehicles according to various embodiments.

[0010] FIG. 4 shows a system overview of a vehicle dispatch apparatus according to various embodiments. [0011] FIG. 5A illustrates a flow diagram for an ontology management process according to various embodiments.

[0012] FIG. 5B illustrates an example of an ontology tree diagram according to various embodiments.

[0013] FIG. 6 illustrates a data flow diagram of a trajectory generation process according to various embodiments.

[0014] FIG. 7 illustrates a flow diagram that shows an example of the knowledge extraction process according to various embodiments.

[0015] FIG. 8 illustrates a flow diagram that shows an example of the semantic recognition process according to various embodiments.

[0016] FIGS. 9A-D show examples of data structures used by the real-time input data and the historical data repository according to various embodiments.

[0017] FIG. 9E shows an example of a data structure generated by the semantic trajectory generator 460 according to various embodiments.

[0018] FIG. 10 shows a data flow diagram for an exemplary process of the future requests prediction server 480 according to various embodiments.

[0019] FIG. 11 shows a data flow diagram for an exemplary process of central management system of autonomous vehicles which will dynamically coordinate the vehicles’ fleet according to various embodiments.

[0020] FIG. 12 shows a flow diagram of a method for dispatching vehicles according to various embodiments.

[0021] FIG. 13 shows a conceptual diagram of a system for dispatching vehicles according to various embodiments.

DESCRIPTION

[0022] Embodiments described below in context of the systems are analogously valid for the respective methods, and vice versa. Furthermore, it will be understood that the embodiments described below may be combined, for example, a part of one embodiment may be combined with a part of another embodiment.

[0023] It will be understood that any property described herein for a specific system may also hold for any system described herein. It will be understood that any property described herein for a specific method may also hold for any method described herein. Furthermore, it will be understood that for any system or method described herein, not necessarily all the components or steps described must be enclosed in the system or method, but only some (but not all) components or steps may be enclosed.

[0024] In this context, the system for dispatching vehicles as described in this description may include a memory which is for example used in the processing carried out in the system. A memory used in the embodiments may be a volatile memory, for example a DRAM (Dynamic Random Access Memory) or a non-volatile memory, for example a PROM (Programmable Read Only Memory), an EPROM (Erasable PROM), EEPROM (Electrically Erasable PROM), or a flash memory, e.g., a floating gate memory, a charge trapping memory, an MRAM (Magnetoresistive Random Access Memory) or a PCRAM (Phase Change Random Access Memory).

[0025] The term “coupled” (or “connected”) herein may be understood as electrically coupled or as mechanically coupled, for example attached or fixed, or just in contact without any fixation, and it will be understood that both direct coupling or indirect coupling (in other words: coupling without direct contact) may be provided.

[0026] In order that the invention may be readily understood and put into practical effect, various embodiments will now be described by way of examples and not limitations, and with reference to the figures.

[0027] In the context of various embodiments, a “trajectory” may include a series of locations denoted by spatiotemporal points.

[0028] In the context of various embodiments, “semantic” may refer to meaningful property/characteristic that is used to forming concepts.

[0029] In the context of various embodiments, a “semantic trajectory” may include a series of locations denoted by semantic annotation which may indicate activities/decision/behavior being carried out in the trajectory. In other words, a semantic trajectory may include a trajectory and information on the activities, decisions or behavior of the user associated with various spatiotemporal points in the trajectory. Every location in the trajectory may have a latent semantic meaning with respect to the user.

[0030] In the context of various embodiments, the phrase “semantic trajectory pattern” may refer to a trajectory pattern which indicates recurring or similar activities in various trajectories. [0031] In the context of various embodiments, “ontology” may refer to specification of keywords for conceptualization and their relationships. Ontology information may define semantic interpretation with respect to domain knowledge.

[0032] According to various embodiments, a method for dispatching vehicles may include determining the sematic content and spatial-temporal data of real-time input data as well as other external data sources to reliably predict “when” and “where” a user may request a vehicle with the highest probability. The method may include predicting the users’ demand for vehicles based on the users’ behaviors, i.e. semantic activity of the users. By doing so, the prediction may be accurate even when there are random fluctuations in the user demand. Consequently, vehicles may be guided to arrive at the predicted location and time periods before actual requests are being initiated and thus it may reduce the consumers’ waiting times for vehicles. Existing methods of predicting user demand are generally based on statistical demand and therefore may result in inaccuracies during random fluctuations.

[0033] According to various embodiments, the vehicles may be autonomous vehicles which may provide personal urban mobility service.

[0034] According to various embodiments, the method for dispatching vehicles may be applied to Autonomous Mobility on Demand (AMOD) services, such that the method may include an AMOD service. Existing AMOD services may dispatch the vehicles according to real-time demand or predicted demand that is determined based on historical statistical data. However, these existing AMOD services may distribute their vehicles inefficiently when the demand deviates from historical trends. Unlike the existing AMOD services, the method of dispatching vehicles according to various embodiments may anticipate future requests for autonomous vehicles to a high level of accuracy by considering the intent and motivation of user activities.

[0035] The method may offer a solution to the traffic woes of crowded cities and may help to curb the car population in the cities. The AMOD service may be able to substitute private car ownership or one-way car sharing with autonomous vehicles for first-and-last-mile connectivity. The method may improve the efficiency of the post-service routing model of AMOD service, for example, by guiding the vehicles’ routes after the passengers have alighted.

[0036] According to various embodiments, the method for dispatching vehicles may include collecting and analysing real-time input data and external resources. The method may further include generating semantic trajectory of daily activities based on the analysis and further based on ontology-based domain knowledge. The method may further include predicting “when” and “where” a user requests a vehicle using semantic trajectory analysis which indicates varying personal mobility demand pattern in real-time. In other words, the method may include building user behavioral models, and predicting user trajectories based on understanding the users’ behavioral models.

[0037] According to various embodiments, the user trajectories predicted by the method may be used to distribute vehicles, as well as for planning of support infrastructure such as charging stations, petrol kiosks, and parking lots. The predicted user trajectories may also be used to recommend activities to users, based on the user behavioral models.

[0038] FIG. 1 shows a conceptual diagram 100 of an on-demand mobility service system according to various embodiments. The on-demand mobility service system may manage ontology-based domain knowledge. The on-demand mobility service system may generate trajectory patterns of commuters (also referred herein as users) based on semantic analysis and the ontology-based domain knowledge, and may predict “when” (time) and “where” (location) a commuter may request for an autonomous vehicle. The on-demand mobility service system may dispatch a vehicle in advance, to the predicted location at the predicted time, so that the commuter’s waiting time may be minimal. The on-demand mobility service system may include external data 110, real-time input data 120, a future requests’ prediction server 130 and a central management system 160. The external resources 110 may include at least one of existing ontologies for travel demands, domain ontologies for private transportation and metadata standard for conceptualization of user behaviors. Existing ontologies for travel demands may include specification of keywords that are specific to the domain of transportation, in other words transportation concepts. Domain ontologies for private transportation may include specific key words for private or personal car usage and demand. The domain ontologies for private transportation may be a subset of the existing ontologies for travel demands that is specific to personalized travel demands. Metadata standard for conceptualization of user behaviors may include supporting information for forming conceptualization. The metadata for conceptualization of user behaviors may describe terminology classification and/or data descriptions, to communicate via compatible knowledge models (ontologies) to support conceptualization. The real-time input data 120 may include at least one of user data, vehicle sensor data and city-wide wireless data. The user data may be data provided directly by the user, for example, through user input into a mobile application. The user data may also include data provided by the user indirectly, for example, through data that the user entered into his/her devices, for example third party mobile applications like maps, calendars etc. The vehicle sensor data may include images, audio, or videos captured by a cameras mounted on the vehicles. The sensor may be mounted on an external surface of the vehicle, or may be mounted inside the vehicle. The city-wide wireless data may be data collected by WiFi access points, when user devices connect to the access points. The on-demand mobility service system may also include a wide area network 150 that communicatively couples the external resources 110, the real-time input data 120, the prediction server 130 (also referred herein as future requests prediction server), and the central management system 160. Alternatively, the prediction server 130 and the central management system 160 may reside on a common computer hardware and may communicate via a data bus. The central management system 160 may control and monitor a fleet of autonomous vehicles 180. The prediction server 130 may include a semantic trajectory analytics processor (STAP) 140. The prediction server 130 may provide information about “when” and “where” a user requests an autonomous vehicle using the STAP 140. The STAP 140 may receive the external data 110 and the real-time input data 120 through the WAN 150, and may perform a semantics analysis on the external data 110 and the real-time input data 120. The central management system 160 may obtain the prediction results from the prediction server 130. The central management system 160 may include a vehicle dispatching system 170. The vehicle dispatching system 170 may determine “which” vehicles 180 to dispatch to the predicted location and timing 190.

[0039] FIG. 2 shows a flow diagram of a method of predicting future requests for autonomous vehicles 200 according to various embodiments. The method may be performed by the prediction server 130. Step 210 may include receiving real-time demand requests in the prediction server 130. The real-time demand requests may be part of the real-time input data 120. Step 210 may also include identifying the current user attributes, also referred herein simply as “current attributes”, from different observation data such as wireless sensor data, images recorded by cameras mounted on vehicles and data collected by mobile applications. The current user attributes may include information such as identities of the user’s travel companions, the user’s current location, the user’s biodata etc. The current user attributes may include keywords to detect ontology-based domain knowledge. The observation data may be part of the external data 110. Step 220 may include detecting concepts by ontology mapping. Step 220 may include searching in an ontology domain database, various keywords to detect predefined concepts. In other words, Step 220 may include conceptualization of the user’ s current demand request which may include the pick up location and the drop-off location. The semantic trajectory analytics processor 140 may detect keywords and may refer the keywords to existing ontology domain knowledge, in each of the demand requests (for example, the locations), the user attributes and any other relevant input data. For example, the semantic trajectory analytics processor 140 may detect keywords such as “kids’ menu”, “ice-cream”, “kinder” in the received data, for example in the names of shops that are the pickup or drop-off point. These keywords may correspond to concepts such as “children” or “family” in the ontology domain knowledge. Step 230 may include extracting correlated semantic knowledge. In Step 230, the semantic trajectory analytics processor 140 may correlate the concepts identified in Step 220 with the ontology-based domain knowledge. The semantic trajectory analytics processor 140 may infer semantic knowledge beyond what is indicated in the data, by correlating the detected concepts with the ontology-based domain knowledge extracted from past and current information. For example, the user is going to the restaurant which is family friendly restaurant nearby. The user may have been in the retail outlet for two hours. Therefore, the semantic trajectory analytics processor 140 may extract semantic features beyond the visited places and may indicate the activities being carried out in these places. Step 240 may include generating the semantic trajectory for the user which may include, for example, “what” activity(s) the user is partaking at the visited places, “whom” are visiting along with the user and “where” are related places for the activity(s) which user may be partaking. Generating the semantic trajectory may include associating activities, user decisions or behavior to locations in the trajectory. The trajectory analytics processor 140 may consequently determine the behaviors of the users. For example, Steps 210 to 230 may establish that the user may be going from a conference centre to a restaurant. The trajectory analytics processor 140 may infer that the user may be attending a seminar and may be heading to the restaurant with colleagues for a working dinner, possible through identifying keywords in the received data for example, in the locations, in the users’ calendar mobile application, and the colleagues’ mobile application data. Step 250 may include similarity classification and spatial-temporal-semantic clustering, to predict “when” and “where” the user may request an autonomous vehicle. For example, the trajectory analytics processor 140 may determine that the user’s trajectory may be similar to other businessman attending the same seminar, or may determine that other users frequently travel from the same conference centre to a nearby street that has many popular upscale restaurants. The trajectory analytics processor 140 may predict that the user and each of his companions may require a vehicle at the nearby street to travel home after dinner.

[0040] FIG. 3 shows a scenario diagram 300 illustrating an example of the method of predicting future requests for autonomous vehicles 200 according to various embodiments. In 310, the prediction server 130 may receive input data on historical trips’ request which may include the pickup and drop-off locations, as well as receiving user attributes such as the user’s travel companions, the user’s age, the user’s gender, the user’s priority etc. The input data may include at least one of the external data 110 and the real-time input data 120. The prediction server 130 may identify current attributes of the user based on the real-time input data 120. In 320, the prediction server 130 may receive ontology data, i.e. data from the ontology domain knowledge database. In 330, the prediction server 130 may generate a historical trajectory of the user. For example, the prediction server 130 may determine that the user has travelled from home to a retail outlet. The prediction server 130 may also determine that the user has brought along his family. The prediction server 130 may determine from the user’s commuting history, that the user may go to the coffee shop and then to a gym after going to the retail outlet; or may go to a restaurant and then go to a recreational park after going to the retail outlet. The prediction server 130 may determine, based on machine learning the user’ s behavior, that when the user brings along his family, he is more likely to go to the restaurant and the recreation park. The prediction server 130 may determine that the user may require transport at the restaurant around 8PM in the evening and may require transport at the recreational park at around 9PM in the evening. In 340, the prediction server 130 may inform the central management system 160 to dispatch a vehicle to the restaurant at 8PM and to dispatch a vehicle to the recreational park at 9PM. As shown in this example, a user’s commuting path may differ, depending on the user’s motivation or intent. In this example, the user may head to the recreational park instead of the gym, as he may be spending time with his family. The prediction server 130 may predict the user’s future commuting trajectory based on the user’s behavioral model.

[0041] FIG. 4 shows a system overview of a vehicle dispatch apparatus 400 according to various embodiments. The apparatus 400 may include, among other components, computing server 430, central management system 440, ontology management system 450, semantic trajectory generator 460, future requests’ prediction server 480 and data repository 470 for storing input data, temporary results and output data. These components may communicate over a network 490. The network 490 may be a fixed wireless network, local area wireless network (LAN), a wireless wide area network (WAN), a virtual private network (VPN) or any other suitable network system. The network 490 may include, or may be part of, the network 150. The network 490 may include equipment for receiving and transmitting data and signals. Input data to the apparatus 400 may include external resources 410 and real-time input data 420. External resources 410 may be used to determine semantic meaning of the trajectory which may be provided in real-time input data 420. The external resources 410 may include existing ontologies 411, new domain ontologies 412 and metadata standards 413. The external resources 410 may also include historical travel patterns of the user.

These data may include keywords for conceptualization of semantic features. The external resources 410 may provide the external data 110. Real-time input data 420 may include user data 421, vehicle sensor data 422 and wireless data 423. The user data 421 may include current travel demand requests, user id, device id etc. The vehicle sensor data 422 may be obtained from cameras mounted on the vehicles. The wireless data 423 may be obtained from access points across the city. The real-time input data 420 may include, or may be part of, the real-time input data 120. The one or more computing servers 430 may include one or more processors 431, system memory 432, and a communication interface 433 that couples the processors 431 and the system memory 432. The system memory 432 may store intermediate results and temporary results of the computing server 430. The computing server 430 may be a point of interaction between several systems such as the central management system 440, the ontology management system 450, the semantic trajectory generator 460, the future requests’ prediction server 480 and the data repository 470. The ontology management system 450 may retrieve data from the external resources 410 and may generate ontology- based domain knowledge which may be utilized by the semantic trajectory generator 460. The ontology management system 450 may include an ontology learning processor 451 and a database that stores ontology-based domain knowledge 452. The ontology learning processor 451 may include a machine learning algorithm that is trained on the ontology-based domain knowledge 452. The semantic trajectory generator 460 may identify current attributes of the user based on real-time input data. The ontology management system 450 may build a behavioral model of the user based on the historical travel patterns of the user. The ontology management system 450 may also map the identified current attributes to predefined ontology information to detect user behavioral concepts. The semantic trajectory generator 460 may perform knowledge extraction 461 and semantic recognition 462 in order to produce semantic trajectories 463 based on the real-time input data 420 and data from the data repository 470. The semantic trajectory generator 460 may include, or may be part of, the semantic trajectory analytics processor 140. The data repository 470 may store historical data including user profiles 471, historical travel demands 472 and past weather or traffic events 473. For example, the user profiles 471 may include data regarding a user’s personal preferences including, but not limited to, the user’s age range, gender, priority, mobility needs as well as travel companions. The historical travel demands 472 may include data regarding prior requests for rides such as “who” made the request, “how many” passengers on the ride, “when” the requests were made, “where” from which the requests were made, “origin” location for the requests and “destination” for the requests and other such data. The pass weather or traffic events 473 may include factors that affect travel demand over a duration of time such as traffic conditions, weather condi tions(rain/snow/sunny), events (concert, national event, public events) and other such data. The future requests prediction server 480 may classify and cluster spatial temporal semantic data to discover semantic trajectory patterns. The future requests prediction server 480 may perform the complex computation of daily activity and travel pattern of one or more individuals’ semantic behaviors. Then, the future requests prediction server 480 may provide information on “when” and “where” a user may require the vehicles, to the central management system 440. The future requests prediction server 480 may include, or may be part of, the prediction server 130. The future requests prediction server 480 may generate a predicted trajectory based on the correlation of the current user attributes and the behavioral model. The central management system 440 may then determine “which” vehicles to dispatch to the predicted location and timing. The central management system 440 may include, or may be part of, the central management system 160. The central management system 440 may dispatch vehicles based on the trajectory generated by the future requests prediction server 480.

[0042] FIG. 5A illustrates a flow diagram 500 for an ontology management process according to various embodiments. In the ontology management process, the ontology management system 450 may utilize external resources 410 in order to build up ontology- based domain knowledge. The ontology-based domain knowledge may include semantic meaning extracted from real-time input data 420. The external resources 410 may include existing ontologies 411, domain ontologies 412 and met data standard 413. The process of learning ontology may include initialization of ontology 510 and keyword determination 520 using ontology learning algorithms 530. Initialization of ontology 510 may be performed using prior knowledge used in the construction of a first version of the ontology. Keyword determination 520 may include detecting new keywords in the input data, for example data form the external resources 410. Keyword determination 520 may further include updating ontological knowledge such as concepts or relations through ontology learning algorithms 530. The ontology learning algorithms 530 may form an ontology-based domain knowledge 452 using machine learning techniques based on knowledge acquisition combined with domain knowledge rules. The ontology management system 450 may generate output data 540. The output data may include an ontology model. The ontology model may include ontology concepts, relations as well as domain knowledge rules, for example, “WHEN event IF condition THEN activity”. The output data may be added to the ontology-based domain knowledge 452.

[0043] FIG. 5B illustrates an example of an ontology tree diagram 550 according to various embodiments. The ontology tree diagram 550 may represent the ontology-based domain knowledge 452. The ontology-based domain knowledge 452 may include concepts 552. In the ontology-based domain knowledge 452, a plurality of concepts 552 may be linked or associated. For example, a person may be associated with various behaviors and/or identities. For example, a place may be associated with a plurality of roads and regions. The instances 554 may be events that occur, for example, events that may be retrieved from the data repository. These instances 554 may be mapped to the relevant concepts 552.

FIG. 6 illustrates a data flow diagram 600 of a trajectory generation process according to various embodiments. The semantic trajectory generator 460 may receive real-time input data 420 and ontology-based domain knowledge 452, as well as information from the data repository 470. The semantic trajectory generator 460 may perform knowledge extraction 461 from the real-time input data 420. The process of knowledge extraction 461 maybe formed using feature engineering process, which may be performed using machine learning. The feature engineering process may include filtering information received from the real-time input data 420 and the data repository 470, to extract spatial, temporal or semantic features that may be strongly correlated to the output data 670. The process of knowledge extraction 461 may be automated and may self-leam over time to improve its extraction of relevant features. The process of knowledge extraction 461 may include keyword detection 610, ontology mapping 620 and spatial-temporal-semantic feature extraction 630. The process of semantic recognition 462 may utilize the output of the spatial-temporal-semantic feature extraction and the ontology-based domain knowledge. The keyword detection 610 may capture various parameters, such as, “who” made the request, “who” are travel companions, “how many” passengers on the ride, “when/where” the requests were made, “origin/destination” location for the requests and related traffic data/weather data/event data. The process of ontology mapping 620 may instantiate semantic meaning from real-time input data 420 by referring to the ontology-based domain knowledge 452. The process of ontology mapping 620 may loop through parameters that were captured in the keyword detection 610. Ontology mapping 620 may transform the incoming parameters into target ontology entities and may relate entities semantically at a conceptual level. Semantic content may be created as a result of the ontology mapping 620. The spatial-temporal-semantic feature extraction 630 may link the semantic content with spatial-temporal features. The semantic trajectory generator 460 may also perform semantic recognition 462 using intermediate results of the spatial-temporal-semantic feature extraction 630. The intermediate results may include semantic content coupled with spatial-temporal features which may be further processed by the semantic recognition process 462. The semantic recognition process 462 may include a semantic knowledge construction process 640 in which spatial-temporal-semantic features may be transformed into semantic knowledge. The semantic recognition process 462 may include a reasoning process 650. The reasoning process 650 may be carried out by a reasoning engine that may include artificial intelligence algorithms. The artificial intelligence algorithms may apply logical rules to semantic knowledge and may generate the semantic trajectory of the user. The semantic trajectory may include information “which” activity(s) the user is participating at the visited places, “whom” are visiting along with the user and “where” is related places for the activity(s) which user may be participating in. Therefore, it may reveal the semantic behaviors of the users. For example, the semantic behavior may reveal the user’s intent, for example, the user may be out for leisure purpose and bringing along the family.

[0044] FIG. 7 illustrates a flow diagram 700 that shows an example of the knowledge extraction process 461 according to various embodiments. Step 702 may include capturing input parameters to determine at least one of user attributes, spatial-temporal data, visualized image data, and related locality boundaries. The input parameters may be captured from at least one of user data, vehicle sensor data, and city-wide wireless data. Step 704 may include detecting keywords. The keywords may include at least one of the identity of the user, i.e. “who” made the request; the identities of travel companions of the user, i.e. “who” are travel companions; the quantity of passengers, i.e. the sum of one user and his/her travel companions, i.e. “how many” passengers on the ride; time and venue of the requests, i.e. “when/where” the requests were made; and the pickup and drop-off locations, i.e. “origin/destination” locations for the requests. Step 706 may include discovering corresponding ontologies by looping through all the keywords. Alternatively, Step 706 may loop through only a subset of the keywords. Step 708 may include instantiating corresponded ontologies based on ontology-based domain knowledge. Step 710 may include determining whether the mapping of the keywords to concepts has been completed. If the mapping has been completed, the knowledge extraction process 461 may proceed to Step 712. If the mapping has not been completed, the knowledge extraction process 461 may repeat Steps 706 and 708. Step 712 may include retrieving recent trip data from the data repository 470 to add additional information obtained from Steps 707 and 708. The additional information may include, for example, user profile, historical travel demands of the user, related traffic data, weather data and event data. Step 714 may include generating syntactic tags for instantiated ontologies to extract spatial-temporal features. The spatial-temporal features may include, for example, locality boundaries and temporal dependencies. Step 716 may include generating syntactic tags for instantiated ontologies to extract semantic contents. The semantic contents may include, for example, the user’s intent, behaviors and lifestyles. Step 718 may include performing canonical correlation analysis to explore the relationships between spatial- temporal-semantic features.

[0045] FIG. 8 illustrates a flow diagram 800 that shows an example of the semantic recognition process 462 according to various embodiments. Step 802 may include capturing spatial-temporal-semantic features along with interdependence relationships. Step 804 may include identifying correlated spatial-temporal-semantic features based on the frequency of occurrence of the individual events or instances. Step 806 may include discovering the semantic knowledge between mobility patterns based on aggregation of a class of users which yields similar semantic content. Step 808 may include applying logical rules to extract individual mobility patterns along with semantic domain knowledge, using the reasoning engine 650. Step 810 may include generating semantic trajectories. The semantic trajectories may reveal the behaviors of the user, for example, whether the user is out on a leisure activity, whether the user is out on a family trip, or whether the user is out for business.

[0046] FIGS. 9A-D show examples of data structures used by real-time input data 420 and historical data repository 470 according to various embodiments. It should be understood that the data structures for real-time input data 420 and historical data repository 470 are not limited to including the fields of the examples shown. [0047] FIG. 9A shows an example of a data structure 900A for storing user profiles. The data structure 900A may include a user ID column 902, a user name column 904, an age range column 906, a gender column 908, and an occupation column 910. Each row in the data structure 900A may store information of a respective user. The user ID column 902 may store a unique code for representing the user. The user name column 904 may store the name of the user. The age range column 906 may store the age, or an age range that the user falls into. The gender column 908 may store information on the user’ s gender. The occupation column 910 may store information on the user’s occupation.

[0048] FIG. 9B shows an example of a data structure 900B for storing information on a journey. The data structure 900B may include a user ID column 902, an origin column 914, a destination column 916, a travel companion column 918 and a time stamp column 920. The origin column 914 may store information on the pickup point, or the start point of the user’s journey. The destination column 916 may store information on the drop-off point, or the end point of the user’s journey. Each of the origin column 914 and the destination column 916 may store geographical locations, for example in GPS coordinates, latitude and longitude. The travel companion column 918 may indicate the relationship between the user’s travel companions and the user, for example, whether they are family members, or colleagues, or friends. Alternatively, or additionally, the travel companion column 918 may indicate the identities (for example represented by user IDs) of the travel companions. The time stamp column 920 may store a time that the journey started, or the time that the user made a request for vehicle to make the journey.

[0049] FIG. 9C shows a data structure 900C for storing information on events. The data structure 900C includes an event type column 922, a data column 924, and a time stamp column 926. The event type column 922 may indicate a category of the event, for example, whether it is a weather event, or a human event. The data column 924 may store a description of the event, for example, the location of the event and what the event was. The time stamp column 926 may store the time that the event took place.

[0050] FIG. 9D shows a user input data table 930, a vehicle sensor data table 940 and a city wide wireless data table 950. The user input data table 930 may include a user ID column 902, a “from” column 934, a “to” column 936, and a time stamp column 938. The “from” column 934 may store a pickup location, whereas the “to” column 936 may store a drop-off location. The time stamp column 938 may store information on the time that the user made the input. The vehicle sensor data table 940 may include a vehicle ID column 942, an image data column 944, and a time stamp column 948. The vehicle ID column 942 may store a unique identifier for the vehicle. The image data column 944 may store an image, or a link to the image. The image may be captured by a sensor that is mounted on the vehicle identified in the vehicle ID column 942. The time stamp column 948 may store information on the time that the sensor captured the image. The city-wide wireless data table 950 may include an access point ID column 952, a location column 954, and a connected device column 958. The access point ID column 952 may store a unique identifier for each wireless connection (for example: WiFi) access point. The location column 954 may store the geographical location of the access point, for example, as GPS coordinates. The connected device column 958 may indicate the user device that connected t to the access point indicated in the access point ID column 952. The user device may be a mobile phone, or a laptop, or any other user devices that are capable of connecting to a wireless connection.

[0051] FIG. 9E shows an example of a data structure 900E generated by the semantic trajectory generator 460 according to various embodiments. It should be understood that the data generated by the semantic trajectory generator 460 is not limited to the fields of the example shown. The data structure 900E may include a user ID column 902, an intent column 964, a trajectory column 966, an activity column 968 and a mobility pattern column 970. The intent column 964 may store a description of the user’s intent for a journey, for example, whether the user is spending leisure time with his family, whether the user is out because it is a sunny day, or because it is the weekend and the user has a regular routine for weekends. The trajectory column 966 may store information on the user’s past trajectory, i.e. where the user has been to, on the same day. In other words, the trajectory column 966 may indicate information on where and when the user’s visits for the day. For example, the trajectory column 966 may store information on the geographical locations that the user has been to, or the name or description of the places that he visited, as well as the time and duration of the visits. The activity column 968 may store a description of the activities that took place during the visits, for example the user may have visited a restaurant and the activity was to have a meal, for example the user may have visited a departmental store and the activity was shopping. The mobility pattern column 970 may store information on a pattern of travel that the semantic trajectory generator 460 has detected. For example, the semantic trajectory generator 460 may detect that the user goes from home, to retail areas, then to a restaurant. [0052] FIG. 10 shows a data flow diagram 1000 for an exemplary process of the future requests prediction server 480 according to various embodiments. The future request prediction server 480 may receive historical semantic trajectories 463 and current semantic trajectories 680 of a plurality of different users. The spatial-temporal-semantic clustering 481 may include a semantic trajectory similarity clustering 710 which may discover patterns in the historical semantic trajectories 463. The semantic trajectory similarity clustering process 710 may identify potential related patterns from the historical data. After that, the future request prediction server 480 may perform the process of similarity classification 720 and spatial-temporal-semantic clustering 481 in parallel. Semantic trajectory classification 720 may include supervised machine learning process by assigning the trajectory to one of several pre-defined semantic knowledge categories based upon the parameters. Spatial-temporal- semantic clustering 481 may perform unsupervised machine learning process that groups the trajectories based upon spatial similarity, temporal similarity as well as semantic similarity. Both classification and clustering process may reveal patterns, commonalities and thus may derive relations between spatial-temporal-semantic features of the trajectories. The real-time prediction engine 482 may perform a prediction process based on patterns, commonalities and relations between semantic trajectories which are generated by classification and clustering process. The prediction engine 482 may combine similarity classification and spatial-temporal-semantic clustering in order to determine “when” and “where” a user may need an autonomous vehicle. The prediction engine 482 may generate output data 730 that may include the location and time of future requests.

FIG. 11 shows a data flow diagram 1100 for an exemplary process of central management system of autonomous vehicles which will dynamically coordinate the vehicles’ fleet according to various embodiments. The central management system 440 may include a dynamic fleet management unit 441. The dynamic fleet management unit 441 may include a fleet tracker 810, a future demand request tracker 820 and a task allocator 830. The fleet tracker 810 may keep track of each vehicle, including its location, and mode. The vehicle mode may refer to whether the vehicle is in transit to a pickup point, or ferrying commuters in a journey, or is in a post-service routing state. The future demand request tracker 820 may request anticipating future requests of vehicles at any regular time interval or at demands which may be specified by external parameters.

The task allocator 830 may assign available vehicles to meet anticipated demand based on vehicles distribution optimization algorithms. The vehicle distribution optimization algorithms may find best-matching of the spatial-temporal distance between the vehicles and future demand requests. The task allocator 830 may generate an output data 840 that indicates the matching between a plurality of vehicles to respective future demand requests. The vehicle dispatching system 442 may dispatch the matched vehicles to pick-up locations according to their respective matched future demand requests based on the output data 840. The vehicles may be autonomous vehicles, and the method may be applied to supplying on- demand autonomous vehicle to commuters.

[0053] FIG. 12 shows a flow diagram 1200 of a method for dispatching vehicles according to various embodiments. Element 1202 may include identifying current attributes of a user based on real-time data. Element 1204 may include detecting user behavioral concepts based on the identified current attributes. Element 1206 may include generating a semantic trajectory of the user based on the detected user behavioral concepts. Element 1208 may include predicting time and location of a future request for a vehicle based on the generated semantic trajectory. Element 1210 may include dispatching the vehicle based on the predicted time and location.

[0054] In other words, the method for dispatching vehicles may include the methods described with respect to FIGS. 2 and 3. The method may include methods of predicting the user’s demand for vehicles. Element 1202 may include, or may be part of Step 210. Element 1204 may include, or may be part of, Step 220. Element 1204 may include mapping the identified current attributes to predefined ontology information to detect spatial-temporal- semantic features. Element 1206 may include, or may be part of Step 240. Generating the semantic trajectory of the user may include correlating the detected user behavioral concepts with ontology-based domain knowledge, like in Step 230. Element 1208 may include, or may be part of, Step 250. Element 1208 may include predicting the time and location of the future request further based on historical semantic trajectories of the user.

[0055] The method may further include building a semantic behavioral model of the user based on historical semantic trajectories of the user. The method may further include correlating the identified current attributes to the behavioral model based on the detected user behavioral concepts, which may include Step 230.The method may further include generating syntactic tags for the identified current attributes, and detecting the user behavioral concepts at least partially based on the syntactic tags. The method may also include predicting the time and location of the future request further based on information on at least one of weather, traffic and events schedule. [0056] According to various embodiments, the real-time data used for identifying current attributes of the user may include at least one of user data received from a mobile phone of the user, sensor data received from the vehicles, and wide area network data received from internet access points. The real-time data may include the real-time input data 120. The current attributes may include at least one of identity of the user, number of companions traveling with the user, identity of the companions, location of the user, pick up point and drop off point.

[0057] FIG. 13 shows a conceptual diagram of a system for dispatching vehicles 1300 according to various embodiments. The system for dispatching vehicles 1300 may include an attribute detector 1302, a behavioral model processor 1304, a semantic trajectory generator 1306, a future request predictor 1308, and a dispatch processor 1310. The attribute detector 1302 may be configured to identify current attributes of a user based on real-time data. The behavioral model processor 1304 may be configured to detect user behavioral concepts based on the identified current attributes. The semantic trajectory generator 1306 may be configured to generate a semantic trajectory of the user based on the detected user behavioral concepts. The future request predictor 1308 may be configured to predict time and location of a future request for a vehicle based on the generated semantic trajectory. The dispatch processor 1310 may be configured to dispatch vehicles based on the predicted time and location. The system for dispatching vehicles 1300 may further include a syntax generator 1312. The syntax generator may be configured to generate syntactic tags for the identified current attributes. The behavioral model processor 1304 may be configured to detect the user behavioral concepts at least partially based on the syntactic tags. The attribute detector 1302, the behavioral model processor 1304, the semantic trajectory generator 1306, the future request predictor 1308, the dispatch processor 1310 and the syntax generator 1312 may be coupled, like indicated by lines 1320, for example electrically coupled, for example using a line or a cable, and/ or mechanically coupled, and/or communicatively coupled.

[0058] In other words, the system for dispatching vehicles 1300 may include, or may be part of, the on-demand mobility service system shown in FIG. 1. More specifically, the system for dispatching vehicles 1300 may include the prediction server 130 and the central management system 160. The attribute detector 1302, the behavioral model processor 1304, the semantic trajectory generator 1306, and the future request predictor 1308 may be part of the prediction server 130. The dispatch processor 1310 may include, or may be part of the central management system 160 or the central management system 440. The attribute detector 1302 may include, or may be part of, the semantic trajectory generator 460. The behavioral model processor 1304 and the semantic trajectory generator 1306 may include, or may be part of, the ontology management system 450 or the semantic trajectory generator 460. The behavioral model processor 1304 may include a machine learning algorithm. The future request predictor 1308 may include, or may be part of, the future requests prediction server 480. The future request predictor 1308 may take into consideration, at least one of weather, traffic and events schedule, in predicting the time and location of the future request. The syntax generator 1312 may be configured to generate syntactic tags for the identified current attributes, and the behavioral model processor 1304 may be configured to detect the user behavioral concepts at least partially based on the syntactic tags.

[0059] According to various embodiments, a non-transitory computer readable medium may be provided. The non-transitory computer readable medium may include instructions, which when executed by a computer, causes the computer to perform a method for dispatching vehicles as described with respect to FIG. 12.

[0060] While embodiments of the invention have been particularly shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The scope of the invention is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced. It will be appreciated that common numerals, used in the relevant drawings, refer to components that serve a similar or the same purpose.

[0061] It will be appreciated to a person skilled in the art that the terminology used herein is for the purpose of describing various embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

[0062] It is understood that the specific order or hierarchy of blocks in the processes / flowcharts disclosed is an illustration of exemplary approaches. Based upon design preferences, it is understood that the specific order or hierarchy of blocks in the processes / flowcharts may be rearranged. Further, some blocks may be combined or omitted. The accompanying method claims present elements of the various blocks in a sample order, and are not meant to be limited to the specific order or hierarchy presented.

[0063] The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects. Unless specifically stated otherwise, the term “some” refers to one or more. Combinations such as “at least one of A, B, or C,” “one or more of A, B, or C,” “at least one of A, B, and C,” “one or more of A, B, and C,” and “A, B, C, or any combination thereof’ include any combination of A, B, and/or C, and may include multiples of A, multiples of B, or multiples of C. Specifically, combinations such as “at least one of A, B, or C,” “one or more of A, B, or C,” “at least one of A, B, and C,” “one or more of A, B, and C,” and “A, B, C, or any combination thereof’ may be A only, B only, C only, A and B, A and C, B and C, or A and B and C, where any such combinations may contain one or more member or members of A, B, or C. All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims. The words “module,” “mechanism,” “element,” “device,” and the like may not be a substitute for the word “means.” As such, no claim element is to be construed as a means plus function unless the element is expressly recited using the phrase “means for.”

Claims

1. A method of dispatching vehicles, the method comprising: identifying current attributes of a user based on real-time data; detecting user behavioral concepts based on the identified current attributes; generating a semantic trajectory of the user based on the detected user behavioral concepts; predicting time and location of a future request for a vehicle based on the generated semantic trajectory; and dispatching the vehicle based on the predicted time and location.

2. The method of claim 1, wherein detecting the user behavioral concepts comprises mapping the identified current attributes to predefined ontology information to detect spatial- temporal- semantic features.

3. The method of any one of claims 1 to 2, wherein predicting the time and the location of the future request is further based on historical semantic trajectories of the user.

4. The method of any one of claims 1 to 3, further comprising: building a semantic behavioral model of the user based on historical semantic trajectories of the user.

5. The method of any one of claims 1 to 4, wherein generating the semantic trajectory of the user comprises correlating the detected user behavioral concepts with ontology-based domain knowledge.

6. The method of any one of claims 1 to 5, wherein the real-time data comprises at least one of user data received from a mobile phone of the user, sensor data received from the vehicles, and wide area network data received from internet access points.

7. The method of any one of claims 1 to 6, wherein the current attributes comprises at least one of identity of the user, number of companions traveling with the user, identity of the companions, location of the user, pick up point and drop off point.

8. The method of any one of claims 1 to 7, further comprising: generating syntactic tags for the identified current attributes; detecting the user behavioral concepts at least partially based on the syntactic tags.

9. The method of any one of claims 1 to 8, further comprising: predicting the time and location of the future request further based on information on at least one of weather, traffic and events schedule.

10. The method of any one of claims 1 to 9, wherein the vehicles are autonomous vehicles.

11. A system for dispatching vehicles, the system comprising: an attribute detector configured to identify current attributes of a user based on real time data; a behavioral model processor configured to detect user behavioral concepts based on the identified current attributes; a semantic trajectory generator configured to generate a semantic trajectory of the user based on the detected user behavioral concepts; a future request predictor configured to predict time and location of a future request for a vehicle based on the generated semantic trajectory; and a dispatch processor configured to dispatch vehicles based on the predicted time and location.

12. The system of claim 11, wherein the behavioral model processor is configured to detect the user behavioral concepts by mapping the identified current attributes to predefined ontology information to detect spatial-temporal-semantic features.

13. The system of any one of claims 11 to 12, wherein the future request predictor is configured to predict the time and the location of the future request further based on historical semantic trajectories of the user.

14. The system of any one of claims 11 to 13, wherein the behavioral model processor is further configured to build a semantic behavioral model of the user based on historical semantic trajectories of the user.

15. The system of any one of claims 11 to 14, wherein the real-time data comprises at least one of user data received from a mobile phone of the user, sensor data received from the vehicles, and wide area network data received from internet access points.

16. The system of any one of claims 11 to 15, wherein the current attributes comprises at least one of identity of the user, number of companions traveling with the user, identity of the companions, location of the user, pick up point and drop off point.

17. The system of any one of claims 11 to 16, further comprising: a syntax generator configured to generate syntactic tags for the identified current attributes; wherein the behavioral model processor is configured to detect the user behavioral concepts at least partially based on the syntactic tags.

18. The system of any one of claims 11 to 17, wherein the future request predictor is configured to predict the time and location of the future request further based on information on at least one of weather, traffic and events schedule.

19. The system of any one of claims 11 to 18, wherein the behavioral model processor comprises a machine learning algorithm.

20. A non-transitory computer readable medium comprising instructions, which when executed by a computer, causes the computer to perform a method of dispatching vehicles, the method comprising: identifying current attributes of a user based on real-time data; detecting user behavioral concepts based on the identified current attributes; generating a semantic trajectory of the user based on the detected user behavioral concepts; predicting time and location of a future request for a vehicle based on the generated semantic trajectory; and dispatching the vehicle based on the predicted time and location.