AU2021105935A4 - System for determining physiological condition of driver in autonomous driving and alarming the driver using machine learning model - Google Patents

Abstract

System for determining physiological condition of driver in autonomous driving and alarming the driver using machine learning model The present invention relates to the field of determining the physiological condition of driver in autonomous driving and alerting the same. With the increasing use of sensor-based technology and machine learning models, there is a need of more reliable and efficient system with faster response time for determining physiological condition of driver and alert/assist the driver before mishappening. The present invention mainly solves the problem in prior art. The system comprises a central server equipped with machine learning model; a GPS system and various sensors installed in the interior of the vehicle as well as sensors available in mobile device and smart wearable of the driver connected to vehicle; capturing, using the sensors installed in the interior of the vehicle, available in mobile device and smart wearable of the driver, the physiological parameters related to the driver; transmitting, by the communication network, captured physiological parameters to the central server continuously; storing the transmitted physiological parameters in the database at the central server; analyzing at central server, by machine learning model, the captured physiological parameters as and when available; determining the event data based on the location of the vehicle according to route of autonomous driving; comparing, by the machine learning model, the event data with the analyzed physiological state of the driver; assist the driver in autonomous driving in simultaneous to alarm the driver in audio/video on the display of the driver accordingly. In this way, physiological condition of the driver can be determined correctly in time and assist/alert the driver before any mishappening. The proposed system provides faster response time and more efficient process using machine learning models that can be used in current system of autonomous vehicle driving. 1 GPS sensors (206) (205) Databases (202) Communication Server based on network (201) deep laning Vehicle (204) Figure 2 Block Diagram of system for determining physiological condition of driver and alarming the same 2

Description

GPS sensors (206) (205) Databases (202)

Communication

Server based on network (201)

deep laning

Vehicle (204)

Figure 2 Block Diagram of system for determining physiological condition of driver and alarming the same

System for determining physiological condition of driver in autonomous driving and alarming the driver using machine learning model

FIELD OF INVENTION

[0001] The present invention relates to the field of autonomous driving of the vehicle. The field of the invention is to provide a method for determining physiological condition of the driver which is to be used in vehicle for autonomous driving.

[0002] More particularly, this present invention relates to the field of determining physiological condition of the driver in autonomous driving and alarming the driver using machine learning models.

BACKGROUND & PRIOR ART

[0003] The subject matter discussed in the background section should not be assumed to be prior art merely as a result of its mention in the background section. Similarly, a problem mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the background section merely represents different approaches, which in-and-of-themselves may also be inventions.

[0004] In today's world, with the advancement in technology, world is moving toward automation. The automation in the field of vehicle and driving is one of the emerging fields with the advent of the technology. Autonomous driving normally refers to self-driving vehicles or transport systems that move without the intervention of a human driver. The level of autonomous vehicle driving may vary from no automation to full automation. The world is moving from manual vehicle driving to autonomous vehicle driving. With the advancement in technology, the level of autonomous vehicle driving is also increasing. The society of autonomous engineers defines 6 levels of driving automation ranging from 0 to 5. Level 0 indicates no driving automation, the which are totally manually controlled. Level 1 indicates the driver assistance, the lowest level of automation in which only single automated system for driver assistance is provided for driver such as cruise control etc. In this way, moving forward to level 5 indicates the full automation in which no driver assistance is required to drive the vehicle. Autonomous vehicles use various technical components to make them automatic. These technical components are various types of advanced sensors, smart control systems, intelligent actuators and various types of machine learning models.

[0005] Further, autonomous vehicle driving is a kind of technology that assist the driver in driving the vehicle according to the condition of the road/way/path. The condition of path on a particular location is determined and vehicle assist the driver or performs automation for optimal driving of the vehicle. This autonomous driving identifies the land topology and captures the path condition and previous act of the vehicle on the same road and according to the condition assist the drive while driving. Further, this autonomous vehicle driving also need to monitor the physiological condition or activity of the driver. The vehicles equipped with autonomous vehicle driving can be able to sense the physiological condition of the driver for safe and comfortable driving. There are various kinds of sensors which are incorporated in the vehicle that can monitor the physiological parameters of the driver and easily able to identity the physiological condition of the driver. The machine learning models are one of the most advanced and current technology which is used for automation. There are various kinds of machine learning models used for providing automation or artificial intelligence which are broadly divided in two types namely supervised learning and unsupervised learning. One kind of machine learning model which is used for automation is deep learning models. Deep learning model is one of the machine learning methodologies used for providing artificial intelligence. Deep learning models are widely used in providing improved performance and high level of features abstraction in comparison to traditional models. Deep learning models are of various types as convolutional Neural Networks (CNNs), Long Short-Term Memory Networks (LSTMs), Recurrent Neural Networks (RNNs), generative adversarial networks, self-organizing maps and many more. The adjective deep in the deep learning models refers to the use of multiple layers in the network. Deep learning is a variation in which unbounded number of layers with bounded size is used for providing faster response, improved performance and high level of abstraction. In deep learning models, the layers involved may be heterogeneous in nature. Most of the deep learning models are based on artificial neural network and especially convolutional neural networks. In deep learning, each level learns to transform its input data into a slightly more abstract and composite representation. Deep learning architectures can be constructed with a greedy layer-by-layer method. Deep learning helps to disentangle these abstractions and pick out which features improve performance.

[0006] Hence, assisting the driver in autonomous driving also involve determining physiological condition of the driver while riding and alarming the driver is one of the challenging and emerging tasks on which research is going now a days. The level of automation in vehicles is increasing with the time. The level of automation in vehicle with the combination of advance sensor technology and machine learning model can be increased up to a great level and with the increasing level of automation in the vehicles and use of these pre-existing sensors can be able to determine the parameters related to the physiological condition of the driver which is used to assist and alarm the driver for achieving safe and easy driving. Sensors incorporated or installed in the vehicle can determine the physical parameters like heart rate, facial expression, eye movement, etc., using various non-invasive sensors. The various kinds of sensors which may be used in determining the physical parameters may include but not limited to sensors installed into the vehicle interior, sensors available within mobile devices and sensors installed in wearable devices like smart watch etc. The sensed physiological parameter data may be used for variety of ways to determine the emotional state and alertness of the driver. Such data may be used with the use of machine learning models to determine the physiological condition of the driver.

[0007] Hence, developing a system for determining the physiological condition of the driver in autonomous driving and alarming the driver accordingly is the aim of the present invention. There is various prior art that aim to resolve the issue of determining the physiological condition of the driver which are discussed below:

[0008] US20150009010 Al - A vision system of a vehicle includes an interior camera disposed in an interior cabin of a vehicle and having a field of view interior of the vehicle that encompasses an area typically occupied by a head of a driver of the vehicle. An image processor is operable to process image data captured by the camera. The image processor is operable to determine the presence of a person's head in the field of view of the camera and to compare features of the person's face to features of an authorized driver. Responsive at least in part to the comparison of features, operation of the vehicle is allowed only to an authorized driver. The system may store features of one or more authorized driver and may allow operation of the vehicle only when the person occupying the driver seat is recognized or identified as an authorized driver.

[0009] US20060214807 Al - A drowsy driving alarm system includes a monitoring mechanism with a camera and an indicator mechanism carrying drowsy driving software and a processor to process data received from the camera regarding drowsiness of a user of the drowsy driving alarm system. The monitoring mechanism can include at least one power source, at least one interface connection, at least one speaker, and a communication bus communicatively interconnecting elements of the monitoring mechanism. The monitoring mechanism can include at least one visual, audible, and/or physical indicator, a microphone, a transceiver, an antenna, at least one sensor and a compass. The indicator mechanism can include at least one display, at least one visual indicator, and a communication bus interconnecting element of the indicator mechanism. The indicator mechanism can include at least one power source, and at least one interface connection. The indicator mechanism can include at least one audible and/or physical indicator.

[0010] US5402109 A - The invention is an eyeglass attachable alarm signal device for automobile and truck drivers, preventing them to fall asleep, while driving. It is designed both for daytime and for night time driving. A beam of a narrow-band light of any colour is used for optical sensing, whether the driver's eyelids are closed or are in an open position. The use of infrared light is preferred, because infrared light generates least distraction to the driver. A tiny slide-adjustable light emitter carrier, sliding along the eyeglass temple, is used for positioning the light emitter on the eyeglass properly for each driver. A narrow-band light beam from this emitter is aimed across the surface of the driver's eye, just above the eyeball, between the eyelids, and it is sensed in the opposite corner of the eye by means of a light sensor, which has a narrow band light filter mounted in front of it.

[0011] US6346887 B1 - An eye activity monitor of the present invention integrates multiple eye activity measures and applies them to general or custom alertness models to determine the onset of operator drowsiness in real time. An advantage of the eye activity monitor of the present invention is that no physical contact with the subject is required. Another advantage is that eye activity measurements may be made in a non-obtrusive manner, avoiding distractions that might impair the performance of the subject.

[0012] US5583590 A - A system for monitoring alertness of a human while performing certain tasks. Alertness is detected herein by determining the tracking ability of the eye with respect to a visual disturbance, and is measured with respect to head motion. A decision circuit is provided when the motor response of the eye is deemed to be impaired as a result of drowsiness, inattention, or substance abuse.

[0013] JP3293308 B2 - A bodily state detection apparatus comprising a CCD camera, an infrared LED device, a pickup image memory, a pupil extraction circuit and a bodily state judgment circuit. The CCD camera inputs images of a predetermined area including the subject person's face. The infrared LED device illuminates the subject person in such a way that the optical axis of the camera and the direction of the illumination coincide with each other. The pickup image memory stores temporarily the output data of the CCD camera. The pupil extraction circuit extracts the subject person's pupil position from the pickup images. The bodily state judgment circuit judges the subject person's bodily state by use of the result of pupil extraction performed by the pupil extraction circuit.

[0014] Besides this, there are various prior arts in the state of the art that claims to resolve the problem of determining the physiological condition of the driver in autonomous driving but the approach adopted for solving the same need to be further refined. Hence, there is a need to provide more efficient and improved process that provide automatic system for determining the physiological condition of the driver in autonomous driving and improve response time. The aim of the present invention is to provide autonomous physiological condition determination system in autonomous driving and timely alert the driver using machine learning models.

[0015] Groupings of alternative elements or embodiments of the invention disclosed herein are not to be construed as limitations. Each group member can be referred to and claimed individually or in any combination with other members of the group or other elements found herein. One or more members of a group can be included in, or deleted from, a group for reasons of convenience and/or patentability. When any such inclusion or deletion occurs, the specification is herein deemed to contain the group as modified thus fulfilling the written description of all Markus groups used in the appended claims.

[0016] As used in the description herein and throughout the claims that follow, the meaning of "a," "an," and "the" includes plural reference unless the context clearly dictate otherwise. Also, as used in the description herein, the meaning of "in" includes "in" and "on" unless the context clearly dictates otherwise.

[0017] The recitation of ranges of values herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within the range. Unless otherwise indicated herein, each individual value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context.

[0018] The use of any and all examples, or exemplary language (e.g. "such as") provided with respect to certain embodiments herein is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention otherwise claimed. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the invention.

[0019] The above information disclosed in this Background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art.

SUMMARY OF THE INVENTION

[0020] Before the present systems and methods, are described, it is to be understood that this application is not limited to the particular systems, and methodologies described, as there can be multiple possible embodiments which are not expressly illustrated in the present disclosure. It is also to be understood that the terminology used in the description is for the purpose of describing the particular versions or embodiments only and is not intended to limit the scope of the present application. This summary is provided to introduce concepts related to methods for autonomous vehicle driving and traffic sign identification based on deep learning models and the concepts are further described below in the detailed description. This summary is not intended to identify essential features of the claimed subject matter nor is it intended for use in determining or limiting the scope of the claimed subject matter.

[0021] The present invention mainly solves the technical problems existing in the prior art. In response to these problems, the present invention discloses a method for autonomous method for determining physiological condition of the driver in autonomous driving and timely alerting the driver using machine learning models. The solution to the said problem needs to be further optimized so that the proposed system for timely alerting the driver should be more reliable, efficient, faster response time, better accuracy and more assistance will be provided to the driver for better and safe driving.

[0022] The proposed invention is based on the machine learning model. The present invention comprises various advanced sensors installed in the vehicle, sensors available in the mobile device and sensors available in smart wearable devices. The present invention works in line with the autonomous vehicle driving model and determine the physiological parameters of the driver using the sensors mentioned above. In autonomous driving, the predetermined path is associated with some action with a event and the same is associated with the location of the path. The whole route is already captured in the system and path conditions along with driving conditions are mapped with the whole predetermined and captured path.

[0023] The proposed invention comprises of central server comprising a database storing the data related to autonomous driving as mentioned before. The said central server is equipped and works on machine learning model. The said machine learning model is trained using initial databased related to physiological conditions and test cases. The data related to the autonomous driving is also stored on the database in central server which is used to control the autonomous vehicle and assist the driver in driving. The autonomous vehicle also comprises inbuilt GPS which is used to determine the location of the vehicle and correlate the location with the event data and action data for controlling the vehicle. Thus, the determination of physiological condition of the driver is also controlled and determined by the central server equipped with machine learning model.

[0024] In the proposed invention, the vehicle installed sensors and various sensors in mobile device and smart wearable devices are used to collect and capture physiological parameters of the driver. There all sensors are non invasive sensors installed inside or interior of the vehicle, mobile device attached to the vehicle through Bluetooth standard protocol or WiFi, and sensors installed in smart wearable devices also connected to the vehicle through mobile device or Bluetooth standard protocol to the vehicle. The physiological state or parameter related to the driver are captured using the above-mentioned sensors continuously and collected/captured physiological parameters are sent to the central server through communication network. The communication network used for transmitting the mentioned data may be but not limited to Wide Area Network, Local Area Network, Bluetooth protocol, WiFi or the combination thereof. The transmitted data to the central server is first stored in the central server and then analyzed by the central server using machine learning model. Then the central server while analyzing the captured physiological data determines the physiological condition of the driver i.e., emotional state and alertness of the driver. The central server determines whether the driver is feeling drowsiness or sleepiness or not. Then the central server based on the current location of the vehicle captured using the GPS system of the vehicle determines the event data or action data associated with the operation of the vehicle. The event data is analyzed by the central server to co-relate the event data with the physiological condition of the driver and determines whether the driver is in the state of performing the action of the event data and assist the driver for performing the event data and alert the driver related to the drowsiness or sleepiness using audio along with video alarm on the display of the vehicle which alert the driver for further actions and ensures the safe and comfortable driving in autonomous driving.

[0025] An aspect of the present disclosure relates to a method for determining physiological condition of the driver in autonomous driving and alarming the driving using machine learning model, the method comprising, capturing, using the sensors installed in the interior of the vehicle, available in mobile device and smart wearable of the driver, the physiological parameters related to the driver; transmitting, by the communication network, captured physiological parameters to the central server continuously; storing the transmitted physiological parameters in the database at the central server; analyzing at central server, by machine learning model, the captured physiological parameters as and when available; determining the event data based on the location of the vehicle according to route of autonomous driving; comparing, by the machine learning model, the event data with the analyzed physiological state of the driver; assist the driver in autonomous driving in simultaneous to alarm the driver in audio/video on the display of the driver accordingly.

[0026] Another aspect of the present disclosure relates to system for determining physiological condition of the driver in autonomous driving and alarming the driving using machine learning model, the method comprising, capturing, using the sensors installed in the interior of the vehicle, available in mobile device and smart wearable of the driver, the physiological parameters related to the driver; transmitting, by the communication network, captured physiological parameters to the central server continuously; storing the transmitted physiological parameters in the database at the central server; analyzing at central server, by machine learning model, the captured physiological parameters as and when available; determining the event data based on the location of the vehicle according to route of autonomous driving; comparing, by the machine learning model, the event data with the analyzed physiological state of the driver; assist the driver in autonomous driving in simultaneous to alarm the driver in audio/video on the display of the driver accordingly.

[0027] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

OBJECTIVE OF THE INVENTION

[0028] A primary object of the present invention is to provide a method for determining physiological condition of the driver in autonomous driving and alarming the driver using machine learning models. The aim of the present method is to provide faster response time for safe autonomous driving.

[0029] Yet another object of the present invention is to provide a system for determining physiological condition of the driver in autonomous driving and alarming the driver using machine learning models. The system comprising a server equipped with machine learning model, a vehicle having a GPS system and various sensors for capturing driver physiological conditions.

BRIEF DESCRIPTION OF DRAWINGS

[0030] To clarify various aspects of some example embodiments of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof which are illustrated in the appended drawings. It is appreciated that these drawings depict only illustrated embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail through the use of the accompanying drawings.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments belong. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing, suitable methods and materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and not intended to be limiting.

[0032] In order that the advantages of the present invention will be easily understood, a detail description of the invention is discussed below in conjunction with the appended drawings, which, however, should not be considered to limit the scope of the invention to the accompanying drawings, in which:

[0033] Figure 1 shows flow diagram of the method for determining physiological condition of the driver in autonomous driving using machine learning model in accordance with the present invention.

[0034] Figure 2 shows a block-diagram of system for determining physiological condition of the driver in autonomous driving using machine learning model in accordance with the present invention.

DETAIL DESCRIPTION

[0035] The present invention relates to a method and system for determining physiological condition of the driver in autonomous driving using machine learning model.

[0036] Although the present disclosure has been described with the purpose of providing a system for determining physiological condition of the driver in autonomous driving using machine learning model, it should be appreciated that the same has been done merely to illustrate the invention in an exemplary manner and to highlight any other purpose or function for which explained structures or configurations could be used and is covered within the scope of the present disclosure.

[0037] Some embodiments of this disclosure, illustrating all its features, will now be discussed in detail. The words and other forms thereof are intended to be open ended in that an item or items following any one of these words is not meant to be an exhaustive listing of such item or items, or meant to be limited to only the listed item or items. It must also be noted that as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Although any systems and methods similar or equivalent to those described herein can be used in the practice or testing of embodiments of the present disclosure, the exemplary systems and methods are now described. The disclosed embodiments are merely exemplary of the disclosure, which may be embodied in various forms.

[0038] Various modifications to the embodiment will be readily apparent to those skilled in the art and the generic principles herein may be applied to other embodiments. However, one of ordinary skill in the art will readily recognize that the present disclosure is not intended to be limited to the embodiments illustrated, but is to be accorded the widest scope consistent with the principles and features described herein.

[0039] Figure 1 show a flow diagram representing the steps involved in process of determining physiological condition of the driver in autonomous driving using machine learning model. According to the present invention, at step 101, capturing, using the sensors installed in the interior of the vehicle, available in mobile device and smart wearable of the driver, the physiological parameters related to the driver; transmitting, by the communication network, captured physiological parameters to the central server continuously at step 102; storing the transmitted physiological parameters in the database at the central server at step 103; analyzing at central server, by machine learning model, the captured physiological parameters as and when available at step 104; determining the event data based on the location of the vehicle according to route of autonomous driving at step 105; comparing, by the machine learning model, the event data with the analyzed physiological state of the driver at step 106; assist the driver in autonomous driving in simultaneous to alarm the driver in audio/video on the display of the driver at step 107 accordingly.

[0040] Figure 2 shows the block-diagram of system determining physiological condition of the driver in autonomous driving using machine learning model comprising a communication network (101) for connecting various embodiments of the system, a database (102) storing various data and test cases of machine learning model, a central server (103) equipped with machine learning model, a vehicle (104) comprising various sensors (105) for capturing driver physiological parameter data, a GPS system (106) for determining location of the vehicle at any point of time. The said system functions as per the process described above.

[0041] The figures and the foregoing description give examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, order of processes described herein may be changed and are not limited to the manner described herein. Moreover, the actions of any flow diagram need not be implemented in the order shown; nor do all of the acts need to be necessarily performed. Also, those acts that are not dependent on other acts may be performed in parallel with the other acts. The scope of embodiments is by no means limited by these specific examples.

[0042] Although implementations for invention have been described in a language specific to structural features and/or methods, it is to be understood that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as examples of implementations for the invention.

Claims (4)

CLAIMS We claim:

1. A method for determining physiological condition of the driver in autonomous driving using machine learning model, the method comprising: capturing, using the sensors installed in the interior of the vehicle, available in mobile device and smart wearable of the driver, the physiological parameters related to the driver (101); transmitting, by the communication network, captured physiological parameters to the central server continuously (102); storing the transmitted physiological parameters in the database at the central server (103); analyzing at central server, by machine learning model, the captured physiological parameters as and when available (104); determining the event data based on the location of the vehicle according to route of autonomous driving (105); comparing, by the machine learning model, the event data with the analyzed physiological state of the driver (106); assist the driver in autonomous driving in simultaneous to alarm the driver in audio/video on the display of the driver (107).

2. The method as claimed in claim 1, wherein the communication network may be but not limited to Wide Area Network, Local Area Network, WiFi, Bluetooth standard protocol or the combination thereof.

3. The method as claimed in claim 1, wherein the said process may involve n number of vehicles.

4. A system for determining physiological condition of the driver in autonomous driving using machine learning model, the system comprising:

a central server (103) equipped with machine learning model;

a communication network (101) for communication between server and vehicle;

a database (102) for storing various test cases and captured data by the system;

a GPS system (106) and various sensors including interior sensors in the vehicle and mobile device/smart wearable sensors (107) installed in the vehicle;

capturing, using the sensors installed in the interior of the vehicle, available in mobile device and smart wearable of the driver, the physiological parameters related to the driver (101); transmitting, by the communication network, captured physiological parameters to the central server continuously (102); storing the transmitted physiological parameters in the database at the central server (103); analyzing at central server, by machine learning model, the captured physiological parameters as and when available (104); determining the event data based on the location of the vehicle according to route of autonomous driving (105); comparing, by the machine learning model, the event data with the analyzed physiological state of the driver (106); assist the driver in autonomous driving in simultaneous to alarm the driver in audio/video on the display of the driver (107).

capturing, using the sensors installed in the interior of the vehicle, available in mobile device and smart wearable of the driver, the physiological parameters related to the driver (101) 2021105935

transmitting, by the communication network, captured physiological parameters to the central server continuously (102)

storing the transmitted physiological parameters in the database at the central server (103)

analyzing at central server, by machine learning model, the captured physiological parameters as and when available (104)

determining the event data based on the location of the vehicle according to route of autonomous driving (105)

comparing, by the machine learning model, the event data with the analyzed physiological state of the driver (106)

assist the driver in autonomous driving in simultaneous to alarm the driver in audio/video on the display of the driver (107)

Figure 1 flow-chart of method for determining physiological condition of the driver and alarming the driver according to the present invention

GPS sensors (206) 2021105935

(205) Databases (202)

Communication network (201) Server based on deep learning model (203)

Vehicle (204)

Figure 2 Block Diagram of system for determining physiological condition of driver and alarming the same