EP3673478A1 - System and method for racing data analysis using telemetry data and wearable sensor data - Google Patents
System and method for racing data analysis using telemetry data and wearable sensor dataInfo
- Publication number
- EP3673478A1 EP3673478A1 EP18849363.9A EP18849363A EP3673478A1 EP 3673478 A1 EP3673478 A1 EP 3673478A1 EP 18849363 A EP18849363 A EP 18849363A EP 3673478 A1 EP3673478 A1 EP 3673478A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- data
- muscle
- vehicle
- driver
- sensor data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 104
- 238000007405 data analysis Methods 0.000 title abstract description 6
- 210000003205 muscle Anatomy 0.000 claims abstract description 48
- 230000008569 process Effects 0.000 claims description 59
- 210000000245 forearm Anatomy 0.000 claims description 31
- 238000012517 data analytics Methods 0.000 claims description 21
- 238000010801 machine learning Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 3
- 239000004744 fabric Substances 0.000 claims description 2
- 238000013502 data validation Methods 0.000 abstract description 13
- 206010049565 Muscle fatigue Diseases 0.000 abstract description 8
- 238000004458 analytical method Methods 0.000 description 19
- 238000012549 training Methods 0.000 description 17
- 238000001303 quality assessment method Methods 0.000 description 12
- 238000013079 data visualisation Methods 0.000 description 8
- 238000010200 validation analysis Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000013459 approach Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000006872 improvement Effects 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000012800 visualization Methods 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 239000002184 metal Substances 0.000 description 2
- 229910044991 metal oxide Inorganic materials 0.000 description 2
- 150000004706 metal oxides Chemical class 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000002040 relaxant effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 241001633663 Iris pseudacorus Species 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000000038 chest Anatomy 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 229920000547 conjugated polymer Polymers 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000011982 device technology Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000005669 field effect Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 229920000642 polymer Polymers 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000004549 pulsed laser deposition Methods 0.000 description 1
- 238000004451 qualitative analysis Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 238000010223 real-time analysis Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
- A61B5/1107—Measuring contraction of parts of the body, e.g. organ, muscle
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/389—Electromyography [EMG]
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B19/00—Teaching not covered by other main groups of this subclass
- G09B19/16—Control of vehicles or other craft
- G09B19/167—Control of land vehicles
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
- A63F13/65—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor automatically by game devices or servers from real world data, e.g. measurement in live racing competition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04Q—SELECTING
- H04Q9/00—Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom
Definitions
- the disclosure relates generally to vehicle insights and in particular to fatigue reduction insights for a racing car.
- IndyCar is an American- based auto racing sanctioning body for Championship auto racing. Unlike other racing formats, such as the Formula One, IndyCar has regulations forbidding the use of power steering. This requires drivers to exert force with their forearms when turning the wheel, which dramatically deteriorates the driver's performance as the forearm muscles become fatigued during a race. Hence, saving a driver's muscle use during a race is a beneficial insight for the driver. Thus, it is desirable to be able to solve the problem of driver muscle fatigue and thus improve driving performance.
- Figure 1 illustrates an example of an implementation of a vehicle data analytics system that may be used for providing insights for forearm muscle fatigue
- FIG. 1 illustrates more details of the vehicle data analytics system
- Figure 3 illustrates more details of the racing analytics engine of the vehicle data analytics system
- Figure 4 illustrates an example of an implementation of the model training engine of the racing analytics engine
- Figure 5 illustrates an example of an implementation of the quality assessment engine of the racing analytics engine
- Figures 6 and 7 illustrate examples of the data validation processes on exemplary test data
- Figure 8 illustrates an example of an insight analysis process that may be performed by the insights analytics engine of the racing analytics engine
- Figure 9 shows the noisy wearable sensor data
- Figures 10-13 illustrate examples of the user interfaces generates by an example of a web based visualization tool
- Figure 14 illustrates an example of the data visualization for the clustering analysis
- Figure 15 illustrates an example of the data visualization for the similarity analysis
- Figure 16 illustrates an example of the EMG data quality assessment process.
- the disclosure is particularly applicable to forearm fatigue insights for racing vehicles that do not permit power steering and it is in this context that the disclosure will be described. It will be appreciated, however, that the system and method has greater utility since it may be used to determine insights for other factors to improve driver performance, it may be used for other types of vehicles and may be implemented in other ways than those disclosed below in the disclosed embodiments.
- the system and method may solve the above problem of assessing driver performance and driver fatigue using one or more wearable sensors, telemetry data from the vehicle and a racing data analysis system that uses the data from the one or more wearable sensors and the telemetry data from the vehicle to generate actionable insights to improve driver performance.
- the system and method may generate actionable insights about how to reduce forearm muscle fatigue in race cars without power steering wherein the actionable insights may be, for example, locations on a race circuit at which the driver may relax his forearm muscles and reduce the fatigue.
- the system and method tackles two major technological challenges including: 1) data validation on noisy signal obtained from wearable sensors in extreme condition as might exist in a race car; and 2) data cultivation to find actionable insights for the driver from heterogeneous racing data.
- the quality of the signal corning from wearable device is very sensitive to whether or not it is properly attached to the body.
- the system and method may use a data quality validation technique that enables the judgment of whether the data is reliable or not.
- This methodology is based on the comparison between actual Electromyogram (EMG) data and predicted EMG data, which is computed by a Machine Learning technique as described below. If the actual EMG deviates significantly from the predicted EMG, the actual EMG is considered not to be valid.
- the feature in the prediction model uses only the car's telemetry information, i.e., excluding the EMG information itself, as a feature since the EMG signal itself may be too noisy to use for prediction, while the car's telemetry information can provide stable signals.
- test data shows that this qualitative analysis based data validation method works with 99.5% accuracy to classify the data reliability.
- the system and method provide a data visualization and interaction engine that enables the race team to cultivate heterogeneous data and discover useful insights in an intuitive manner.
- the computation method behind this engine may be, in one implementation, a multi-modal analysis of EMG and car telemetry data.
- the analysis may be unsupervised learning using the following technique: 1. cluster data points in a geographical fashion; and 2. find similarity between EMG and the car's telemetry data. Based on this analysis, locations are identified where the driver exerts unnecessary force during the race, in other words, where the driver may be able to rest and recover as shown for example in Figure 15
- Figure 1 illustrates an example of an implementation of a vehicle data analytics system
- the system may generally receive data about the vehicle and data about the driver and generate, using machine learning in part, driving performance insights.
- the system and method may receive/obtain vehicle telemetry data, data from the muscles of one or more forearms of a driver of the vehicle and generate insights about how the driver may reduce muscle fatigue of the forearms while driving the vehicle that does not have power steering.
- the system 10 may include a vehicle 12, such as an Indycar racing car for example, from which various car telemetry data, such as for example accelerometer data (latitude, longitude, vertical), steering angle, speed (mph), throttle pressure, brake pressure, engine rpm, angular acceleration, etc.
- the system 10 may be obtained and communicated to a racing analytics system 18.
- the system 10 may also have a driver garment 14 having one or more wearable sensors 16, such as for example electromyogram (EMG) sensors on each forearm in the implementation for forearm muscle fatigue, wherein the data from the one or more wearable sensors may be communicated to the racing analytics system 18.
- the racing analytics system 18 may receive the telemetry data and sensor data, perform the data validation on noisy signal obtained from wearable sensors in extreme condition as might exist in a race car and perform the data cultivation to find actionable insights for the driver from heterogeneous racing data.
- the system may generate, for example, locations on the race circuit at which the driver may be able to relax his forearms and thus reduce the fatigue of the forearm muscles.
- FIG 2 illustrates more details of the vehicle data analytics system 18.
- the vehicle data analytics system 18 may be implemented in hardware or software.
- each element of the racing analytics engine 206 (shown in more detail in Figure 3) may be a hardware device that is specialized to perform the analysis and data validation as described below.
- one or more elements of the racing analytics engine 206 may be implemented using the same hardware device.
- a racing analytics engine 206 and each element of the racing analytics engine 206 may be implemented using a plurality of lines of instructions/computer code that may be executed by a processor of a computer system that hosts the racing analytics engine 206 so that the processor of the computer system is configured to perform the operations and processes of each of the elements of the racing analytics engine 206.
- the computer system that hosts the elements of the racing analytics engine 206 in a software implementation may be one or more server computers, one or more application servers, one or more cloud computing resources and the like.
- the computer system may include storage 200, memory 202 and one or more processors 204.
- the storage 200 may be a software or hardware implemented storage mechanism that may store, for example, telemetry data, the EMG data, user data, the computer code used to perform the operations and processes of the each of the elements of the racing analytics engine 206 and the results of the data analytics process as described below.
- the memory 202 may temporarily store the telemetry and EMG data, may store the code being executed by the processor to perform the operations and processes of the each of the elements of the racing analytics engine 206 and may store the results of the data analytics.
- the processor 204 may execute the code/plurality of instructions so that the computer system is configured to perform the operations and processes of the each of the elements of the racing analytics engine 206.
- the vehicle data analytics system 18 may receive the telemetry data and the sensor data, such as EMG data for the forearm fatigue example use case, and may generate one or more driver performance insights based on a combination of the telemetry data and the sensor data to improve the performance of the driver.
- the one or more driver performance insights may be one or more locations on a racing circuit (a route over which the vehicle is racing that may be a track, a speedway or streets/roads) at which the driver may be able to relax the forearm muscles.
- the insights may indicate that a straight portion of the racing circuit should be a location on the racing circuit during which the driver should be relaxing his forearm muscles.
- FIG 3 illustrates more details of the racing analytics engine 206 of the vehicle data analytics system.
- the racing analytics engine 206 may include a model training engine 300 and quality assessment engine 302 that may together perform the wearable sensor validation process and thus, for example, determine if the received wearable sensor data is valid and usable to generate the insights of the system.
- the model training engine 300 may implement a machine learning process (shown in more detail in Figure 6) that trains a machine learning model so that the machine learning model may be used to validate the wearable sensor data and determine if the wearable sensor data is valid and thus usable during the insight analytics process.
- the quality assessment engine 302 uses the trained model (trained by the model training engine 300) to generate predictions (using machine learning techniques) to validate each piece of wearable sensor data.
- the quality assessment engine 302 may compare the predicted sensor value by the trained model against the actual sensor data to validate each piece of wearable sensor data since the sensor data is noisy as described above.
- the model training engine 300 and the quality assessment engine 302 provide a technical solution and improvement to the process of driver improvement data and solve the problem of noisy sensor data described above.
- the racing analytics engine 206 may further include an insight analytics engine 304 that receives the telemetry data, the validated sensor data and GPS data to generate the one or more insights.
- this engine 304 may perform clustering and similarity analysis to generate the one or more insights.
- this engine may generate insights for locations on the race circuit at which the drive should relax his forearm muscles.
- the insight analytics engine 304 provides a technical solution and improvement to the process of generating driver insights and solves the problem of being able to automatously generate the driver performance insights.
- the racing analytics engine 206 may further include a user interface engine 306 that may generate the visual displays of the processes, such as reports, data visualizations of the actionable insights and the like as shown in Figures 10-15.
- Data Collection may further include a user interface engine 306 that may generate the visual displays of the processes, such as reports, data visualizations of the actionable insights and the like as shown in Figures 10-15.
- the one or more wearable sensors may include electrocardiogram (ECG) sensors located around the rib cage and one or more sensors that collect signals for EMG with sensors located around the forearm.
- ECG electrocardiogram
- the sensors communicate through a bluetooth receiver connected to an onboard telemetry system of the vehicle and are at some point communicated to the system 18.
- the bluetooth receiver may capture data at 200 samples per second and each race offers different number of laps ranging between 50 to 300 laps with an average distance of 2.2 miles per lap.
- the method may collect from the onboard telemetry system of the vehicle, telemetry data including accelerometer data (latitude, longitude, vertical), steering angle, speed (mph), throttle pressure, brake pressure, engine rpm, and/or angular acceleration.
- the system 10 may also gather GPS coordinates (relative to a fixed point on the track), and seconds of gap between the car ahead and behind the driver.
- the data above may be collected through a private network that is accessible during the race allowing the team to do near-real time analysis although the system 10 may also perform post-race analysis with the data that may be downloaded from the vehicle's telemetry system after the race.
- the system may also collect timestamps from the real time clock (RTC) onboard the vehicle.
- RTC real time clock
- the RTC is used to anchor data collected across differing frequencies. Using this RTC channel value, data from all channels is realigned to the highest frequency channel, while listing a blank value where the source channel collected data at a lower frequency.
- the wearable garment may be a hitoe wearable sensor developed by NTT Docomo that is described in an article by Kazuhiko Takagahara et al. ""hitoe”— A Wearable Sensor Developed through Cross-industrial Collaboration” that may be found at https://vvww.ntt-revievv.jp/archive/ntttechmcai.php?contents-nt.r2()1409ral .h ril which is incorporated herein by reference.
- This hitoe wearable sensor technology may include the one or more EMG sensors to generate the EMG data since each EMG sensor is known and commercially available.
- the system may perform validation of the wearable sensor data to determine if the wearable sensor data can be used for generating the one or more insights generated by the system.
- the data validation process may be performed by the model training engine 300 and the quality assessment engine 302 shown in Figure 3.
- the data validation process for vehicle racing is particularly important since the quality of the signal coming from wearable devices are very sensitive to whether the sensor is properly attached to the body or not.
- the sensor data is highly affected by the extreme forces experienced in racing conditions. If the data is not valid and reliable, it may lead to faulty analysis and incorrect insights.
- the methodology for data validation is based on the comparison of actual EMG data and predicted EMG as shown in Figure 5 and performed by the quality assessment engine 302. If the driver's actual EMG significantly deviates from the predicted EMG, the collected EMG is not considered valid (as indicated by the generated quality score).
- the methodology relies on predicting the EMG value with high accuracy in a reliable manner via Machine Learning using the model training engine 300 shown in Figure 4.
- the system and method use only heterogeneous and reliable car telemetry data as features for the prediction model, i.e., excluding the collected EMG information as a feature.
- the data validation may be performed for each lap of the race course since further analysis depends on lap data.
- FIG. 4 illustrates an example of an implementation of the model training engine 300 of the racing analytics engine.
- the model training engine 300 may train an EMG data prediction model 400.
- the data set may include: [3-axis acceleration respectively [m/s A 2], throttle pedal[%], gyro[rad/s], lap number, pressure brake, speed, steering, hitoe EMG.
- the model 400 may be trained via machine learning.
- the data and features used for the training may be the various telemetry data.
- the machine learning may use ensemble learning that aggregates different prediction algorithms such as Random Forest, XGBoost, etc.. This way, the model 400 can leverage diverse prediction models to produce more accurate results.
- the ensemble learning is one example of the process by which the model may be trained.
- the features for the machine learning are simply designed by car telemetry data.
- the EMG data is not used as a feature since it is highly affected by how the wearable device is attached to body as described above. In the extreme conditions of IndyCar, temporary detachment from the driver's body can easily happen.
- the process only leverages reliable and heterogeneous data (e.g., the telemetry data).
- the model training process may use clean labeled EMG data and method may use datasets obtained from firmly attached wearable fabric (less noisy data) for this training phase.
- the quality of each piece of wearable sensor data is assessed.
- Figure 5 illustrates an example of an implementation of the quality assessment engine 302 of the racing analytics engine that may be used to perform the quality assessment.
- the data quality assessment process 500 is based on the error between actual EMG values and predicted EMG values.
- the system and method may validate data quality lap by lap for the next level of analysis as shown in Figure 16.
- wearable sensor data may be predicted using the features of the telemetry data and the model trained during the model training process 400. Again, the features for this model are only composed of car telemetry information which provide stable signals. Thus, these predicted EMG values are assumed to be able to be acceptably accurate.
- a first line 1602 (a green line in a color drawing) shows the original EMG value coming from sensor and a second line 1604 (shown in red in a color drawing) shows the predicted value from EMG using only car telemetry data.
- the process 500 may then perform an error determination process 502 in which the error between the predicted wearable sensor data and the actual wearable sensor data is determined.
- RMSE Root Mean Squared Error
- N t represents the number of data point of EMG in lap I
- ⁇ ' J represents the actual EMG at index h.
- the process 500 may then have a quality validation process 504 that determined if the data at lap I can be used for advanced analytics or not. This classification is based on threshold against e ⁇ . This threshold is experimentally setup, which performs the best classification performance on training data
- the above data validation process was tested against a clean dataset of wearable sensor data.
- This dataset includes 40 laps of racing data in 10 different practice runs which are composed of 1,655,102 data points.
- the evaluation uses the 5-fold cross validation method. Each segment is divided by laps and the prediction model is constructed using 10 different sizes of dataset from 10% to 100% for analytics experiment.
- the results of the EMG prediction are shown in Figure 6.
- the best prediction error against the test dataset is about 0.220, while the error against the training data is 0.088 with 100% of the data.
- the EMG data is scaled by standardization from the original data, meaning the mean equals 0 and the standard deviation equals 1.0. Therefore, this model results in approximately 22% error in the predicted EMG signal and this prediction outperforms a random prediction model as shown in Figure 6.
- the prediction performance can be improved by further refinement of the machine learning model. Also, as Figure 5 shows, the error gets smaller as the size of dataset increases.
- Figure 7 shows the results of the quality validation and shows the histogram of RMSE for both clean and dirty classification. Note that this figure shows all results obtained by 5 repeated evaluation at once.
- the RMSE of the clean data is relatively small because the EMG prediction performs well for clean data.
- RMSE of dirty data varies widely and is relatively large since the predicted EMG value deviates from the actual EMG value. Because the car telemetry data is not kely to be affected by noise, the prediction should roughly perform within 22% error. Therefore, the actual EMG value is considered to have no abnormality.
- the classification threshold is set to be the minimum value of the RMSE of dirty data, 0.507 in this case, the accuracy of classification becomes 99.48%. Realistically, the threshold should be set up conservatively. Although this causes some loss from the clean dataset available for further analysis, including noisy data for further analytics could lead to faulty analysis, which is far worse than losing some valid data.
- this actionable insights analysis may be performed by the insights analytics engine 304 as shown in Figure 3.
- the actionable insights may be potential points where the driver can improve performance.
- the actionable insights may be locations on the race circuit in which the driver should be relaxing his forearms during the race to reduce forearm fatigue and therefore increase drive performance.
- a process for actionable insights 800 may utilize unsupervised learning.
- the process 800 may identify the EMG correlations with various data points from the car's telemetry information along various GPS coordinates along the race track.
- the process may receive car telemetry data, GPS data and the EMG data.
- the process 800 may include a filter process 802 for filtering the EMG data to prevent the unreasonable or spiky noise, a normalization process 804 for normalizing the EMG data and the telemetry data, a clustering process 806 that uses the GPS data and performs, for example fe-means clustering, on the GPS location data to aggregate the data from multiple laps in the same vicinity, a similarity process 808 that receives the filtered and normalized EMG data and the telemetry data and the clustered GPS data and determines the similarity between normalized EMG and car telemetry data is computed at each clustered location, and a data visualization process 810 that produces data visualization using an interactive tool (an example of which is shown in Figures 8 (for the pipeline) and Figures 10-13 showing the interactive tool.)
- the filter process 802 may remove unreasonable or spiky noise from the wearable sensor data. As shown in Figure 9, the original EMG signal has spiky noise within 0.1 seconds with the interval of about 0.7 seconds. This appears to be due to the sensor measuring the driver's pulse within the EMG data that may be removed.
- the filter process 802 may use a Chebyshev type2 filter. Other possible filters that may be used include Butterworth,
- the clustering process 806, such as &-means clustering in one example, allows the process to understand the general behavior at each GPS location by aggregating the locations on the track.
- the initial centroids for jfe-means clustering are determined based on the complete GPS data on track for one lap. For example, each centroid may be picked 0.5 second intervals. Since the sampling rate of GPS is 0.1 second, clustering aggregates approximately 5 data samples within one location. For example, the method lets denote the data point within 0.1 second, where I is the lap indax, c is the cluster index, and is the data index in cluster c.
- the cluster index c and data index is determined, while I is given in raw data. This owns data of both EMG data and car telemetry data with one GPS data point. Note that this clustering only considers clean data.
- the classification above determines lap index ⁇ whose data meets quality levels for further clustering analysis and, if the data in lap I is classified as not clean, it is simply excluded.
- An example of the data resulting from the clustering for an exemplary race circuit is shown in Figure 12.
- normalization and linear interpolation may be used, as heterogeneous data points have different scales and sampling rates.
- every data point is standardized, meaning the mean equals 0 and the standard deviation equals 1.0.
- the similarity process 808 may determine a similarity between EMG and car telemetry data that can be computed as follows
- the average and the standard deviation are computed as follows. These information is used for next ste , data visualization.
- FIG. 15 An example of the resulting data from the above similarity analysis process for an exemplary race circuit is shown in Figure 15.
- the example in Figure 15 also shows actionable insights (potential relaxation points around the race circuit.
- the visualization process 810 may use a data visualization tool with a web-based user interface. This tool offers the ability to choose a parameter and instantly searching for an analytics result.
- Figure 10 shows an initial screen. Using this tool, users can choose the parameters of race, lap and the data analytics tool to be used. Once users choose the parameters, the result is displayed as shown in Figure 11. Here, users can PAN, zoom or perform other manipulation of the data.
- the examples of clustering analytics results are shown in Figure 12 and Figure 13. The meanings of the shapes and colors are described in Figure 8 above.
- system and method disclosed herein may be implemented via one or more components, systems, servers, appliances, other subcomponents, or distributed between such elements.
- systems may include an/or involve, inter alia, components such as software modules, general-purpose CPU, RAM, etc. found in general- purpose computers,.
- a server may include or involve components such as CPU, RAM, etc., such as those found in general-purpose computers.
- system and method herein may be achieved via implementations with disparate or entirely different software, hardware and/or firmware components, beyond that set forth above.
- components e.g., software, processing
- aspects of the innovations herein may be implemented consistent with numerous general purpose or special purpose computing systems or configurations.
- Various exemplary computing systems, environments, and/or configurations that may be suitable for use with the innovations herein may include, but are not limited to: software or other components within or embodied on personal computers, servers or server computing devices such as routing/connectivity components, hand-held or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, consumer electronic devices, network PCs, other existing computer platforms, distributed computing environments that include one or more of the above systems or devices, etc.
- aspects of the system and method may be achieved via or performed by logic and/or logic instructions including program modules, executed in association with such components or circuitry, for example.
- program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular instructions herein.
- the inventions may also be practiced in the context of distributed software, computer, or circuit settings where circuitry is connected via communication buses, circuitry or links. In distributed settings, control/instructions may occur from both local and remote computer storage media including memory storage devices.
- the software, circuitry and components herein may also include and/or utilize one or more type of computer readable media.
- Computer readable media can be any available media that is resident on, associable with, or can be accessed by such circuits and/or computing components.
- Computer readable media may comprise computer storage media and communication media.
- Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.
- Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and can accessed by computing component.
- Communication media may comprise computer readable instructions, data structures, program modules and/or other components. Further, communication media may include wired media such as a wired network or direct- wired connection, however no media of any such type herein includes transitory media.
- the terms component, module, device, etc. may refer to any type of logical or functional software elements, circuits, blocks and/or processes that may be implemented in a variety of ways.
- the functions of various circuits and/or blocks can be combined with one another into any other number of modules.
- Each module may even be implemented as a software program stored on a tangible memory (e.g., random access memory, read only memory, CD-ROM memory, hard disk drive, etc.) to be read by a central processing unit to implement the functions of the innovations herein.
- the modules can comprise programming instructions transmitted to a general purpose computer or to processing/graphics hardware via a transmission carrier wave.
- the modules can be implemented as hardware logic circuitry implementing the functions encompassed by the innovations herein.
- the modules can be implemented using special purpose instructions (SIMD instructions), field programmable logic arrays or any mix thereof which provides the desired level performance and cost.
- SIMD instructions special purpose instructions
- features consistent with the disclosure may be implemented via computer-hardware, software and/or firmware.
- the systems and methods disclosed herein may be embodied in various forms including, for example, a data processor, such as a computer that also includes a database, digital electronic circuitry, firmware, software, or in combinations of them.
- a data processor such as a computer that also includes a database
- digital electronic circuitry such as a computer
- firmware such as a firmware
- aspects of the method and system described herein, such as the logic may also be implemented as functionality programmed into any of a variety of circuitry, including programmable logic devices (“PLDs”), such as field programmable gate arrays (“FPGAs”), programmable array logic (“PAL”) devices, electrically programmable logic and memory devices and standard cell-based devices, as well as application specific integrated circuits.
- PLDs programmable logic devices
- FPGAs field programmable gate arrays
- PAL programmable array logic
- Some other possibilities for implementing aspects include: memory devices, microcontrollers with memory (such as EEPROM), embedded microprocessors, firmware, software, etc.
- aspects may be embodied in microprocessors having software-based circuit emulation, discrete logic (sequential and combinatorial), custom devices, fuzzy (neural) logic, quantum devices, and hybrids of any of the above device types.
- the underlying device technologies may be provided in a variety of component types, e.g., metal-oxide
- MOSFET semiconductor field-effect transistor
- CMOS complementary metal- oxide semiconductor
- ECL emitter-coupled logic
- polymer technologies e.g., silicon-conjugated polymer and metal-conjugated polymer-metal structures
- mixed analog and digital and so on.
- Computer- readable media in which such formatted data and/or instructions may be embodied include, but are not limited to, non-volatile storage media in various forms (e.g., optical, magnetic or semiconductor storage media) though again does not include transitory media.
- non-volatile storage media in various forms (e.g., optical, magnetic or semiconductor storage media) though again does not include transitory media.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Pathology (AREA)
- Biophysics (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Business, Economics & Management (AREA)
- Educational Technology (AREA)
- Hospice & Palliative Care (AREA)
- Psychiatry (AREA)
- Entrepreneurship & Innovation (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Child & Adolescent Psychology (AREA)
- Educational Administration (AREA)
- Computer Networks & Wireless Communication (AREA)
- Developmental Disabilities (AREA)
- Psychology (AREA)
- Social Psychology (AREA)
- Physiology (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201762549415P | 2017-08-23 | 2017-08-23 | |
PCT/US2018/047615 WO2019040675A1 (en) | 2017-08-23 | 2018-08-22 | System and method for racing data analysis using telemetry data and wearable sensor data |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3673478A1 true EP3673478A1 (en) | 2020-07-01 |
EP3673478A4 EP3673478A4 (en) | 2021-04-28 |
Family
ID=70803010
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP18849363.9A Withdrawn EP3673478A4 (en) | 2017-08-23 | 2018-08-22 | System and method for racing data analysis using telemetry data and wearable sensor data |
Country Status (1)
Country | Link |
---|---|
EP (1) | EP3673478A4 (en) |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20080046779A (en) * | 2006-11-23 | 2008-05-28 | 삼성전자주식회사 | Sleepiness preventive system based on electromyogram and method thereof using a mobile phone |
-
2018
- 2018-08-22 EP EP18849363.9A patent/EP3673478A4/en not_active Withdrawn
Also Published As
Publication number | Publication date |
---|---|
EP3673478A4 (en) | 2021-04-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10566084B2 (en) | System and method for racing data analysis using telemetry data and wearable sensor data | |
US10520947B2 (en) | Machine learning for event detection and classification in autonomous vehicles | |
Xing et al. | Identification and analysis of driver postures for in-vehicle driving activities and secondary tasks recognition | |
Reddy et al. | Real-time driver drowsiness detection for embedded system using model compression of deep neural networks | |
Vlahogianni et al. | Driving analytics using smartphones: Algorithms, comparisons and challenges | |
Gateau et al. | Real-time state estimation in a flight simulator using fNIRS | |
CN110765807B (en) | Driving behavior analysis and processing method, device, equipment and storage medium | |
Khodairy et al. | Driving behavior classification based on oversampled signals of smartphone embedded sensors using an optimized stacked-LSTM neural networks | |
JP7074069B2 (en) | Information processing equipment and methods, and programs | |
Manawadu et al. | Multiclass classification of driver perceived workload using long short-term memory based recurrent neural network | |
US10413204B2 (en) | Apparatus and method for brain computer interface | |
CN107233103B (en) | High-speed rail dispatcher fatigue state evaluation method and system | |
US10660536B2 (en) | Wearable biometric measurement device | |
Alamri et al. | An effective bio-signal-based driver behavior monitoring system using a generalized deep learning approach | |
Ramirez et al. | Inferring drivers’ visual focus attention through head-mounted inertial sensors | |
Tavakoli et al. | Multimodal driver state modeling through unsupervised learning | |
Gatteschi et al. | Comparing algorithms for aggressive driving event detection based on vehicle motion data | |
US10984364B2 (en) | Summarization of a recording for quality control | |
Campos-Ferreira et al. | Vehicle and driver monitoring system using on-board and remote sensors | |
CN110931125A (en) | Discrimination signal identification method and device for cerebral apoplexy | |
CN108898795B (en) | Pedestrian state real-time analysis method and system based on mobile device | |
Amadori et al. | Decision anticipation for driving assistance systems | |
US20230367398A1 (en) | Leveraging machine learning and fractal analysis for classifying motion | |
Dong et al. | Doublecheck: Detecting single-hand cycling with inertial measurement unit of smartphone | |
EP3673478A1 (en) | System and method for racing data analysis using telemetry data and wearable sensor data |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20200323 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
A4 | Supplementary search report drawn up and despatched |
Effective date: 20210326 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G09B 19/16 20060101AFI20210322BHEP Ipc: H04N 21/235 20110101ALI20210322BHEP Ipc: H04Q 9/00 20060101ALI20210322BHEP |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20230203 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN |
|
18W | Application withdrawn |
Effective date: 20230602 |