WO2010083562A1 - Détection d'activité - Google Patents

Détection d'activité Download PDF

Info

Publication number
WO2010083562A1
WO2010083562A1 PCT/AU2010/000056 AU2010000056W WO2010083562A1 WO 2010083562 A1 WO2010083562 A1 WO 2010083562A1 AU 2010000056 W AU2010000056 W AU 2010000056W WO 2010083562 A1 WO2010083562 A1 WO 2010083562A1
Authority
WO
WIPO (PCT)
Prior art keywords
activity
clusters
time series
list
cluster
Prior art date
Application number
PCT/AU2010/000056
Other languages
English (en)
Inventor
Peter Sunehag
Matthew Robards
Original Assignee
National Ict Australia Limited
Australian Sports Commission
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2009900258A external-priority patent/AU2009900258A0/en
Application filed by National Ict Australia Limited, Australian Sports Commission filed Critical National Ict Australia Limited
Publication of WO2010083562A1 publication Critical patent/WO2010083562A1/fr

Links

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B69/00—Training appliances or apparatus for special sports
    • A63B69/12—Arrangements in swimming pools for teaching swimming or for training
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1123—Discriminating type of movement, e.g. walking or running
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2503/00—Evaluating a particular growth phase or type of persons or animals
    • A61B2503/10—Athletes
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2208/00—Characteristics or parameters related to the user or player
    • A63B2208/03—Characteristics or parameters related to the user or player the user being in water
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/40—Acceleration
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/80—Special sensors, transducers or devices therefor
    • A63B2220/83—Special sensors, transducers or devices therefor characterised by the position of the sensor
    • A63B2220/836—Sensors arranged on the body of the user
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0003—Analysing the course of a movement or motion sequences during an exercise or trainings sequence, e.g. swing for golf or tennis
    • A63B24/0006—Computerised comparison for qualitative assessment of motion sequences or the course of a movement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30221—Sports video; Sports image
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients

Definitions

  • the disclosure concerns activity detection, such as but not limited to training of an activity detector and use of an activity detector to detect activities represented in input time series data.
  • activities are swimming stokes, such as freestyle and breaststroke.
  • aspects of the invention include methods, computer systems and software.
  • Clustering analysis is a tool used in the data mining community and beyond. In essence, clustering allows the information of a large data set X to be summarised by creating a much smaller set C of classes and a membership map relating each point in X to its representative in C .
  • a time series data set is a special type of data set as it has a temporal ordering on its data points. That is,
  • (1) whole series clustering - generally the data comprises a number of time series of equal length (say n ).
  • a vector space is formed of dimension n so that each time series is represented by a single point in the space. Clustering then takes place in the usual way and groupings of similar time series are returned.
  • subsequence clustering - generally the data comprises a single long time series data set X ,
  • the aim is to find repeating sequences of features (i.e. a pattern).
  • subsequences of a certain length say p
  • say k the windows considered would be [0,100], [10,110], [20,120],...,[ « -100, « ].
  • Every subsequence is a vector of dimension 100 in this case.
  • Every detected subsequence i.e. segment
  • Every detected subsequence i.e. segment
  • a label identifying that segment as having a pattern that is similar to other segments that also is assigned the same label.
  • time series data could be data captured from sensors (e.g. accelerometers) worn by a moving body. Clustering and pattern identification of this data can enable the identification of distinct movements of the body.
  • the time series data of three dimension observations of acceleration could be identified as a series of primitive actions (activity atoms) undertaken by the wearer of the sensors. In this example detailed monitoring of athletes is possible, which is an important part of their training.
  • a first aspect provides a computer implemented method of training an activity detector to detect activities represented in time series data, the method comprises the steps of:
  • the activity detector can be trained using minimal input, that is, only the time series data and the (received or known) list of ordered activity types. No annotated data is required.
  • semi-Markov clustering repeating patterns of segments can be identified in the data that can be associated with activity types.
  • the detector having been trained what an activity (i.e. segment) of an activity type is represented in the data, can now use the features of these segments for detection of activity types on future data.
  • the activities may be based on human movement.
  • the activity may be a swimming stroke and the activity types may be any two or more of freestyle, breaststroke, butterfly and backstroke.
  • the method may further comprise the step of receiving the list of activity types.
  • the semi-Markov clustering may iteratively attempt to improve a similarity measure of segments within each cluster, such as by extremising a cost function.
  • the segments may be of varying length.
  • the similarity measure between segments of a cluster may comprise determining a feature vector of each segment which may be independent of segment duration. This feature vector represents the segment and may, for example, contain the segment's mean, variance and/or other summarising statistics.
  • the cost function may be based on a measure of the distance between the feature vector of a segment of a candidate activity and a centroid measure of the corresponding cluster of a candidate activity type.
  • Each repeat of step (b) may rely on the set of clusters of a previous repetition of step (b).
  • Step (b) may be repeated until a further criteria is met, such as based on a number of repeats.
  • step (b) may comprise for each repeat determining a score, such as an accuracy score, for the set of clusters, and step (b) may be repeated until a change in the score over a one or more repeats is less than an amount, such as representative of little improvement in the accuracy score.
  • the clustering of step (b) may be K-means clustering, where K is equal to, or greater than, the number of different activity types in the list.
  • the criteria of step (c) may be for a set of clusters, the number of transitions of activity types in the ordered list is the same as the number of transitions of clusters of that set of clusters, where the clusters are in time order based on the underlying time order of sets of consecutively timed segments that comprise each cluster. That is the timing of a cluster is the time of each first segment of each set of time consecutive segments within that cluster.
  • the criteria of step (c) may be for a set of clusters, a number of consecutive segments in time order that are in the same cluster satisfies a minimum number criteria.
  • the criteria of step (c) may comprise the set of clusters and segments within each cluster conforming to a given grammar.
  • the method may further comprise determining a score for each set of clusters, and the criteria of step (c) comprises that the set of clusters have a score representative of being the best solution compared to the other sets of clusters.
  • For each cluster associating an activity type based on the order of the activity types in the list may comprise associating in time order the activity types in the list to clusters that are in time order based on the underlying time order of sets of consecutively timed segments that comprise each cluster. That is, pairing time consecutive segments in the same cluster to activity types in the list in time order. More specifically, the first cluster comprised of the first segment in the time series data and any consecutive segments also comprising the first cluster is associated with the first activity type on the list. The next segment in the time series data associated with a second cluster and any consecutive segments also comprising the second cluster is associated with the second activity type on the list.
  • Associating an activity type to a cluster may comprise associating the activity type with each segment that comprised that cluster.
  • the method may further comprise determining a pattern characteristic that can be used to describe each activity type based on the characteristics of segments within clusters associated with that activity type.
  • a computer system such as an activity detector trainer, is provided for training an activity detector to detect activities represented in time series data
  • the computer system comprising: an input port to receive time series data representing an ordered sequence of activities; a processor to repeatedly generate a set of clusters of the time series data by performing semi-Markov clustering on the data by segmenting the time series data into segments and clustering segments into clusters, where each segment represents a candidate activity and each cluster represents a candidate activity type; to select a set of clusters that satisfy a criteria that is based on at least a list of activity types, wherein the activity types in the list are in the order they are represented in the time series data; and for each cluster of the selected set of clusters, associating an activity type based on the order of the activity types in the list.
  • software is provided, that is computer readable instructions stored on a computer readable medium, that when installed on a computer system causes it to perform the method described above.
  • an automated activity detector for detecting activities represented in time series data, wherein the detector is trained according to the method described above.
  • step (b) the time series data is repeatedly segmented resulting in dynamic segmentation of the time series data.
  • the method is unsupervised.
  • the ordered list of activity types make it possible for us to connect the segments with names provided by the user. This list also helps us choose among the results of the restarts for the semi-Markov clustering. It does this by narrowing the choice down to results that has the property that the activity types that have persisted for at least a specified number of consecutive instances all have a unique corresponding type in the provided list of activity types.
  • the method is suited to learning complex tasks as it avoids the creation of a hard coded rule system for each activity type.
  • the training of the activity detection system in this way has the advantage of only requiring a list of ordered activity types and will automatically detect activities within the input data from which many details about the activities in the data can be extracted.
  • the method is also able to learn to detect multiple activities and to deal with increasingly complex inputs of activity type lists. Further, the method can be applied repeatedly to learn to detect activities that belong to say, different users. In this way the method is able to easily accommodate for differences between users.
  • the method does not rely on sliding windows to perform time series clustering, instead semi-Markov clustering is used.
  • Fig. l is a flow chart of the semi-Markov Kmeans clustering training technique described
  • Fig. 2 is a sample flow diagram of this technique
  • Figs. 3 to 9 are pseudo code explaining in further detail steps of the flow diagram of Fig. 2;
  • Fig. 10 is a schematic diagram of a computer system used in one example
  • Fig. 11 (a) schematically shows a sample grammar and Fig. l l(b) shows that grammar applied to the swimming example of Fig. 10;
  • Fig. 12 schematically shows a special case of the grammar in Fig. 11;
  • Fig. 13 schematically shows the application of the additional grammar and heuristics to of segmenting in the time series data
  • Figs. 14 and 16 show graphically the results of various implementations.
  • Subsequence clustering aims to find patterns that appear repeatedly in time series data. We describe here a novel subsequence clustering technique that is called here "semi- Markov kmeans clustering".
  • the aim of the training module is to train a computerised activity detector to detect activities represented in time series data.
  • the training module produces the time series data that is segmented into primitive activities (i.e. activity atoms). That is, each segment represents a primitive activity (such as a swimming stoke, step or push up).
  • Each segment of the same activity type is given the same label, and each label is associated with an activity type in a ordered list of activities types that are represented in the time series data.
  • This training method is unsupervised and does not require segments of fixed length - instead it works with all possible segmentations to find one way of segmenting the sequence into consistent patterns, preferably an optimal way.
  • the output from the training module can then be used by an activity detector to automatically identify the activities in further time series data.
  • the data x consists of a sequence of observations X ' , l ⁇ l> — > TM .
  • labels ' where ⁇ ⁇ - ⁇ *> • ⁇ > L ) denotes the set of possible labels.
  • the set S(n,l) is defined by a combination of a grammar 20(d) which says which label transitions are allowed, e.g. in a gym a person who is working out on the training bicycle cannot directly transition to working out on the rowing machine without having some state like standing or walking in between. Grammars will be described in further detail below. We also restrict the possible durations of a segment by giving minimum and maximum allowed length and we can also for computational reasons introduce what we call a step size which regulates which segment lengths between the minimum and the maximum lengths that we will consider.
  • a stopping criteria is provided, such as a convergence threshold or number of iterations.
  • Inputs of number of clusters, stopping criteria and the grammar may be predetermined and accessed from memory by the training module.
  • Semi-Markov means dependencies are local, but based on segments rather than points.
  • This loss function is designed to obtain a segmentation where all segments which are given the same label are found to be "similar", according to some notion of similarity with cluster centroids.
  • the similarity is measured based on a square euclidean distance between a feature vector of a proposed segment and the centroid corresponding to the proposed cluster.
  • These centroids are found in an unsupervised fashion, and are analogous to the cluster centroids in the traditional kMeans algorithm.
  • These calculations can be distributed over the sensors, as long as the features for each sensor are calculated based only on the data from that sensor. In other words features and similarities can be calculated for each individual sensor, in parallel, and later combined.
  • the number of bins can be varied, and in our ECG implementation below we also modify the bins by subtracting the mean of the whole segment. This is important if one wants to be able to recognize the same pattern at different height.
  • derivative features which are defined as the difference between a given mean bin and the preceeding mean bin. Ten mean bins therefore result in nine derivative features.
  • the third kind of features we use is the variance over the whole segment. Variance can be binned like mean features, however, it has been found unnecessary for this study. For every implementation described below we will describe the exact choice of features used. In our semi-Markov kMeans algorithm, we find a labeled segmentation y * of a given data sequence x by performing the following minimization:
  • the predicted segmentation is calculated by minimizing (3).
  • This prediction step is actually in itself a novel supervised method that we call the semi-Markov nearest centroid method. It can be trained from labeled segmentations by simply calculating the centroid of each label through (4). Dealing with the problem of how to perform the minimization in a tractable manner through dynamic programming is described further below.
  • ⁇ (/) is the covariance matrix for our model of class / .
  • centroids and standard deviations are chosen through maximum likelihood estimation for each class using the gaussian density function.
  • centroid ⁇ in (10) corresponds to that found in (9).
  • ⁇ (/) a matrix that agrees with it for the diagonal elements but whose non- diagonal elements are zero.
  • dynamic programming can be carried out in the following manner;
  • V ⁇ n,l a score function and let U(n,l) be a tuple of a position and label.
  • U(n,l) be a tuple of a position and label.
  • V(n,l): m ⁇ xV ⁇ n',l')+ f ⁇ n',n,l) (13)
  • the output is a labeled segmentation of y .
  • the matrix ® is initialised 34, and this is described in further detail in the pesudo code of Fig. 4.
  • the matrix ® is initialised from a random sampling of the raw data.
  • ⁇ is comprised of KCLASS columns 36 and NUMBINS rows 38.
  • Each columns 36 (a single class) corresponds to a particular type of segment/atom and arranged by bin type.
  • Each bin corresponds to a particular shape in the underlying data.
  • the segment/atom is a non linear function of the data.
  • INITIALISEMAP can rely on the results of any previous segmentations rather than generating ® at random.
  • Line 7 of Fig. 4 calculates the "bin statistic" for the bin.
  • bins are: 1. mean bins: the statistic is the mean of the data segment 42 2. difference bins: the statistic is the difference between two adjacent mean bins 44
  • variance bins the statistic is the variance on the data segment 46 The enumeration above defined the values for t in line 4.
  • the VITERBI algorithm (shown in Fig. 6) of GENERATESEGMENTS effectively tries every possible labelling. Viterbi is, in this case, enabled by the Markov restriction on the atoms. Further, the recursive step forces the next segment to start at the end of the current segment, while allowing the segments to be of variable length (see lines 5 and 11 of Fig. 6). Since the method is Markov with respect to atoms, it is semi-Markov with respect to the underlying data.
  • the determined Y is then used to update the matrix map, see Fig. 8 for UPDATEMAP.
  • the selected set of clusters that will be used as the basis for the output of the method meets certain criteria.
  • the criteria based on the ordering of the activity list is shown in Fig. 3 as being tested by REALITYCHECK within the repeating loop of 60.
  • these criteria can be applied outside the loop 60, such as testing all the generated sets of clusters for the one set that meets the REALITYCHECK and is the best scored outside repeating loop 60 in order to select the optimal set of segments.
  • the activity detector can now be used to detect activities in further time series data received, such as input from the swimmer's further training.
  • Max-margin semi-Markov model can be used.
  • One among many alternative detection algorithms is the semi-Markov nearest centroid method which appears in the training above.
  • Using the semi-Markov nearest centroid method is equivalent to using the GENERATESEGMENTS function with the patterns stored during training in the matrix ® .
  • Implementation 1 In this example time series data taken from sensors worn by a swimmer during training is provided. Initially an activity detector is trained, which then can identify the strokes in further received data. This in turn allows for a comprehensive break-down of training sessions, including lap times, detailed statistics of strokes and turns.
  • the swimmer 10 is wearing sensors 12 while performing an activity.
  • the sensors 12 may be three dimensional accelerometers that measure the swimmer's movement in three directions.
  • the multiple sensors 12 are attached to key swimming movement areas of the swimmer and are each wireless transmitters that cause the sensed data to be transmitted in real time to the computer 14.
  • the time series data may be provided to a computer after the activity is completed, such as through a wired connection.
  • the time series data representing an ordered sequence of activities, in this case swimming strokes, is received at the input/output port 16 of the computer 14, such as a bluetooth receiver.
  • the processor 18 causes the received sensed data is stored in the internal memory 20 of the computer 14.
  • an interface such as an interface driven by the processor and displayed on a monitor (not shown) connected to the port 16
  • a user such as the swimmer or the swimmer's coach is able to provide further input of a list of activity types in an order they are represented in the time series data. This may be done by selecting the swimming strokes in order from a pick list displayed on the interface.
  • This activity list is also received at the port 16 and stored in internal memory 20 of the computer 14.
  • a training module 18(a) There are two main modules operated by the processor 18, a training module 18(a) and an activity detection module 18(b). This technique described concerns the training module 18(a). Using installed software, the processor 18 accesses the internal memory 20 to obtain the required input and operates in the method described in detail above.
  • the aim of this training is to process the received data as described above to identify segments in the data of primitive activities, so that each segment represents a certain type of swimming stroke, and all segments representing the same swimming stroke share similarities.
  • the output of the segmented data is provided as port 22 and again is displayed using the monitor to the user.
  • the output may be stored in internal memory 20 as an invertible map from clusters to activity types in the provided list which can translate the cluster list to the given activity type list and back again. This map is ready for use by the activity detection module 18(b) on further data received to guide the segmentation process.
  • the timing of the start and end of each segment, and segment groups, can be used by the processor 18 to provide further output on various statistics on timing that are sent to the output port.
  • training module 18(a) and the training module 18(b) are contained within the same computer system 14. In other embodiments, these modules 18(a) and 18(b) may be provided on separate computer systems.
  • grammars can be provided as input to guide the clustering technique. Grammars provide flexibility by avoiding hard coding of these rules. These predefined grammars define allowable state transitions. In this example, a grammar with one or more repeating states is used.
  • Fig. 11 (a) shows a sample grammar that can be applied to repeatable activities, such as gymnasium exercises or swimming.
  • This grammar displayed includes a simple repeating pattern 50, that is one state that repeats.
  • This simple repeating pattern 50 is contained within a complex repeating pattern having multiple states in a specified order.
  • This case the complex repeating pattern includes an activity 52 before the simple repeating pattern 50 and a further activity 53 after 50.
  • this grammar 52, 50 and 53 are repeated in this order.
  • the grammar also takes account that some activities may occur at the very start 56 and at the very end 58 that are not part of the complex repeating grammar.
  • Fig. l l(b) shows the grammar of Fig. 11 (a) applied to swimming.
  • the simple repeating pattern 50 is the swimming strokes.
  • the push 52 and a turn 53 can bound every lap of a particular stroke.
  • an activity 56 such as a dive or splash that is not part of a repeating pattern can be provided for, in the same way climbing out of the pool 58 is provided outside the complex repeating pattern.
  • Fig. 12 schematically shows a four stroke system, with repeating stroke types. This grammar can be considered a special case of the grammar of Fig. 1 l(a).
  • Fig. 13 shows the application of the grammar of Fig. l l(b) and additional heuristics to the detection and understanding of the received data 100.
  • the raw data 100 is a time series of unlabeled measurements from the sensors described for Fig. 10. Labels are also received, such as Freestyle laps and then a butterfly lap, but no additional timing information is provided.
  • the first pass 108 of the algorithm shown in Fig. 3 detects simple repeating patterns.
  • the predetermine grammar is then accessed 110 and the simple repeating pattern of the grammar to condense each set of repeating states into 1 meta-state group.
  • a meta state is a variable in length in that they contain different number of strokes.
  • the new "meta data" now contains unknown, unclassified data 104 and classified data 102.
  • a second pass 112 of the algorithm of Fig. 3 is applied to detect complex patterns in combined group and unknown data. For example, using the complex grammar to evaluate the presence of turns or terminals in the data.
  • This step may also use heuristics, such as a lap of one type of stroke, followed by a lap of a stroke with a short intervening section of unclassified data has a high probability of being a lap-turn-lp sequence.
  • a heuristic may be applied to the unknown data to estimate likelihood of "turn" within the data, given prior knowledge of the surrounding stroke-types.
  • This process may be continued for successively more complex activity grammars.
  • the simple-repeating activity may be a "step” on a treadmill or a "lift” of a weight and the grammar can be applied to the gymnasium setting.
  • the original CBF task was formulated by Saito (Saito, 1994) and consists of classifying a time series as one of three classes; Cylinder, Bell or Funnel.
  • the time series are defined from these numbers by letting
  • Fig. 14 shows the original CBF data with segment boundaries and tokens as found by semi-Markov kmeans.
  • Fig. 15 shows a "reconstruction" of the original CBF data by replacing each segment with the corresponding centroid found by semi-Markov kmeans. More specifically this reconstruction is found by dilating the centroid such that it is the same length as the corresponding segments.
  • the activities undertaken were back extensions, side twists, push-ups, sit-ups and squat jumping.
  • the features used were 20 mean bins, 19 derivative bins and the variance of the segment.
  • We demanded a consistency of at least 5 consecutive segments of the same type. This was run with k 10 .
  • the examples can be applied to other sports, such as skiing, as well as numerous domains, such as patient care, chronic disease management and promotion of lifelong health and well-being for the aging population. For example, monitoring the rehabilitation of patients wearing sensors in post-ambulatory conditions, detecting abnormalities in ECG data, gait detection for children suffering cerebral palsy and monitoring Parkinson's disease patients.
  • the examples can be applied to data that is captured in time series from which repeating atoms can be identified, such as financial and physiological data. For example, with financial data the example can be used learn and detect rule discovery.
  • the example can be used in the task of making mobile devices aware of the context of the user.
  • the sensor data may again be interpreted as consisting of primitive activities whose frequency can indicate higher level context like driving a car.
  • K-means used as the clustering technique.
  • Other clustering methods can be extended to the semi-Markov framework, such as Gaussian mixture models trained with the Expectation Maximization (EM) method and hierarchical clustering.
  • EM Expectation Maximization
  • the supervised classification and segmentation part can be another method that can learn from fully segmented and labelled time series training data, such as semi-Markov conditional random fields (semi-CRF) and hidden Markov models models (HMM).
  • the function of the computer 14 shown in Fig. 10 may be contained in a housing with a sensor, such as a device worn on the wrist.
  • the processing may be distributed, for example the processing may be distributed between two or more of the sensor device, a remote PC or a computer device permanent at the site of the activity itself, such as the pool.
  • Suitable computer readable media may include volatile (e.g. RAM) and/or non-volatile (e.g. ROM, disk) memory, carrier waves and transmission media (e.g. copper wire, coaxial cable, fibre optic media).
  • exemplary carrier waves may take the form of electrical, electromagnetic or optical signals conveying digital data steams along a local network or a publically accessible network such as the internet.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Public Health (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Multimedia (AREA)
  • Physiology (AREA)
  • Psychiatry (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Fuzzy Systems (AREA)
  • Signal Processing (AREA)
  • Evolutionary Computation (AREA)
  • Mathematical Physics (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

La description décrit une détection d'activité, telle que l'apprentissage d'un détecteur d'activité pour détecter des activités représentées dans des données en série temporelle d'entrée. Un exemple d'activité est la nage, telle que la nage libre et la brasse. Des données en série temporelle représentant une séquence ordonnée d'activités 30 sont fournies en entrée. Le procédé comprend de manière répétée 60 la génération d'un ensemble de groupes des données en série temporelle par la réalisation de manière itérative 40 d'un regroupement semi-Markov sur les données 30. Un ensemble de groupes qui satisfont un critère qui est fondé sur au moins une liste de types d'activité 32 est sélectionné de telle sorte que pour chaque groupe de l'ensemble sélectionné de groupes, il y a association d'un type d'activité sur la base de l'ordre des types d'activité dans la liste. Un avantage est que le détecteur d'activité peut subir un apprentissage à l'aide d'une entrée minimale, autrement dit, uniquement les données en série temporelle et la liste (reçue ou connue) de types d'activité ordonnés. Aucune donnée annotée n'est requise. Des aspects comprennent des procédés, des systèmes informatiques, des détecteurs d'activité et un logiciel.
PCT/AU2010/000056 2009-01-22 2010-01-21 Détection d'activité WO2010083562A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2009900258A AU2009900258A0 (en) 2009-01-22 Activity Detection
AU2009900258 2009-01-22

Publications (1)

Publication Number Publication Date
WO2010083562A1 true WO2010083562A1 (fr) 2010-07-29

Family

ID=42355435

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2010/000056 WO2010083562A1 (fr) 2009-01-22 2010-01-21 Détection d'activité

Country Status (1)

Country Link
WO (1) WO2010083562A1 (fr)

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2464062C1 (ru) * 2011-02-09 2012-10-20 Борис Аронович Дышко Способ оценки техники плавания
US20130179110A1 (en) * 2012-01-09 2013-07-11 Invensense, Inc. Activity classification in a multi-axis activity monitor device
WO2013140075A1 (fr) * 2012-03-20 2013-09-26 Commissariat à l'énergie atomique et aux énergies alternatives Dispositif et procede d'identification d'un mouvement cyclique, programme d'ordinateur correspondant
GB2511833A (en) * 2013-03-15 2014-09-17 Suunto Oy Device and method for monitoring swimming performance
CN104899248A (zh) * 2015-04-24 2015-09-09 济南大学 一种通用的运动捕获数据检索方法
WO2015187344A1 (fr) * 2014-06-02 2015-12-10 Qualcomm Incorporated Obtention de caractéristiques d'utilisateurs à partir de fichiers journaux d'utilisateurs
WO2015195765A1 (fr) * 2014-06-17 2015-12-23 Nant Vision, Inc. Systèmes et procédés de reconnaissance d'activité
EP3032455A1 (fr) * 2014-12-09 2016-06-15 Movea Dispositif et procédé pour la classification et la reclassification d'activité d'un utilisateur
WO2016122591A1 (fr) * 2015-01-30 2016-08-04 Hewlett Packard Enterprise Development Lp Test de performance basé sur une segmentation et un regroupement de longueur variable de données chronologiques
KR20160120331A (ko) * 2014-02-14 2016-10-17 쓰리엠 이노베이티브 프로퍼티즈 캄파니 가속도계 데이터를 사용하는 활동 인식
US10486049B2 (en) 2013-03-15 2019-11-26 Amer Sports Digital Services Oy Device and method for monitoring swimming performance
CN111351524A (zh) * 2018-12-21 2020-06-30 亚玛芬体育数字服务公司 传感器数据管理
US10856776B2 (en) 2015-12-21 2020-12-08 Amer Sports Digital Services Oy Activity intensity level determination
US11137820B2 (en) 2015-12-01 2021-10-05 Amer Sports Digital Services Oy Apparatus and method for presenting thematic maps
US11144107B2 (en) 2015-12-01 2021-10-12 Amer Sports Digital Services Oy Apparatus and method for presenting thematic maps
US11145272B2 (en) 2016-10-17 2021-10-12 Amer Sports Digital Services Oy Embedded computing device
US11210299B2 (en) 2015-12-01 2021-12-28 Amer Sports Digital Services Oy Apparatus and method for presenting thematic maps
US11215457B2 (en) 2015-12-01 2022-01-04 Amer Sports Digital Services Oy Thematic map based route optimization
US11284807B2 (en) 2015-12-21 2022-03-29 Amer Sports Digital Services Oy Engaging exercising devices with a mobile device
US11541280B2 (en) 2015-12-21 2023-01-03 Suunto Oy Apparatus and exercising device
US11587484B2 (en) 2015-12-21 2023-02-21 Suunto Oy Method for controlling a display
US11607144B2 (en) 2015-12-21 2023-03-21 Suunto Oy Sensor based context management
US11703938B2 (en) 2016-10-17 2023-07-18 Suunto Oy Embedded computing device
CN116884094A (zh) * 2023-09-07 2023-10-13 武汉理工大学 基于视角和行为解耦的多视角行为识别方法及系统
US11838990B2 (en) 2015-12-21 2023-12-05 Suunto Oy Communicating sensor data in wireless communication systems

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6441846B1 (en) * 1998-06-22 2002-08-27 Lucent Technologies Inc. Method and apparatus for deriving novel sports statistics from real time tracking of sporting events
US20050256817A1 (en) * 2004-05-12 2005-11-17 Wren Christopher R Determining temporal patterns in sensed data sequences by hierarchical decomposition of hidden Markov models
US6993462B1 (en) * 1999-09-16 2006-01-31 Hewlett-Packard Development Company, L.P. Method for motion synthesis and interpolation using switching linear dynamic system models
US20080030460A1 (en) * 2000-07-24 2008-02-07 Gesturetek, Inc. Video-based image control system
US20090016610A1 (en) * 2007-07-09 2009-01-15 Honeywell International Inc. Methods of Using Motion-Texture Analysis to Perform Activity Recognition and Detect Abnormal Patterns of Activities

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6441846B1 (en) * 1998-06-22 2002-08-27 Lucent Technologies Inc. Method and apparatus for deriving novel sports statistics from real time tracking of sporting events
US6993462B1 (en) * 1999-09-16 2006-01-31 Hewlett-Packard Development Company, L.P. Method for motion synthesis and interpolation using switching linear dynamic system models
US20080030460A1 (en) * 2000-07-24 2008-02-07 Gesturetek, Inc. Video-based image control system
US20050256817A1 (en) * 2004-05-12 2005-11-17 Wren Christopher R Determining temporal patterns in sensed data sequences by hierarchical decomposition of hidden Markov models
US20090016610A1 (en) * 2007-07-09 2009-01-15 Honeywell International Inc. Methods of Using Motion-Texture Analysis to Perform Activity Recognition and Detect Abnormal Patterns of Activities

Cited By (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2464062C1 (ru) * 2011-02-09 2012-10-20 Борис Аронович Дышко Способ оценки техники плавания
US20130179110A1 (en) * 2012-01-09 2013-07-11 Invensense, Inc. Activity classification in a multi-axis activity monitor device
US11026600B2 (en) * 2012-01-09 2021-06-08 Invensense, Inc. Activity classification in a multi-axis activity monitor device
WO2013140075A1 (fr) * 2012-03-20 2013-09-26 Commissariat à l'énergie atomique et aux énergies alternatives Dispositif et procede d'identification d'un mouvement cyclique, programme d'ordinateur correspondant
FR2988501A1 (fr) * 2012-03-20 2013-09-27 Commissariat Energie Atomique Dispositif et procede d'identification d'un mouvement cyclique, programme d'ordinateur correspondant
US9952043B2 (en) 2012-03-20 2018-04-24 Commissariat à l'énergie atomique et aux énergies alternatives Device and method for identifying a cyclic movement and corresponding computer program
GB2511833A (en) * 2013-03-15 2014-09-17 Suunto Oy Device and method for monitoring swimming performance
US10486049B2 (en) 2013-03-15 2019-11-26 Amer Sports Digital Services Oy Device and method for monitoring swimming performance
GB2511833B (en) * 2013-03-15 2016-01-13 Suunto Oy Device and method for monitoring swimming performance
US10881327B2 (en) 2014-02-14 2021-01-05 3M Innovative Properties Company Activity recognition using accelerometer data
AU2017251830B2 (en) * 2014-02-14 2019-11-28 3M Innovative Properties Company Activity recognition using accelerometer data
KR20160120331A (ko) * 2014-02-14 2016-10-17 쓰리엠 이노베이티브 프로퍼티즈 캄파니 가속도계 데이터를 사용하는 활동 인식
KR102306064B1 (ko) 2014-02-14 2021-09-30 쓰리엠 이노베이티브 프로퍼티즈 캄파니 가속도계 데이터를 사용하는 활동 인식
WO2015187344A1 (fr) * 2014-06-02 2015-12-10 Qualcomm Incorporated Obtention de caractéristiques d'utilisateurs à partir de fichiers journaux d'utilisateurs
JP2017525070A (ja) * 2014-06-17 2017-08-31 ナント ホールディングス アイピー， エルエルシー 行動認識システム及び方法
US9886625B2 (en) 2014-06-17 2018-02-06 Nant Holdings Ip, Llc Activity recognition systems and methods
CN106462747A (zh) * 2014-06-17 2017-02-22 河谷控股Ip有限责任公司 活动识别系统和方法
US10216984B2 (en) 2014-06-17 2019-02-26 Nant Holdings Ip, Llc Activity recognition systems and methods
US9547678B2 (en) 2014-06-17 2017-01-17 Nant Holdings Ip, Llc Activity recognition systems and methods
US10572724B2 (en) 2014-06-17 2020-02-25 Nant Holdings Ip, Llc Activity recognition systems and methods
US11232292B2 (en) 2014-06-17 2022-01-25 Nant Holdings Ip, Llc Activity recognition systems and methods
WO2015195765A1 (fr) * 2014-06-17 2015-12-23 Nant Vision, Inc. Systèmes et procédés de reconnaissance d'activité
US11837027B2 (en) 2014-06-17 2023-12-05 Nant Holdings Ip, Llc Activity recognition systems and methods
EP3032455A1 (fr) * 2014-12-09 2016-06-15 Movea Dispositif et procédé pour la classification et la reclassification d'activité d'un utilisateur
WO2016122591A1 (fr) * 2015-01-30 2016-08-04 Hewlett Packard Enterprise Development Lp Test de performance basé sur une segmentation et un regroupement de longueur variable de données chronologiques
US10643138B2 (en) 2015-01-30 2020-05-05 Micro Focus Llc Performance testing based on variable length segmentation and clustering of time series data
CN104899248A (zh) * 2015-04-24 2015-09-09 济南大学 一种通用的运动捕获数据检索方法
US11144107B2 (en) 2015-12-01 2021-10-12 Amer Sports Digital Services Oy Apparatus and method for presenting thematic maps
US11210299B2 (en) 2015-12-01 2021-12-28 Amer Sports Digital Services Oy Apparatus and method for presenting thematic maps
US11137820B2 (en) 2015-12-01 2021-10-05 Amer Sports Digital Services Oy Apparatus and method for presenting thematic maps
US11215457B2 (en) 2015-12-01 2022-01-04 Amer Sports Digital Services Oy Thematic map based route optimization
US11284807B2 (en) 2015-12-21 2022-03-29 Amer Sports Digital Services Oy Engaging exercising devices with a mobile device
US11541280B2 (en) 2015-12-21 2023-01-03 Suunto Oy Apparatus and exercising device
US11587484B2 (en) 2015-12-21 2023-02-21 Suunto Oy Method for controlling a display
US11607144B2 (en) 2015-12-21 2023-03-21 Suunto Oy Sensor based context management
US11838990B2 (en) 2015-12-21 2023-12-05 Suunto Oy Communicating sensor data in wireless communication systems
US10856776B2 (en) 2015-12-21 2020-12-08 Amer Sports Digital Services Oy Activity intensity level determination
US11145272B2 (en) 2016-10-17 2021-10-12 Amer Sports Digital Services Oy Embedded computing device
US11703938B2 (en) 2016-10-17 2023-07-18 Suunto Oy Embedded computing device
CN111351524A (zh) * 2018-12-21 2020-06-30 亚玛芬体育数字服务公司 传感器数据管理
TWI729596B (zh) * 2018-12-21 2021-06-01 芬蘭商亞瑪芬體育數字服務公司 感測器資料管理
CN116884094A (zh) * 2023-09-07 2023-10-13 武汉理工大学 基于视角和行为解耦的多视角行为识别方法及系统
CN116884094B (zh) * 2023-09-07 2023-12-12 武汉理工大学 基于视角和行为解耦的多视角行为识别方法及系统

Similar Documents

Publication Publication Date Title
WO2010083562A1 (fr) Détection d'activité
CN111027487B (zh) 基于多卷积核残差网络的行为识别系统、方法、介质及设备
Namkung Machine learning methods for microbiome studies
Zhang et al. A feature selection-based framework for human activity recognition using wearable multimodal sensors
Brajdic et al. Walk detection and step counting on unconstrained smartphones
Crema et al. IMU-based solution for automatic detection and classification of exercises in the fitness scenario
CN103892840B (zh) 一种智能穿戴设备及人体运动的特征提取方法
Bu Human motion gesture recognition algorithm in video based on convolutional neural features of training images
CN104135911A (zh) 多轴线活动监测装置中的活动分类
CN109376663A (zh) 一种人体姿态识别方法及相关装置
CN111583194B (zh) 基于贝叶斯粗糙集和布谷鸟算法的高维特征选择算法
Hausberger et al. IMU-based smart fitness devices for weight training
CN103908259A (zh) 一种智能穿戴设备及人体运动的监测与识别方法
Crema et al. Characterization of a wearable system for automatic supervision of fitness exercises
CN114913547B (zh) 基于改进的Transformer网络的跌倒检测方法
Luqian et al. Human activity recognition using time series pattern recognition model-based on tsfresh features
CN114816069A (zh) 基于多源域集成迁移的跨用户肌电手势识别方法及系统
Yang et al. Comparing cross-subject performance on human activities recognition using learning models
CN112784812B (zh) 一种深蹲动作识别方法
CN104021295B (zh) 用于运动识别的聚类特征融合方法和装置
CN116747495A (zh) 一种动作计数方法、装置、终端设备及可读存储介质
Hosein et al. Wearable sensor selection, motion representation and their effect on exercise classification
Gerych et al. Gan for generating user-specific human activity data from an incomplete training corpus
Kumari et al. Effect of reduced dimensionality on deep learning for human activity recognition
CN115410267A (zh) 一种基于人体骨骼与肌肉相互配合动作分析数据统计算法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10733144

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10733144

Country of ref document: EP

Kind code of ref document: A1