WO2023274792A1 - Methods, architectures, apparatuses and systems directed to recognizing an activity in a set of activities - Google Patents

Methods, architectures, apparatuses and systems directed to recognizing an activity in a set of activities Download PDF

Info

Publication number
WO2023274792A1
WO2023274792A1 PCT/EP2022/066914 EP2022066914W WO2023274792A1 WO 2023274792 A1 WO2023274792 A1 WO 2023274792A1 EP 2022066914 W EP2022066914 W EP 2022066914W WO 2023274792 A1 WO2023274792 A1 WO 2023274792A1
Authority
WO
WIPO (PCT)
Prior art keywords
activity
signatures
activities
sub
signature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2022/066914
Other languages
French (fr)
Inventor
Abdullah HASKOU
Ali Louzir
Anthony Pesin
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
InterDigital CE Patent Holdings SAS
Original Assignee
InterDigital CE Patent Holdings SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by InterDigital CE Patent Holdings SAS filed Critical InterDigital CE Patent Holdings SAS
Publication of WO2023274792A1 publication Critical patent/WO2023274792A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50Systems of measurement based on relative movement of target
    • G01S13/52Discriminating between fixed and moving objects or between objects moving at different speeds
    • G01S13/56Discriminating between fixed and moving objects or between objects moving at different speeds for presence detection
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/66Radar-tracking systems; Analogous systems
    • G01S13/72Radar-tracking systems; Analogous systems for two-dimensional [2D] tracking, e.g. combination of angle and range tracking, track-while-scan radar
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/88Radar or analogous systems specially adapted for specific applications
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • G06V40/23Recognition of whole body movements, e.g. for sport training
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20Movements or behaviour, e.g. gesture recognition
    • G06V40/28Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/06Systems determining position data of a target
    • G01S13/08Systems for measuring distance only
    • G01S13/32Systems for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated
    • G01S13/34Systems for measuring distance only using transmission of continuous waves, whether amplitude-, frequency-, or phase-modulated, or unmodulated using transmission of continuous, frequency-modulated waves while heterodyning the received signal, or a signal derived therefrom, with a locally-generated signal related to the contemporaneously transmitted signal
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/02Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
    • G01S13/50Systems of measurement based on relative movement of target
    • G01S13/58Velocity or trajectory determination systems; Sense-of-movement determination systems
    • G01S13/583Velocity or trajectory determination systems; Sense-of-movement determination systems using transmission of continuous unmodulated waves, amplitude-, frequency-, or phase-modulated waves and based upon the Doppler effect resulting from movement of targets
    • G01S13/584Velocity or trajectory determination systems; Sense-of-movement determination systems using transmission of continuous unmodulated waves, amplitude-, frequency-, or phase-modulated waves and based upon the Doppler effect resulting from movement of targets adapted for simultaneous range and velocity measurements

Definitions

  • the present disclosure relates to the domain of activity (e.g., gesture) sensing and recognition.
  • Gesture-based interfaces may allow users to intuitively control devices, with, for example, motions of parts of the body.
  • Applications using gesture recognition may be based on computer vision image processing techniques and may rely on cameras for capturing images of a gesture to be recognized. Some applications may benefit from gesture recognition, e.g., without having access to a camera and/or the processing resources for processing captured images.
  • the present disclosure has been designed with the foregoing in mind.
  • a signature of an activity may be obtained based on a radar processing of a signal that may be obtained in presence of a user performing the activity e.g., in the range of a radar transceiver.
  • the obtained signature of the activity may be divided in a set of sub-signatures of the activity according to a set of intervals of intensity (e.g., values).
  • reference signatures of the activities of the set of activities may be divided in reference sub signatures according to the same set of intervals of intensity.
  • the activity may be recognized by correlating the different sub signatures of the obtained signature to different reference sub signatures according to the set of intervals of intensity.
  • FIG. 1 is a diagram illustrating an example of a radar transceiver for obtaining a signature of an activity
  • FIG. 2 is a diagram illustrating an example of a TV set that may be controlled via radar-based activity recognition
  • FIG. 3 is a system diagram illustrating an example of radar sensor integrated in a TV set
  • FIG. 4 is a diagram illustrating examples of power vs. range and time radar signatures of a set of activities performed in front of a TV-set radar transceiver;
  • FIG. 5 is a diagram illustrating five examples of radar range-time signatures of respectively five activities
  • FIG. 6A is a diagram illustrating an example of a processing device for recognizing an activity in a set of activities
  • Figure 6B represents an exemplary architecture of the processing device described in Figure 6A.
  • FIG. 7 is a diagram illustrating an example of a method for recognizing an activity in a set of activities.
  • interconnected is defined to mean directly connected to or indirectly connected with through one or more intermediate components. Such intermediate components may include both hardware and software-based components.
  • interconnected is not limited to a wired interconnection and also includes wireless interconnection.
  • processor or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, read only memory (ROM) for storing software, random access memory (RAM), and non volatile storage.
  • DSP digital signal processor
  • ROM read only memory
  • RAM random access memory
  • any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
  • any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function.
  • the disclosure as defined by such claims resides in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
  • any of the following 7”, “and/or”, and “at least one of”, for example, in the cases of “A/B”, “A and/or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B).
  • such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C).
  • This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed.
  • Embodiments described herein may be related to any of methods architectures, apparatuses and systems radio frequency (RF) based activity (e.g., gesture) sensing and recognition.
  • RF radio frequency
  • Embodiments described herein may allow, for example, hands-free control of any of mobile and wearable devices.
  • Embodiments, described herein may allow to enhance (e.g., improve intuitiveness of) any user interface of any consumer electronics (CE) device with gesture sensing and recognition.
  • a user interface may allow to trigger any kind of action based on a recognized gesture.
  • embodiments described herein may be applicable to any of user posture, activity recognition and game control.
  • radio detection and ranging may be referred to herein as radar.
  • Embodiments described herein may be related to radar-based techniques for any of detection and classification (e.g., recognition).
  • detection and classification e.g., recognition
  • embodiments described herein may be applicable to human activity detection and, for example, classification (e.g., recognition).
  • embodiments described herein may be applicable to other kinds of applications.
  • ICs integrated circuits
  • Al artificial intelligence
  • ML machine learning
  • Machine learning may be used to classify human activities (such as e.g., walking, sitting, falling etc.), by extracting, for example, one feature from the acquired signal, such that only a small part of the available data may be used for activity detection and e.g., classification. This may limit the system accuracy. For example, determining the relevant features to be extracted may represent a difficult task and these features may exhibit large variations, depending on the individuals being monitored e.g., in terms of any of size, habits, and health conditions. In another example, feature extraction may be bypassed by using deep learning techniques, which may be based on large datasets collection for the model training and may suffer from overfitting and extensive computation time.
  • deep learning techniques may be based on large datasets collection for the model training and may suffer from overfitting and extensive computation time.
  • correlation-based methods may be used for radar data processing e.g., as alternative to AI/ML techniques. Compared to AI/ML techniques, correlation methods may be less complex and may involve shorter computation time. Furthermore, in contrast to ML techniques that may be based (e.g., only) on selected features for any of activity and object classification, correlation methods may be based on the entire available data, such that the result may be improved.
  • the dynamic power- range e.g., the power ratio between the highest echo signal and lowest echo signal
  • the dynamic power- range may be very large, such that the (e.g., 2D) correlation result may be dominated by the highest echo and any of subtle and low power information may be lost (e.g., ignored).
  • Embodiments described herein may allow to improve the results of radar- based (e.g., human) activity recognition (e.g., classification) based on improved (2D) correlation techniques.
  • radar- based e.g., human
  • activity recognition e.g., classification
  • 2D improved
  • a (e.g., large dynamic) range radar signature may be sub divided into a plurality of (e.g., smaller dynamic) range signatures.
  • a first set of correlations e.g., values
  • the (e.g., overall) correlation e.g., value
  • the large dynamic range radar signature may be obtained as any of a sum and an average of the smaller dynamic range correlations (e.g., values).
  • Subdividing the signature in sub-signatures and computing correlations on the sub-signatures may allow to preserve more (e.g., all) information, e.g., including small power echoes such that more information may be taken into account in the decision, improving the overall accuracy.
  • FIG. 1 is a diagram illustrating an example of a radar transceiver for obtaining a signature of an activity.
  • a radar transceiver may comprise a waveform generator 13, a transmit antenna 11 , a receive antenna 12, a mixer 14, a low pass filter 15, an analogue to digital converter (ADC) 16 and a processing module 17 (e.g., a processor).
  • the waveform generator 13 may be configured to generate a radar signal (e.g., waveform) that may be, for example, any of a frequency modulated continuous wave (FMCW), a stepped-frequency continuous wave (SFCW), and a digital code modulation (DCM). Any kind of radar waveform may be applicable to embodiments described herein.
  • FMCW frequency modulated continuous wave
  • SFCW stepped-frequency continuous wave
  • DCM digital code modulation
  • the generated radar signal may be transmitted e.g., via the transmit antenna 11 , and may reflect on (e.g., different) target(s) 10 at a (e.g., different) distance(s) from the radar transceiver.
  • an echo of the transmitted radar signal may be received by the radar transceiver via the receive antenna 12.
  • radar processing may be applied on the received echo to obtain (e.g., different) target(s) information (such as e.g., any of range, velocity, angle of arrival, ).
  • applying a radar processing on the received echo may comprise any of mixing 14 the received echo with the generated radar signal, applying low pass filtering 15, ADC 16 and further (e.g., radar-based) signal processing 17.
  • the target information that may be obtained based on a radar processing of a received echo may be used to obtain a (e.g., radar) signature, which may also be referred to herein as a heatmap.
  • a (e.g., radar) signature may comprise a (e.g., representative) characteristic of an activity that may be performed in a scene illuminated by the radar transceiver.
  • a (e.g., radar) signature may comprise a set of power vs. range and time values, e.g., as illustrated in Figure 5.
  • a (e.g., radar) signature may comprise any of a set of power vs. velocity and time values, a set of power vs. angle of arrival and time values, etc. More generally any set of values, that may be obtained from a radar processing of a received echo, and that may be representative of an activity performed by a user reflecting the echo may be applicable to embodiments described herein.
  • an activity may represent (e.g., be defined as) a sequence of movements (e.g., gestures) that a human may perform (such as e.g., any of walking, sitting, washing dishes, cooking, etc.) in a relative long time duration.
  • a gesture may represent (e.g., be defined as) a single (e.g., simple) movement that a human may perform in a relative short time duration, for example, with one part (e.g., member) of the body, such as e.g., any of a hand, a foot, the head, etc.
  • gestures may include, without limitations, any of nodding the head up and down, raising a hand, etc.
  • a gesture may correspond to a single (e.g., unitary) movement.
  • an activity may correspond to a single gesture.
  • an activity may comprise any number of gestures (e.g., unitary movements) resulting in a sequence of movements (e.g., gestures).
  • a new activity may be defined based on a new combination of (e.g., different existing) gestures.
  • embodiments are described herein for human activities, but they are not limited to human activity recognition. Any kind of activities, that may be performed, for example, by any kind of animals may be applicable to embodiments described herein.
  • a user may perform an activity in proximity (e.g., in the range) of a radar transceiver.
  • the activity may be recognized in a set of reference activities.
  • the set of reference activities (which may be referred to herein simply as “activities”) may be represented by (e.g., associated with) signatures (e.g., matrices, heatmaps).
  • each activity in the set of activity may be associated with a reference signature (e.g., matrix, heatmap).
  • the different activities may be referred to herein as (a, b, ).
  • the different reference signatures may be referred to herein as (A r , B r , ).
  • a signature may comprise, for example, power level values as a function of the distance (e.g., to the radar transceiver) and the time, e.g., P(d,t).
  • training data may be (e.g., collected) and arranged in a set of signatures (e.g., matrices, heatmaps) which may be referred to herein as (Ai, A2, ..., Bi, B2, ...), where A and Bi respectively represent two sample signatures (i) of two different activities a, b.
  • signatures e.g., matrices, heatmaps
  • Different reference signatures (A r , B r , ...) may be obtained for the different activities (a, b, ... ), for example by averaging a set of sample signatures for a same activity.
  • a r mean(A 1 ,A 2 , ... ).
  • the reference signatures (A r , B r , ...) may be decomposed (e.g., divided) into respective sets of reference sub signatures (e.g., sub-matrix) based on an intensity level (e.g., of the power values).
  • the reference sub signatures may be referred to herein as (A rl , A r2 , A r3 , ...,B rl ,B r2 , B r3 , ... ).
  • a reference sub signature (e.g., sub-matrix An) of a reference signature A r may have the same dimension (e.g., comprise the same number of values) as the reference signature A r .
  • a reference sub signature An may comprise the elements (e.g., values) of the reference signature A r with the intensity levels (e.g., only) in a predefined (e.g., i th ) range of values (which may be referred to herein as an interval of intensity). Elements of the reference signature A r outside of this range may be replaced by zeros.
  • the set of ranges e.g., intervals of intensity
  • the set of ranges may be such that all the values of the reference signature may be comprised in at least one range of values (e.g., interval of intensity).
  • the set of ranges e.g., intervals of intensity
  • a user may perform an activity in the range of the radar transceiver.
  • a signature of the performed activity may be obtained based on a radar processing of a signal obtained in presence of the user performing the activity.
  • the signature may be arranged as a set (e.g., matrix) Xi of power values.
  • the signature Xi may be divided in a set of sub-signatures (3 ⁇ 4,3 ⁇ 4,3 ⁇ 4, ) of the activity, in a same way as the reference signatures.
  • the different sub signatures of the signature Xi may comprise values of the different (e.g., non-overlapping) intervals of intensity.
  • a same set of intervals of intensity may be used to divide the reference signatures in reference sub signatures, and to divide the signature in sub signatures.
  • the different sub signatures Xij of the (e.g., performed) activity may be correlated ( corr(A ri , Xu)) to the corresponding reference sub signatures and may be any of added and averaged to obtain different correlations (e.g., matrices) between the performed activity and the different activities of the set of activities.
  • the different correlations (e.g., matrices) to the different activities may be referred to herein as CA, CB, ...
  • the correlations (e.g., matrices) of the performed activity to the respective activities (CA, CB, ...) may be given by:
  • the activity may be recognized (e.g., as a recognized activity) based on the (e.g., highest) correlation value.
  • the recognized activity may be the activity to which the correlation (e.g., matrix) of the performed activity may comprise the highest value among the correlations (e.g., matrices) of the performed activity to (e.g., all) the activities.
  • the correlation matrix with the highest value may be determined as the correlation matrix to the recognized activity.
  • the performed activity may be determined (e.g., recognized) as a given activity, on a condition that the correlation (e.g., matrix) to that given activity comprises a highest value (among the correlation values).
  • a subsequent signature of a subsequent activity may be obtained, and the same steps may be repeated to recognize the subsequent activity.
  • the (e.g., 2D) correlation of an M-by-N matrix, X, and a P-by- Q matrix, H may be represented as a matrix, C, of size M+P-1 by N+Q-1 , which elements (e.g., values) may be given by:
  • the correlation may be tested for (e.g., all) the possible shifts between the two matrices (k,l).
  • the maximum correlation’s value may be independent from the shift between the two matrices and (e.g., only) its location (in the matrix) may change.
  • different weighting may be applied to the different sub-matrices as described herein.
  • a training set of signature samples (Ai, A2, ... , Bi, B2, ..) may be obtained, wherein the signature samples (Ai, A2, ..., Bi, B2, ..) of the training set may be divided in sub signatures based on the same set of intervals of intensity as for the reference signatures (e.g., (A ⁇ ,A ⁇ ,A ⁇ , ... , B lt , B 12 , B 13 , ... )).
  • a weighting coefficient (w’) corresponding to a given interval of intensity (w’i) may be obtained by dividing a co-correlation of the reference sub-signatures to different sub signature samples of a same activity for the given interval of intensity by a cross correlation of the reference sub-signatures to sub-signature samples of different activities for the given interval of intensity.
  • the weighting e.g., coefficients
  • the weighting may be given by:
  • the weighting (e.g., coefficients) wi, W2 may be normalized to obtain a sum of one, e.g.,
  • the correlation (e.g., matrix) CA of a performed activity to a (e.g., given) activity A may be obtained by a weighted sum of the correlations of the sub-signatures to the respective sub reference signatures of that given activity for the different intervals of intensity.
  • the different sub-matrices from the obtained radar signature matrix may be correlated to its peers from the reference signatures and a weighted sum on the sub-matrices may be performed as e.g.,
  • FIG. 2 is a diagram illustrating an example of a TV set that may be controlled via radar-based activity recognition.
  • a TV set 20 may include a radar sensor (e.g., transceiver) 21 , for example, embedded in the front panel.
  • the radar sensor (e.g., transceiver) 21 may be configured to enable the remote control of a TV-set 20 e.g., without any hand-held device (such as e.g., a remote-control unit).
  • the different activities of the set of activities may be associated with (e.g., different) control operations (e.g., commands) of the TV set 20, as described herein.
  • FIG. 3 is a system diagram illustrating an example of radar sensor integrated in a TV set.
  • a radar sensor (e.g., transceiver) 31 may be integrated (e.g., included, embedded) in a TV set.
  • the radar sensor (e.g., transceiver) 31 may include at least one Tx antenna and at least one Rx antenna.
  • the radar sensor (e.g., transceiver) 31 may (e.g., continuously) obtain (e.g., measure instantaneous) radar signatures of an activity performed by a user.
  • the radar signature may comprise (e.g., be based on) a range profile (RP) (e.g., a set of power values for different distances to the radar sensor).
  • RP range profile
  • the radar signature(s) may be sent to a processor (e.g., a processing unit (PU)) 32, that may be equipped with an internal or external memory 34.
  • a processor e.g., a processing unit (PU) 32
  • the PU 32 and the memory 34 may be any of the same as the TV’s ones and a separate one.
  • the PU and the TV processor 33 may be any of a same and two different processors.
  • the PU 32 may store the RPs in the memory 34.
  • (e.g., reference) radar signatures e.g., heatmaps, sets of power values for different values of time and distance
  • Figure 4 is a diagram illustrating examples of power vs. range and time radar signatures of a set of activities (e.g., gestures) performed in front of a TV-set radar transceiver e.g., in the 6 GHz band.
  • the set of activities may comprise any of the following activities: sitting down 41 (e.g., going towards and sitting in a sofa), standing up 42 (e.g., standing up and leaving the sofa), sweep left 43 (e.g., sweeping hand from right to left, e.g., to scroll through the programs), sweep right 44 (e.g., sweeping hand from left e.g., to right to scroll through the programs in the reverse direction), hand up 45 (e.g., moving hand up e.g., to increase the volume, hand down 46 (e.g., moving hand down e.g., to turn down the volume).
  • sitting down 41 e.g., going towards and sitting in a sofa
  • standing up 42 e.g., standing up and leaving the
  • the (e.g., different) activities may be associated with (e.g., different) control operations of the TV.
  • the (reference signature of the) sitting down activity 41 may be associated with a power on control operation (e.g., if the TV set power is off when the sitting down activity
  • any of the (reference signatures of the) sweep left 43 and sweep right 44 activity may be associated with a channel change control operation in one or the other direction.
  • any of the (reference signatures of the) hand up 45 and hand down 46 activity may be associated with any of a volume increase and a volume decrease control operation.
  • Any other (reference signature of any) activity e.g., gesture
  • any kind of control operation may be applicable to embodiments described herein.
  • the set of activities 41 , 42, 43, 44, 45, 46 may be associated with corresponding reference signatures 410, 420, 430, 440, 450, 460 as illustrated in Figure 4.
  • the reference signatures 410, 420, 430, 440, 450, 460 show different levels of grey color for different power levels for different values of time (vertical axis) and distance (horizontal axis).
  • the set of reference signatures may be any of pre-configured in the TV set and downloaded from e.g., a server via a network connection.
  • a (e.g., fixed, pre determined) set of gestures e.g., activities
  • the set of activities may be configured in the TV set, for example, during an installation phase.
  • a user may be requested, e.g., via a user interface, to perform an intuitive gesture and to associate the intuitive gesture with a control operation.
  • the user may choose to raise his hand to increase the device volume, move his hand down to decrease the device volume, sweep his hand from right to left to change to the next channel, sweep his hand from left to right to change to the previous channel and so on.
  • the user may perform a gesture and enter information via the user interface for associating the performed gesture with a control operation of the TV.
  • the user may be requested to repeat the same intuitive gesture for a number of times which (may be referred to herein as N, N being any integer value), such that the TV set may acquire a training set of signature samples for the different activities (and associated control operations).
  • N may be referred to herein as any integer value
  • the TV set may acquire a training set of signature samples for the different activities (and associated control operations).
  • the PU may generate the reference signatures of the reference activities.
  • weighted coefficients may be computed by the PU based on the training set.
  • the TV set may be controlled based on recognized gestures as described herein. For example, after a gesture may have been recognized, the control command associated with the recognized gesture may be executed by the TV set. For example, the control command associated with the recognized gesture may be sent to the TV processor for control command execution.
  • the user presence may be detected by the radar transceiver.
  • the TV may be switched on and a program menu may be displayed, through which the user may navigate based on (e.g., configured) intuitive hand gestures.
  • intuitive hand gestures For example, (e.g., at the end), in a case where the user stands up and leaves the room, the TV may be automatically switched off.
  • any of the TV switch on and off may also be performed (e.g., operated, forced) by a recognition of specific gestures, that may have been learned by the TV set during the training phase (e.g., or configured via another method).
  • any of “entering the room” and “leaving the room” types of activities may be processed (e.g., detected, classified, recognized) in the same manner as (e.g. simple) gestures may be processed (e.g., detected, classified, recognized) but in a different (e.g., larger) time scale to better focus on the relevant event (such as e.g., six seconds for processing an activity event and two seconds for a (e.g., simple) gesture event.
  • a signature of an activity may be obtained (e.g., acquired) over any type of duration. Any duration for obtaining a signature of an activity may be applicable to embodiments described herein.
  • a reference signature may be associated with (e.g., have been acquired in) a given duration, and a signature sample (to be evaluated against the reference signature) may be acquired over at least the same duration.
  • the reference signature may be automatically updated (e.g., enriched, improved) with new signatures (e.g., samples) of the gestures, for example, as a background task.
  • new signatures e.g., samples
  • the reference signature may be automatically updated in a case where there is no activity in the room (e.g. in a case where background is detected).
  • the radar transceiver may be packaged as standalone module such as e.g., an accessory, (e.g., separated) device (e.g., not integrated in the TV).
  • the radar transceiver may be, for example, connected to the TV (using any of wired and wireless connection) for sending control commands to the TV.
  • FIG. 5 is a diagram illustrating five examples of radar range-time signatures of respectively five activities.
  • the radar range-time signatures 501 , 502, 503, 504, 505 represented in Figure 5 may have been obtained based on the AWR 1243 circuit from Texas Instrument, using Tl reference design.
  • a first 501 , a second 502, a third 503, a fourth 504 and a fifth 505 signature may be obtained from the radar transceiver in presence of a user respectively performing a first, a second, a third, a forth and a fifth activity in the range of the radar transceiver.
  • the first activity which may be referred to herein as a background activity may correspond to no activity (e.g., no user) in the range of the radar.
  • the second activity which may be referred to herein as a walking activity, may correspond to a user walking into the room and crossing below the radar in any of the two directions.
  • the third activity which may be referred to herein as a sitting activity may correspond to a user walking into the room and sitting below (e.g., in front of) the radar.
  • the fourth and the fifth activities which may be referred to herein as respectively a knee falling activity and a face falling activity may correspond to a user walking into the room and falling below (e.g., in front of) the radar respectively on his knee and on his face.
  • the radar range-time signatures 501 , 502, 503, 504, 505 of respectively five activities may comprise a set of power value for different time value (e.g., vertical axis) and different range values (e.g., horizontal axis).
  • Other types of signatures such as e.g., without limitation, range-doppler, micro-doppler, spectrogram, etc. may also be applicable to embodiments described herein.
  • a signature may be obtained based on a (e.g., sliding) time window.
  • range-power may be measured and added to a set of value (e.g., matrix) fora duration of T.
  • the duration may be of seven seconds.
  • the first line of the matrix e.g., corresponding to 1s
  • the instantaneous measured range-power value may be added to end of this matrix (e.g., as in a circular buffer).
  • the five signatures 501 , 502, 503, 504, 505 may appear visually different such that they may be detected and classified based on a signal processing method.
  • the transmit (Tx) and receive (Rx) antennas’ coupling e.g., self-reflection shown by the vertical lines from 0m to 0.3m
  • the reflection from the ground static clutter shown by the vertical lines from 2.5m to 3.8m
  • Tx and Rx antennas may be the dominant reflections.
  • these reflections may be removed by performing static clutter removal (by e.g., setting the points that may not change between two consecutive measurements beyond a (e.g., threshold) value (e.g., 5%), to a (e.g., minimum) value, to e.g., OdB.
  • background subtraction may be performed on (e.g., any of reference and sample) signatures, by e.g., performing a measurement e.g., without any activity in the room and by subtracting the background signature from other (e.g., any of reference and sample) signatures.
  • Figure 5 further shows five radar range-time signatures 511 , 512, 513, 514, 515 in the bottom that correspond to the radar range-time signatures 501 , 502, 503, 504, 505 shown at the top, after background subtraction. It may be observed that the self-reflection and the static clutter, including the ground effect, may be reduced (e.g., minimized). Background subtraction may allow to better differentiate the radar range-time signatures.
  • Table 1 is a confusion matrix that may be obtained by processing the radar range-time signatures illustrated in Figure 5 based on a regular correlation method.
  • regular correlation method it is meant performing a (e.g., 2D) correlation of the signatures in a single correlation without subdividing in submatrices before performing multiple correlations between submatrices as described herein.
  • the average accuracy (calculated as the average value of the confusion matrix’s diagonal) is equal to 64.66%. This may be due to the large dynamic of power values of the signatures (e.g., with relative power values range from OdB to 30dB, e.g., 1 to 1000)
  • Table 2 is a confusion matrix that may be obtained by processing the radar range-time signatures illustrated in Figure 5 based on the correlation as described herein (e.g., based on subdividing the signatures in sub signatures and correlating sub-signatures to reference sub-signatures).
  • Embodiments described herein may allow to improve the classification result by enabling each part of the signature to contribute to the correlation product. Using the proposed method, each part of the matrix may contribute to the correlation product. As shown in Table 2, applying the correlation method described herein may allow to provide 99.23% true detection.
  • the signature may be divided according to the following power ranges (e.g., intervals of intensity): [0, 1], [1 , 10], [10, 100], [100, 1000] and [1000, ⁇ ].
  • Table 3 is a confusion matrix that may be obtained by processing the radar range-time signatures illustrated in Figure 5 based on the weighting correlation variant.
  • FIG. 6A is a diagram illustrating an example of a processing device 6A for recognizing an activity in a set of activities.
  • the processing device 6A may comprise a radar transceiver 43 that may be configured to process a signal that may be obtained in presence of a user performing an activity in the range of the radar transceiver 43.
  • the radar transceiver 43 may be coupled to a processing module 45, configured to obtain a signature of the activity based on a radar processing of the obtained signal.
  • the processing module 45 may be further configured to divide the signature of the activity to obtain a set of sub-signatures of the activity according to a set of intervals of intensity, wherein different sub-signatures of the activity may correspond to different intervals of intensity, and wherein reference signatures of the activities in the set of activities may be divided in respective sets of reference sub signatures according to the (e.g., same) set of intervals of intensity, wherein a reference sub signature of an interval of intensity may correspond to (e.g., be associated with) a sub-reference signature of the same interval of intensity.
  • the processing module 45 may be further configured to recognize (e.g., classify) the activity in the set of activities based on a processing (e.g., a correlation) of the sub-signatures of the activity with (e.g., to) corresponding reference sub signatures of the activities of the set of activities according to the set of intervals of intensity.
  • a processing e.g., a correlation
  • the processing module 45 may be coupled to an optional interface module 47.
  • the interface module 47 may be a network interface.
  • the network interface may be any of wired and wireless network interface, any of a local and wide area network interface.
  • the interface module 47 may be a user interface, running (e.g., and displayed) locally on the processing device 6A.
  • the user interface may be running on another device, communicating with the processing device 6A via the network interface.
  • the user interface may allow the processing device 6A to interact with a user, for example, by associating a control operation with a specific activity and by triggering the control operation based on the recognized activity.
  • control operations may include any of unlocking a door, unlocking a device, lighting a lamp, switching on/off a device, increasing volume, decreasing volume, changing channel...
  • the user interface may propose (e.g., any of display an image displaying, play an audio including, ...) a set of options to a user, wherein an option may be associated with a specific activity.
  • the user interface may be configured to validate (e.g., select) the option corresponding to the recognized activity.
  • the processing device 6A may be a (e.g., basic) sensing device that may be any of separable from and couplable to another (e.g., more complex) processing device (such as e.g., a TV). Coupling / separation between both devices may be accomplished via any of a bus interface, a network interface, with e.g., connectors.
  • FIG. 6B represents an exemplary architecture of the processing device 6A described herein.
  • the processing device 6B may comprise one or more processor(s) 610, which may be, for example, any of a CPU, a GPU a DSP (English acronym of Digital Signal Processor), along with internal memory 620 (e.g. any of RAM, ROM, EPROM).
  • the processing device 6A may comprise any number of Input/Output interface(s) 630 adapted to send output information and/or to allow a user to enter commands and/or data (e.g. any of a keyboard, a mouse, a touchpad, a webcam, a display), and/or to send / receive data over a network interface; and a power source 640 which may be external to the processing device 6A.
  • the processing device 6A may further comprise a computer program stored in the memory 620.
  • the computer program may comprise instructions which, when executed by the processing device 6A, in particular by the processor(s) 610, make the processing device 6A carrying out the processing method described with reference to figure 7.
  • the computer program may be stored externally to the processing device 6A on a non-transitory digital data support, e.g. on an external storage medium such as any of a SD Card, HDD, CD-ROM, DVD, a read-only and/or DVD drive, a DVD Read/Write drive, all known in the art.
  • the processing device 6A may comprise an interface to read the computer program. Further, the processing device 6A may access any number of Universal Serial Bus (USB)-type storage devices (e.g., “memory sticks.”) through corresponding USB ports (not shown).
  • USB Universal Serial Bus
  • the processing device 6A may be any of a server, a desktop computer, a laptop computer, a networking device, a TV set, a tablet, a smartphone, a phablet, a set-top box, an internet gateway, a game console, a head mounted device, an internet of things (loT) device, a wearable device, a doorbell system, a locker, ...
  • a server a desktop computer, a laptop computer, a networking device, a TV set, a tablet, a smartphone, a phablet, a set-top box, an internet gateway, a game console, a head mounted device, an internet of things (loT) device, a wearable device, a doorbell system, a locker, ...
  • loT internet of things
  • Figure 7 is a diagram illustrating an example of a method for recognizing an activity in a set of activities.
  • the method may be implemented in a processing device that may be coupled to (e.g., integrating) a radar transceiver.
  • a signature of the activity may be obtained based on a radar processing of a signal that may be obtained in presence of a user performing the activity e.g., in the range of the radar transceiver.
  • the signature of the activity may be divided in (e.g., to obtain) a set of sub-signatures of the activity according to a set of intervals of intensity (e.g., power values).
  • a set of intervals of intensity e.g., power values.
  • different sub-signatures of the activity may correspond to different intervals of intensity of the signature.
  • each sub-signature of the activity may correspond to a different interval of intensity of the signature.
  • reference signatures of the (e.g., set of) activities in the set of activities may be divided in respective sets of reference sub signatures e.g., according to the same intervals of intensity.
  • different (e.g., each of the) reference sub signatures of a set may correspond to the (e.g., same) different intervals of intensity.
  • a same set of intervals of intensity may be used to divide the signature in sub-signatures, and to divide a (e.g., each) reference signature in a set of reference sub signatures, such that for a same interval of intensity a sub-signature of a signature may correspond to (e.g., be associated with) a reference sub signature of a reference signature.
  • the activity may be recognized in the set of activities (e.g., as a recognized activity) based on a processing of the sub signatures of the activity with corresponding reference sub signatures of the activities of the set of activities according to the set of intervals of intensity.
  • the processing may, for example, comprise correlating the sub signatures of the activity with the corresponding reference sub signatures of the (e.g., set of) activities according to the set of intervals of intensity.
  • the processing of the sub-signatures of the activity with the corresponding reference sub signatures may comprise obtaining correlation matrices of the activity to respective other activities.
  • a correlation matrix of the activity to a given activity may be obtained based on a sum of correlation sub-matrices for the different intervals of intensity.
  • a correlation sub matrix (e.g., for an interval of intensity) may be obtained by correlating the sub-signature of the activity to the corresponding reference sub signature of the given activity (e.g., for the same interval of intensity).
  • the activity may be recognized in the set of activities based on a maximum value of the correlation matrices.
  • the activity may be recognized as a recognized activity in the set of activities on a condition that the correlation matrix of the activity to the recognized activity comprises the maximum value of the correlation matrices.
  • the activity may be recognized as the recognized activity in the set of activities on a further condition that the maximum value is above a predetermined value.
  • the sum may be weighted with different weighting coefficients corresponding to the different intervals of intensity.
  • a weighting coefficient corresponding to a given interval of intensity may be obtained by dividing a co-correlation of the reference sub signatures to different sub-signature samples of a same activity for the given interval of intensity by a cross-correlation of the reference sub-signatures to sub signature samples of different activities for the given interval of intensity.
  • the weighting coefficients may be normalized.
  • the activity may comprise a (e.g., single) gesture.
  • the activity may comprise any number (e.g., a sequence) of gestures.
  • the reference signatures of the (e.g., set of) activities may be obtained based on a training set of signature samples of the (e.g., set of) activities.
  • the reference signatures of the (e.g., set of) activities may be any of pre-configured and downloaded from a server.
  • the reference signatures of the activities may be any of pre configured and downloaded from a server.
  • different reference signatures may be associated with different control operations of the processing device.
  • the processing device may be any of a TV set, a set-top-box, a media player and a game console, that may be controllable by the set of activities.
  • control operation may be any of a switch on, a switch off, a volume increase, a volume decrease, and a channel change control operation.
  • present embodiments may be employed in any combination or sub-combination.
  • present principles are not limited to the described variants, and any arrangement of variants and embodiments may be used.
  • embodiments described herein are not limited to the gestures and activities described herein and any other types of gestures/activities (e.g., involving any body parts) may be compatible with the embodiments described herein.
  • Any characteristic, variant or embodiment described for a method is compatible with an apparatus device comprising means for processing the disclosed method, with a device comprising a processor configured to process the disclosed method, with a computer program product comprising program code instructions and with a non-transitory computer-readable storage medium storing program instructions.
  • non-transitory computer-readable storage media include, but are not limited to, a read only memory (ROM), random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
  • ROM read only memory
  • RAM random access memory
  • register cache memory
  • semiconductor memory devices magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
  • processing platforms, computing systems, controllers, and other devices containing processors are noted. These devices may contain at least one Central Processing Unit (“CPU”) and memory.
  • CPU Central Processing Unit
  • FIG. 1 A block diagram illustrating an exemplary computing system
  • FIG. 1 A block diagram illustrating an exemplary computing system
  • FIG. 1 A block diagram illustrating an exemplary computing system
  • FIG. 1 A block diagram illustrating an exemplary computing system
  • FIG. 1 A block diagram illustrating an exemplary computing system
  • memory may contain at least one Central Processing Unit (“CPU”) and memory.
  • CPU Central Processing Unit
  • Such acts and operations or instructions may be referred to as being "executed,” “computer executed” or "CPU executed.”
  • an electrical system represents data bits that can cause a resulting transformation or reduction of the electrical signals and the maintenance of data bits at memory locations in a memory system to thereby reconfigure or otherwise alter the CPU's operation, as well as other processing of signals.
  • the memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties corresponding to or representative of the data bits. It should be understood that the representative embodiments are not limited to the above-mentioned platforms or CPUs and that other platforms and CPUs may support the provided methods.
  • the data bits may also be maintained on a computer readable medium including magnetic disks, optical disks, and any other volatile (e.g., Random Access Memory (“RAM”)) or non-volatile (e.g., Read-Only Memory (“ROM”)) mass storage system readable by the CPU.
  • RAM Random Access Memory
  • ROM Read-Only Memory
  • the computer readable medium may include cooperating or interconnected computer readable medium, which exist exclusively on the processing system or are distributed among multiple interconnected processing systems that may be local or remote to the processing system. It is understood that the representative embodiments are not limited to the above- mentioned memories and that other platforms and memories may support the described methods.
  • any of the operations, processes, etc. described herein may be implemented as computer-readable instructions stored on a computer-readable medium.
  • the computer-readable instructions may be executed by a processor of a mobile unit, a network element, and/or any other computing device.
  • the use of hardware or software is generally (e.g., but not always, in that in certain contexts the choice between hardware and software may become significant) a design choice representing cost vs. efficiency tradeoffs.
  • Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs); Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (1C), and/or a state machine.
  • DSP digital signal processor
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Standard Products
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • FPGAs Field Programmable Gate Arrays
  • DSPs digital signal processors
  • ASICs Application Specific Integrated Circuits
  • FPGAs Field Programmable Gate Arrays
  • DSPs digital signal processors
  • FIG. 1 ASICs
  • FIG. 1 ASICs
  • FIG. 1 ASICs
  • FIG. 1 ASICs
  • FIG. 1 ASICs
  • FIG. 1 ASICs
  • FIG. 1 Application Specific Integrated Circuits
  • FPGAs Field Programmable Gate Arrays
  • DSPs digital signal processors
  • a signal bearing medium examples include, but are not limited to, the following: a recordable type medium such as a floppy disk, a hard disk drive, a CD, a DVD, a digital tape, a computer memory, etc., and a transmission type medium such as a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.).
  • a signal bearing medium include, but are not limited to, the following: a recordable type medium such as a floppy disk, a hard disk drive, a CD, a DVD, a digital tape, a computer memory, etc.
  • a transmission type medium such as a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.).
  • any two components so associated may also be viewed as being “operably connected”, or “operably coupled”, to each other to achieve the desired functionality, and any two components capable of being so associated may also be viewed as being “operably couplable” to each other to achieve the desired functionality.
  • operably couplable include but are not limited to physically mateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components.
  • the phrase “A or B” will be understood to include the possibilities of “A” or “B” or “A and B.”
  • the terms “any of” followed by a listing of a plurality of items and/or a plurality of categories of items, as used herein, are intended to include “any of,” “any combination of,” “any multiple of,” and/or “any combination of multiples of” the items and/or the categories of items, individually or in conjunction with other items and/or other categories of items.
  • the term “set” or “group” is intended to include any number of items, including zero.
  • the term “number” is intended to include any number, including zero.

Landscapes

  • Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Social Psychology (AREA)
  • Human Computer Interaction (AREA)
  • Psychiatry (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

A signature of an activity may be obtained based on a radar processing of a signal that may be obtained in presence of a user performing the activity e.g., in the range of a radar transceiver. For example, the obtained signature of the activity may be divided in a set of sub-signatures of the activity according to a set of intervals of intensity (e.g., values). For example, reference signatures of the activities of the set of activities may be divided in reference sub signatures according to the same set of intervals of intensity. For example, the activity may be recognized by correlating the different sub signatures of the obtained signature to different reference sub signatures according to the set of intervals of intensity.

Description

METHODS, ARCHITECTURES, APPARATUSES AND SYSTEMS DIRECTED TO RECOGNIZING AN ACTIVITY IN A SET OF ACTIVITIES
1. CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims the benefit of European Patent Application No. 21305889.4, filed June 29, 2021 , which is incorporated herein by reference in its entirety.
2. TECHNICAL FIELD
The present disclosure relates to the domain of activity (e.g., gesture) sensing and recognition.
3. BACKGROUND ART
Gesture-based interfaces may allow users to intuitively control devices, with, for example, motions of parts of the body. Applications using gesture recognition may be based on computer vision image processing techniques and may rely on cameras for capturing images of a gesture to be recognized. Some applications may benefit from gesture recognition, e.g., without having access to a camera and/or the processing resources for processing captured images. The present disclosure has been designed with the foregoing in mind.
4. SUMMARY
Methods, architectures, apparatuses, and systems directed to recognizing an activity in a set of activities are described herein. For example, a signature of an activity may be obtained based on a radar processing of a signal that may be obtained in presence of a user performing the activity e.g., in the range of a radar transceiver. For example, the obtained signature of the activity may be divided in a set of sub-signatures of the activity according to a set of intervals of intensity (e.g., values). For example, reference signatures of the activities of the set of activities may be divided in reference sub signatures according to the same set of intervals of intensity. For example, the activity may be recognized by correlating the different sub signatures of the obtained signature to different reference sub signatures according to the set of intervals of intensity.
5. BRIEF DESCRIPTION OF THE DRAWINGS
- Figure 1 is a diagram illustrating an example of a radar transceiver for obtaining a signature of an activity;
- Figure 2 is a diagram illustrating an example of a TV set that may be controlled via radar-based activity recognition;
- Figure 3 is a system diagram illustrating an example of radar sensor integrated in a TV set;
- Figure 4 is a diagram illustrating examples of power vs. range and time radar signatures of a set of activities performed in front of a TV-set radar transceiver;
- Figure 5 is a diagram illustrating five examples of radar range-time signatures of respectively five activities;
- Figure 6A is a diagram illustrating an example of a processing device for recognizing an activity in a set of activities;
- Figure 6B represents an exemplary architecture of the processing device described in Figure 6A; and
- Figure 7 is a diagram illustrating an example of a method for recognizing an activity in a set of activities.
It should be understood that the drawing(s) are for purposes of illustrating the concepts of the disclosure and are not necessarily the only possible configuration for illustrating the disclosure.
6. DESCRIPTION OF EMBODIMENTS
It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input/output interfaces. Herein, the term " interconnected " is defined to mean directly connected to or indirectly connected with through one or more intermediate components. Such intermediate components may include both hardware and software-based components. The term “interconnected” is not limited to a wired interconnection and also includes wireless interconnection.
All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor to furthering the art and are to be construed as being without limitation to such specifically recited examples and conditions.
Moreover, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudocode, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, read only memory (ROM) for storing software, random access memory (RAM), and non volatile storage.
Other hardware, conventional and/or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
In the claims hereof, any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements that performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. The disclosure as defined by such claims resides in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. It is thus regarded that any means that can provide those functionalities are equivalent to those shown herein.
It is to be appreciated that the use of any of the following 7”, “and/or”, and “at least one of”, for example, in the cases of “A/B”, “A and/or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and/or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended, as is clear to one of ordinary skill in this and related arts, for as many items as are listed.
Overview
Embodiments described herein may be related to any of methods architectures, apparatuses and systems radio frequency (RF) based activity (e.g., gesture) sensing and recognition.
Embodiments described herein may allow, for example, hands-free control of any of mobile and wearable devices. Embodiments, described herein may allow to enhance (e.g., improve intuitiveness of) any user interface of any consumer electronics (CE) device with gesture sensing and recognition. For example, a user interface may allow to trigger any kind of action based on a recognized gesture. More generally, embodiments described herein may be applicable to any of user posture, activity recognition and game control. In embodiments described herein, radio detection and ranging may be referred to herein as radar.
Embodiments described herein may be related to radar-based techniques for any of detection and classification (e.g., recognition). For example, embodiments described herein may be applicable to human activity detection and, for example, classification (e.g., recognition). For example, embodiments described herein may be applicable to other kinds of applications.
The recent advances in integrated circuits (ICs) and the growth of processing power combined with any of artificial intelligence (Al) and machine learning (ML) techniques may allow to apply radar processing techniques for developing new radar applications, e.g., including any of user sensing, user localization, gesture recognition, vital signs sensing (breathing, heartbeat), etc...
Machine learning may be used to classify human activities (such as e.g., walking, sitting, falling etc.), by extracting, for example, one feature from the acquired signal, such that only a small part of the available data may be used for activity detection and e.g., classification. This may limit the system accuracy. For example, determining the relevant features to be extracted may represent a difficult task and these features may exhibit large variations, depending on the individuals being monitored e.g., in terms of any of size, habits, and health conditions. In another example, feature extraction may be bypassed by using deep learning techniques, which may be based on large datasets collection for the model training and may suffer from overfitting and extensive computation time. In yet another example, correlation-based methods may be used for radar data processing e.g., as alternative to AI/ML techniques. Compared to AI/ML techniques, correlation methods may be less complex and may involve shorter computation time. Furthermore, in contrast to ML techniques that may be based (e.g., only) on selected features for any of activity and object classification, correlation methods may be based on the entire available data, such that the result may be improved.
For example, for radar sensing applications (in particular those operating indoors (with e.g., a short range)) the dynamic power- range (e.g., the power ratio between the highest echo signal and lowest echo signal) may be very large, such that the (e.g., 2D) correlation result may be dominated by the highest echo and any of subtle and low power information may be lost (e.g., ignored).
Embodiments described herein may allow to improve the results of radar- based (e.g., human) activity recognition (e.g., classification) based on improved (2D) correlation techniques.
For example, a (e.g., large dynamic) range radar signature may be sub divided into a plurality of (e.g., smaller dynamic) range signatures. For example, a first set of correlations (e.g., values) may be obtained (e.g., computed) for the different (e.g., each) smaller dynamic range radar signatures. For example, the (e.g., overall) correlation (e.g., value) of the large dynamic range radar signature may be obtained as any of a sum and an average of the smaller dynamic range correlations (e.g., values). Subdividing the signature in sub-signatures and computing correlations on the sub-signatures may allow to preserve more (e.g., all) information, e.g., including small power echoes such that more information may be taken into account in the decision, improving the overall accuracy.
Example of a Radar Transceiver
Figure 1 is a diagram illustrating an example of a radar transceiver for obtaining a signature of an activity. For example, a radar transceiver may comprise a waveform generator 13, a transmit antenna 11 , a receive antenna 12, a mixer 14, a low pass filter 15, an analogue to digital converter (ADC) 16 and a processing module 17 (e.g., a processor). The waveform generator 13 may be configured to generate a radar signal (e.g., waveform) that may be, for example, any of a frequency modulated continuous wave (FMCW), a stepped-frequency continuous wave (SFCW), and a digital code modulation (DCM). Any kind of radar waveform may be applicable to embodiments described herein. The generated radar signal may be transmitted e.g., via the transmit antenna 11 , and may reflect on (e.g., different) target(s) 10 at a (e.g., different) distance(s) from the radar transceiver. For example, an echo of the transmitted radar signal may be received by the radar transceiver via the receive antenna 12. For example, radar processing may be applied on the received echo to obtain (e.g., different) target(s) information (such as e.g., any of range, velocity, angle of arrival, ...). For example, applying a radar processing on the received echo may comprise any of mixing 14 the received echo with the generated radar signal, applying low pass filtering 15, ADC 16 and further (e.g., radar-based) signal processing 17.
Example of a Radar Signature
For example, the target information that may be obtained based on a radar processing of a received echo, may be used to obtain a (e.g., radar) signature, which may also be referred to herein as a heatmap. A (e.g., radar) signature may comprise a (e.g., representative) characteristic of an activity that may be performed in a scene illuminated by the radar transceiver. For example, a (e.g., radar) signature may comprise a set of power vs. range and time values, e.g., as illustrated in Figure 5. In other examples, a (e.g., radar) signature may comprise any of a set of power vs. velocity and time values, a set of power vs. angle of arrival and time values, etc. More generally any set of values, that may be obtained from a radar processing of a received echo, and that may be representative of an activity performed by a user reflecting the echo may be applicable to embodiments described herein.
Activity and Gesture Examples
For example, an activity may represent (e.g., be defined as) a sequence of movements (e.g., gestures) that a human may perform (such as e.g., any of walking, sitting, washing dishes, cooking, etc.) in a relative long time duration. For example, a gesture may represent (e.g., be defined as) a single (e.g., simple) movement that a human may perform in a relative short time duration, for example, with one part (e.g., member) of the body, such as e.g., any of a hand, a foot, the head, etc. Examples of gestures may include, without limitations, any of nodding the head up and down, raising a hand, etc. For example, a gesture may correspond to a single (e.g., unitary) movement. In a first example, an activity may correspond to a single gesture. In another example, an activity may comprise any number of gestures (e.g., unitary movements) resulting in a sequence of movements (e.g., gestures). For example, a new activity may be defined based on a new combination of (e.g., different existing) gestures. For the sake of clarity, embodiments are described herein for human activities, but they are not limited to human activity recognition. Any kind of activities, that may be performed, for example, by any kind of animals may be applicable to embodiments described herein.
Example of a Method for Recognizing an Activity
For example, a user may perform an activity in proximity (e.g., in the range) of a radar transceiver. The activity may be recognized in a set of reference activities. The set of reference activities (which may be referred to herein simply as “activities”) may be represented by (e.g., associated with) signatures (e.g., matrices, heatmaps). For example, each activity in the set of activity may be associated with a reference signature (e.g., matrix, heatmap). The different activities may be referred to herein as (a, b, ...). The different reference signatures may be referred to herein as (Ar, Br, ...). A signature may comprise, for example, power level values as a function of the distance (e.g., to the radar transceiver) and the time, e.g., P(d,t).
For example, training data may be (e.g., collected) and arranged in a set of signatures (e.g., matrices, heatmaps) which may be referred to herein as (Ai, A2, ..., Bi, B2, ...), where A and Bi respectively represent two sample signatures (i) of two different activities a, b.
Different reference signatures (Ar, Br, ...) may be obtained for the different activities (a, b, ... ), for example by averaging a set of sample signatures for a same activity. For example, Ar = mean(A1,A2, ... ).
For example, the reference signatures (Ar, Br, ...) may be decomposed (e.g., divided) into respective sets of reference sub signatures (e.g., sub-matrix) based on an intensity level (e.g., of the power values). For example, the reference sub signatures may be referred to herein as (Arl, Ar2, Ar3, ...,Brl,Br2, Br3, ... ). For example, a reference sub signature (e.g., sub-matrix An) of a reference signature Ar may have the same dimension (e.g., comprise the same number of values) as the reference signature Ar. For example, a reference sub signature An may comprise the elements (e.g., values) of the reference signature Ar with the intensity levels (e.g., only) in a predefined (e.g., ith) range of values (which may be referred to herein as an interval of intensity). Elements of the reference signature Ar outside of this range may be replaced by zeros. The set of ranges (e.g., intervals of intensity) may be such that all the values of the reference signature may be comprised in at least one range of values (e.g., interval of intensity). For example, the set of ranges (e.g., intervals of intensity) may not overlap between themselves. For example, the reference signature Ar of an activity may be obtained by a sum of the divided reference sub signatures An of that activity (e.g., Ar = Arl + Ar2 +
Figure imgf000010_0001
For example, a user may perform an activity in the range of the radar transceiver. For example, a signature of the performed activity may be obtained based on a radar processing of a signal obtained in presence of the user performing the activity. For example, the signature may be arranged as a set (e.g., matrix) Xi of power values. For example, the signature Xi may be divided in a set of sub-signatures (¾,¾,¾, ) of the activity, in a same way as the reference signatures. For example, the different sub signatures of the signature Xi may comprise values of the different (e.g., non-overlapping) intervals of intensity. For example, a same set of intervals of intensity may be used to divide the reference signatures in reference sub signatures, and to divide the signature in sub signatures.
For example, the different sub signatures Xij of the (e.g., performed) activity may be correlated ( corr(Ari , Xu)) to the corresponding reference sub signatures and may be any of added and averaged to obtain different correlations (e.g., matrices) between the performed activity and the different activities of the set of activities. The different correlations (e.g., matrices) to the different activities may be referred to herein as CA, CB, ... More formally, the correlations (e.g., matrices) of the performed activity to the respective activities (CA, CB, ...) may be given by:
Figure imgf000010_0002
For example, the activity may be recognized (e.g., as a recognized activity) based on the (e.g., highest) correlation value. For example, the recognized activity may be the activity to which the correlation (e.g., matrix) of the performed activity may comprise the highest value among the correlations (e.g., matrices) of the performed activity to (e.g., all) the activities. In other words, among the correlations (e.g., matrices) CA, CB, of the performed activity to the different activities, the correlation matrix with the highest value may be determined as the correlation matrix to the recognized activity.
For example, the performed activity may be determined (e.g., recognized) as a given activity, on a condition that the correlation (e.g., matrix) to that given activity comprises a highest value (among the correlation values). For example, the performed activity may be determined (e.g., recognized) as a given activity on a further condition that the highest correlation value is above a predetermined value (e.g., a threshold). More formally: if CA = max (CA, CB, ... ) & CA > threshold ® the event may be A if CA = max (CA, CB, ... ) & CA < threshold ® the event may be unknown
For example, a subsequent signature of a subsequent activity may be obtained, and the same steps may be repeated to recognize the subsequent activity.
For example, the (e.g., 2D) correlation of an M-by-N matrix, X, and a P-by- Q matrix, H, may be represented as a matrix, C, of size M+P-1 by N+Q-1 , which elements (e.g., values) may be given by:
Figure imgf000011_0001
For example, it may be seen from equation (1), that the correlation may be tested for (e.g., all) the possible shifts between the two matrices (k,l). For example, the maximum correlation’s value may be independent from the shift between the two matrices and (e.g., only) its location (in the matrix) may change.
For example, in a variant, different weighting (e.g., coefficients) may be applied to the different sub-matrices as described herein. For example, a training set of signature samples (Ai, A2, ... , Bi, B2, ..) may be obtained, wherein the signature samples (Ai, A2, ..., Bi, B2, ..) of the training set may be divided in sub signatures based on the same set of intervals of intensity as for the reference signatures (e.g., (A^,A^,A^, ... , Blt, B12, B13, ... )). For example, a weighting coefficient (w’) corresponding to a given interval of intensity (w’i) may be obtained by dividing a co-correlation of the reference sub-signatures to different sub signature samples of a same activity for the given interval of intensity by a cross correlation of the reference sub-signatures to sub-signature samples of different activities for the given interval of intensity. More formally, the weighting (e.g., coefficients) may be given by:
Figure imgf000012_0001
For example, the weighting (e.g., coefficients) wi, W2 may be normalized to obtain a sum of one, e.g.,
Figure imgf000012_0002
For example, in the weighting correlation variant, the correlation (e.g., matrix) CA of a performed activity to a (e.g., given) activity A may be obtained by a weighted sum of the correlations of the sub-signatures to the respective sub reference signatures of that given activity for the different intervals of intensity. More formally, the different sub-matrices from the obtained radar signature matrix may be correlated to its peers from the reference signatures and a weighted sum on the sub-matrices may be performed as e.g.,
Figure imgf000012_0003
Example of TV Control Based on Activity Recognition
Figure 2 is a diagram illustrating an example of a TV set that may be controlled via radar-based activity recognition. For example, a TV set 20 may include a radar sensor (e.g., transceiver) 21 , for example, embedded in the front panel. For example, the radar sensor (e.g., transceiver) 21 may be configured to enable the remote control of a TV-set 20 e.g., without any hand-held device (such as e.g., a remote-control unit). For example, the different activities of the set of activities may be associated with (e.g., different) control operations (e.g., commands) of the TV set 20, as described herein.
Figure 3 is a system diagram illustrating an example of radar sensor integrated in a TV set. For example, a radar sensor (e.g., transceiver) 31 may be integrated (e.g., included, embedded) in a TV set. The radar sensor (e.g., transceiver) 31 may include at least one Tx antenna and at least one Rx antenna. For example, the radar sensor (e.g., transceiver) 31 may (e.g., continuously) obtain (e.g., measure instantaneous) radar signatures of an activity performed by a user. For example, the radar signature may comprise (e.g., be based on) a range profile (RP) (e.g., a set of power values for different distances to the radar sensor). For example, the radar signature(s) may be sent to a processor (e.g., a processing unit (PU)) 32, that may be equipped with an internal or external memory 34. For example, any of the PU 32 and the memory 34 may be any of the same as the TV’s ones and a separate one. For example, the PU and the TV processor 33 may be any of a same and two different processors. For example, the PU 32 may store the RPs in the memory 34. For example, (e.g., reference) radar signatures (e.g., heatmaps, sets of power values for different values of time and distance) may be stored in the memory 34, and may be associated with (e.g., different) control operations for controlling the TV-set.
Figure 4 is a diagram illustrating examples of power vs. range and time radar signatures of a set of activities (e.g., gestures) performed in front of a TV-set radar transceiver e.g., in the 6 GHz band. For example, the set of activities may comprise any of the following activities: sitting down 41 (e.g., going towards and sitting in a sofa), standing up 42 (e.g., standing up and leaving the sofa), sweep left 43 (e.g., sweeping hand from right to left, e.g., to scroll through the programs), sweep right 44 (e.g., sweeping hand from left e.g., to right to scroll through the programs in the reverse direction), hand up 45 (e.g., moving hand up e.g., to increase the volume, hand down 46 (e.g., moving hand down e.g., to turn down the volume).
For example, the (e.g., different) activities (e.g., reference signatures) may be associated with (e.g., different) control operations of the TV. For example, the (reference signature of the) sitting down activity 41 may be associated with a power on control operation (e.g., if the TV set power is off when the sitting down activity
41 is recognized). For example, the (reference signature of the) standing up activity
42 (e.g., including a walk away of the TV set) may be associated with a power off control operation (e.g., if the TV set power is on when the standing-up activity 42 is recognized). For example, any of the (reference signatures of the) sweep left 43 and sweep right 44 activity may be associated with a channel change control operation in one or the other direction. For example, any of the (reference signatures of the) hand up 45 and hand down 46 activity may be associated with any of a volume increase and a volume decrease control operation. Any other (reference signature of any) activity (e.g., gesture) that may be associated with any kind of control operation may be applicable to embodiments described herein.
For example, the set of activities 41 , 42, 43, 44, 45, 46 may be associated with corresponding reference signatures 410, 420, 430, 440, 450, 460 as illustrated in Figure 4. The reference signatures 410, 420, 430, 440, 450, 460 show different levels of grey color for different power levels for different values of time (vertical axis) and distance (horizontal axis).
For example, the set of reference signatures (e.g., associated with different control operations), may be any of pre-configured in the TV set and downloaded from e.g., a server via a network connection. For example, a (e.g., fixed, pre determined) set of gestures (e.g., activities) may be determined by e.g., the TV manufacturer for all the users in e.g., a default database.
In another example, the set of activities may be configured in the TV set, for example, during an installation phase. For example, a user may be requested, e.g., via a user interface, to perform an intuitive gesture and to associate the intuitive gesture with a control operation. For example, the user may choose to raise his hand to increase the device volume, move his hand down to decrease the device volume, sweep his hand from right to left to change to the next channel, sweep his hand from left to right to change to the previous channel and so on. For example, the user may perform a gesture and enter information via the user interface for associating the performed gesture with a control operation of the TV. For example, the user may be requested to repeat the same intuitive gesture for a number of times which (may be referred to herein as N, N being any integer value), such that the TV set may acquire a training set of signature samples for the different activities (and associated control operations). For example, after having obtained N samples of the different activities, the PU may generate the reference signatures of the reference activities. For example, in the weighting variant, weighted coefficients may be computed by the PU based on the training set.
For example, the TV set may be controlled based on recognized gestures as described herein. For example, after a gesture may have been recognized, the control command associated with the recognized gesture may be executed by the TV set. For example, the control command associated with the recognized gesture may be sent to the TV processor for control command execution.
For example, different types of activities, with different levels of complexity may be applicable to embodiments described herein. For example, in a case where a user enters the living room and sits in the sofa in front of the TV, the user presence may be detected by the radar transceiver. For example, the TV may be switched on and a program menu may be displayed, through which the user may navigate based on (e.g., configured) intuitive hand gestures. For example, (e.g., at the end), in a case where the user stands up and leaves the room, the TV may be automatically switched off. For example, any of the TV switch on and off may also be performed (e.g., operated, forced) by a recognition of specific gestures, that may have been learned by the TV set during the training phase (e.g., or configured via another method). For example, any of “entering the room” and “leaving the room” types of activities may be processed (e.g., detected, classified, recognized) in the same manner as (e.g. simple) gestures may be processed (e.g., detected, classified, recognized) but in a different (e.g., larger) time scale to better focus on the relevant event (such as e.g., six seconds for processing an activity event and two seconds for a (e.g., simple) gesture event. More generally a signature of an activity may be obtained (e.g., acquired) over any type of duration. Any duration for obtaining a signature of an activity may be applicable to embodiments described herein. For example, a reference signature may be associated with (e.g., have been acquired in) a given duration, and a signature sample (to be evaluated against the reference signature) may be acquired over at least the same duration.
For example, the reference signature may be automatically updated (e.g., enriched, improved) with new signatures (e.g., samples) of the gestures, for example, as a background task. In another example, (e.g., new) reference signatures may be automatically updated in a case where there is no activity in the room (e.g. in a case where background is detected).
In a variant, the radar transceiver may be packaged as standalone module such as e.g., an accessory, (e.g., separated) device (e.g., not integrated in the TV). The radar transceiver may be, for example, connected to the TV (using any of wired and wireless connection) for sending control commands to the TV.
Examples of Indoor Activity Detection
Figure 5 is a diagram illustrating five examples of radar range-time signatures of respectively five activities. For example, the radar range-time signatures 501 , 502, 503, 504, 505 represented in Figure 5 may have been obtained based on the AWR 1243 circuit from Texas Instrument, using Tl reference design. For example, a first 501 , a second 502, a third 503, a fourth 504 and a fifth 505 signature may be obtained from the radar transceiver in presence of a user respectively performing a first, a second, a third, a forth and a fifth activity in the range of the radar transceiver. For example, the first activity, which may be referred to herein as a background activity may correspond to no activity (e.g., no user) in the range of the radar. For example, the second activity, which may be referred to herein as a walking activity, may correspond to a user walking into the room and crossing below the radar in any of the two directions. For example, the third activity, which may be referred to herein as a sitting activity may correspond to a user walking into the room and sitting below (e.g., in front of) the radar. For example, the fourth and the fifth activities, which may be referred to herein as respectively a knee falling activity and a face falling activity may correspond to a user walking into the room and falling below (e.g., in front of) the radar respectively on his knee and on his face.
For example, the radar range-time signatures 501 , 502, 503, 504, 505 of respectively five activities, that may be obtained from a radar range profile, may comprise a set of power value for different time value (e.g., vertical axis) and different range values (e.g., horizontal axis). Other types of signatures, such as e.g., without limitation, range-doppler, micro-doppler, spectrogram, etc. may also be applicable to embodiments described herein. For example, (e.g., in real time situations), a signature may be obtained based on a (e.g., sliding) time window. For example, (e.g., instantaneous) range-power may be measured and added to a set of value (e.g., matrix) fora duration of T. In the example illustrated by Figure 5, the duration may be of seven seconds. After the duration, at a (e.g., each) instance, the first line of the matrix (e.g., corresponding to 1s) may be deleted and the instantaneous measured range-power value may be added to end of this matrix (e.g., as in a circular buffer).
For example, as illustrated in Figure 5, the five signatures 501 , 502, 503, 504, 505 may appear visually different such that they may be detected and classified based on a signal processing method. For example, the transmit (Tx) and receive (Rx) antennas’ coupling (e.g., self-reflection shown by the vertical lines from 0m to 0.3m) and the reflection from the ground (static clutter shown by the vertical lines from 2.5m to 3.8m) may be the dominant reflections. For example, these reflections may be removed by performing static clutter removal (by e.g., setting the points that may not change between two consecutive measurements beyond a (e.g., threshold) value (e.g., 5%), to a (e.g., minimum) value, to e.g., OdB. For example, background subtraction may be performed on (e.g., any of reference and sample) signatures, by e.g., performing a measurement e.g., without any activity in the room and by subtracting the background signature from other (e.g., any of reference and sample) signatures.
Figure 5 further shows five radar range-time signatures 511 , 512, 513, 514, 515 in the bottom that correspond to the radar range-time signatures 501 , 502, 503, 504, 505 shown at the top, after background subtraction. It may be observed that the self-reflection and the static clutter, including the ground effect, may be reduced (e.g., minimized). Background subtraction may allow to better differentiate the radar range-time signatures.
Table 1 is a confusion matrix that may be obtained by processing the radar range-time signatures illustrated in Figure 5 based on a regular correlation method.
Figure imgf000018_0001
By “regular” correlation method it is meant performing a (e.g., 2D) correlation of the signatures in a single correlation without subdividing in submatrices before performing multiple correlations between submatrices as described herein. As shown from the confusion matrix described in Table 1, the average accuracy (calculated as the average value of the confusion matrix’s diagonal) is equal to 64.66%. This may be due to the large dynamic of power values of the signatures (e.g., with relative power values range from OdB to 30dB, e.g., 1 to 1000)
Table 2 is a confusion matrix that may be obtained by processing the radar range-time signatures illustrated in Figure 5 based on the correlation as described herein (e.g., based on subdividing the signatures in sub signatures and correlating sub-signatures to reference sub-signatures).
Figure imgf000018_0002
Embodiments described herein may allow to improve the classification result by enabling each part of the signature to contribute to the correlation product. Using the proposed method, each part of the matrix may contribute to the correlation product. As shown in Table 2, applying the correlation method described herein may allow to provide 99.23% true detection. In this example, the signature may be divided according to the following power ranges (e.g., intervals of intensity): [0, 1], [1 , 10], [10, 100], [100, 1000] and [1000, ¥].
Table 3 is a confusion matrix that may be obtained by processing the radar range-time signatures illustrated in Figure 5 based on the weighting correlation variant.
Figure imgf000019_0001
Table 3: Confusion matrix using the weighting correlation variant described herein
For example, by applying the weighting correlation variant, the weights may be obtained with the following values [0.2282, 0.2505, 0.1945, 0.1790, 0.1478] and the average true detection ratio may be 98.46% as shown in Table 3. Figure 6A is a diagram illustrating an example of a processing device 6A for recognizing an activity in a set of activities. For example, the processing device 6A may comprise a radar transceiver 43 that may be configured to process a signal that may be obtained in presence of a user performing an activity in the range of the radar transceiver 43. For example, the radar transceiver 43 may be coupled to a processing module 45, configured to obtain a signature of the activity based on a radar processing of the obtained signal. For example, the processing module 45 may be further configured to divide the signature of the activity to obtain a set of sub-signatures of the activity according to a set of intervals of intensity, wherein different sub-signatures of the activity may correspond to different intervals of intensity, and wherein reference signatures of the activities in the set of activities may be divided in respective sets of reference sub signatures according to the (e.g., same) set of intervals of intensity, wherein a reference sub signature of an interval of intensity may correspond to (e.g., be associated with) a sub-reference signature of the same interval of intensity. For example, the processing module 45 may be further configured to recognize (e.g., classify) the activity in the set of activities based on a processing (e.g., a correlation) of the sub-signatures of the activity with (e.g., to) corresponding reference sub signatures of the activities of the set of activities according to the set of intervals of intensity.
For example, the processing module 45 may be coupled to an optional interface module 47. For example, the interface module 47 may be a network interface. For example, the network interface may be any of wired and wireless network interface, any of a local and wide area network interface. For example, the interface module 47 may be a user interface, running (e.g., and displayed) locally on the processing device 6A. For example, the user interface may be running on another device, communicating with the processing device 6A via the network interface. The user interface may allow the processing device 6A to interact with a user, for example, by associating a control operation with a specific activity and by triggering the control operation based on the recognized activity. Without limitations, control operations may include any of unlocking a door, unlocking a device, lighting a lamp, switching on/off a device, increasing volume, decreasing volume, changing channel... In another example, the user interface may propose (e.g., any of display an image displaying, play an audio including, ...) a set of options to a user, wherein an option may be associated with a specific activity. Upon recognition of the activity as one of the specific activities, the user interface may be configured to validate (e.g., select) the option corresponding to the recognized activity. According to embodiments, the processing device 6A may be a (e.g., basic) sensing device that may be any of separable from and couplable to another (e.g., more complex) processing device (such as e.g., a TV). Coupling / separation between both devices may be accomplished via any of a bus interface, a network interface, with e.g., connectors.
Figure 6B represents an exemplary architecture of the processing device 6A described herein. The processing device 6B may comprise one or more processor(s) 610, which may be, for example, any of a CPU, a GPU a DSP (English acronym of Digital Signal Processor), along with internal memory 620 (e.g. any of RAM, ROM, EPROM). The processing device 6A may comprise any number of Input/Output interface(s) 630 adapted to send output information and/or to allow a user to enter commands and/or data (e.g. any of a keyboard, a mouse, a touchpad, a webcam, a display), and/or to send / receive data over a network interface; and a power source 640 which may be external to the processing device 6A.
For example, the processing device 6A may further comprise a computer program stored in the memory 620. The computer program may comprise instructions which, when executed by the processing device 6A, in particular by the processor(s) 610, make the processing device 6A carrying out the processing method described with reference to figure 7. According to a variant, the computer program may be stored externally to the processing device 6A on a non-transitory digital data support, e.g. on an external storage medium such as any of a SD Card, HDD, CD-ROM, DVD, a read-only and/or DVD drive, a DVD Read/Write drive, all known in the art. The processing device 6A may comprise an interface to read the computer program. Further, the processing device 6A may access any number of Universal Serial Bus (USB)-type storage devices (e.g., “memory sticks.”) through corresponding USB ports (not shown).
According to embodiments, the processing device 6A may be any of a server, a desktop computer, a laptop computer, a networking device, a TV set, a tablet, a smartphone, a phablet, a set-top box, an internet gateway, a game console, a head mounted device, an internet of things (loT) device, a wearable device, a doorbell system, a locker, ...
Figure 7 is a diagram illustrating an example of a method for recognizing an activity in a set of activities. For example, the method may be implemented in a processing device that may be coupled to (e.g., integrating) a radar transceiver.
For example, in a step 710, a signature of the activity may be obtained based on a radar processing of a signal that may be obtained in presence of a user performing the activity e.g., in the range of the radar transceiver. For example, in a step 720, the signature of the activity may be divided in (e.g., to obtain) a set of sub-signatures of the activity according to a set of intervals of intensity (e.g., power values). For example, different sub-signatures of the activity may correspond to different intervals of intensity of the signature. For example, each sub-signature of the activity may correspond to a different interval of intensity of the signature. For example, reference signatures of the (e.g., set of) activities in the set of activities may be divided in respective sets of reference sub signatures e.g., according to the same intervals of intensity. For example, different (e.g., each of the) reference sub signatures of a set may correspond to the (e.g., same) different intervals of intensity. In other words, a same set of intervals of intensity may be used to divide the signature in sub-signatures, and to divide a (e.g., each) reference signature in a set of reference sub signatures, such that for a same interval of intensity a sub-signature of a signature may correspond to (e.g., be associated with) a reference sub signature of a reference signature.
For example, in a step 730, the activity may be recognized in the set of activities (e.g., as a recognized activity) based on a processing of the sub signatures of the activity with corresponding reference sub signatures of the activities of the set of activities according to the set of intervals of intensity. For example, the processing may, for example, comprise correlating the sub signatures of the activity with the corresponding reference sub signatures of the (e.g., set of) activities according to the set of intervals of intensity.
For example, the processing of the sub-signatures of the activity with the corresponding reference sub signatures may comprise obtaining correlation matrices of the activity to respective other activities. For example, a correlation matrix of the activity to a given activity may be obtained based on a sum of correlation sub-matrices for the different intervals of intensity. For example, a correlation sub matrix (e.g., for an interval of intensity) may be obtained by correlating the sub-signature of the activity to the corresponding reference sub signature of the given activity (e.g., for the same interval of intensity).
For example, the activity may be recognized in the set of activities based on a maximum value of the correlation matrices. For example, the activity may be recognized as a recognized activity in the set of activities on a condition that the correlation matrix of the activity to the recognized activity comprises the maximum value of the correlation matrices.
For example, the activity may be recognized as the recognized activity in the set of activities on a further condition that the maximum value is above a predetermined value.
For example, the sum may be weighted with different weighting coefficients corresponding to the different intervals of intensity.
For example, a weighting coefficient corresponding to a given interval of intensity may be obtained by dividing a co-correlation of the reference sub signatures to different sub-signature samples of a same activity for the given interval of intensity by a cross-correlation of the reference sub-signatures to sub signature samples of different activities for the given interval of intensity.
For example, the weighting coefficients may be normalized.
For example, the activity may comprise a (e.g., single) gesture.
For example, the activity may comprise any number (e.g., a sequence) of gestures.
For example, the reference signatures of the (e.g., set of) activities may be obtained based on a training set of signature samples of the (e.g., set of) activities.
For example, the reference signatures of the (e.g., set of) activities may be any of pre-configured and downloaded from a server.
For example, the reference signatures of the activities may be any of pre configured and downloaded from a server.
For example, different reference signatures may be associated with different control operations of the processing device.
For example, after the activity may have been recognized (e.g., according to a reference signature), it may be determined with which control operation the reference signature of the recognized activity may be associated, and the control operation associated with the reference signature of the recognized activity may be executed on (e.g., by) the processing device. For example, the processing device may be any of a TV set, a set-top-box, a media player and a game console, that may be controllable by the set of activities.
For example, the control operation may be any of a switch on, a switch off, a volume increase, a volume decrease, and a channel change control operation.
Conclusion
While not explicitly described, the present embodiments may be employed in any combination or sub-combination. For example, the present principles are not limited to the described variants, and any arrangement of variants and embodiments may be used. Moreover, embodiments described herein are not limited to the gestures and activities described herein and any other types of gestures/activities (e.g., involving any body parts) may be compatible with the embodiments described herein.
Any characteristic, variant or embodiment described for a method is compatible with an apparatus device comprising means for processing the disclosed method, with a device comprising a processor configured to process the disclosed method, with a computer program product comprising program code instructions and with a non-transitory computer-readable storage medium storing program instructions.
Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element can be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer readable medium for execution by a computer or processor. Examples of non-transitory computer-readable storage media include, but are not limited to, a read only memory (ROM), random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
Moreover, in the embodiments described above, processing platforms, computing systems, controllers, and other devices containing processors are noted. These devices may contain at least one Central Processing Unit ("CPU") and memory. In accordance with the practices of persons skilled in the art of computer programming, reference to acts and symbolic representations of operations or instructions may be performed by the various CPUs and memories. Such acts and operations or instructions may be referred to as being "executed," "computer executed" or "CPU executed."
One of ordinary skill in the art will appreciate that the acts and symbolically represented operations or instructions include the manipulation of electrical signals by the CPU. An electrical system represents data bits that can cause a resulting transformation or reduction of the electrical signals and the maintenance of data bits at memory locations in a memory system to thereby reconfigure or otherwise alter the CPU's operation, as well as other processing of signals. The memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties corresponding to or representative of the data bits. It should be understood that the representative embodiments are not limited to the above-mentioned platforms or CPUs and that other platforms and CPUs may support the provided methods.
The data bits may also be maintained on a computer readable medium including magnetic disks, optical disks, and any other volatile (e.g., Random Access Memory ("RAM")) or non-volatile (e.g., Read-Only Memory ("ROM")) mass storage system readable by the CPU. The computer readable medium may include cooperating or interconnected computer readable medium, which exist exclusively on the processing system or are distributed among multiple interconnected processing systems that may be local or remote to the processing system. It is understood that the representative embodiments are not limited to the above- mentioned memories and that other platforms and memories may support the described methods.
In an illustrative embodiment, any of the operations, processes, etc. described herein may be implemented as computer-readable instructions stored on a computer-readable medium. The computer-readable instructions may be executed by a processor of a mobile unit, a network element, and/or any other computing device. There is little distinction left between hardware and software implementations of aspects of systems. The use of hardware or software is generally (e.g., but not always, in that in certain contexts the choice between hardware and software may become significant) a design choice representing cost vs. efficiency tradeoffs. There may be various vehicles by which processes and/or systems and/or other technologies described herein may be effected (e.g., hardware, software, and/or firmware), and the preferred vehicle may vary with the context in which the processes and/or systems and/or other technologies are deployed. For example, if an implementer determines that speed and accuracy are paramount, the implementer may opt for a mainly hardware and/or firmware vehicle. If flexibility is paramount, the implementer may opt for a mainly software implementation. Alternatively, the implementer may opt for some combination of hardware, software, and/or firmware.
The foregoing detailed description has set forth various embodiments of the devices and/or processes via the use of block diagrams, flowcharts, and/or examples. Insofar as such block diagrams, flowcharts, and/or examples contain one or more functions and/or operations, it will be understood by those within the art that each function and/or operation within such block diagrams, flowcharts, or examples may be implemented, individually and/or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs); Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (1C), and/or a state machine.
Although features and elements are provided above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element can be used alone or in any combination with the other features and elements. The present disclosure is not to be limited in terms of the particular embodiments described in this application, which are intended as illustrations of various aspects. Many modifications and variations may be made without departing from its spirit and scope, as will be apparent to those skilled in the art. No element, act, or instruction used in the description of the present application should be construed as critical or essential to the invention unless explicitly provided as such. Functionally equivalent methods and apparatuses within the scope of the disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing descriptions. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure is to be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled. It is to be understood that this disclosure is not limited to particular methods or systems.
In certain representative embodiments, several portions of the subject matter described herein may be implemented via Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), digital signal processors (DSPs), and/or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein, in whole or in part, may be equivalently implemented in integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and/or writing the code for the software and or firmware would be well within the skill of one of skill in the art in light of this disclosure. In addition, those skilled in the art will appreciate that the mechanisms of the subject matter described herein may be distributed as a program product in a variety of forms, and that an illustrative embodiment of the subject matter described herein applies regardless of the particular type of signal bearing medium used to actually carry out the distribution. Examples of a signal bearing medium include, but are not limited to, the following: a recordable type medium such as a floppy disk, a hard disk drive, a CD, a DVD, a digital tape, a computer memory, etc., and a transmission type medium such as a digital and/or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.). The herein described subject matter sometimes illustrates different components contained within, or connected with, different other components. It is to be understood that such depicted architectures are merely examples, and that in fact many other architectures may be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality may be achieved. Hence, any two components herein combined to achieve a particular functionality may be seen as "associated with" each other such that the desired functionality is achieved, irrespective of architectures or intermediate components. Likewise, any two components so associated may also be viewed as being "operably connected", or "operably coupled", to each other to achieve the desired functionality, and any two components capable of being so associated may also be viewed as being "operably couplable" to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically mateable and/or physically interacting components and/or wirelessly interactable and/or wirelessly interacting components and/or logically interacting and/or logically interactable components.
With respect to the use of substantially any plural and/or singular terms herein, those having skill in the art can translate from the plural to the singular and/or from the singular to the plural as is appropriate to the context and/or application. The various singular/plural permutations may be expressly set forth herein for sake of clarity.
It will be understood by those within the art that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "includes" should be interpreted as "includes but is not limited to," etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, where only one item is intended, the term "single" or similar language may be used. As an aid to understanding, the following appended claims and/or the descriptions herein may contain usage of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles "a" or "an" limits any particular claim containing such introduced claim recitation to embodiments containing only one such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and indefinite articles such as "a" or "an" (e.g., "a" and/or "an" should be interpreted to mean "at least one" or "one or more"). The same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations," without other modifiers, means at least two recitations, or two or more recitations).
Furthermore, in those instances where a convention analogous to "at least one of A, B, and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). In those instances where a convention analogous to "at least one of A, B, or C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, or C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and/or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and/or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B." Further, the terms "any of" followed by a listing of a plurality of items and/or a plurality of categories of items, as used herein, are intended to include "any of," "any combination of," "any multiple of," and/or "any combination of multiples of" the items and/or the categories of items, individually or in conjunction with other items and/or other categories of items. Moreover, as used herein, the term "set" or “group” is intended to include any number of items, including zero. Additionally, as used herein, the term "number" is intended to include any number, including zero.
In addition, where features or aspects of the disclosure are described in terms of Markush groups, those skilled in the art will recognize that the disclosure is also thereby described in terms of any individual member or subgroup of members of the Markush group.
Moreover, the claims should not be read as limited to the provided order or elements unless stated to that effect. In addition, use of the terms "means for" in any claim is intended to invoke 35 U.S.C. §112, IT 6 or means-plus-function claim format, and any claim without the terms "means for" is not so intended.

Claims

1. A method comprising: obtaining a signature of an activity based on a radar processing of a signal obtained in presence of a user performing the activity; dividing the signature of the activity in a set of sub-signatures of the activity according to a set of intervals of intensity, wherein different sub-signatures of the activity correspond to different intervals of intensity, and wherein reference signatures of a set of activities are divided in respective sets of reference sub signatures according to the set of intervals of intensity; and recognizing the activity in the set of activities by correlating the sub signatures of the activity to corresponding reference sub signatures of the set of activities according to the set of intervals of intensity.
2. The method according to claim 1 , wherein correlating the sub-signatures of the activity to corresponding reference sub signatures comprises obtaining correlation matrices of the activity to respective other activities, wherein a correlation matrix of the activity to a given activity is a sum of correlation sub-matrices for the different intervals of intensity, wherein a correlation sub matrix for an interval of intensity is obtained by correlating a sub-signature of the activity to a corresponding reference sub signature of the given activity for the interval of intensity.
3. The method according to claim 2, wherein the activity is recognized in the set of activities based on a maximum value of the correlation matrices.
4. The method according to claim 3, wherein the activity is recognized as a recognized activity in the set of activities on a condition that the correlation matrix of the activity to the recognized activity comprises the maximum value of the correlation matrices.
5. The method according to claim 4, wherein the activity is recognized as the recognized activity in the set of activities on a further condition that the maximum value is above a predetermined value.
6. The method according to any of claims 2 to 5, wherein the sum is weighted with different weighting coefficients corresponding to the different intervals of intensity.
7. The method according to claim 6, wherein the weighting coefficients are obtained on a training set of signature samples of different activities, the signature samples being divided in sub-signature samples corresponding to the different intervals of intensity.
8. The method according to claim 7, wherein a weighting coefficient corresponding to a given interval of intensity is obtained by dividing a co-correlation of the reference sub signatures to different sub-signature samples of a same activity for the given interval of intensity by a cross-correlation of the reference sub signatures to sub-signature samples of different activities for the given interval of intensity.
9. The method according to any of claims 6 to 8, wherein the weighting coefficients are normalized.
10. The method according to any of claims 1 to 9, wherein the activity comprises a gesture.
11. The method according to any of claims 1 to 10, wherein the activity comprises a sequence of gestures.
12. The method according to any of claims 1 to 11 , wherein the reference signatures of the set of activities are obtained based on a training set of signature samples of the set of activities.
13. The method according to any of claims 1 to 11 , wherein the reference signatures of the set of activities are any of pre-configured and downloaded from a server.
14. The method according to any of claims 1 to 13, wherein different reference signatures are associated with different control operations of a device.
15. The method according to claims 14, wherein after the activity has been recognized according to a reference signature, it is determined with which control operation the reference signature of the recognized activity is associated, and the control operation associated with the reference signature of the recognized activity is executed on the device.
16. The method according to any of claims 14 to 15, wherein the device is any of a TV set, a set-top-box, a media player and a game console, controllable by the set of activities.
17. The method according to any of claims 14 to 16, wherein the control operation is any of a switch on, a switch off, a volume increase, a volume decrease, and a channel change.
18. An apparatus comprising: a radar transceiver configured to process a signal obtained in presence of a user performing an activity; a processor configured to: obtain a signature of the activity based on a radar processing of the signal; divide the signature of the activity in a set of sub-signatures of the activity according to a set of intervals of intensity, wherein different sub signatures of the activity correspond to different intervals of intensity, and wherein reference signatures of the set of activities are divided in respective sets of reference sub signatures according to the set of intervals of intensity; and recognize the activity in the set of activities based on a correlation of the sub-signatures of the activity to corresponding reference sub signatures of the set of activities according to the set of intervals of intensity.
19. The apparatus according to claim 18, wherein different reference signatures are associated with different control operations of the apparatus.
20. The apparatus according to any of claims 18 to 19, wherein the apparatus is any of a TV set, a set-top-box, a media player and a game console, controllable by the set of activities, and wherein a control operation is any of a switch on, a switch off, a volume increase, a volume decrease, and a channel change.
PCT/EP2022/066914 2021-06-29 2022-06-21 Methods, architectures, apparatuses and systems directed to recognizing an activity in a set of activities Ceased WO2023274792A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP21305889 2021-06-29
EP21305889.4 2021-06-29

Publications (1)

Publication Number Publication Date
WO2023274792A1 true WO2023274792A1 (en) 2023-01-05

Family

ID=76845165

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2022/066914 Ceased WO2023274792A1 (en) 2021-06-29 2022-06-21 Methods, architectures, apparatuses and systems directed to recognizing an activity in a set of activities

Country Status (1)

Country Link
WO (1) WO2023274792A1 (en)

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
AHMED SHAHZAD ET AL: "Hand Gestures Recognition Using Radar Sensors for Human-Computer-Interaction: A Review", REMOTE SENSING, vol. 13, no. 3, 1 February 2021 (2021-02-01), CH, pages 527, XP055823644, ISSN: 2072-4292, DOI: 10.3390/rs13030527 *
LOUZIR ALI ET AL: "A Simple RF-Based Solution for Gesture Recognition", 2021 IEEE INTERNATIONAL CONFERENCE ON CONSUMER ELECTRONICS (ICCE), IEEE, 10 January 2021 (2021-01-10), pages 1 - 5, XP033916478, DOI: 10.1109/ICCE50685.2021.9427740 *

Similar Documents

Publication Publication Date Title
Fu et al. Sensing technology for human activity recognition: A comprehensive survey
JP7623106B2 (en) Method, Apparatus and System for Wireless Tracking, Scanning and Surveillance - Patent application
JP6816201B2 (en) Type-independent RF signal representation
CN111695420B (en) A gesture recognition method and related devices
US11567580B2 (en) Adaptive thresholding and noise reduction for radar data
US8885890B2 (en) Depth map confidence filtering
US20210365124A1 (en) Radar-Enabled Sensor Fusion
US10529071B2 (en) Facial skin mask generation for heart rate detection
EP2839719B1 (en) Sensory lighting system and method for characterizing an illumination space
US8571263B2 (en) Predicting joint positions
CN105122183B (en) Detect natural user input engagement
US20150363005A1 (en) Techniques for using human gestures to control gesture unaware programs
CN113064483A (en) Gesture recognition method and related device
US20110221755A1 (en) Bionic motion
CN106796656A (en) Away from the depth of time-of-flight camera
US20140085625A1 (en) Skin and other surface classification using albedo
US20160106329A1 (en) Utilizing different color channels for rgb-image-based differential heart rate detection
Tang et al. MDPose: Human skeletal motion reconstruction using WiFi micro-Doppler signatures
CN114581958B (en) Static human body posture estimation method based on CSI signal arrival angle estimation
Deng et al. Midas++: Generating training data of mmwave radars from videos for privacy-preserving human sensing with mobility
CN113168700B (en) Electronic device and control method thereof
CN114708663A (en) Millimeter wave radar sensing gesture recognition method based on few-sample learning
CN116520282B (en) Radar continuous motion detection and behavior recognition method for intelligent endowment application
Janakaraj et al. STAR: Simultaneous tracking and recognition through millimeter waves and deep learning
CN116561700A (en) Indoor Human Pose Recognition Method Based on Millimeter Wave Radar

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22737814

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22737814

Country of ref document: EP

Kind code of ref document: A1