CN110378214A - Activity recognition and response method, equipment, Intelligent treatment equipment and medium - Google Patents
Activity recognition and response method, equipment, Intelligent treatment equipment and medium Download PDFInfo
- Publication number
- CN110378214A CN110378214A CN201910507405.7A CN201910507405A CN110378214A CN 110378214 A CN110378214 A CN 110378214A CN 201910507405 A CN201910507405 A CN 201910507405A CN 110378214 A CN110378214 A CN 110378214A
- Authority
- CN
- China
- Prior art keywords
- behavior
- data
- characteristic
- activity recognition
- real object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses Activity recognitions and response method, equipment, Intelligent treatment equipment and medium.Behavior identification includes: that building behavior corresponds to table with response method；For each real object behavior, the image data of subjects is acquired, and extracts characteristic from image data, is used as standard feature data, so that each real object behavior has corresponding multiple standard feature data, constitutive characteristic library；Feature database construction step includes data screening step, by using the sensing data of hardware sensing device, the image data captured simultaneously by image capturing device is screened, with extraction standard characteristic.It is accurately screened by the characteristic in the feature database to the Activity recognition for real object, to ensure the accuracy of the characteristic for Activity recognition, to greatly improve the precision of the Activity recognition of real object.
Description
Technical field
The present invention relates to Activity recognition and response technology more particularly to Activity recognition and response method, equipment, Intelligent treatments
Equipment and medium.
Background technique
In Activity recognition field, presently, there are the not high problems of accuracy of identification.For example, the Activity recognition side of the prior art
Case is not accurate enough for the movement of real object (such as user etc.) and/or Expression Recognition, particularly with subtle movement (such as
Smile, blink etc.) be difficult to accurately identify out.
In addition, matching and responding field in behavior, since the Activity recognition precision for real object is insufficient, cause virtual
Object is not high for the matching degree for the respondent behavior that real object behavior is made, and seriously affects user experience.
Therefore, a kind of Activity recognition precision that can be improved real object is needed, and/or improves virtual objects, response
In real object behavior and the solution of the matching degree of respondent behavior made.
Summary of the invention
One of in order to solve problem above, the present invention provides a kind of Activity recognitions and response method, equipment, Intelligent treatment
Equipment and medium, to improve Activity recognition and/or the matched precision of behavior.
According to one embodiment of present invention, a kind of Activity recognition and response method are provided, comprising: behavior corresponds to table building
Step, building is for providing the corresponding table of the behavior of the corresponding relationship between real object behavior and virtual objects behavior；Feature database
Construction step corresponds to each real object behavior in table for behavior respectively, acquires subjects by image capturing device
The image data when behavior is repeatedly made, and extracts the spy of the behavior from the image data captured when making the behavior every time
Data, the standard feature data as the behavior are levied, so that behavior corresponds to each real object behavior in table with corresponding
Multiple standard feature data, and correspond to by behavior each real object behavior and corresponding standard feature data in table
Constitutive characteristic library；Wherein, the feature database construction step includes data screening step, in the data screening step: trying
When testing object and repeatedly making each real object behavior, the hardware sensing device in subjects is carried to the behavior
It is sensed, obtains the sensing data of hardware sensing device, and the image capturing device carries out shadow to the behavior simultaneously
As capturing, image data is obtained；By using the sensing data, for the image captured simultaneously by the image capturing device
Data are screened, with from the image data extraction standard characteristic after screening.
Optionally, in the data screening step, use feeling measured data screening image data includes: based on sensing data
Activity recognition as a result, rejecting the sensing data that are not consistent with goal behavior, and will capture simultaneously with the sensing data being removed
Image data also accordingly reject, the image data after obtaining the screening.
Optionally, in the data screening step, the image data after also making the screening passes through as feature point
The machine learning model of class model, obtain being consistent characteristic and the characteristic that is not consistent, and the characteristic that will be consistent is as standard
Characteristic is put into feature database, and the characteristic rejecting that will not be consistent.
Optionally, the Activity recognition and response method further include: Activity recognition step, from the behavior for being directed to real object
The image data captured extracts characteristic, and by calculating the standard feature data in this feature data and the feature database
Characteristic similarity, obtain matched standard feature data, and be based on matched standard feature data, obtained from the feature database
Real object behavior corresponding with the standard feature data, as corresponding Activity recognition result.
Optionally, by calculating the distance between characteristic, the characteristic similarity between characteristic is obtained.
Optionally, the Activity recognition and response method further include: behavior response step, based on being obtained in behavior identification step
To Activity recognition as a result, according to for providing that the behavior of the corresponding relationship between object behavior and virtual objects behavior is corresponding
Table makes virtual objects make corresponding respondent behavior.
An exemplary embodiment according to the present invention provides a kind of Activity recognition and response apparatus, the Activity recognition
Include: that behavior corresponds to table construction device with response apparatus, is configured for building behavior and corresponds to table, the behavior corresponds to table and is used for
Provide the corresponding relationship between real object behavior and virtual objects behavior；Feature database construction device is configured for needle respectively
Each real object behavior in table is corresponded to behavior, when repeatedly making the behavior by image capturing device acquisition subjects
Image data, and from the image data captured when making the behavior every time extract the behavior characteristic, be used as the row
For standard feature data so that behavior corresponds to each real object behavior in table with corresponding multiple standard feature numbers
According to, and correspond to by behavior each real object behavior and corresponding standard feature data constitutive characteristic library in table；Its
In, when subjects repeatedly make each real object behavior, carry the hardware sensing device pair in subjects
The behavior is sensed, and obtains the sensing data of hardware sensing device, and the image capturing device is simultaneously to the row
To carry out image capturing, image data is obtained, wherein the feature database construction device includes data screening component, the data
Screening part is configured for: by using the sensing data, for the image captured simultaneously by the image capturing device
Data are screened, with from the image data extraction standard characteristic after screening.
Optionally, the operation of the data screening component use feeling measured data screening image data includes: based on sensing number
According to Activity recognition as a result, rejecting the sensing data that are not consistent with goal behavior, and will catch simultaneously with the sensing data being removed
The image data caught also accordingly is rejected, the image data after obtaining the screening.
Optionally, the data screening component is additionally configured to so that the image data after the screening passes through as special
Levy disaggregated model machine learning model, the characteristic that obtains being consistent with the characteristic that is not consistent, the characteristic that will be consistent as
Standard feature data are put into feature database, and the characteristic rejecting that will not be consistent.
Optionally, the Activity recognition and response apparatus further include:
Activity recognition device, the image data for being configured for being captured from the behavior for real object extract characteristic
According to, and the characteristic similarity by calculating the standard feature data in this feature data and the feature database, obtain matched mark
Quasi- characteristic, and matched standard feature data are based on, it is obtained from the feature database corresponding with the standard feature data true
Real object behavior, as corresponding Activity recognition result.
Optionally, the Activity recognition device is obtained between characteristic by calculating the distance between characteristic
The characteristic similarity.
Optionally, the Activity recognition and response apparatus further include: behavior response device is configured for by behavior
The Activity recognition that identification device obtains is as a result, according to for the corresponding relationship between regulation object behavior and virtual objects behavior
Behavior corresponds to table, and virtual objects is made to make corresponding respondent behavior.
Still another embodiment in accordance with the present invention provides a kind of Intelligent treatment equipment, comprising: processor；And memory,
It is stored thereon with executable code, when the executable code is executed by the processor, executes the processor above
One of method of description.
According to still another embodiment of the invention, a kind of non-transitory machinable medium is provided, is stored thereon with
Executable code makes the processor execute one of method described above when the executable code is executed by processor.
In the present invention, the characteristic in the feature database for the Activity recognition of real object is accurately sieved first
Choosing, to ensure the accuracy of the characteristic for Activity recognition, to greatly improve the precision of the Activity recognition of real object.
In addition, for the real object behavior identified, passing through based on real object Activity recognition with high precision
Based on characteristic similarity, in the corresponding table of behavior for providing the corresponding relationship between real object behavior and virtual objects behavior
The middle respondent behavior searched to determine virtual objects can effectively improve between virtual corresponding behavior and real object behavior
Matching degree.
Detailed description of the invention
Disclosure illustrative embodiments are described in more detail in conjunction with the accompanying drawings, the disclosure above-mentioned and its
Its purpose, feature and advantage will be apparent, wherein in disclosure illustrative embodiments, identical appended drawing reference
Typically represent same parts.
Fig. 1 gives the Activity recognition of an exemplary embodiment according to the present invention and the schematic flow of response method
Figure.
Fig. 2 gives the schematic diagram of the frequency signal feature of an illustrative user behavior.
Fig. 3 gives the Activity recognition of an exemplary embodiment according to the present invention and the schematic frame of response apparatus
Figure.
Fig. 4 gives the schematic block diagram of the Intelligent treatment equipment of an exemplary embodiment according to the present invention.
Specific embodiment
The preferred embodiment of the disclosure is more fully described below with reference to accompanying drawings.Although showing the disclosure in attached drawing
Preferred embodiment, however, it is to be appreciated that may be realized in various forms the disclosure without the embodiment party that should be illustrated here
Formula is limited.On the contrary, these embodiments are provided so that this disclosure will be more thorough and complete, and can be by the disclosure
Range is completely communicated to those skilled in the art.What needs to be explained here is that number, serial number and attached drawing in the application
Mark it is merely for convenience description and occur, for step of the invention, sequence etc. be not limited in any way, unless
The execution that step has been explicitly pointed out in specification has specific sequencing.
As described above, in the present invention, first to the characteristic in the feature database for the Activity recognition of real object
It is accurately screened, to ensure the accuracy of the characteristic for Activity recognition, to greatly improve the behavior of real object
The precision of identification.
In addition, for the real object behavior identified, passing through based on real object Activity recognition with high precision
Based on characteristic similarity, in the corresponding table of behavior for providing the corresponding relationship between real object behavior and virtual objects behavior
The middle respondent behavior searched to determine virtual objects can effectively improve between virtual corresponding behavior and real object behavior
Matching degree.
In the following, the Activity recognition method that will first introduce an exemplary embodiment according to the present invention.
Fig. 1 gives the Activity recognition of an exemplary embodiment according to the present invention and the schematic flow of response method
Figure.
As shown in Figure 1, corresponding to table construction step S110 in behavior, construct for providing real object behavior and virtual objects
The behavior of corresponding relationship between behavior corresponds to table.
It may include real object behavior and corresponding with real object behavior that above-mentioned behavior, which corresponds to the data item in table,
One or more virtual objects behaviors.
Here, it should be pointed out that, it is corresponded in table in behavior, between each real object behavior and each virtual objects behavior not
It must be one-to-one relationship.For example, for some or certain real object behaviors for being less susceptible to respond or distinguish, it can
To match with specific one or several virtual objects behaviors as its respondent behavior.For example, if true right as one
As the movement of behavior excessively distort or excessive velocities and be difficult to, or if the movement as a real object behavior exists
Behavior is corresponded in table and be can not find, then for example following respondent behavior similar in this way can be used respond "！It is really too cruel
?！", " may I ask you is the friend from Mars? ".
In addition, for the corresponding relationship between real object behavior and virtual objects behavior, i.e. real object behavior and void
Association between quasi- object behavior, can construct according to actual needs, can also correspond in table and make in behavior according to demand
It updates and modifies.
In feature database construction step S120, each real object behavior in table is corresponded to for behavior respectively, passes through image
Capture device (such as can be by using photographic device of smart machine), acquisition subjects repeatedly (such as 100 times or 500
It is secondary etc.) then image data when making the behavior extracts the characteristic of the behavior from the image data of the behavior,
As the standard feature data of the behavior, behavior corresponds to each real object behavior in table with corresponding multiple marks as a result,
Quasi- characteristic is corresponded to each real object behavior and corresponding standard feature data constitutive characteristic in table by behavior
Library.
Specifically, in above-mentioned feature database, it is (each true right in table corresponding with behavior to be stored with each real object behavior
As behavior corresponds) and corresponding standard feature data, wherein a real object behavior can correspond to multiple and different
Standard feature data.For example, the examination for " lift hand " this movement (as a real object behavior), as real object
It tests object (such as testing crew) and repeatedly makes this movement, capture for the photographic device of smart machine to acquire as the movement
As the image data of multiple frame image sequences, then, multiple characteristics of the movement are extracted from the image data of the movement
According to the standard feature data set as the movement.
It for example can be as follows from the method for image data extraction standard characteristic.
As described above, the form of image data can be extracted for frame image sequence by multiple frame image sequences
Standard feature data.Specifically, the photographic device of smart machine can capture one from an action process of subjects
(for example, going out a frame image sequence from continuous 50 frame image zooming-out, " frame image sequence " will be slightly for continuous frame image sequence
After specifically describe), then can from this frame image sequence, extract characteristics of image (for example, profile, color, texture etc.)
Data, using the characteristic of extraction as the movement standard feature data (here, standard feature data for example can be to
The form of amount or serial data).
Multiple standard feature data obtained from testing crew makes the same movement repeatedly, can be described as the movement (should
Real object behavior) standard feature data set.Here, the corresponding real object behavior of a standard feature data set, row
It is stored in above-mentioned feature database for the corresponding all standard feature data sets of all real object behaviors specified in corresponding table.
The standard feature data set of different real object behaviors (including movement, expression etc.) constitutes a standard feature data as a result,
Library (herein referred as " feature database ").
It wherein, optionally, can also include data screening step S121, for leading in feature database construction step S120
Cross using by carry the hardware sensing device senses in subjects to the sensing data of behavior made of subjects, it is right
It is screened in the image data captured by smart machine, to obtain more accurate standard feature number from the image data after screening
According to.
Firstly, since the present inventor is it is considered that in the prior art, for describe subjects make it is true
Directly shadow is understood in the accuracy of the standard feature data of the feature of object behavior (can be generally thought is " object behavior of standard ")
Therefore the precision for ringing Activity recognition in the present invention, the accuracy of characteristic is improved using special data screening mechanism
For example, subjects need repeatedly to make same movement, (for example " lift hand " or " blink " etc is moved
Make), standard feature data are obtained to acquire image data, but, not ensuring that the movement made every time all is to meet mark
Quasi- movement (can be described as " goal behavior "), therefore, the characteristic obtained from the image data of acquisition not necessarily can act as
" standard feature data ".
Specifically, in this step, for example, can first based on the hardware sensing device senses worn by testing crew to
Sensing data Activity recognition as a result, the real behavior made of elimination test object cannot be with target line from sensing data
For the data being consistent, the precision of data is sensed with raising.For example, subjects repeatedly do the movement (expression) of " smile ", wear
Hardware sensing device with real object passes through movement the sense of access measured data of sensing subjects, but in sensing process
In, as described above, some " smiles " movement of possible subjects is made deformation, " smile " can not act at last, this can
To judge that the result of judgement is as Activity recognition result from the sensing data of hardware sensing device (frequency signal feature)
(for example, whether the movement belongs to " smile " movement).Therefore, by sensing data, for example, passing through the Activity recognition of sensing data
As a result, the sensing data for meeting goal behavior can be filtered out.
Then, optionally, further, by each real object behavior and image data institute corresponding to sensing data
Corresponding each real object behavior is consistent, and (shooting of the sensing and photographic device of hardware sensing device carries out simultaneously, institute
Movement while being directed to the same subjects with them), therefore, the sense that can be rejected and be removed from image data
Part image data corresponding to measured data because real object behavior corresponding to this partial image data being removed with
Goal behavior is not inconsistent, and described in the example acted such as " smile " above, is movement deformation " smile " movement in fact, cannot calculate
It is real " smile " movement.If above-mentioned by being used as after the corresponding image data progress feature extraction of " smile " movement of deformation
Feature database in standard feature data necessarily will affect row then such characteristic is treated as normal data
For the precision of identification.And reject such data not being consistent in the present invention, the data being consistent are left, necessarily can be improved row
For the precision of identification.
Using screening sensing data to image data carry out primary screening after, image data can also be carried out into
The screening of one step.For example, (such as can be can be for finding by using the machine learning model as tagsort model
The ant group algorithm of path optimizing), image data is divided into two classes: one kind is the characteristic range obtained by the machine learning model
Compare the data of concentration, it is believed that be " be consistent data ", another kind of is the bigger data of coverage bias, it is believed that is " not phase
Symbol data " are (for example, it may be possible to which the image data prevented since the image of capture is not clear enough is from reflecting true object
Behavior etc.), therefore, it can will be rejected from image data " be not consistent data ", to improve image data and standard feature number
According to accuracy, to improve the precision of Activity recognition.
On the other hand, the machine learning model as disaggregated model can also be first passed through to screen image data, retained special
Sign range compares the image data of concentration, and rejects the biggish image data of characteristic range deviation ratio, and correspondingly delete sensing
Corresponding sensing data portion in data.It is then possible to by by remaining sensing data, Activity recognition result and target
The sensing data that behavior is not inconsistent are rejected, and the sensing data that retention behavior recognition result is consistent with goal behavior, and based on reservation
Sensing data retain corresponding image data in image data, thus filter out the image number of " being consistent with goal behavior "
According to, thus obtain the standard feature data of " being consistent with goal behavior ", thus improve Activity recognition precision.
By above-mentioned data screening mechanism, as described above, the image data of " being consistent with goal behavior " can be obtained, by
This obtains the standard feature data of " being consistent with goal behavior ", thus greatly improves the precision of Activity recognition.
By above-mentioned each step, especially data screening step, more accurate feature database can be obtained, is thus mentioned significantly
High row is accuracy of identification.
Here, smart machine may include for example mobile phone, Intelligent flat, intelligent computer, AR (Augmented Reality,
Augmented reality) various smart machines, the present invention such as equipment, VR (Virtual Reality, virtual reality) equipment do not make volume to this
Outer limitation.
Wherein, the mode that subjects carry hardware sensing device may, for example, be wearing, wearing, carrying etc., this hair
It is bright that this is not intended to be limited in any.
In addition, above-mentioned sensing device can be such as inertial sensor.Inertial sensor can detecte and measure acceleration
The movement such as degree, inclination, shock and vibration, rotation.Certainly, above-mentioned sensing device is also possible to for example individual acceleration sensing
Device, vibrating sensor etc..The present invention does not make additional limitation for the pattern and classification of sensing device itself, as long as it can
Sense user behavior characteristics.
Still further, above-mentioned hardware sensing device, which can for example be dressed, is physically used as body sensor or wears
Face or head are worn over as face or head sensor or is carried at other positions etc. for needing to sense.
Wherein, the behavior of above-mentioned measurand may include conventional behavior act, such as sit, lie, standing, walking,
Run, jump, going upstairs, going downstairs, take a lift upstairs, take a lift downstairs, cycle it is static, run, jump, sitting back and waiting；It can also wrap
Include it is subtle, be difficult to the behavior act distinguished, such as blink, smile, put out one's tongue etc.；Moreover, it is also possible to include complicated row
For combination, such as the combination of conventional behavior, the combination of subtle behavior, or even the combination etc. of conventional behavior and subtle behavior.
In addition, the form of sensing data can be the frequency fluctuation (signal characteristic) sensed.
Still further, the form of image data can be frame image sequence.For example, above-mentioned photographic device is in a period
Object is captured in (such as 0.5 second or 1 second etc.) and does the image of a movement (real object behavior), and generates image sequence
(such as 50 frame images) then can uniformly take several (such as 500) pixels in each frame image, and record institute
There is the position of pixel, then according to the position of institute's capture vegetarian refreshments in this 50 frame image, multiple frame image sequences is obtained, by mentioning
These frame image sequences taken, record and the characteristic (such as the variation track etc. that can be characteristics of image) for retaining image.
Optionally, the above-mentioned machine learning model as tagsort model for example can be the calculation of the ant colony for finding path optimizing
Method etc..
Optionally, for sense data, due to by hardware sensor sense be waveform frequency change, so can
The behavior of object is identified with the waveform frequency variation obtained by it, the Activity recognition result obtained in this way is usually than calibrated
True.
It, can be with for example, the waveform shown in Fig. 2 (signal characteristic) that is sensed of acceleration transducer carried by object
The behavior for determining the object is to walk.In Fig. 2, the longitudinal axis is indicated vertical (Z-direction) acceleration A z (unit is rice/square second), horizontal
Axis indicates time T (unit is the second).
It further, can be from the image number for capturing an obtained real object behavior in behavior identification step S130
According to characteristic, and the characteristic similarity by calculating the standard feature data in this feature data and feature database is extracted, obtain
Matched real object behavior, as corresponding Activity recognition result.
Here, it not necessarily fits like a glove with the standard feature data in feature database due to actually obtaining characteristic, so,
It can be as described above, finding matched characteristic in a manner of using calculating characteristic similarity to obtain matched real object
Behavior.When characteristic similarity is less than some threshold value, it is believed that matching.Conversely, when characteristic similarity about some threshold value,
It is considered that mismatching.It, can be by the real object when all can not find matched standard feature data in entire feature database
Behavior is determined as " abnormal behaviour " (or can be used as " unrecognizable behavior "), in subsequent behavior is matched and is responded,
The real object behavior can be replied or be interacted with some or certain specific respondent behaviors.
It further optionally, can be by distance, such as Euclidean distance, Hamming distance etc., to calculate characteristic
Characteristic similarity between (usually vector data).If similarity is big apart from small；If similar apart from big
It spends small.It is considered that two characteristics are similar, it can think this when characteristic similarity is less than or equal to some threshold value
Two characteristics are matched.It, can when there are the situation of a characteristic and at least one standard feature Data Matching
Using that lesser standard feature data of selected characteristic similarity as matched standard feature data.
Then, optionally, the present invention can also include behavior response step S140.
Here " behavior response " can be the behavior in response to real object, the responsive trip that virtual objects are correspondingly made
For.Matching between respondent behavior and the behavior of real object may include same or like behavior (such as imitate row
For), it also may include opposite behavior, answer sexual behaviour etc., the present invention is not intended to be limited in any this.
In addition, the respondent behavior of virtual objects includes behavior, the movement, expression etc. that virtual objects are made, can also wrap
Include the sound of virtual objects sending, the data drawing list provided etc..That is, present document relates to " virtual objects behavior " be not limited to
Behavior act, and can be the content of any possible visualization and/or Small Enclosure.
It, can be based on the Activity recognition obtained in step S130 as a result, according to for providing in behavior response of step S140
The corresponding table of the behavior of corresponding relationship between object behavior and virtual objects behavior, makes virtual objects make corresponding responsive trip
For.
As a result, in the present invention, by accurate Activity recognition, can greatly improve virtual objects, for object (example
Such as user) matching degree of the respondent behavior of behavior.
Fig. 3 gives the Activity recognition of an exemplary embodiment according to the present invention and the schematic frame of response apparatus
Figure.
As described in Figure 3, the Activity recognition and response apparatus 100 of an exemplary embodiment according to the present invention may include
Following device:
Behavior corresponds to table construction device 110, can be configured for building behavior and correspond to table, the behavior corresponds to table and is used for
Provide the corresponding relationship between real object behavior and virtual objects behavior；
Feature database construction device 120 can be configured for corresponding to each real object row in table for behavior respectively
To acquire image data when subjects repeatedly make the behavior by image capturing device, and make the behavior from each
When the image data that captures in extract the characteristic of the behavior, the standard feature data as the behavior, so that behavior is corresponding
Each real object behavior in table has corresponding multiple standard feature data, and correspond to by behavior it is each really right in table
As behavior and corresponding standard feature data constitutive characteristic library.
Wherein, when subjects repeatedly make each real object behavior, the hardware in subjects is carried
Sensing device senses subjects, obtains the sensing data of its behavior of hardware sensing device senses, and image is caught
It catches device and image capturing is carried out to the subjects simultaneously, obtain the image data for the behavior that it is made.
In addition, above-mentioned feature database construction device 120 may include data screening component 121.
Wherein, data screening component 121 can be configured to using above-mentioned sensing data, for being caught by image
It catches device while the image data captured is screened, with from the image data extraction standard characteristic after screening.
Optionally, the operation of 121 use feeling measured data of the data screening component screening image data includes: based on sensing
The Activity recognitions of data as a result, reject the sensing data not being consistent with goal behavior, and by with the sensing data that are removed simultaneously
The image data of capture is also accordingly rejected, the image data after obtaining the screening.
Optionally, the data screening component 121 may be further configured for so that the image data after the screening is logical
The machine learning model as tagsort model is crossed, obtain being consistent characteristic and the characteristic that is not consistent, by the feature that is consistent
Data are put into feature database as standard feature data, and the characteristic rejecting that will not be consistent.
Optionally, the Activity recognition and response apparatus 100 can also include Activity recognition device 130, can be matched
Set for extracting characteristic from the image data that is captured of behavior for real object, and by calculate this feature data with
The characteristic similarity of standard feature data in the feature database obtains matched standard feature data, and is based on matched mark
Quasi- characteristic obtains real object behavior corresponding with the standard feature data from the feature database, as corresponding behavior
Recognition result.
Optionally, the Activity recognition device 130 is obtained between characteristic by calculating the distance between characteristic
The characteristic similarity.
Optionally, the Activity recognition and response apparatus 100 can also include behavior response device 140, be configured to use
In based on the Activity recognition obtained by Activity recognition device as a result, according between regulation object behavior and virtual objects behavior
The behavior of corresponding relationship correspond to table, so that virtual objects is made corresponding respondent behavior.
Here, above-mentioned device 110-140,121 and steps described above S110-S140, step S121 are similar, herein
Excessive description no longer is done to its details.
In the present invention, the characteristic in the feature database for the Activity recognition of real object is accurately sieved first
Choosing, to ensure the accuracy of the characteristic for Activity recognition, to greatly improve the precision of the Activity recognition of real object.
In addition, for the real object behavior identified, passing through based on real object Activity recognition with high precision
Based on characteristic similarity, in the corresponding table of behavior for providing the corresponding relationship between real object behavior and virtual objects behavior
The middle respondent behavior searched to determine virtual objects can effectively improve between virtual corresponding behavior and real object behavior
Matching degree.
Fig. 4 gives the schematic block diagram of the Intelligent treatment equipment of an exemplary embodiment according to the present invention.
Referring to fig. 4, which includes memory 10 and processor 20.
Processor 20 can be the processor of a multicore, also may include multiple processors.In some embodiments, locate
Reason device 20 may include a general primary processor and one or more special coprocessors, such as graphics processor
(GPU), digital signal processor (DSP) etc..In some embodiments, the circuit realization of customization can be used in processor 20,
Such as application-specific IC (ASIC, Application Specific Integrated Circuit) or scene can
Programmed logic gate array (FPGA, Field Programmable Gate Arrays).
It is stored with executable code on memory 10, when the executable code is executed by the processor 20, makes institute
It states processor 20 and executes one of method described above.Wherein, memory 10 may include various types of storage units, such as
Installed System Memory, read-only memory (ROM) and permanent storage.Wherein, ROM can store processor 20 or computer
The static data or instruction that other modules need.Permanent storage can be read-write storage device.Permanently store dress
Set the non-volatile memory device that the instruction and data of storage will not be lost can be after computer circuit breaking.In some realities
It applies in mode, permanent storage device is used as permanent storage using mass storage device (such as magnetically or optically disk, flash memory).
In other embodiment, permanent storage device can be removable storage equipment (such as floppy disk, CD-ROM drive).In system
It deposits and can be read-write storage equipment or the read-write storage equipment of volatibility, such as dynamic random access memory.Installed System Memory
It can store the instruction and data that some or all processors need at runtime.In addition, memory 10 may include any
The combination of computer readable storage medium, including (DRAM, SRAM, SDRAM, flash memory can for various types of semiconductor memory chips
Program read-only memory), disk and/or CD can also use.In some embodiments, memory 10 may include readable
And/or the removable storage equipment write, such as laser disc (CD), read-only digital versatile disc (such as DVD-ROM, it is double
Layer DVD-ROM), read-only Blu-ray Disc, super disc density, flash card (such as SD card, min SD card, Micro-SD card etc.),
Magnetic floppy disc etc..Computer readable storage medium does not include carrier wave and the momentary electron signal by wirelessly or non-wirelessly transmitting.
In addition, being also implemented as a kind of computer program or computer program product, the meter according to the method for the present invention
Calculation machine program or computer program product include the calculating for executing the above steps limited in the above method of the invention
Machine program code instruction.
Alternatively, the present invention can also be embodied as a kind of (or the computer-readable storage of non-transitory machinable medium
Medium or machine readable storage medium), it is stored thereon with executable code (or computer program or computer instruction code),
When the executable code (or computer program or computer instruction code) by electronic equipment (or calculate equipment, server
Deng) processor execute when, so that the processor is executed each step according to the above method of the present invention.
Those skilled in the art will also understand is that, various illustrative logical blocks, mould in conjunction with described in disclosure herein
Block, circuit and algorithm steps may be implemented as the combination of electronic hardware, computer software or both.
What flow chart and block diagram in attached drawing etc. showed the system and method for multiple embodiments according to the present invention can
The architecture, function and operation being able to achieve.In this regard, each box in flowchart or block diagram can represent a mould
A part of block, program segment or code, a part of the module, section or code include one or more for realizing rule
The executable instruction of fixed logic function.It should also be noted that in some implementations as replacements, the function of being marked in box
It can also be occurred with being different from the sequence marked in attached drawing.For example, two continuous boxes can actually be substantially in parallel
It executes, they can also be executed in the opposite order sometimes, and this depends on the function involved.It is also noted that block diagram and/
Or the combination of each box in flow chart and the box in block diagram and or flow chart, can with execute as defined in function or
The dedicated hardware based system of operation is realized, or can be realized using a combination of dedicated hardware and computer instructions.
Various embodiments of the present invention are described above, above description is exemplary, and non-exclusive, and
It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill
Many modifications and changes are obvious for the those of ordinary skill in art field.The selection of term used herein, purport
In the principle, practical application or improvement to the technology in market for best explaining each embodiment, or make the art
Other those of ordinary skill can understand each embodiment disclosed herein.
Claims (14)
1. a kind of Activity recognition and response method, which is characterized in that the Activity recognition and response method include:
Behavior corresponds to table construction step, constructs for providing the corresponding relationship between real object behavior and virtual objects behavior
Behavior corresponds to table；
Feature database construction step corresponds to each real object behavior in table for behavior respectively, is adopted by image capturing device
Collection subjects repeatedly make the image data when behavior, and extract from the image data captured when making the behavior every time
The characteristic of the behavior, the standard feature data as the behavior, so that behavior corresponds to each real object behavior in table
With corresponding multiple standard feature data, and correspond to by behavior each real object behavior and corresponding mark in table
Quasi- characteristic constitutive characteristic library；
Wherein, the feature database construction step includes data screening step, in the data screening step:
When subjects repeatedly make each real object behavior, the hardware sensing device pair in subjects is carried
The behavior is sensed, and obtains the sensing data of hardware sensing device, and the image capturing device is simultaneously to the row
To carry out image capturing, image data is obtained；
By using the sensing data, the image data captured simultaneously by the image capturing device is screened, with
From the image data extraction standard characteristic after screening.
2. Activity recognition as described in claim 1 and response method, which is characterized in that in the data screening step, make
Include: with sensing data screening image data
Activity recognition based on sensing data as a result, reject the sensing data that are not consistent with goal behavior, and by be removed
The image data that sensing data capture simultaneously is also accordingly rejected, the image data after obtaining the screening.
3. Activity recognition as claimed in claim 2 and response method, which is characterized in that in the data screening step, also
So that the image data after the screening by the machine learning model as tagsort model, obtains being consistent characteristic and
Be not consistent characteristic, and the characteristic that will be consistent is put into feature database as standard feature data, and the characteristic that will not be consistent is picked
It removes.
4. Activity recognition and response method as described in any one in claims 1 to 3, which is characterized in that the behavior
Identification and response method further include:
Activity recognition step, the image data captured from the behavior for real object extracts characteristic, and passes through calculating
The characteristic similarity of standard feature data in this feature data and the feature database, obtains matched standard feature data, and
Based on matched standard feature data, real object behavior corresponding with the standard feature data is obtained from the feature database, is made
For corresponding Activity recognition result.
5. Activity recognition as claimed in claim 4 and response method, which is characterized in that by calculate characteristic between away from
From, obtain characteristic between the characteristic similarity.
6. Activity recognition as claimed in claim 4 and response method, which is characterized in that the Activity recognition and response method are also
Include:
Behavior response step, based on the Activity recognition obtained in behavior identification step as a result, according to for regulation object behavior with
The behavior of corresponding relationship between virtual objects behavior corresponds to table, and virtual objects is made to make corresponding respondent behavior.
7. a kind of Activity recognition and response apparatus, which is characterized in that the Activity recognition and response apparatus include:
Behavior corresponds to table construction device, is configured for building behavior and corresponds to table, it is true right for providing that the behavior corresponds to table
As the corresponding relationship between behavior and virtual objects behavior；
Feature database construction device is configured for corresponding to each real object behavior in table for behavior respectively, passes through image
Image data when capture device acquisition subjects repeatedly make the behavior, and from the image captured when making the behavior every time
The characteristic of the behavior, the standard feature data as the behavior are extracted in data, so that behavior corresponds to each of table very
Real object behavior has corresponding multiple standard feature data, and by behavior correspond to each real object behavior in table and with
Its corresponding standard feature data constitutive characteristic library；
Wherein, when subjects repeatedly make each real object behavior, the hardware sensing in subjects is carried
Device senses the behavior, obtains the sensing data of hardware sensing device, and the image capturing device is right simultaneously
The behavior carries out image capturing, obtains image data,
Wherein, the feature database construction device includes data screening component, and the data screening component is configured for:
By using the sensing data, the image data captured simultaneously by the image capturing device is screened, with
From the image data extraction standard characteristic after screening.
8. Activity recognition as claimed in claim 7 and response apparatus, which is characterized in that the data screening component uses sensing
The operation of data screening image data includes:
Activity recognition based on sensing data as a result, reject the sensing data that are not consistent with goal behavior, and by be removed
The image data that sensing data capture simultaneously is also accordingly rejected, the image data after obtaining the screening.
9. Activity recognition as claimed in claim 8 and response apparatus, which is characterized in that the data screening component is also configured
For making the image data after the screening by the machine learning model as tagsort model, the characteristic that is consistent is obtained
According to the characteristic that is not consistent, the characteristic that will be consistent is put into feature database as standard feature data, and the characteristic that will not be consistent
According to rejecting.
10. Activity recognition and response apparatus as described in any one in claim 7~9, which is characterized in that the behavior
Identification and response apparatus further include:
Activity recognition device, the image data for being configured for being captured from the behavior for real object extract characteristic,
And the characteristic similarity by calculating the standard feature data in this feature data and the feature database, it is special to obtain matched standard
Data are levied, and are based on matched standard feature data, it is corresponding with the standard feature data true right to obtain from the feature database
As behavior, as corresponding Activity recognition result.
11. Activity recognition as claimed in claim 10 and response apparatus, wherein the Activity recognition device is by calculating feature
The distance between data obtain the characteristic similarity between characteristic.
12. Activity recognition as claimed in claim 10 and response apparatus, which is characterized in that the Activity recognition and response apparatus
Further include:
Behavior response device is configured for the Activity recognition obtained by Activity recognition device as a result, according to for providing
The corresponding table of the behavior of corresponding relationship between object behavior and virtual objects behavior, makes virtual objects make corresponding responsive trip
For.
13. a kind of Intelligent treatment equipment, comprising:
Processor；And
Memory is stored thereon with executable code, when the executable code is executed by the processor, makes the processing
Device executes the method as described in any one of claim 1~6.
14. a kind of non-transitory machinable medium, is stored thereon with executable code, when the executable code is located
When managing device execution, the processor is made to execute the method as described in any one of claim 1~6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910507405.7A CN110378214B (en) | 2019-06-12 | 2019-06-12 | Behavior feature processing method, behavior feature processing device, intelligent processing device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910507405.7A CN110378214B (en) | 2019-06-12 | 2019-06-12 | Behavior feature processing method, behavior feature processing device, intelligent processing device and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110378214A true CN110378214A (en) | 2019-10-25 |
| CN110378214B CN110378214B (en) | 2021-08-20 |
Family
ID=68250129
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910507405.7A Active CN110378214B (en) | 2019-06-12 | 2019-06-12 | Behavior feature processing method, behavior feature processing device, intelligent processing device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110378214B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103984036A (en) * | 2013-02-08 | 2014-08-13 | 索尼公司 | Screening method for operating a plurality of screening sensors, and screening system |
| CN104463087A (en) * | 2013-11-25 | 2015-03-25 | 安徽寰智信息科技股份有限公司 | Action identification device and method |
| CN105551182A (en) * | 2015-11-26 | 2016-05-04 | 吉林大学 | Driving state monitoring system based on Kinect human body posture recognition |
-
2019
- 2019-06-12 CN CN201910507405.7A patent/CN110378214B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103984036A (en) * | 2013-02-08 | 2014-08-13 | 索尼公司 | Screening method for operating a plurality of screening sensors, and screening system |
| CN104463087A (en) * | 2013-11-25 | 2015-03-25 | 安徽寰智信息科技股份有限公司 | Action identification device and method |
| CN105551182A (en) * | 2015-11-26 | 2016-05-04 | 吉林大学 | Driving state monitoring system based on Kinect human body posture recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110378214B (en) | 2021-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230206720A1 (en) | System for counting quantity of game tokens | |
| CN105617638B (en) | Badminton racket swing action identification method and device | |
| US9489042B2 (en) | Scenario-specific body-part tracking | |
| CN108352072A (en) | Object tracking methods, object tracking apparatus and program | |
| CN106780906A (en) | A kind of testimony of a witness unification recognition methods and system based on depth convolutional neural networks | |
| CN108932479A (en) | A kind of human body anomaly detection method | |
| US20120040727A1 (en) | Table game tracking | |
| US20060252554A1 (en) | Gaming object position analysis and tracking | |
| CN104346802B (en) | A kind of personnel leave the post monitoring method and equipment | |
| Ganiyusufoglu et al. | Spatio-temporal features for generalized detection of deepfake videos | |
| CN109086711A (en) | Facial Feature Analysis method, apparatus, computer equipment and storage medium | |
| CN111914633B (en) | Face-changing video tampering detection method based on face characteristic time domain stability and application thereof | |
| US20190180149A1 (en) | System and method of classifying an action or event | |
| CN109086734A (en) | The method and device that pupil image is positioned in a kind of pair of eye image | |
| CN109543526A (en) | True and false facial paralysis identifying system based on depth difference opposite sex feature | |
| Niitani et al. | Sampling techniques for large-scale object detection from sparsely annotated objects | |
| CN103279744B (en) | Imitation fingerprint detection methods based on multiple dimensioned three mode texture feature and system | |
| JP2010072723A (en) | Tracking device and tracking method | |
| CN110268440A (en) | Image analysis apparatus, method for analyzing image and image analysis program | |
| CN109271848A (en) | A kind of method for detecting human face and human face detection device, storage medium | |
| CN107305693A (en) | Information processor, information processing method and storage medium | |
| JP7088281B2 (en) | Product analysis system, product analysis method and product analysis program | |
| Bayat et al. | Scene grammar in human and machine recognition of objects and scenes | |
| CN112560584A (en) | Face detection method and device, storage medium and terminal | |
| CN113822254B (en) | Model training method and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |