CN109144349A - One kind is virtual, enhance or mixed reality head shows direction of motion recognition methods and system - Google Patents
One kind is virtual, enhance or mixed reality head shows direction of motion recognition methods and system Download PDFInfo
- Publication number
- CN109144349A CN109144349A CN201810890022.8A CN201810890022A CN109144349A CN 109144349 A CN109144349 A CN 109144349A CN 201810890022 A CN201810890022 A CN 201810890022A CN 109144349 A CN109144349 A CN 109144349A
- Authority
- CN
- China
- Prior art keywords
- user
- virtual
- axis
- mixed reality
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
- G06F18/24155—Bayesian classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
Abstract
The invention discloses a kind of virtual, enhancings or mixed reality head to show direction of motion recognition methods and system, this method comprises: guidance user is mobile, collects user's mobile data and establishes data set；Data set after optimization is stored and establishes disaggregated model；Judge and set the initial position of user；Whether system starts to move according to threshold decision user, and user starts to move, then collects record mobile data；System according to threshold decision user whether tenth skill, user's tenth skill, then stop collect record mobile data；Data set is substituted into disaggregated model, judges user movement direction, system provides the direction of motion, completes direction of motion judgement；According to the user movement direction of judgement, the corresponding function of system trigger.Method provided by the invention overcomes the defect that existing method is not easy to operate, expensive and accuracy is not high, realizes low cost, high efficiency and low misjudgment rate.
Description
Technical field
The present invention relates to virtual, enhancing and mixed reality fields, and in particular to one kind is virtual, enhance or mixed reality head is aobvious
Direction of motion recognition methods and system.
Background technique
Direction of motion identification is a branch under action recognition field, it can be used in 1) human-computer interaction；2) it visualizes
And selection technique；3) fields such as remote health monitoring system.Realize the direction of motion identify maximum challenge first is that how quick
Accurately identify the direction of motion of user.For this problem, there is following correlative study in the prior art.
The Doppler shift information that exercise induced is extracted using the Wi-Fi router of business level, is completed to the direction of motion
Anticipation.Its key is carefully to eliminate random frequency displacement using antenna diversity, while keeping relevant Doppler frequency shift.This method
Need for three Wi-Fi routers to be placed in one 3 meters × 3 meters of grid edge.Using this method, to 8 directions of user
On moving direction judging nicety rate reach 92%.However, this method needs one 3 meters × 3 meters of realistic space, and arrange 3
A Wi-Fi router, this is not easy to operate and unrealistic.
The moving direction of user indoors is analyzed using the Inertial Measurement Unit built in mobile phone, utilizes inertia measurement
Unit has calculated the Eulerian angles numerical value in X, Y, Z3 directions, and collects and analyze user and move towards 8 directions of motion
When Eulerian angles variation.91.42% is up to using the accuracy of random forest grader.But rotation model of the equipment in trouser pocket
It is with limit, numerical value of the Eulerian angles in head-mounted display can not be as reference.
Although existing method can be realized direction of motion identification, but there are certain defects.In addition to cannot directly apply to
In virtually/enhancing/mixed reality, the problems such as there is also not easy to operate, price is relatively expensive, accuracy is not high.Therefore, it is necessary to mention
A kind of scheme for the direction of motion identification can be used in head-mounted display out.The limitation of " attachment constraint " should be got rid of, again side
Just user uses, while also to guarantee identification accuracy.
Summary of the invention
Object of the present invention is to: virtual one kind, enhancing or mixed reality head are provided and show direction of motion recognition methods, is overcome existing
The defect that method is not easy to operate, expensive and accuracy is not high is realized low cost, high efficiency and low misjudgment rate and can be used
The purpose of menu selection, navigation or the visualization under three-dimensional environment and selection is carried out in virtual, enhancing or mixed reality.
The technical scheme is that a kind of virtual, enhancing or mixed reality head show direction of motion recognition methods, it is based on head
Display and its corollary equipment are worn, the direction of motion recognition methods includes:
Guide user by prompt towards the mobile M different distance level scales in N number of direction, the N is the integer more than or equal to 2, and M is big
In the integer for being equal to 1, collects user's mobile data and establish data set；
Optimize the data set, and the data set after optimization is stored and establishes disaggregated model；
Judge and set the initial position of user；
Whether system starts to move according to threshold decision user, if it is determined that user starts to move, then collects record mobile data；
According to threshold decision user, whether tenth skill then stops collecting record movement if it is determined that user's tenth skill system
Data；
According to collected mobile data, data set is substituted into disaggregated model, system recommends movement most like out according to feature
Direction judges user movement direction；According to recommending data, system finally provides the direction of motion, completes direction of motion judgement；
According to the user movement direction of judgement, the corresponding function of system trigger.
In one embodiment, the data set includes user's any movement in either direction in N number of direction
Moving distance is poor；The movement speed of this movement；The acceleration of this movement；This displacement for being projected in X, Z both direction of movement away from
From；For distance change in X-axis divided by slope obtained by the distance change on Z axis, described X, Z are the horizontal plane of rectangular coordinate system in space
Two reference axis.
In one embodiment, N number of direction and M different distance level scales be specially east, south, west, north, northeast,
Northwest, southwest and the direction of the southeast 8 and the closely movement, remote mobile two distance level scales of each direction.
In one embodiment, the method for the optimization data set is as follows:
It is mobile for short distance, it filters X-axis or Z axis moving distance is less than the data of the mobile threshold value of the short distance set, retain big
In the data set of the threshold value；
For moving at a distance, filters X-axis or Z axis moving distance is less than the data of the remote mobile threshold value set, retain big
In the data set of the threshold value；
The threshold size is set as adaptively being adjusted according to user's own situation, it is described adjustment based on user closely and
The peak value moved at a distance.
In one embodiment, the disaggregated model uses k-nearest neighbor classifier.
In one embodiment, the K of the k-nearest neighbor is equal to 4.
In one embodiment, the method for judgement user's initial position are as follows: when user terminates mobile recurrence starting point,
System obtains location information at this time and compared with the location information before setting out, when front-rear position is at three-dimensional space X, Y and Z tri-
When alternate position spike is respectively less than the threshold speed of speed and setting on the alternate position spike threshold value set while three directions on direction, determine
Initial position is returned to, and sets position at this time as initial position.
In one embodiment, the alternate position spike threshold value is 0.05m, and the threshold speed is 0.2m/s.
In one embodiment, described whether to start mobile criterion according to threshold decision user are as follows: three-dimensional space
The absolute value of moving distance is greater than the threshold value of setting in interior X-axis or Z axis.Reach condition, system determine user start it is mobile and
Start to collect exercise data, data include moving distance and speed and acceleration of the user in three-dimensional space in X, Y, Z axis
Degree, and calculate the moving distance in three-dimensional space in X-axis compared to the slope of the moving distance on Z axis and it is total it is mobile away from
From.
In one embodiment, it is described according to threshold decision user whether the criterion of tenth skill are as follows: three-dimensional space
Interior X-axis or Z axis moving distance are greater than the moving distance threshold value and the speed of user's all directions of setting and the speed threshold less than setting
Value reaches condition, and system determines user i.e. by stop motion, and data are collected in pause.
In one embodiment, the moving distance threshold value and threshold speed are in short distance movement and remote situation of movement
Lower threshold value is of different sizes,
Mobile for short distance, the nearly moving distance that X-axis or Z axis moving distance are greater than setting in three-dimensional space terminates threshold value and use
The speed of family all directions and terminate threshold value less than the short distance movement speed of setting；
For moving at a distance, the remote moving distance that X-axis or Z axis moving distance are greater than setting in three-dimensional space terminates threshold value and use
The speed of family all directions and terminate threshold value less than the remote movement speed of setting；
The threshold size is set as adaptively being adjusted according to user's own situation.
In one embodiment, the substitution disaggregated model judges the move distance etc. that user is also judged when the direction of motion
Grade, and corresponding function is triggered according to the direction of motion of user and distance level scale in the next steps.
In one embodiment, by the head-mounted display and its corollary equipment built in Inertial Measurement Unit sensor
The mobile data of user is obtained with depth transducer.
In one embodiment, the guidance user is mobile, when collecting mobile data and establishing data set, uses 3D arrow
It prompts the user whether also to need mobile completion data collection.
In one embodiment, the recognition methods is when completing the user movement direction of judgement and triggering corresponding function
To frame out or sound feedback, it is additionally provided with following steps after the completion: judging whether user instruction terminates, if so, task is complete
Finish；If it is not, the step of then returning to judgement and setting user's initial position, repeats subsequent step, until instruction terminates.
The embodiment of the invention also provides a kind of computer readable storage medium, the computer-readable recording medium storage
There is the computer program for executing virtual, enhancing described in any of the above embodiments or the aobvious direction of motion recognition methods of mixed reality head.
The embodiment of the invention also provides a kind of virtual, enhancings or mixed reality head to show direction of motion identifying system, comprising:
Data collection module, including Inertial Measurement Unit sensor and depth transducer, for acquiring the mobile data of user；
Display module is worn, for showing virtual, enhancing or mixed reality space and interactive interface；
Control and memory module, including processor and memory, the memory, which is stored with, executes void described in any of the above embodiments
The computer program of quasi-, enhancing or the aobvious direction of motion recognition methods of mixed reality head, the processor by execute it is described based on
Calculation machine program.
The invention has the advantages that proposing what the direction of motion under a kind of virtual/enhancing/mixed reality head-mounted display identified
Method.This method can replace the controller under virtual/enhancing/mixed reality at this stage, for virtual/enhancing/mixed reality
Middle carry out menu selection, navigation or visualization and selection under three-dimensional environment.The technology can be implanted in exercise type game
(Exergame), such as, the present invention has been used, user is without using dance rug, the equipment such as dancing machine.Based on we
Method, we design and develop a dancing and game and corresponding menu interface.Wherein, our dancing and game is based on user's
The direction of motion, user complete the movement in corresponding sports direction as indicated, and game can identify simultaneously the movement that user makes
Feedback (sound, vision) and reward (score) are provided according to the performance of user.Our dancing and game is bought compared to needs and is jumped
The common dancing and game of blanket, dancing machine is waved, advantage is not needing the additional purchase relevant device of user, and simultaneously operation index is more
Multiplicity, the exercise for giving user are more obvious.And in effect performance, our invention combines machine learning techniques,
Judge that the precision in user movement direction is high and speed is fast.
The present invention takes full advantage of the included depth transducer and Inertial Measurement Unit of virtual/enhancing/mixed reality equipment,
The comprehensive method for using machine learning, the present invention realize inexpensive (not needing to buy extras), high efficiency and low misdeem
Accidentally the direction of motion of rate knows method for distinguishing.Hand held data entry device is eliminated using the comprehensive circular menu of the identification in body kinematics direction
Demand.When these equipment are unavailable or inconvenient in use, the present invention is particularly useful.Meanwhile the present invention can also be used in exercise type
Game (Exergame) allows user to be in and tempers body while playing game in the case where buying redundant equipment without user
Body, low cost and beneficial physical and mental health.
Detailed description of the invention
The invention will be further described with reference to the accompanying drawings and embodiments:
Fig. 1 is the direction of motion recognition methods flow chart that the virtual embodiment of the present invention, enhancing or mixed reality head are shown；
When Fig. 2 is that the embodiment of the present invention uses virtual, enhancing or mixed display head-mounted display, user makes long distance backward repeatedly
Head position data variation line chart from shift action；
When Fig. 3 is that the embodiment of the present invention uses virtual, enhancing or mixed display head-mounted display, user makes long distance backward repeatedly
Head speed data from shift action changes line chart；
When Fig. 4 is that the embodiment of the present invention uses virtual, enhancing or mixed display head-mounted display, user makes long distance backward repeatedly
Head acceleration data variation line chart from shift action；
Fig. 5 is the embodiment of the present invention when using k-nearest neighbor (default parameters) on Weka software, 10 folding cross validation categorization results
Accuracy and classification confusion matrix result screenshot；
Fig. 6 is the embodiment of the present invention when using Naive Bayes Classifier (default parameters) on Weka software, and 10 foldings intersection is tested
Demonstrate,prove the accuracy of categorization results and the confusion matrix result screenshot of classification；
Fig. 7 is the embodiment of the present invention when using support vector machines (default parameters) on Weka software, and 10 folding cross validations are sorted out
As a result the confusion matrix result screenshot of accuracy and classification；
Fig. 8 is the embodiment of the present invention when using random tree (default parameters) on Weka software, 10 folding cross validation categorization results
Accuracy and classification confusion matrix result screenshot；
Fig. 9 is the embodiment of the present invention when using k-nearest neighbor (parameter k=4) on Weka software, 10 folding cross validation categorization results
Accuracy and classification confusion matrix result screenshot；
Figure 10 is the round interactive interface that the embodiment of the present invention is used cooperatively；
Figure 11 is the round navigation interface that the embodiment of the present invention is used cooperatively；
Figure 12 is that user of the embodiment of the present invention uses schematic diagram.
Specific embodiment
The principle and features of the present invention will be described below with reference to the accompanying drawings, and the given examples are served only to explain the present invention, and
It is non-to be used to limit the scope of the invention.
The direction of motion recognition methods flow chart that virtual, enhancing as shown in Figure 1 or mixed reality head are shown, the virtual, increasing
The aobvious direction of motion recognition methods of strong or mixed reality head, is based on head-mounted display and its corollary equipment, and the direction of motion is known
Other method includes the following steps: to guide user by prompt towards the mobile M different distance level scales in N number of direction, the N be greater than etc.
In 2 integer, M is the integer more than or equal to 1, collects user's mobile data and establishes data set；Optimize the data set, and will
Data set after optimization stores and establishes disaggregated model；Judge and set the initial position of user；System is used according to threshold decision
Whether family, which starts, is moved, if it is determined that user starts to move, then collects record mobile data；System is according to threshold decision user
No tenth skill, if it is determined that user's tenth skill, then stop collecting record mobile data；According to collected mobile data,
Data set is substituted into disaggregated model, system recommends the direction of motion most like out according to feature, judges user movement direction；According to
Recommending data, system finally provide the direction of motion, complete direction of motion judgement；According to the user movement direction of judgement, system touching
Send out function corresponding, it is preferred that frame out or sound feedback can also be given simultaneously.The guidance user is mobile and collects mobile number
Initial data set is established according to being, system needs to identify how many a directions, and user will be guided to complete how many when initial
The movement of corresponding direction can also move repeatedly different distances, two features in direction and distance as needed in each direction
A corresponding option, i.e. a data set, the quantity of data set can be such as most simple with arbitrary extension or compression according to practical use
The identification of single realization both direction, then only need to collect the movement of user in two directions.If necessary to east, south, west,
North, northeast, northwest, southwest and the direction of the southeast 8, the different moving distances in 2, each direction, then just need to guide in the vectoring phase
User is mobile to 8 directions, closely movement, at a distance mobile two distance level scales, i.e. 16 movements of each direction, and collects
The data moved each time.It can be with after above-mentioned steps in order to facilitate use this method identification direction of user's circulation
Include the following steps: to judge whether user instruction terminates, if so, task finishes；If it is not, at the beginning of then returning to judgement and setting user
The step of beginning position, repeats subsequent step, until instruction terminates.
In numerous embodiments of above-described embodiment, the exercise data of user in all directions is collected.Data set
(dataset) include following characteristics (attribute): head in space the movement in three-dimensional space on 3 directions (XYZ axis) away from
From (formula 1), the acceleration of the movement speed (formula 2) in head 3 directions in space, head 3 directions in three dimensions (comes
From Inertial Measurement Unit), slope (formula 3) of the moving distance in x-axis compared to the moving distance in z-axis in three-dimensional space
With the shift value (formula 4) of X-axis in three-dimensional space and Z axis and precision analysis is classified to it with polyalgorithm.
Moving distance is poor (X, Y, Z coordinate axis are applicable in):(formula 1)
In formulaFor user's coordinate position in X-axis at present,Start the mobile preceding seat in X-axis for user
Cursor position,The distance moved in X-axis for user；
Speed (X, Y, Z coordinate axis are applicable in):(formula 2)
In formulaThe speed for being user in X-axis,For the distance that user moves in X-axis, t is this frame of software to upper
Time difference used in one frame；
Slope:(formula 3)
K is slope in formula,It is the distance that user moves in X-axis,It is the distance that user moves on Z axis；
Total moving distance:(formula 4)
In formulaFor total moving distance,It is the distance that user moves in X-axis,It is the distance that user moves on Z axis；
As shown in Fig. 2, the number fluctuation in three-dimensional space on Z axis (front-rear direction) is the most obvious, most represents and moves this backward
One behavior.Since user will appear the slightly unstable situation of body, the data in three-dimensional space in X-axis (left and right directions) are deposited
In error by a small margin, but fluctuating range is significantly less than Z axis.Numerical fluctuations situation and Z in three-dimensional space Y-axis (up and down direction)
Axis is consistent, and when user, which steps paces, reaches maximum, Y-axis numerical value reduces most.Because what Y-axis represented is changed in height
Difference, when user steps paces, the position of head-mounted display reaches minimum point naturally, when user, which steps back, to stand erectly, from
So return to highest point (difference is minimum).
What Fig. 3 and Fig. 4 was respectively represented is the situation of change for once stepping head velocity and acceleration in the work that reverses backward.
Calculation method thinks that the direction of motion is positive value forward, and the direction of motion is negative value backward, therefore the velocity variations in figure are after first bearing
Just, this fluctuation is conducive to distinguish instead and be moved forward and backward.This moves generated broken line to simple analysis backward, wears
The movement speed of display is consistent with the fluctuation situation of position change amount, and the speed in three-dimensional space in Y-axis reaches minimum value
Z axis speed reaches maximum value simultaneously and Z axis velocity variations are maximum.Inertia measurement list of the acceleration value from head-mounted display
Member has carried out smoothing processing to noise present in data (noise) using Kalman filtering, shown in Fig. 4, X in three-dimensional space
Acceleration on axis remains unchanged substantially, and Z axis acceleration can be moved back because of user, fluctuation occurs.It is therefore preferred that the number
It include that the moving distance of any movement is poor in either direction in N number of direction by user according to collection；The movement speed of this movement；This
Mobile acceleration；This moves the shift length projected in X, Z both direction；Distance change in X-axis divided by Z axis away from
From variation gained slope, described X, Z are two reference axis of the horizontal plane of rectangular coordinate system in space.Features described above in mobile data
Data are most effective accurate in the algorithm for judging moving direction and distance.
There are many method for building up of disaggregated model described in above-described embodiment, Fig. 5, and 6,7,8 be to utilize k-nearest neighbor, simple shellfish
The classification results of Ye Si, support vector machines and random tree use 10 folding cross validations, and append confusion matrix result figure,
Wherein it is minimum (less than 0.01 second) not only to model the time for k-nearest neighbor, and 10 folding cross validation accuracy highests, reaches
99.977%.Therefore, in real-time measurement, k nearest neighbour method is also used.Parameter is also adjusted simultaneously, when k-nearest neighbor k=4
When can obtain optimal accuracy, reach 99.846%, as a result see Fig. 9.Therefore preferably k-nearest neighbor classifier establishes classification mould
Type, it is further preferred that K=4.
In one embodiment specifically, influencing machine since the data closely moved at a distance in the same direction have overlapping
The accuracy rate that device learning algorithm determines.Data set can be advanced optimized, the method for the optimization data set is as follows: for close
Distance is mobile, filters X-axis or Z axis moving distance is less than the data of the mobile threshold value of the short distance set, such as 0.1 meter, retain big
In the data set of the threshold value；For moving at a distance, filters X-axis or Z axis moving distance is less than the remote mobile threshold value of setting
Data, such as 0.3 meter, remain larger than the data set of the threshold value；The threshold size be set as according to user's own situation into
The adaptive adjustment of row, the peak value that the adjustment closely and is at a distance moved based on user.
When calibrating the initial position of user is a problem for needing to overcome.The supporting leg of user can be in moving process
There is offset phenomena, the position of recurrence is not the position started before movement.Further, since the prior art technically lack
It falls into, even if supporting leg does not deviate, still there are miss caused by noise (noise) according to the calculated position of sensor for algorithm
Difference.
Using formula one and formula two, in three-dimensional space on tri- directions xyz alternate position spike and speed calculated,
As long as the alternate position spike in three-dimensional space on xyz axis is less than 0.05 meter, while speed in three-dimensional space on tri- directions xyz and small
In 0.2 meter per second.Determine that user returns " initial " position, system is ready for the calibration for " initial " position, new position is regarded
For " initial " position, accuracy rate is promoted.In embodiment specifically, the method for judgement user's initial position are as follows: Yong Hujie
When Shu Yidong returns starting point, system obtains location information at this time and compared with the location information before setting out, and works as front-rear position
On tri- directions three-dimensional space X, Y and Z alternate position spike be respectively less than the alternate position spike threshold value set simultaneously the speed on three directions and
When the threshold speed of setting, judgement returns to initial position, and sets position at this time as initial position.Preferably, the position
Poor threshold value is 0.05m, and the threshold speed is 0.2m/s.To optimize the judgement of initial position.
In one embodiment specifically, whether described start mobile criterion according to threshold decision user are as follows: three
The absolute value of moving distance is greater than the threshold value of setting on X-axis or Z axis in dimension space, and the preferred threshold value is 0.1m.Reach item
Part, system determine that user starts to move and start to collect exercise data, and data include user in three-dimensional space in X, Y, Z axis
Moving distance and velocity and acceleration, and calculate the moving distance in three-dimensional space in X-axis compared on Z axis
The slope of moving distance and total moving distance.
In one embodiment specifically, it is described according to threshold decision user whether the criterion of tenth skill are as follows: three
X-axis or Z axis moving distance are greater than the moving distance threshold value and the speed of user's all directions of setting and less than setting in dimension space
Threshold speed reaches condition, and system determines user i.e. by stop motion, and data are collected in pause.Preferably, short distance is moved
It is dynamic, the nearly moving distance that X-axis or Z axis moving distance are greater than setting in three-dimensional space terminate threshold value and the speed of user's all directions and
Short distance movement speed less than setting terminates threshold value；Preferably, it is 0.2m, the low coverage that the nearly moving distance, which terminates threshold value,
Terminating threshold value from movement speed is 0.2m/s.For moving at a distance, X-axis or Z axis moving distance are greater than setting in three-dimensional space
Remote moving distance terminate the speed of threshold value and user's all directions and terminate threshold value less than the remote movement speed of setting；It is described
Threshold size is set as adaptively being adjusted according to user's own situation.Preferably, the remote moving distance terminates threshold value and is
0.45m, it is 0.2m/s that the remote movement speed, which terminates threshold value,.The program facilitates the movement side that system prejudges user in advance
To and it is close remote, make system before user does not also return " initial " position, just realize user movement direction and closely remote judgement.
In one embodiment specifically, the substitution disaggregated model judges the movement for also judging user when the direction of motion
Distance level scale, and corresponding function is triggered according to the direction of motion of user and distance level scale in the next steps.
In one embodiment specifically, passing through the Inertial Measurement Unit built in the head-mounted display and its corollary equipment
Sensor and depth transducer obtain the mobile data of user.Hardware configuration: the present invention can be based on existing equipment, such as:
Meta 2 (augmented reality head-mounted display), Oculus Rift (virtual reality head-mounted display) or Samsung odyssey(are mixed
Close display head-mounted display).For Meta2 and Samsung odyssey, the present invention uses the Inertial Measurement Unit (IMU) built in it
Sensor and depth transducer judge the current position of user, user's head movement speed and acceleration；For Oculus
Rift, the present invention using built in it Inertial Measurement Unit (IMU) sensor and subsidiary two depth transducers judge to use
Account portion position and user's head movement speed and acceleration.
In one embodiment specifically, the guidance user is mobile, when collecting mobile data and establishing data set, use
3D arrow prompt the user whether also to need it is mobile complete data collection, in the next steps can also each step all to user into
The prompt such as row sound, picture or vibration, enables a user to smoothly complete relevant operation.
It can be seen that the direction of motion recognition methods that the application is a kind of virtual, enhancing or mixed reality head are aobvious both got rid of it is " attached
The limitation of part constraint ", but it is user-friendly, while also guaranteeing identification accuracy.
On the other hand the application also provides a kind of computer readable storage medium, the computer-readable recording medium storage
Have and executes the computer program that virtual any of the above-described described one kind, enhancing or mixed reality head show direction of motion recognition methods.
On the other hand the application also provides virtual one kind, enhancing or mixed reality head and shows direction of motion identifying system, comprising:
Data collection module, including Inertial Measurement Unit sensor and depth transducer, for acquiring the mobile data of user；It wears aobvious
Show module, for showing virtual, enhancing or mixed reality space and interactive interface；Control and memory module, including processor and
Memory, the memory are stored with the direction of motion knowledge that execution one kind described above is virtual, enhancing or mixed reality head are aobvious
The computer program of other method, the processor can be built-in for executing the computer program, the data collection module
Display module is worn in described, also can be made independent external component, the control and memory module can be external meter
Calculate machine equipment or be built in head-mounted display, the data collection module and wear display module with control and memory module
Establish communication connection.
A kind of virtual, enhancing of above-described embodiment or mixed reality head show direction of motion recognition methods, solve virtual, enhancing
Or direction of motion identification under mixed reality head-mounted display, position correction, without extras interface alternation the problems such as.Mainly with
The coordinate position of user's head-mounted display and the features such as the movement speed of head-mounted display and acceleration are data set, with machine
The method of study realizes quickly accurate direction of motion identification.Implementation method is as follows in concrete case:
Case one: in conjunction with the use of round interactive interface, it may replace existing remote controler；
Figure 10 is circular menu interface, and the application method in conjunction with round interactive menu is introduced below.
Assuming that user wants selection " weather ", user steps forward a small step first, and head is suitably displaced holding with body
Balance, when head movement distance in three-dimensional space X-axis or Z axis more than 0.1 meter of threshold value, system judgement user start to move, and
Start to collect exercise data (three-dimensional space head portion moving distance is poor, head movement speed, head translational acceleration).
The leg of taking a step of user is stepped on the moment (criterion are as follows: X-axis or Z axis are greater than 0.2 meter in three-dimensional space of target position
And the speed of user's all directions and less than 0.2 meter per second, wherein the distance threshold present invention can adjust automatically according to user situation
It is whole), system according to collected real-time motion data, using before classification model automatic identification the direction of motion and paces it is big
It is small.
System successfully determines that user wants selection " weather " function, and " weather " function is automatically loaded, and simultaneity factor issues choosing
Select audio.
Case two and scheme one are almost the same: the navigation under virtual, enhancing or mixed display head-mounted display；
Figure 11 is round navigation interface, is introduced below and realizes navigation at VR in conjunction with round navigation interface.
Assuming that user wants selection, direction is moved at a distance northeastward, and user is first to right front huge step, head
Holding balance, when X-axis or Z axis are more than 0.1 meter of threshold value to head movement distance in three-dimensional space, system are suitably displaced with body
Determine user start to move, and start collect exercise data (three-dimensional space head portion moving distance is poor, head movement speed, head
Portion's translational acceleration).
The leg of taking a step of user is stepped on the moment (criterion are as follows: X-axis or Z axis are greater than 0.2 meter in three-dimensional space of target position
And the speed of user's all directions and less than 0.2 meter per second, wherein the distance threshold present invention can adjust automatically according to user situation
It is whole), system according to collected real-time motion data, using before classification model automatic identification the direction of motion and paces it is big
It is small.
System successfully determines that user wants to select northeastward remote, and control personage navigates from the trend direction.
Case three: the use in virtual reality dancing and game.
As Figure 12 shows that user's completion moves right the movement completion figure of a small step and major step.
User's small step to the right first, head is suitably displaced holding balance with body, when head movement distance is in X-axis or Z
For axis more than 0.1 meter of threshold value, system determines that user starts to move, and start to collect exercise data (three-dimensional space head portion it is mobile away from
Deviation, head movement speed, head translational acceleration).
The leg of taking a step of user is stepped on the moment (criterion are as follows: X-axis or Z axis are greater than 0.2 meter in three-dimensional space of target position
And the speed of user's all directions and less than 0.2 meter per second, wherein the distance threshold present invention can adjust automatically according to user situation
It is whole), system according to collected real-time motion data, using before classification model automatic identification the direction of motion and paces it is big
It is small.
If user action is correct, score is given, and feedback (instruction Icon Color changes, and celebration sound occurs in background) is provided；
If user's not execution as indicated, prompts current action mistake, not score, and provides feedback and (indicate that Icon Color changes
Become, encouragement sound occurs in background).
Subsequent user returns to initial position, system calibration initial position, criterion: the position in three-dimensional space on xyz axis
It sets difference and is respectively smaller than 0.05 meter and the speed on xyz axis and less than 0.2 meter per second.
Next instruction movement is the major step that moves right.
User is execution, to the right huge step, and head is suitably displaced holding balance with body, when head it is mobile away from
From X-axis in three-dimensional space or Z axis more than 0.1 meter of threshold value, system determines that user starts to move, and starts to collect exercise data
(three-dimensional space head portion moving distance is poor, head movement speed, head translational acceleration).
The leg of taking a step of user is stepped on the moment (criterion are as follows: X-axis or Z axis are greater than 0.2 meter in three-dimensional space of target position
And the speed of user's all directions and less than 0.2 meter per second, wherein the distance threshold present invention can adjust automatically according to user situation
It is whole), system according to collected real-time motion data, using before classification model automatic identification the direction of motion and paces it is big
It is small.
If user action is correct, score is given, and feedback (instruction Icon Color changes, and celebration sound occurs in background) is provided；
The execution as indicated if user fails, prompts current action mistake, not score, and provides feedback (instruction icon
There is encouragement sound in color change, background).
It should be appreciated by those skilled in the art each modules or each step of, the above-mentioned embodiment of the present invention can be with general
Computing device realize that they can be concentrated on a single computing device, or be distributed in multiple computing devices and formed
Network on, optionally, they can be realized with the program code that computing device can perform, it is thus possible to which they are stored
It is performed by computing device in the storage device, and in some cases, it can be to be different from shown in sequence execution herein
Out or description the step of, perhaps they are fabricated to each integrated circuit modules or by them multiple modules or
Step is fabricated to single integrated circuit module to realize.In this way, the embodiment of the present invention is not limited to any specific hardware and soft
Part combines.
Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with
Relevant hardware is instructed to complete by computer program, the program can be stored in a non-volatile computer and can be read
In storage medium, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, provided herein
Each embodiment used in any reference to memory, storage, database or other media, may each comprise non-volatile
And/or volatile memory.Nonvolatile memory may include that read-only memory (ROM), programming ROM (PROM), electricity can be compiled
Journey ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include random access memory
(RAM) or external cache.By way of illustration and not limitation, RAM is available in many forms, such as static state RAM
(SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDRSDRAM), enhanced SDRAM
(ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) directly RAM (RDRAM), straight
Connect memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM) etc..
The above embodiments merely illustrate the technical concept and features of the present invention, and its object is to allow person skilled in the art
It cans understand the content of the present invention and implement it accordingly, it is not intended to limit the scope of the present invention.It is all to lead according to the present invention
The modification for wanting the Spirit Essence of technical solution to be done, should be covered by the protection scope of the present invention.
Claims (17)
1. a kind of virtual, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that based on head-mounted display and
Its corollary equipment, the direction of motion recognition methods include:
Guide user by prompt towards the mobile M different distance level scales in N number of direction, the N is the integer more than or equal to 2, and M is big
In the integer for being equal to 1, collects user's mobile data and establish data set；
Optimize the data set, and the data set after optimization is stored and establishes disaggregated model；
Judge and set the initial position of user；
Whether system starts to move according to threshold decision user, if it is determined that user starts to move, then collects record mobile data；
According to threshold decision user, whether tenth skill then stops collecting record movement if it is determined that user's tenth skill system
Data；
According to collected mobile data, data set is substituted into disaggregated model, system recommends movement most like out according to feature
Direction judges user movement direction, and system provides the direction of motion, completes direction of motion judgement；
According to the user movement direction of judgement, the corresponding function of system trigger.
2. virtual as described in claim 1, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that described
Data set includes that the moving distance of any movement is poor in either direction in N number of direction by user；The movement speed of this movement；
The acceleration of this movement；This moves the shift length projected in X, Z both direction；Distance change in X-axis is divided by Z axis
Slope obtained by distance change, described X, Z are two reference axis of the horizontal plane of rectangular coordinate system in space.
3. virtual as claimed in claim 2, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that the N
A direction and M different distance level scales are specially east, south, west, north, northeast, northwest, southwest and the direction of the southeast 8, and every
The closely movement, remote mobile two distance level scales of a direction.
4. virtual as claimed in claim 3, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that described
The method for optimizing data set is as follows:
It is mobile for short distance, it filters X-axis or Z axis moving distance is less than the data of the mobile threshold value of the short distance set, retain big
In the data set of the threshold value；
For moving at a distance, filters X-axis or Z axis moving distance is less than the data of the remote mobile threshold value set, retain big
In the data set of the threshold value；
The threshold size is set as adaptively being adjusted according to user's own situation, it is described adjustment based on user closely and
The peak value moved at a distance.
5. virtual as described in claim 1, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that described
Disaggregated model uses k-nearest neighbor classifier.
6. virtual as claimed in claim 5, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that the K
The K of nearest neighbour method is equal to 4.
7. virtual as claimed in claim 2, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that described
Judge the method for user's initial position are as follows: user terminate it is mobile when returning starting point, system obtain location information at this time and with
Location information before setting out compares, when alternate position spike is respectively less than the position set to front-rear position on tri- directions three-dimensional space X, Y and Z
When setting the threshold speed of the speed and setting on poor threshold value while three directions, judgement returns to initial position, and sets at this time
Position is initial position.
8. virtual as claimed in claim 7, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that described
Alternate position spike threshold value is 0.05m, and the threshold speed is 0.2m/s.
9. virtual as described in claim 1, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that described
Whether start mobile criterion according to threshold decision user are as follows: in three-dimensional space on X-axis or Z axis moving distance absolute value
Greater than the threshold value of setting, reach condition, system determines that user starts to move and start to collect exercise data, and data include user
Moving distance and velocity and acceleration in X, Y, Z axis in three-dimensional space, and calculate the shifting in X-axis in three-dimensional space
Dynamic slope of the distance compared to the moving distance on Z axis and total moving distance.
10. virtual as described in claim 1, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that described
According to threshold decision user whether the criterion of tenth skill are as follows: X-axis or Z axis moving distance are greater than setting in three-dimensional space
Moving distance threshold value and the speed of user's all directions and the threshold speed less than setting, reach condition, and system determines that user will
Data are collected in stop motion, pause.
11. virtual as claimed in claim 10, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that institute
It is mobile of different sizes with remote situation of movement lower threshold value in short distance to state moving distance threshold value and threshold speed,
Mobile for short distance, the nearly moving distance that X-axis or Z axis moving distance are greater than setting in three-dimensional space terminates threshold value and use
The speed of family all directions and terminate threshold value less than the short distance movement speed of setting；
For moving at a distance, the remote moving distance that X-axis or Z axis moving distance are greater than setting in three-dimensional space terminates threshold value and use
The speed of family all directions and terminate threshold value less than the remote movement speed of setting；
The threshold size is set as adaptively being adjusted according to user's own situation.
12. virtual as described in claim 1, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that described
It substitutes into disaggregated model and judges the move distance grade for also judging user when the direction of motion, and in the next steps according to user's
The direction of motion and distance level scale trigger corresponding function.
13. virtual as described in claim 1, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that pass through
Inertial Measurement Unit sensor and depth transducer built in the head-mounted display and its corollary equipment obtain the movement of user
Data.
14. virtual as described in claim 1, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that described
When guiding user's movement, collecting mobile data and establish data set, prompt the user whether also to need mobile completion using 3D arrow
Data collection.
15. virtual as described in claim 1, enhancing or mixed reality head show direction of motion recognition methods, which is characterized in that described
Recognition methods, to frame out or sound feedback, is gone back after the completion when completing the user movement direction of judgement and triggering corresponding function
Setting has the following steps: judging whether user instruction terminates, if so, task finishes；If it is not, then returning to judgement and setting user
The step of initial position, repeats subsequent step, until instruction terminates.
16. a kind of computer readable storage medium, which is characterized in that the computer-readable recording medium storage has perform claim
It is required that the computer program for the direction of motion recognition methods that 1 to 15 described in any item virtual, enhancings or mixed reality head are shown.
17. a kind of virtual, enhancing or mixed reality head show direction of motion identifying system characterized by comprising
Data collection module, including Inertial Measurement Unit sensor and depth transducer, for acquiring the mobile data of user；
Display module is worn, for showing virtual, enhancing or mixed reality space and interactive interface；
Control and memory module, including processor and memory, the memory are stored with perform claim and require any one of 1 to 15
Virtual, enhancing or the aobvious direction of motion recognition methods of mixed reality head the computer program, the processor is for executing
The computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810890022.8A CN109144349A (en) | 2018-08-07 | 2018-08-07 | One kind is virtual, enhance or mixed reality head shows direction of motion recognition methods and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810890022.8A CN109144349A (en) | 2018-08-07 | 2018-08-07 | One kind is virtual, enhance or mixed reality head shows direction of motion recognition methods and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109144349A true CN109144349A (en) | 2019-01-04 |
Family
ID=64792200
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810890022.8A Pending CN109144349A (en) | 2018-08-07 | 2018-08-07 | One kind is virtual, enhance or mixed reality head shows direction of motion recognition methods and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109144349A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111260793A (en) * | 2020-01-10 | 2020-06-09 | 中国电子科技集团公司第三十八研究所 | Remote virtual-real high-precision matching positioning method for augmented and mixed reality |
| CN112337082A (en) * | 2020-10-20 | 2021-02-09 | 深圳市杰尔斯展示股份有限公司 | AR immersive virtual visual perception interaction system and method |
| US11288877B2 (en) | 2020-01-10 | 2022-03-29 | 38th Research Institute, China Electronics Technology Group Corp. | Method for matching a virtual scene of a remote scene with a real scene for augmented reality and mixed reality |
| WO2022143110A1 (en) * | 2020-12-30 | 2022-07-07 | 华为技术有限公司 | Neck motion data collection and processing method and apparatus |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102184020A (en) * | 2010-05-18 | 2011-09-14 | 微软公司 | Method for manipulating posture of user interface and posture correction |
| CN103164139A (en) * | 2011-12-15 | 2013-06-19 | 英顺源（上海）科技有限公司 | System of displaying target data according to track properties and method thereof |
| CN104571823A (en) * | 2015-01-12 | 2015-04-29 | 济南大学 | Non-contact virtual human-computer interaction method based on smart television set |
| CN104866221A (en) * | 2015-04-21 | 2015-08-26 | 上海墨百意信息科技有限公司 | Touch operation response method and device based on wearable equipment |
| CN105320275A (en) * | 2014-07-30 | 2016-02-10 | 三星电子株式会社 | Wearable device and method of operating the same |
| CN105431764A (en) * | 2013-07-25 | 2016-03-23 | Lg电子株式会社 | Head mounted display and method of controlling therefor |
| CN106959762A (en) * | 2017-04-24 | 2017-07-18 | 英华达（上海）科技有限公司 | Virtual reality system and method |
| US20180088678A1 (en) * | 2015-09-01 | 2018-03-29 | Kabushiki Kaisha Toshiba | Eyeglasses-type wearable device and method using the same |
| CN108140360A (en) * | 2015-07-29 | 2018-06-08 | 森赛尔股份有限公司 | For manipulating the system and method for virtual environment |
| CN108369345A (en) * | 2015-10-20 | 2018-08-03 | 奇跃公司 | Virtual objects are selected in three dimensions |
-
2018
- 2018-08-07 CN CN201810890022.8A patent/CN109144349A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102184020A (en) * | 2010-05-18 | 2011-09-14 | 微软公司 | Method for manipulating posture of user interface and posture correction |
| CN103164139A (en) * | 2011-12-15 | 2013-06-19 | 英顺源（上海）科技有限公司 | System of displaying target data according to track properties and method thereof |
| CN105431764A (en) * | 2013-07-25 | 2016-03-23 | Lg电子株式会社 | Head mounted display and method of controlling therefor |
| CN105320275A (en) * | 2014-07-30 | 2016-02-10 | 三星电子株式会社 | Wearable device and method of operating the same |
| CN104571823A (en) * | 2015-01-12 | 2015-04-29 | 济南大学 | Non-contact virtual human-computer interaction method based on smart television set |
| CN104866221A (en) * | 2015-04-21 | 2015-08-26 | 上海墨百意信息科技有限公司 | Touch operation response method and device based on wearable equipment |
| CN108140360A (en) * | 2015-07-29 | 2018-06-08 | 森赛尔股份有限公司 | For manipulating the system and method for virtual environment |
| US20180088678A1 (en) * | 2015-09-01 | 2018-03-29 | Kabushiki Kaisha Toshiba | Eyeglasses-type wearable device and method using the same |
| CN108369345A (en) * | 2015-10-20 | 2018-08-03 | 奇跃公司 | Virtual objects are selected in three dimensions |
| CN106959762A (en) * | 2017-04-24 | 2017-07-18 | 英华达（上海）科技有限公司 | Virtual reality system and method |
Non-Patent Citations (2)
| Title |
|---|
| 张萍: "《教育信息技术应用教程》", 30 September 2016 * |
| 李新成: "《信息技术教育应用教程》", 31 August 2007 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111260793A (en) * | 2020-01-10 | 2020-06-09 | 中国电子科技集团公司第三十八研究所 | Remote virtual-real high-precision matching positioning method for augmented and mixed reality |
| CN111260793B (en) * | 2020-01-10 | 2020-11-24 | 中国电子科技集团公司第三十八研究所 | Remote virtual-real high-precision matching positioning method for augmented and mixed reality |
| US11288877B2 (en) | 2020-01-10 | 2022-03-29 | 38th Research Institute, China Electronics Technology Group Corp. | Method for matching a virtual scene of a remote scene with a real scene for augmented reality and mixed reality |
| CN112337082A (en) * | 2020-10-20 | 2021-02-09 | 深圳市杰尔斯展示股份有限公司 | AR immersive virtual visual perception interaction system and method |
| WO2022143110A1 (en) * | 2020-12-30 | 2022-07-07 | 华为技术有限公司 | Neck motion data collection and processing method and apparatus |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109144349A (en) | One kind is virtual, enhance or mixed reality head shows direction of motion recognition methods and system | |
| JP6082101B2 (en) | Body motion scoring device, dance scoring device, karaoke device, and game device | |
| CN101964047B (en) | Multiple trace point-based human body action recognition method | |
| JP2022500790A (en) | Face model generation methods, devices, electronic devices and computer readable storage media | |
| WO2021011888A1 (en) | System and method for error detection and correction in virtual reality and augmented reality environments | |
| US9144744B2 (en) | Locating and orienting device in space | |
| CN109407603B (en) | Method and device for controlling mechanical arm to grab object | |
| KR102388939B1 (en) | Method for providing feedback service for user's motion correction | |
| CN107015637B (en) | Input method and device in virtual reality scene | |
| CN108355347B (en) | Interaction control method and device, electronic equipment and storage medium | |
| EP3366359A1 (en) | Evaluation information providing system and evaluation information providing method | |
| CN108638062A (en) | Robot localization method, apparatus, positioning device and storage medium | |
| JP7006379B2 (en) | Golf shot analyzer | |
| CN109782901A (en) | Augmented reality exchange method, device, computer equipment and storage medium | |
| JP6197702B2 (en) | Input method, program, and input device | |
| CN109766882A (en) | Label identification method, the device of human body luminous point | |
| CN109255817A (en) | A kind of the vision method for relocating and device of smart machine | |
| CN109918718A (en) | Unmanned plane during flying method for evaluating quality, device, storage medium and electronic equipment | |
| JPWO2020003670A1 (en) | Information processing device and information processing method | |
| KR101504103B1 (en) | 3d character motion synthesis and control method and device for navigating virtual environment using depth sensor | |
| CN102184006A (en) | Systems and methods for motion recognition with minimum delay | |
| US20120265516A1 (en) | Peripheral device simulation | |
| CN109769206A (en) | A kind of indoor positioning fusion method, device, storage medium and terminal device | |
| CN109901940A (en) | Promote to be that touch-screen gesture assessment generates standardized test based on model data | |
| CN109885598A (en) | Fault recognition method, device, computer readable storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190104 |
|
| RJ01 | Rejection of invention patent application after publication |