CN109308132A - Implementation method, device, equipment and the system of the handwriting input of virtual reality - Google Patents

Implementation method, device, equipment and the system of the handwriting input of virtual reality Download PDF

Info

Publication number
CN109308132A
CN109308132A CN201811013714.0A CN201811013714A CN109308132A CN 109308132 A CN109308132 A CN 109308132A CN 201811013714 A CN201811013714 A CN 201811013714A CN 109308132 A CN109308132 A CN 109308132A
Authority
CN
China
Prior art keywords
virtual reality
handwriting
handwriting pad
information
reality helmet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811013714.0A
Other languages
Chinese (zh)
Inventor
孔庆磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qingdao Pico Technology Co Ltd
Original Assignee
Qingdao Pico Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qingdao Pico Technology Co Ltd filed Critical Qingdao Pico Technology Co Ltd
Priority to CN201811013714.0A priority Critical patent/CN109308132A/en
Publication of CN109308132A publication Critical patent/CN109308132A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/043Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using propagating acoustic waves
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text

Abstract

The invention discloses a kind of implementation method of the handwriting input of virtual reality, device, equipment and systems, this method comprises: determining location information of the handwriting pad relative to the virtual reality helmet;Location information according to the handwriting pad relative to the virtual reality helmet determines the position of the virtual handwriting pad created in the display scene of the virtual reality helmet;Obtain contact position information when writing pencil is contacted with the handwriting pad;Contact position information when being contacted according to the writing pencil and the handwriting pad, determines the handwriting information of the writing pencil, and the handwriting information is presented on the virtual handwriting pad.

Description

Implementation method, device, equipment and the system of the handwriting input of virtual reality
Technical field
The present invention relates to technical field of virtual reality, more particularly, to a kind of realization of the handwriting input of virtual reality Method, a kind of realization device of the handwriting input of virtual reality, a kind of virtual reality helmet and a kind of virtual reality system.
Background technique
Currently, virtual reality device occupies more and more important position in people's daily life, it can give people Bring unprecedented Flow experience, either video or game.
Virtual reality products can be applied in a variety of different practical fields.But virtual reality products are in use process In have a problem that: user can not see external environment when wearing virtual reality products, so that user can not watch The information inputted to the external interactive device by virtual reality products, reduces the experience of user.For example, user is wearing void After quasi- reality helmet, write on the external input board of virtual reality helmet by writing pencil, user can not Watch the content write in external input board.
Accordingly, it is desirable to provide a kind of new technical method, improves for above-mentioned the problems of the prior art.
Summary of the invention
It is an object of the present invention to provide a kind of new solutions of the handwriting input of virtual reality.
According to the first aspect of the invention, a kind of implementation method of the handwriting input of virtual reality is provided, comprising:
Determine location information of the handwriting pad relative to the virtual reality helmet;
Location information according to the handwriting pad relative to the virtual reality helmet is determined in the virtual reality The position of the virtual handwriting pad created in the display scene of helmet;
Obtain contact position information when writing pencil is contacted with the handwriting pad;
Contact position information when being contacted according to the writing pencil and the handwriting pad determines the person's handwriting letter of the writing pencil Breath, and the handwriting information is presented on the virtual handwriting pad.
Optionally, while contact position information when obtaining writing pencil and the handwriting pad and contacting, the method is also Include:
Determine location information of the writing pencil relative to the virtual reality helmet, and according to the writing pencil phase For the location information of the virtual reality helmet, determination creates in the display scene of the virtual reality helmet Virtual writing pencil position;And/or
The data that the Inertial Measurement Unit measurement of the writing pencil obtains are obtained, and according to the inertia measurement of the writing pencil The data that unit measurement obtains determine the posture information of the virtual writing pencil.
Optionally, while contact position information when obtaining writing pencil and the handwriting pad and contacting, the method is also Include:
Obtain the person's handwriting pressure information that the detection of writing pencil obtains;
According to the person's handwriting pressure information, the thickness information for the person's handwriting being presented on the virtual handwriting pad is determined.
Optionally, the virtual reality helmet is provided with ultrasonic transmitter, and the handwriting pad is provided with ultrasonic wave Receiver, alternatively, the virtual reality helmet is provided with ultrasonic receiver, the handwriting pad is provided with ultrasonic wave transmitting Device, wherein
Determine location information of the handwriting pad relative to the virtual reality helmet, comprising:
The ultrasonic signal information received according to the ultrasonic receiver, determine the ultrasonic receiver with it is described The distance of ultrasonic transmitter;
According to the ultrasonic receiver at a distance from the ultrasonic transmitter, determine the handwriting pad relative to described The location information of virtual reality helmet.
Optionally, the virtual reality helmet is provided with binocular camera, and the handwriting pad is provided with luminescent device Array,
Determine location information of the handwriting pad relative to the virtual reality helmet, comprising:
According to the imaging of the light-emitting element array in the handwriting pad image of binocular camera shooting, handwriting pad phase is determined For the location information of the virtual reality helmet.
Optionally, while determining location information of the handwriting pad relative to the virtual reality helmet, the side Method further include:
Obtain the data that the Inertial Measurement Unit measurement of the handwriting pad obtains;
According to the data that the Inertial Measurement Unit measurement of the handwriting pad obtains, determine in the virtual reality helmet Display scene in the posture information of virtual handwriting pad that creates.
According to the second aspect of the invention, a kind of realization device of the handwriting input of virtual reality is provided, comprising:
Location information determining module, for determining location information of the handwriting pad relative to the virtual reality helmet;
Location information determining module is believed for the position according to the handwriting pad relative to the virtual reality helmet Breath determines the position of the virtual handwriting pad created in the display scene of the virtual reality helmet;
Module is obtained, for obtaining contact position information when writing pencil and the handwriting pad contact;
Handwriting information determining module, contact position information when for being contacted according to the writing pencil and the handwriting pad, It determines the handwriting information of the writing pencil, and the handwriting information is presented on the virtual handwriting pad.
According to the third aspect of the invention we, a kind of realization device of the handwriting input of virtual reality is provided, comprising: packet It includes: memory and processor, wherein the memory stores executable instruction, and the executable instruction controls the processor It is operated in the method according to any one in first aspect of execution.
According to the fourth aspect of the invention, a kind of virtual reality helmet is provided, including such as second aspect or third The realization device of the handwriting input of virtual reality described in aspect.
According to the fifth aspect of the invention, a kind of virtual reality system is provided, which is characterized in that including such as fourth aspect Virtual reality helmet, handwriting pad and the writing pencil, wherein
The contact position information that the handwriting pad is used for when contacting the writing pencil and the handwriting pad is sent to described Virtual reality helmet;
The person's handwriting pressure information that the writing pencil is used to will test is sent directly to the virtual reality helmet, Alternatively, the person's handwriting pressure information that will test is sent to the handwriting pad, by the handwriting pad by the person's handwriting pressure information It is forwarded to the virtual reality helmet.
According to one embodiment of present invention, virtual reality helmet can show that creation is virtual hand-written in scene at it Plate, and lettering pen of the original subscriber on the handwriting pad external with virtual reality helmet is really gone back on virtual handwriting pad Mark allows user to watch extraneous handwriting input in the display scene of virtual reality helmet.
By referring to the drawings to the detailed description of exemplary embodiment of the present invention, other feature of the invention and its Advantage will become apparent.
Detailed description of the invention
It is combined in the description and the attached drawing for constituting part of specification shows the embodiment of the present invention, and even With its explanation together principle for explaining the present invention.
Fig. 1 is the process flow diagram of the implementation method of the handwriting input of virtual reality according to an embodiment of the invention.
Fig. 2 a is the ultrasonic wave that is arranged on virtual reality helmet according to an embodiment of the invention and on handwriting pad The schematic diagram of device.
Fig. 2 b is the ultrasound that is arranged on virtual reality helmet in accordance with another embodiment of the present invention and on handwriting pad The schematic diagram of wave device.
Fig. 2 c is the ultrasound that is arranged on virtual reality helmet in accordance with another embodiment of the present invention and on handwriting pad The schematic diagram of wave device.
Fig. 2 d is the ultrasound that is arranged on virtual reality helmet in accordance with another embodiment of the present invention and on handwriting pad The schematic diagram of wave device.
Fig. 3 a is the schematic diagram of virtual reality helmet according to an embodiment of the present invention.
Fig. 3 b is the schematic diagram of handwriting pad according to an embodiment of the present invention.
Fig. 4 is the functional block diagram of the realization device of the handwriting input of virtual reality according to an embodiment of the present invention.
Fig. 5 is the hardware block diagram of the realization device of the handwriting input of virtual reality according to an embodiment of the present invention.
Fig. 6 is the structural schematic diagram of virtual reality system according to an embodiment of the invention.
Fig. 7 is the structural schematic diagram of the virtual reality system of another embodiment according to the present invention.
Specific embodiment
Carry out the various exemplary embodiments of detailed description of the present invention now with reference to attached drawing.It should also be noted that unless in addition having Body explanation, the unlimited system of component and the positioned opposite of step, numerical expression and the numerical value otherwise illustrated in these embodiments is originally The range of invention.
Be to the description only actually of at least one exemplary embodiment below it is illustrative, never as to the present invention And its application or any restrictions used.
Technology, method and apparatus known to person of ordinary skill in the relevant may be not discussed in detail, but suitable In the case of, the technology, method and apparatus should be considered as part of specification.
It is shown here and discuss all examples in, any occurrence should be construed as merely illustratively, without It is as limitation.Therefore, other examples of exemplary embodiment can have different values.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi It is defined in a attached drawing, then in subsequent attached drawing does not need that it is further discussed.
An embodiment provides a kind of implementation methods of the handwriting input of virtual reality.The virtual reality The implementation method of handwriting input is implemented by virtual reality helmet.
According to Fig. 1, the implementation method of the handwriting input of virtual reality at least includes the following steps:
Step S101 determines location information of the handwriting pad relative to virtual reality helmet.
In one embodiment of the invention, virtual reality helmet is provided with ultrasonic transmitter, handwriting pad setting There is ultrasonic receiver.Alternatively, virtual reality helmet is provided with ultrasonic receiver, handwriting pad is provided with ultrasonic wave transmitting Device.Firstly, the ultrasonic signal information received according to ultrasonic receiver, determines ultrasonic receiver and ultrasonic transmitter Distance.Then, according to ultrasonic receiver at a distance from ultrasonic transmitter, determine that handwriting pad is worn relative to virtual reality The location information of equipment.
In one example, according to fig. 2 shown in a, virtual reality helmet is provided at least three ultrasonic transmitters, Handwriting pad is provided with ultrasonic receiver.
When ultrasonic transmitter emits ultrasonic signal, ultrasonic receiver can be notified to start timing by radiofrequency signal. When ultrasonic receiver receives ultrasonic signal, ultrasonic receiver terminates timing.Pass through the timing of ultrasonic receiver As a result, determining the time needed for ultrasonic receiver receives the ultrasonic signal of each ultrasonic transmitter transmitting.According to ultrasonic wave Spread speed and ultrasonic receiver receive the ultrasonic signal of each ultrasonic transmitter transmitting needed for the time, be calculated Distance of the ultrasonic receiver to each ultrasonic transmitter.
Three ultrasonic transmitters on virtual reality helmet are set in virtual reality helmet ontology coordinate The position of system is expressed as (x1, y1, z1)、(x2, y2, z2)、(x3, y3, z3).By the position coordinates of three ultrasonic transmitters and surpass The distance of acoustic receiver to each ultrasonic transmitter substitutes into distance between two points calculation formula, obtains equation group.Equation group packet Three linear functions are included, it is as follows,
Wherein, d1、d2、d3For ultrasonic receiver to the distance of each ultrasonic transmitter.According to above three first power Location information of the ultrasonic receiver under virtual reality helmet body coordinate system can be calculated in journey.Ultrasonic wave is connect Location information of the device under virtual reality helmet body coordinate system is received as handwriting pad relative to virtual reality helmet Location information.
In another example, according to fig. 2 shown in b, virtual reality helmet is provided with the reception of at least three ultrasonic waves Device, handwriting pad are provided with ultrasonic transmitter.
When ultrasonic transmitter emits ultrasonic signal, radiofrequency signal is sent to each ultrasonic receiver.Each ultrasonic wave connects Device is received when receiving radiofrequency signal, starts timing.When each ultrasonic receiver receives ultrasonic signal, each ultrasonic wave is connect Receiving device terminates timing.By the timing result of each ultrasonic receiver, determine that each ultrasonic receiver receives ultrasonic transmitter Time needed for the ultrasonic signal of transmitting.Ultrasonic wave transmitting is received according to the spread speed of ultrasonic wave and each ultrasonic receiver Device transmitting ultrasonic signal needed for the time, be calculated each ultrasonic receiver to ultrasonic transmitter distance.
Using above-mentioned equation group, position of the ultrasonic transmitter under virtual reality helmet body coordinate system is calculated Confidence breath.Wherein, (x1, y1, z1)、(x2, y2, z2)、(x3, y3, z3) it is respectively three be arranged on virtual reality helmet Ultrasonic receiver is in the position of virtual reality helmet body coordinate system, d1、d2、d3For each ultrasonic receiver to ultrasound The distance of wave launcher.Using location information of the ultrasonic transmitter under virtual reality helmet body coordinate system as hand-written Location information of the plate relative to virtual reality helmet.
In another example, according to fig. 2 shown in c, virtual reality helmet is provided with the transmitting of at least three ultrasonic waves Device, handwriting pad are provided at least three ultrasonic receivers.Alternatively, virtual reality helmet is provided with according to fig. 2 shown in d At least three ultrasonic receivers, handwriting pad are provided at least three ultrasonic transmitters.This mode can obtain more Range information, and then positioning accuracy can be improved.
In one embodiment of the invention, virtual reality helmet is provided with binocular camera, and handwriting pad is provided with Light-emitting element array.The light that light-emitting element array on the jotting surface issues, which is arranged, can be visible light, be also possible to invisible Light (for example, infrared light).If the light that luminescent device issues is infrared light, binocular camera is infrared camera.
In the embodiment of the present invention, the imaging of the light-emitting element array in handwriting pad image shot according to binocular camera, Determine location information of the handwriting pad relative to virtual reality helmet.
Fig. 3 a is the schematic diagram of virtual reality helmet according to an embodiment of the present invention.It is virtual existing according to Fig. 3 a Real helmet is provided with binocular camera.Fig. 3 b is the schematic diagram of handwriting pad according to an embodiment of the present invention.According to Fig. 3 b institute Show, there are three LED light emitting device for handwriting pad setting, which surrounds triangular shaped.
In the embodiment of the present invention, according to the imaging shape for the shape that the LED light emitting device that binocular camera is shot surrounds Shape and imaging size, determine location information of the handwriting pad relative to virtual reality helmet.For example, using binocular visual positioning Algorithm handles the imaging for the shape that the LED light emitting device that binocular camera is shot surrounds, it is opposite to obtain handwriting pad In the location information of virtual reality helmet.
It should be noted that the number that luminescent device on the jotting surface is arranged is not limited to three.In order to improve positioning More luminescent devices (for example, five, ten, 100 etc.) are arranged on the jotting surface, and are arranged in handwriting pad for precision On luminescent device can surround specific geometry, for example, rectangular, polygon etc..
Step S102, the location information according to handwriting pad relative to virtual reality helmet are determined in virtual reality head Wear the position of the virtual handwriting pad created in the display scene of equipment.
In the embodiment of the present invention, the location information by handwriting pad relative to virtual reality helmet is converted into world coordinates Location information under system, and using location information of the handwriting pad under world coordinate system as the display field of virtual reality helmet The position of virtual handwriting pad in scape.
Step S103 obtains contact position information when writing pencil is contacted with handwriting pad.
In the embodiment of the present invention, writing pencil can be time writer, and handwriting pad can be electromagnetic induction handwriting pad.Electricity is utilized in user For magnetic pen when writing on electromagnetic induction handwriting pad, electromagnetic induction handwriting pad acquires the pen tip coordinate information of writing pencil.Electromagnetic induction Its collected pen tip coordinate information is sent to virtual reality helmet by handwriting pad.Virtual reality helmet is received Contact position information when being contacted to pen tip coordinate information as writing pencil and handwriting pad.
In one embodiment of the invention, writing pencil is provided with pressure detecting module.The pressure detecting module can be Pressure sensor.The pressure detecting module is for detecting person's handwriting pressure information.Writing pencil can detect pressure detecting module To person's handwriting pressure information be sent to handwriting pad, then, received person's handwriting pressure information is forwarded to virtually by handwriting pad again Real helmet.Alternatively, the person's handwriting pressure information that pressure detecting module detects can be sent directly to void by writing pencil Quasi- reality helmet.
Virtual reality helmet determines the thickness letter for the person's handwriting being presented on virtual handwriting pad according to person's handwriting pressure information Breath.For example, virtual reality helmet prestores the mapping table of person's handwriting pressure information and handwriting thickness information.Virtual reality Helmet can determine the corresponding handwriting thickness information of person's handwriting pressure information according to the mapping table.
Step S104, contact position information when being contacted according to writing pencil and handwriting pad, determines the handwriting information of writing pencil, And handwriting information is presented on virtual handwriting pad.
The implementation method of the handwriting input of virtual reality provided in an embodiment of the present invention, virtual reality helmet can be Its show scene in creates virtual handwriting pad, and on virtual handwriting pad really also original subscriber with virtual reality helmet Written handwriting on external handwriting pad allows user to watch the external world in the display scene of virtual reality helmet Handwriting input.
In one embodiment of the invention, handwriting pad is provided with Inertial Measurement Unit.Virtual reality helmet can be with Obtain the data that the Inertial Measurement Unit measurement of handwriting pad obtains, and the number obtained according to the Inertial Measurement Unit measurement of handwriting pad According to the posture information of the determining virtual handwriting pad created in the display scene of virtual reality helmet.
In the embodiment of the present invention, the Inertial Measurement Unit being arranged on the jotting surface can be nine axle sensors, including accelerate Degree meter, gyroscope and magnetometer.The data that Inertial Measurement Unit measurement obtains include at least the measured value a and gyro of accelerometer The measured value ω of instrument.According to the measured value ω of the measured value a of accelerometer and gyroscope, determine in virtual reality helmet The posture information of the virtual handwriting pad created in display scene.
In one embodiment of the invention, not only creation is virtual hand-written in display scene for virtual reality helmet Plate can also create virtual writing pencil.
In the embodiment of the present invention, virtual reality helmet obtains position of the writing pencil relative to virtual reality helmet The data that the Inertial Measurement Unit measurement of information and writing pencil obtains.Then, virtual reality helmet is opposite according to writing pencil In the location information of virtual reality helmet, determination creates virtual hand-written in the display scene of virtual reality helmet The position of pen, and, according to the data that the Inertial Measurement Unit measurement of writing pencil obtains, determine the posture letter of virtual writing pencil Breath.
In one example, virtual reality helmet is provided with ultrasonic transmitter, and writing pencil is provided with ultrasonic wave and connects Receive device.Alternatively, virtual reality helmet is provided with ultrasonic receiver, writing pencil is provided with ultrasonic transmitter.Firstly, root The ultrasonic signal information received according to ultrasonic receiver, determines ultrasonic receiver at a distance from ultrasonic transmitter.So Afterwards, according to ultrasonic receiver at a distance from ultrasonic transmitter, position of the writing pencil relative to virtual reality helmet is determined Confidence breath.Writing pencil relative to the location information of virtual reality helmet determination method can referring in particular to handwriting pad relative to The determination method of the location information of virtual reality helmet, does not do excessively repeat herein.
In another example, virtual reality helmet is provided with binocular camera, and writing pencil is provided with luminescent device Array.The light that the light-emitting element array that is arranged on writing pencil issues can be visible light, be also possible to black light (for example, Infrared light).In the embodiment of the present invention, the imaging of the light-emitting element array in writing pencil image shot according to binocular camera, Determine location information of the writing pencil relative to virtual reality helmet.For example, being sent out according to the LED that binocular camera is shot The imaged shape and imaging size for the shape that optical device surrounds, determine that writing pencil is believed relative to the position of virtual reality helmet Breath.
Based on the same inventive concept, one embodiment of the invention provides a kind of realization dress of the handwriting input of virtual reality It sets.Fig. 4 is the functional block diagram of the realization device of the handwriting input of virtual reality according to an embodiment of the present invention.
According to Fig.4, the realization device of the handwriting input of virtual reality include: first location information determining module 410, Second location information determining module 420 obtains module 430, handwriting information determining module 440.
First location information determining module 410, for determining that handwriting pad is believed relative to the position of virtual reality helmet Breath.
Second location information determining module 420 is believed for the position according to handwriting pad relative to virtual reality helmet Breath determines the position of the virtual handwriting pad created in the display scene of virtual reality helmet.
Module 430 is obtained, for obtaining contact position information when writing pencil and handwriting pad contact.
Handwriting information determining module 440, contact position information when for being contacted according to writing pencil and handwriting pad, determines hand The handwriting information of pen is write, and handwriting information is presented on virtual handwriting pad.
In one embodiment of the invention, virtual reality helmet is provided with ultrasonic transmitter, handwriting pad setting There is ultrasonic receiver, alternatively, virtual reality helmet is provided with ultrasonic receiver, handwriting pad is provided with ultrasonic wave transmitting Device.First location information determining module 410 is further used for: the ultrasonic signal information received according to ultrasonic receiver, Determine ultrasonic receiver at a distance from ultrasonic transmitter;According to ultrasonic receiver at a distance from ultrasonic transmitter, really Determine location information of the handwriting pad relative to virtual reality helmet.
In one embodiment of the invention, virtual reality helmet is provided with binocular camera, and handwriting pad is provided with LED light emitting device array.First location information determining module 410 is further used for: the handwriting pad shot according to binocular camera The imaging of LED light emitting device array in image determines location information of the handwriting pad relative to virtual reality helmet.
In one embodiment of the invention, the realization device of the handwriting input of virtual reality further include: posture information is true Cover half block.Posture information determining module is for obtaining the data that the Inertial Measurement Unit measurement of handwriting pad obtains;According to handwriting pad The obtained data of Inertial Measurement Unit measurement, determine created in the display scene of virtual reality helmet it is virtual hand-written The posture information of plate.
In one embodiment of the invention, first location information determining module 410 is also used to: determine writing pencil relative to The location information of virtual reality helmet.Second location information determining module 420 is also used to according to writing pencil relative to virtual The location information of real helmet determines the position of the virtual writing pencil created in the display scene of virtual reality helmet It sets.
In one embodiment of the invention, posture information determining module is also used to: obtaining the inertia measurement list of writing pencil The data that member measurement obtains, and the data obtained according to the Inertial Measurement Unit measurement of writing pencil, determine the appearance of virtual writing pencil State information.
In one embodiment of the invention, it obtains module 430 to be also used to: obtaining the person's handwriting pressure that the detection of writing pencil obtains Force information.Handwriting information determining module 440 is also used to: according to person's handwriting pressure information, determining the pen being presented on virtual handwriting pad The thickness information of mark.
Fig. 5 is the hardware block diagram of the realization device of the handwriting input of virtual reality according to an embodiment of the present invention.
According to Fig.5, the realization device of the handwriting input of virtual reality includes: memory 510 and processor 520.It deposits Reservoir 510 stores executable instruction, and executable instruction control processor 520 is operated to execute any of the above-described embodiment and provide Virtual reality handwriting input implementation method.
Based on the same inventive concept, one embodiment of the present of invention additionally provides a kind of virtual reality helmet.The void Quasi- reality helmet includes at least the realization device of the handwriting input of the virtual reality shown in Fig. 4 or Fig. 5.
Based on the same inventive concept, one embodiment of the present of invention additionally provides a kind of virtual reality system.This is virtually existing Real system includes at least virtual reality helmet, handwriting pad and writing pencil.
The contact position information that handwriting pad is used for when contacting writing pencil and handwriting pad is sent to virtual reality helmet.
The person's handwriting pressure information that writing pencil is used to will test is sent directly to virtual reality helmet, alternatively, will It detects obtained person's handwriting pressure information and is sent to handwriting pad, person's handwriting pressure information is forwarded to virtual reality by handwriting pad wears and set It is standby.
Fig. 6 is the structural schematic diagram of virtual reality system according to an embodiment of the invention.
According to Fig.6, virtual reality helmet includes at least first processor, the first ultrasonic node lattice array, the One inertia measurement sensor, display module, first communication module.
Handwriting pad includes at least second processor, the second ultrasonic node lattice array, handwriting information detection module, the second inertia Measurement sensor, second communication module.
Writing pencil includes at least third processor, pressure detecting module, third ultrasonic node lattice array, third inertia measurement Sensor, third communication module.
In the embodiment of the present invention, virtual reality helmet and handwriting pad can be communicated to connect, and handwriting pad and writing pencil can With communication connection.The communication connection can be wire communication connection, be also possible to wireless communication connection.
First ultrasonic node lattice array can be at least three ultrasonic transmitters, and the second ultrasonic node lattice array can be At least one ultrasonic receiver.Alternatively, the first ultrasonic node lattice array can be at least three ultrasonic receivers, the second surpass Sound wave node array can be at least one ultrasonic transmitter.
Virtual reality helmet can be according between the first ultrasonic node lattice array and the second ultrasonic node lattice array Ultrasound signal transmission information determines location information of the handwriting pad relative to virtual reality helmet, later according to handwriting pad Relative to the location information of virtual reality helmet, determination creates virtual in the display scene of virtual reality helmet The position of handwriting pad.
The data that second Inertial Measurement Unit measurement of handwriting pad obtains are sent to second processor.Second processor The data that second Inertial Measurement Unit measurement obtains are sent to virtual reality helmet.The first of virtual reality helmet The data that processor is obtained according to the second Inertial Measurement Unit measurement, determination are created in the display scene of virtual reality helmet The posture information for the virtual handwriting pad built.
First ultrasonic node lattice array can be at least three ultrasonic transmitters, and third ultrasonic node lattice array can be At least one ultrasonic receiver.Alternatively, the first ultrasonic node lattice array can be at least three ultrasonic receivers, third is super Sound wave node array can be at least one ultrasonic transmitter.
Virtual reality helmet can be according between the first ultrasonic node lattice array and third ultrasonic node lattice array Ultrasound signal transmission information determines location information of the writing pencil relative to virtual reality helmet, later according to writing pencil Relative to the location information of virtual reality helmet, determination creates virtual in the display scene of virtual reality helmet The position of writing pencil.
The data that the third Inertial Measurement Unit measurement of writing pencil obtains are sent to handwriting pad, and then, handwriting pad is again The data forwarding that the third Inertial Measurement Unit measurement received is obtained is to virtual reality helmet.Virtual reality, which is worn, to be set The data that standby first processor is obtained according to third Inertial Measurement Unit measurement determine the display in virtual reality helmet The posture information of the virtual writing pencil created in scene.
When user is write on the jotting surface using writing pencil, the handwriting information detection module of handwriting pad can acquire hand-written The pen tip coordinate information of pen, and its collected pen tip coordinate information is sent to second processor.Second processor is by the pen Sharp coordinate information is sent to virtual reality helmet.The first processor of virtual reality helmet is believed according to the pen tip coordinate The handwriting information for determining writing pencil is ceased, and handwriting information is presented on virtual handwriting pad.
When user is write on the jotting surface using writing pencil, the pressure detecting module of writing pencil can detecte person's handwriting pressure Information, and the person's handwriting pressure information detected is sent to third processor.The third processor of writing pencil is by pressure detecting The person's handwriting pressure information that module detects is sent to handwriting pad, and then, handwriting pad is again by received person's handwriting pressure information It is forwarded to virtual reality helmet.The first processor of virtual reality helmet is determined and is presented according to person's handwriting pressure information The thickness information of person's handwriting on virtual handwriting pad, and the information of the thickness of person's handwriting is presented on virtual handwriting pad.
Fig. 7 is the structural schematic diagram of the virtual reality system of another embodiment according to the present invention.
According to Fig.7, virtual reality helmet is used to including at least first processor, binocular camera shooting head module, first Property measurement sensor, display module, first communication module.
Handwriting pad includes at least second processor, the first light-emitting element array, handwriting information detection module, the second inertia and surveys Quantity sensor, second communication module.
Writing pencil includes at least third processor, pressure detecting module, the second light-emitting element array, third inertia measurement and passes Sensor, third communication module.
In the embodiment of the present invention, virtual reality helmet and handwriting pad can be communicated to connect, virtual reality helmet It can be communicated to connect with writing pencil.The communication connection can be wire communication connection, be also possible to wireless communication connection.
The first processor of virtual reality helmet is according to the first light-emitting element array of binocular camera module photograph Imaging, determine location information of the handwriting pad relative to virtual reality helmet.
The data that second Inertial Measurement Unit measurement of handwriting pad obtains are sent to second processor.Second processor The data that second Inertial Measurement Unit measurement obtains are sent to virtual reality helmet.The first of virtual reality helmet The data that processor is obtained according to the second Inertial Measurement Unit measurement, determination are created in the display scene of virtual reality helmet The posture information for the virtual handwriting pad built.
The first processor of virtual reality helmet is according to the second light-emitting element array of binocular camera module photograph Imaging, determine location information of the writing pencil relative to virtual reality helmet.
The data that the third Inertial Measurement Unit measurement of writing pencil obtains are sent to third processor.Third processor The data that third Inertial Measurement Unit measurement obtains are sent to virtual reality helmet.The first of virtual reality helmet The data that processor is obtained according to third Inertial Measurement Unit measurement, determination are created in the display scene of virtual reality helmet The posture information for the virtual writing pencil built.
When user is write on the jotting surface using writing pencil, the handwriting information detection module of handwriting pad can acquire hand-written The pen tip coordinate information of pen, and its collected pen tip coordinate information is sent to second processor.Second processor is by the pen Sharp coordinate information is sent to virtual reality helmet.The first processor of virtual reality helmet is believed according to the pen tip coordinate The handwriting information for determining writing pencil is ceased, and handwriting information is presented on virtual handwriting pad.
When user is write on the jotting surface using writing pencil, the pressure detecting module of writing pencil can detecte person's handwriting pressure Information, and the person's handwriting pressure information detected is sent to third processor.The third processor of writing pencil is by pressure detecting The person's handwriting pressure information that module detects is sent to virtual reality helmet.The first processor of virtual reality helmet According to person's handwriting pressure information, the thickness information for the person's handwriting being presented on virtual handwriting pad is determined, and by the information of the thickness of person's handwriting It is presented on virtual handwriting pad.
The present invention can be system, method and/or computer program product.Computer program product may include computer Readable storage medium storing program for executing, containing for making processor realize the computer-readable program instructions of various aspects of the invention.
Computer readable storage medium, which can be, can keep and store the tangible of the instruction used by instruction execution equipment Equipment.Computer readable storage medium for example can be-- but it is not limited to-- storage device electric, magnetic storage apparatus, optical storage Equipment, electric magnetic storage apparatus, semiconductor memory apparatus or above-mentioned any appropriate combination.Computer readable storage medium More specific example (non exhaustive list) includes: portable computer diskette, hard disk, random access memory (RAM), read-only deposits It is reservoir (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), portable Compact disk read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical coding equipment, for example thereon It is stored with punch card or groove internal projection structure and the above-mentioned any appropriate combination of instruction.Calculating used herein above Machine readable storage medium storing program for executing is not interpreted that instantaneous signal itself, the electromagnetic wave of such as radio wave or other Free propagations lead to It crosses the electromagnetic wave (for example, the light pulse for passing through fiber optic cables) of waveguide or the propagation of other transmission mediums or is transmitted by electric wire Electric signal.
Computer-readable program instructions as described herein can be downloaded to from computer readable storage medium it is each calculate/ Processing equipment, or outer computer or outer is downloaded to by network, such as internet, local area network, wide area network and/or wireless network Portion stores equipment.Network may include copper transmission cable, optical fiber transmission, wireless transmission, router, firewall, interchanger, gateway Computer and/or Edge Server.Adapter or network interface in each calculating/processing equipment are received from network to be counted Calculation machine readable program instructions, and the computer-readable program instructions are forwarded, for the meter being stored in each calculating/processing equipment In calculation machine readable storage medium storing program for executing.
Computer program instructions for executing operation of the present invention can be assembly instruction, instruction set architecture (ISA) instructs, Machine instruction, machine-dependent instructions, microcode, firmware instructions, condition setup data or with one or more programming languages The source code or object code that any combination is write, the programming language include the programming language-of object-oriented such as Smalltalk, C++ etc., and conventional procedural programming languages-such as " C " language or similar programming language.Computer Readable program instructions can be executed fully on the user computer, partly execute on the user computer, be only as one Vertical software package executes, part executes on the remote computer or completely in remote computer on the user computer for part Or it is executed on server.In situations involving remote computers, remote computer can pass through network-packet of any kind It includes local area network (LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as benefit It is connected with ISP by internet).In some embodiments, by utilizing computer-readable program instructions Status information carry out personalized customization electronic circuit, such as programmable logic circuit, field programmable gate array (FPGA) or can Programmed logic array (PLA) (PLA), the electronic circuit can execute computer-readable program instructions, to realize each side of the invention Face.
Referring herein to according to the method for the embodiment of the present invention, the flow chart of device (system) and computer program product and/ Or block diagram describes various aspects of the invention.It should be appreciated that flowchart and or block diagram each box and flow chart and/ Or in block diagram each box combination, can be realized by computer-readable program instructions.
These computer-readable program instructions can be supplied to general purpose computer, special purpose computer or other programmable datas The processor of processing unit, so that a kind of machine is produced, so that these instructions are passing through computer or other programmable datas When the processor of processing unit executes, function specified in one or more boxes in implementation flow chart and/or block diagram is produced The device of energy/movement.These computer-readable program instructions can also be stored in a computer-readable storage medium, these refer to It enables so that computer, programmable data processing unit and/or other equipment work in a specific way, thus, it is stored with instruction Computer-readable medium then includes a manufacture comprising in one or more boxes in implementation flow chart and/or block diagram The instruction of the various aspects of defined function action.
Computer-readable program instructions can also be loaded into computer, other programmable data processing units or other In equipment, so that series of operation steps are executed in computer, other programmable data processing units or other equipment, to produce Raw computer implemented process, so that executed in computer, other programmable data processing units or other equipment Instruct function action specified in one or more boxes in implementation flow chart and/or block diagram.
The flow chart and block diagram in the drawings show the system of multiple embodiments according to the present invention, method and computer journeys The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation One module of table, program segment or a part of instruction, the module, program segment or a part of instruction include one or more use The executable instruction of the logic function as defined in realizing.In some implementations as replacements, function marked in the box It can occur in a different order than that indicated in the drawings.For example, two continuous boxes can actually be held substantially in parallel Row, they can also be executed in the opposite order sometimes, and this depends on the function involved.It is also noted that block diagram and/or The combination of each box in flow chart and the box in block diagram and or flow chart, can the function as defined in executing or dynamic The dedicated hardware based system made is realized, or can be realized using a combination of dedicated hardware and computer instructions.It is right For those skilled in the art it is well known that, by hardware mode realize, by software mode realize and pass through software and It is all of equal value that the mode of combination of hardware, which is realized,.
Various embodiments of the present invention are described above, above description is exemplary, and non-exclusive, and It is not limited to disclosed each embodiment.Without departing from the scope and spirit of illustrated each embodiment, for this skill Many modifications and changes are obvious for the those of ordinary skill in art field.The selection of term used herein, purport In the principle, practical application or technological improvement to the technology in market for best explaining each embodiment, or lead this technology Other those of ordinary skill in domain can understand each embodiment disclosed herein.The scope of the present invention is limited by appended claims It is fixed.

Claims (10)

1. a kind of implementation method of the handwriting input of virtual reality characterized by comprising
Determine location information of the handwriting pad relative to the virtual reality helmet;
Location information according to the handwriting pad relative to the virtual reality helmet, determination are worn in the virtual reality The position of the virtual handwriting pad created in the display scene of equipment;
Obtain contact position information when writing pencil is contacted with the handwriting pad;
Contact position information when being contacted according to the writing pencil and the handwriting pad, determines the handwriting information of the writing pencil, And the handwriting information is presented on the virtual handwriting pad.
2. the method according to claim 1, wherein contacting when obtaining writing pencil and being contacted with the handwriting pad While location information, the method also includes:
Determine location information of the writing pencil relative to the virtual reality helmet, and according to the writing pencil relative to The location information of the virtual reality helmet determines the void created in the display scene of the virtual reality helmet The position of quasi- writing pencil;And/or
The data that the Inertial Measurement Unit measurement of the writing pencil obtains are obtained, and according to the Inertial Measurement Unit of the writing pencil Obtained data are measured, determine the posture information of the virtual writing pencil.
3. the method according to claim 1, wherein contacting when obtaining writing pencil and being contacted with the handwriting pad While location information, the method also includes:
Obtain the person's handwriting pressure information that the detection of writing pencil obtains;
According to the person's handwriting pressure information, the thickness information for the person's handwriting being presented on the virtual handwriting pad is determined.
4. method according to claim 1 to 3, which is characterized in that the virtual reality helmet is provided with super Pinger, the handwriting pad are provided with ultrasonic receiver, alternatively, the virtual reality helmet is provided with ultrasonic wave Receiver, the handwriting pad are provided with ultrasonic transmitter, wherein
Determine location information of the handwriting pad relative to the virtual reality helmet, comprising:
The ultrasonic signal information received according to the ultrasonic receiver, determines the ultrasonic receiver and the ultrasound The distance of wave launcher;
According to the ultrasonic receiver at a distance from the ultrasonic transmitter, determine the handwriting pad relative to described virtual The location information of real helmet.
5. method according to claim 1 to 3, which is characterized in that the virtual reality helmet is provided with double Mesh camera, the handwriting pad are provided with light-emitting element array,
Determine location information of the handwriting pad relative to the virtual reality helmet, comprising:
According to the binocular camera shooting handwriting pad image in light-emitting element array imaging, determine handwriting pad relative to The location information of the virtual reality helmet.
6. the method according to claim 1, wherein being set determining that handwriting pad is worn relative to the virtual reality While standby location information, the method also includes:
Obtain the data that the Inertial Measurement Unit measurement of the handwriting pad obtains;
According to the data that the Inertial Measurement Unit measurement of the handwriting pad obtains, determine in the aobvious of the virtual reality helmet Show the posture information of the virtual handwriting pad created in scene.
7. a kind of realization device of the handwriting input of virtual reality characterized by comprising
First location information determining module, for determining location information of the handwriting pad relative to the virtual reality helmet;
Second location information determining module is believed for the position according to the handwriting pad relative to the virtual reality helmet Breath determines the position of the virtual handwriting pad created in the display scene of the virtual reality helmet;
Module is obtained, for obtaining contact position information when writing pencil and the handwriting pad contact;
Handwriting information determining module, contact position information when for being contacted according to the writing pencil and the handwriting pad, determines The handwriting information of the writing pencil, and the handwriting information is presented on the virtual handwriting pad.
8. a kind of realization device of the handwriting input of virtual reality characterized by comprising it include: memory and processor, In, the memory stores executable instruction, and the executable instruction controls the processor and operated to execute according to power Benefit requires method described in any one in 1-6.
9. a kind of virtual reality helmet, which is characterized in that including the hand-written of virtual reality as claimed in claim 7 or 8 The realization device of input.
10. a kind of virtual reality system, which is characterized in that including virtual reality helmet as claimed in claim 9, hand-written Plate and writing pencil, wherein
The contact position information that the handwriting pad is used for when contacting the writing pencil and the handwriting pad is sent to described virtual Real helmet;
The person's handwriting pressure information that the writing pencil is used to will test is sent directly to the virtual reality helmet, or Person, the person's handwriting pressure information that will test are sent to the handwriting pad, are turned the person's handwriting pressure information by the handwriting pad It is sent to the virtual reality helmet.
CN201811013714.0A 2018-08-31 2018-08-31 Implementation method, device, equipment and the system of the handwriting input of virtual reality Pending CN109308132A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811013714.0A CN109308132A (en) 2018-08-31 2018-08-31 Implementation method, device, equipment and the system of the handwriting input of virtual reality

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811013714.0A CN109308132A (en) 2018-08-31 2018-08-31 Implementation method, device, equipment and the system of the handwriting input of virtual reality

Publications (1)

Publication Number Publication Date
CN109308132A true CN109308132A (en) 2019-02-05

Family

ID=65224280

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811013714.0A Pending CN109308132A (en) 2018-08-31 2018-08-31 Implementation method, device, equipment and the system of the handwriting input of virtual reality

Country Status (1)

Country Link
CN (1) CN109308132A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111427447A (en) * 2020-03-04 2020-07-17 青岛小鸟看看科技有限公司 Display method of virtual keyboard, head-mounted display equipment and system
CN111857365A (en) * 2019-04-19 2020-10-30 苹果公司 Stylus-based input for head-mounted devices

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150363980A1 (en) * 2014-06-17 2015-12-17 Valorisation-Recherche, Limited Partnership 3d virtual environment interaction system
CN105247453A (en) * 2012-11-29 2016-01-13 伊姆兰·哈迪斯 Virtual and augmented reality instruction system
CN106095174A (en) * 2016-06-12 2016-11-09 苏州芯沃科电子科技有限公司 Electronic whiteboard actively pen and touch input system and driving method
WO2018075054A1 (en) * 2016-10-21 2018-04-26 Hewlett-Packard Development Company, L.P. Virtual reality input
CN108267715A (en) * 2017-12-26 2018-07-10 青岛小鸟看看科技有限公司 Localization method and device, the virtual reality device and system of external equipment
CN108459782A (en) * 2017-02-17 2018-08-28 阿里巴巴集团控股有限公司 A kind of input method, device, equipment, system and computer storage media

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105247453A (en) * 2012-11-29 2016-01-13 伊姆兰·哈迪斯 Virtual and augmented reality instruction system
US20150363980A1 (en) * 2014-06-17 2015-12-17 Valorisation-Recherche, Limited Partnership 3d virtual environment interaction system
CN106095174A (en) * 2016-06-12 2016-11-09 苏州芯沃科电子科技有限公司 Electronic whiteboard actively pen and touch input system and driving method
WO2018075054A1 (en) * 2016-10-21 2018-04-26 Hewlett-Packard Development Company, L.P. Virtual reality input
CN108459782A (en) * 2017-02-17 2018-08-28 阿里巴巴集团控股有限公司 A kind of input method, device, equipment, system and computer storage media
CN108267715A (en) * 2017-12-26 2018-07-10 青岛小鸟看看科技有限公司 Localization method and device, the virtual reality device and system of external equipment

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111857365A (en) * 2019-04-19 2020-10-30 苹果公司 Stylus-based input for head-mounted devices
CN111427447A (en) * 2020-03-04 2020-07-17 青岛小鸟看看科技有限公司 Display method of virtual keyboard, head-mounted display equipment and system
CN111427447B (en) * 2020-03-04 2023-08-29 青岛小鸟看看科技有限公司 Virtual keyboard display method, head-mounted display device and system

Similar Documents

Publication Publication Date Title
KR102606785B1 (en) Systems and methods for simultaneous localization and mapping
US8933931B2 (en) Distributed asynchronous localization and mapping for augmented reality
US9613262B2 (en) Object detection and tracking for providing a virtual device experience
CN105378801B (en) Hologram snapshot grid
EP3067883B1 (en) Electronic device, method for recognizing playing of string instrument in electronic device, and method for providing feedback on playing of string instrument in electronic device
US10481689B1 (en) Motion capture glove
EP3283938B1 (en) Gesture interface
US20130010071A1 (en) Methods and systems for mapping pointing device on depth map
CN110457414A (en) Offline map processing, virtual objects display methods, device, medium and equipment
CN105264460A (en) Holographic object feedback
US20140009384A1 (en) Methods and systems for determining location of handheld device within 3d environment
CN108267715A (en) Localization method and device, the virtual reality device and system of external equipment
CN109255749A (en) From the map structuring optimization in non-autonomous platform of advocating peace
CN115956259A (en) Generating an underlying real dataset for a virtual reality experience
CN106527670A (en) Hand gesture interaction device
CN113672099A (en) Electronic equipment and interaction method thereof
CN110796703A (en) Pose tracking system, pose tracking method and storage medium
CN106527671A (en) Method for spaced control of equipment
CN109804638A (en) The double mode augmented reality interface of mobile device
US10671163B2 (en) Refining virtual mesh models through physical contacts
CN109308132A (en) Implementation method, device, equipment and the system of the handwriting input of virtual reality
Devi et al. Low cost tangible glove for translating sign gestures to speech and text in Hindi language
CN110262667B (en) Virtual reality equipment and positioning method
KR20240028481A (en) System and method for generating a three-dimensional map of an indoor space
CN106527669A (en) Interaction control system based on wireless signal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20190205