CN102540673A - Laser dot position determining system and method - Google Patents
Laser dot position determining system and method Download PDFInfo
- Publication number
- CN102540673A CN102540673A CN2012100766562A CN201210076656A CN102540673A CN 102540673 A CN102540673 A CN 102540673A CN 2012100766562 A CN2012100766562 A CN 2012100766562A CN 201210076656 A CN201210076656 A CN 201210076656A CN 102540673 A CN102540673 A CN 102540673A
- Authority
- CN
- China
- Prior art keywords
- photographic images
- position coordinates
- positional information
- relative position
- hot spot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B21/00—Projectors or projection-type viewers; Accessories therefor
- G03B21/14—Details
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B17/00—Details of cameras or camera bodies; Accessories therefor
- G03B17/48—Details of cameras or camera bodies; Accessories therefor adapted for combination with other photographic or optical apparatus
- G03B17/54—Details of cameras or camera bodies; Accessories therefor adapted for combination with other photographic or optical apparatus with projector
Abstract
The invention discloses a projector and a projected image calibrating method thereof. The method comprises the following steps of: acquiring a shot image from a shooting unit, wherein the shot image is obtained through shooting the projection area, which is projected on a display screen by the projector, by the shooting unit; and determining the exact position information of indication laser dot in the shot image according the received movement data, wherein the movement data is movement data of a laser emitting device, which is acquired by a gravity sensor unit in a laser recognition device, and the indication laser dot is formed by a visual light beam, which is emitted by the laser recognition device, on the display screen. According to the invention, the movement data of the laser emitting device is acquired and anti-shaking treatment is carried out according to the movement direction of the laser emitting device, so that the anti-shaking treatment can be carried out with no need of adopting complex algorithms such as filtering, the anti-shaking treatment of the laser dot can be realized only by carrying out simple numerical comparison, and the occupied computing resources of the system are reduced, thus the laser dot position determining speed is quicker, and the laser dot position determining efficiency is higher.
Description
Technical field
The present invention relates to shadow casting technique, relate in particular to the technology that in projection display system, prevents to cause maloperation because of the laser point shake.
Background technology
Projector is a kind of projection arrangement that is used on screen enlarged and displayed image, has been widely used in the demonstration activities such as multimedia teaching, meeting, speech at present.For more people also can see the content of demonstration in larger distance, generally all be that the file that needs show is projected on the display screen after converting light signal into.In general presentation process, needing has interactive function between demo content and the demonstrator.For this reason, the mutual method of present a kind of laser is risen gradually.Basic functional principle and process that laser is mutual are following:
Projector will need display image to convert the projected image that light signal projects formation amplification on the display screen into.Laser beam emitting device (or claiming laser designator) is a handheld apparatus, on display screen, forms hot spot in order to launch narrow visible light beam, is used for indicating interested object in the speech projected image, perhaps sends operational order through the control to it to computing machine.Signal processing unit intactly taken and be sent to the shooting headstock can with the projected image that shows on the display screen before display screen.The consecutive image that signal processing unit is taken camera is analyzed, and obtains indicating on the display screen hot spot position and the mobile message of (or claiming laser point).If the speaker points to figure with laser beam emitting device and goes up a certain zone of control; And push button when sending instruction; Signal processing unit not only will be known the type of button, but also will know indication hot spot this key of where supressing at display screen, just can make correct reaction.
But, laser beam emitting device since when hand-held inevitable shake can cause the difference of indication hot spot position in the camera photographic images, thereby to signal processing unit difficulty is brought in the location of indication hot spot.If judged the positional information of indication hot spot by accident, just may cause maloperation.For example, A as shown in Figure 1, B, C zone place button B place when the speaker will indicate hot spot, and push button when sending click commands, because the shake of hand may cause indicating hot spot to rock button A place, or button C place.If camera is taken in the projected image that gets off, the indication hot spot rocks button A place, and then may to judge the operation for the speaker by accident be to have clicked button A to signal processing unit, and cause maloperation.Therefore, when confirming the positional information of indication hot spot, need prevent trembling processing, some indication hot spots that cause because of shake are rocked neglect, obtain the positional information of definite indication hot spot, avoid causing maloperation.
At present, in order to prevent to judge by accident the positional information of indication hot spot, and cause maloperation in the prior art, the anti-shake algorithm of employing is comparatively complicated filter, data processing algorithm normally, to calculate the positional information that the indication hot spot is confirmed.Yet; Anti-shake filtering, the data processing algorithm of prior art are generally understood more complicated, need a large amount of calculating, make that the processing time of the positional information that judgement indication hot spot is confirmed is longer; And take the system with more computational resource, thereby cause confirming that the laser point position and speed is slow, efficient is low.
Summary of the invention
The embodiment of the invention provides a kind of laser point location determining method and system, and in order to simplify the anti-shake algorithm of laser point, the raising system confirms the efficient of laser point position.
According to an aspect of the present invention, a kind of laser point position determination system is provided, has comprised: laser beam emitting device, it comprises laser emission element, gravity sensor unit and first radio communication unit; The laser recognition device, it comprises that image unit, second radio communication unit and positional information confirm the unit; Wherein, said laser emission element be used for the visible emitting bundle on display screen, form the indication hot spot; Said gravity sensor unit is used to obtain the exercise data of said laser beam emitting device; First radio communication unit is used for said exercise data is carried out wireless transmission; Second radio communication unit is used to receive the exercise data that said first radio communication unit sends; The view field that said image unit is used for continuously projector being projected on display screen takes, and obtains the photographic images of successive frame; Said positional information confirms that the unit is used for obtaining photographic images from said image unit, and according to the exercise data that receives from second radio communication unit, confirms the definite positional information of indication hot spot in the said photographic images.
More excellent ground, said positional information confirm that the unit confirms that the definite positional information of indication hot spot in the said photographic images comprises: obtain first relative position coordinates, first relative position coordinates is the relative position coordinates of indication hot spot in the photographic images of pre-treatment; And confirm the distance value between first relative position coordinates and second relative position coordinates, wherein, second relative position coordinates is the relative position coordinates of indicating hot spot in the former frame photographic images of said photographic images when pre-treatment; After confirming the direction of motion of said laser beam emitting device,, determine said definite positional information of in the photographic images of pre-treatment, indicating hot spot according to said direction of motion and distance value according to said exercise data.
More excellent ground, said positional information confirms that the unit comprises: position coordinates is confirmed subelement, is used to obtain first relative position coordinates; Relative distance is confirmed subelement, is used for confirming the distance value between first relative position coordinates and second relative position coordinates; Direction of motion is confirmed subelement, is used for confirming according to said exercise data the direction of motion of said laser beam emitting device; Positional information is judged subelement, is used for according to said direction of motion and distance value, determines said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
Wherein, said positional information judges that subelement is used for: when said direction of motion is horizontal direction, be said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is confirmed second relative position coordinates less than first threshold; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
Said positional information judges that subelement also is used for: when said direction of motion is vertical direction, be said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is confirmed second relative position coordinates less than second threshold value; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
According to another aspect of the present invention, a kind of laser beam emitting device is provided, has comprised: laser emission element is used for the visible emitting bundle and on display screen, forms the indication hot spot; The gravity sensor unit is used to obtain the exercise data of said laser beam emitting device; First radio communication unit is used for said exercise data is carried out wireless transmission.
According to another aspect of the present invention, a kind of laser recognition device is provided also, has comprised: second radio communication unit is used to receive the exercise data that first radio communication unit of laser beam emitting device sends; Image unit, the view field that is used for continuously projector being projected on display screen takes, and obtains the photographic images of successive frame; Positional information is confirmed the unit, is used for obtaining photographic images from said image unit, and according to the exercise data that receives from second radio communication unit, confirms the definite positional information of indication hot spot in the said photographic images.
More excellent ground, said positional information confirms that the unit specifically is used to obtain first relative position coordinates, first relative position coordinates is the relative position coordinates of indication hot spot in the photographic images of pre-treatment; And confirm the distance value between first relative position coordinates and second relative position coordinates, wherein, second relative position coordinates is the relative position coordinates of indicating hot spot in the former frame photographic images of said photographic images when pre-treatment; After confirming the direction of motion of said laser beam emitting device,, determine said definite positional information of in the photographic images of pre-treatment, indicating hot spot according to said direction of motion and distance value according to said exercise data.
More excellent ground, said positional information confirms that the unit comprises: position coordinates is confirmed subelement, is used to obtain first relative position coordinates; Relative distance is confirmed subelement, is used for confirming the distance value between first relative position coordinates and second relative position coordinates; Direction of motion is confirmed subelement, is used for confirming according to said exercise data the direction of motion of said laser beam emitting device; Positional information is judged subelement, is used for according to said direction of motion and distance value, determines said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
Wherein, said positional information judges that subelement specifically is used for when said direction of motion is horizontal direction, is said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is then confirmed second relative position coordinates less than first threshold; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
Said positional information judges that subelement also is used for when said direction of motion is vertical direction, is said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is then confirmed second relative position coordinates less than second threshold value; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
According to another aspect of the present invention; A kind of laser point location determining method also is provided; Comprise: obtain photographic images from image unit, said photographic images is that view field that the image unit in the laser recognition device projects on display screen projector takes and obtains; According to the exercise data that receives, confirm the definite positional information of indication hot spot in the said photographic images; Wherein, Said exercise data is the exercise data of the said laser beam emitting device that obtained by the gravity sensor unit in the said laser beam emitting device; And via the transmission of first radio communication unit in the said laser beam emitting device, second radio communication unit reception in the said laser recognition device obtains; Said indication hot spot is that the laser emission element visible light emitted bundle in the said laser beam emitting device forms on display screen.
More excellent ground; Said according to the exercise data that receives; Confirm the definite positional information of indication hot spot in the said photographic images, comprising: obtain first relative position coordinates, first relative position coordinates is the relative position coordinates of indication hot spot in the photographic images of pre-treatment; Confirm the distance value between first relative position coordinates and second relative position coordinates, wherein, second relative position coordinates is the relative position coordinates of indication hot spot in the former frame photographic images of said photographic images when pre-treatment; Confirm the direction of motion of said laser beam emitting device according to said exercise data; According to said direction of motion and distance value, determine said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
Said according to said direction of motion and distance value; Determining said definite positional information when indication hot spot in the photographic images of pre-treatment comprises: when said direction of motion is horizontal direction; If it is said definite positional information of in the photographic images of pre-treatment, indicating hot spot that said distance value, is confirmed second relative position coordinates less than first threshold; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot; When said direction of motion is vertical direction, be said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is confirmed second relative position coordinates less than second threshold value; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
Be provided with gravity sensor owing in the laser beam emitting device in the laser point position determination system of the embodiment of the invention; The exercise data of the laser beam emitting device that can obtain; And laser beam emitting device sends to the laser recognition device with exercise data, makes the laser recognition device can judge the direction of motion of laser beam emitting device according to exercise data, prevents trembling processing according to the direction of motion of laser beam emitting device; Then needn't adopt complicated algorithms such as filtering to prevent trembling processing; That only need carry out that simple numeric ratio can realize laser point anti-ly trembles processing, has simplified the process of the definite positional information of judging the indication hot spot greatly, has reduced taking system resources in computation; Make and confirm that the laser point position and speed is faster, efficient is higher.
Description of drawings
Fig. 1 for the laser beam emitting device of prior art because of the hand-held synoptic diagram that causes indicating the hot spot shake;
Fig. 2 is the laser point position determination system inner structure synoptic diagram of the embodiment of the invention;
Fig. 3 a, 3b are that the positional information of the embodiment of the invention confirms that the unit confirms in the m frame photographic images method flow diagram of indication facula position information;
Fig. 4 be the embodiment of the invention according to direction of motion with apart from l, confirm in the m frame photographic images method flow diagram of the definite positional information of indication hot spot;
Fig. 5 confirms unit inner structure block diagram for the positional information of the embodiment of the invention.
Embodiment
For making the object of the invention, technical scheme and advantage clearer, below with reference to accompanying drawing and enumerate preferred embodiment, to further explain of the present invention.Yet, need to prove that many details of listing in the instructions only are in order to make the reader to one or more aspects of the present invention a thorough understanding arranged, even if there are not these specific details also can realize these aspects of the present invention.
Terms such as " module " that the application uses, " system " are intended to comprise the entity relevant with computing machine, such as but not limited to hardware, firmware, combination thereof, software or executory software.For example, module can be, but be not limited in: the thread of the process of moving on the processor, processor, object, executable program, execution, program and/or computing machine.For instance, the application program of moving on the computing equipment can be a module with this computing equipment.One or more modules can be positioned at an executory process and/or thread, and module also can be on the computing machine and/or be distributed between two or more the computing machines.In addition, can carry out these modules through the multiple computer-readable medium of having stored plurality of data structures.These modules can communicate through signal utilization this locality and/or remote process.
Laser beam emitting device is because the shake that causes when hand-held often has certain rules, can be because the reason of gravity or the reason of button firmly downwards cause the jitter amplitude on the vertical direction bigger, and the jitter amplitude of horizontal direction are less comparatively speaking.Main thought of the present invention is: G-sensor (Gravity-sensor, gravity sensor or title acceleration transducer) is installed, in order to obtain the exercise data of laser beam emitting device on hand-held laser beam emitting device; Also be provided with wireless communication module on the laser beam emitting device, exercise data is sent to the laser recognition device.Simultaneously, the image unit in the laser recognition device is taken the view field that projector projects on display screen, the indication hot spot that has laser beam emitting device visible emitting bundle on display screen, to form in the photographic images that obtains.The laser recognition device is determined the direction of motion of laser beam emitting device according to the exercise data that receives, and according to the direction of motion of laser beam emitting device the indication facula position in the photographic images prevent trembling processing, judges the definite positional information of indicating hot spot.
Specify the technical scheme of the embodiment of the invention below in conjunction with accompanying drawing.Laser point position determination system as shown in Figure 2, that the embodiment of the invention provides comprises: laser beam emitting device 201, laser recognition device 202.
Above-mentioned laser beam emitting device 201 is a hand-held, on display screen, forms hot spot in order to the visible emitting bundle, the object in the indication speech projected image.Comprise in the laser beam emitting device 201: laser emission element 211, gravity sensor unit 212, first radio communication unit 213.
After first radio communication unit 213 receives the exercise data that sends gravity sensor unit 212, send the exercise data that receives through wireless communication mode.This wireless communication mode can comprise various wireless communication modes such as bluetooth, wifi, 3G.
Include in the laser recognition device 202: image unit 221, second radio communication unit 222, positional information are confirmed unit 223.
Second radio communication unit 222 in the laser recognition device 202 is used to receive the exercise data that first radio communication unit 213 of laser beam emitting device 201 sends, and the exercise data that receives is sent to positional information confirms unit 223.
The view field that image unit 221 is used for continuously projector being projected on display screen takes, and obtains the photographic images of successive frame.Usually the brightness of view field will be higher than the brightness of the non-view field that does not have projection ray's irradiation, and image unit 221 view field that brightness is higher intactly takes.
Positional information confirms that unit 223 is used for obtaining photographic images from image unit 221, and according to the exercise data that receives from second radio communication unit 222, confirms the definite positional information of indication hot spot in the said photographic images.Be that position information confirms that unit 223 obtains every frame photographic images that image unit 221 is taken, and, confirm the definite positional information of indication hot spot in every frame photographic images according to the exercise data that receives from second radio communication unit 222.Positional information is confirmed to adopt the anti-algorithm of trembling in the positional information process of unit 223 indication hot spot in confirming every frame photographic images; Neglect the offset of the indication hot spot that causes because of shake; To judge the definite positional information of indication hot spot in photographic images; After preventing positional information because of shake erroneous judgement indication hot spot, bring to next step operation imprecise information is provided, cause next step maloperation.Positional information is confirmed to indicate the concrete grammar of the positional information of hot spot to introduce follow-up in unit 223 definite every frame photographic images.
Positional information confirms that the function of unit 223 can pass through processor such as arm processor, for example general processor, digital signal processor (DSP), and hardware such as processor that perhaps also can be through any routine, controller, microcontroller are realized.
The method that positional information in the embodiment of the invention laser point position determination system is confirmed to confirm indication facula position information in every frame photographic images in unit 223 comprises the steps: shown in Fig. 3 a
S310: obtain photographic images from image unit 221.The photographic images that obtains is that view field that the 221 pairs of projector of image unit in the laser recognition device 202 project on display screen takes and obtains.
S311:, confirm the definite positional information of indication hot spot in the photographic images according to the exercise data that receives.
The exercise data of the laser beam emitting device 201 that the gravity sensor unit 212 in the laser beam emitting device 201 obtains, and send via first radio communication unit 213 in the laser beam emitting device 201; Second radio communication unit 222 in the laser recognition device 202 receives and obtains the exercise data that first radio communication unit 213 sends.Positional information confirms that exercise data that unit 223 receives according to second radio communication unit 222 confirms the definite positional information of indication hot spot in the photographic images.
Particularly; Positional information confirms that unit 223 obtains the photographic images of successive frame from image unit 221; Confirm shown in Fig. 3 b, to comprise the steps: the flow process (m for greater than 1 natural number) of indication facula position information in the m frame photographic images according to the exercise data that receives
S301: the relative position coordinates P that obtains indication hot spot in m frame (or being called present frame) photographic images
m(X
m, Y
m).
Suppose that positional information confirms that the photographic images that pre-treatment is worked as in unit 223 is a m frame photographic images.After positional information confirms that hot spot is indicated in unit 223 in determining m frame photographic images, be reference point, calculate the relative coordinate of indication hot spot with respect to the upper left point of view field with the upper left point of view field.With this relative coordinate as P
m(X
m, Y
m).Certainly, those skilled in the art also can obtain the relative position coordinates of indication hot spot through other method, for example, are reference point with the upper right point of view field, calculate the relative coordinate of indication hot spot with respect to the upper right point of view field.
Because the position coordinates P that calculates
m(X
m, Y
m) be relative coordinate value with respect to reference point, therefore, can be because of the distance of projector distance, perhaps the distance of taking module distance causes in the photographic images view field to amplify or dwindle, and makes position coordinates P
mChange thereupon.
S302: calculate P
mWith P
M-1Between apart from l.
The formula of computed range l is specific as follows:
Wherein, X
M-1, Y
M-1Be respectively P
M-1X, the Y coordinate of relative position coordinates.
Wherein, P
M-1It is the relative position coordinates of indication hot spot in the m-1 frame photographic images.M-1 frame photographic images is the former frame photographic images of m frame photographic images, promptly works as the former frame photographic images of the photographic images of pre-treatment.Definite method of the relative position coordinates of indication hot spot repeats no more with identical described in the step S301 here in m-1 frame (or claiming at the preceding frame) photographic images.
S304:, confirm the direction of motion of laser emission element 211 according to the exercise data that receives from second radio communication unit 222.
Said exercise data specifically can be after the G-sensor of gravity sensor unit 212 detects the motion of laser beam emitting device 201, the acceleration information of three directions of XYZ of generation.According to the acceleration information of three directions of XYZ that receive, can determine the direction of motion of laser emission element 211.
The acceleration information of three directions of XYZ of for example, supposing to receive is respectively X
1, Y
1, Z
1If, X
1, Y
1, Z
1In the scope expressed like lower inequality, the direction of motion of then determining laser beam emitting device 201 is vertical direction, and is direction upwards:
-1.3G＜Z
1＜-0.7G (inequality 1)
-0.5G＜X
1＜0.5G (inequality 2)
-0.5G＜Y
1＜0.5G (inequality 3)
Wherein, G is a gravity unit of acceleration.Acceleration range in the above-mentioned inequality 1,2,3 is rule of thumb set for those skilled in the art.
If X
1, Y
1, Z
1Satisfy inequality 4,5,6 simultaneously, the direction of motion of then determining laser beam emitting device 201 is vertical direction, and is downward direction:
0.7G＜Z
1＜1.3G (inequality 4)
-0.5G＜X
1＜0.5G (inequality 5)
-0.5G＜Y
1＜0.5G (inequality 6)
In like manner, the method for determined level direction is also similar: if X
1, Y
1, Z
1Satisfy inequality 7,8,9 simultaneously, the direction of motion of then determining laser beam emitting device 201 is horizontal direction, and for forwards to:
-1.3G＜Y
1＜-0.7G (inequality 7)
-0.5G＜X
1＜0.5G (inequality 8)
-0.5G＜Z
1＜0.5G (inequality 9)
If X
1, Y
1, Z
1Satisfy inequality 10,11,12 simultaneously, the direction of motion of then determining laser beam emitting device 201 is horizontal direction, and is backward directions:
0.7G＜Y
1＜1.3G (inequality 10)
-0.5G＜X
1＜0.5G (inequality 11)
-0.5G＜Z
1＜0.5G (inequality 12)
If X
1, Y
1, Z
1Satisfy inequality 13,14,15 simultaneously, the direction of motion of then determining laser beam emitting device 201 is horizontal direction, and is direction to the right:
-1.3G＜X
1＜-0.7G (inequality 13)
-0.5G＜Y
1＜0.5G (inequality 14)
-0.5G＜Z
1＜0.5G (inequality 15)
If X
1, Y
1, Z
1Satisfy inequality 16,17,18 simultaneously, the direction of motion of then determining laser beam emitting device 201 is horizontal direction, and is direction left:
0.7＜X
1＜1.3G (inequality 16)
-0.5G＜Y
1＜0.5G (inequality 17)
-0.5G＜Z
1＜0.5G (inequality 18)
Owing to confirm the technology that direction of motion is well known to those skilled in the art according to exercise data, introduce no longer in detail here.
S305:, confirm the definite positional information of indication hot spot in the m frame photographic images according to the direction of motion of determining with apart from l.
Among the step S305, according to the direction of motion of determining with apart from l, confirm the concrete grammar of the definite positional information of indication hot spot in the m frame photographic images, flow process is as shown in Figure 4, comprises the steps:
S401: judge that direction of motion is horizontal direction or vertical direction; If horizontal direction, then execution in step S402; If vertical direction, then execution in step S403.
S402: relatively apart from l and first threshold; Whether judging distance l less than first threshold, if, execution in step S404 then; Otherwise, execution in step S405.
First threshold is a threshold value that is provided with on the horizontal direction, for those skilled in the art rule of thumb are provided with: can think when the unconscious shake of hand-held laser beam emitting device P
mWith P
M-1Between can be smaller apart from l; If hand-held laser beam emitting device changes indication facula position, then P consciously
mWith P
M-1Between can be bigger apart from l.According to this experience,, thereby judge the relative position coordinates P that indicates hot spot in the m frame photographic images through the size of comparison apart from l and first threshold
mBe to depart from the P in the former frame photographic images because of shake
M-1The position, still change is indicated facula position and is departed from the P in the former frame photographic images consciously
M-1The position.
If less than first threshold, then think P apart from l
mBe to depart from the P in the former frame photographic images because of shake
M-1The position; Otherwise, P
mBe because change is indicated facula position and departed from the P in the former frame photographic images consciously
M-1The position.
For judging P
mBe to depart from the P in the former frame photographic images because of shake
M-1The situation of position, then execution in step S404 neglects P
m, think that the position of working as indication hot spot in the photographic images of pre-treatment does not have to change with respect to the position of indication hot spot in the former frame photographic images, has promptly carried out preventing trembling processing, with P
M-1Definite positional information as indication hot spot in the photographic images (i.e. m frame photographic images) of pre-treatment.
For judging P
mBe because change is indicated facula position and departed from the P in the former frame photographic images consciously
M-1The situation of position, then execution in step S405 confirms that the position of indication hot spot in the photographic images of pre-treatment has changed to P really
mThe position.
S403: relatively apart from the l and second threshold value; Whether judging distance l less than second threshold value, if, execution in step S404 then; Otherwise, execution in step S405.
Similarly, those skilled in the art are provided with the threshold value on the vertical direction according to actual conditions and experience, i.e. second threshold value.Rule of thumb it has been generally acknowledged that when the unconscious shake of hand-held laser beam emitting device P
mWith P
M-1Between can be smaller apart from l; If hand-held laser beam emitting device changes indication facula position, then P consciously
mWith P
M-1Between can be bigger apart from l.According to this experience,, thereby judge the relative position coordinates P that indicates hot spot in the m frame photographic images through the size of comparison apart from the l and second threshold value
mBe to depart from the P in the former frame photographic images because of shake
M-1The position, still change is indicated facula position and is departed from the P in the former frame photographic images consciously
M-1The position.
If less than second threshold value, then think P apart from l
mBe to depart from the P in the former frame photographic images because of shake
M-1The position; Otherwise, P
mBe because change is indicated facula position and departed from the P in the former frame photographic images consciously
M-1The position.
For judging P
mBe to depart from the P in the former frame photographic images because of shake
M-1The situation of position, then execution in step S404 neglects P
m, think that the position of working as indication hot spot in the photographic images of pre-treatment does not have to change with respect to the position of indication hot spot in the former frame photographic images, has promptly carried out preventing trembling processing, with P
M-1Definite positional information as indication hot spot in the photographic images (i.e. m frame photographic images) of pre-treatment.
For judging P
mBe because change is indicated facula position and departed from the P in the former frame photographic images consciously
M-1The situation of position, then execution in step S405 confirms that the position of indication hot spot in the photographic images of pre-treatment has changed to P really
mThe position.
Because hand-held laser beam emitting device usually can be greater than the jitter amplitude of horizontal direction at the jitter amplitude of vertical direction, therefore, those skilled in the art can be according to the actual conditions setting, and the ratio first threshold that second threshold value is provided with is big.
S404: confirm P
M-1Definite positional information for indication hot spot in the photographic images of pre-treatment.Promptly determine and indicate facula position information to be specially P in the m frame photographic images
M-1Coordinate figure.
S405: confirm P
mDefinite positional information for indication hot spot in the photographic images of pre-treatment.Promptly determine and indicate facula position information to be specially P in the m frame photographic images
mCoordinate figure.
Above-mentioned positional information is confirmed a kind of concrete structure that unit 223 is inner, and is as shown in Figure 5, comprising: position coordinates confirms that subelement 501, relative distance confirms that subelement 502, direction of motion confirms that subelement 503, positional information judge subelement 504.
Position coordinates confirms that subelement 501 is used to obtain first relative position coordinates, and first relative position coordinates refers to the relative position coordinates of indication hot spot in the photographic images of pre-treatment.
Relative distance confirms that subelement 502 is used for confirming the distance value between first relative position coordinates and second relative position coordinates, and second relative position coordinates refers to the relative position coordinates of indication hot spot in the former frame photographic images of the photographic images of pre-treatment.
Direction of motion confirms that subelement 503 is used for confirming that according to positional information exercise data that unit 223 receives confirms the direction of motion of laser beam emitting device 201.
Positional information judges that subelement 504 is used for confirming that according to direction of motion direction of motion and relative distances that subelement 503 is confirmed confirm the distance value that subelement 502 is confirmed, determines the definite positional information of indication hot spot in the photographic images of pre-treatment.
Particularly, if direction of motion confirms that subelement 503 determined direction of motion are horizontal direction: then positional information is judged subelement 504 more said distance value and first thresholds; If it is said definite positional information of in the photographic images of pre-treatment, indicating hot spot that said distance value, is then confirmed second relative position coordinates less than first threshold; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
If direction of motion confirms that subelement 503 determined direction of motion are vertical direction: then positional information is judged the subelement 504 more said distance values and second threshold value; If it is said definite positional information of in the photographic images of pre-treatment, indicating hot spot that said distance value, is then confirmed second relative position coordinates less than second threshold value; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
Above-mentioned laser recognition device 202 can be built in the projector; Positional information is confirmed the main control module of the positional information of the indication hot spot that unit 223 outputs are continuous to projector; Main control module can be confirmed the positional information of the indication hot spot that unit 223 sends according to positional information; And out of Memory, operate accordingly and respond like current operation interface, current key information etc.: for example, switch the page, select option etc.The main control module can be processor such as the arm processor on embedded main board; For example general processor, digital signal processor (DSP), special IC (ASIC), field programmable gate array (FPGA) or other PLD, discrete gate or transistor logic device, discrete hardware components or its combination in any perhaps also can be processor, controller, microcontroller or the state machine of any routine.Said main control module also possibly be embodied as the combination of computing equipment, and for example, the combination of DSP and microprocessor, a plurality of microprocessor, one or more microprocessor combine with the DSP kernel, perhaps any other this kind structure.
Perhaps, laser recognition device 202 can be arranged at computing machine that projector links to each other in, the positional information of continuous indication hot spot is provided for computing machine.Computing machine can be confirmed the positional information of the indication hot spot that unit 223 sends according to positional information, and mouse location information is related with it.Promptly let the mouse present position with the indication hot spot position synchronous; Then according to mouse location information; And out of Memory, operate accordingly and respond like current operation interface, current key information etc.: for example, switch the page, select option, preserve operations such as document.
Be provided with gravity sensor owing in the laser beam emitting device in the laser point position determination system of the embodiment of the invention; The exercise data of the laser beam emitting device that can obtain; And laser beam emitting device sends to the laser recognition device with exercise data, makes the laser recognition device can judge the direction of motion of laser beam emitting device according to exercise data, prevents trembling processing according to the direction of motion of laser beam emitting device; Then needn't adopt complicated algorithms such as filtering to prevent trembling processing; That only need carry out that simple numeric ratio can realize laser point anti-ly trembles processing, has simplified the process of the definite positional information of judging the indication hot spot greatly, has reduced taking system resources in computation; Make and confirm that the laser point position and speed is faster, efficient is higher.
One of ordinary skill in the art will appreciate that all or part of step that realizes in the foregoing description method is to instruct relevant hardware to accomplish through program; This program can be stored in the computer read/write memory medium, as: ROM/RAM, magnetic disc, CD etc.
The above only is a preferred implementation of the present invention; Should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the principle of the invention; Can also make some improvement and retouching, these improvement and retouching also should be regarded as protection scope of the present invention.
Claims (10)
1. laser point position determination system comprises:
Laser beam emitting device, it comprises laser emission element, gravity sensor unit and first radio communication unit;
The laser recognition device, it comprises that image unit, second radio communication unit and positional information confirm the unit;
Wherein, said laser emission element be used for the visible emitting bundle on display screen, form the indication hot spot;
Said gravity sensor unit is used to obtain the exercise data of said laser beam emitting device;
First radio communication unit is used for the said exercise data of wireless transmission;
Second radio communication unit is used to receive the exercise data that said first radio communication unit sends;
The view field that said image unit is used for continuously projector being projected on display screen takes, and obtains the photographic images of successive frame;
Said positional information confirms that the unit is used for obtaining photographic images from said image unit, and according to the exercise data that receives from second radio communication unit, confirms the definite positional information of indication hot spot in the said photographic images.
2. the system of claim 1; It is characterized in that; Said positional information confirms that the unit confirms in the said photographic images that the definite positional information of indication hot spot comprises: obtain first relative position coordinates, first relative position coordinates is the relative position coordinates of indication hot spot in the photographic images of pre-treatment; And confirm the distance value between first relative position coordinates and second relative position coordinates, wherein, second relative position coordinates is the relative position coordinates of indicating hot spot in the former frame photographic images of said photographic images when pre-treatment; After confirming the direction of motion of said laser beam emitting device,, determine said definite positional information of in the photographic images of pre-treatment, indicating hot spot according to said direction of motion and distance value according to said exercise data; And,
Said positional information confirms that the unit comprises:
Position coordinates is confirmed subelement, is used to obtain first relative position coordinates;
Relative distance is confirmed subelement, is used for confirming the distance value between first relative position coordinates and second relative position coordinates;
Direction of motion is confirmed subelement, is used for confirming according to said exercise data the direction of motion of said laser beam emitting device;
Positional information is judged subelement, is used for according to said direction of motion and distance value, determines said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
3. system as claimed in claim 2 is characterized in that, said positional information judges that subelement is used for:
When said direction of motion is horizontal direction, be said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is confirmed second relative position coordinates less than first threshold; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot;
When said direction of motion is vertical direction, be said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is confirmed second relative position coordinates less than second threshold value; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
4. laser beam emitting device comprises:
Laser emission element is used for the visible emitting bundle and on display screen, forms the indication hot spot;
The gravity sensor unit is used to obtain the exercise data of said laser beam emitting device;
First radio communication unit is used for said exercise data is carried out wireless transmission.
5. laser recognition device comprises:
Second radio communication unit is used to receive the exercise data that first radio communication unit of laser beam emitting device sends;
Image unit, the view field that is used for continuously projector being projected on display screen takes, and obtains the photographic images of successive frame;
Positional information is confirmed the unit, is used for obtaining photographic images from said image unit, and according to the exercise data that receives from second radio communication unit, confirms the definite positional information of indication hot spot in the said photographic images.
6. device as claimed in claim 5 is characterized in that, said positional information confirms that the unit is used for:
Obtain first relative position coordinates, first relative position coordinates is the relative position coordinates of indication hot spot in the photographic images of pre-treatment; And confirm the distance value between first relative position coordinates and second relative position coordinates, wherein, second relative position coordinates is the relative position coordinates of indicating hot spot in the former frame photographic images of said photographic images when pre-treatment; After confirming the direction of motion of said laser beam emitting device,, determine said definite positional information of in the photographic images of pre-treatment, indicating hot spot according to said direction of motion and distance value according to said exercise data; And,
Said positional information confirms that the unit comprises:
Position coordinates is confirmed subelement, is used to obtain first relative position coordinates;
Relative distance is confirmed subelement, is used for confirming the distance value between first relative position coordinates and second relative position coordinates;
Direction of motion is confirmed subelement, is used for confirming according to said exercise data the direction of motion of said laser beam emitting device;
Positional information is judged subelement, is used for according to said direction of motion and distance value, determines said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
7. device as claimed in claim 6 is characterized in that, said positional information judges that subelement is used for:
When said direction of motion is horizontal direction, be said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is confirmed second relative position coordinates less than first threshold; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot;
When said direction of motion is vertical direction, be said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is then confirmed second relative position coordinates less than second threshold value; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
8. laser point location determining method comprises:
Obtain photographic images from image unit, said photographic images is that view field that the image unit in the laser recognition device projects on display screen projector takes and obtains;
According to the exercise data that receives, confirm the definite positional information of indication hot spot in the said photographic images;
Wherein, Said exercise data is the exercise data of the said laser beam emitting device that obtained by the gravity sensor unit in the said laser beam emitting device; And via the transmission of first radio communication unit in the said laser beam emitting device, second radio communication unit reception in the said laser recognition device obtains; Said indication hot spot is that the laser emission element visible light emitted bundle in the said laser beam emitting device forms on display screen.
9. method as claimed in claim 8, said according to the exercise data that receives, confirm to indicate in the said photographic images the definite positional information of hot spot, comprising:
Obtain first relative position coordinates, first relative position coordinates is the relative position coordinates of indication hot spot in the photographic images of pre-treatment;
Confirm the distance value between first relative position coordinates and second relative position coordinates, wherein, second relative position coordinates is the relative position coordinates of indication hot spot in the former frame photographic images of said photographic images when pre-treatment;
Confirm the direction of motion of said laser beam emitting device according to said exercise data;
According to said direction of motion and distance value, determine said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
10. method as claimed in claim 9, said according to said direction of motion and distance value, determine the said definite positional information of hot spot of in the photographic images of pre-treatment, indicating and comprise:
When said direction of motion is horizontal direction, be said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is confirmed second relative position coordinates less than first threshold; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot;
When said direction of motion is vertical direction, be said definite positional information of in the photographic images of pre-treatment, indicating hot spot if said distance value, is confirmed second relative position coordinates less than second threshold value; Otherwise, confirm that first relative position coordinates is said definite positional information of in the photographic images of pre-treatment, indicating hot spot.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510285173.7A CN104918028A (en) | 2012-03-21 | 2012-03-21 | Jitter removing method for laser spot positions on projection screen |
| CN201210076656.2A CN102540673B (en) | 2012-03-21 | 2012-03-21 | Laser dot position determining system and method |
| CN201510285207.2A CN104834165B (en) | 2012-03-21 | 2012-03-21 | Position determining method for laser spot on projection screen |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210076656.2A CN102540673B (en) | 2012-03-21 | 2012-03-21 | Laser dot position determining system and method |
Related Child Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510285207.2A Division CN104834165B (en) | 2012-03-21 | 2012-03-21 | Position determining method for laser spot on projection screen |
| CN201510285173.7A Division CN104918028A (en) | 2012-03-21 | 2012-03-21 | Jitter removing method for laser spot positions on projection screen |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102540673A true CN102540673A (en) | 2012-07-04 |
| CN102540673B CN102540673B (en) | 2015-06-10 |
Family
ID=46347872
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510285173.7A Pending CN104918028A (en) | 2012-03-21 | 2012-03-21 | Jitter removing method for laser spot positions on projection screen |
| CN201210076656.2A Active CN102540673B (en) | 2012-03-21 | 2012-03-21 | Laser dot position determining system and method |
| CN201510285207.2A Active CN104834165B (en) | 2012-03-21 | 2012-03-21 | Position determining method for laser spot on projection screen |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510285173.7A Pending CN104918028A (en) | 2012-03-21 | 2012-03-21 | Jitter removing method for laser spot positions on projection screen |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510285207.2A Active CN104834165B (en) | 2012-03-21 | 2012-03-21 | Position determining method for laser spot on projection screen |
Country Status (1)
| Country | Link |
|---|---|
| CN (3) | CN104918028A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105912145A (en) * | 2016-04-15 | 2016-08-31 | 武汉邮电科学研究院 | Laser pen mouse system and image positioning method thereof |
| CN107110637A (en) * | 2014-12-22 | 2017-08-29 | 赛博光学公司 | Calibration to three-dimension measuring system is updated |
| CN107740964A (en) * | 2017-11-21 | 2018-02-27 | 武汉亚格光电技术股份有限公司 | A kind of optical dynamic therapy combined light source system |
| CN110793393A (en) * | 2019-10-10 | 2020-02-14 | 成都贝瑞光电科技股份有限公司 | Laser multi-shooting-point identification system |
| CN112702586A (en) * | 2020-12-21 | 2021-04-23 | 成都极米科技股份有限公司 | Projector virtual touch tracking method, device and system based on visible light |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106774992A (en) * | 2016-12-16 | 2017-05-31 | 深圳市虚拟现实技术有限公司 | The point recognition methods of virtual reality space location feature |
| CN109669567B (en) * | 2017-10-13 | 2020-10-16 | 华为技术有限公司 | Control method and terminal |
| WO2019072254A1 (en) | 2017-10-13 | 2019-04-18 | 华为技术有限公司 | Control method and terminal |
| CN108509072B (en) * | 2017-10-30 | 2019-09-24 | 嘉兴仁光乌镇科技有限公司 | The method, apparatus, equipment and computer readable storage medium of coordinate anti-trembling on screen |
| CN108509071B (en) * | 2017-10-30 | 2019-08-09 | 嘉兴仁光乌镇科技有限公司 | The method, apparatus, equipment and computer readable storage medium of coordinate anti-trembling on screen |
| CN109828681B (en) * | 2017-11-20 | 2022-10-11 | 深圳市Tcl高新技术开发有限公司 | Laser pen optical flow trajectory tracking method, projection equipment and computer readable storage medium |
| TWI701497B (en) * | 2018-10-17 | 2020-08-11 | 英錡科技股份有限公司 | Projection device with auto-correction of spots |
| CN114947664A (en) * | 2022-06-22 | 2022-08-30 | 汇智机器人科技(深圳)有限公司 | Control method and system applied to laser guidance of sweeping robot |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN200950629Y (en) * | 2006-05-25 | 2007-09-19 | 光峰科技股份有限公司 | Projector capable of picking up image and bulletin system having the same |
| CN102279656A (en) * | 2010-06-10 | 2011-12-14 | 鼎亿数码科技(上海)有限公司 | Laser plane positioning system and implementation method thereof |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPWO2006085580A1 (en) * | 2005-02-10 | 2008-06-26 | 株式会社タクラム・デザイン・エンジニアリング | Pointer light tracking method, program and recording medium therefor |
| CN202033723U (en) * | 2011-03-31 | 2011-11-09 | 西安电子科技大学 | Wireless control input device of projector |
-
2012
- 2012-03-21 CN CN201510285173.7A patent/CN104918028A/en active Pending
- 2012-03-21 CN CN201210076656.2A patent/CN102540673B/en active Active
- 2012-03-21 CN CN201510285207.2A patent/CN104834165B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN200950629Y (en) * | 2006-05-25 | 2007-09-19 | 光峰科技股份有限公司 | Projector capable of picking up image and bulletin system having the same |
| CN102279656A (en) * | 2010-06-10 | 2011-12-14 | 鼎亿数码科技(上海)有限公司 | Laser plane positioning system and implementation method thereof |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107110637A (en) * | 2014-12-22 | 2017-08-29 | 赛博光学公司 | Calibration to three-dimension measuring system is updated |
| CN107110637B (en) * | 2014-12-22 | 2019-11-01 | 赛博光学公司 | The calibration of three-dimension measuring system is updated |
| CN105912145A (en) * | 2016-04-15 | 2016-08-31 | 武汉邮电科学研究院 | Laser pen mouse system and image positioning method thereof |
| CN107740964A (en) * | 2017-11-21 | 2018-02-27 | 武汉亚格光电技术股份有限公司 | A kind of optical dynamic therapy combined light source system |
| CN110793393A (en) * | 2019-10-10 | 2020-02-14 | 成都贝瑞光电科技股份有限公司 | Laser multi-shooting-point identification system |
| CN110793393B (en) * | 2019-10-10 | 2022-06-14 | 成都贝瑞光电科技股份有限公司 | Laser multi-shooting-point identification system |
| CN112702586A (en) * | 2020-12-21 | 2021-04-23 | 成都极米科技股份有限公司 | Projector virtual touch tracking method, device and system based on visible light |
Also Published As
| Publication number | Publication date |
|---|---|
| CN102540673B (en) | 2015-06-10 |
| CN104918028A (en) | 2015-09-16 |
| CN104834165A (en) | 2015-08-12 |
| CN104834165B (en) | 2017-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102540673A (en) | Laser dot position determining system and method | |
| US7522151B2 (en) | Coordinate-based display object movement restriction method | |
| CN106999772B (en) | System, program, and method for operating screen by linking display with a plurality of controllers connected via network | |
| US9304607B2 (en) | Pointer positioning method of handheld pointer device | |
| US9122123B2 (en) | Projector having focus adjusting section for adjusting projection based on projection distance information, computer program product, and image projecting method | |
| CN111078018A (en) | Touch control method of display, terminal device and storage medium | |
| CN111578839B (en) | Obstacle coordinate processing method and device, electronic equipment and readable storage medium | |
| JP2016045874A (en) | Information processor, method for information processing, and program | |
| US20140118246A1 (en) | Gesture recognition using an electronic device including a photo sensor | |
| US20160188018A1 (en) | System, drawing method and information processing apparatus | |
| JP2011209579A (en) | Image display system and control method thereof | |
| JP2007114820A (en) | Portable pointer device and display system | |
| JP4836099B2 (en) | Input system and method, and computer program | |
| JP2015118442A (en) | Information processor, information processing method, and program | |
| CN108801259A (en) | A kind of indoor navigation method, device and mobile terminal | |
| JP6401268B2 (en) | Non-contact operation detection device | |
| US20220236854A1 (en) | Personal digital assistant | |
| US10067576B2 (en) | Handheld pointer device and tilt angle adjustment method thereof | |
| CN102736378B (en) | Projection apparatus and projection method | |
| JPWO2008093395A1 (en) | Input system and method, and computer program | |
| CN116048313B (en) | Cursor control method, cursor control device and storage medium | |
| EP3032380A1 (en) | Image projection apparatus, and system employing interactive input-output capability | |
| US20180307302A1 (en) | Electronic device and method for executing interactive functions | |
| CN115793873A (en) | Method and device for acquiring cursor position and storage medium | |
| JP5830899B2 (en) | Projection system, projection apparatus, projection method and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |