US11313970B2 - Time of flight camera - Google Patents
Time of flight camera Download PDFInfo
- Publication number
- US11313970B2 US11313970B2 US17/113,121 US202017113121A US11313970B2 US 11313970 B2 US11313970 B2 US 11313970B2 US 202017113121 A US202017113121 A US 202017113121A US 11313970 B2 US11313970 B2 US 11313970B2
- Authority
- US
- United States
- Prior art keywords
- light
- component
- time
- reflecting
- flight camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
- G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/481—Constructional features, e.g. arrangements of optical elements
- G01S7/4814—Constructional features, e.g. arrangements of optical elements of transmitters alone
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B21/00—Projectors or projection-type viewers; Accessories therefor
- G03B21/14—Details
- G03B21/20—Lamp housings
- G03B21/2006—Lamp housings characterised by the light source
- G03B21/2033—LED or laser light sources
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B21/00—Projectors or projection-type viewers; Accessories therefor
- G03B21/14—Details
- G03B21/20—Lamp housings
- G03B21/208—Homogenising, shaping of the illumination light
Definitions
- the present disclosure relates to the technical field of camera devices, particularly to a time-of-flight camera using a light-diffusing component to uniformize the intensity of projected light.
- the three-dimensional structured light measurement is a method first to project scattered spots or a code pattern then obtain the depth of the photographed object by matching with feature points.
- the time-of-flight measurement is a method first to use the phase difference between the projected light and the received light for the calculation of the flight time of the light between the camera and the object, then the flight time is used for the calculation of the distance between each of the points on the object and the camera, and finally the depth of each of the points on the object can be calculated. Since the method of three-dimensional structured light measurement requires a clear reflection image for measurement, objects to be measured should be within a certain distance which is not far. Objects in a farther distance could be measured by the method of time-of-flight measurement as it measures the phase difference between projected light and received light.
- the intensity of the reflected light is relatively low at where the intensity of the projected light is low, so the received light cannot be accurately measured.
- the embodiments of the present disclosure provide a time-of-flight camera to solve the problem of incomplete or inaccurate measurement results caused by uneven light intensity of conventional time-of-flight cameras.
- the present disclosure provides a time-of-flight camera, comprising a light-emitting module and a light-receiving module.
- the light-emitting module comprises a light source component, a light-reflecting component, and a light-diffusing component.
- the light source component emits a first light in a first direction.
- the light-reflecting component and the light-diffusing component are disposed on the light path of the first light.
- a second light formed by the first light passes through the light-reflecting component and the light-diffusing component.
- the second light travels toward a second direction to an object to be measured.
- the object to be measured reflects the second light.
- the first direction is intersecting with the second direction.
- the light-receiving module receives the reflected second light and performs the function of sensing.
- the intensity of the first light from the light source component can be uniformized after being diffused by diffuser.
- the problem of uneven intensity of the projected light of conventional time-of-flight cameras can be handled, allowing the measurement result to be comprehensive and accurate for possible measurements on farther objects.
- FIG. 1 is a schematic diagram of the structural configuration of a time-of-flight camera of the first embodiment of the present disclosure
- FIG. 2 is an operation status diagram of the time-of-flight camera in FIG. 1 ;
- FIG. 3 is a schematic diagram showing the comparison between the light intensity of the time-of-flight camera of the first embodiment and the light intensity of a conventional time-of-flight camera.
- FIG. 4 is a schematic diagram of a time-of-flight camera of the second embodiment of the present disclosure.
- FIG. 5 is a schematic diagram of a time-of-flight camera of the third embodiment of the present disclosure.
- FIG. 6 is a schematic diagram of a time-of-flight camera of the fourth embodiment of the present disclosure.
- the terms “include”, “contain”, and any variation thereof are intended to cover a non-exclusive inclusion. Therefore, a process, method, object, or device that includes a series of elements not only includes these elements, but also includes other elements not specified expressly, or may include inherent elements of the process, method, object, or device. If no more limitations are made, an element limited by “include a/an . . . ” does not exclude other same elements existing in the process, the method, the article, or the device which includes the element.
- FIG. 1 is a schematic diagram of the structural configuration of a time-of-flight camera of the first embodiment of the present disclosure.
- FIG. 2 is an operation status diagram of the time-of-flight camera in FIG. 1 .
- this embodiment provides a time-of-flight camera 100 comprising a light-emitting module 10 and a light-receiving module 20 . Light is emitted from the light-emitting module 10 to the object to be measured, then is reflected by the object to be measured and finally enters the light-receiving module 20 .
- the travel time of the light can be calculated, and hence the distance of each of the points on the object to be measured can be calculated to measure the depth of each of the points on the object to be measured. Finally, by simulation, a three-dimensional image can be obtained.
- the light-emitting module 10 and the light-receiving module 20 are disposed in a housing 30 , forming an emitting module accommodating space 31 and a receiving module accommodating space 32 separated by a partition wall W.
- a light-emitting opening 33 is provided on the housing 30 at a position corresponding to the emitting module accommodating space 31
- a light-receiving opening 34 is provided on the housing 30 at a position corresponding to the receiving module accommodating space 32 .
- the light-emitting module 10 comprises a light source component 11 , a light-reflecting component 12 , and a light-diffusing component 13 .
- the light source component 11 emits a first light B 1 toward a first direction L 1 .
- the first light B 1 passes through the light-diffusing component 13 , it is scattered by the light-diffusing component 13 to form a beam having uniform intensity.
- the beam with uniform intensity is then reflected by the light-reflecting component 12 to form a second light beam B 2 , which travels in a second direction L 2 and leaves the light-emitting module 10 through the light-emitting opening 33 until it reaches the object to be measured.
- the light source component 11 , the light-reflecting component 12 , and the light-diffusing component 13 are disposed in the emitting module accommodating space 31 of the housing 30 .
- the light source component 11 is installed on a base plate 14 , the light source component 11 and the base plate 14 are disposed in a container 15 , and the light-diffusing component 13 is disposed at the opening of the container 15 .
- the container 15 , the light source component 11 , and the light-diffusing component 13 are disposed at a position in the emitting module accommodating space 31 close to the housing 30 and meanwhile on the opposite side of partition wall W.
- the light-reflecting component 12 is adjacent to the light-emitting opening 33 of the housing 30 and is adjacent to the partition wall W of the housing 30 .
- the container 15 , the light source component 11 , and the light-diffusing component 13 could also be disposed at a position adjacent to the partition wall W of the housing 30
- the light-reflecting component 12 is disposed at a position in the emitting module accommodating space 31 where the position is close to the wall surface of the housing 30 and is opposite to the partition wall W
- the light-emitting opening 33 corresponds to the position of the light-reflecting component 12 .
- the positions of the container 15 and the light-reflecting component 12 are not limited, as long as the light path of the light-emitting module 10 passes through the light-diffusing component 13 and the light-reflecting component 12 and then passes through the light-emitting opening 33 of the housing 30 .
- the light-diffusing component 13 is disposed on the light path of the first light B 1 between the light source component 11 and the light-reflecting component 12 , and the first light B 1 first passes through the light-diffusing component 13 and then the light-reflecting component 12 .
- the light-diffusing component 13 does not need to be limited to be disposed at the opening of the container 15 , as long as it is on the light path of the first light B 1 .
- the first direction L 1 and the second direction L 2 intersect, that is, the first direction L 1 and the second direction L 2 form an angle. By appropriately adjusting the reflecting surface of the light-reflecting component 12 , the first direction L 1 can be perpendicular to the second direction L 2 .
- the light source component 11 could be a laser chip, such as a vertical-cavity surface-emitting laser chip (VCSEL), and the base plate 14 could be, for example, a ceramic base plate.
- the light-diffusing component 13 could be an optical film with a plurality of particles dispersed inside, by which the light can be scattered to generate uniform intensity light.
- the light-diffusing component 13 could also be an optical film with a plurality of microstructures inside, for example, a tapered bump microstructure, by which the light can be refracted or reflected in different directions to diffuse.
- the beam angle of the first light B 1 is between 22 and 26 degrees.
- the beam angle of the second light B 2 can be in the range of 45 to 60 degrees.
- FIG. 3 shows the relationship between the light intensity multiples of the emitted light of the time-of-flight camera of this embodiment and the camera of prior art in the horizontal direction (X-axis) and the angle range.
- the curve C 1 represents the distribution of the light intensity multiples of the time-of-flight camera of prior art in the horizontal direction (X-axis)
- the curve C 2 represents the distribution of the light intensity multiples of the time-of-flight camera of this embodiment in the horizontal direction (X-axis).
- the light of the time-of-flight camera of the prior art is concentrated in the range of ⁇ 20 degrees of the horizontal direction.
- the range of the light distribution is expanded to a range of ⁇ 40 degrees though the light intensity in the center (0 degrees) is just slightly weakened.
- the light-reflecting component 12 comprises a base 121 and a prism 122 .
- the prism 122 could be, for example, a right-angle cone prism, of which the hypotenuse (reflective surface) is configured to be on an inclined surface of the base 121 .
- the first light B 1 is incident on one side surface of the prism 122 , and after being reflected by the reflective surface, the first light B 1 exits the prism 122 from the other side surface of the prism 122 and becomes a second light B 2 , allowing the direction of light incident on the prism 122 to be perpendicular to the direction of light exiting from the prism 122 .
- the light-reflecting component can also include a base and a reflecting mirror. The reflecting mirror can reflect the first light B 1 traveling in the first direction L 1 to make it travel in the second direction L 2 .
- the second light B 2 After arriving at the object to be measured, the second light B 2 is reflected to form a third light B 3 .
- the third light B 3 is then received by the light-receiving module 20 after passing through the light-receiving opening 34 .
- the light-receiving module 20 comprises an optical lens component 21 and a light-sensing component 22 .
- the third light B 3 is imaged on the light-sensing component 22 through the optical lens component 21 , and each of the pixels of the light-sensing component 22 would generate a corresponding image data to be transmitted to the data processing module 40 , which is able to calculate the phase difference between the first light B 1 and the third light B 3 based on the image data from each of the pixels and the data of the first light B 1 of the light source component 11 for the travel time of the light.
- the distance of each of the points of the object to be measured corresponding to each of the pixels can be calculated, and the depth information of each of the points of the object to be measured can be further calculated based on the above information to simulate a three-dimensional image.
- the light-sensing component 22 could be a CCD or a CMOS
- the data processing module 40 could be a processor and a digital/analog conversion component.
- FIG. 4 is a schematic diagram of a time-of-flight camera of the second embodiment of the present disclosure. Partial components of this embodiment are identical to those of the first embodiment shown in FIG. 1 , the same components are given the same symbols, and descriptions are omitted. The difference between this embodiment and the first embodiment shown in FIG. 1 is that the time-of-flight camera 100 ′ of this embodiment further comprises a driving module 50 .
- the driving module 50 is connected to the data processing module 40 in a manner capable of transmitting signals and is connected to the base 121 of the light-reflecting component 12 .
- the data processing module 40 sends a control signal according to the light-receiving state of the light-sensing component 22 , and the driving module 50 receives the control signal from the data processing module 40 to drive the base 121 of the light-reflecting component 12 to move linearly in any direction.
- the prism 122 can be moved to change the projecting area of the second light B 2 , so that the reflected third light B 3 could cover the entire light-sensing component 22 .
- the driving module 50 could be, for example, a stepper motor.
- FIG. 5 is a schematic diagram of a time-of-flight camera of the third embodiment of the present disclosure. Partial components of this embodiment are identical to those of the first embodiment shown in FIG. 1 , the same components are given the same symbols, and descriptions are omitted. The difference between this embodiment and the first embodiment shown in FIG. 1 is that the time-of-flight camera 100 ′′ of this embodiment further comprises a driving module 50 ′.
- the driving module 50 ′ is connected to the data processing module 40 in a manner capable of transmitting signals and is connected to the base 121 of the light-reflecting component 12 .
- the data processing module 40 sends a control signal according to the light-receiving state of the light-sensing component 22 , and the driving module 50 receives the control signal from the data processing module 40 to drive the base 121 of the light-reflecting component 12 to rotate within any range of angle.
- the prism 122 can be moved to change the projecting direction and area of the second light B 2 , so that the reflected third light B 3 could cover the entire light-sensing component 22 .
- the driving module 50 ′ could be, for example, a voice coil motor.
- the driving module could also be a combination of driving module 50 and driving module 50 ′. That is, the base 121 could be driven to move linearly and rotate so that the prism 122 can be moved to change the projecting direction and area of the second light B 2 .
- FIG. 6 is a schematic diagram of a time-of-flight camera of the fourth embodiment of the present disclosure. Partial components of the time-of-flight camera 100 ′′′ of this embodiment are identical to those of the first embodiment shown in FIG. 1 , the same components are given the same symbols, and descriptions are omitted.
- the difference between this embodiment and the first embodiment shown in FIG. 1 is that the position of the light-diffusing component 13 of this embodiment is different from that of the first embodiment.
- the light-diffusing component 13 is disposed at the light-emitting opening 33 of the housing 30
- the light-diffusing component 13 of the first embodiment is disposed at the opening of the container 15 .
- the light source component 11 emits a first light B 1 in a first direction L 1 .
- the first light B 1 is first reflected by the light-reflecting component 12 and travels in the second direction L 2 , and then passes through the light-diffusing component 13 , to form a second light B 2 .
- the light-diffusing component 13 is on the light path of the first light B 1 emitted by the light-emitting module 10 , and the first light B 1 passes through the light-reflecting component 12 and then passes through the light-diffusing component 13 , but the light-diffusing component 13 is not limited to be disposed at the light-emitting opening 33 of the housing 30 , as long as it is on the path of the reflected light of the first light B 1 reflected by the light-reflecting component 12 .
- the second light B 2 passing through the light-reflecting component 12 is also diffused.
- the beam angle of the first light B 1 is between 22 to 26 degrees. After being diffused by the light-diffusing component 13 , the beam angle of the second light B 2 can be in the range of 45 to 60 degrees.
- embodiments of the present disclosure provide a time-of-flight camera comprising a light-diffusing component diffusing the light emitted by the light source component into light with uniform intensity (amount of light per unit area), thereby increasing the light intensity of the low light area.
- a time-of-flight camera of the present disclosure can also be applied to a light-emitting module emitting light with a small beam angle for objects in the same distance, presenting measurement results which are the same.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Electromagnetism (AREA)
- Optics & Photonics (AREA)
- Optical Radar Systems And Details Thereof (AREA)
- Studio Devices (AREA)
Abstract
Description
Claims (9)
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201922270742.7U CN211478651U (en) | 2019-12-17 | 2019-12-17 | TOF camera |
| CN201922270742.7 | 2019-12-17 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20210181345A1 US20210181345A1 (en) | 2021-06-17 |
| US11313970B2 true US11313970B2 (en) | 2022-04-26 |
Family
ID=72363396
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US17/113,121 Active US11313970B2 (en) | 2019-12-17 | 2020-12-07 | Time of flight camera |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US11313970B2 (en) |
| CN (1) | CN211478651U (en) |
| TW (1) | TWM609585U (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021258256A1 (en) * | 2020-06-22 | 2021-12-30 | 欧菲光集团股份有限公司 | Camera module, and vehicle-mounted device and electronic device having same |
| CN114545366B (en) * | 2022-01-13 | 2025-07-08 | 昆山丘钛微电子科技股份有限公司 | TOF camera module and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5892610A (en) * | 1997-07-10 | 1999-04-06 | Agfa Division--Bayer Corporation | Scanning system with error-correcting deflector |
| CN101673036A (en) * | 2008-09-11 | 2010-03-17 | 台达电子工业股份有限公司 | Light guide assembly and projection device comprising same |
| US20110310356A1 (en) * | 2009-02-27 | 2011-12-22 | Epicrystals Oy | Image projector and an illuminating unit suitable for use in an image projector |
| JP2019032815A (en) * | 2017-06-06 | 2019-02-28 | トヨタ モーター エンジニアリング アンド マニュファクチャリング ノース アメリカ,インコーポレイティド | System and method for optimizing battery precharge using adjusted traffic prediction |
| JP2020144048A (en) * | 2019-03-07 | 2020-09-10 | ソニーセミコンダクタソリューションズ株式会社 | Light source device and distance measurement device |
-
2019
- 2019-12-17 CN CN201922270742.7U patent/CN211478651U/en active Active
-
2020
- 2020-12-03 TW TW109215990U patent/TWM609585U/en unknown
- 2020-12-07 US US17/113,121 patent/US11313970B2/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5892610A (en) * | 1997-07-10 | 1999-04-06 | Agfa Division--Bayer Corporation | Scanning system with error-correcting deflector |
| CN101673036A (en) * | 2008-09-11 | 2010-03-17 | 台达电子工业股份有限公司 | Light guide assembly and projection device comprising same |
| US20110310356A1 (en) * | 2009-02-27 | 2011-12-22 | Epicrystals Oy | Image projector and an illuminating unit suitable for use in an image projector |
| JP2019032815A (en) * | 2017-06-06 | 2019-02-28 | トヨタ モーター エンジニアリング アンド マニュファクチャリング ノース アメリカ,インコーポレイティド | System and method for optimizing battery precharge using adjusted traffic prediction |
| JP2020144048A (en) * | 2019-03-07 | 2020-09-10 | ソニーセミコンダクタソリューションズ株式会社 | Light source device and distance measurement device |
Also Published As
| Publication number | Publication date |
|---|---|
| US20210181345A1 (en) | 2021-06-17 |
| CN211478651U (en) | 2020-09-11 |
| TWM609585U (en) | 2021-03-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6862097B2 (en) | Three-dimensional shape measuring method, and three-dimensional shape measuring apparatus | |
| US4534637A (en) | Camera with active optical range finder | |
| CN102124368B (en) | distance measuring system | |
| US11629959B2 (en) | Surveying instrument | |
| JP7565342B2 (en) | Projector for solid-state LIDAR systems | |
| JP2005140996A (en) | Projector | |
| JP7360298B2 (en) | surveying equipment | |
| US11313970B2 (en) | Time of flight camera | |
| JPH10221064A (en) | Optical distance-measuring device | |
| JP2017072466A (en) | Light wave distance measuring device | |
| CN109358435A (en) | A kind of the adjustment device and method of adjustment of doubly telecentric camera lens verticality | |
| JP4185397B2 (en) | Optical detector | |
| CN112710237B (en) | Alignment system and alignment method | |
| JP7421437B2 (en) | surveying equipment | |
| CN116697886A (en) | measuring device | |
| CN115507768A (en) | Depth imaging device, method, system and computer readable storage medium | |
| CN112004000A (en) | Light-emitting device and image acquisition device using same | |
| CN211505897U (en) | TOF module, three-dimensional scanning device, and electronic device | |
| CN112415740A (en) | Structured light control device | |
| CN216956618U (en) | Structured light projector, camera module and electronic equipment | |
| TW202026659A (en) | Sensing device with conical reflector for making two-dimensional optical radar | |
| TWI742473B (en) | Structure light emitting module and image collecting device | |
| CN112867905A (en) | Optical engine for three-dimensional detection and three-dimensional detection equipment | |
| CN111537977B (en) | Sensing device of two-dimensional optical radar manufactured by conical reflecting mirror | |
| EP4160145B1 (en) | Surveying instrument |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AS | Assignment |
Owner name: GUANGZHOU LUXVISIONS INNOVATION TECHNOLOGY LIMITED, CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:YEN, SHIH-CHIEH;REEL/FRAME:054557/0512 Effective date: 20201201 |
|
| FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: APPLICATION DISPATCHED FROM PREEXAM, NOT YET DOCKETED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
| STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
| MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |