CN107517340B - Camera module and electronic equipment - Google Patents
Camera module and electronic equipment Download PDFInfo
- Publication number
- CN107517340B CN107517340B CN201710865186.0A CN201710865186A CN107517340B CN 107517340 B CN107517340 B CN 107517340B CN 201710865186 A CN201710865186 A CN 201710865186A CN 107517340 B CN107517340 B CN 107517340B
- Authority
- CN
- China
- Prior art keywords
- image
- lens
- state
- visible light
- light signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
- G06F21/31—User authentication
- G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Security & Cryptography (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Studio Devices (AREA)
Abstract
The embodiment of the invention discloses a camera module and electronic equipment, wherein the camera module comprises: the first lens is used for collecting visible light signals; the first photosensitive chip is used for generating a first image according to the visible light signal collected by the first lens; the second lens is used for collecting visible light signals in a first state and collecting infrared light signals in a second state; the second photosensitive chip is used for generating a second image according to the visible light signal collected by the second lens and generating a third image according to the infrared light signal reflected by the human eyes collected by the second lens; the infrared light signal transmitter is used for transmitting an infrared light signal; the switching device is used for controlling the second lens to be switched between the first state and the second state so as to realize double shooting, improve the safety of the electronic equipment, reduce the volume of the camera module, and facilitate the development of light and thin of the electronic equipment when being applied to the electronic equipment.
Description
Technical Field
The invention relates to the technical field of camera shooting, in particular to a camera shooting module and electronic equipment comprising the same.
Background
With the development of electronic technology, more and more electronic devices integrate functions such as mobile phone payment and password unlocking. However, security identification using security information such as an account number and a password has a problem that the security information is easy to leak, which causes a potential safety hazard in the hidden information of the existing electronic device.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present invention provide a camera module, which reduces the size of the camera module while achieving dual-camera and improving the security of an electronic device, and is beneficial to the development of light and thin electronic devices when being applied to the electronic devices.
The embodiment of the invention provides the following technical scheme:
a camera module, comprising:
the first lens is used for collecting visible light signals;
the first photosensitive chip is used for generating a first image according to the visible light signal collected by the first lens;
the second lens is used for collecting visible light signals in a first state and collecting infrared light signals in a second state;
the second photosensitive chip is used for generating a second image according to the visible light signal collected by the second lens and generating a third image according to the infrared light signal reflected by human eyes collected by the second lens;
the infrared light signal transmitter is used for transmitting an infrared light signal;
and the switching device is used for controlling the second lens to be switched between the first state and the second state.
Optionally, the second lens includes a filter structure, the optical signal passed by the filter structure in the first state includes a visible light signal, and the filter structure passes only an infrared light signal in the second state.
Optionally, the filtering structure is a filter.
Optionally, the first lens is a main lens, the second lens is an auxiliary lens, and the second image is used for compensating the preset parameter of the first image.
Optionally, the preset parameter is depth of field or brightness or color saturation.
An electronic device, comprising:
the camera module of any of the above;
a processor for generating an output visible light image from the first image and the second image and for matching security information from the third image.
Optionally, the processor is configured to obtain a distance between a camera object and the electronic device according to the first image and the second image, and generate a depth image of the camera object.
Optionally, the processor is configured to perform preset feature extraction according to the third image, generate a fourth image according to the preset feature, compare the fourth image with the preset image, generate a control instruction when the fourth image matches the preset image, and respond to the control instruction.
Optionally, the preset features include an iris, a pupil, a sclera, and an eyelid of the human eye.
Compared with the prior art, the technical scheme has the following advantages:
when the camera module provided by the embodiment of the invention is applied to electronic equipment, the second lens can be used for collecting the infrared light signals reflected by human eyes in a second state, and the second photosensitive chip is used for generating a third image comprising human eye images according to the infrared light signals, so that the third image is used for matching and identifying subsequent passwords, and the security of the privacy information of the electronic equipment is improved. In addition, in the camera module provided in the embodiment of the present invention, the second lens collects a visible light signal in the first state, the second photosensitive chip generates the second image according to the visible light signal collected by the second lens in the first state, so that the second image and the first image can jointly generate a captured image of a camera image, thereby implementing double shooting and improving a shooting effect, and the third image and the second image share the second lens and the second photosensitive signal, thereby implementing double shooting and improving the safety of an electronic device, reducing the volume of the camera module, and facilitating the development of thinning and lightening of the electronic device when being applied to the electronic device.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a camera module according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a principle of computing a depth of field by an electronic device according to an embodiment of the present invention;
fig. 4 is a schematic diagram of the electronic device shown in the figure for calculating the principle equivalent of the depth of field.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.
Next, the present invention will be described in detail with reference to the drawings, wherein the cross-sectional views illustrating the structure of the device are not enlarged partially according to the general scale for convenience of illustration when describing the embodiments of the present invention, and the drawings are only examples, which should not limit the scope of the present invention. In addition, the three-dimensional dimensions of length, width and depth should be included in the actual fabrication.
As described in the background section, security identification is performed by using security information such as an account number and a password, which causes a problem that the security information is easy to leak, resulting in a potential safety hazard in the hidden information of the existing electronic device.
In view of this, an embodiment of the present invention provides a camera module, as shown in fig. 1, including:
the first lens 10, the first lens 10 is used for collecting visible light signals;
the first photosensitive chip 20, the first photosensitive chip 20 is configured to generate a first image according to the visible light signal collected by the first lens 10;
the second lens 30 is used for collecting visible light signals in a first state and collecting infrared light signals in a second state;
the second photosensitive chip 40 is configured to generate a second image according to the visible light signal collected by the second lens 30, and generate a third image according to the infrared light signal reflected by the human eye collected by the second lens 30;
an infrared light signal transmitter (not shown in the figure) for transmitting an infrared light signal;
a switching device 50 for controlling the second lens 30 to switch between the first state and the second state.
Because the irises of the human eyes are used as individual characteristic information, the problems of copying and stealing do not exist, the irises of different people are different and have specificity, and the irises are used as passwords for privacy information such as safe payment and unlocking, so that the safety is extremely high. Therefore, when the camera module provided by the embodiment of the invention is applied to an electronic device, the second lens 30 can be used to collect the infrared light signal reflected by human eyes in the second state, and the second photosensitive chip 40 is used to generate a third image including a human eye image according to the infrared light signal, so that the third image is used to perform matching identification of subsequent passwords, and the security of the privacy information of the electronic device is improved. Moreover, in the camera module provided in the embodiment of the present invention, the second lens 30 collects the visible light signal in the first state, the second photosensitive chip 40 generates the second image according to the visible light signal collected by the second lens 30 in the first state, so that the second image and the first image can jointly generate the captured image of the camera image, thereby implementing double-shot, and improving the shooting effect, and the third image and the second image share the second lens 30 and the second photosensitive signal, so that the volume of the camera module is reduced while implementing double-shot, and improving the security of the electronic device, and when the camera module is applied to the electronic device, the development of the lightness and thinness of the electronic device is facilitated.
On the basis of the foregoing embodiment, in an embodiment of the present invention, the second lens includes a filter structure, and the optical signal passed by the filter structure in the first state includes a visible light signal, and only an infrared light signal passes in the second state, so that the second lens 30 can collect the visible light signal in the first state and collect the infrared light signal in the second state.
Specifically, on the basis of the above embodiment, in an embodiment of the present invention, the filter structure is a filter, and the optical signal that passes through the filter in a state where no voltage is applied includes a visible light signal, and only passes through an infrared light signal in an external voltage state.
On the basis of any of the above embodiments, in an embodiment of the present invention, the first lens 10 is a main lens, the second lens 30 is an auxiliary lens, and the second image is used for compensating for a preset parameter of the first image; in other embodiments of the present invention, the first lens 10 is an auxiliary lens, the second lens 30 is a main lens, and the first image is used for compensating for a preset parameter of the second image, which is not limited in the present invention, as the case may be.
The following description will be given taking the first lens 10 as a main lens and the second lens 30 as an auxiliary lens as an example.
On the basis of the above embodiment, in an embodiment of the present invention, the preset parameter is color saturation, the first image is an entire image of a shooting object, and the second image is a partial image; in another embodiment of the present invention, the preset parameter is brightness, the first image is a color image, the second image is a black-and-white image, and the brightness of the second image is greater than the brightness of the first image; in another embodiment of the present invention, the preset parameter is a depth of field, the first image is an entire image of the subject, and the second image is also an entire image of the subject, which is not limited in this respect, as the case may be.
In addition, the camera module provided by the embodiment of the invention further comprises: and a dual-lens holder 60 for fixing the first lens 10 and the second lens 30.
Correspondingly, an embodiment of the present invention further provides an electronic device, as shown in fig. 2, where the electronic device includes: the camera module 100 and the processor 200 according to any of the embodiments above, wherein the processor 200 is configured to generate an output visible light image according to the first image and the second image, and perform security information matching according to the third image.
On the basis of the foregoing embodiments, in an embodiment of the present invention, the processor 200 is configured to obtain a distance between a camera object and the electronic device according to the first image and the second image, and generate a depth image of the camera object.
Specifically, as shown in fig. 3, the distance between the optical center Q1 of the first lens and the optical center Q2 of the second lens is b; the distance from the shooting object P to the optical center connection of the first lens and the second lens is z; an imaging point of the photographic object on the first photosensitive chip is P1, and an imaging point on the second photosensitive chip is P2; the focal length of the first lens and the second lens (i.e. the distance between the first lens and the first photosensitive chip, and the distance between the second lens and the second photosensitive chip) is f. The processor obtains a distance x1 from an imaging point P1 on the first photo-sensitive chip to the first lens optical center Q1 in a first direction from the first image, and obtains a distance x2 from an imaging point P2 on the second photo-sensitive chip to the second lens optical center Q2 in the first direction from the second image. Wherein the first direction is parallel to a connecting line direction of an optical center Q1 of the first lens and an optical center Q2 of the second lens.
For convenience of calculation, the position of the first photosensitive chip is equivalent to the position between the first lens and the shooting object, the position of the second photosensitive chip is equivalent to the position between the second lens and the shooting object, as shown in fig. 4, the distance between an imaging point P1 on the first photosensitive chip and the optical center Q1 of the first lens is x1, the distance between an imaging point P2 on the second photosensitive chip and the optical center Q2 of the second lens in the first direction obtained from the second image is x2, and then (z + f)/z is (b + x1+ x2)/b according to the triangle-like principle, so that f/z is (x1+ x2)/b, and further the distance z between the shooting object P and the electronic device is f b/(x1+ x2), thereby realizing the depth of field effect.
On the basis of any of the foregoing embodiments, in an embodiment of the present invention, the processor is further configured to control the second lens to switch between the first state and the second state by controlling the switching device. Specifically, when the switching device is a filter structure, the processor controls the filter structure to switch between the first state and the second state by applying a voltage and not applying a voltage to the filter structure and controlling the filter structure to pass only an infrared light signal and a visible light signal, but the invention is not limited thereto, and is determined as the case may be.
On the basis of the foregoing embodiment, in an embodiment of the present invention, the processor is further configured to activate the infrared light signal emitter to emit an infrared light signal when the second lens is in the second state.
On the basis of any of the foregoing embodiments, in a specific embodiment of the present invention, the processor is configured to perform preset feature extraction according to the third image, generate a fourth image according to the preset feature, compare the fourth image with a preset image, and generate a control instruction when the fourth image matches the preset image, and respond to the control instruction. Wherein the predetermined characteristics include an iris, a pupil, a sclera, and an eyelid of the human eye.
Specifically, the processor extracts the iris, the pupil, the sclera and the eyelid of the human eye from the third image, determines an inner circle through the iris and the pupil, determines an outer circle through the iris and the sclera, determines a boundary through the eyelid and the iris, generates a fourth image, compares the fourth image with a preset image, generates a control instruction when the fourth image is matched with the preset image, and responds to the control instruction to realize functions such as unlocking or payment.
On the basis of the above embodiment, in an embodiment of the present invention, before performing the preset feature extraction according to the third image, the processor is further configured to adjust the size of the iris to a suitable size through normalization, and perform processing such as brightness, contrast, and smoothing to provide an iris identification degree; when the fourth image is compared with a preset image, iris characteristic information is extracted from the iris image through a specific algorithm (such as two-dimensional Gabor wavelet) and is coded so as to match the iris characteristic information with the preset image.
In summary, the camera module and the electronic device provided in the embodiments of the present invention may collect the infrared light signal reflected by the human eyes in the second state by using the second lens, and generate the third image including the human eye image according to the infrared light signal by using the second photosensitive chip, so that the third image is used to perform matching identification on the subsequent password, thereby improving the security of the privacy information of the electronic device. In addition, in the camera module provided in the embodiment of the present invention, the second lens collects a visible light signal in the first state, the second photosensitive chip generates the second image according to the visible light signal collected by the second lens in the first state, so that the second image and the first image can jointly generate a captured image of a camera image, thereby implementing double shooting and improving a shooting effect, and the third image and the second image share the second lens and the second photosensitive signal, thereby implementing double shooting and improving the safety of an electronic device, reducing the volume of the camera module, and facilitating the development of thinning and lightening of the electronic device when being applied to the electronic device.
In the description, each part is described in a progressive manner, each part is emphasized to be different from other parts, and the same and similar parts among the parts are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. The utility model provides a module of making a video recording which characterized in that includes:
the first lens is used for collecting visible light signals;
the first photosensitive chip is used for generating a first image according to the visible light signal collected by the first lens;
the second lens is used for collecting visible light signals in a first state and collecting infrared light signals in a second state;
the second photosensitive chip is used for generating a second image according to the visible light signal collected by the second lens and generating a third image according to the infrared light signal reflected by human eyes collected by the second lens;
the infrared light signal transmitter is used for transmitting an infrared light signal;
switching means for controlling the second lens to switch between the first state and the second state;
the second lens comprises a filtering structure, the filtering structure comprises a filter, the filter passes through optical signals including visible light signals under the state of no voltage application, and only passes through infrared light signals under the state of external voltage.
2. The camera module of claim 1, wherein the optical signals passed by the filter structure in the first state comprise visible light signals and the filter structure in the second state passes only infrared light signals.
3. The camera module of claim 1, wherein the first lens is a primary lens and the second lens is a secondary lens, and the second image is used for compensating for a preset parameter of the first image.
4. The camera module of claim 3, wherein the predetermined parameter is depth of field or brightness or color saturation.
5. An electronic device, comprising:
the camera module of any one of claims 1-4;
a processor for generating an output visible light image from the first image and the second image and for matching security information from the third image.
6. The electronic device according to claim 5, wherein the processor is configured to obtain a distance between a camera object and the electronic device from the first image and the second image, and generate a depth image of the camera object.
7. The electronic device of claim 5, wherein the processor is configured to perform a preset feature extraction according to the third image, generate a fourth image according to the preset feature, compare the fourth image with a preset image, and generate a control command when the fourth image matches the preset image, and respond to the control command.
8. The electronic device of claim 7, wherein the predetermined features include an iris, a pupil, a sclera, and an eyelid of a human eye.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710865186.0A CN107517340B (en) | 2017-09-22 | 2017-09-22 | Camera module and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710865186.0A CN107517340B (en) | 2017-09-22 | 2017-09-22 | Camera module and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107517340A CN107517340A (en) | 2017-12-26 |
| CN107517340B true CN107517340B (en) | 2020-11-27 |
Family
ID=60726600
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710865186.0A Active CN107517340B (en) | 2017-09-22 | 2017-09-22 | Camera module and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107517340B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020047860A1 (en) * | 2018-09-07 | 2020-03-12 | Guangdong Oppo Mobile Telecommunications Corp., Ltd. | Electronic device and image processing method |
| CN111147700A (en) * | 2018-11-02 | 2020-05-12 | 南昌欧菲光电技术有限公司 | Camera device and intelligent terminal |
| CN111147699B (en) * | 2018-11-02 | 2022-01-07 | 南昌欧菲光电技术有限公司 | Electronic equipment, camera device and mounting base thereof |
| CN110248073B (en) * | 2019-07-08 | 2021-04-09 | Oppo（重庆）智能科技有限公司 | Camera module, electronic device and manufacturing method of camera module |
| CN111291731B (en) * | 2020-03-27 | 2024-07-19 | 深圳阜时科技有限公司 | Optical sensing device and electronic equipment |
| CN113242364B (en) * | 2021-03-29 | 2023-04-18 | 杭州涂鸦信息技术有限公司 | Black light camera |
| CN114650359A (en) * | 2022-03-22 | 2022-06-21 | 维沃移动通信有限公司 | Camera module and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103123433A (en) * | 2011-11-21 | 2013-05-29 | 鸿富锦精密工业（深圳）有限公司 | Lens module |
| CN203193791U (en) * | 2013-03-26 | 2013-09-11 | 深圳市中控生物识别技术有限公司 | Network camera with face identification function |
| CN104301633A (en) * | 2014-11-03 | 2015-01-21 | 倪蔚民 | Visible light and biological recognition combined photo-electronic imaging system and method of mobile terminal |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103152517B (en) * | 2013-02-06 | 2018-06-22 | 北京中科虹霸科技有限公司 | Imaging modules and mobile equipment for mobile iris identification equipment |
| KR102261857B1 (en) * | 2014-11-27 | 2021-06-07 | 삼성전자주식회사 | Image sensor and apparatus and method for acquiring image by using the same |
| CN206042197U (en) * | 2016-08-16 | 2017-03-22 | 上海通铭信息科技股份有限公司 | Law enforcement that two cameras zoomed can be realized and appearance is taken notes |
| CN206370880U (en) * | 2017-01-25 | 2017-08-01 | 徐鹤菲 | A kind of dual camera imaging system and mobile terminal |
-
2017
- 2017-09-22 CN CN201710865186.0A patent/CN107517340B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103123433A (en) * | 2011-11-21 | 2013-05-29 | 鸿富锦精密工业（深圳）有限公司 | Lens module |
| CN203193791U (en) * | 2013-03-26 | 2013-09-11 | 深圳市中控生物识别技术有限公司 | Network camera with face identification function |
| CN104301633A (en) * | 2014-11-03 | 2015-01-21 | 倪蔚民 | Visible light and biological recognition combined photo-electronic imaging system and method of mobile terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107517340A (en) | 2017-12-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107517340B (en) | Camera module and electronic equipment | |
| US10956714B2 (en) | Method and apparatus for detecting living body, electronic device, and storage medium | |
| CN105187726B (en) | Multifunctional mobile image processing device and processing method | |
| WO2017049922A1 (en) | Image information collection apparatus, image collection method, and application thereof | |
| US11321575B2 (en) | Method, apparatus and system for liveness detection, electronic device, and storage medium | |
| US10205883B2 (en) | Display control method, terminal device, and storage medium | |
| CN106716451B (en) | Iris recognition device and manufacturing method and application thereof | |
| US9798927B2 (en) | Mobile terminal iris recognition method and device having human-computer interaction mechanism | |
| US20230360254A1 (en) | Pose estimation method and related apparatus | |
| CN109993115B (en) | Image processing method and device and wearable device | |
| CN110287900B (en) | Verification method and verification device | |
| US10855885B2 (en) | Image processing apparatus, method therefor, and storage medium | |
| CN104933344A (en) | Mobile terminal user identity authentication device and method based on multiple biological feature modals | |
| KR20180109109A (en) | Method of recognition based on IRIS recognition and Electronic device supporting the same | |
| CN204948210U (en) | A kind of image information collecting device | |
| CN101533473B (en) | Equipment for acquisition and processing of binocular iris images with single USB interfaces | |
| CN106503680B (en) | Guidance for mobile terminal iris recognition indicates man-machine interface system and method | |
| CN103106401A (en) | Mobile terminal iris recognition device with human-computer interaction mechanism and method | |
| CN209328043U (en) | With the electronic equipment shielded comprehensively | |
| CN113741681B (en) | Image correction method and electronic equipment | |
| CN209170442U (en) | Electronic equipment with high screen accounting | |
| CN204791017U (en) | Mobile terminal users authentication device based on many biological characteristics mode | |
| CN111598065B (en) | Depth image acquisition method, living body identification method, apparatus, circuit, and medium | |
| CN111291338B (en) | User identification method, user identification device, storage medium and head-mounted device | |
| CN105956538A (en) | Image rendering apparatus and method based on an RGB camera and iris camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |