CN112711135B - Scanning dot matrix projection system and electronic equipment - Google Patents

Scanning dot matrix projection system and electronic equipment Download PDF

Info

Publication number
CN112711135B
CN112711135B CN201911019606.9A CN201911019606A CN112711135B CN 112711135 B CN112711135 B CN 112711135B CN 201911019606 A CN201911019606 A CN 201911019606A CN 112711135 B CN112711135 B CN 112711135B
Authority
CN
China
Prior art keywords
light
angle
sawtooth
scanning
diffusion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911019606.9A
Other languages
Chinese (zh)
Other versions
CN112711135A (en
Inventor
魏文雄
董泳江
王帆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN201911019606.9A priority Critical patent/CN112711135B/en
Publication of CN112711135A publication Critical patent/CN112711135A/en
Application granted granted Critical
Publication of CN112711135B publication Critical patent/CN112711135B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B26/00Optical devices or arrangements for the control of light using movable or deformable optical elements
    • G02B26/08Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
    • G02B26/10Scanning systems
    • G02B26/106Scanning systems having diffraction gratings as scanning elements, e.g. holographic scanners
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00Reducing energy consumption in communication networks
    • Y02D30/70Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Mechanical Optical Scanning Systems (AREA)

Abstract

The embodiment of the application discloses scanning dot matrix projection system, including light source, steering mirror and the permeable diffusion subassembly of light. The light source emits collimated light, the steering mirror is arranged in the emitting direction of the collimated light, the collimated light which enters the steering mirror is reflected to obtain reflected light, and the steering mirror is arranged to be capable of deflecting, so that a first emitting angle of the reflected light which is obtained through the steering mirror is changed within a preset angle range. The diffusion subassembly sets up in the outgoing direction of reflection ray, and when reflection ray incided diffusion subassembly, diffusion subassembly can be to this reflection ray diffusion for the angle of emergence when reflection ray jets out from diffusion subassembly is greater than the incident angle when reflection ray jets into diffusion subassembly, and the light that jets out from diffusion subassembly shines on the measured object face, in order to realize the scanning to the measured object face. Due to the diffusion effect of the diffusion component, the scanning angle is enlarged, and the complexity and the power consumption of a driving system of the micromirror are reduced.

Description

Scanning dot matrix projection system and electronic equipment
Technical Field
The application relates to the technical field of optical three-dimensional sensing, in particular to a scanning dot matrix projection system and electronic equipment.
Background
With the application of three-dimensional imaging technology, 3-dimensional (3D) sensors become hot spots of a new generation of terminal sensors, such as mobile phone sensors. The terminal may utilize a 3D sensor to enable 3D imaging and recognition of the target. Technologies applicable to the field Of 3D sensing include stereo imaging, structured light, a Time Of Flight (TOF) camera, a TOF camera, and a TOF scanner, wherein the TOF scanner has the advantages Of long working distance, high resolution, and the like, and is a preferred technology Of the next generation Of 3D sensors.
The scanning type dot matrix projection system based on the micro-mirror is the most common scheme of the TOF scanner transmitting end, and the scanning of the target is realized by utilizing the two-dimensional rotation of the micro-mirror, so that the three-dimensional imaging is realized at the TOF scanner receiving end.
However, the requirement Of the solution on the two-dimensional rotation angle Of the micromirror, the modulation speed Of the emission light source (the two-dimensional rotation speed Of the micromirror) and the like is very high, taking 30 frames per second Of Video Graphics Array (VGA) image calculation as an example, at least 9 million points need to be collected per second, the rotation angle Of the micromirror, i.e. the Field Of View (FOV), is required to exceed ± 30 degrees, and the complexity and the power consumption Of the driving system Of the micromirror are increased to a great extent.
Disclosure of Invention
The embodiment of the application provides a scanning dot matrix projection system and electronic equipment, even if the deflection angle of a steering mirror is very small, a larger scanning angle can be obtained through diffusion, and therefore the complexity and the power consumption of a driving system of a micro mirror are reduced.
In order to solve the above technical problem, an embodiment of the present application provides the following technical solutions:
the embodiment of the application provides a scanning dot matrix projection system, which comprises a light source, a steering mirror and a diffusion component which can penetrate light rays. The light source can emit collimated light, and resolution reduction caused by overlarge light spots formed on the surface of the measured object finally due to light divergence is avoided. The steering mirror is arranged in the emergent direction of the collimated light, the collimated light which enters the steering mirror can be reflected to obtain reflected light, and the steering mirror is arranged to be capable of deflecting, so that the first emergent angle of the reflected light which is obtained through the steering mirror is changed within a preset angle range. The diffusion subassembly sets up in the exit direction of reflection light, and when reflection light incided diffusion subassembly, the diffusion subassembly can be to this reflection light diffusion for exit angle when reflection light jets out from the diffusion subassembly is greater than the incident angle when reflection light jets into the diffusion subassembly, and the light that jets out from the diffusion subassembly shines on the measured object face, can form the facula for example, in order to realize the scanning to the measured object face. According to the optical imaging principle, the incident angle of the reflected light ray when entering the diffusion component is determined according to the first emergent angle, the first emergent angle changes within the preset angle range along with the deflection of the steering mirror, therefore, even if the steering mirror deflects a small angle, the reflected light ray can be emitted from the diffusion component at a larger second emergent angle due to the diffusion effect of the diffusion component, and the scanning angle is enlarged. Compared with the traditional system, the system has the advantages that the deflection angle of the steering mirror is small when the same scanning angle is achieved, so that the complexity and the power consumption of a driving system of the micro mirror are reduced.
It should be understood that the elements capable of achieving the angle diffusion function can be used as a diffusion component, and the diffusion component can include, for example, an optical film with a sawtooth structure, wherein the refractive index of the optical film with the sawtooth structure is greater than that of air, and light incident at a smaller angle can be converted into light emergent at a larger angle due to refraction, so that the scanning angle can be enlarged. The diffusion component can also comprise an optical film with a grating structure, and the scanning angle can be expanded by utilizing the principle of optical diffraction.
When the diffusion member is an optical film having a saw-tooth structure, the kind of the inclination angle of the saw-tooth inclined plane in the saw-tooth structure may be various. Because the inclination angle of the sawtooth inclined plane in the sawtooth structure can determine the position of the light irradiating on the measured object plane, if the inclination angle of the sawtooth inclined plane in the sawtooth structure is different, the light irradiates on different positions on the measured object plane, multi-point scanning is realized, the requirement on the deflection speed of the deflection mirror is further reduced, and the complexity and the power consumption of a driving system are reduced. In addition, the requirement for the duty cycle of the light source is reduced.
Wherein, the number of the positions irradiated on the measured object surface is the same as the number of the inclination angles.
In some cases, in order to simplify the processing of the saw tooth structure, the saw teeth in the saw tooth structure may be provided with a greater number of saw teeth to achieve a similar function instead of a greater number of large saw teeth. At this time, the sawteeth in the sawtooth structure can be regularly arranged according to a period, and the types of the inclination angles of the sawtooth inclined planes in the sawtooth structure in one period are various. The sawteeth with different inclination angles are repeatedly used, so that the number of the inclination angles of the sawteeth required to be designed is reduced, and the design and processing difficulty is reduced.
In one possible implementation, the widths of the different serrations in the serration structure are the same. Thus, when the incident angle is zero degree, the emergent light energy of different angles is consistent, for example, the energy of different light spots formed on the surface of the measured object is consistent, thereby reducing the requirement on the energy of the light source.
It can be understood that the sawtooth inclined planes in the sawtooth structure can be designed into various arrangements, the inclined directions of the sawtooth inclined planes in the sawtooth structure are the same or different, under the common condition, the inclined directions of the sawtooth inclined planes in the sawtooth structure are different relative to the inclined directions, the optical film has stronger diffusion effect on light rays, the requirement on the deflection angle of the steering mirror is lower, and the performance is more advantageous.
When the inclined directions of the inclined surfaces of the sawteeth in the sawtooth structure are the same, in some cases, a sawtooth structure with an inclination angle increasing or decreasing can be adopted; when the inclined directions of the inclined surfaces of the sawteeth in the sawtooth structure are different, two sawteeth with the same inclination angle can form a mirror symmetry structure; furthermore, the sawteeth with the same inclination angle value and the opposite signs on the inclined plane are spliced together to form an obtuse angle structure, so that the processing difficulty of the position of the sharp angle can be reduced, the saw tooth structure is more wear-resistant in the using process, and the reliability of products is improved.
In order to further increase the scanning speed and further reduce the deflection speed requirement of the turning mirror, the number of points scanned simultaneously can be further increased, so that in the embodiment of the present application, the optical film with the sawtooth structure can comprise multiple layers. Therefore, the reflected light can be refracted for multiple times, the number of positions irradiated on the surface of the measured object is further increased, the number of points scanned simultaneously is increased, the scanning speed is further increased, and the requirement on the deflection speed of the steering mirror is further reduced.
When the optical film with the sawtooth structure comprises a plurality of layers, all the optical films in the plurality of layers of optical films can be orthogonally arranged, all the optical films can be parallelly arranged, and both the optical films can be parallelly arranged and orthogonally arranged. By means of orthogonal placement (namely, the optical films which are stacked orthogonally along the emergent direction of the reflected light exist in the plurality of layers of optical films with the sawtooth structures), the scanning angle can be enlarged in different scanning directions, so that the transverse scanning angle and the longitudinal scanning angle are obviously reduced, and the deflection angle of the steering mirror is reduced in multiple directions.
When a plurality of layers of optical films with sawtooth structures are adopted, the plurality of layers of optical films can be bonded, for example, the optical films can be bonded in an electrostatic adsorption mode, or bonded by a vacuum adsorption method, or bonded by glue. When the optical films are attached, the optical films with the sawtooth structures can be filled with refraction materials, the refraction index of the refraction materials is smaller than that of materials used by the sawtooth structures, and at the moment, the optical films are attached through optical glue. If an air layer is reserved among the plurality of layers of optical films with the sawtooth structures, the optical films can be attached to each other only at the tooth tips of the sawteeth through glue.
When the diffusion component comprises an optical film with a grating structure, the grating structure may be a one-dimensional grating structure or a two-dimensional grating structure, etc. When the grating structure is a one-dimensional grating structure, a plurality of irradiation positions decomposed by the grating structure are arranged in a one-dimensional manner; when the grating structure is a two-dimensional grating structure, a plurality of irradiation positions decomposed by the grating structure are arranged in two dimensions, and the effect is similar to the effect achieved by orthogonally arranging a plurality of layers of optical films with sawtooth structures.
In addition, based on the scanning dot matrix projection system, an embodiment of the present application further provides an electronic device, where the electronic device includes a controller and the scanning dot matrix projection system, the controller is configured to control a light source and a steering mirror in the scanning dot matrix projection system, and the scanning dot matrix projection system is as described in any of the above implementation manners. The scanning dot matrix projection system can enlarge the scanning angle, and the deflection angle of the steering mirror is small when the same scanning angle is reached, so that the complexity and the power consumption of a driving system of the micromirror are reduced, and the scanning angle can be enlarged by electronic equipment comprising the scanning dot matrix projection system, and the complexity and the power consumption of the driving system of the micromirror are reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on the provided drawings without creative efforts.
FIG. 1 is a block diagram of a conventional scanning micro-mirror based dot matrix projection system;
FIG. 2 is a block diagram of a scanning dot matrix projection system according to an embodiment of the present disclosure;
fig. 3 is a structural diagram of a receiving end cooperating with a scanning dot matrix projection system according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram illustrating an embodiment of a scanning dot matrix projection system according to the present disclosure;
FIG. 5 is a diagram illustrating an example of an inclination angle of a sawtooth inclined plane in a sawtooth structure according to an embodiment of the present application;
FIG. 6 is a diagram illustrating an example of a multi-point scan of an optical film with sawtooth structures provided in an embodiment of the present application;
FIG. 7a is a diagram illustrating an exemplary arrangement of saw teeth according to an embodiment of the present application;
FIG. 7b is a schematic diagram of an exemplary arrangement of sawtooth structures provided in an embodiment of the present application;
FIG. 8 is a diagram illustrating an example of a sawtooth structure according to an embodiment of the present disclosure;
FIG. 9 is a diagram illustrating an example of a sawtooth structure according to an embodiment of the present disclosure;
FIG. 10a is a schematic diagram of an exemplary sawtooth arrangement provided in an embodiment of the present application;
FIG. 10b is a schematic diagram of an exemplary sawtooth arrangement provided in an embodiment of the present application;
FIG. 10c is a schematic diagram of an exemplary sawtooth arrangement provided in an embodiment of the present application;
fig. 11 is an exemplary diagram of the emergence of incident light with different angles from sawtooth inclined planes with two inclination angles of 35 ° and 15 ° provided in the embodiment of the present application;
fig. 12 is an exemplary diagram of an arrangement of a plurality of decomposed light spots according to an embodiment of the present application;
FIG. 13 is a diagram illustrating an example of two layers of optical films with sawtooth structures disposed orthogonally to obtain a plurality of light spots;
fig. 14 is a block diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," "fourth," and the like in the description and claims of this application and in the above-described drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
As shown in fig. 1, a light emitted from a light source 101 is incident on a micro-mirror 104, the micro-mirror 104 can reflect the received light to obtain a reflected light, and the reflected light is projected onto a surface 105 to be measured to form a light spot. The light source 101 may be a pulse light source shown as 102, and the micro mirrors 104 may rotate under the driving of the rotation driving system 103, so as to change the emergent light of the reflected light, change the projection position of the light spot on the object to be measured 105, and form a scanning dot matrix shown as 106, thereby implementing the scanning on the object to be measured 105.
However, in order to realize 3D imaging and recognition, a large number of points per second need to be collected, and thus, the requirement for the two-dimensional rotation angle and the two-dimensional rotation speed of the micromirror is extremely high, which greatly increases the complexity and power consumption of the driving system of the micromirror.
Therefore, the embodiment of the application provides a scanning dot matrix projection system, a diffusion component is arranged in the exit direction of the reflected light obtained by the steering mirror, and due to the diffusion effect of the diffusion component, the exit angle when the reflected light exits from the diffusion component is larger than the incident angle when the reflected light enters the diffusion component, so that even if the deflection angle of the steering mirror is small, a larger scanning angle can be obtained by diffusion, and the complexity and the power consumption of a driving system of a micro mirror are reduced.
Next, embodiments of the present application will be described with reference to the drawings.
Referring to fig. 2, fig. 2 shows a block diagram of a scanning dot matrix projection system comprising a light source 201, a turning mirror 202 and a light transmissive diffuser assembly 203.
The light source 201 is used for emitting collimated light, so that the problem that resolution is reduced due to the fact that light spots formed on the surface of a measured object finally are too large due to light divergence is avoided.
The Light source 201 may be one or more lasers or Light Emitting Diodes (LEDs) with a collimated Light path, and the collimated Light path may be a lens, a lens group, or a microlens array; the light source may also be a collimated emitting laser, such as a pulsed laser.
The turning mirror 202 is disposed in the emitting direction of the collimated light, and is used for reflecting the collimated light incident to the turning mirror 202 to obtain a reflected light. The turning mirror 202 is provided to be capable of deflecting so that a first exit angle of the reflected light obtained by the turning mirror 202 varies within a preset angle range.
A plurality of electrodes may be disposed around the turning mirror 202, and deflection of the turning mirror 202 is achieved by applying an operating voltage between the plurality of electrodes. The turning mirror 202 may be deflectable in one direction or multiple directions.
The turning mirror 202 may be a mirror structure processed based on Micro Electro Mechanical System (MEMS) technology.
The diffusing member 203 is disposed in an exit direction of the reflected light, and the diffusing member 203 is configured to diffuse the reflected light passing through the diffusing member 203 such that a second exit angle when the reflected light exits from the diffusing member 203 is larger than an incident angle when the reflected light enters the diffusing member 203. The light emitted from the diffusing member 203 is irradiated on the surface of the object to be measured, for example, a light spot may be formed on the surface of the object to be measured, so that the irradiation position of the light on the surface of the object to be measured may be changed with the deflection of the turning mirror 202, thereby realizing the scanning of the surface of the object to be measured. Due to the diffusing action of the diffusing member, the scanning angle is enlarged. In this way, the desired comparatively large scanning angle can be achieved with a small deflection angle of the steering mirror 202.
The scanning dot matrix projection system provided by the embodiment of the application can be used as a projection end of a 3D sensor and can be applied to terminal equipment, the scanning dot matrix projection system is started under the control of a processor of the terminal equipment, is matched with a receiving end of the 3D sensor to complete three-dimensional scanning imaging of a measured object surface, and then performs feature extraction and judgment on imaging data to realize the functions of 3D imaging and target identification.
Referring to fig. 3, the receiving end may include an optical lens 204 and an image sensor 205. When the light emitted from the diffusion member 203 is incident on the surface of the object to be measured, taking the reflected light from the surface of the object to be measured shown in fig. 3 as an example, the reflected light from the surface of the object to be measured is received by the optical lens 204 and imaged on the image sensor 205. With the deflection of the steering mirror 202, the whole measured object surface can be scanned, so that three-dimensional scanning imaging is completed on the image sensor 205.
The scanning dot matrix projection system that this application embodiment provided includes light source, steering mirror and the diffusion subassembly that light can pass. The steering mirror is arranged in the emergent direction of the collimated light, the collimated light which enters the steering mirror is reflected to obtain reflected light, and the steering mirror is arranged to be capable of deflecting, so that the first emergent angle of the reflected light which is obtained through the steering mirror is changed within a preset angle range. The diffusion subassembly sets up in the outgoing direction of reflection ray, and when reflection ray incidented the diffusion subassembly, the diffusion subassembly can spread this reflection ray for the exit angle when reflection ray jets out from the diffusion subassembly is greater than the incident angle when reflection ray jetted into the diffusion subassembly, and the light that jets out from the diffusion subassembly shines on the measured object face, in order to realize the scanning to the measured object face. According to the optical imaging principle, the incident angle of the reflected light ray when entering the diffusion component is determined according to the first emergent angle, the first emergent angle changes within the preset angle range along with the deflection of the steering mirror, therefore, even if the steering mirror deflects a small angle, the reflected light ray can be emitted from the diffusion component at a larger second emergent angle due to the diffusion effect of the diffusion component, and the scanning angle is enlarged. Compared with the traditional system, the system has the advantages that when the same scanning angle is reached, the deflection angle of the steering mirror is small, so that the complexity and the power consumption of a driving system of the micro mirror are reduced, and the process difficulty is also reduced.
It should be noted that the diffusion member 203 may have various forms, and an element capable of achieving an angle diffusion function may be used as the diffusion member 203. In one possible implementation, the diffuser element 203 may be an optical film with a sawtooth structure. After the reflected light is incident on the inclined surface of the sawtooth structure, light is refracted to change the emergent angle, and because the refractive index of the optical film is greater than that of air, the light incident at a smaller angle can be converted into light emergent at a larger angle, so that the second emergent angle when the reflected light is emitted from the diffusion component 203 is greater than the incident angle when the reflected light is incident on the diffusion component 203.
The optical film can be made of glass or optical plastic as a base material, and the sawtooth structure is processed by etching, nanoimprint, microreplication and other processes. The optical plastic may be, for example, polymethyl methacrylate (PMMA), Polycarbonate (PC), Polyethylene terephthalate (PET), or the like.
It will be appreciated that, in general, a single-point light source can form a single spot on the surface of the object to be measured, so that a single-point scan can be achieved with the spot. In the process of realizing three-dimensional scanning imaging, due to the requirement of image calculation, a large number of points need to be collected every second, which requires the steering mirror 202 to reach a higher deflection speed. In order to further reduce the complexity and power consumption of the driving system of the steering mirror 202 and ensure that the number of collected points can meet the requirement of image calculation, in some cases, the number of light spots formed on the surface of a measured object can be increased, a single-point scanning mode is converted into a multipoint scanning mode, and as the multipoint scanning mode is utilized, the requirement on the deflection speed of the steering mirror 202 can be reduced, and the complexity and power consumption of the driving system can be further reduced.
For this reason, the diffusion member 203 may be further configured to split a spot of the reflected light incident on the diffusion member 203 into a plurality of spots to be projected on the surface of the object. The way of decomposing one light spot into a plurality of light spots by the diffusing member 203 may include various ways, and if the diffusing member 203 is an optical film with a sawtooth structure, since the inclination angle of the sawtooth inclined plane in the sawtooth structure may determine the position of the light spot projected on the surface of the object to be measured, different light spots may be formed on the surface of the object to be measured if the inclination angle of the sawtooth inclined plane in the sawtooth structure is different. Therefore, in some possible implementations, the inclination angles of the inclined planes of the saw teeth in the saw tooth structure may be various, so that one light spot of the reflected light incident on the diffusion component 203 is decomposed into a plurality of light spots to be projected on the surface of the object to be measured.
Referring to fig. 4, on the measured object surface, it can be naturally seen that the measured object surface is divided into a plurality of regions by different projection points (light spots), each region has only one projection point, and each projection point simultaneously completes scanning in each region through deflection of the steering mirror 202, thereby completing scanning of the whole measured object surface. The image sensor 205 may select a minimum array arrangement in which the number of photodetectors (e.g., white dots on the top view of the image sensor in fig. 4) and the number of projection points are the same, and correspond to the scanning area covered by each projection point; or the image sensor 205 may employ a larger number of photodetectors, in which case the number of photodetectors is not strictly limited, and the size of each photodetector may even be smaller than the projection point, as process capability permits.
Wherein the inclination angle includes an angle value and a direction, as shown in fig. 5, the inclination angles of the inclined surface 1 and the inclined surface 2 are 35 °, but the inclination directions of the inclined surface 1 and the inclined surface 2 are opposite, and the inclination directions are opposite. In the present embodiment, it is considered that the inclination angle direction of the inclined surface 2 is "+" and the inclination angle direction of the inclined surface 1 is "-", the inclination angle of the inclined surface 1 is 35 °, and the inclination angle of the inclined surface 2 is-35 °.
It will be appreciated that the number of spots resolved is related to the number of tilt angles. In some cases, the number of spots resolved is the same as the number of tilt angles. Referring to fig. 6, the inclination angles of the sawtooth inclined planes in the sawtooth structure shown in fig. 6 include 4, which are 35 °, 15 °, -35 °, -15 ° from left to right, respectively, and reflected light is incident on different sawtooth inclined planes, respectively, and undergoes a refraction mechanism of light, thereby forming four light spots, one for each sawtooth inclined plane.
The single-point light source input is used for realizing multi-point output, and the single-point scanning mode can be changed into multi-point simultaneous scanning by matching with the array type detector, so that the requirement on the duty ratio of the light source is reduced (the duty ratio of the light source refers to the number of light pulses sent by the light source in unit time), and the requirement on the deflection speed of the steering mirror 202 is also reduced.
In the embodiment of the present application, the sawtooth structure (for example, the shape and layout of the sawtooth) of the optical film may adopt different structural designs according to factors such as application requirements, processing difficulty, or product reliability, as long as a required scanning angle can be achieved. In some cases, the inclined planes of the saw teeth in the saw tooth structure are inclined in the same direction or in different directions, and in general, when the inclined planes of the saw teeth in the saw tooth structure are not inclined in the same direction relative to the inclined planes, the optical film has stronger light diffusion effect, lower deflection angle requirement on the turning mirror 202, and more advantageous performance.
When the inclined directions of the inclined surfaces of the saw teeth in the saw tooth structure are the same, in some cases, a saw tooth structure having an increasing or decreasing inclination angle may be employed, as shown in fig. 7a and 7b, respectively. When the inclined planes of the sawteeth in the sawtooth structure are different in inclination direction, two sawteeth with the same inclination angle can be adopted to form a mirror symmetry structure, for example, as shown in fig. 8; or, as shown in fig. 9, further, the saw teeth with the same inclination angle value and the opposite sign on the inclined plane are spliced together to form an obtuse angle structure, so that the processing difficulty of the sharp angle position can be reduced, the wear resistance is improved in the using process, and the product reliability is improved.
Since the size of the saw teeth may affect the processing difficulty of the saw tooth structure in the optical film, for example, the too large height of the saw teeth may cause the processing difficulty, in order to simplify the processing of the saw tooth structure, the saw teeth in the saw tooth structure may be set to be more saw teeth to replace more large saw teeth to achieve a similar function. In order to reduce the number of the sawtooth inclination angles required to be designed, in the present embodiment, the sawteeth with various inclination angles may be reused, and the arrangement manner of the sawteeth with different inclination angles is not limited in the present embodiment.
In some cases, the saw teeth with different inclination angles may be regularly arranged, for example, the saw teeth in the saw tooth structure are regularly arranged according to a period, and the inclination angles of the inclined surfaces of the saw teeth in the saw tooth structure within one period may be various. The period can be determined according to actual requirements, and in general, the value range of the period can be 40um-200 um.
For example, the inclination angles of the inclined planes of the saw teeth in the sawtooth structure are 4, the saw teeth in each period can form a mirror-symmetrical structure shown in fig. 8, the 4 inclination angles are included in one period, and then the arrangement of the period is repeated, so that the arrangement of the sawtooth structure in the optical film can be seen in fig. 10 a.
For another example, the types of the inclination angles of the inclined surfaces of the saw teeth in the sawtooth structures are 4, the saw teeth in each period can form an obtuse angle structure shown in fig. 9, each period includes 4 inclination angles, and then the arrangement manner of the period is repeated, so that the arrangement manner of the sawtooth structures in the optical film can be shown in fig. 10 b.
For another example, the types of the inclination angles of the inclined planes of the saw teeth in the saw tooth structure are 3, the saw teeth in each period can form a unidirectional inclined structure shown in fig. 7a, the period includes 3 inclination angles, and then the arrangement of the period is repeated, so that the arrangement of the saw tooth structure in the optical film can be seen from fig. 10 c.
It should be noted that, after the sawtooth arrangement mode of the sawtooth structure is determined, the specific inclination angle of the sawtooth inclined plane can be determined according to the required scanning angle. If the FOV of the emergent light when the reflected light is emitted from the diffusing component 203 is required to reach ± 32 ° (scan angle is ± 32 °), and the inclination angle of each sawtooth inclined plane can support the scan of ≧ 8 °, for example, 4 inclination angles are included in one period, and taking the arrangement manner of the sawtooth structure as fig. 10a as an example, the inclination angles adopted in each period can be respectively ± 35 ° and ± 15 °.
The reason why the sawtooth structure satisfies the required scan angle is analyzed below. Fig. 11 shows the emergence of incident light with two sawtooth inclined planes with 35 ° and 15 ° inclination angles along different angles, wherein (a), (b) and (c) in fig. 11 show the emergence of incident light with two sawtooth inclined planes with 35 ° inclination angles along different angles, and (d), (e) and (f) in fig. 11 show the emergence of incident light with two sawtooth inclined planes with 15 ° inclination angles along different angles, and the-35 ° and-15 ° sawteeth are mirror images, and are not described again.
When the incident angle of the reflected light ray incident on the diffusing member 203 is 0 ° (the inclination angles of the inclined surfaces of the saw teeth are 35 ° and 15 °, respectively), as shown in (a), the second exit angle of the reflected light ray exiting from the diffusing member 203 is-24 °, as shown in (d), the second exit angle of the reflected light ray exiting from the diffusing member 203 is-8 °, and the two exit angles are spaced by 16 °. After the incident angle is changed, since the exit angle (second exit angle) and the incident angle satisfy the law of refraction rather than the linear relationship, the second exit angle interval at the time when the inclination angles of the sawtooth inclined planes are 35 ° and 15 °, respectively, is not maintained at 16 °. If the inclination angle of the sawtooth inclined plane is 35 °, see (b) diagram, the second exit angle when the reflected light exits from the diffusing member 203 is-32 ° at the incident angle of-4.28 °, see (c) diagram, the second exit angle when the reflected light exits from the diffusing member 203 is-16 ° at the incident angle of 5.7 °, i.e., when the incident angle is deviated from-4.28 ° to 5.7 °, the second exit angle is deviated from-32 ° to-16 ° (the-32 ° to-16 ° scanning is achieved), i.e., the scanning angle range of ± 8 ° (with respect to the initial angle of-24 °) is achieved; if the inclination angle of the sawtooth inclined surface is 15 °, see (e) diagram, the second exit angle when the reflected light ray exits from the diffusing member 203 is-16 ° when the incident angle is-7.63 °, see (f) diagram, the second exit angle when the reflected light ray exits from the diffusing member 203 is 0 ° when the incident angle is 7.61 °, i.e., when the incident angle is deflected from-7.63 ° to 7.61 °, the second exit angle is deflected from-16 ° to 0 °, and a scanning angle range of ± 8 ° is also achieved (with respect to the initial angle of-8 °). Since the incident angle scan range required for the sawtooth slope with an inclination angle of 15 ° is larger than the sawtooth slope with an inclination angle of 35 ° when the same scan angle is achieved, the outgoing light from the sawtooth slope with an inclination angle of 35 ° is wasted in the range of the extra scan angle (because its outgoing angle exceeds the FOV to ± 32 °).
When the saw-tooth structure shown in fig. 10a is also employed, the deflection angle of the turning mirror 202 can be further reduced by further adjusting the inclination angle of the saw-tooth inclined surface. For example, the refractive index of the material of the optical film with the saw-tooth structure is 1.5, and the set tilt angles are ± 12.6 ° and ± 32.9 °, respectively, and then the scan angle range of ± 32 ° can be realized as long as the deflection angle of the turning mirror 202 satisfies ± 6.37 °.
When the types of the inclination angles of the sawtooth inclined planes in the sawtooth structure are multiple, the second exit angles of the reflected light rays incident on the sawtooth inclined planes with different inclination angles at the same incident angle are different, and the positions of the reflected light rays irradiated on the object surface are different. In order to ensure that the energy of the outgoing light from different angles is the same when the incident angle is zero, for example, the energy of different light spots formed by irradiating on the surface of the object to be measured is the same, thereby reducing the requirement for the energy of the light source, in a possible implementation manner, different sawteeth in the sawtooth structure are set to have the same width.
Under the condition that the types of the inclination angles of the sawtooth inclined planes in the sawtooth structure are various, multipoint scanning can be realized by utilizing the scanning dot matrix projection system provided by the embodiment of the application. In order to further increase the scanning speed and further reduce the requirement on the deflection speed of the steering mirror 202, the number of the light spots obtained by decomposition can be further increased, and the number of points of simultaneous scanning can be increased. The way of increasing the number of spots obtained by the decomposition may include various ways, and one way may be to increase the number of different inclination angles, for example, to increase the number of sawtooth inclination planes including 4 different inclination angles in one cycle to the number of sawtooth inclination planes including 8 different inclination angles in one cycle.
Another way may be to increase the number of layers of the optical film with a sawtooth structure, that is, the diffusion component 203 in the scanning dot matrix projection system provided in this embodiment of the present application is a multilayer optical film with a sawtooth structure, so that the reflected light incident on the diffusion component 203 undergoes multiple refractions, and the scanning angle is further enlarged. At this time, the number of spots obtained by the decomposition is related to the number of tilt angles per layer and the number of layers of the optical film. The multilayer optical film may be optical films with the same sawtooth structure, or optical films with different sawtooth structures.
For example, two optical films having the same saw-tooth structure are used, and the saw-tooth structure arrangement of each optical film is shown in fig. 10a, one optical film having the saw-tooth structure can decompose the light spot incident to the diffusion member 203 into 4 light spots, and when the reflected light passes through the two optical films having the saw-tooth structure, the light spot incident to the diffusion member 203 can be decomposed into 16 light spots.
By increasing the number of layers of the optical film with the sawtooth structure, multipoint simultaneous scanning is further realized, the scanning angle is enlarged, and the requirement on the steering mirror is further reduced.
An air layer may be left between the optical films or the optical films may be filled with a refractive material having a refractive index lower than that of a material used for the saw tooth structure of the optical films, but the refractive material may change the refraction of the interface between the originally designed saw tooth structure and the air layer, so that the inclination angle of each saw tooth inclined plane in the saw tooth structure needs to be redesigned. Of course, in the case of only one layer of the optical film, the sawtooth structures can be filled with a refractive material having a refractive index lower than that of the material used for the sawtooth structures of the optical film.
In the embodiments of the present application, there are various placement manners among the multilayer optical films, and the embodiments of the present application mainly describe the parallel placement and the orthogonal placement among the multilayer optical films. The parallel placement means parallel in the arrangement direction of the sawtooth structures, and the orthogonal placement means orthogonal in the arrangement direction of the sawtooth structures.
When the multilayer optical films are arranged in parallel, if the multilayer optical films are two layers, the arrangement mode of the sawtooth structure in one period adopted by each layer of the optical film is as shown in fig. 8, at the moment, 16 light spots are formed on the surface of a measured object, the arrangement mode of the light spots is as shown in fig. 12, and the 16 light spots are arranged in a one-dimensional mode according to the arrangement mode of the sawtooth structure for generating the light spots. Each spot corresponds to a scanning area, and in this case, one spot corresponds to a transverse scanning angle which is significantly smaller, but the longitudinal scanning angle is still unchanged.
Further, on the basis of realizing multipoint scanning, in order to reduce the deflection angle of the steering mirror 202 in multiple directions, the structure of the diffusion component 203 can be designed, so that one light spot of the reflected light incident on the diffusion component 203 is decomposed into multiple two-dimensionally arranged light spots to be projected on the surface of the measured object.
In one possible implementation, the multilayer optical films may be arranged in an orthogonal manner, that is, there are optical films stacked orthogonally along the exit direction of the reflected light in the multilayer optical film with the sawtooth structure. For example, as shown in the left side of fig. 13, the multilayer optical film has two layers, and the zigzag structure arrangement in one period adopted by each layer of optical film is as shown in fig. 8, where 16 light spots are formed on the surface of the object to be measured. Since the two optical films are orthogonally arranged, when the reflected light sequentially passes through the two optical films to be emitted, light spots formed on the surface of the measured object are as shown on the right side of fig. 13, and 16 light spots are two-dimensionally arranged by 4 × 4.
It can be seen that in this case one spot corresponds to a significantly smaller transverse scan angle and longitudinal scan angle, thereby reducing the deflection angle of the steering mirror 202 in multiple directions.
When the optical film with the sawtooth structure includes a plurality of layers, all of the optical films in the plurality of layers may be orthogonally disposed, all of the optical films may be parallel disposed, and both of the optical films may be orthogonally disposed.
When a plurality of optical films with a sawtooth structure are adopted, the interval between the two optical films is as small as the process allows, and the interval is as small as possible. Each optical film may be fixed separately as a discrete component, or multiple optical films may be bonded together, for example, by electrostatic adsorption, vacuum adsorption, or glue. In general, the positions of the sawtooth tips of the previous optical film and the bottom surface of the next optical film (the bottom surface is the side of the optical film without the sawtooth structures) in the optical path direction may be adhered together by glue.
When the adhesive is used for bonding, the bonding mode is different according to the difference of reserving an air layer or filling a refraction material between the multilayer optical films. If the refractive material is filled between the optical films with the sawtooth structures, the optical films can be bonded through the optical glue. The refractive index of the optical cement can be smaller than, equal to or larger than the refractive index of the material used for the sawtooth structure and the refractive index of the refractive material. If an air layer is reserved between the optical films with the sawtooth structures, the optical films can be attached to each other only at the tooth tips of the sawteeth through glue.
While the foregoing describes the use of an optical film having a sawtooth structure including a plurality of tilt angles to split a spot of reflected light incident on the diffuser element 203 into a plurality of spots, in some cases, an optical film having a grating structure may be used to split a spot of reflected light incident on the diffuser element 203 into a plurality of spots, that is, the diffuser element is an optical film having a grating structure.
The grating structure is composed of a large number of parallel slits with equal width and equal spacing, one light spot incident to the diffusion component 203 can be decomposed into a plurality of light spots to be projected onto a measured object surface by utilizing the principle of optical diffraction, and meanwhile, the scanning angle can be expanded. Wherein the number of the light spots obtained by the decomposition is related to the number of the slits.
The grating structure can be a one-dimensional grating structure or a two-dimensional grating structure. When the grating structure is a one-dimensional grating structure, a plurality of light spots decomposed by the grating structure are arranged in a one-dimensional manner; when the grating structure is a two-dimensional grating structure, a plurality of light spots decomposed by the grating structure are arranged in two dimensions, and the effect is similar to the effect realized by orthogonally arranging a plurality of layers of optical films with sawtooth structures.
Based on the scanning dot matrix projection system provided in the foregoing embodiments, the present application further provides an electronic device, and referring to fig. 14, the electronic device 1400 includes a controller 1401 and a scanning dot matrix projection system 1402. The structure and the functions of each structure of the scanning dot matrix projection system 1402 are described in the foregoing embodiments, and are not further described herein. The controller 1401 may control the light sources and the steering mirrors in the scanning dot matrix projection system 1402. The controller 1401, when controlling the light source, may control, for example, the intensity of the light source, a pulse signal of the light source, the luminance of the light source, and the like; the controller may control, for example, the deflection of the steering mirror when controlling the steering mirror.
In the several embodiments provided in the present application, it should be understood that the disclosed system may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
It should be understood that in the present application, "at least one" means one or more, "a plurality" means two or more. "and/or" for describing an association relationship of associated objects, indicating that there may be three relationships, e.g., "a and/or B" may indicate: only A, only B and both A and B are present, wherein A and B may be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one of the following" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items. For example, at least one (one) of a, b, or c, may represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims (11)

1. A scanning dot matrix projection system comprising a light source, a turning mirror and a light transmissive diffusing component:
the light source is used for emitting collimated light;
the steering mirror is arranged in the emergent direction of the collimated light and is used for reflecting the collimated light incident to the steering mirror to obtain reflected light; the steering mirror is arranged to be capable of deflecting so that a first exit angle of the reflected light obtained by the steering mirror changes within a preset angle range;
the diffusion component is arranged in the emergent direction of the reflected light, and is used for diffusing the reflected light passing through the diffusion component, so that a second emergent angle when the reflected light is emitted from the diffusion component is larger than an incident angle when the reflected light is emitted into the diffusion component; the light emitted from the diffusion component irradiates on the surface of the object to be measured so as to realize the scanning of the surface of the object to be measured;
the diffusion component comprises a plurality of layers of optical films with sawtooth structures.
2. The system of claim 1, wherein the inclination angles of the sawtooth inclined planes in the sawtooth structure are various.
3. The system of claim 1 or 2, wherein the saw teeth in the saw tooth structure are arranged according to a periodic rule, and the inclination angles of the inclined surfaces of the saw teeth in the saw tooth structure in one period are various.
4. The system of claim 1 or 2, wherein the widths of different teeth in the tooth structure are the same.
5. The system of claim 1 or 2, wherein the inclined planes of the saw teeth in the saw tooth structure are inclined in different directions.
6. The system of claim 1, wherein the plurality of layers of optical films with a sawtooth structure comprise optical films stacked orthogonally to the direction of the reflected light beam, which is perpendicular to the direction of the collimated light beam.
7. The system according to claim 1 or 6, wherein a refractive material is filled between the plurality of layers of optical films with the sawtooth structures, the refractive index of the refractive material is smaller than that of the material used for the sawtooth structures, and the optical films are attached by optical glue.
8. The system according to claim 1 or 6, wherein an air layer is reserved between the plurality of layers of optical films with the sawtooth structures, and the optical films are attached to each other at the positions of the sawtooth tips by glue.
9. The system of claim 1, wherein the diffuser assembly comprises an optical film with a grating structure.
10. The system of claim 9, wherein the grating structure is a two-dimensional grating structure.
11. An electronic device, characterized in that the electronic device comprises a controller and a scanning dot matrix projection system according to any of the claims 1-10, the controller being adapted to control a light source and a steering mirror in the scanning dot matrix projection system.
CN201911019606.9A 2019-10-24 2019-10-24 Scanning dot matrix projection system and electronic equipment Active CN112711135B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911019606.9A CN112711135B (en) 2019-10-24 2019-10-24 Scanning dot matrix projection system and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911019606.9A CN112711135B (en) 2019-10-24 2019-10-24 Scanning dot matrix projection system and electronic equipment

Publications (2)

Publication Number Publication Date
CN112711135A CN112711135A (en) 2021-04-27
CN112711135B true CN112711135B (en) 2022-09-02

Family

ID=75540454

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911019606.9A Active CN112711135B (en) 2019-10-24 2019-10-24 Scanning dot matrix projection system and electronic equipment

Country Status (1)

Country Link
CN (1) CN112711135B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101203802A (en) * 2005-06-20 2008-06-18 松下电器产业株式会社 2-dimensional image display device, illumination light source, and exposure illumination device
CN105807421A (en) * 2016-05-10 2016-07-27 天津大学 Parallel-light-emerging angle magnification MEMS scanning method and optical system
CN107209265A (en) * 2015-03-20 2017-09-26 摸索科技有限公司 light detection and ranging device
CN109814253A (en) * 2019-02-21 2019-05-28 浙江水晶光电科技股份有限公司 Structure optical mode group and three-dimensional sensing device

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TW200801688A (en) * 2006-06-21 2008-01-01 T & Amp T Ind Corp An optical film with both light-diffusion and brightness-enhancement effects
JP2009162825A (en) * 2007-12-28 2009-07-23 Seiko Epson Corp Image display apparatus
KR101548445B1 (en) * 2011-06-22 2015-08-28 히다치 막셀 가부시키가이샤 Image display device
JP5780129B2 (en) * 2011-11-22 2015-09-16 ソニー株式会社 Light beam expansion device, image display device, and optical device
US10305247B2 (en) * 2016-08-30 2019-05-28 Apple Inc. Radiation source with a small-angle scanning array
CN108828786A (en) * 2018-06-21 2018-11-16 深圳市光鉴科技有限公司 A kind of 3D camera
CN208999565U (en) * 2018-10-12 2019-06-18 北醒(北京)光子科技有限公司 A kind of scanning means and laser radar
CN109164464A (en) * 2018-10-12 2019-01-08 北醒(北京)光子科技有限公司 A kind of scanning means and laser radar

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101203802A (en) * 2005-06-20 2008-06-18 松下电器产业株式会社 2-dimensional image display device, illumination light source, and exposure illumination device
CN107209265A (en) * 2015-03-20 2017-09-26 摸索科技有限公司 light detection and ranging device
CN105807421A (en) * 2016-05-10 2016-07-27 天津大学 Parallel-light-emerging angle magnification MEMS scanning method and optical system
CN109814253A (en) * 2019-02-21 2019-05-28 浙江水晶光电科技股份有限公司 Structure optical mode group and three-dimensional sensing device

Also Published As

Publication number Publication date
CN112711135A (en) 2021-04-27

Similar Documents

Publication Publication Date Title
EP1078293B1 (en) Illumination system and method for spatial modulators
KR101827180B1 (en) Integrated structured-light projector
CN111722241B (en) Multi-line scanning distance measuring system, method and electronic equipment
KR102329110B1 (en) Vehicle monitoring system
CN109557550B (en) Three-dimensional solid-state laser radar device and system
KR20040010549A (en) Light beam display with interlaced light beam scanning
CN107533237B (en) Optical device
KR20190018351A (en) nanostructured optical element comprising, depth sensor and electronic device
US10151629B2 (en) Spectral imaging sensors and methods with time of flight sensing
AU2003286237B2 (en) Structured light projector
CN108107663A (en) Laser emitter, optoelectronic device, depth camera and electronic device
KR20150090680A (en) Camera apparatus
CN112711135B (en) Scanning dot matrix projection system and electronic equipment
US11019328B2 (en) Nanostructured optical element, depth sensor, and electronic device
US20220373656A1 (en) Increasing VCSEL projector spatial resolution
EP4280600A1 (en) Camera module
CN115867824A (en) Laser radar system with rough angle control
US5168167A (en) Optical scanner having controllable light sources
CN114616484A (en) Light source, sensor and method of illuminating a scene
KR102103919B1 (en) Multi-image projector and electronic device having multi-image projector
EP4204759B1 (en) Structured and diffuse light generation
US20230168380A1 (en) Method and device for acquiring image data
US11815677B1 (en) Display using scanning-based sequential pupil expansion
CN112567286A (en) Scanning display with increased uniformity
EP4283330A1 (en) Lidar device with spatial light modulators

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant