CN111580372A - Synthetic CGH (Carrier grade height) -based speckle noise suppressed large visual area calculation holographic display method - Google Patents

Synthetic CGH (Carrier grade height) -based speckle noise suppressed large visual area calculation holographic display method Download PDF

Info

Publication number
CN111580372A
CN111580372A CN202010423114.2A CN202010423114A CN111580372A CN 111580372 A CN111580372 A CN 111580372A CN 202010423114 A CN202010423114 A CN 202010423114A CN 111580372 A CN111580372 A CN 111580372A
Authority
CN
China
Prior art keywords
cgh
size
object point
slm
speckle noise
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010423114.2A
Other languages
Chinese (zh)
Other versions
CN111580372B (en
Inventor
刘素娟
李萍萍
翟凤潇
刘楠楠
郝蕴琦
张志峰
杨坤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhengzhou University of Light Industry
Original Assignee
Zhengzhou University of Light Industry
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhengzhou University of Light Industry filed Critical Zhengzhou University of Light Industry
Priority to CN202010423114.2A priority Critical patent/CN111580372B/en
Publication of CN111580372A publication Critical patent/CN111580372A/en
Application granted granted Critical
Publication of CN111580372B publication Critical patent/CN111580372B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03HHOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/22Processes or apparatus for obtaining an optical image from holograms
    • G03H1/2202Reconstruction geometries or arrangements
    • G03H1/2205Reconstruction geometries or arrangements using downstream optical component
    • GPHYSICS
    • G03PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03HHOLOGRAPHIC PROCESSES OR APPARATUS
    • G03H1/00Holographic processes or apparatus using light, infrared or ultraviolet waves for obtaining holograms or for obtaining an image from them; Details peculiar thereto
    • G03H1/0005Adaptation of holography to specific applications
    • G03H2001/0088Adaptation of holography to specific applications for video-holography, i.e. integrating hologram acquisition, transmission and display

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Holo Graphy (AREA)

Abstract

The invention provides a synthetic CGH-based large visual area computer-generated holographic display method with suppressed speckle noise. In the method, a recorded object generates a plurality of object point groups through pixel separation, and information of each object point group is recorded into a large-size CGH. In addition, Diffractive Optical Elements (DOEs) with random phase distribution are fabricated. And respectively splicing different DOEs and a large-size CGH in the horizontal direction, so that one object point group correspondingly generates a plurality of synthesized CGHs. Finally, the synthesized CGH is loaded on three SLMs of the plane arrangement structure for reproduction by utilizing a time multiplexing technology, and the diffraction boundary angle of each interference pattern is equal to the maximum diffraction angle of the SLM, so that the expansion of a reproduced image visual area is realized; while speckle noise on the reproduced image is significantly suppressed by the averaging effect and the separation between adjacent pixels.

Description

Synthetic CGH (Carrier grade height) -based speckle noise suppressed large visual area calculation holographic display method
One, the technical field
The invention relates to the field of holographic display, in particular to a large visual area calculation holographic display method with speckle noise suppressed.
Second, background Art
A Spatial Light Modulator (SLM) -based computational holographic display technology can provide a 3D reconstructed image satisfying all visual perception functions for humans, and is considered as one of the most ideal 3D display modes. There are still some challenges that prevent further development of this technique, such as speckle noise in the reconstructed image, which severely degrades the quality of the reconstructed image; in addition, in the conventional method for generating the Computer Generated Hologram (CGH), the diffraction boundary angle of the interferogram corresponding to the object point cannot reach the maximum diffraction angle of the SLM at the same time, and the maximum diffraction angle of the SLM is limited by the pixel interval of the SLM, so that the viewing visual area of the holographically reproduced image is small, the requirement of people on binocular viewing cannot be met, and the viewing experience is influenced.
In order to suppress speckle noise in a reproduced image, researchers have proposed various methods. The iterative algorithm achieves speckle noise suppression by increasing the number of iterations of CGH, but when the number of iterations reaches a certain value, the convergence of the algorithm becomes poor, and the noise suppression effect is reduced. The weak coherent light method employs a light source (e.g., LED) with poor coherence as reproduction light, thereby reducing speckle noise caused by its random coherent addition, but at the same time, causing blurring of a reproduction image. The time averaging method is used for reproducing a plurality of CGH, the reproduction image quality is improved by averaging speckle noise, but the calculation amount of the CGH is greatly increased. The pixel separation method suppresses speckle noise by reducing the overlapping area between adjacent pixels, but it is difficult to realize dynamic display.
In order to enlarge the viewing zone of the reproduced image, an advanced diffraction light method, an SLM resolution redistribution method, a multi-SLM curved surface array method, and the like are proposed. The higher order diffraction method visually combines different orders of reproduced light by time multiplexing to enlarge the viewing zone, but causes unevenness in the light intensity distribution of the reproduced image. The SLM resolution redistribution method increases the horizontal resolution of the SLM using a 4f imaging system, and loses the vertical viewing area while achieving the enlargement of the viewing area in the horizontal direction. The multi-SLM curved surface array method can significantly expand the viewing area of the reproduced image, but the requirement for seamless splicing technology between SLMs is high, and the complexity and cost of the reproduction system are also greatly increased, which is not favorable for marketization.
The above results improve the viewing quality of the reproduced image by enlarging the viewing zone of the reproduced image and suppressing speckle noise, respectively. However, there are few methods that can simultaneously achieve suppression of speckle noise of a reproduced image and enlargement of a viewing zone by a simple operation. Therefore, the large visual area calculation holographic display method with the speckle noise suppressed is provided, and has important significance for improving the quality of the holographic reproduction image.
Third, the invention
The invention provides a synthetic CGH-based large visual area computer-generated holographic display method with suppressed speckle noise. In the method, a recorded object generates a plurality of object point groups through pixel separation, and information of each object point group is recorded into a large-size CGH. In addition, Diffractive Optical Elements (DOEs) with random phase distribution are fabricated. And respectively splicing different DOEs and a large-size CGH in the horizontal direction, so that one object point group correspondingly generates a plurality of synthesized CGHs. Finally, the synthesized CGH is loaded on three SLMs of the plane arrangement structure for reproduction by utilizing a time multiplexing technology, and the diffraction boundary angle of each interference pattern is equal to the maximum diffraction angle of the SLM, so that the expansion of a reproduced image visual area is realized; while speckle noise on the reproduced image is significantly suppressed by the averaging effect and the separation between adjacent pixels.
As shown in fig. 1, the large visual area computer generated hologram display method with speckle noise suppressed based on the synthesized CGH of the present invention mainly includes the following steps and the study of the visual area is performed only in the horizontal direction.
The method comprises the following steps: the recorded object is separated into N by a pixel separation interval N (N is 2, 3, 4 … …) using a pixel separation method2And the object point groups ensure that any object point on the recorded object and the object points which are separated from each other by N pixels are separated into the same object point group in the vertical and horizontal directions.
Step two: and recording the information of each object point group to generate a large-size CGH. In the horizontal direction, the size of the object group is determined to be D, the size of the working area of the SLM is determined to be H, H is more than D, and the SLM is arranged inThe point is the origin of coordinates and the distance between the set of object points and the SLM is L. For any object point I (x) on the object point group0L), the information of which is recorded as an interferogram according to the principle of scalar diffraction. The size of the interference pattern is increased by designating the abscissa of the rightmost end point and the leftmost end point of the interference pattern as x1And x2They satisfy formula (1) and formula (2):
x1=x0+H/2+D/2 (1)
x2=x0-(H/2+D/2) (2)
the size H of the interferogram1Satisfies formula (3):
H1=x1-x2=H+D (3)
and superposing interference patterns of all object points in the object point group to obtain the large-size CGH. Its dimension HCGHSatisfies formula (4):
HCGH=H+2D (4)
this step generates a CGH that is larger than the size of a single SLM but smaller than the size of three SLMs.
Step three: manufacture of VN2(V ═ 1, 2, 3, … …) webs of DOEs with random phase distribution, and the horizontal dimension of the DOEs was 2(H-D), the vertical dimension was the same as that of the large size CGH.
Step four: and (3) dividing one DOE into two parts in the horizontal direction, and splicing the two parts with the leftmost end and the rightmost end of the large-size CGH to generate the synthetic CGH with the size of 3H. And for each object point group, splicing the DOEs with different V amplitudes with the large-size CGH respectively. Therefore, the present invention co-generates VN2And (5) amplitude synthesizing the CGH.
Step five: to VN2The synthesized CGH is loaded to the SLM for reproduction in time sequence, and the reproduction light path is shown in figure 2. The laser is a reproduction light source, the filter and the lens expand and collimate the laser, the beam splitter divides a reproduction light beam into two beams, the plane mirror changes the propagation direction of the reproduction light, the three reflective phase-type SLMs are arranged according to a plane structure, the beam splitter behind the three reflective phase-type SLMs realizes seamless splicing among the SLMs, and the receiving plate receives the reproduction image. A composite CGH is divided into three parts and loaded on corresponding SLM for carrying outThe diffraction boundary angle of each interferogram on the composite CGH is equal to the maximum diffraction angle of the SLM, so that the viewing area of the resulting sub-image is enlarged. At a viewing distance R (R > L), the viewing zone size satisfies formula (5):
V=[R(D+H)-L(H+2D)]/L(5)
meanwhile, the reproduced light is modulated by DOE on the synthesized CGH and appears on the sub-image in the form of speckle noise, V synthesized CGH corresponding to one object point group generates V sub-images with different speckle noises, the V sub-images form a final sub-image through time multiplexing, and the speckle noise on the final sub-image is inhibited through an averaging effect according to a statistical law of the speckle noise. N generated by different object point groups2And the final sub-images are subjected to time multiplexing to form a reproduced image containing complete information, and adjacent image points are separated in space, so that speckle noise caused by superposition of the adjacent image points is further suppressed, and finally the large-visual-area holographic reproduced image with the suppressed speckle noise is obtained.
Description of the drawings
The foregoing and additional aspects and advantages of the present invention will be further apparent and readily appreciated from the following detailed description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flow chart of a large visual area computer generated hologram display method with speckle noise suppressed based on a synthesized CGH according to the present invention.
FIG. 2 is a schematic diagram of the structure of an optical reproducing system according to the method of the present invention.
The reference numbers in the figures are:
1 laser, 2 filters, 3 lenses, 4 beam splitters, 5 plane mirrors, 6SLM and 7 receiving plates.
It should be understood that the above-described figures are merely schematic and are not drawn to scale.
Fifth, detailed description of the invention
The present invention will be described in further detail below with reference to an exemplary embodiment of a large-view computed holographic display method with speckle noise suppression based on a synthesized CGH according to the present invention.
The invention relates to a large visual area computer-generated holographic display method based on synthesized CGH (CGH) and with suppressed speckle noise, which mainly comprises the following steps, and the study of a visual area is only carried out in the horizontal direction.
The method comprises the following steps: by using a pixel separation method, the recorded object is separated into 4 object point groups by the pixel separation interval N-2, and any one object point on the recorded object and the object point which is 2 pixels away from the object point are separated into the same object point group in the vertical and horizontal directions.
Step two: and recording the information of each object point group to generate a large-size CGH. In the horizontal direction, the size of the object group is determined to be 2.88mm, the size of the working area of the SLM is determined to be 8.64mm, H is guaranteed to be larger than D, the middle point of the SLM is the coordinate origin, and the distance between the object point group and the SLM is 300 mm. For any object point I (x) on the object point group0300mm) whose information is recorded as an interferogram according to the principle of scalar diffraction. The size of the interference pattern is increased by designating the abscissa of the rightmost end point and the leftmost end point of the interference pattern as x1And x2They satisfy the formula x1=x0+H/2+D/2=x0+8.64mm/2+2.88mm/2=x0+5.76mm and formula x2=x0-(H/2+D/2)=x0-(8.64mm/2+2.88mm/2)=x05.76mm, the size H of the interferogram1Satisfies the formula H1=x1-x2H + D8.64 mm +2.88mm 11.52 mm. Superposing the interference patterns of all object points in the object point group to obtain a large-size CGH, the size H of which isCGHSatisfies the formula HCGHH +2D 8.64mm +2 x 2.88mm 14.40 mm. This step generates a CGH that is larger than the size of a single SLM but smaller than the size of three SLMs.
Step three: manufacture of VN216 (V-4) webs of DOEs with random phase distribution, and the horizontal dimension of the DOEs is 2(H-D) × (8.64mm-2.88mm) × 11.52mm, the vertical dimension is the same as that of the large size CGH.
Step four: and (3) dividing one DOE into two parts in the horizontal direction, and splicing the two parts with the leftmost end and the rightmost end of the large-size CGH to generate the synthetic CGH with the size of 3H-3 x 8.64 mm-25.92 mm. For each object point group, 4 different DOEs are respectively spliced with the large-size CGH. Therefore, the present invention generates 16 synthetic CGHs in total.
Step five: the 16-amplitude synthesized CGH time sequence is loaded to the SLM for reproduction, green laser is a reproduction light source, a filter and a lens expand and collimate the laser, a beam splitter divides a reproduction light beam into two beams, a plane mirror changes the propagation direction of the reproduction light, three reflection type phase-type SLMs are arranged according to a plane structure, the beam splitters behind the three reflection type phase-type SLMs realize seamless splicing among the SLMs, and a receiving plate receives the reproduction image. A composite CGH is divided into three parts and loaded on a corresponding SLM for reproduction, the diffraction boundary angle of each interference pattern on the composite CGH is equal to the maximum diffraction angle of the SLM, and therefore the visual area of the generated sub-image is enlarged. At a viewing distance R of 800mm, the viewing zone size satisfies the formula V ═ R (D + H) -L (H +2D) ]/L ═ 800(2.88+8.64) -300(8.64+2 × 2.88) ]/300mm ═ 16.32 mm. Meanwhile, the reproduced light is modulated by DOE on the synthesized CGH and appears on the sub-image in the form of speckle noise, 4 synthesized CGHs corresponding to one object point group are combined into 4 sub-images with different speckle noises, the 4 sub-images form a final sub-image through time multiplexing, and the speckle noise on the final sub-image is inhibited through an averaging effect according to a statistical law of the speckle noise. 4 final sub-images generated by different object point groups are subjected to time multiplexing to form a reproduced image containing complete information, and adjacent image points are separated in space, so that speckle noise caused by superposition of the adjacent image points is further suppressed, and finally the large-visual-area holographic reproduced image with suppressed speckle noise is obtained.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.

Claims (2)

1. The invention relates to a large visual area computer-generated holographic display method based on synthesized CGH with speckle noise suppressed, which is characterized by mainly comprising the following steps of researching a visual area only in the horizontal direction,
the method comprises the following steps: the recorded object is separated into N by a pixel separation interval N (N is 2, 3, 4 … …) using a pixel separation method2The object point groups ensure that any object point on the recorded object and object points which are separated from each other by N pixels are separated into the same object point group in the vertical and horizontal directions;
step two: recording the information of each object point group, generating a large-size CGH, determining the size of the object group as D and the size of the working area of the SLM as H in the horizontal direction, ensuring that H is more than D, taking the middle point of the SLM as a coordinate origin, and the distance between the object point group and the SLM as L, and aiming at any object point I (x) on the object point group0L), according to the principle of scalar diffraction, the information is recorded as an interferogram, the size of which is increased if the abscissa of the rightmost and leftmost end of the interferogram is denoted x1And x2They satisfy the formula x1=x0+ H/2+ D/2 and x2=x0- (H/2+ D/2), the size H of the interferogram1Satisfies the formula H1=x1-x2Superposing interference patterns of all object points in the object point group to obtain a large-size CGH with the size HCGHSatisfies the formula HCGHH +2D, this step generates a CGH size larger than the size of a single SLM and smaller than the size of three SLMs;
step three: manufacture of VN2(V ═ 1, 2, 3, … …) webs of DOEs with random phase distribution, and the horizontal dimension of the DOEs is 2(H-D), the vertical dimension is the same as that of the large size CGH;
step four: equally dividing one DOE into two parts in the horizontal direction, splicing the two parts with the leftmost end and the rightmost end of the large-size CGH to generate a synthetic CGH with the size of 3H, and for each object point group, respectively splicing DOEs with different V amplitudes with the large-size CGH, so that VN is generated in the invention2Amplitude synthesizing CGH;
step five: to VN2The amplitude synthesis CGH is loaded to the SLM for reproduction in time sequence, and the laser is used for reproductionThe laser beam expanding and collimating device comprises a light source, a filter and a lens, wherein a beam splitter divides a beam into two beams, a plane mirror changes the propagation direction of the beam, three reflective phase-type SLMs are arranged according to a plane structure, the beam splitters behind the three phase-type SLMs realize seamless splicing among the SLMs, a receiving plate supports a reproduced image, and finally a large visual area holographic reproduced image with suppressed spot noise is obtained.
2. The synthetic CGH-based, speckle-noise-suppressed, large-view-area computed holographic display method of claim 1, in which step five is VN2Loading the synthesized CGH to the SLM for reproduction in time sequence, dividing one synthesized CGH into three parts, loading the three parts to the corresponding SLM for reproduction, wherein the diffraction boundary angle of each interference pattern on the synthesized CGH is equal to the maximum diffraction angle of the SLM, so that the visual area of the generated sub-image is enlarged, and at the viewing distance R (R is more than L), the size of the visual area satisfies the formula V ═ R (D + H) -L (H +2D)]/L
Meanwhile, the reproduced light is modulated by DOE on the synthesized CGH and appears on the sub-image in the form of speckle noise, V synthesized CGH corresponding to one object point group generates V sub-images with different speckle noises, the V sub-images form a final sub-image through time multiplexing, the speckle noise on the final sub-image is inhibited through an averaging effect according to a statistical law of the speckle noise, and N generated by different object point groups2And the final sub-images are subjected to time multiplexing to form a reproduced image containing complete information, and adjacent image points are separated in space, so that speckle noise caused by superposition of the adjacent image points is further suppressed, and finally the large-visual-area holographic reproduced image with the suppressed speckle noise is obtained.
CN202010423114.2A 2020-05-19 2020-05-19 Large visual area calculation holographic display method with speckle noise suppressed Active CN111580372B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010423114.2A CN111580372B (en) 2020-05-19 2020-05-19 Large visual area calculation holographic display method with speckle noise suppressed

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010423114.2A CN111580372B (en) 2020-05-19 2020-05-19 Large visual area calculation holographic display method with speckle noise suppressed

Publications (2)

Publication Number Publication Date
CN111580372A true CN111580372A (en) 2020-08-25
CN111580372B CN111580372B (en) 2021-07-30

Family

ID=72125160

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010423114.2A Active CN111580372B (en) 2020-05-19 2020-05-19 Large visual area calculation holographic display method with speckle noise suppressed

Country Status (1)

Country Link
CN (1) CN111580372B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023000394A1 (en) * 2021-07-19 2023-01-26 北京航空航天大学 Holographic 3d display system based on spatial light modulator virtual array splicing

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107121917A (en) * 2017-05-09 2017-09-01 四川大学 A kind of method for suppressing to calculate holographic speckle noise
CN107247399A (en) * 2017-05-09 2017-10-13 四川大学 A kind of space-time multiplexing method for suppressing to calculate holographic display dot noise
CN107526279A (en) * 2017-08-16 2017-12-29 四川大学 A kind of method for expanding holographic reconstructed image viewing vision area
JP2019066885A (en) * 2016-11-17 2019-04-25 大日本印刷株式会社 Illumination device and manufacturing method of the same
CN110095969A (en) * 2018-10-23 2019-08-06 郑州轻工业学院 A method of inhibit to calculate holographic reconstructed image speckle noise based on single SLM
US20190294103A1 (en) * 2018-03-21 2019-09-26 Carl Zeiss Meditec Ag Visualization system and method for generating holographic presentations from optical signals

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2019066885A (en) * 2016-11-17 2019-04-25 大日本印刷株式会社 Illumination device and manufacturing method of the same
CN107121917A (en) * 2017-05-09 2017-09-01 四川大学 A kind of method for suppressing to calculate holographic speckle noise
CN107247399A (en) * 2017-05-09 2017-10-13 四川大学 A kind of space-time multiplexing method for suppressing to calculate holographic display dot noise
CN107526279A (en) * 2017-08-16 2017-12-29 四川大学 A kind of method for expanding holographic reconstructed image viewing vision area
US20190294103A1 (en) * 2018-03-21 2019-09-26 Carl Zeiss Meditec Ag Visualization system and method for generating holographic presentations from optical signals
CN110095969A (en) * 2018-10-23 2019-08-06 郑州轻工业学院 A method of inhibit to calculate holographic reconstructed image speckle noise based on single SLM

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023000394A1 (en) * 2021-07-19 2023-01-26 北京航空航天大学 Holographic 3d display system based on spatial light modulator virtual array splicing

Also Published As

Publication number Publication date
CN111580372B (en) 2021-07-30

Similar Documents

Publication Publication Date Title
JP6797746B2 (en) How to calculate a hologram
JP5468537B2 (en) Method for generating video holograms in real time to improve 3D rendering graphics pipeline
JP5266223B2 (en) Method for generating computer video holograms in real time using propagation
EP2663901A1 (en) An apparatus and methods for holographic display
JP2010501903A (en) Method for generating video holograms in real time using sub-holograms
JP5541924B2 (en) Method and apparatus for reducing speckle
CN110032053B (en) Holographic speckle noise suppression method based on effective utilization of spatial light modulator
CN111538223B (en) Holographic projection method based on light beam deflection
EP3770692B1 (en) Holographic display apparatus and method for providing expanded viewing window
CN111580372B (en) Large visual area calculation holographic display method with speckle noise suppressed
Yang et al. Computer generated full-parallax synthetic hologram based on frequency mosaic
KR20090011973A (en) Method for reproducing hologram 3d image by integral imaging scheme and apparatus thereof
CN109946943B (en) Holographic display system for inhibiting speckle noise based on beam shaping
CN111240176B (en) Large visual area calculation holographic display method with speckle noise suppressed
Li et al. Large-size holographic display method based on effective utilization of two spatial light modulators
KR20240055642A (en) Hologram calculation for compact head-up display
Kang et al. Color holographic wave-front printing technique
CN117850186A (en) Fresnel holographic reconstruction image controllable amplifying reconstruction system and method
Abookasis et al. Computer-generated three types of holograms of three-dimensional objects synthesized by multiple perspectives

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant