CN107330933B - Arbitrary focal surface shooting method based on camera array - Google Patents

Arbitrary focal surface shooting method based on camera array Download PDF

Info

Publication number
CN107330933B
CN107330933B CN201710578545.4A CN201710578545A CN107330933B CN 107330933 B CN107330933 B CN 107330933B CN 201710578545 A CN201710578545 A CN 201710578545A CN 107330933 B CN107330933 B CN 107330933B
Authority
CN
China
Prior art keywords
camera array
camera
parallax image
plane
focal surface
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710578545.4A
Other languages
Chinese (zh)
Other versions
CN107330933A (en
Inventor
王琼华
熊召龙
赵敏
李大海
罗令
任慧
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan University
Original Assignee
Sichuan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan University filed Critical Sichuan University
Priority to CN201710578545.4A priority Critical patent/CN107330933B/en
Publication of CN107330933A publication Critical patent/CN107330933A/en
Application granted granted Critical
Publication of CN107330933B publication Critical patent/CN107330933B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G06T7/85—Stereo camera calibration

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention provides a camera array-based arbitrary focal surface shooting method, which utilizes a calibrated camera array to obtain a parallax image sequence of a three-dimensional scene, and utilizes the redundancy of three-dimensional information data obtained by the camera array to realize arbitrary surface focusing in the three-dimensional scene. The method comprises two processes of calculating the pixel offset of the parallax image and synthesizing the shot image of any focal surface, has the advantages of high speed, adjustability and simplicity in calibration, and breaks through the limitation that the traditional camera can only focus on a plane.

Description

Arbitrary focal surface shooting method based on camera array
Technical Field
The invention relates to a technology for shooting a curved surface of a focus in a three-dimensional scene, in particular to a method for shooting any curved surface of the focus based on a camera array.
Background
The camera shooting technology is a technology in which a three-dimensional scene is projected onto a two-dimensional sensor through an optical imaging system, and optical information associated with the three-dimensional scene is obtained. The traditional camera has a specific depth of field when shooting, and the focal plane of the traditional camera is a plane in the object space of the imaged three-dimensional scene. Due to the limitation of the diameter of an entrance pupil, the depth of field range of an image shot by a traditional camera is fixed, and the later-stage adjustment cannot be carried out, and the focusing cannot be carried out on a specific curved surface of a three-dimensional scene space. The camera array is a device for acquiring three-dimensional information of a three-dimensional scene from different angles, and the acquired three-dimensional information data has great redundancy. At present, a camera array is mainly applied to the fields of three-dimensional reconstruction and three-dimensional display, and meanwhile, the camera array also has application in the fields of target detection and identification.
Disclosure of Invention
The invention aims to realize a shooting method which is based on a camera array and can focus on any curved surface of a three-dimensional scene. The method has the advantages of high speed, adjustability and simple calibration. In order to achieve the purpose, the invention provides an arbitrary focal surface shooting method based on a camera array. The method is based on a camera array, and realizes equivalent virtual cameras by combining a camera array calibration method according to the pose of each camera in the camera array. And shooting the three-dimensional scene by using an equivalent virtual camera, calculating the pixel offset corresponding to each pixel of the target focal curved surface, and synthesizing a shot image of the target focal curved surface. The method comprises two processes of calculating the pixel offset of the parallax image and synthesizing any focal surface shot image.
The invention utilizes the camera array to obtain an equivalent virtual camera, which can focus on a specific plane in a three-dimensional scene and can focus on any curved surface in the three-dimensional scene, as shown in figure 1. The focal surface in the invention can be a three-dimensional scene surface and can also be a curved surface with any numerical value.
The flow of the method provided by the invention is as shown in figure 2, the pose of each camera in the camera array is adjusted, the position of a central depth plane and the range of shooting a three-dimensional scene are determined, the calibration parameters of the camera array are obtained by utilizing a camera array calibration method, and then the parallax images obtained by the camera array are corrected to obtain a corrected parallax image sequence; and calculating the pixel offset corresponding to each pixel on the target focal surface according to the determined target focal surface and by combining the index number of each camera, and finally synthesizing the shot image of the target focal surface.
In the calculation process of the pixel offset of the parallax image, firstly, the pose of each camera of the camera array is adjusted. In the camera array, the number of cameras isM×NAdjusting each camera in the camera array to ensure that all the cameras are in the same plane, and the interval between two adjacent cameras is the same and is deltaCSimultaneously, the shooting ranges of different cameras in the camera array are overlapped as much as possible, and the position of the central depth plane is determinedZ 0The plane of the camera array is parallel to the central depth plane and the distance between the two planes isLAs shown in fig. 3. Shooting chessboard calibration board by camera array, obtaining camera array parameters by using camera array calibration method, and further calculating calibration matrix of parallax image sequenceH m n,Wherein, in the step (A),mandnrespectively, of camera arraysmColumn, firstnThe index number corresponding to the line camera,m∈{1, 2, 3, …,M}，n∈{1, 2, 3,…,N}. Then, a corrected parallax image sequence is obtained. Shooting a three-dimensional scene by using a camera array to obtain a parallax image sequence with the resolution ofW P ×H P . First of camera arraymColumn, firstnThe parallax image obtained by the line camera isI m n,(x,y) Wherein, in the step (A),xandyrespectively, the three-dimensional scene parallax image pixel coordinates. Using calibration matricesH m n,Calculating to obtain corresponding corrected parallax image sequenceI' m n,(x',y')，I' m n,(x',y') AndI m n,(x,y) Satisfies the following conditions:
Figure 357519DEST_PATH_IMAGE001
（1）
wherein:
Figure 438608DEST_PATH_IMAGE002
（2）
finally, the shift direction and the shift magnitude of each pixel of the parallax image are calculated. As shown in FIG. 3, the point of presence on the focal surfacePAt a distance Δ from the central depth planeZPoint of contactPThe projection through the view volume of different cameras in the camera array has different projection coordinates on the central depth plane. In the invention, the camera at the central position of the camera array is taken as a reference camera, and the point isPThrough the first stepmColumn, firstnThe projection coordinates of the line camera view frame body on the central depth plane have a delta value compared with the projection coordinates of the reference camera view frame body on the central depth planeSPixel shift amount of (1), ΔSSatisfies the following conditions:
Figure 257659DEST_PATH_IMAGE003
（3）
Figure 957762DEST_PATH_IMAGE004
（4）
Figure 670503DEST_PATH_IMAGE005
（5）
wherein, DeltaS X And ΔS Y Are each ΔSIn thatxShaft andythe component of the axis is such that,round(x) denotes rounding to the nearest integer,W S andH S respectively parallax image sequence resolutionW P AndH P and the real range of the three-dimensional scene corresponding to the position of the central depth plane.
The synthesis process of the shot image of the arbitrary focal surface comprises the steps of firstly, determining the distance delta between the target focal surface and the central depth planeZ(x,y). In the invention, the target focal surface is an arbitrary curved surface, and the depth value of the target focal surface is expressed asZ(x,y) Wherein, in the step (A),xandyrespectively are the pixel coordinates of the target focal surface. According to the position of the central depth planeZ 0And calculating to obtain a target focal curved surfaceZ(x,y) Distance Δ from the central depth planeZ(x,y)：
Figure 30815DEST_PATH_IMAGE006
（6）
Then, calculating the target focal surface with each pixel at the secondmColumn, firstnPixel shift amount delta on parallax image corresponding to line cameraS m n,(x,y). Substituting equation (6) into equations (3), (4) and (5) yields:
Figure 461796DEST_PATH_IMAGE007
（7）
wherein, DeltaS m n X, ,(x,y) And ΔS m n Y, ,(x,y) Are each ΔS m n,(x,y) In thatxShaft andythe component of the axis. Finally, synthesizing the target focal surface shot imageO(x',y')，O(x',y') AndI' m n,(x',y') Satisfies the following conditions:
Figure 700011DEST_PATH_IMAGE008
（8）
wherein the content of the first and second substances,m∈{1, 2, 3, …,M}，n∈{1, 2, 3, …,N}. The images taken based on the different focal planes of the camera array are shown in fig. 4.
The invention provides a camera array-based arbitrary focal surface shooting method, which utilizes a calibrated camera array to acquire a parallax image sequence of a three-dimensional scene, breaks through the limitation that a traditional camera can only focus on a plane, has the advantages of high speed, adjustability and simple calibration, and can realize arbitrary curved surface focusing on the three-dimensional scene.
Drawings
FIG. 1 is a schematic diagram of an arbitrary focal surface shooting method based on a camera array
FIG. 2 is a flow chart of the method of the present invention
FIG. 3 is a schematic diagram of a process for calculating the pixel shift amount of a parallax image according to the present invention
FIG. 4 is (a 1) a Gaussian-shaped focal surface captured composite image, (a 2) a corresponding Gaussian-shaped focal surface, (b 1) an arbitrary focal surface I captured composite image, (b 2) a corresponding arbitrary focal surface I, (c 1) an arbitrary focal surface II captured composite image, (c 2) an arbitrary focal surface II corresponding to the arbitrary focal surface II, and (d) a focal surface depth value comparison table
The reference numbers in the figures are:
the system comprises a camera array 1, a virtual camera 2 equivalent to the camera array, a 3-focal plane I, a 4-focal plane II, a 5-focal curved surface, a 6-center depth plane and a 7-camera.
It should be understood that the above-described figures are merely schematic and are not drawn to scale.
Detailed Description
The present invention will be described in further detail below with reference to an exemplary embodiment of a camera array-based arbitrary focal surface photographing method according to the present invention. It should be noted that the following examples are only for illustrative purposes and should not be construed as limiting the scope of the present invention, and that the skilled person in the art may make modifications and adaptations of the present invention without departing from the scope of the present invention.
The invention provides an arbitrary focal surface shooting method based on a camera array.
The present embodiment utilizes the camera array to obtain an equivalent virtual camera, which can focus on not only a specific plane in the three-dimensional scene, but also any curved surface in the three-dimensional scene, as shown in fig. 1. The focal surface in this embodiment may be a three-dimensional scene surface, or may be an arbitrary numerical value surface.
The flow of this embodiment is as shown in fig. 2, the pose of each camera in the camera array is adjusted, the position of the central depth plane and the range for shooting the three-dimensional scene are determined, the calibration parameters of the camera array are obtained by using a camera array calibration method, and then the parallax images obtained by the camera array are corrected to obtain a corrected parallax image sequence; and calculating the pixel offset corresponding to each pixel on the target focal surface according to the determined target focal surface and by combining the index number of each camera, and finally synthesizing the shot image of the target focal surface.
In the calculation process of the pixel offset of the parallax image, firstly, the pose of each camera of the camera array is adjusted. In the camera array, the number of cameras isM×N= 11 × 11, each camera in the camera array is adjusted to ensure that all cameras are in the same plane, and the intervals between two adjacent cameras are the same and are ΔC= 20mm, while simultaneously making the shooting ranges of the different cameras in the camera array coincide as much as possible, while determining the center depth plane positionZ 0= 995mm, the plane of the camera array is parallel to the central depth plane, and the distance between the two planes isL= 995mm as shown in figure 3. Shooting chessboard calibration board by camera array, obtaining camera array parameters by using camera array calibration method, and further calculating calibration matrix of parallax image sequenceH m n,Wherein, in the step (A),mandnrespectively, of camera arraysmColumn, firstnThe index number corresponding to the line camera,m∈{1, 2, 3, …, 11}，ne {1,2,3, …, 11 }. Then, a corrected parallax image sequence is obtained. Shooting a three-dimensional scene by using a camera array to obtain a parallax image sequence with the resolution ofW P ×H P = 512 × 389. First of camera arraymColumn, firstnThe parallax image obtained by the line camera isI m n,(x,y) Wherein, in the step (A),xandyrespectively, the three-dimensional scene parallax image pixel coordinates. Using calibration matricesH m n,Calculating to obtain corresponding corrected parallax image sequenceI' m n,(x',y')，I' m n,(x',y') AndI m n,(x,y) Satisfies the following conditions:
Figure DEST_PATH_IMAGE009
（1）
wherein:
Figure 204941DEST_PATH_IMAGE010
（2）
finally, the shift direction and the shift magnitude of each pixel of the parallax image are calculated. As shown in FIG. 3, the point of presence on the focal surfacePAt a distance Δ from the central depth planeZ= 100mm, pointPThe projection through the view volume of different cameras in the camera array has different projection coordinates on the central depth plane. In the present embodiment, the camera at the center of the camera array is used as the reference camera, and the point is setPThrough the first stepmColumn, firstnThe projection coordinates of the line camera view frame body on the central depth plane have a delta value compared with the projection coordinates of the reference camera view frame body on the central depth planeSPixel shift amount of (1), ΔSSatisfies the following conditions:
Figure 237620DEST_PATH_IMAGE003
（3）
Figure 341184DEST_PATH_IMAGE004
（4）
Figure 710986DEST_PATH_IMAGE005
（5）
wherein, DeltaS X And ΔS Y Are each ΔSIn thatxShaft andythe component of the axis is such that,round(x) denotes rounding to the nearest integer,W S andH S respectively parallax image sequence resolutionW P AndH P in the real range of the three-dimensional scene corresponding to the position of the central depth plane, in this embodiment,W S = 2560mm andH S = 1945mm。
the synthesis process of the shot image of the arbitrary focal surface comprises the steps of firstly, determining the distance delta between the target focal surface and the central depth planeZ(x,y). In this embodiment, the target focal surface is an arbitrary surface, and the depth value thereof is expressed asZ(x,y) Wherein, in the step (A),xandyrespectively are the pixel coordinates of the target focal surface. According to the position of the central depth planeZ 0And calculating to obtain a target focal curved surfaceZ(x,y) Distance Δ from the central depth planeZ(x,y)：
Figure 8106DEST_PATH_IMAGE006
（6）
Then, the target focal surfaces are each calculatedThe pixel is at the secondmColumn, firstnPixel shift amount delta on parallax image corresponding to line cameraS m n,(x,y). Substituting equation (6) into equations (3), (4) and (5) yields:
Figure 805161DEST_PATH_IMAGE007
（7）
wherein, DeltaS m n X, ,(x,y) And ΔS m n Y, ,(x,y) Are each ΔS m n,(x,y) In thatxShaft andythe component of the axis. Finally, synthesizing the target focal surface shot imageO(x',y')，O(x',y') AndI' m n,(x',y') Satisfies the following conditions:
Figure DEST_PATH_IMAGE011
（8）
wherein the content of the first and second substances,m∈{1, 2, 3, …, 11}，ne {1,2,3, …, 11 }. In this embodiment, the target focal surfaceZ(x,y) When the curve is a gaussian curve, the corresponding focal curve is captured as shown in fig. 4 (a 1) and (a 2); as-target focal surfaceZ(x,y) When the arbitrary focal plane I and the arbitrary focal plane II are taken, the corresponding focal plane captured images are shown in FIGS. 4 (b 1), (b 2) (c 1) and (c 2); the depth value comparison table of the focal plane is shown in FIG. 4 (d).

Claims (1)

1. A method for shooting any focal surface based on a camera array is characterized by comprising two processes of calculating pixel offset of a parallax image and synthesizing any focal surface shot image, wherein in the process of calculating the pixel offset of the parallax image, firstly, the pose of each camera of the camera array is adjusted, the number of the cameras in the camera array is MxN, all the cameras are ensured to be positioned on the same plane, the interval between every two adjacent cameras is the same and is delta C, and meanwhile, the camera array is ensured to be positioned on the same planeCentered depth plane position Z0The plane of the camera array is parallel to the central depth plane, the distance between the plane and the central depth plane is L, the camera array parameters are obtained by utilizing a camera array calibration method, and then a calibration matrix H of the parallax image sequence is calculatedm,nWherein M and N are index numbers corresponding to the M-th column and the N-th row of the camera array respectively, M belongs to {1,2,3, …, M }, and N belongs to {1,2,3, …, N }; then, shooting a three-dimensional scene by using a camera array to obtain a parallax image sequence, wherein the resolution of the parallax image sequence is WP×HPThe parallax images obtained by the m-th and n-th row cameras of the camera array are Im,n(x, y), wherein x and y are respectively the pixel coordinates of the parallax image of the three-dimensional scene, and a calibration matrix H is utilizedm,nCalculating to obtain a corresponding corrected parallax image sequence I'm,n(x ', y'), where x 'and y' are corrected parallax image pixel coordinates, I ', respectively'm,n(x ', y') and Im,n(x, y) satisfy l'm,n(x′,y′)＝Im,n(x, y) wherein
Figure FDA0002316866380000011
Finally, when there is a point P on the focal surface, the distance from the focal plane is Δ Z, and the camera at the center position of the camera array is used as the reference camera, the projected coordinates of the point P on the focal plane through the mth column and the nth line of the camera view frame body have a pixel shift amount of Δ S compared with the projected coordinates on the focal plane through the reference camera view frame body, and Δ S satisfies Δ S ═ Δ S (Δ S satisfies Δ S ═ SX,ΔSY)，
Figure FDA0002316866380000012
Wherein, Delta SXAnd Δ SYIs the component of Δ S on the X and Y axes, round (X) means rounding to the nearest integer, WSAnd HSRespectively parallax image sequence resolution WPAnd HPIn the real range of the three-dimensional scene corresponding to the position of the central depth plane, the synthesis process of the shot image of any focal surface firstly determines the distance delta Z (x, y) between the target focal surface and the central depth plane and the target focalThe curved surface is an arbitrary curved surface, and the depth value of the curved surface is expressed as Z (x, y), wherein x and y are respectively the pixel coordinates of the target focal surface according to the central depth plane position Z0Calculating to obtain the distance delta Z (x, y) between the target focal curved surface Z (x, y) and the central depth plane as Z (x, y) -Z0(ii) a Then, the pixel offset of each pixel of the target focal surface on the parallax image corresponding to the m-th column and the n-th row of cameras is calculated
Figure FDA0002316866380000013
Wherein, Delta Sm,n,X(x, y) and Δ Sm,n,Y(x, y) are each Δ Sm,n(X, Y) components on the X-axis and Y-axis, m and n are index numbers corresponding to the m-th column and n-th row of the camera array, respectively, round (X) represents rounding the nearest integer, and WSAnd HSRespectively parallax image sequence resolution WPAnd HPIn a three-dimensional scene real range corresponding to the position of the central depth plane, the number of the cameras is MxN, the interval between two adjacent cameras is delta C, and the distance between the plane where the camera array is located and the central depth plane is L; finally, the target focal surface captured images O (x ', y'), O (x ', y') and I 'are synthesized'm,n(x ', y') satisfies
Figure FDA0002316866380000021
Wherein M belongs to {1,2,3, …, M }, and N belongs to {1,2,3, …, N }.
CN201710578545.4A 2017-07-17 2017-07-17 Arbitrary focal surface shooting method based on camera array Active CN107330933B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710578545.4A CN107330933B (en) 2017-07-17 2017-07-17 Arbitrary focal surface shooting method based on camera array

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710578545.4A CN107330933B (en) 2017-07-17 2017-07-17 Arbitrary focal surface shooting method based on camera array

Publications (2)

Publication Number Publication Date
CN107330933A CN107330933A (en) 2017-11-07
CN107330933B true CN107330933B (en) 2020-05-12

Family

ID=60227052

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710578545.4A Active CN107330933B (en) 2017-07-17 2017-07-17 Arbitrary focal surface shooting method based on camera array

Country Status (1)

Country Link
CN (1) CN107330933B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113409391B (en) * 2021-06-25 2023-03-03 浙江商汤科技开发有限公司 Visual positioning method and related device, equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102122393A (en) * 2010-01-09 2011-07-13 鸿富锦精密工业（深圳）有限公司 Method and system for building three-dimensional model and modeling device with system
CN104954779A (en) * 2015-06-23 2015-09-30 四川大学 Integral imaging three-dimensional display center depth plane adjusting method
CN105303574A (en) * 2015-07-30 2016-02-03 四川大学 Integrated imaging camera array calibration method based on homography transformation

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090101947A1 (en) * 2007-10-17 2009-04-23 Visera Technologies Company Limited Image sensor device and fabrication method thereof

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102122393A (en) * 2010-01-09 2011-07-13 鸿富锦精密工业（深圳）有限公司 Method and system for building three-dimensional model and modeling device with system
CN104954779A (en) * 2015-06-23 2015-09-30 四川大学 Integral imaging three-dimensional display center depth plane adjusting method
CN105303574A (en) * 2015-07-30 2016-02-03 四川大学 Integrated imaging camera array calibration method based on homography transformation

Also Published As

Publication number Publication date
CN107330933A (en) 2017-11-07

Similar Documents

Publication Publication Date Title
KR102487546B1 (en) Improved camera calibration system, target, and process
CN106846415B (en) A kind of multichannel fisheye camera binocular calibration device and method
CN105488810B (en) A kind of focusing light-field camera inside and outside parameter scaling method
CN109859272B (en) Automatic focusing binocular camera calibration method and device
CN107248178B (en) Fisheye camera calibration method based on distortion parameters
CN105959669B (en) It is a kind of based on the micro- pattern matrix rapid generation of integration imaging remapped
CN111667536A (en) Parameter calibration method based on zoom camera depth estimation
EP1087336A3 (en) Apparatus and method for stereoscopic image processing
CN112700502B (en) Binocular camera system and binocular camera space calibration method
CN102243432A (en) Panoramic three-dimensional photographing device
CN107424195B (en) Light field distance estimation method
CN106340045B (en) Calibration optimization method in three-dimensional facial reconstruction based on binocular stereo vision
CN109712232B (en) Object surface contour three-dimensional imaging method based on light field
CN110505468B (en) Test calibration and deviation correction method for augmented reality display equipment
WO2019065260A1 (en) Information processing device, information processing method, and program, and interchangeable lens
CN113175899A (en) Camera and galvanometer combined three-dimensional imaging model of variable sight line system and calibration method thereof
CN108230401A (en) 3D four-wheel position finder automatic camera calibration method and system
CN113554708A (en) Complete calibration method of linear structured light vision sensor based on single cylindrical target
CN114359405A (en) Calibration method of off-axis Samm 3D line laser camera
JP2011117787A (en) Distance image input device and device for monitoring vehicle exterior
CN104634323B (en) A kind of multistage shooting tri-dimensional photographing system and method
CN108805921A (en) Image-taking system and method
CN112950727B (en) Large-view-field multi-target simultaneous ranging method based on bionic curved compound eye
CN107330933B (en) Arbitrary focal surface shooting method based on camera array
CN113963065A (en) Lens internal reference calibration method and device based on external reference known and electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant