CN113759554B - Projection display system, method and storage medium - Google Patents
Projection display system, method and storage medium Download PDFInfo
- Publication number
- CN113759554B CN113759554B CN202111069160.8A CN202111069160A CN113759554B CN 113759554 B CN113759554 B CN 113759554B CN 202111069160 A CN202111069160 A CN 202111069160A CN 113759554 B CN113759554 B CN 113759554B
- Authority
- CN
- China
- Prior art keywords
- light
- image
- light source
- sub
- source end
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0132—Head-up displays characterised by optical features comprising binocular systems
- G02B2027/0134—Head-up displays characterised by optical features comprising binocular systems of stereoscopic type
Abstract
The embodiment of the application provides a projection display system, a projection display method and a storage medium. The projection display system includes: a light source end and a liquid crystal display panel; the liquid crystal display panel displays a first image and a second image, the first image and the second image are respectively projected to the retina of a left eye and the retina of a right eye under the irradiation of a first light source end and a second light source end, the focal lengths of the lens of the left eye and the lens of the right eye are adjusted to be matched with the virtual object distance of an object shared by the first image and the second image, and the opening size of each pixel area of the liquid crystal display panel is adjusted according to the virtual object distance, so that the splicing state of each sub-target projection image at the retina is realized; the problem of vergence adjustment in the display projection can be solved while preserving the focal length adjustment capability of the lens.
Description
Technical Field
The application relates to the technical field of projection display, in particular to a projection display system, a projection display method and a storage medium.
Background
The focusing mechanism of human eyes is that when a real world object is watched, the scenes seen by the left eye and the right eye have slight differences, the binocular parallax is analyzed by the brain to obtain the approximate information of the distance between the object and the human eyes immediately, at the moment, the lens of a single eye can rapidly adjust the focal length in an uncontrolled manner (actually controlled by involuntary nerves and not controlled by the consciousness of a user), and the object focused by the human eyes is ensured to be accurately imaged by each eye.
However, this focusing mechanism of the human eye can create convergence modulation problems in AR (Augmented Reality ) or VR (Virtual Reality) displays. Specifically, in the existing AR/VR technology, only the image accepted by the left and right monocular eyes can be a planar image, the depth of field of the image is fixed at a certain position in front of eyes, and as a result, the proper focal segment considered by the brain is not matched with the actual focal segment felt by the monocular eyes, so that serious internal consumption is generated: the brain judges that the virtual distance between an object in the image and human eyes is x meters according to the left eye image and the right eye image; the brain automatically controls the binocular lens to adjust the focal length so that the focal segment of the lens is matched with x meters; however, the depth of field of the left and right eye images is that the actual distance between the display device and human eyes is y meters, and because x is generally not equal to y, the imaging blur of the left and right eye images on retina is caused; the brain finds imaging blur, controls the lens to readjust, and finally adjusts the focal segment to be matched with y meters, so that imaging is clear; however, the brain again automatically controls accommodation of the binocular lens to a focal segment matching x meters according to binocular parallax. The cycle is repeated. The internal consumption not only can lead to ciliary muscle regulation fatigue, but also can greatly consume human brain energy, so that people can feel tired, dizziness and overuse brain, and the application of AR/VR technology is greatly limited.
Retinal projection technology is considered as one of the solutions to the problem of vergence adjustment. One conventional retinal projection method includes using a highly collimated backlight to transmit through an LCD (Liquid Crystal Display ) screen, condensing a parallel light beam containing display information to a lens optical center (the direction of the light beam passing through the lens optical center corresponding to a concave lens is not deflected) by a prism, and then projecting the light beam directly onto the retina.
However, with the existing retinal projection method, since the imaging on the retina is always clear, the focusing mechanism of the human eye cannot be triggered, and the lens loses the focal length adjusting function thoroughly, which requires that the distance between the device and the eyeball is strictly controlled, and no error is allowed, which cannot be achieved in practical application, and in practical operation, the image seen by the eye (i.e. the imaging on the retina) is often easily caused to be abnormal, for example, the sizes of the images seen by the left eye and the right eye are inconsistent, the image overlapping or splitting phenomenon occurs, and the brain vision treatment is abnormal.
Therefore, the technical scheme for realizing convergence consistency and keeping certain focal length adjustment capability of the crystalline lens is provided, and the technical problem to be solved by the person skilled in the art is provided.
Disclosure of Invention
The application aims at the defects of the prior art and provides a projection display system, a projection display method and a storage medium, which are used for solving the technical problems of convergence adjustment or incapability of adjusting a crystalline lens in the prior art.
In a first aspect, an embodiment of the present application provides a projection display system, including: a light source end and a liquid crystal display panel;
the light source end comprises a first light source end and a second light source end which are respectively positioned at one side of the liquid crystal display panel far away from the lens design position;
the liquid crystal display panel comprises a first liquid crystal display area and a second liquid crystal display area, and is used for respectively displaying a first image and a second image, so that the first image and the second image are respectively projected to the retina of a left eye and the retina of a right eye under the irradiation of the first light source end and the second light source end, and the focal lengths of a lens of the left eye and a lens of the right eye are adjusted to be matched with the virtual object distance of a common object in the first image and the second image; and adjusting the opening size of each pixel region of the liquid crystal display panel according to the virtual object distance, and displaying each sub-target image of the target image in each pixel region, so that each sub-target image is projected to the splicing state of each sub-target projection image at the retina through a lens with focal length matched with the virtual object distance
Optionally, the light source end includes at least one of the following:
the diameter of the output light beam of the first light source end is smaller than or equal to 0.01 millimeter;
the diameter of the output light beam of the second light source end is smaller than or equal to 0.01 millimeter.
Optionally, the pitch of the pixel regions is less than 50 microns.
Optionally, the first light source end includes:
a laser source for outputting collimated laser;
the light guide plate is positioned at the light emitting side of the laser source and is used for converting the collimated laser into a two-dimensional lattice type laser matrix;
and the lens group is positioned on the light emitting side of the light guide plate and is used for converging each split beam in the two-dimensional lattice type laser matrix into a point light source, and each point light source forms a two-dimensional lattice type point light source matrix.
Optionally, the light guide plate includes:
the first beam splitting assembly is used for splitting the collimated laser into n first split beams; the light intensity of each first split beam is equal, 1/n of the light intensity of the collimated laser is obtained, and n is a positive integer;
the light emitting surface corresponds to the light entering surface of the first light splitting component and is used for splitting each first light splitting beam into m second light splitting beams; the light intensity of each second sub-beam is equal, 1/m of the light intensity of the first sub-beam is equal, and m is a positive integer; the two-dimensional lattice type laser matrix comprises n x m second sub-beams.
Optionally, the first light splitting component includes n light splitting pieces arranged in a row, the n light splitting pieces include 1 total reflection plate and n-1 transmission reflection plate, and the total reflection plate is farthest from the collimated laser;
the collimated laser is a light beam with n x m parts of light intensity;
in the first light splitting assembly, the 1 st transparent reflecting plate is the transparent reflecting plate farthest from the total reflecting plate; the ith transmission reflecting plate is used for reflecting m parts of light intensity light beams in the (n-i+1) m parts of light intensity incident collimated laser beams in a first direction to form a first split light beam, and transmitting the (n-i) m parts of light intensity light beams in the (n-i+1) m parts of light intensity incident collimated laser beams to form the (i+1) th incident collimated laser beams transmitted through the reflecting plate; n is a positive integer greater than 2, i is an integer value, and the first direction is a direction parallel to the light emitting surface of the light guide plate.
Optionally, the second light splitting component includes n second sub-group components, an incident light path of each second sub-group component corresponds to a reflected light path of one light splitting component in the first light splitting component, the second sub-group component includes m light splitting components arranged in a row along the first direction, the m light splitting components include 1 total reflection plate and m-1 transmission reflection plate, and the total reflection plate is farthest from the first light splitting component;
In the second light splitting assembly, the 1 st transparent reflecting plate is the transparent reflecting plate closest to the first light splitting assembly; the j-th transmission reflecting plate is used for reflecting 1 part of light intensity light beams in the first incident split light beams with the light intensity of m-j+1 parts to a second direction to form a second split light beam, and transmitting m-j parts of light intensity light beams in the first incident split light beams with the light intensity of m-j+1 parts to form the j-th first incident split light beams transmitted through the reflecting plate; m is a positive integer greater than 2, j is an integer value, and the second direction is a direction perpendicular to the light emitting surface of the light guide plate.
In a second aspect, an embodiment of the present application provides a projection display method, including:
displaying a first image and a second image respectively, so that the first image and the second image are projected to the retina of a left eye and the retina of a right eye respectively under the irradiation of a first light source end and a second light source end, and the focal lengths of a lens of the left eye and a lens of the right eye are adjusted to be matched with the virtual object distance of an object in common between the first image and the second image;
and adjusting the opening size of each pixel area of the liquid crystal display panel according to the virtual object distance, and displaying each sub-target image of the target image in each pixel area, so that each sub-target image is projected to each sub-target projection image at the retina through a lens with a focal length matched with the virtual object distance to be in a spliced state.
Optionally, the first image comprises a left eye image and the second image comprises a right eye image.
Optionally, displaying the first image and the second image respectively, so that the first image and the second image are projected to retinas of a left eye and a right eye respectively under the illumination of the first light source end and the second light source end, including:
displaying a plurality of first sub-images included in the first image in a first liquid crystal display area of the liquid crystal display panel, so that each first sub-projection image projected to the retina of the left eye under the irradiation of a first light source end is in an overlapped state or a split state;
in a third aspect of displaying a plurality of second sub-images included in the second image in a second liquid crystal display area of the liquid crystal display panel, so that each of the second sub-images is projected to a retina of a right eye under the irradiation of a second light source end, an embodiment of the present application provides a computer readable storage medium having stored thereon a computer program that when executed by a processor implements the projection control method provided in the second aspect of the present application.
The technical scheme provided by the embodiment of the application has the beneficial technical effects that:
The projection display system provided by the embodiment of the application adopts the point light source to irradiate the target image and project the target image to the retinas of the left eye and the right eye, and adopts the retinal projection technology as a whole, so that the problem of convergence adjustment is solved, and the target projection image projected to the retinas is always clear.
Moreover, most or all of the light rays in the application do not pass through the optical center of the lens, so that the focal length adjustment capability of the lens is reserved, and the feasibility of adjusting the size of a projection image at the retina by triggering the focal length adjustment of the lens is reserved. Specifically:
according to the projection display system provided by the embodiment of the application, the focusing mechanism of the human eyes is triggered through the first image and the second image respectively projected to the left eye and the right eye, namely the focal length adjustment of the binocular lens is triggered, so that the focal length of the binocular lens is matched with the virtual object distance of an object shared by the first image and the second image. And, adjust the opening size of each pixel area of the liquid crystal display panel according to the virtual object distance, make the focal length of the lens of the left and right eyes and opening size of each pixel area of the liquid crystal display panel match with the virtual object distance, make each sub-goal image projected to each sub-goal projection image of retina place splice state, namely each sub-goal projection image will not be in the split state because of too small, will not be in the overlapping state because of too big, but the size just benefits the state of splice.
Further, since the lens focal lengths of the two eyes are matched with the uniform virtual object distance, the sizes of the target projection images of the two eyes obtained by projection of the lens of the two eyes after the focal lengths are matched are consistent. In addition, by using the technical scheme of the application, the size of each monocular target projection image can be independently adjusted based on the first liquid crystal display area and the second liquid crystal display area which are independent of each other.
Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.
Drawings
The foregoing and/or additional aspects and advantages of the application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a schematic diagram of a projection display system according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of a light source end of a projection display system according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a projection display system for projecting images to left and right eyes, respectively, according to an embodiment of the present application;
FIG. 4 is a schematic flow chart of a projection display method according to an embodiment of the present application;
Fig. 5 is a schematic diagram of a splicing state of a retinal imaging site according to an embodiment of the present application;
FIG. 6 is a schematic diagram of overlapping states of retinal imaging sites according to an embodiment of the present application;
FIG. 7 is a diagram illustrating a separation state of retina imaging according to an embodiment of the present application;
FIG. 8 is a schematic view of a stitching state of an image "88" at a retinal imaging site according to an embodiment of the present application;
FIG. 9 is a schematic diagram of an overlapping state of images "88" at retinal imaging, according to an embodiment of the present application;
fig. 10 is a schematic diagram of the separation state of an image "88" at retinal imaging according to an embodiment of the present application.
The reference numerals of the drawings are explained as follows:
100-a first light source end; 110-point light sources; 200-a first liquid crystal display region; 210-imaging shadows; 220-a target projection image; 230-target image; 240-pixel area; 250-a first sub-target image; 260-a second sub-target image; 2501-first sub-target projection image; 2601-a second sub-target projection image; 300-crystalline lens; 310-retina;
600-laser source; 610-collimated laser; 400-light guide plate; 620-a first split beam; 630-second split beam;
70-a first light splitting assembly; 700-a first transflector; 701-a second transflector; 702-a third transflector; 703-fourth transflector; 704-fifth pass through reflection plate; 705-total reflection plate;
71-a second light splitting component; 710-sixth-seventh-730-eighth transmissive-reflective plates;
810-a first image; 820-a second image; 830-common object; 800-projection display device.
Detailed Description
The present application is described in detail below, examples of embodiments of the application are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar components or components having the same or similar functions throughout. Further, if detailed description of the known technology is not necessary for the illustrated features of the present application, it will be omitted. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the application.
It will be understood by those skilled in the art that all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs unless defined otherwise. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It should be understood that the term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.
The following describes the technical scheme of the present application and how the technical scheme of the present application solves the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
The embodiment of the application provides a projection display system, which comprises a light source end and a liquid crystal display panel.
The light source end comprises a first light source end and a second light source end which are respectively positioned at one side of the liquid crystal display panel far away from the lens design position;
The liquid crystal display panel comprises a first liquid crystal display area and a second liquid crystal display area, and is used for respectively displaying a first image and a second image, so that the first image and the second image are respectively projected to the retina of a left eye and the retina of a right eye under the irradiation of a first light source end and a second light source end, and the focal lengths of a lens of the left eye and a lens of the right eye are adjusted to be matched with the virtual object distance of a common object in the first image and the second image; and adjusting the opening size of each pixel area of the liquid crystal display panel according to the virtual object distance, and displaying each sub-target image of the target image in each pixel area, so that each sub-target image is projected to the splicing state of each sub-target projection image at the retina through a lens with the focal length matched with the virtual object distance.
The projection display system provided by the embodiment of the application adopts the point light source 110 to irradiate the target image 230 to project to the retina 310 of the left eye and the right eye, and adopts the retina projection technology as a whole, so that the problem of convergence adjustment is solved, and the target projection image projected to the retina is always clear.
Moreover, most or all of the light rays in the present application do not pass through the optical center of lens 300, preserving the focal length adjustment capabilities of lens 300, preserving the feasibility of achieving a projected image size adjustment at retina 310 by triggering focal length adjustment of lens 300. Specifically:
According to the projection display system provided by the embodiment of the application, the focusing mechanism of the human eyes is triggered through the first image and the second image respectively projected to the left eye and the right eye, namely the focal length adjustment of the binocular lens 300 is triggered, so that the focal length of the binocular lens 300 is matched with the virtual object distance of an object shared by the first image and the second image. And, the opening size of each pixel region 340 of the liquid crystal display panel is adjusted according to the virtual object distance so that the focal length of the lens 300 of the left and right eyes and the opening size of each pixel region 340 of the liquid crystal display panel are matched with the virtual object distance, so that each sub-target image is projected to the spliced state of each sub-target projection image at the retina 310, that is, each sub-target projection image is not in the split state due to too small or in the overlapped state due to too large, but the size is just in the spliced state.
Further, since the focal lengths of the lenses 300 of both eyes are matched with the uniform virtual object distance, the sizes of the target projection images of both eyes projected by the lenses 300 of both eyes after the focal lengths are matched are identical. In addition, with the technical scheme of the present application, the size of each monocular target projection image can be independently adjusted based on the first liquid crystal display region 200 and the second liquid crystal display region which are independent of each other.
Alternatively, the first image and the second image projected to the left eye and the right eye both belong to the target image 230.
Alternatively, as shown in fig. 1, the first light source end 100 outputs a point light source 110, the point light source 110 transmits through a pixel area 240 of a first display area 200 of the liquid crystal display panel, displays a target image 230, and forms an imaging shadow 210 of the target image 230, and the imaging shadow 210 of the target image 230 is projected onto retinas 310 of the left and right eyes under the action of crystalline lenses 300 of the left and right eyes to form a target projection image 220.
Alternatively, the target image 230 is composed of at least two sub-images including the first sub-target image 250 and the second sub-target image 260, and the arrangement states of the first sub-target projection image 2501 and the second sub-target projection image 2601, in which the first sub-target image 250 and the second sub-target image 260 are projected onto the retina through the lens 300, include a mosaic state.
Alternatively, the first and second images projected to the left and right eyes represent different appearances of an object at the left and right eye viewing angles, as shown in fig. 3, there is a slight difference in the common object 830 of the first image 810 projected to the left eye and the second image 820 projected to the right eye, and the virtual object distance of the object can be determined according to the slight difference in the common object 830.
The brain determines the virtual object distance of the object by analyzing the subtle differences that exist in the common object 830 of the first image 810 projected to the left eye and the second image 820 projected to the right eye, and adjusts the lenses of the left and right eyes to a state that matches the virtual object distance of the object, i.e., a proper focal length state.
The projection display system in the projection display device 800 determines the virtual object distance of the object based on the subtle difference between the common object 830 projected to the first image 810 for the left eye and the second image 820 for the right eye, and adjusts the opening size of the liquid crystal display panel pixel area 240 corresponding to the left eye and the right eye to match the virtual object distance of the object, i.e. the opening size of the suitable liquid crystal display panel pixel area 240.
At a proper focal length state matched with the virtual object distance of the object and a proper opening size of the pixel region 240 of the liquid crystal display panel, the imaging of the first image 810 projected to the left eye retina and the imaging of the second image 820 projected to the right eye retina are both in a stitched state, and the images seen by the human eye are displayed normally.
Optionally, the liquid crystal display panel is an LCD (Liquid Crystal Display ).
Optionally, the projection display system of the embodiment of the application is applicable to the field of virtual 3D projection.
The inventor of the present application considers that since there is a slight difference between the first image 810 and the second image 820 projected to the left and right eyes, there is a disturbance in the projection setting of the left and right eye images and the adjustment of the pixel area if the common light source terminal 100 and the display area of the liquid crystal display panel are set.
Based on the above consideration, the embodiment of the application sets two light source ends and the display area of the liquid crystal display panel, which correspond to the left eye and the right eye respectively, so that the projection setting of the image and the adjustment of the pixel area are more accurate.
In some embodiments, the first light source end 100 outputs a beam of light having a diameter less than or equal to 0.01 millimeters.
In some embodiments, the second light source end outputs a beam of light having a diameter less than or equal to 0.01 millimeters.
The output beam of the first light source end 100 is a point light source 110. When the diameter of the point light source 110 is less than or equal to 0.01 mm, the retina display level can be approached, and the display device is suitable for general application scenes and meets display requirements.
Alternatively, the pitch of the pixel regions 240 is less than 50 microns.
When the pitch of the pixel area 240 is less than 50 micrometers, the retina display level can be approached, and the method is suitable for general application scenes and meets display requirements.
In some embodiments, as shown in fig. 2, the first light source end 100 includes:
A laser source 600 for outputting collimated laser light 610;
the light guide plate 400 is located at the light emitting side of the laser source 600, and is used for converting the collimated laser 610 into a two-dimensional lattice type laser matrix;
and the lens group is positioned on the light emitting side of the light guide plate 400 and is used for converging each split beam in the two-dimensional lattice type laser matrix into a point light source, and each point light source forms a two-dimensional lattice type point light source matrix.
Alternatively, the lens group is composed of lenses 500 in one-to-one correspondence with each of the split beams in the two-dimensional lattice laser matrix, converging the split beams into a point light source.
Optionally, the split beam is a second split beam 630 obtained by reflecting the collimated laser 610 twice in the light guide plate 400, all the second split beams 630 equally divide the total light intensity of the collimated laser 610, and the obtained second split beam 630 can obtain a point light source meeting the display requirement through the convergence of lenses.
In some embodiments, the light guide plate 400 includes:
a first beam splitting assembly 70 for splitting the collimated laser light into n first split beams 620; the light intensity of each first sub-beam 620 is equal, 1/n of the light intensity of the collimated laser, n being a positive integer;
the second light splitting component 71, the light emitting surface corresponds to the light incident surface of the first light splitting component, and is configured to split each of the first light splitting beams 620 into m second light splitting beams 630; the light intensity of each second sub-beam 630 is equal, 1/m of the light intensity of the first sub-beam 620, m being a positive integer; the two-dimensional lattice laser matrix includes n x m second sub-beams 630.
Optionally, the first light-splitting assembly 70 includes n light-splitting elements arranged in a row, where the n light-splitting elements include 1 total reflection plate and n-1 transmission reflection plates, and the total reflection plate is farthest from the collimated laser light;
the collimated laser is a light beam with n x m parts of light intensity;
in the first beam splitter assembly, the 1 st transmission reflecting plate is the transmission reflecting plate farthest from the total reflecting plate; the ith transmission reflecting plate is used for reflecting m parts of light intensity light beams in the (n-i+1) m parts of light intensity incident collimated laser beams in a first direction to form a first split light beam, and transmitting (n-i) m parts of light intensity light beams in the (n-i+1) m parts of light intensity incident collimated laser beams to form the (i+1) th transmission collimated laser beams; n is a positive integer greater than 2, i is an integer value, and the first direction is a direction parallel to the light emitting surface of the light guide plate.
Optionally, the second light splitting component 71 includes n second sub-group components, where an incident light path of each second sub-group component corresponds to a reflected light path of one light splitting element in the first light splitting component, and the second sub-group component includes m light splitting elements arranged in a row along the first direction, where the m light splitting elements include 1 total reflection plate and m-1 transmission reflection plates, and the total reflection plate is furthest away from the first light splitting component;
In the second light splitting component, the 1 st transparent reflecting plate is the transparent reflecting plate closest to the first light splitting component; the j-th transmission reflecting plate is used for reflecting 1 part of light intensity light beams in the first incident split light beams with the light intensity of m-j+1 parts to a second direction to form a second split light beam, and transmitting m-j parts of light intensity light beams in the first incident split light beams with the light intensity of m-j+1 parts to form j+1-th first incident split light beams transmitted through the reflecting plate; m is a positive integer greater than 2, j is an integer value, and the second direction is a direction perpendicular to the light emitting surface of the light guide plate.
Optionally, six light splitting elements are disposed along the incident direction of the collimated laser 610 to form a first light splitting assembly 70, including five transmissive reflecting plates and a total reflecting plate, where the total reflecting plate is furthest away from the incident point of the collimated laser 610, the first light splitting assembly 70 forms 45 ° with the incident direction of the collimated laser 610, so that the reflected light of the collimated laser on the transmissive reflecting plate of the first light splitting assembly 70 is used as a first light splitting beam 620, the direction of the first light splitting beam 620 is perpendicular to the original incident direction and is denoted as a first direction, and the first direction is parallel to the light emitting surface direction of the light guide plate 400, so that the collimated laser 610 is split into six first light splitting beams 620 with the same light intensity, which are all 1/6 of the light intensity of the collimated laser 610; the second beam splitting component 71 is formed by arranging four beam splitting components along the first direction in the direction of the first beam splitting component 620, and comprises three transmission reflecting plates and a total reflecting plate, wherein the total reflecting plate is farthest from the incidence point of the first beam splitting component 620, the incidence direction of the second beam splitting component 71 and the first beam splitting component 620 is 45 degrees, so that the reflected light of the first beam splitting component 620 on the transmission reflecting plate of the second beam splitting component 71 is taken as a second beam splitting component 630, the direction of the second beam splitting component 630 is perpendicular to the direction of the first beam splitting component 620 and is marked as a second direction, and the second direction is perpendicular to the light emitting surface direction of the light guiding plate 400, so that the first beam splitting component 620 is split into four second beam splitting components 630 with the same light intensity, and the light intensity is 1/4 of the light intensity of the first beam splitting component 620.
The first beam splitter 70 includes a first transmissive/reflective plate 700, a second transmissive/reflective plate 701, a third transmissive/reflective plate 702, a fourth transmissive/reflective plate 702, a fifth transmissive/reflective plate 704, and a total reflection plate 705 along the incident direction of the collimated laser beam 610, and the second beam splitter 71 includes a sixth transmissive/reflective plate 710, a seventh transmissive/reflective plate 720, an eighth transmissive/reflective plate 730, and a total reflection plate 705 along the direction of the first beam splitter 620.
Assuming that the light intensity of the collimated laser light 610 is 24 light intensities, transmission of the first transmissive and reflective plate 700, the second transmissive and reflective plate 701, the third transmissive and reflective plate 702, the fourth transmissive and reflective plate 702, and the fifth transmissive and reflective plate 704 are provided: the reflection ratio is 20:4, 16:4, 12:4, 8:4, and 4:4, so that the incident collimated laser 610 reflects 4 parts of the first split beam 620 with light intensity in the first direction under the action of the first transmissive reflector 700, the collimated laser with light intensity passing through 20 parts is used as the incident collimated laser with light intensity passing through the second transmissive reflector 701, the first split beam 620 with light intensity reflecting 4 parts in the first direction under the action of the second transmissive reflector 701, the collimated laser with light intensity passing through 16 parts is used as the incident collimated laser with light intensity passing through the third transmissive reflector 702, and so on, the total reflector 705 receives the collimated laser with light intensity of 4 parts transmitted through the fifth transmissive reflector 704 and reflects all the collimated laser with light intensity as reflected light in the first direction.
The transmission of the sixth, seventh and eighth transmissive-reflective plates 710, 720 and 730 is provided: the reflection ratio is 3:1, 2:1, and 1:1, so that the incident first split beam 620 reflects the second split beam 630 with 1 part of light intensity in the second direction under the action of the sixth transmissive and reflective plate 710, the first split beam with 3 parts of light intensity is used as the incident split beam with the seventh transmissive and reflective plate 720, the second split beam 630 with 1 part of light intensity is reflected in the second direction under the action of the seventh transmissive and reflective plate 720, the split beam with 2 parts of light intensity is used as the incident split beam with the eighth transmissive and reflective plate 730, and so on, the total reflective plate 705 receives the incident split beam with the eighth transmissive and reflective plate 730 and totally reflects as the reflected light in the second direction.
A 6*4 two-dimensional lattice type laser matrix composed of second sub-beams 630 with the light intensity of 1 part along the second direction is obtained, each second sub-beam 630 is converged into a point light source under the action of a corresponding lens 500 in front of the light guide plate, and the laser source 600 forms a two-dimensional lattice type point light source matrix under the combined action of the light guide plate 400 and the lens group to complete the configuration of a light source end.
A projection display control method, comprising:
Displaying the first image and the second image respectively, so that the first image and the second image are projected to the retina of the left eye and the retina of the right eye respectively under the irradiation of the first light source end and the second light source end, and the focal lengths of the lens of the left eye and the lens of the right eye are adjusted to be matched with the virtual object distance of a common object in the first image and the second image;
and adjusting the opening size of each pixel area of the liquid crystal display panel according to the virtual object distance, and displaying each sub-target image of the target image in each pixel area, so that each sub-target image is projected to each sub-target projection image at the retina through a lens with the focal length matched with the virtual object distance to be in a spliced state.
Based on the same inventive concept, in matching with the projection display system provided by the present application, an embodiment of the present application provides a projection display control method, and a flow chart of the method is shown in fig. 4, where the method includes steps S401 and S402 as follows:
s401: the first image and the second image are displayed respectively, so that the first image and the second image are projected to the retina of the left eye and the retina of the right eye respectively under the illumination of the first light source end and the second light source end, and the focal lengths of the lens of the left eye and the lens of the right eye are adjusted to be matched with the virtual object distance of a common object in the first image and the second image.
Optionally, the first image comprises a left eye image and the second image comprises a right eye image.
Optionally, displaying the first image and the second image respectively, so that the first image and the second image are projected to retinas of the left eye and the right eye respectively under the illumination of the first light source end and the second light source end, including:
displaying a plurality of first sub-images included in the first image in a first liquid crystal display area of the liquid crystal display panel, so that each first sub-projection image projected to the retina of the left eye under the irradiation of a first light source end is in an overlapped state or a split state;
and displaying a plurality of second sub-images included in the second image in a second liquid crystal display area of the liquid crystal display panel, so that each second sub-projection image projected to the retina of the right eye under the irradiation of the second light source end is in an overlapped state or a split state.
Alternatively, the first and second sub-target projection images 2501 and 2601 are formed in three types of stitching, overlapping, and dividing as shown in fig. 5 to 7, and the first and second sub-target projection images 2501 and 2601 are mainly affected by the focusing state of the lens 300 and the opening size of the pixel region 240.
As shown in fig. 5, when the focal length of the lens 300 is proper, the first sub-target projection image 2501 and the second sub-target projection image 2601 are in a spliced state on the retina, the image margins a and b of the first sub-target projection image 2501 and the second sub-target projection image 2601 are 0, and at this time, the image seen by the human eye is displayed normally; as shown in fig. 6, when the focusing power of the lens 300 is too weak, the first sub-target projection image 2501 and the second sub-target projection image 2601 are in an overlapped state on the retina, the image margins a and b of the first sub-target projection image 2501 and the second sub-target projection image 2601 are not 0 and overlap, and at this time, the image seen by the human eye shows abnormality; as shown in fig. 7, when the focusing power of the lens 300 is too strong, the first sub-target projection image 2501 and the second sub-target projection image 2601 are in a divided state on the retina, the image margins a and b of the first sub-target projection image 2501 and the second sub-target projection image 2601 are not 0 and are dispersed, and at this time, the image seen by the human eye is abnormal in display.
The images seen by the human eye show an abnormal perception of a blurred state, and the lens can be adjusted to a proper focal length state by brain adjustment and concentration, so that the imaging on the retina is in a spliced state.
Meanwhile, the size of the opening of the pixel region 240 shown in fig. 1 is adjustable, which is represented by scaling the target image 230, and the first sub-target image 250 and the second sub-target image 260 scale together with the scaling of the target image 230, so that the size and the combination state of the first sub-target projection image 2501 and the second sub-target projection image 2601 projected onto the retina 310 are affected, and the perception of the sharpness of the target projection image 220 on the retina 310 is affected.
In a lens state where the focusing power is unchanged and the size of the opening of the pixel region 240 is proper, the first sub-target projection image 2501 and the second sub-target projection image 2601 are in a spliced state on the retina, similar to the image edge distance a and b of the first sub-target projection image 2501 and the second sub-target projection image 2601 shown in fig. 5, which is 0, and the image seen by the human eye is displayed normally; when the opening size of the pixel region 240 is too large, the first sub-target projection image 2501 and the second sub-target projection image 2601 are in a large and overlapping state on the retina, and like the one shown in fig. 6, the image margins a and b of the first sub-target projection image 2501 and the second sub-target projection image 2601 are not 0 and overlap, and at this time, the image display seen by the human eye is abnormal; when the opening size of the pixel region 240 is too small, the first and second sub-target projection images 2501 and 2601 are imaged on the retina too small and in a divided state, similarly to the case shown in fig. 7, the image margins a and b of the first and second sub-target projection images 2501 and 2601 are not 0 and are dispersed, and the image seen by the human eye is displayed abnormally at this time
The abnormal image display perceived by the human eye is perceived as a blurred state, and the imaging on the retina can be brought into a spliced state by adjusting the opening size of the pixel region 240 of the liquid crystal display panel.
Fig. 8-10 correspond to the specific state of the three imaging modalities of stitching, overlapping and splitting shown in fig. 5-7 described above when the human eye sees "88". As shown in fig. 8, in the appropriate lens state and the appropriate opening size of the liquid crystal display panel pixel region 240, the target image "88" is imaged in a stitched state, the retina is imaged normally, and the target image "88" is imaged clearly; as shown in fig. 9, when in a lens state where focusing power is too weak and/or the opening size of the liquid crystal display panel pixel region 240 is too large, the target image "88" is imaged in an overlapped state, retinal imaging is abnormal, and the target image "88" is imaged blurred; as shown in fig. 10, when the lens state is in which focusing power is too strong and/or the opening size of the pixel region 240 of the liquid crystal display panel is too small, the target image "88" is imaged in a divided state, retinal imaging is abnormal, and the target image "88" is blurred.
S402: and adjusting the opening size of each pixel area of the liquid crystal display panel according to the virtual object distance, and displaying each sub-target image of the target image in each pixel area, so that each sub-target image is projected to each sub-target projection image at the retina through a lens with the focal length matched with the virtual object distance to be in a spliced state.
Based on the same inventive concept, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, which when executed by a processor, implements the projection control method provided by the present application.
By applying the embodiment of the application, at least the following beneficial effects can be realized:
the projection display system provided by the embodiment of the application adopts the point light source to irradiate the target image and project the target image to the retinas of the left eye and the right eye, and adopts the retinal projection technology as a whole, so that the problem of convergence adjustment is solved, and the target projection image projected to the retinas is always clear.
Moreover, most or all of the light rays in the application do not pass through the optical center of the lens, so that the focal length adjustment capability of the lens is reserved, and the feasibility of adjusting the size of a projection image at the retina by triggering the focal length adjustment of the lens is reserved. Specifically:
according to the projection display system provided by the embodiment of the application, the focusing mechanism of the human eyes is triggered through the first image and the second image respectively projected to the left eye and the right eye, namely the focal length adjustment of the binocular lens is triggered, so that the focal length of the binocular lens is matched with the virtual object distance of an object shared by the first image and the second image. And, adjust the opening size of each pixel area of the liquid crystal display panel according to the virtual object distance, make the focal length of the lens of the left and right eyes and opening size of each pixel area of the liquid crystal display panel match with the virtual object distance, make each sub-goal image projected to each sub-goal projection image of retina place splice state, namely each sub-goal projection image will not be in the split state because of too small, will not be in the overlapping state because of too big, but the size just benefits the state of splice.
Further, since the lens focal lengths of the two eyes are matched with the uniform virtual object distance, the sizes of the target projection images of the two eyes obtained by projection of the lens of the two eyes after the focal lengths are matched are consistent. In addition, by using the technical scheme of the application, the size of each monocular target projection image can be independently adjusted based on the first liquid crystal display area and the second liquid crystal display area which are independent of each other.
Those of skill in the art will appreciate that the various operations, methods, steps in the flow, acts, schemes, and alternatives discussed in the present application may be alternated, altered, combined, or eliminated. Further, other steps, means, or steps in a process having various operations, methods, or procedures discussed herein may be alternated, altered, rearranged, disassembled, combined, or eliminated. Further, steps, measures, schemes in the prior art with various operations, methods, flows disclosed in the present application may also be alternated, altered, rearranged, decomposed, combined, or deleted.
In the description of the present application, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like indicate orientations or positional relationships based on the orientation or positional relationships shown in the drawings, merely to facilitate describing the present application and simplify the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present application.
The terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include one or more such feature. In the description of the present application, unless otherwise indicated, the meaning of "a plurality" is two or more.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
The foregoing is only a partial embodiment of the present application, and it should be noted that it will be apparent to those skilled in the art that modifications and adaptations can be made without departing from the principles of the present application, and such modifications and adaptations should and are intended to be comprehended within the scope of the present application.
Claims (10)
1. A projection display system, comprising: a light source end and a liquid crystal display panel;
the light source end comprises a first light source end and a second light source end which are different, and the first light source end and the second light source end are respectively positioned at one side of the first liquid crystal display area and the second liquid crystal display area of the liquid crystal display panel, which are far away from the lens design position; the light beam output by the first light source end comprises a plurality of point light sources, and each point light source forms a two-dimensional dot matrix type point light source matrix; the first light source end comprises a laser source, a light guide plate and a lens group; the laser source is used for outputting collimated laser; the light guide plate is positioned at the light emitting side of the laser source and is used for converting the collimated laser into a two-dimensional lattice type laser matrix; the lens group is positioned at the light-emitting side of the light guide plate and is used for converging each split beam in the two-dimensional lattice type laser matrix into a point light source, and each point light source forms a two-dimensional lattice type point light source matrix;
The liquid crystal display panel comprises a first liquid crystal display area and a second liquid crystal display area, and is used for respectively displaying a first image and a second image, so that the first image and the second image are respectively projected to retina of a left eye and retina of a right eye under the irradiation of each point light source output by the first light source end and each point light source output by the second light source end, and focal lengths of a lens of the left eye and a lens of the right eye are adjusted to be matched with virtual object distances of a common object in the first image and the second image; and adjusting the opening size of each pixel area of the liquid crystal display panel according to the virtual object distance, and displaying each sub-target image of the target image in each pixel area, so that each sub-target projection image of each sub-target image projected to the retina is in a spliced state, wherein each sub-target projection image is formed by projecting each sub-target image through a lens with a focal length matched with the virtual object distance.
2. The projection display system of claim 1, wherein the light source end comprises at least one of:
the diameter of the output light beam of the first light source end is smaller than or equal to 0.01 millimeter;
The diameter of the output light beam of the second light source end is smaller than or equal to 0.01 millimeter.
3. The projection display system of claim 1, wherein the pixel areas are less than 50 microns apart.
4. The projection display system of claim 1, wherein the light guide plate comprises:
the first beam splitting assembly is used for splitting the collimated laser into n first split beams; the light intensity of each first split beam is equal, 1/n of the light intensity of the collimated laser is obtained, and n is a positive integer;
the light incident surface corresponds to the light emergent surface of the first light splitting component and is used for splitting each first light splitting beam into m second light splitting beams; the light intensity of each second sub-beam is equal, 1/m of the light intensity of the first sub-beam is equal, and m is a positive integer; the two-dimensional lattice type laser matrix comprises n x m second sub-beams.
5. The projection display system of claim 4, wherein the first light splitting assembly comprises n light splitting elements arranged in a row, the n light splitting elements comprising 1 total reflection plate and n-1 transflector, the total reflection plate being furthest from the collimated laser light;
the collimated laser is a light beam with n x m parts of light intensity;
In the first light splitting assembly, the 1 st transparent reflecting plate is the transparent reflecting plate farthest from the total reflecting plate; the ith transmission reflecting plate is used for reflecting m parts of light intensity light beams in the (n-i+1) m parts of light intensity incident collimated laser beams in a first direction to form a first split light beam, and transmitting the (n-i) m parts of light intensity light beams in the (n-i+1) m parts of light intensity incident collimated laser beams to form the (i+1) th incident collimated laser beams transmitted through the reflecting plate; n is a positive integer greater than 2, i is an integer value, and the first direction is a direction parallel to the light emitting surface of the light guide plate.
6. The projection display system of claim 4, wherein the second light splitting assembly includes n second sub-assemblies, each of the second sub-assemblies having an incident light path corresponding to a reflected light path of one of the light splitting members of the first light splitting assembly, the second sub-assemblies including m light splitting members arranged in a row along a first direction, the m light splitting members including 1 total reflection plate and m-1 transmission reflection plate, the total reflection plate being farthest from the first light splitting assembly;
in the second light splitting assembly, the 1 st transparent reflecting plate is the transparent reflecting plate closest to the first light splitting assembly; the j-th transmission reflecting plate is used for reflecting 1 part of light intensity light beams in the first incident split light beams with the light intensity of m-j+1 parts to a second direction to form a second split light beam, and transmitting m-j parts of light intensity light beams in the first incident split light beams with the light intensity of m-j+1 parts to form the j-th first incident split light beams transmitted through the reflecting plate; m is a positive integer greater than 2, j is an integer value, and the second direction is a direction perpendicular to the light emitting surface of the light guide plate.
7. A projection display control method applied to the projection display system according to any one of claims 1 to 6, comprising:
respectively displaying a first image and a second image, so that the first image and the second image are respectively projected to the retina of a left eye and the retina of a right eye under the irradiation of each point light source output by a first light source end and each point light source output by a second light source end, and the focal lengths of a lens of the left eye and a lens of the right eye are adjusted to be matched with the virtual object distance of a common object in the first image and the second image;
and adjusting the opening size of each pixel area of the liquid crystal display panel according to the virtual object distance, and displaying each sub-target image of the target image in each pixel area, so that each sub-target image is projected to each sub-target projection image at the retina through a lens with a focal length matched with the virtual object distance to be in a spliced state.
8. The projection display control method of claim 7, wherein the first image comprises a left eye image and the second image comprises a right eye image.
9. The projection display control method according to claim 7, wherein displaying the first image and the second image, respectively, such that the first image and the second image are projected onto retinas of left and right eyes, respectively, under irradiation of the first light source end and the second light source end, respectively, comprises:
Displaying a plurality of first sub-images included in the first image in a first liquid crystal display area of the liquid crystal display panel, so that each first sub-projection image projected to the retina of the left eye under the irradiation of a first light source end is in an overlapped state or a split state;
and displaying a plurality of second sub-images included in the second image in a second liquid crystal display area of the liquid crystal display panel, so that each second sub-projection image projected to the retina of the right eye under the irradiation of a second light source end is in an overlapped state or a split state.
10. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the projection control method as claimed in any one of claims 7-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111069160.8A CN113759554B (en) | 2021-09-13 | 2021-09-13 | Projection display system, method and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111069160.8A CN113759554B (en) | 2021-09-13 | 2021-09-13 | Projection display system, method and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113759554A CN113759554A (en) | 2021-12-07 |
| CN113759554B true CN113759554B (en) | 2023-10-10 |
Family
ID=78795159
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111069160.8A Active CN113759554B (en) | 2021-09-13 | 2021-09-13 | Projection display system, method and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113759554B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105717640A (en) * | 2014-12-05 | 2016-06-29 | 北京蚁视科技有限公司 | Next-to-eye displayer based on microlens array |
| CN112904566A (en) * | 2021-03-22 | 2021-06-04 | 深圳珑璟光电科技有限公司 | Illumination optical system and optical display apparatus |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101414425B (en) * | 2007-10-16 | 2013-07-17 | 宋学锋 | Display device and display method |
| JP4725595B2 (en) * | 2008-04-24 | 2011-07-13 | ソニー株式会社 | Video processing apparatus, video processing method, program, and recording medium |
| CN107153270B (en) * | 2012-04-23 | 2019-11-22 | 北京蚁视昂维科技有限公司 | Small-hole-projectionear-eye near-eye display |
| IL252585A0 (en) * | 2017-05-29 | 2017-08-31 | Eyeway Vision Ltd | Eye projection system and method for focusing management |
| WO2021002641A1 (en) * | 2019-07-04 | 2021-01-07 | Samsung Electronics Co., Ltd. | Electronic device and method for displaying augmented reality |
-
2021
- 2021-09-13 CN CN202111069160.8A patent/CN113759554B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105717640A (en) * | 2014-12-05 | 2016-06-29 | 北京蚁视科技有限公司 | Next-to-eye displayer based on microlens array |
| CN112904566A (en) * | 2021-03-22 | 2021-06-04 | 深圳珑璟光电科技有限公司 | Illumination optical system and optical display apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113759554A (en) | 2021-12-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20220365361A1 (en) | Directional backlight | |
| CN104854864B (en) | Time multiplexing display with lateral operation pattern and longitudinal operator scheme | |
| CN107894666B (en) | Head-mounted multi-depth stereo image display system and display method | |
| CN107533166B (en) | Split pupil optical system for virtual and augmented reality and method for displaying image using the same | |
| US6124954A (en) | Projection screen based on reconfigurable holographic optics for implementation in head-mounted displays | |
| US8248458B2 (en) | Variable fixation viewing distance scanned light displays | |
| US5457574A (en) | Autostereoscopic display with high power efficiency | |
| US20200301239A1 (en) | Varifocal display with fixed-focus lens | |
| US20060033992A1 (en) | Advanced integrated scanning focal immersive visual display | |
| EP2866073A1 (en) | Three-dimensional focal field spectacles display | |
| US9049441B2 (en) | 3D image display apparatus and method | |
| JP2010503009A (en) | Autostereoscopic display device | |
| JP2010224129A (en) | Stereoscopic image display device | |
| CN108646412A (en) | Nearly eye display device and nearly eye display methods | |
| CN114527581B (en) | Display system and display control method thereof | |
| US6191876B1 (en) | Light diffusion control by electrically reconfigurable holographic optical elements | |
| CN103389612B (en) | The projection arrangement of various visual angles image is provided | |
| KR20140111553A (en) | Stereoscopic display apparatus, and display method thereof | |
| CN113759554B (en) | Projection display system, method and storage medium | |
| CN112859337A (en) | Super multi-viewpoint display system | |
| CN112748585B (en) | Small-distance visual area guiding type three-dimensional display system and method | |
| JP2002156599A (en) | Image observation apparatus and image observation system using the same | |
| KR102135888B1 (en) | Projection device | |
| JP2002228976A (en) | Stereoscopic image display device and stereoscopic image display method | |
| CN112433386A (en) | Compact optical structure for light field display |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |