CN113411565B - Control method for three-dimensional display of interactable body based on photo-phoresis capture - Google Patents

Control method for three-dimensional display of interactable body based on photo-phoresis capture Download PDF

Info

Publication number
CN113411565B
CN113411565B CN202110678338.2A CN202110678338A CN113411565B CN 113411565 B CN113411565 B CN 113411565B CN 202110678338 A CN202110678338 A CN 202110678338A CN 113411565 B CN113411565 B CN 113411565B
Authority
CN
China
Prior art keywords
particles
dimensional
capturing
controlling
display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110678338.2A
Other languages
Chinese (zh)
Other versions
CN113411565A (en
Inventor
范超
韩东成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui Easpeed Technology Co Ltd
Original Assignee
Anhui Easpeed Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui Easpeed Technology Co Ltd filed Critical Anhui Easpeed Technology Co Ltd
Priority to CN202110678338.2A priority Critical patent/CN113411565B/en
Publication of CN113411565A publication Critical patent/CN113411565A/en
Application granted granted Critical
Publication of CN113411565B publication Critical patent/CN113411565B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/363Image reproducers using image projection screens
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B26/00Optical devices or arrangements for the control of light using movable or deformable optical elements
    • G02B26/08Optical devices or arrangements for the control of light using movable or deformable optical elements for controlling the direction of light
    • G02B26/10Scanning systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30Image reproducers
    • H04N13/398Synchronisation thereof; Control thereof
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02BCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
    • Y02B20/00Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
    • Y02B20/40Control techniques providing energy savings, e.g. smart controller or presence detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Control Of Indicators Other Than Cathode Ray Tubes (AREA)

Abstract

The invention discloses a control method for three-dimensional display of an interactable body based on photophoresis capture, which comprises the following steps: s1, inputting an image to be displayed, and selecting a mode of capturing and scanning particles; s2, adjusting the power of a laser, controlling the light modulator 1 to generate corresponding light beams, and capturing particles by controlling the three-dimensional scanning system 1 and the particle container in the process that the light beams sequentially pass through the three-dimensional scanning system 1 and the particle container; s3, controlling the three-dimensional scanning system 1 to control the light beam carried particles to rapidly scan in the display area; controlling the RGB laser system, the light modulator 2 and the three-dimensional scanning system 2 to project an image to be displayed to a display area; and S4, identifying information such as the position, the action and the like of the human body in the image display process, controlling the particle control system and the projection system according to the identification result, realizing the optimal viewing angle and carrying out interaction.

Description

Control method for three-dimensional display of interactable body based on photo-phoresis capture
Technical Field
The invention relates to the technical fields of true three-dimensional display technology, laser beam modulation technology, optical capturing technology, projection technology and automatic control, in particular to a control method for three-dimensional display of interactable bodies based on photophoresis capturing.
Background
The world we live in is a three-dimensional world, and the traditional display device can only display graphics of a two-dimensional plane, and the two-dimensional display lacks depth of field relative to the three-dimensional display. With the rapid development of science and technology, people also put forward higher requirements on the display effect of the display. In recent years, various three-dimensional display technologies have been developed, and generally classified into the following: stereoscopic (Stereoscopic Display), autostereoscopic3 DDisplay), holographic (Holographic Di splay), and Volumetric3 DDisplay.
The stereoscopic technology requires wearing special glasses, and realizes a three-dimensional effect by utilizing binocular parallax. This allows the viewer's field of view to be limited and the images seen when viewed from different angles to be identical, without a true stereoscopic impression; the principle of the autostereoscopic three-dimensional display technique is the same as that of the stereoscopic technique, but no special glasses need to be worn. This display technique allows images to be viewed from only one particular spatial window, and therefore, a set of images can be projected into a series of windows where the viewer can receive different images from the two glasses when viewing, thereby creating a stereoscopic impression. The disadvantage of this display technique is the limited range of viewing, which is inconvenient for many people to view at the same time. Both methods essentially provide only a psychological depth of field and no physical depth of field. When the eyes observe the images, the eyes cannot observe the images like changing the focal length of the eyes when observing real objects, and the three-dimensional stereoscopic effect is synthesized through the brain.
Holographic display technology is the recording, analysis and reproduction of information based on the reconstruction of wave fronts. With the development of computer technology, computer-generated holographic technology is increasingly applied to holographic display technology. However, due to the limitation of the spatial light modulation device, the display quality and the viewing angle of the current computational holographic display technology are greatly limited.
The volumetric three-dimensional display technique refers to a technique for reproducing image information in a real three-dimensional space. The three-dimensional object displayed by the volume three-dimensional display technology has both psychological depth of field and physical depth of field. It is possible to realize that any plurality of observers directly observe the three-dimensional object from any angle without any auxiliary equipment. Existing volumetric three-dimensional display techniques in free space can be divided into the following: laser-induced plasma display, modified air display, acoustic levitation display, and the like. Laser-induced plasma display is currently unable to achieve color display and occlusion effects of three-dimensional objects in free space. The display effect of the improved air display and acoustic levitation display is too rough.
The basic unit in two-dimensional display is a pixel, and the basic unit corresponding thereto in three-dimensional display technology is a Voxel (Voxel). When the same image is displayed, the more voxels in the activated three-dimensional space are, the more the displayed three-dimensional image is, the more the details are, the higher the resolution is, and the better the display effect is. In two-dimensional displays, the display of an image is achieved by periodically, rapidly scanning the pixels of the display to activate the pixels. Also, in the volumetric three-dimensional display technique, some means is also required to scan voxels in the three-dimensional space, thereby achieving the display of a three-dimensional image.
In a laser-induced plasma display, laser light is focused through a lens to ionize air so as to be observed by human eyes. By controlling the focus of the laser, the rapid scanning is performed in the three-dimensional space, thereby realizing the display of the three-dimensional graph. This display technique requires a laser to have a high energy, and the color of light emitted by air ionization is related to the kind of gas ionized in the air, so that it is difficult to realize color display.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a control method for three-dimensional display of an interactable body based on photo-phoresis capture, so as to realize three-dimensional display and interaction in a free space and improve the three-dimensional display effect. The technical scheme of the invention is as follows:
the invention provides a control method for three-dimensional display of interactable body based on photo-phoresis capture, which comprises the following steps:
s1, inputting an image to be displayed, and selecting a mode of capturing particles and a mode of scanning, wherein the mode of capturing the particles is divided into four modes of single-point capturing, one-dimensional capturing, two-dimensional capturing and three-dimensional capturing; s2, after the particle capturing and scanning modes are determined, adjusting the power of a laser, controlling an optical modulator to generate corresponding light beams, and controlling the three-dimensional scanning system 1 and a particle container to capture particles in the process that the light beams sequentially pass through the three-dimensional scanning system 1 and the particle container;
s3, after particle capturing is completed, controlling the three-dimensional scanning system 1 to control the light beam to carry particles to rapidly scan in a display area; at the same time, the RGB laser system, the light modulator 2 and the three-dimensional scanning system 2 are controlled to project the image to be displayed into the display area;
and S4, in the process that the image is displayed in the display area, the information such as the position, the action and the like of the human body can be identified through the interaction system, and the particle control system and the projection system are controlled according to the identification result, so that the optimal viewing angle is realized and interaction is carried out.
Optionally, in the step S3, the image to be displayed is projected into the display area, where the projection manner includes point-by-point projection, layered projection, stereoscopic projection, or holographic projection.
Optionally, when the interactable three-dimensional display device further includes a display quality detection device, the control method further includes step S5: in the process of displaying the image in the display area, the real-time detection can be carried out through the display quality detection device to judge whether the display quality meets the requirement; when a reduction in display quality is detected or a requirement is not met, adjustments are made to the particle control system and projection system to optimize the quality of the display.
Optionally, a suitable system operating mode is selected according to the image to be displayed: the system comprises a particle arrangement mode, a particle scanning mode and an image projection mode, and is used for detecting the display quality of the system in real time by combining a display quality detection device, and when the display quality is detected to be reduced or the display quality is not satisfied, the operations of controlling the laser power, the scanning speed, re-capturing the particles and the like ensure that a light beam can stably carry a sufficient number of particles to scan in space.
Optionally, when the single-point capturing mode is used, the light beam captures the particles and then carries the particles to perform linear scanning in the display area; when a one-dimensional capturing mode is used, captured particles are arranged linearly, and a light beam carries the captured particles to scan in a display area; when a two-dimensional capturing mode is used, captured particles are arranged in a two-dimensional array, and the light beam carries the captured particles to scan in a display area; when the three-dimensional capturing method is used, captured particles are arranged in a three-dimensional array, three-dimensional projection can be directly performed, and the particles can be controlled to scan in a display area.
Optionally, the capturing and displaying steps when the single-point capturing mode is adopted include: selecting to carry out single-point capturing on the particles; then adjusting the power of the laser, and controlling the light modulator 1 to modulate the light beam so that a single light bottle structure can be generated at a focus, and capturing particles by controlling the three-dimensional scanning system and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system 1 and the particle container; after the capture of the particles is completed, controlling the three-dimensional scanning system 1 to control the light beam to carry the particles to carry out rapid scanning in the display area, and if a three-dimensional image needs to be displayed, scanning of the particles in three dimensions needs to be carried out in the display area; at the same time, the RGB laser system, the light modulator 2 and the three-dimensional scanning system 2 are controlled; when the particles are scanned to a certain position, the RGB laser system outputs three color light beams simultaneously, the color light beams needed to be displayed at the position are generated through an additive color method, and the projection system irradiates the color light beams needed to be displayed at the position of the particles to the particles.
Optionally, the capturing and displaying steps when the one-dimensional capturing mode is adopted include: selecting to carry out one-dimensional capture on the particles; then, the power of a laser is regulated, and the light modulator 1 is controlled to modulate the light beam, so that a one-dimensional linear arrangement light bottle structure can be generated at a focus, and the capture of particles is realized by controlling the three-dimensional scanning system 1 and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system 1 and the particle container; after the capture of the particles is completed, the three-dimensional scanning system 1 is controlled to control the light beam to carry the particles to carry out rapid scanning in the display area, and at the moment, if a three-dimensional image is required to be displayed, the particles only need to be scanned in 2 dimensions; at the same time, the RGB laser system, the light modulator 2 and the three-dimensional scanning system 2 are controlled, when the particles in linear arrangement are scanned to a certain position, the projection system divides the colors required to be displayed in the linear region where the particles are positioned into three frames, the three colors are respectively projected onto the particles by the single combined RGB laser system through the light modulator 2, or the RGB laser system consisting of the green, red and blue laser which are not combined is modulated by the three light modulators and then simultaneously projects the three colors onto the particles through the three-dimensional scanning system.
Optionally, the capturing and displaying steps when the two-dimensional capturing mode is adopted include: selecting two-dimensional capture of particles; then, the power of a laser is regulated, and the light modulator 1 is controlled to modulate the light beam, so that a two-dimensional plane-arrayed light bottle structure can be generated at a focus, and the capture of particles is realized by controlling the three-dimensional scanning system and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system 1 and the particle container; after the capture of the particles is finished, controlling the three-dimensional scanning system 1 to control the light beam to carry the particles to carry out rapid scanning in the display area, wherein at the moment, if only a two-dimensional image is required to be displayed, the projection system is directly utilized to carry out projection, and if the three-dimensional image is required to be displayed, the particles only need to be scanned in 1 dimension; at the same time, the RGB laser system, the light modulator 2 and the three-dimensional scanning system 2 are controlled, when the particles in two-dimensional arrangement are scanned to a certain position, the projection system divides the colors required to be displayed in the linear region where the particles are positioned into three frames, the three colors are respectively projected onto the particles by the single combined RGB laser system through the light modulator 2, or the RGB laser system consisting of the green, red and blue laser which are not combined is modulated by the three light modulators and then simultaneously projected onto the particles by the three-dimensional scanning system.
Optionally, the capturing and displaying steps when the three-dimensional capturing mode is adopted include: selecting to capture particles in three dimensions; then adjusting the power of a laser, and controlling the light modulator 1 to modulate the light beam so that the light beam can generate a three-dimensionally arranged light bottle structure at a focus, and capturing particles by controlling the three-dimensional scanning system 1 and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system 1 and the particle container; after the capture of the particles is finished, the three-dimensional scanning system 1 is controlled to control the light beam to carry the particles to carry out rapid scanning in a display area, at the moment, if a three-dimensional contour image is required to be displayed, the particles are directly controlled to be arranged according to the contours required to be displayed, then the projection system is used for projection, and if the three-dimensional contour image required to be displayed is overlarge, the images required to be displayed can be segmented and displayed in turn, and when the speed is high enough, a complete three-dimensional contour image can be observed;
the method comprises the steps of controlling an RGB laser system, a light modulator 2 and a three-dimensional scanning system 2 while scanning particles, dividing colors required to be displayed in a linear region where the particles are positioned into three frames by a projection system when the particles are scanned to a certain position, respectively projecting three colors onto the particles by a single combined RGB laser system through the light modulator 2 by utilizing three-dimensional holographic projection, or simultaneously projecting three colors onto the particles by utilizing the three-dimensional holographic projection after modulating the RGB laser system consisting of three non-combined green, red and blue lasers through the three light modulators by utilizing the three-dimensional holographic projection.
The beneficial effects of the invention are as follows: the invention captures particles by electrophoresis, so that the particles are continuously scanned in a display area, and a display image is projected to the position of the particles when the particles are scanned, and light projected onto the particles can be scattered, thereby being capable of being observed. The optical modulator 1 of the present invention can modulate the laser beam, and thereby can arbitrarily arrange captured particles; the captured particles can be arranged in one dimension, two dimensions or three dimensions by modulating the laser beam, so that parallel scanning is performed, and the resolution is obviously increased; the three-dimensional scanning system 1 can control the position of a light beam focus to realize rapid scanning in a three-dimensional space; the RGB laser system can realize projection at any position in space by combining the light modulator 2 and the three-dimensional scanning system 2; the upper computer can select a proper system working mode according to the image to be displayed, including an arrangement mode of particles, a scanning mode of particles, a projection mode of the image and the like, and can detect the display quality of the system in real time by combining a display quality detection device, and the optimal display quality is realized by adjusting the parameters of the system, updating the particles and the like; the laser adopted by the invention has lower power and lower cost compared with other three-dimensional display technologies. In addition, the interaction system can recognize information such as positions, gestures and actions of a human body, and transmits instructions to the upper computer according to recognition results, and the upper computer realizes the optimal viewing angle and performs interaction by controlling the system. In summary, compared with the existing free space three-dimensional display technology, the invention can realize true three-dimensional imaging with wide color gamut, high resolution and low speckle in free space, and the safety and the cost of the system are more advantageous.
Drawings
Figure 1 is a schematic diagram of the main structure and upper computer control principle of the present invention,
FIG. 2 is a schematic diagram of a control method of an interactable three-dimensional display based on a photo phoretic capture according to the invention.
The three-dimensional display device comprises a 1-input, a 2-upper computer, a 3-particle control system, a 4-projection system, a 5-laser, a 6-light modulator 1, a 7-three-dimensional scanning system 1, an 8-particle container, a 9-RGB laser system, a 10-light modulator 2, a 11-three-dimensional scanning system 2, a 12-interaction system, a 13-display quality detection device, a 14-display area, a 15-light absorbing material and a 16-transparent shell.
Detailed Description
Embodiments of the present invention will be described in detail below, by way of example with reference to the accompanying drawings.
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
As shown in fig. 1, the embodiment of the invention discloses an interactable body three-dimensional display device based on photo-phoresis capture, which comprises a particle control system (3), an upper computer (2), a projection system (4), an interaction system (12) and a display quality detection device (13); the particle control system (3) comprises a laser (5), a light modulator (1) (6), a three-dimensional scanning system (1) (7) and a particle container (8); the projection system (4) comprises an RGB laser system (9), a light modulator (10) and a three-dimensional scanning system 2 (11); the upper computer (2) is respectively connected with the laser (5), the light modulator (6), the three-dimensional scanning system 1 (7), the particle container (8), the RGB laser system (9), the light modulator 2 (10), the three-dimensional scanning system 2 (11), the interaction system (12) and the display quality detection device (13). The input (1) is a three-dimensional image or video file input into the upper computer by a user control computer.
The wavelength range of the laser (5) is less than 450nm or more than 650nm, the output power of the laser is controlled by the upper computer, and the output light beam irradiates the light modulator (1) (6).
The optical modulator 1 (6) comprises a lens and an optical modulation device, the lens is used for expanding and collimating light beams and modulating the diameters of the light beams, the optical modulation device is a spatial optical modulator (SLM), the SLM can be used for modulating laser beams emitted by the laser (5), and the modulated light beams can generate a single or three-dimensional light bottle structure which is arranged into a one-dimensional array, a two-dimensional array or a three-dimensional array after being focused by the lens and can be used for carrying out electrophoresis capture on particles.
The three-dimensional scanning system 1 (7) consists of a galvanometer and a lens system, wherein the galvanometer is used for controlling light beams to scan an XY plane, the lens system realizes scanning in the Z direction through zooming, and the combination of the galvanometer and the lens system can realize large-scale three-dimensional dynamic focusing scanning.
The particle container (8) is arranged on a transmission device, when particles need to be provided for a display area, the transmission device drives the particle container to move into the display area (14), and the three-dimensional scanning system 1 (7) controls light beams to carry out photo-phoretic capture on the particles. When the trapping process is completed, the actuator moves the particle container away from the display area (14). The particles in the particle container (8) are absorbent particles, the diameter of the absorbent particles is less than 500 microns, and the shape of the particles is not particularly required. Scattering phenomena occur when visible light impinges on the particles.
The RGB laser system comprises three lasers capable of respectively emitting red light, green light and blue light, wherein the three lasers can be used for combining beams through a dichroic mirror and outputting three colors of light at the same time, and a light beam with any color is obtained through an additive method and is then irradiated on particles; the three lasers can be placed side by side, light beams are output, light beams with three colors are irradiated on the particles at the same time after being modulated respectively, and the particles can scatter light with any color according to an additive color method.
The optical modulator 2 (10) comprises a lens for expanding, collimating and modulating the diameter of the light beam and an optical modulation device, which is a digital micromirror array (DMD); the light modulator 2 has two functions, one of which is to cooperate with a three-dimensional scanning system to irradiate the light beams with any color on the particles; secondly, the upper computer divides each frame of picture to be projected into three frames according to red, green and blue, when the laser emits laser light with corresponding color, the three frames of light beams are respectively modulated correspondingly through 3 digital micromirror arrays, and are projected to corresponding positions through the three-dimensional scanning system 2, and when the speed is fast enough, the projection of color images can be realized at the positions.
The three-dimensional scanning system 2 (11) consists of a galvanometer and a lens system, can realize large-scale three-dimensional dynamic focusing scanning like the three-dimensional scanning system 1, and can also project image information modulated by a digital micro-mirror array to any position in a display area.
The interactive system (12) is composed of a Kinect (motion sensing controller) and can detect the position and gesture of a human body, recognize various action instructions of fingers, transmit the instructions to the upper computer (2), control the particle control system (3) and the projection system (4) through the upper computer (2) to control the displayed images, provide the optimal display effect and give corresponding feedback to the instructions. The method comprises the following steps: the upper computer (2) controls the particle control system (3) and the projection system (4) according to the position information of the human body, and adjusts the displayed graph to realize the optimal viewing angle; and the upper computer (2) controls the particle control system (3) and the projection system (4) according to the action information of the human body, and adjusts the displayed graph to realize interaction.
The display quality detection device (13) consists of an image acquisition device and image analysis software, and in the working process of the display system, the display quality detection device detects the display quality in real time, and when the display quality is detected to be incapable of meeting the requirement, a signal is sent to the upper computer (2), and the upper computer (2) optimizes system parameters and recaptures particles. The method comprises the following steps: the upper computer (2) updates captured particles by adjusting the power of the laser (5), the modulation mode of the optical modulator (1) (6), the scanning mode and scanning speed of the three-dimensional scanning system (1) (7) and the captured particles according to the instructions, adjusts the power of the RGB laser system (9), changes the projection mode of the projection system and the like, and optimizes the display quality.
The display area (14) is free space.
The light absorbing material (15) is arranged in the vicinity of the display area (14) for absorbing stray light influencing the display effect.
The transparent housing (16) is made of transparent material and is used for protecting the display area (14) from being influenced by air flow. When the airflow environment of the display area is too complex, the display effect can be affected, and the transparent shell can be used for protecting the display area, so that the display effect is ensured. The transparent case may not be used when the air flow environment of the display area is gentle.
Example 1: single point capture
The embodiment provides a control method of an interactable body three-dimensional display device based on photophoresis capture, which comprises the following steps: the image to be displayed is a three-dimensional image (such as a relatively simple stereoscopic figure portrait), and a user selects to perform single-point capturing on the particles through an upper computer; and then a user adjusts the power of the laser through the upper computer and controls the light modulator 1 to modulate the light beam, so that a single light bottle structure can be generated at a focus, and the capture of particles is realized by controlling the three-dimensional scanning system 1 and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system 1 and the particle container. After the capture of the particles is completed, a user controls the three-dimensional scanning system 1 through the upper computer to control the light beam to carry the particles to carry out rapid scanning in the display area, and at the moment, if a three-dimensional image needs to be displayed, the particles need to be scanned in three dimensions in the display area; at the same time, the RGB laser system, the light modulator 2 and the three-dimensional scanning system 2 are controlled by the upper computer, when the particles are scanned to a certain position, the RGB laser system outputs three color light beams simultaneously, the light beams of the colors required to be displayed at the position are generated by an additive color method, the projection system irradiates the light beams of the colors required to be displayed at the position of the particles on the particles, and the light beams irradiated on the particles are scattered, so that the colors required to be displayed can be observed at the point. When the scanning speed is sufficiently fast, a three-dimensional image can be observed in the display area according to the visual residual effect.
In the process of image display, a user can also recognize the information of the position, the action and the like of a human body through the interaction system, and transmit an instruction to the upper computer according to the recognition result, and the upper computer adjusts the scanning position of particles by controlling the light modulator and the three-dimensional scanning system in the particle control system and the projection system according to the position information of the human body, so that the angle of image display is changed, and the optimal watching effect is realized; and the upper computer controls the particle control system, the light modulator and the three-dimensional scanning system in the projection system according to the motion information of the human body, and adjusts the displayed graph to realize interaction.
The user can also detect in real time through the display quality detection device to judge whether the display quality meets the requirement. When the display quality is detected to be reduced or the display quality is not satisfied in the working process of the display system, the operations of controlling the laser power, the scanning speed, re-capturing particles and the like through the upper computer ensure that the light beam can stably carry a sufficient number of particles to scan in space, thereby optimizing the display quality.
Example 2: one-dimensional capture
The embodiment provides a control method of an interactable body three-dimensional display device based on photophoresis capture, which comprises the following steps: the image to be displayed is a three-dimensional image (such as a relatively complex stereoscopic figure portrait), and a user selects particles to capture in one dimension through an upper computer; and then a user adjusts the power of the laser through the upper computer and controls the light modulator 1 to modulate the light beam, so that the light beam can generate a one-dimensional light bottle structure which is linearly arranged at a focus, and the capture of particles is realized by controlling the three-dimensional scanning system 1 and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system 1 and the particle container. After the capture of the particles is completed, a user controls the three-dimensional scanning system 1 through the upper computer to control the light beam to carry the particles to carry out rapid scanning in the display area, and at the moment, if a three-dimensional image is required to be displayed, the particles only need to be scanned in 2 dimensions; meanwhile, the RGB laser system, the light modulator 2 and the three-dimensional scanning system 2 are controlled by the upper computer, when the particles in linear arrangement are scanned to a certain position, the projection system divides the colors required to be displayed in the linear region where the particles are positioned into three frames, the three colors are respectively projected onto the particles by the single combined RGB laser system through the light modulator 2, or the three colors are simultaneously projected onto the particles by the three-dimensional scanning system after being modulated by the three light modulators by the RGB laser system formed by the three green, red and blue laser devices which are not combined. The light beam impinging on the particles will scatter and at this point the desired color of the display can be observed. When the scanning speed is sufficiently fast, a three-dimensional image can be observed in the display area.
In the process of image display, a user can also recognize the information of the position, the action and the like of a human body through the interaction system, and transmit an instruction to the upper computer according to the recognition result, and the upper computer adjusts the scanning position of particles by controlling the light modulator and the three-dimensional scanning system in the particle control system and the projection system according to the position information of the human body, so that the angle of image display is changed, and the optimal watching effect is realized; and the upper computer controls the particle control system, the light modulator and the three-dimensional scanning system in the projection system according to the motion information of the human body, and adjusts the displayed graph to realize interaction.
The user can also detect in real time through the display quality detection device to judge whether the display quality meets the requirement. When the display quality is detected to be reduced or the display quality is not satisfied in the working process of the display system, the operations of controlling the laser power, the scanning speed, re-capturing particles and the like through the upper computer ensure that the light beam can stably carry a sufficient number of particles to scan in space, thereby optimizing the display quality.
Example 3: two-dimensional capture
The embodiment provides a control method of an interactable body three-dimensional display device based on photophoresis capture, which comprises the following steps: the image to be displayed is a three-dimensional image (such as a relatively complex stereoscopic figure portrait), and a user selects to capture particles in two dimensions through an upper computer; and then a user adjusts the power of the laser through the upper computer and controls the light modulator 1 to modulate the light beam, so that the light beam can generate a two-dimensional plane arranged light bottle structure at a focus, and the capture of particles is realized by controlling the three-dimensional scanning system 1 and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system 1 and the particle container. After the capture of the particles is finished, a user controls the three-dimensional scanning system 1 to control the light beam to carry the particles to carry out rapid scanning in the display area through the upper computer, at the moment, if only a two-dimensional image is required to be displayed, the projection system is directly used for projection, and if the three-dimensional image is required to be displayed, the particles only need to be scanned in 1 dimension; meanwhile, the RGB laser system, the light modulator 2 and the three-dimensional scanning system 2 are controlled by the upper computer, when two-dimensionally arranged particles are scanned to a certain position, the projection system divides the colors required to be displayed in the linear region where the particles are positioned into three frames, the three colors are respectively projected onto the particles by the single combined RGB laser system through the light modulator 2, or the three colors are simultaneously projected onto the particles by the three-dimensional scanning system after being modulated by the three light modulators by the RGB laser system formed by the three green, red and blue laser devices which are not combined. The light beam impinging on the particles will scatter and at this point the desired color of the display can be observed. When the speed of scanning is sufficiently fast, the displayed image can be observed in the display area.
In the process of image display, a user can also recognize the information of the position, the action and the like of a human body through the interaction system, and transmit an instruction to the upper computer according to the recognition result, and the upper computer adjusts the scanning position of particles by controlling the light modulator and the three-dimensional scanning system in the particle control system and the projection system according to the position information of the human body, so that the angle of image display is changed, and the optimal watching effect is realized; and the upper computer controls the particle control system, the light modulator and the three-dimensional scanning system in the projection system according to the motion information of the human body, and adjusts the displayed graph to realize interaction.
The user can also detect in real time through the display quality detection device to judge whether the display quality meets the requirement. When the display quality is detected to be reduced or the display quality is not satisfied in the working process of the display system, the operations of controlling the laser power, the scanning speed, re-capturing particles and the like through the upper computer ensure that the light beam can stably carry a sufficient number of particles to scan in space, thereby optimizing the display quality.
Example 4: three-dimensional capture
The embodiment provides a control method of an interactable body three-dimensional display device based on photophoresis capture, which comprises the following steps: the image to be displayed is a three-dimensional image (such as a complex stereoscopic figure portrait), and a user selects to capture particles in three dimensions through an upper computer; and then a user adjusts the power of the laser through the upper computer and controls the light modulator 1 to modulate the light beam, so that the light beam can generate a three-dimensional arranged light bottle structure at a focus, and the capture of particles is realized by controlling the three-dimensional scanning system 1 and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system 1 and the particle container. After the capture of the particles is completed, a user controls the three-dimensional scanning system 1 to control the light beam to carry the particles to rapidly scan in the display area through the upper computer, at the moment, if a three-dimensional contour image is required to be displayed, the particles are directly controlled to be arranged according to the contour required to be displayed, then the projection system is utilized to project the particles, and if the three-dimensional contour image required to be displayed is too large, the images required to be displayed can be segmented and alternately displayed, and when the speed is high enough, a complete three-dimensional contour image can be observed. When particles are scanned, an RGB laser system, a light modulator 2 and a three-dimensional scanning system 2 are controlled by an upper computer, when the particles in three-dimensional arrangement are scanned to a certain position, a projection system divides colors required to be displayed in a linear region where the particles are positioned into three frames, three colors are respectively projected onto the particles by a single combined RGB laser system through the light modulator 2 by utilizing three-dimensional holographic projection, or three colors are simultaneously projected onto the particles by utilizing the three-dimensional holographic projection after being modulated by utilizing three light modulators by utilizing an RGB laser system formed by three green, red and blue laser devices which are not combined. The light beam impinging on the particles will scatter and at this point the desired color of the display can be observed. When the speed of scanning is sufficiently fast, the displayed image can be observed in the display area.
In the process of image display, a user can also recognize the information of the position, the action and the like of a human body through the interaction system, and transmit an instruction to the upper computer according to the recognition result, and the upper computer adjusts the scanning position of particles by controlling the light modulator and the three-dimensional scanning system in the particle control system and the projection system according to the position information of the human body, so that the angle of image display is changed, and the optimal watching effect is realized; and the upper computer controls the particle control system, the light modulator and the three-dimensional scanning system in the projection system according to the motion information of the human body, and adjusts the displayed graph to realize interaction.
The user can also detect in real time through the display quality detection device to judge whether the display quality meets the requirement. When the display quality is detected to be reduced or the display quality is not satisfied in the working process of the display system, the operations of controlling the laser power, the scanning speed, re-capturing particles and the like through the upper computer ensure that the light beam can stably carry a sufficient number of particles to scan in space, thereby optimizing the display quality.
In summary, when the image to be displayed is relatively simple and the required resolution is low, a single-point capturing mode can be selected, but if the image is relatively complex and the required resolution is high, the frame rate of the display is greatly reduced due to the limitation of the movement speed of the particles, and the viewing effect is affected, so that other modes such as one-dimensional, two-dimensional and three-dimensional capturing are required to be selected, and the frame rate of the display can be increased. Therefore, the interactive three-dimensional display device and the control method thereof provided by the embodiment of the invention not only can realize 360-degree full-view naked eye three-dimensional image display in free space and interaction, but also can greatly improve the three-dimensional display effect.
The foregoing embodiments are merely illustrative of the preferred embodiments of the present invention and are not intended to limit the scope of the invention, and various modifications and improvements made by those skilled in the art to which the invention pertains without departing from the invention as defined in the appended claims.
While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.

Claims (8)

1. A control method for three-dimensional display of interactable body based on photo-phoresis capture is characterized by comprising the following steps:
s1, inputting an image to be displayed, and selecting a mode of capturing particles and a mode of scanning, wherein the mode of capturing the particles is divided into four modes of single-point capturing, one-dimensional capturing, two-dimensional capturing and three-dimensional capturing;
when the single-point capturing mode is used, the light beam captures particles and then carries the particles to carry out linear scanning in a display area;
when a one-dimensional capturing mode is used, captured particles are arranged linearly, and a light beam carries the captured particles to scan in a display area;
when a two-dimensional capturing mode is used, captured particles are arranged in a two-dimensional array, and the light beam carries the captured particles to scan in a display area;
when the three-dimensional capturing mode is used, captured particles are arranged in a three-dimensional array, and three-dimensional projection is directly carried out, or the particles are controlled to scan in a display area;
s2, after the particle capturing and scanning modes are determined, adjusting the power of a laser, controlling an optical modulator to generate corresponding light beams, and controlling the three-dimensional scanning system and the particle container to capture particles in the process that the light beams sequentially pass through the three-dimensional scanning system and the particle container;
s3, after particle capturing is completed, controlling a three-dimensional scanning system to control a beam to carry particles to rapidly scan in a display area; simultaneously, controlling an RGB laser system, a light modulator and a three-dimensional scanning system to project an image to be displayed into a display area;
and S4, identifying the position and action information of the human body through the interaction system in the process that the image is displayed in the display area, and controlling the particle control system and the projection system according to the identification result to realize the optimal viewing angle and perform interaction.
2. The method according to claim 1, wherein the step S3 is performed by projecting the image to be displayed into the display area, and the projection method includes point-by-point projection, layered projection, stereoscopic projection, or holographic projection.
3. A control method for three-dimensional display of interactable based on photo-phoresis capturing according to claim 1 wherein when said interactable three-dimensional display device further comprises a display quality detection means, said control method further comprises step S5: in the process of displaying the image in the display area, the real-time detection is also carried out through the display quality detection device, and whether the display quality meets the requirement is judged; when a reduction in display quality is detected or a requirement is not met, adjustments are made to the particle control system and projection system to optimize the quality of the display.
4. A control method for three-dimensional display of interactable bodies based on photophoresis capturing according to claim 3, characterized in that the appropriate system operation mode is selected according to the image to be displayed: the system comprises a particle arrangement mode, a particle scanning mode and an image projection mode, and is used for detecting the display quality of the system in real time by combining a display quality detection device, and when the display quality is detected to be reduced or the display quality is not satisfied, the operation of controlling the laser power, the scanning speed and the operation of recapturing the particles ensures that the light beam can stably carry a sufficient number of particles to scan in space.
5. The method for controlling three-dimensional display of interactable body based on photo-phoresis capturing according to claim 1 wherein the capturing and displaying steps when the single-point capturing mode is adopted comprise:
selecting to carry out single-point capturing on the particles;
then adjusting the power of the laser, and controlling the light modulator to modulate the light beam so that the light beam can generate a single light bottle structure at a focus, and capturing particles by controlling the three-dimensional scanning system and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system and the particle container;
after the capture of the particles is completed, controlling a three-dimensional scanning system to control the light beam to carry the particles to carry out rapid scanning in a display area, and if a three-dimensional image needs to be displayed, scanning of the particles in three dimensions needs to be carried out in the display area; simultaneously, controlling an RGB laser system, a light modulator and a three-dimensional scanning system;
when the particles are scanned to a certain position, the RGB laser system outputs three color light beams simultaneously, the color light beams needed to be displayed at the position are generated through an additive color method, and the projection system irradiates the color light beams needed to be displayed at the position of the particles to the particles.
6. The method for controlling three-dimensional display of interactable body based on photo-phoresis capturing according to claim 1, wherein the capturing and displaying steps when adopting a one-dimensional capturing mode comprise:
selecting to carry out one-dimensional capture on the particles;
then adjusting the power of the laser, and controlling the light modulator to modulate the light beam so that the light beam can generate a one-dimensional light bottle structure which is linearly arranged at a focus, and capturing particles by controlling the three-dimensional scanning system and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system and the particle container;
after the capture of the particles is completed, controlling a three-dimensional scanning system to control the light beam to carry the particles to carry out rapid scanning in a display area, and at the moment, if a three-dimensional image is required to be displayed, only 2-dimensional scanning of the particles is required;
at the same time, the RGB laser system, the light modulator and the three-dimensional scanning system are controlled, when the particles in linear arrangement are scanned to a certain position, the projection system divides the colors required to be displayed in the linear region where the particles are positioned into three frames, the three colors are respectively projected onto the particles by the single combined RGB laser system through the light modulator, or the RGB laser system consisting of the green, red and blue lasers which are not combined is modulated by the three light modulators and then simultaneously projects the three colors onto the particles through the three-dimensional scanning system.
7. The method for controlling three-dimensional display of interactable body based on photo-phoresis capturing according to claim 1 wherein the capturing and displaying steps when two-dimensional capturing mode is adopted comprise:
selecting two-dimensional capture of particles;
then adjusting the power of the laser, and controlling the light modulator to modulate the light beam so that the light beam can generate a two-dimensional plane arranged light bottle structure at a focus, and capturing particles by controlling the three-dimensional scanning system and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system and the particle container;
after the capture of the particles is finished, controlling a three-dimensional scanning system to control a beam to carry the particles to carry out rapid scanning in a display area, wherein at the moment, if only a two-dimensional image is required to be displayed, the projection system is directly utilized to project, and if the three-dimensional image is required to be displayed, the particles only need to be scanned in 1 dimension;
simultaneously, an RGB laser system, a light modulator and a three-dimensional scanning system are controlled, when particles in two-dimensional arrangement are scanned to a certain position, the projection system divides colors required to be displayed in a linear region where the particles are positioned into three frames, the three colors are respectively projected onto the particles by the RGB laser system of a single combined beam through the light modulator, or the RGB laser system formed by three green, red and blue lasers without combined beams is modulated by the three light modulators and then simultaneously projected onto the particles by the three-dimensional scanning system.
8. The method for controlling three-dimensional display of interactable body based on photo-phoresis capturing according to claim 1 wherein the capturing and displaying steps when adopting the three-dimensional capturing mode comprise:
selecting to capture particles in three dimensions;
then adjusting the power of the laser, and controlling the light modulator to modulate the light beam so that the light beam can generate a three-dimensionally arranged light bottle structure at a focus, and capturing particles by controlling the three-dimensional scanning system and the particle container in the process that the light beam sequentially passes through the three-dimensional scanning system and the particle container;
after the capture of the particles is finished, controlling a three-dimensional scanning system to control a beam to carry the particles to carry out rapid scanning in a display area, directly controlling the particles to be arranged according to the contours required to be displayed if a three-dimensional contour image is required to be displayed, then projecting by a projection system, and dividing the images required to be displayed if the three-dimensional contour image required to be displayed is too large, and alternately displaying;
when particles are scanned, an RGB laser system, a light modulator and a three-dimensional scanning system are controlled, when the particles in three-dimensional arrangement are scanned to a certain position, the projection system divides the colors required to be displayed in the linear region where the particles are positioned into three frames, the three colors are respectively projected onto the particles by a single combined RGB laser system through the light modulator by utilizing three-dimensional holographic projection, or the three colors are simultaneously projected onto the particles by utilizing the three-dimensional holographic projection after the RGB laser system consisting of three non-combined green, red and blue lasers is modulated by utilizing the three light modulators.
CN202110678338.2A 2018-08-19 2018-08-19 Control method for three-dimensional display of interactable body based on photo-phoresis capture Active CN113411565B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110678338.2A CN113411565B (en) 2018-08-19 2018-08-19 Control method for three-dimensional display of interactable body based on photo-phoresis capture

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810944347.XA CN110891169B (en) 2018-08-19 2018-08-19 Interactive three-dimensional display device based on photophoretic capture and control method thereof
CN202110678338.2A CN113411565B (en) 2018-08-19 2018-08-19 Control method for three-dimensional display of interactable body based on photo-phoresis capture

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201810944347.XA Division CN110891169B (en) 2018-08-19 2018-08-19 Interactive three-dimensional display device based on photophoretic capture and control method thereof

Publications (2)

Publication Number Publication Date
CN113411565A CN113411565A (en) 2021-09-17
CN113411565B true CN113411565B (en) 2023-07-18

Family

ID=69591844

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201810944347.XA Active CN110891169B (en) 2018-08-19 2018-08-19 Interactive three-dimensional display device based on photophoretic capture and control method thereof
CN202110678338.2A Active CN113411565B (en) 2018-08-19 2018-08-19 Control method for three-dimensional display of interactable body based on photo-phoresis capture

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201810944347.XA Active CN110891169B (en) 2018-08-19 2018-08-19 Interactive three-dimensional display device based on photophoretic capture and control method thereof

Country Status (2)

Country Link
CN (2) CN110891169B (en)
WO (1) WO2020037693A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114690438A (en) * 2020-12-31 2022-07-01 上海誉沛光电科技有限公司 Apparatus and method for floating display

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN201163802Y (en) * 2007-12-11 2008-12-10 上海高意激光技术有限公司 Novel colorful laser projection display device
CN106500628A (en) * 2016-10-19 2017-03-15 杭州思看科技有限公司 A kind of 3-D scanning method containing multiple different wave length laser instrument and scanner
CN107479189A (en) * 2017-08-14 2017-12-15 中国科学院西安光学精密机械研究所 Non-paraxial self-acceleration light beam generation method and device based on angular spectrum regulation

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6611380B2 (en) * 2001-12-21 2003-08-26 Eastman Kodak Company System and method for calibration of display system with linear array modulator
US6977732B2 (en) * 2002-12-26 2005-12-20 National Taiwan University Miniature three-dimensional contour scanner
CN101329451A (en) * 2008-07-24 2008-12-24 南京大学 Three-dimensional space laser imaging apparatus
US8864313B2 (en) * 2009-06-15 2014-10-21 Eastman Kodak Company Dynamic illumination control for laser projection display
CN101609210B (en) * 2009-07-14 2012-01-25 王一诺 Method for realizing display of image and character in three-dimensional space by laser scanning
CN102589476B (en) * 2012-02-13 2014-04-02 天津大学 High-speed scanning and overall imaging three-dimensional (3D) measurement method
CN103905808A (en) * 2012-12-27 2014-07-02 北京三星通信技术研究有限公司 Device and method used for three-dimension display and interaction.
CN104427325B (en) * 2013-09-04 2018-04-27 北京三星通信技术研究有限公司 Fast integration image generating method and the naked eye three-dimensional display system with user mutual
US10129517B2 (en) * 2014-12-05 2018-11-13 Brigham Young University Full-color freespace volumetric display with occlusion
US10228653B2 (en) * 2016-04-07 2019-03-12 Pixie Dust Technologies, Inc. System and method for rendering interactive aerial volumetric graphics and generating spatial audio using femtosecond lasers
US10554961B2 (en) * 2016-11-08 2020-02-04 Kevin Vora Three-dimensional volumetric display using photoluminescent materials
CN106707484B (en) * 2016-12-16 2019-06-28 上海理工大学 Super-resolution optical micro imaging method based on the illumination of Particle Scattering light near field
CN107277494A (en) * 2017-08-11 2017-10-20 北京铂石空间科技有限公司 three-dimensional display system and method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN201163802Y (en) * 2007-12-11 2008-12-10 上海高意激光技术有限公司 Novel colorful laser projection display device
CN106500628A (en) * 2016-10-19 2017-03-15 杭州思看科技有限公司 A kind of 3-D scanning method containing multiple different wave length laser instrument and scanner
CN107479189A (en) * 2017-08-14 2017-12-15 中国科学院西安光学精密机械研究所 Non-paraxial self-acceleration light beam generation method and device based on angular spectrum regulation

Also Published As

Publication number Publication date
CN110891169B (en) 2021-07-09
CN110891169A (en) 2020-03-17
WO2020037693A1 (en) 2020-02-27
CN113411565A (en) 2021-09-17

Similar Documents

Publication Publication Date Title
CN108139518B (en) Virtual/augmented reality system with reverse angle diffraction grating
KR101670927B1 (en) Display apparatus and method
EP1126728B1 (en) Method of computer generating autostereoscopic images
US8760499B2 (en) Three-dimensional imager and projection device
US9148658B2 (en) Light-based caustic surface calibration
US9438886B2 (en) Parallax scanning methods for stereoscopic three-dimensional imaging
EP1209508A2 (en) Display for 3D images
JP2005515487A (en) 3D image projection using retroreflective screen
JP6921973B2 (en) Microscope device for capturing and displaying 3D images of samples
KR20190043999A (en) Three dimensional glasses free light field display using eye location
JP2008146221A (en) Image display system
KR20160043786A (en) Digital holographic table-top display apparatus and method thereof
JP2000047138A (en) Image display device
JP2007316461A (en) Projector and image projection method
CN106773469A (en) Scene reproduces line holographic projections display system
CN113411565B (en) Control method for three-dimensional display of interactable body based on photo-phoresis capture
GB2570903A (en) Projection array light field display
Rawson Vibrating varifocal mirrors for 3-D imaging
CN107577113A (en) Laser projection device, system and method
JP3403048B2 (en) Three-dimensional image reproducing device and three-dimensional subject information input device
US20210218946A1 (en) Image display device and image display method
CN110865467B (en) Interactive three-dimensional display device based on laser induction and control method thereof
CN106772821B (en) Interactive naked eye 3D system
KR101741227B1 (en) Auto stereoscopic image display device
KR101567002B1 (en) Computer graphics based stereo floting integral imaging creation system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant