JP6935094B2 - Methods and systems for generating interactive aerial volmetric images and spatial audio using femtosecond lasers - Google Patents

Description

The present invention generally relates to an interactive vormetric audiovisual display. More specifically, the present invention relates to methods and systems for producing high-resolution, safe, and interactive 3D images and aerial audio using high-intensity ultrashort pulsed laser light sources.

Three-dimensional (3D) displays have received a lot of attention over the last five years. The 3D virtual object is originally displayed on a head-mounted display. Since then, continuous efforts have been made to explore 3D displays with planes, and several techniques have been developed to provide binocular stereoscopic vision. The 3D display employs glasses that realize binocular stereoscopic vision based on, for example, the following techniques (anaglyph, time division, and polarized light). On the other hand, those that do not rely on 3D display glasses are based on, for example, the following techniques (parallax barrier and lenticular lens array). While these techniques can provide effective 3D images, they require accurate image calculations and generation for multiple viewpoints and limit the viewing angle of the user.

Another approach to achieving advanced 3D displays is to use physical 3D space to render images instead of planes. This approach forms a visual representation of an object in three physical dimensions, rather than a traditional screen planar image that uses a variety of visual effects to simulate depth. These 3D displays are called vormetric displays and can show the user a display image from any angle. The volmetric display displays "voxels" arranged in three-dimensional space.

Lasers can be used to place light spots (“voxels”) of various media at any point in three-dimensional space. Some of the advantages of the light spots placed by the laser are (1) no special material need to be placed and no need to suppress the emission of light from the medium, and (2) obstruction of the line of sight. Wireless communication that can avoid the resulting structure, and (3) precise control of the laser with advances in optical technology. In an aerial laser-based volmetric display, aerial voxels, or plasmas, are generated by a high-intensity laser, which is achieved by shortening the pulse duration within the limits of total power.

When the rendering speed was only 100 dot / sec, the first demonstration of a volmetric display was performed using a nanosecond laser (Hidei Kimura, Taro Uchiyama, and Hiroyuki Yoshikawa. 2006. Laser produced 3D display in the air. In ACM SIGGRAPH 2006 Emerging Technologies (SIGGRAPH'06). ACM, New York, NY, Article 20. DOI: http://dx.doi.org/10.1145/1179133.1179154 ). Later, a femtosecond laser achieved 1000 dots / sec ( Hideo Saito, Hidei Kimura, Satoru Shimada, Takeshi Naemura, Jun Kayahara, Songkran Jarusirisawad, Vincent Nozick, Hiroyo Ishikawa, Toshiyuki Murakami, Jun Aoki, Akira Asano, Tatsumi Kimura, Masayuki Kakehata, Fumio Sasaki, Hidehiko Yashiro, Masahiko Mori, Kenji Torizuka, and Kouta Ino. 2008. Laser-plasma scanning 3D display for putting digital contents in free space. In Proceedings of SPIE 6803 (2008), 680309-680309-10. DOI: http://dx.doi.org/10.1117/12.768068 ). However, the resolution of these systems is low.

Demonstrations of laser-based volmetric displays on media other than air were also conducted. When rendering speeds of 50,000 dot / sec were achieved, nanosecond laser-based volmetric displays were developed underwater (Hidei Kimura, Akira Asano, Issei Fujishiro, Ayaka Nakatani, and Hayato Watanabe. 2011. True 3D display. In ACM SIGGRAPH 2011 Emerging Technologies (SIGGRAPH'11). ACM, New York, NY, Article 20, 1 page. DOI: http://dx.doi.org/10.1145/2048259.2048279 ). Later, a femtosecond laser volmetric display in a fluorescent medium with parallel optical access using computer-generated holograms was developed to achieve higher resolution ( Satoshi Hasegawa and Yoshio Hayasaki. 2013. Liquid volumetric display with parallel optical). access by computer-generated locally. In Digital Holography and Three-Dimensional Imaging (2013), DTh2A.7. DOI: http://dx.doi.org/ 10.1364/DH.2013.DTh2A.7 ) In these systems, You cannot access the derived light spot.

Aerial volmetric displays usually involve user interaction. Incorporating aerial tactile sensations into an aerial volmetric display has the advantage of providing a touch of interaction between virtual objects and the user. Aerial tactile sensation has the advantage of being able to apply force from a distance without physical contact or wearable devices, and has high programmability. That is, since no physical actuator is required, it can be rearranged at an arbitrary position in the three-dimensional space. For example, in a recent study, Takayuki Hoshi, Masafumi Takahashi, Takayuki Iwamoto, and Hiroyuki Shinoda. 2010. Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Transactions on Haptics 3, 3, 155-165 . DOI: http://dx.doi.org/10.1109/TOH.2010.4 , Tom Carter, Sue Ann Seah, Benjamin Long, Bruce Drinkwater, and Sriram Subramanian. 2013. Ultrahaptics: Multi-point mid-air haptic feedback for touch surfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, ACM, New York, NY, USA, UIST '13, 505-514. DOI: http://dx.doi.org/10.1145/2501988.2502018 ; Seki Inoue, Koseki J. Kobayashi-Kirschvink, Yasuaki Monnai, Keisuke Hasegawa, Yasutoshi Makino, Hiroyuki Shinoda. 2014. HORN: The hapt-optic reconstruction. In ACM SIGGRAPH 2014 Emerging Technologies, ACM, New York, NY, USA, SIGGRAPH' 14, 11: 1. DOI: http://dx.doi.org/10.1145/2614066.2614092 ) or air vortices for non-contact tactile stimuli and feedback ( Rajinder Sodhi, Ivan Poupyrev, Matthew Glisson, and Ali Israr. 2013 .Aireal: Interactive tactile experiences in free air. ACM Trans. Graph. 32, 4 (July), 134: 1-134: 10. DOI: http://dx.doi.org/10.1145/2461912.2462007 ; Sidhant Gupta, Dan Morris, Shwetak N. Patel, and Desney Tan. 2013. AirWave: Non-contact haptic feedback using air vortex rings. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, ACM, New York, NY, USA, UbiComp '13, 419-428. DOI: http://dx.doi.org/10.1145/2493432.2493463 ) is under consideration. These approaches lack spatial accuracy and have limited operating distance. Recent studies have also considered the use of low-power nanosecond lasers to evoke tactile sensations ( Jae-Hoon Jun, Jong-Rak Park, Sung-Phil Kim, Young Min Bae, Jang-Yeon Park, Hyung). -Sik Kim, Seungmoon Choi, Sung Jun Jung, Seung Hwa Park, Dong-Il Yeom, Gu-In Jung, Ji-Sun Kim and SoonCheol Chung. 2015. Laser-induced thermoelastic effects can evoke tactile sensations. Scientific Reports 5, 11016 . DOI: http://dx.doi.org/10.1038/srep11016 , Hojin Lee, Ji-Sun Kim, Seungmoon Choi, Jae-Hoon Jun, Jong-Rak Park, A-Hee Kim, Han-Byeol Oh, Hyung- Sik Kim, and Soon-Cheol Chung. 2015. Midair tactile stimulation using laser-induced thermoelastic effects: The first study for indirect radiation. In World Haptics Conference (WHC), 2015, 374-380. DOI: http: // dx. doi.org/10.1109/WHC.2015.7177741 ). However, even low-power nanosecond lasers have been shown to instantly raise the temperature of human skin.

Aerial volmetric displays usually come with a sound system. Previous studies on the control of spatial acoustic distribution in free space include multi-channel audio synthesis and ultrasound-based super-directional speakers (parametric speakers) as means for generating 3D sound. Traditional surround sound speakers simulate an immersive sound environment and generate spatial patterns through the interference of audible sound waves ( K. Shinagawa, Y. Amemiya, H. Takemura, S. Kagami, and H). Mizoguchi. 2007. Three dimensional simulation and measurement of sound pressure distribution generated by 120 ch plane loudspeaker array. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics, 278-283. ). However, the quality of the auditory experience depends on the listener's position with respect to the environment of the surround sound speakers, and the auditory experience is optimized for the listener in the center of the target area. Parametric speakers can generate a narrow beam of audible sound using an ultrasonic transducer array, so that a specific person targeted by the device can hear the sound ( M. Yoneyama, J. Fujimoto). , Y. Kawamo, and S. Sasabe. 1983. The audio spotlight: An application of nonlinear interaction of sound waves to a new type of loudspeaker design. The Journal of the Acoustical Society of America 73, 5, 1532-1536. ). In all of these sound systems, audible sound is generated from outside the target area and emitted towards the target area.

Therefore, it is desirable to have a high resolution and scalable aerial volmetric display. It is also desirable to have an aerial volmetric display with improved non-contact tactile feedback. It is also desirable to have an immersive spatial audio experience that is personalized to accompany a 3D visual experience from a particular perspective.

The present invention is a system and method for generating aerial volmetric graphics, which causes a physical phenomenon in which a femtosecond laser emits light at an arbitrary three-dimensional position.

It is an object of the present invention to utilize spatial light modulators and computational phase modulation to improve the scalability and resolution of laser-based aerial vormetric displays. Another object of the present invention is to utilize a femtosecond laser to provide a secure interaction between the user and the volmetric display. Another object of the present invention is to realize a functional aerial audio speaker by utilizing a femtosecond laser. The plasma induced in the focal point of the ultrashort pulse laser produces a shock wave such as an impulse, and the focal point can be manipulated and placed at any position in three-dimensional space. Computational graphical design methods are used to implement spatial audio speed.

According to one aspect, the system comprises an ultrafast femtosecond laser light source, a spatial light modulator, and a three-dimensional position scanner. The system is simultaneously addressable, can be combined with computer-generated holograms, can provide near-field laser plasma displays, can provide tactile-based plasma interactions, and provides immersive audio. It can be provided.

According to the present invention, the laser-induced plasma obtained from the femtosecond light source can be used for interactive control including tactile interaction. Low-energy plasma minimizes the risk posed to those who come into contact with the plasma.

One aspect of the present invention is
Equipped with a femtosecond light source that produces a laser pulse beam
Equipped with a processor that produces computer-generated holograms
A spatial light modulator that modulates the laser pulse beam according to the computer-generated hologram.
A three-dimensional scanner optically connected to the spatial light modulator is provided.
The three-dimensional scanner directs the modulated laser pulse beam to one or more focal points in the air.
A lens that focuses the modulated laser pulse beam.
It is a plasma generator.

In another aspect of the invention, the modulated laser pulse beam induces a luminescent effect at a single focal point.

In another aspect of the invention, the modulated laser pulse beam induces a luminous effect at multiple focal points at the same time.

In another aspect of the invention, the three-dimensional scanner comprises a galvano scanner and a varifocal lens.

In another aspect of the invention, the plasma generator comprises a sensor that detects a change in the brightness of the luminescence effect.

In another aspect of the invention, the modulated laser pulse beam produces a palpable light field at one or more focal points.

In another aspect of the invention, the plasma generator comprises an ultrasonic phased array that produces a palpable acoustic field at one or more focal points.

In another aspect of the invention, the plasma generator comprises an ultrasonic phased array that produces a palpable acoustic field around one or more focal points.

In another aspect of the invention, the tactile light field comprises a tactile image pattern.

In another aspect of the invention, the lens is a microlens array.

In another aspect of the invention, the plasma generator comprises a sensor that detects the position of an object.

In another aspect of the invention, the plasma generator comprises an amplitude modulator that modifies the intensity of the laser pulse beam according to an audio signal.

One aspect of the present invention is
With steps to generate a femtosecond laser pulse beam
With steps to generate computer-generated holograms
A step of generating one or more modulated laser pulse beams by modulating the femtosecond laser pulse beam according to the computer-generated hologram.
A step of directing the one or more laser pulse beams to one or more focal points in the air.
This is a plasma generation method.

In another aspect of the invention, the plasma generation method comprises the step of inducing a luminescence effect at one or more foci by focusing one or more modulated laser pulse beams at one or more foci.

In another aspect of the invention, the plasma generation method comprises a step of detecting a change in the brightness of the luminescence effect.

In another aspect of the invention, the plasma generation method comprises the step of generating a palpable light field at one or more focal points.

In another aspect of the invention, the plasma generation method produces ultrasonic radiation pressure at one or more focal points by focusing one or more modulated laser pulse beams at one or more focal points.

In another aspect of the invention, the plasma generation method comprises the step of inducing a luminescence effect around one or more focal points by focusing one or more modulated laser pulse beams at one or more focal points. ..

In another aspect of the invention, the plasma generation method comprises the step of inducing the luminescence effect at a rate sufficient to generate an afterimage.

In another aspect of the invention, the plasma generation method comprises the step of determining a new one or more focal combinations in the vicinity of the one or more focal points.

In another aspect of the invention, the plasma generation method comprises the step of using the detected change as an input selection.

In another aspect of the invention, the plasma generation method comprises the step of deriving a light emitting effect that generates sound waves by directing one or more modulated laser pulse beams at one or more focal points.

In another aspect of the invention, the sound wave is in the audible frequency band.

In another aspect of the invention, the sound wave is in the ultrasonic frequency band.

In another aspect of the invention, the plasma generation method comprises the step of adjusting the intensity of the femtosecond laser pulse beam according to an audio signal.

In another aspect of the invention, the plasma generation method comprises a step of weakening the femtosecond laser pulse beam.

In another aspect of the invention, the plasma generation method produces directional sound waves by arranging the one or more focal points at positions corresponding to the computer-generated hologram.

In another aspect of the invention, the plasma generation method
With steps to generate multiple fame and second laser pulse beams
A step of directing the plurality of laser pulse beams to two or more focal points is provided.

In another aspect of the invention, the two or more focal points include a tactile image pattern.

In another aspect of the invention, the plasma generation method comprises the step of generating ultrasonic radiation pressure in the vicinity of the two or more focal points.

Other features and advantages of embodiments of the present invention will be readily apparent from the detailed description below, the accompanying drawings, and the appended claims.

The laser inductive effect for each medium is shown. FIG. 1 (a) shows plasma emission caused by ionization of air. FIG. 1 (b) shows the fluorescence induced in the fluorescent solution. FIG. 1 (c) shows the fluorescence that can be induced in the fluorescence plate. FIG. 1 (d) shows the light scattered by the microbubbles generated from the cavitation of water. The system which generates the vormetric graphics of this embodiment is shown. The system which generates the vormetric graphics of another aspect of this embodiment is shown. 4 (a) to 4 (c) show the spectra of the 100 fs laser, the 30 fs laser, and the 269 fs laser, respectively. FIG. 4D shows the peak intensities and pulse widths of the 100 fs laser, the 30 fs laser, and the 269 fs laser, respectively. FIG. 5A shows an example of an original image. FIG. 5B shows a converted spot array image of the original image. FIG. 5C shows a hologram of the original image to be displayed on the spatial light modulator. The experimental result of the brightness of the light emission induced in the air by a 30 fs laser and a 100 fs laser is shown. The experimental results of the brightness of the emission in the fluorescent solution, water, and air induced by the 30 fs laser are shown. The evaluation result of the brightness addressed to the plasma dots at the same time is shown. FIG. 8 (a) shows the normalized intensity of the plasma dots with respect to the simultaneously generated voxel pulse energies. FIG. 8B shows a magnified photograph simultaneously addressed to the plasma dots and a computer-generated hologram used to generate the plasma dot pattern. FIG. 8C shows a comparison result of the intensity of plasma dots with respect to the pulse energy of voxels generated simultaneously in a series of photographs. The experimental results regarding the damage to the skin are shown. FIG. 9A shows the effect when the leather sheet is irradiated with a laser pulse of 30 fs for 50 ms. FIG. 9B shows the effect when the leather sheet is irradiated with a laser pulse of 30 fs for 2000 ms. FIG. 9C shows a damaged region when the leather sheet is irradiated with a laser pulse of 30 fs for 2000 ms. FIG. 9D shows the effect of irradiating the leather sheet with laser pulses of 30 fs and 100 fs for each irradiation time of 50 ms to 6000 ms. Various applications of this embodiment are shown. FIG. 10A shows an example of spatial virtual reality. FIG. 10B shows an example of tactile feedback. FIG. 10 (c) shows an example of aerial volmetric graphics. FIG. 10 (d) shows an example of aerial volmetric graphics surrounded by a transparent wall (eg, a glass wall). The experimental results when the address is specified to the fluorescent liquid at the same time are shown. The experimental results when the address is specified to the fluorescent plate at the same time are shown. The experimental results when addressing water at the same time are shown. The experimental results regarding aerial rendering are shown. FIG. 14A shows a logo rendered in the air. FIG. 14B shows a cylinder rendered in the air. FIG. 14 (c) shows the heart and interaction effects rendered in the air. FIG. 14 (d) shows a fairy rendered in the air. FIG. 14 (e) shows "bean sprouts" that germinate from seeds as an example of virtual reality. FIG. 14 (f) shows a light spot that turns into a "jewel" after contacting the ring. FIG. 14 (g) shows the direct interaction between the light spot and the finger. The tactile feedback generated by the laser and ultrasonic crossfield setup is shown. The system which generates the vormetric graphics and the cross-field tactile feedback of this embodiment is shown. The aerial volmetric display of this embodiment and the system for generating cross-field tactile feedback are shown. The experimental results regarding the perceptual threshold are shown. FIG. 18A shows the perceptual threshold of the shock wave. FIG. 18B shows the perceptual threshold of ultrasonic radiation pressure. FIG. 18 (c) shows a cross-field perceptual threshold that includes a shock wave of laser plasma loaded with an ultrasonic vibration stimulus that is weaker than the perceptual threshold. Shown is a series of photographs of the spatial pattern generated by a laser plasma containing dots, lines, and boxes. The experimental results regarding the identification of the spatial pattern generated by the laser plasma are shown. The system which generates the palpable ultrasonic field of this embodiment is shown. The figure of the control system of this embodiment is shown. A multi-resolution 3D image generated by laser and ultrasound is shown. FIG. 23 (a) shows a user view of visual and tactile virtual reality. FIG. 23B shows a laser plasma. FIG. 23 (c) shows the ultrasonic field visualized by dry ice. FIG. 23D shows a camera view of the virtual reality marker. FIG. 23 (e) shows one of the cameras in the system setup. The characters in the Braille alphabet set and the corresponding generated plasma images of the computer-generated holograms are shown. FIG. 25 (a) shows a comparison of normalized sound pressures emitted from a single focal point by laser power. FIG. 25B shows the microphone position around the plasma spot with respect to the propagation direction of the laser beam. A chart comparison of the directivity of the focal position for each focal length is shown. A chart comparison of laser power and normalized sound pressure emitted from a single focal point for each pulse width is shown. The sound wave emitted from the laser plasma is shown. The superimposed waveform of the light field ultrasonic field is shown. An input / output screen of a graphical simulator that simulates the interference pattern of sound points is shown. FIG. 30A shows a graphical user interface that allows the user to enter the position of a sound source by selecting points on the grid. FIG. 30B shows the result of directivity for the combination of input points. FIG. 30 (c) shows the result of directivity for other combinations of input points. FIG. 30 (d) shows a heat map for a particular combination of input points. FIG. 30 (e) shows a heat map for other combinations of input points. The system which generates the aerial sound source of this embodiment is shown. Another system for generating the aerial sound source of this embodiment is shown. Another system for generating the aerial sound source of this embodiment is shown. Another system for generating the aerial sound source of this embodiment is shown. The sound wave characteristic and frequency characteristic of a single focus are shown. FIG. 35 (a) shows the time domain. FIG. 35 (b) shows the frequency domain. FIG. 36 (a) shows a microlens array having a specific irradiation pattern of this embodiment. FIG. 36B shows a microlens array with another irradiation pattern of this embodiment. FIG. 36 (c) shows the theoretical sound pressure distribution corresponding to the plasma distribution induced using the microlens array of FIG. 36 (a). FIG. 36 (d) shows the theoretical sound pressure distribution corresponding to the plasma distribution induced using the microlens array of FIG. 36 (b). FIG. 36 (e) shows an experimental setup for measuring the sound pressure distribution produced by the microlens array. FIG. 37 (a) shows a system for generating the sound of the present embodiment. FIG. 37B shows a system for changing the frequency of the generated sound of the present embodiment. FIG. 37 (c) shows another system for changing the frequency of the generated sound of this embodiment. FIG. 37 (d) shows a system for changing the frequency of the generated sound of the present embodiment. The polarity characteristics of each frequency component of the sound radiated from the two sound sources generated by using LCOS-SLM are shown. The polarity characteristics of each frequency component of the sound radiated from the four sound sources generated by using LCOS-SLM are shown. The system for amplitude modulation of this embodiment is shown. Various speaker configurations that can be implemented in this embodiment are shown. Various applications of spatial sound of this embodiment are shown. FIG. 42 (a) shows the use of spatial sound with 3D graphics. FIG. 42 (b) shows the use of spatial sound with aerial graphics. FIG. 42 (c) shows the use of spatial sound with interactive aerial graphics. Other applications of spatial sound with 3D graphics of this embodiment are shown. FIG. 44 shows the directivity characteristics of 10 kHz, 20 kHz, 30 kHz, 40 kHz, 50 kHz, 60 kHz, 70 kHz, 80 kHz, and 90 kHz.

The present invention provides a system that uses a femtosecond laser to generate touchable aerial volmetric graphics and generate immersive audio. According to the present embodiments disclosed and described herein, laser-induced spots generated in the air are used to provide an interactive audiovisual experience. An object of the present embodiment is to realize a safe and scalable high-intensity laser for a wide range of applications.

There are three types of laser inductive effects that produce light spots, including fluorescence, cavitation, and ionization. The particular effect involved depends on the display medium. 1 (a)-(d) show various laser-induced effect media for each display.

Laser-induced fluorescence occurs when a fluorescent solution or phosphor is excited using a laser. First, when an atom absorbs one or more photons, the orbital electrons in the molecule or atom are excited. Then, as the electrons relax, new photons are emitted. When two photons are absorbed at the same time, the wavelength of the emitted photon is half that of the original photon. The wavelength required to excite an electron depends on the type of fluorescent substance. When N photons are absorbed at the same time, the synchrotron radiation has a short wavelength N times. This effect occurs with relatively low intensity lasers (energy from nJ to mJ is sufficient).

Laser-induced cavitation occurs when a liquid medium is excited using a laser. Microbubbles are generated at the focal point of the laser in the liquid medium. This localized cluster of microbubbles diffuses the incident laser so that the laser is observed as a pointed light. The color of this point light depends directly on the wavelength of the incident laser. This fact indicates that RGB images can be represented using multiple lasers. This effect requires an intense laser to generate microbubbles, regardless of the material.

Finally, laser-induced ionization occurs when the gas medium is excited using a laser. In particular, tunnel ionization can generate sufficient visible light and occurs particularly predominantly when the ^{laser intensity is greater than 1014 W / cm 2.} The well-shaped potential of a molecule or atom is deformed by the electric field of a high-intensity laser with a potential barrier, and then the electrons have the opportunity to leave (ie, ionize) the atom based on the tunneling effect. It is known that the higher the intensity of the laser, the higher the probability of tunnel ionization, that is, the more electrons are ionized. The ionized electrons are recombined with atoms after a half cycle and photons are emitted (this effect is called laser destruction). The emitted light has a bluish white color.

This embodiment focuses on the ionization effect because it is easier to implement in the air and has a wider range of applications, but consider that other effects are discussed for each display medium. Furthermore, the plasma induced in this embodiment can be touched. In addition, the induced plasma produces an impulse-like shock wave that can be modulated to generate sound waves.

FIG. 2 shows a system 100 that generates simultaneous multipoint volmetric graphics of the present embodiment. The system 100 includes a system controller 101, a femtosecond laser light source 110, a spatial light modulator 120, a three-dimensional position scanner (including a galvanometer mirror scanner unit 130 and a varifocal lens 135), an objective lens 160, and an optical lens. And mirrors 140, 142, 144, 146, and 148. System 100 can be used to display images on various display media (including, for example, air and water).

The system controller 101 is operatively coupled to the spatial light modulator 120, the galvano scanner unit 130, and the varifocal lens lens 135. Image 195 is formed in workspace 190 by directing a laser pulse beam through an optical circuit. The system controller 101 forms the graphics generated by the system controller 101 by associating these components with the object image and synchronizing the femtosecond light source 110.

As shown in FIG. 2, the femtosecond light source 110 produces a laser pulse beam 112 modulated by the spatial light modulator 120. The modulated laser pulse beam passes through two lenses 140 and 142 that act as beam reducers. Next, the direction of the laser pulse beam is changed by the galvano scanner unit 130, and the focus of the laser light on the XY axis in the workspace 190 is determined. The altered laser pulse beam passes through two lenses 144 and 146 that act as beam expanders. The laser pulse beam is then redirected by the mirror 148 and passes through the varifocal lens unit 135 to determine the focus of the laser light on the Z axis in the workspace 190. Finally, the laser pulse beam is incident on the objective lens 160 and focused on a focal point that excites a particular position on the display medium (eg, air, water, fluorescent screen, or fluorescent fluid) in the workspace 190.

The system controller 101 can be, for example, a personal computer (hereinafter referred to as “PC”) having a conventional video output port (for example, a DVI port) and a USB (Universal Serial Bus) port.

A commercially available laser can be used as the femtosecond light source 120. Ultrashort pulses can be generated by adults who convert low intensity and long time pulses into high brightness and short time pulses. When the time average laser output is constant, the peak intensity depends on the pulse width. For example, the 30 fs pulse width has a peak intensity greater than the 100 fs pulse width in terms of the same time average output. For aerial plasma generation, the laser peak intensity is more important than the pulse width.

The display medium is a major factor in the selection of femtosecond light sources for volmetric displays. Available wavelengths vary depending on the method of inducing the light spot. In the case of ionization, the plasma color is wavelength independent, so the use of invisible wavelengths (eg, infrared or ultraviolet) is appropriate. In the case of fluorescence, multi-electron fluorescence is appropriate because a plurality of photons are absorbed by the numerator and become a single photon having a shorter wavelength. Thus, invisible UV light is available only when the emission is visible. On the other hand, when cavitation is applied, it is necessary to use a visible wavelength as a light emitter because the incident wavelength is diffused by microbubbles, observed, and universal.

As the spatial light modulator 120, a commercially available product of SLM (Spatial Light Modulator), which is an optical device that can be used for forming a hologram, can be used. SLMs modulate at least one of phase and intensity and generate various spatial distributions of light based on interference. It can be used to generate any laser pattern. This is achieved by applying a computer-generated hologram (CGH), i.e., computational phase modulation based on a two-dimensional cross section of the laser beam. Examples of the desired output image and the corresponding CGH are shown in FIGS. 5 (a) and 5 (c). FIG. 5A shows a target image. FIG. 5C shows the CGH obtained from the target image. The SLM uses a CGH image to modulate the laser pulse beam. Therefore, the SLM produces one or more focal points in three-dimensional space from one laser pulse beam (including voxels addressed at the same time).

In this embodiment, any desired 3D graphics can be generated by computational holography using SLM as follows.

ここで、ＳＬＭに表示されるホログラム面の振幅及び位相は、それぞれ、再構成される面の振幅及び位相である。単純化すると、ＣＧＨに対する放射光が均一な強度分布を有する平面波として近似することができる場合、一定であるとみなせる。これは、ＯＲＡ（Optimal Rotation Angle）法により導出される。再構成の空間強度分布が、｜Ｕ_ｒ｜^２＝ａ_ｒ ^２として、実際に観察された。 The spatial phase control of light allows the focal position to be controlled along both the lateral (XY) and axial (Z) directions. Computer generated holograms (CGH) the complex amplitude of the reconstruction from _{U r} (CA) is given by the Fourier transform of the designed CGH pattern _{U h.}

Here, the amplitude and phase of the hologram surface displayed on the SLM are the amplitude and phase of the surface to be reconstructed, respectively. To simplify, if the synchrotron radiation to CGH can be approximated as a plane wave with a uniform intensity distribution, it can be considered constant. This is derived by the ORA (Optimal Rotation Angle) method. The spatial intensity distribution of the reconstruction was actually observed as | _Ur ^{| 2} = a _r ^2.

ここで、誘起スポットの大きさがレーザの焦点の大きさに等しい場合、ＳＬＭの空間分解能が最小焦点距離を決める。 For controlling the focusing position along the lateral (XY) direction, the CGH is designed based on the superposition of CAs of blaze gratings with various azimuth angles. If the reconstruction has N multi-focused spots, the CGH comprises an N blaze grating. In controlling the focusing position along the axis (Z) direction, the phase Fresnel lens pattern is simply added to _{φ h.}

Here, if the magnitude of the induced spot is equal to the magnitude of the focal point of the laser, the spatial resolution of the SLM determines the minimum focal length.

ここで、Ｕ_ｈｒは、ＣＧＨ平面上のピクセルｈから再構成平面上のピクセルｒまで影響を受けるＣＡであり、φ_ｈｒは、ピクセルｈからピクセルｒまでの光伝搬によって影響を受ける位相であり、ω_ｒ ^（ｉ）は、ピクセルｒの光強度を制御する重み係数である。ピクセルｒ毎の光強度の合計Σ_ｒ｜Ｕ_ｒ ^（ｉ）｜^２を最大化するために、ピクセルｈでφ_ｈ ^（ｉ）に加算される位相変化Δφ_ｈ ^（ｉ）は、以下の数式を用いて計算される。

ここで、ω_ｒは、再構成平面上のピクセルｒでの位相である。ＣＧＨφ_ｈ ^（ｉ）は、Δφ_ｈ ^（ｉ）によって以下のように更新される。

さらに、ω_ｒ ^（ｉ）は、また、再構成平面上のピクセルｒでの光強度を制御するために、式（７）のフーリエ変換によって得られる再構成の光強度に応じて更新される。

ここで、Ｉ_ｒ ^（ｉ）＝｜Ｕ_ｒ ^（ｉ）｜^２は、ｉ番目の反復プロセスにおいて、最高性平面上のピクセルｒでの光強度であり、ｉ_ｒ ^（ｄ）は、所望の光強度であり、αは一定である。Ｉ_ｒ ^（ｉ）がＩ_ｒ ^（ｄ）に略等しくなるまで、位相変化Δφ_ｈ ^（ｉ）は、上記繰り返しプロセス（式（４）〜（８））によって最適化される。したがって、ＯＲＡ法は、高品質なＣＧＨの生成を容易にする。 The ORA method is an optimization algorithm for obtaining a reconstruction of a CGH composed of spot arrays having uniform intensities. 4 (a) and 4 (c) show the original image and an example of the CGH corresponding to the original image. This means that in the i-th iterative process, the amplitude of pixel h is a _h , the phase of pixel h is φ _h , and the complex amplitude of pixel r corresponding to the focusing position on the reconstruction plane is (CA) _Ur ^(i). Then, in the computer, it is described as follows.

Here, U _hr is a CA that is affected from pixel h on the CGH plane to pixel r on the reconstruction plane, and φ _hr is a phase that is affected by light propagation from pixel h to pixel r. ω _r ⁽ⁱ⁾ is a weighting coefficient that controls the light intensity of the pixel r. The phase change Δφ _h ⁽ⁱ⁾ _{added to φ h} ⁽ⁱ⁾ at pixel h in order to maximize the total light intensity for each pixel r Σ _r | _Ur ⁽ⁱ⁾ | ² is calculated by the following formula. Calculated using.

Here, ω _r is the phase at pixel r on the reconstruction plane. CGHφ _h ⁽ⁱ⁾ is _{updated by Δφ h} ⁽ⁱ⁾ as follows.

Further, ω _r ⁽ⁱ⁾ is also updated according to the light intensity of the reconstruction obtained by the Fourier transform of the equation (7) in order to control the light intensity at the pixel r on the reconstruction plane.

_{^{Here, I r (i) = |}} U r (i) | 2 is the i th iterative process, a light intensity at a pixel r on the highest of the _plane, ^{i r (d)} the desired optical It is the intensity, and α is constant. I _r ⁽ⁱ⁾ until substantially equal to _I ^{r (d),} the phase change [Delta] [phi _h ⁽ⁱ⁾ is optimized by the iterative process (equation (4) to (8)). Therefore, the ORA method facilitates the production of high quality CGH.

In general, an SLM has an array of pixels that modulates at least one of the intensity and phase of the laser beam. SLMs have dynamically reconstructable pixels. For example, SLMs include LCOS SLMs that modulate the phase and DMD SLMs that modulate the intensity. Dual mask SLMs can modulate both phase and intensity.

Liquid crystal SLMs include a layer of liquid crystal molecules. The orientation of the liquid crystal molecules in this layer is controlled by the electrodes (ie, pixels), and the phase of the light rays reflected or passing through this layer is spatially modulated according to the orientation of the liquid crystal molecules. That is, there are two types of liquid crystal-based SLMs (liquid crystal (LC) -SLMs and liquid crystal (LCOS) -SLMs on silicon).

The LC-SLM is a parallel oriented nematic liquid crystal spatial light modulator (PAL-SLM) coupled with a liquid crystal display (LCD) and a laser diode (LD). This device is often used to display real-time CGHs. The PAL-SLM has a liquid crystal layer, a dielectric mirror layer in a specified wavelength range, and an optically addressed photoconductive layer containing amorphous silicon. These layers are sandwiched between two transparent indium tin oxide electrodes. The liquid crystal molecules in the liquid crystal layer are arranged in parallel. When the incident light enters the photoconductive layer, the impedance of the photoconductive layer decreases, and the electric field of the liquid crystal layer increases accordingly. With this increase in electrolysis, the liquid crystal molecules are inclined in the propagation direction of the readout light, and the effective refractive index of the liquid crystal layer decreases. Pure phase modulation occurs only when the polarization direction of the femtosecond laser is parallel to the direction of the liquid crystal molecules. The CGH pattern on the LCD illuminated by the LD is applied to the photoconductive layer via imaging optics.

The LCOS-SLM is a spatial light modulator having a structure in which a liquid crystal layer is arranged on a silicon substrate. The electrical addressing circuit is formed on a silicon substrate by semiconductor technology. The top layer contains pixels, each independently made of aluminum electrodes that control its electrical potential. A glass substrate is placed on the silicon substrate while maintaining a certain gap, and the gap is filled with a liquid crystal material. The liquid crystal molecules are oriented in parallel with no twist between the two substrates by the orientation control technique applied to the silicon substrate and the glass substrate. The electric field sandwiching the liquid crystal layer can be controlled on a pixel-by-pixel basis. This tilts the liquid crystal molecules in response to the electric field so that the phase of the light can be modulated. In addition, the dielectric mirror layer reduces internal absorption by improving the reflectance and enables operation with a high-power laser.

A digital micromirror device (DMD) includes a layer of a microscope, a pixel, and two pairs of electrodes in each mirror that controls the position of the mirror. Each microscope can be turned on or off individually. The amplitude of the light rays reflected by this layer is spatially adjusted according to the direction of the mirror.

In other words, the SLMs function as an optical phased array. By precisely controlling the wave of light in this way, the SLM can be used to form a light beam pattern that produces a hologram. With reference to equations (3) to (8), for LCOS SLM, a _h is fixed at 1 and φ _h is calculated. Instead, for DMD SLM, φ _h is fixed at 0 and a _h is calculated as 0 ≦ a _h ≦ 1. After that, a _h is rounded to 0 or 1. When a _h is 0, 0 ≦ a _h ≦ 0.5, and when a _h is 0.5, 0.5 < _ah ≦ 1.

図２は、反射型ＳＬＭｓに基づく光学回路を示すが、反射型ＳＬＭｓに代えて、透過型ＳＬＭｓも利用可能である。好ましくは、ＳＬＭ１２０は、ネマチック液晶素子を使用する反射型線形アレイＳＬＭであり、例えば、液晶ＳＬＭｓ（ＬＣ−ＳＬＭｓ）及びシリコンｅｋしようＳＬＭｓ（ＬＣＯＳ−ＳＬＭｓ）を含む。ＳＬＭ１２０のエネルギー変換効率は、約６５％〜約９５％になるべきである。 In selecting an SLM for a vormetric display, the resolution and speed of the SLM are the main factors to consider. LCOS SLMs have a lower operating frequency than DMD SLMs, even though they have higher diffraction efficiencies. This means that LCSO SLMs are slow but have high resolution. The DMD SLM has a higher thermal resistance and a higher operating frequency than the LCOS-SLM. Table 1 outlines the factors to consider.

FIG. 2 shows an optical circuit based on reflective SLMs, but transmissive SLMs can be used instead of reflective SLMs. Preferably, the SLM 120 is a reflective linear array SLM that uses a nematic liquid crystal element and includes, for example, liquid crystal SLMs (LC-SLMs) and silicon eks SLMs (LCOS-SLMs). The energy conversion efficiency of SLM120 should be from about 65% to about 95%.

Since the SLMs currently on the market are not resistant to a strong laser light source, the laser power of the laser light source cannot be fully utilized. Further, the optical circuit requires careful development and processing because ionization may occur in the optical circuit when a high-intensity laser is used. This reduces the output energy and may destroy the optics.

An alternative to SLM is a modulator that is a passive small lens array, i.e. a microlens array. A cover having holes (for example, a paper sheet) is set in the opening of the microlens array. The modulator provides multi-access because each lens has its own focus. Although the modulator is static, it has the advantage of high resolution.

The three-dimensional position scanner can be realized by combining commercially available optical elements. The three-dimensional position scanner shown in FIG. 2 includes a galvano scanner unit 130 and a varifocal lens 135. The galvano scanner unit 130 scans the light spots along the lateral direction (X scan and Y scan), and the varifocal lens 135 can change the focal point in the axial direction (Z scan). The galvano scanner unit 130 and the varifocal lens 135 are controlled by the system controller 101. In addition, the system controller may be combined with additional control circuitry.

In another exemplary embodiment, the SLM can also be used as a three-dimensional scanner, as spatial phase control of light allows control of the focusing position along both the lateral (XY) and axial (Z) directions. good.

Therefore, for 3D position scanners, one of ordinary skill in the art should understand that there are three options for placing points at any position. The first option is to guide the laser by adjusting the galvano scanner and varifocal lens. The second option is to change the focus by modulating the cross-sectional distribution of the laser with SLM. The SLM and the combination of the galvano scanner unit and the varifocal lens can form graphics in substantially the same area. The conditions and response times of these devices determine a suitable 3D position scanner. The third option is a combination of SLM, galvano scanner unit, and varifocal lens.

The SLM is used to form additional dots in a single frame, and the galvano scanner is primarily used to position the formed hologram. For femtosecond lasers pulsed at a frequency of 1 kHz, the theoretical rendering limit is 33 dots per second for 30 frames per second.

The system controller 101 is a general personal computer (hereinafter referred to as "PC"). The PC runs a custom software application and is directly or indirectly connected to the 3D position scanner (eg, via a USB cable or an optical interface circuit board, ie a PCI driver board). Preferably, the system controller 101 is connected to the galvano scanner unit 130 and the vocalifocal lens 135 via USB.

SLMs have control interface ports that can be connected to standard computer system video output ports (eg, DVI ports). The orientation of the liquid crystal molecules or micromirrors can be controlled on a pixel-by-pixel basis. Preferably, the SLM 120 is connected to the control controller 101 as an external display via a DVI port.

The system controller 101 drives the CGH based on the desired output image and controls the SLM 120, the galvano scanner unit 130, and the varifocal lens 135 in synchronization with the femtosecond laser light source 110 to control the workspace. The output image is displayed on 190. A camera can be connected to the system controller 101 to monitor the workspace 190.

A commercially available optical lens can be used as the objective lens. The objective lens is not a special lens, but an ordinary optical lens arranged at the end of an optical circuit. Objective lenses are required to generate aerial plasma. Laser plasma generation requires laser power (PW / cm ² ). Therefore, an objective lens is needed to generate an aerial plasma by focusing light and forming a focal point. Since the angular range of the galvano scanner unit 130, i.e. XY scanning, is limited by the aperture size of the objective lens, the aperture size of the objective lens defines the maximum workspace. The larger the aperture size, the wider the angle range of the galvano scanner, but the smaller the aperture size, the larger the laser power.

In other exemplary embodiments, the objective lens is optional. The laser power required to excite water is less than the laser power required to excite air. Therefore, an objective lens is not required for a volmetric display that uses water as a display medium. The size of the workspace of these displays is limited by the angular range of the galvano scanner and the depth range of the varifocal lens.

In the embodiment shown in FIG. 2, the optical circuit includes a pair of beam expander lenses and a pair of reducer lenses. These beam expander and reducer lenses are commercially available optical lenses. These are used for adjusting the beam spot size of the laser beam and are added for commercial purposes, that is, when the elements of the optical circuit operate for each beam spot size.

The display medium is a key factor in determining potential interactions, as the brightness of the voxels depends on the absorption rate of the selected medium. The order of energy required decreases from air to water. Thus, air breakdown for tunnel ionization requires energy on the order of ^{PW / cm 2} while water requires energy on the order of ^{MW / cm 2.} In addition, the softness of the medium also determines the interaction.

Aerial plasma allows the user to insert and touch the plasma. When a full-color laser source is adopted, the liquid voxels caused by cavitation provide a full-color representation. However, the air breakdown method can only provide a monochrome representation.

もう１つは、レーザビームのパスに平行な直径ｗ_ｄである。これは、ａ：ｗ_ｆ＝ｒ：ｗ_ｄ／２の関係から、幾何学的に以下のように導かれる。

高輝度レーザによって生成された発光ドットは、伝搬方向（フィラ）に沿った尾を有する。尾は、光学的カー効果に起因して振る舞う自己集束として生成される。それは、レーザビームの自然回折と競合し、且つ、空中に３次元グラフィックスを形成する場合には望ましくないものである。特に、この効果は、人間の目には見えないものである。これは、焦点での光がより明るいためである。 The size of the voxel (ie, the luminescence effect) relates to the size of the focal point of the laser. The focal point is usually oval with two diameters. _{One is the diameter w f} perpendicular to the path of the laser beam. This is the diffraction limit and is determined by the original beam width a, the focal length r, and the wavelength λ, and is expressed as follows.

The other is a parallel diameter w _d in the path of the laser beam. This is geometrically derived from the relationship of a: w _f = r: w _{d / 2 as follows.}

The emitting dots generated by the high-intensity laser have a tail along the propagation direction (filler). The tail is produced as a self-focusing that behaves due to the optical Kerr effect. It competes with the natural diffraction of the laser beam and is undesirable when forming 3D graphics in the air. In particular, this effect is invisible to the human eye. This is because the light at the focal point is brighter.

フレーム毎のドット数は、Ｎ_ｄｏｔ、レーザパルスの繰り返し周波数Ｆ_ｒｅｐ、及び、フレーム時間Ｔ_ｆによって、ヒューマンビジョンの持続性に基づいて以下のように決定される。 The spatiotemporal resolution of the volmetric display is defined by the number of dots per frame (dpf). When the dots appear in the dark, the minimum energy requirement per dot is equal to the _{laser breakdown threshold Elbd.} The total output energy E _tot is divided between the dots by the SLM. _Assuming that the number of dots for each laser pulse is N dot, the following equation holds.

The number of dots per frame is determined by N _dot , laser pulse repetition frequency F _rep , and frame time T _f as follows based on the sustainability of human vision.

例えば、Ｎ_ｄｏｔ＝１００、Ｆ_ｒｅｐ＝１ｋＨｚ、及び、Ｔ_ｆ＝１００ｍｓである場合、１００００ｄｐｆのアニメーションは１０ｆｐｓで再生される。実際には、毎フレームのドット数は、Ｆ_ｒｅｐに代えて、ガルバノスキャナ及び／又はＳＬＭの応答時間のボトルネックによって決まる。

For example, when N _dot = 100, F _rep = 1 kHz, and T _f = 100 ms, the animation of 10000 dpf is reproduced at 10 fps. In practice, the number of dots per frame is determined by the response time bottleneck of the galvano scanner and / or SLM instead of the _Flep.

Spatio-temporal resolution can be improved by using SLM to generate voxels that are simultaneously addressed and by increasing the repetition frequency of the laser source. Fourier CGH is used to generate voxels that are simultaneously addressed. Fourier CGH can be optimized by using the ORA (Optimal Rotation Angle) method. The amount of resolution improvement obtained by using voxels addressed at the same time depends on (1) the energy of the laser source, (2) the durability of the SLM, (3) the refresh rate of the SLM, and (4) the resolution of the SLM. do. However, due to the nature of the liquid crystal molecules of SLM, there is a limit to improving the refresh rate and durability of SLM. Therefore, in order to realize high resolution of aerial plasma imaging, the repetition frequency plays an important role in increasing the resolution. Higher energies and higher repetition frequencies solve the resolution problem.

On the other hand, since the energy required to induce underwater emission is smaller, SLM plays an important role in realizing a high-resolution image in an aqueous medium. Therefore, high resolution of the volmetric display can be achieved by parallel access to the three-dimensional position.

(experiment)
The following experiments were carried out to prove the potential of the present invention and to show various advantages.

The experiments below were performed at 20.5 ° C. under normal atmosphere (normal air with a mixture of 80% N ₂ and 20% O _{2 above sea level).} The water was tap water.

For each of Visual Experiments 1-6, one or more preferred embodiments below were used.

Preferred embodiments (referred to herein as "system A") are described below based on the optical system setup shown in FIG.

System A includes a femtosecond laser light source manufactured by Coherent. The fame and second laser light source has a center wavelength of 800 nm, a repetition frequency of 1 kHz, a maximum pulse energy of 2 mJ, and a pulse duration adjustable from 30 fs to 100 fs. 4 (a) to 4 (d) show the spectral and pulse energies of the 30 fs pulse setting and the 100 fs pulse setting with this light source. If the average laser pulse energy is invariant, the peak energy will vary for each laser pulse duration. In practice, the 30 fs pulse duration has a peak energy three times greater than the 100 fs pulse duration at the same average energy setting. In visual experiments 1-6, the experiments were performed with the pulse width set to 30 and the pulse duration set to 100 fs. Experiments and results are shown as System A (30 fs) and System A (100 fs), respectively. Peak brightness, rather than the average power of the laser, is important for the generation of aerial plasma, even for short pulses. System A has a peak brightness sufficient to excite air and generate an ionized plasma.

System A is further equipped with LC-SLM manufactured by Hamamatsu Photonics. The LC-SLM comprises a PAL-SLM connected to an LCD and a 680 nm laser diode. The device is capable of performing only two or more radian phase modulations and has a resolution of 768 pixels x 768 pixels, ^{a pixel size of 20 x 20 μm 2} , and a response time of 100 ms.

System A further comprises two lenses. The two lenses have focal lengths of 450 mm and 150 mm, respectively. The two lens units reduce the beam spot size by a factor of three.

System A further comprises a galvano scanner unit. The galvano scanner unit includes a scan head unit (Canon GH-315) driven by a scanner controller board (Canon GB-501). The scan head unit (Canon GH-315) has a beam diameter of 10 to 14 mm, a scan angle of ± 0.17 rad, an error of 5 μrad or less, and a resolution of 20 bits. The scan head covers an area of ^{about 10x10 mm 2 or more.} The scanner controller board (Canon GB-501) controls the scan head unit and the laser unit that directs the laser beam to arbitrary coordinates in the XY plane. The standardized PCI bus interfaces with the PC and obtains instructions from PC commands.

System A further comprises two lenses. The two lenses have focal lengths of 540 mm and 150 mm, respectively, and are located behind the galvano scanner unit. These two lens units increase the beam spot size by 1.5 times.

System A is further equipped with a varifocal lens lens unit (EL-10-30 manufactured by Optotune). The aperture of the varifocal lens (EL-10-30) is 10 mm, the response time is less than 2.5 ms, and the focal length is +45 to +120 mm. The varifocal lens (EL-10-30) adjusts the Z-axis focal position of the laser beam on the volmetric screen.

System A further comprises an objective lens. The focal length of the objective lens is 40 mm.

System A further comprises a system controller. The system controller includes a PC running an operating system (Windows®). All programs that control the operation of the SLM, the galvano scanner unit, and the varifocal lens are coded in C ++. The galvano scanner unit and varifocal lens unit operate in their own thread and synchronize when a new drawing pattern is received. User input may be captured at 20 Hz. The control system also comprises a USB microscope. USB microscopes are used to monitor the interaction between the optical setup and the display medium.

The energy conversion rate of system A is 53%.

Another preferred embodiment based on the optical system setup of FIG. 3 (hereinafter referred to as "system B") will be described below.

System B is equipped with a femtosecond laser (FCPA μJewel DE1050 manufactured by IMRA America). The femtosecond laser has a center wavelength of 1045 mm, a repetition frequency of 200 kHz, a maximum pulse energy of 50 μJ, and a pulse duration of 269 fs. System B has sufficient peak brightness for excitation of air and generation of ionized plasma.

System B is further equipped with a galvano scanner (Intelliscan 20i). The galvano scanner scan angle is ± 0.35 rad, the error is less than 5 μrad, and the resolution is 20 bits.

System B is further equipped with a varifocal lens unit (EL-10-30 manufactured by Optotune). The aperture of the varifocal lens unit (EL-10-30) is 10 mm, the response time is less than 2.5 ms, and the focal length range is +45 to +120 mm. The varifocal lens unit (EL-10-30) adjusts the Z-axis focus of the laser beam on the volmetric screen.

System B further comprises two lenses. The focal lengths of the two lenses are 50 mm and 80 mm.

System B further comprises an objective lens. The focal length of the objective lens is 20 mm.

System B further comprises a control system. The control system includes a PC running an operating system (Windows®). All programs that control the operation of the galvano scanner unit and varifocal lens are coded in C ++. The control system also includes a camera. The camera is used to monitor the interaction between the optical setup and the display medium.

The energy conversion efficiency of system B is 80%.

The laser-induced plasma emission effect in air requires laser power on the ^{order of petawatts (PW / cm 2) per square centimeter.}

(Visual experiment 1: Energy vs. brightness)
We conducted an experiment to evaluate the relationship between plasma generation energy and the brightness of the resulting image. Voxel brightness with respect to input energy is important for improving spatiotemporal resolution. The purpose of this experiment is to verify the possibility of System A and to investigate how to apply it to voxel display. Therefore, the minimum peak energy value was determined.

The experiment was carried out using System A with the pulse width set to 30 fs. The femtosecond laser source (Coherent) provided an output of up to 7 W, but some elements of the optical circuit prior to the objective lens caused air destruction due to the high output. As such, the full power of the laser light source cannot be used. In addition, the energy capacity of SLM (manufactured by Hamamatsu Photonics) is not guaranteed to be 2W. Therefore, the experiment was carried out in an output range of 0.05 to 1.00 W. The microscope camera was used to capture the resulting image.

(Visual experiment 2: Pulse width vs. brightness)
The inventor conducted an experiment to evaluate the relationship between pulse duration and voxel brightness. This is important in scalability, especially in the development of faster laser sources. Since peak energy plays an important role in plasma generation, the relationship between the peak pulse and the brightness of the resulting image was also tested.

Experiments were performed using System A with pulse widths of 30 fs and 100 fs and an output range of 0.05 to 1.00 W. The microscope camera was used to capture the resulting image.

The results are shown in FIG. The pulses 30 fs and 100 fs produce different spectra and peak energies with the same average output. The 30 fs laser produced more than three times the peak pulse. Experiments have shown that a 100 fs laser can generate plasma starting at 0.45 mJ per pulse and has a peak brightness of 24 PW / cm ² .

(Visual experiment 3: Experiment for each display medium)
We compare the energy consumption performance between various laser-induced effects, including gas ionized plasma, photon absorption, and cavitation, with exploratory means for applying femtosecond laser systems to various display technologies. Therefore, an experiment was conducted for each display medium.

The experiment was performed using System A with a pulse width of 30 fs. The microscope camera was used to capture the resulting image. As shown in FIG. 7, the experimental results show that the required pulse energy values vary by orders of magnitude depending on the display medium. Fluorescence occurs at about 0.01 μJ, cavitation occurs at about 2 μJ, and ionization occurs at energies greater than 100 μJ.

(Visual experiment 4: Simultaneous addressing in the air)
We have conducted experiments to determine the scalability of CGHs in order to modulate the phase of a laser pulse beam that produces high resolution graphics in the air. In the conventional system, it was not possible to generate multiple voxels at the same time. This experiment was specifically performed using SLM to explore the scalability of resolution by generating simultaneously addressed voxels from a single light source. Simultaneous addressing is important because it improves spatiotemporal resolution, but because energy is distributed among voxels, voxels addressed at the same time are darker than single voxels. The inventor has assumed that the use of CGHs can be used to generate multiple plasma spots at the same time. Simultaneous addressing is available for both the side (X, Y) and beam (Z) axes by displaying the appropriate \ hologram on a single SLM ↑. However, in this experiment, for the sake of simplification, only the simultaneous addressing of the horizontal axes was investigated.

This experiment was performed using System A, which had a pulse width of 30 fs and a laser power range of 0.05 to 1.84 W.

8 (a)-(c) show the results and the CGHs used in SLM. As shown in FIG. 8 (c), voxels addressed at the same time are visible. As a result, up to four parallel accesses could be observed under the constraints of the power supply of System A. The diffraction effect of SLM was about 50%.

(Video experiment 5: Simultaneous addressing underwater)
We conducted experiments to determine the feasibility of using CGHs that modulate the phase of a laser pulse beam to produce high resolution graphics in water. This experiment was performed using System A, which had a pulse width of 30 fs and a laser power range of 0.05 to 1.84 W. The microscope camera was used to capture the resulting image.

(Visual application 1: Aerial display)
The laser-excited plasma floats in the air. In another embodiment, a high resolution anthelmintic volmetric display is provided. 10 (c) to 10 (d) show vormetric images generated in the air. Aerial volmetric displays utilizing tunnel ionization are feasible with systems A and B. 14 (a)-(d) show various graphics generated in the air using systems A and B. For systems A and B, the workplaces are 10x10x10mm ³ and 8x8x8mm ³ , respectively. Compared to previous studies, the workspace is smaller, but the resolution is 10 to 200 times higher. The maximum spatiotemporal resolution is 4000 dots per second for system A and 200,000 dots per second for system B. The image frame rate depends on the number of vertices used in the model.

(Visual application 2: Spatial virtual reality for objects in the real world)
According to another embodiment of the invention, an augmented reality display is provided. Rendered plasma images are available with real-world objects. For example, FIG. 10A shows a plasma image 11 generated as an extension or accessory of the reality object 10. As another example of virtual reality, FIG. 14 (e) is a photograph of bean sprouts sprouted from seeds. The advantage of the technology of applying spatial AR to real-world objects is that AR content is on the same scale as virtual objects. In addition, the aerial display can be used in combination with a microscope camera. The microscope camera can detect the object in the workspace, can detect the timing when the plasma touches the object, and can superimpose the AR content on the object to express the extension of the object.

This is more advantageous than the conventional approach in that it corresponds to a three-dimensional spatial position. With conventional AR technology, it is difficult to show AR content at any three-dimensional position. However, the present invention allows the generation of plasma spots at any location in real space within the workspace.

Visual Application 3: Underwater Color Volmetric Display According to another embodiment of the present invention, a display medium for a color laser-based vormetric display using water is provided. Systems A and B use water as the display medium. In this configuration, the workspaces are 1 and 10 cm ³ , respectively. These workspaces depend on the focal area of the objective lens. In this application, the principles used to obtain voxels are different from other applications. In this application, the voxels use microbubbles to reflect light, and the user can see the color of the laser light at a point in a three-dimensional space filled with water. The minimum energy required to excite water is in the joule range. With system B, the wavelength of this system is 1064 mm so that the user can only see the bubbles.

(Scalability)
Workspace size scalability is a major concern. The size scalability depends on the type of display medium, as the display medium determines the amount of energy required to generate the laser excitation effect.

For aerial displays, energy on the order of ^{PW / cm 2} is required because a large amount of power is required in the focal region to induce tunnel ionization. Therefore, the generation of aerial plasma is mainly limited to the attributes of the objective lens that focuses the laser beam at the focal point. The larger the aperture of the objective lens, the larger the workspace.

Experiments with average power and high intensity peak pulses show that for everyday applications, three factors are key to achieving safety, stability, and adequate workspace size: The three factors are an increase in the power of the laser light source, (2) a shortening of the pulse and an increase in the peak energy, and (3) an increase in the scanning speed. By achieving these factors, the workspace can be expanded while maintaining tactile sensation and visibility in the application.

For vormetric displays containing water as the display medium, the energy required to induce cavitation is small enough that the workspace is not constrained by the objective lens. The workspace of these types of displays is constrained by the workspace of the 3D scanner (ie, galvano scanner unit and varifocal lens unit). In general, galvano scanner units and varifocal lens units are fast enough for scanning large spaces. Multiple laser focusing systems can be used to achieve a larger workspace for these displays.

(Tactile interaction)
In general, plasma has high energy and is therefore dangerous to humans. However, femtosecond lasers emit long and short pulses and are often used for non-thermal braking for industrial purposes. Therefore, we see that contact with the plasma induced by the femtosecond laser is less dangerous to humans if the light source is limited to peak intensity.

(Tactile experiment 1: Skin exposure)
The inventor conducted a search experiment to see if exposure of plasma induced by a femtosecond laser to human skin causes damage.

The experiment was carried out using System A composed of 30 fs at 1 W and System A composed of 100 fs at 1 W. The plasma exposure period varied from 50 to 6000 ms. FIG. 9D shows the experimental results, and the 30 fs pulse and the 100 fs pulse show almost the same effect on the skin. As mentioned above, the 30 fs pulse has a peak energy of 3 times or more and can generate brighter voxels. However, in the 50 ms period (50 shots), there is almost no difference between the 30 fs result and the 100 fs result. In this experiment, average power is a determinant of the results. Exposures less than 2000 ms (2000 shots) revealed holes with a diameter of 100 μm and no thermal damage to the leather. For a period longer than 2000 ms, a thermal effect was observed around the pores.

Tests with continuous (non-pulse) nanosecond lasers were performed for comparison with this result. When a nanosecond laser was used, the leather sprouted within 100 ms. This means that pulse duration, number of iterations, and energy are important factors influencing the level of damage caused by the laser.

The present inventor has verified that an ultrashort pulse laser has a non-thermal effect different from that of a nanosecond laser ( Jae-Hoon Jun, Jong-Rak Park, Sung-Phil Kim, Young Min Bae, Jang-Yeon Park, Hyung-Sik Kim, Seungmoon Choi, Sung Jun Jung, Seung Hwa Park, Dong-Il Yeom, Gu-In Jung, Ji-Sun Kim and SoonCheol Chung. 2015. Laser-induced thermoelastic effects can evoke tactile sensations . Scientific Reports 5, 11016. DOI: http://dx.doi.org/10.1038/srep11016 ; Hojin Lee, Ji-Sun Kim, Seungmoon Choi, Jae-Hoon Jun, Jong-Rak Park, A-Hee Kim, Han -Byeol Oh, Hyung-Sik Kim, and Soon-Cheol Chung. 2015. Midair tactile stimulation using laser-induced thermoelastic effects: The first study for indirect radiation. In World Haptics Conference (WHC), 2015, 374-380. DOI: See http://dx.doi.org/10.1109/WHC.2015.7177741 ). Therefore, the ultra-short laser pulse emitted from the femtosecond laser is safer. Systems A and B induce plasma spots that are bright and have a less concentrated average output.

(Tactile experiment 2: Contact effect)
The inventor has conducted tests to explore what happens when plasma comes into contact with human skin. This test was performed using System A.

When the present inventor touches the plasma boxel of the aerial image with a finger, the plasma generates a shock wave at the same time as the contact, and the present inventor impulses the aerial image to the finger so that the plasma has some physical substances. I was able to feel. The touch sensation is based on the evaporation effect of the femtosecond laser pulse. It excises the surface of the skin and produces a shock wave. The sensation is crisp and sharp, resembling electrical stimulation, such as electrostatic discharge or coarse sand paper.

The inventor also noted that the contact between the plasma and the finger brightens the plasma. The difference in density between air and human skin changes the brightness of light. This effect is shown in FIGS. 14 (c) and 14 (g) and can be used to display contacts in interactive applications.

(Contact experiment 3: Perceptual threshold)
The present inventor conducted a study for evaluating the perceptual threshold of the shock wave of laser plasma with respect to the skin. FIG. 16 (a) shows the optical circuit setup used in this study. As shown in FIG. 16A, a femtosecond laser light source 1610 with adjustable power settings emits a laser pulse 1612 towards an objective lens 1660 that focuses the laser pulse at a focal point to induce plasma. Be done. Since it is difficult to measure the evaporation effect as a force (N), the threshold was measured with respect to the laser output power (W). The laser output power was set to 0.05, 0.10, 0.13, or 0.16 W. The lowest power was limited by the femtosecond laser source used, and the highest power was determined by preliminary safety tests.

Seven subjects participated in this study (mean age 22.5 years, 5 men, 2 grants). Subjects were asked to touch a plasma-induced femtosecond laser with their right index finger. Eight trials were performed per subject. In each trial, subjects touched 10 plasma dots and asked if they felt anything on their index finger. The order of the output power settings used to generate the plasma dots was randomly determined, and each output power setting was repeated at least once. Subjects wore blindfolds to exclude visual information and headphones to exclude auditory information to reproduce white noise.

The results are shown in FIG. 18 (a). The perception rate according to the number of trials is the ratio of the number of trials in which the subject felt stimulation to the number of trials of each laser power. The 50% threshold seems to be 0.03 to 0.04 W. Subjects confidently felt the stimulus at 0.16 W (ie, greater than 90%).

Tactile feedback is possible even if no aerial plasma is generated. The shock wave is generated on the surface of the skin by a focused laser that does not have sufficient power to generate plasma in the air. This shock wave results from skin ablation.

(Tactile experiment 4: Pattern detection)
The present inventor conducted an experiment to test whether the subject was able to distinguish the spatial pattern formed by the laser plasma. FIG. 16B shows the optical circuit setup used for this verification. As shown in FIG. 16B, the femtosecond laser light source 1610 emits a laser pulse 1612 to a galvano scanner unit 1630 capable of scanning a programmed pattern. The objective lens 1660 focuses the pulses for inducing plasma dots that form a programmed pattern.

FIG. 19 shows an example of a pattern formed by repeated galvanoscans of laser plasma. Two spatial patterns (dots and lines) were used in this experiment. Subjects were asked to touch the plasma pattern with their right index finger. The subject is the same as the subject of the tactile experiment 3. Eight trials were performed for each subject. Each trial included contact with 10 plasma patterns and a question of the pattern felt on the index finger. Plasma patterns were randomly generated and each plasma pattern was repeated at least once. Subjects wore blindfolds to exclude visual information and headphones to exclude auditory information to reproduce white noise.

The results are shown in FIG. The combined results show fluorescence in which the subject was able to distinguish between the two patterns but gave the opposite answer. The accuracy rate will improve once the pattern is recognized. However, some subjects could not distinguish all patterns. In addition, two types of trends emerged. One is the ambiguous group and the other is the "bias vs. line" group.

(Tactile experiment 5: Crossfield effect)
Ultrasonic tactile feedback has been closely studied for many years. Ultrasound Tactile Feedback ( Takayuki Hoshi, Masafumi Takahashi, Takayuki Iwamoto, and Hiroyuki Shinoda. 2010. Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Transactions on Haptics 3, 3, 155-165. DOI: http: // dx .doi.org/10.1109/TOH.2010.4 ; Tom Carter, Sue Ann Seah, Benjamin Long, Bruce Drinkwater, and Sriram Subramanian. 2013. Ultrahaptics: Multi-point mid-air haptic feedback for touch surfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, ACM, New York, NY, USA, UIST '13, 505-514. DOI: http://dx.doi.org/10.1145/2501988.2502018 ; Seki Inoue, Koseki J. Kobayashi- Kirschvink, Yasuaki Monnai, Keisuke Hasegawa, Yasutoshi Makino, Hiroyuki Shinoda. 2014. HORN: The hapt-optic reconstruction. In ACM SIGGRAPH 2014 Emerging Technologies, ACM, New York, NY, USA, SIGGRAPH '14, 11: 1. DOI: http://dx.doi.org/10.1145/2614066.2614092 ) is highly programmable for the use of ultrasonic phased arrays. Ultrasound tactile feedback has relatively high spatial resolution compared to other spatial haptic feedback techniques and is limited by wavelength (8.5 mm at 40 kHz ultrasound). Higher frequency ultrasound (ie shorter wavelengths) is not suitable for tactile feedback due to absorption loss in air. Another constraint is the weakness of the stimulus, which is insufficient to reproduce impulses such as the moment of contact. The maximum force generated by an 18x18 array can be as low as 16 mN [ Takayuki Hoshi, Masafumi Takahashi, Takayuki Iwamoto, and Hiroyuki Shinoda. 2010. Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Transactions on Haptics 3 , 3, 155-165. DOI: http://dx.doi.org/10.1109/TOH.2010.4 ]. And to get more power, you need a bigger array. [ Keisuke Hasegawa and Hiroyuki Shinoda. 2013. Aerial display of vibrotactile sensation with high spatialtemporal resolution using large-aperture airborne ultrasound phased array. In World Haptics Conference (WHC), 2013, 31-36. DOI: http://dx.doi .org / 10.1109 / WHC.2013.6548380 ]. Ultrasonic haptics are based on acoustic radiation pressure, which presses against the surface of the skin, not vibration. It can be applied to the skin for a long time, but it is relatively weak (10-20 mN). The sensation resembles a laminar air flow within a small area.

We use the dull tactile sensation of the sound field to improve the tactile sensation and reduce the stinging sensation that the subject feels when touching the femtosecond laser-induced plasma. We examined whether the sharp tactile sensation of the field could be enhanced. The two fields of laser-induced plasma are physically independent of each other and can therefore be applied at the same location and time and can be mixed into the skin as elastic waves and / or as neural signals within the nervous system. For example, a laser field simulates the first contact between the skin and a virtual object, after which the ultrasonic field creates a continuous contact between them.

We conducted a series of experiments to investigate the tactile sensation when a femtosecond laser beam field was combined with an ultrasonic sound field. We hypothesized that combining two fields of different physical quantities would bring about not only the superposition effect proposed above, but also synergistic effects such as sensory modification.

FIG. 21 shows an exemplary embodiment of the ultrasonic transducer array system 500 of the present invention. The system 500 includes a system controller 510 and one or more ultrasonic phased arrays 520. Each phased array 520 includes two circuit boards 521 and 525. The first circuit board is an array 525 of ultrasonic transducers 526. The second circuit board includes a drive circuit 521 that drives the ultrasonic transducer 526. The two circuit boards (hence the oscillator array 525 and the drive circuit 521) are interconnected.

As shown in FIG. 21, the ultrasonic transducer array 525 includes several hundred ultrasonic transducers 526 arranged in a grid pattern. Each ultrasonic transducer 526 is individually controlled with sufficient time delay or sufficient phase delay. These time delays or phase delays are specified by the system controller 510 and applied by the drive circuit 521. In this way, each array 525 of the ultrasonic transducer 526 can generate ultrasonic waves of various distributions.

矩形振動子アレイから生成される超音波の空間分布は、ｓｉｎｃ関数のような形状であることが理論的および実験的に示されている［Takayuki Hoshi, Masafumi Takahashi, Takayuki Iwamoto, and Hiroyuki Shinoda. 2010. Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Transactions on Haptics 3, 3, 155-165. DOI: http://dx.doi.org/10.1109/TOH.2010.4]。長方形配列の辺に平行なメインローブの幅ｗは式（１４）で表される。

ここで、λは波長であり、Ｒは焦点距離であり、Ｄは長方形アレイの一辺の長さである。この数式は、空間分解能とアレイサイズとの間にトレードオフがあることを意味する。
(I, j) the time delay Delta] t _ij of the oscillator 526 of the transducer array 525 corresponding to the -th is given by equation 13.

The spatial distribution of ultrasonic waves generated from a rectangular oscillator array has been theoretically and experimentally shown to be shaped like a sinc function [ Takayuki Hoshi, Masafumi Takahashi, Takayuki Iwamoto, and Hiroyuki Shinoda. 2010 . Noncontact tactile display based on radiation pressure of airborne ultrasound. IEEE Transactions on Haptics 3, 3, 155-165. DOI: http://dx.doi.org/10.1109/TOH.2010.4 ]. The width w of the main lobe parallel to the sides of the rectangular arrangement is expressed by the equation (14).

Here, λ is the wavelength, R is the focal length, and D is the length of one side of the rectangular array. This formula means that there is a trade-off between spatial resolution and array size.

ここで、ｆ_ｐは、式１３に基づいて生成された超音波焦点ｋであり、ｐは、音圧であり、ｔは時間である。 The ultrasonic transducer array system 500 can be controlled to generate a distribution of ultrasonic waves that form a haptic image. Haptic image _{H i} is the time sum of the focal (formula (15)).

Here, f _p is the ultrasonic focal point k generated based on the equation 13, p is the sound pressure, and t is the time.

Referring to FIG. 21, the drive circuit 521 includes a USB interface circuit 522, a field programmable gate array FPGA 523, and a driver 524 (not shown).

As shown in FIG. 21, the system controller 510 controls the ultrasonic transducer array 525 under the direction of the control application 512 to create a sound field generated by one or more ultrasonic transducer arrays 525. Brings the desired change. The system controller 510 can be a PC. The system controller 510 controls each ultrasonic phased array 520 via the USB cable 530.

In one embodiment according to the invention, the control application 512 is developed in C ++ on a Windows® operating system. The system controller 510 transmits necessary data including the X-coordinate, Y-coordinate, and Z-coordinate of the focal point and the required output intensity of the ultrasonic beam to the drive board 521. The drive circuit 521 receives this data using the USB interface 522. Next, the phase calculator 527 of the FPGA 523 calculates an appropriate time delay (or phase delay) of each ultrasonic transducer 526 of the ultrasonic transducer array 525 based on the equation (13) or the equation (15). The signal generator 528 then bases each oscillator in the oscillator array 525 based on the beam intensity data provided by the system controller 510 and the time delay (or phase delay) calculated by the phase calculator 527. Generates a drive signal for. The drive signal is transmitted to the oscillator 526 of the oscillator array 525 via the driver's push-pull amplifier.

Correction of the time delay (or phase delay) relative to the drive signal 529 applied to each oscillator 526 is performed to change the distribution of the sound field generated by one or more ultrasonic phased arrays 525. NS. The output intensity of each oscillator 526 is varied using pulse width modulation (PWM) control of the drive signal 529 applied to the oscillator.

(Tactile experiment 5a: Tactile threshold of ultrasonic waves)
The present inventors have carried out verification for evaluating the tactile threshold of the acoustic radiation pressure excited by focused ultrasonic waves. FIG. 16 (c) shows the basic setup used for this verification. As shown in FIG. 16 (c), the ultrasonic phased array 1680 is used to generate a palpable sound field 1682 at contact point 1602.

The direct current output of ultrasonic waves is imperceptibly weak, but human skin has sensory receptors that respond to vibrations. Specifically, the Pasina globules (PC) and Meissner globules (RA) existing in the epidermis layer have frequencies of 10 to 200 Hz (peak of 10 to 50 Hz) and 40 to 800 Hz (peak of 200 to 300 Hz), respectively. Responds to range vibrations ( SJ Bolanowski, Jr., GA Gescheider, RT Verrillo, and CM Checkosky. 1968. Four channels mediate the mechanical aspects of touch. The Journal of the Acoustical Society of America. 84, 5, 1680-1694 . ). Therefore, in this study, we examined the application of vibrating and tactile stimuli modulated by square waves of 200 and 50 Hz to the index finger.

The diameter of the ultrasonic focus is about 20 mm. Since this is larger than the width of the index finger, the force acting on the index finger is slightly less than the output setting of the ultrasonic phased array. The output was set to one of 14 values centered on the threshold estimated by the preliminary experiment to determine the output power value perceived by the participants.

The subjects were the same as the participants in Tactile Experiments 3 and 4. Subjects were asked to use the index finger of their right hand to touch an ultrasonic field configured to vibrate simultaneously at 50-200 Hz. 14 tests were performed per subject. Each test included touching the ultrasonic field and asked each subject if they felt something on their index finger. The order of the output settings used to generate the ultrasonic field was randomized, and each output setting was used once. Subjects wore blindfolds to exclude visual information and headphones to exclude auditory information to reproduce white noise.

Experiments were performed using a preferred embodiment (referred to herein as "system C") based on the configuration of the ultrasonic transducer array system shown in FIG. System C will be described later.

With reference to FIG. 21, system C includes an ultrasonic phased array 525 with a resonant frequency of 40 kHz. The focal position is digitally controlled by a resolution of 1/16 of the wavelength (about 0.5 mm with 40 kHz ultrasound) and can be refreshed at 1 kHz. The 40 kHz phased array consists of 285 T4010A1 transducers 526 manufactured by Nippon Ceramic Co., Ltd. These transducers have a diameter of 10 mm and are arranged in an area of ^{170 × 170 mm 2.} When the focal length R = 200 mm, the sound pressure at the peak of the focal point is 2585 PaRMS (measured value). The size and weight of a single phased array are 19x19x5 cm ³ and 0.6 kg, respectively. The workspace is 30x30x30 cm ³ , but it can be expanded depending on the size of the phased array.

Referring to FIG. 21, system C further includes a drive circuit 521 with a USB interface 522, an FPGA 523, and a signal driver 524 (not shown). The USB interface 522 of the drive circuit can be implemented by a USB board that employs the FT2232H Hi-Speed Dual USB UART / FIFO integrated circuit manufactured by Future Technology Devices International of Glasgow, England. The FPGA 523 can be implemented on an FPGA board that includes an Altera III FPGA from Altera, San Jose, California. The signal driver 524 (not shown) of the signal can be implemented using a push-pull amplifier IC.

In the experiment, the tactile perception of vibration at 50 Hz and 200 Hz was tested. The result is shown in FIG. 18 (b). The perception rate is the ratio of the number of trials that the subject felt the stimulus to the number of trials of each ultrasonic output. The 50% thresholds for 200 Hz and 50 Hz stimuli are about 1.1 mN and 1.6 mN, respectively. Subjects reliably (ie, 90%) felt 200 Hz and 50 Hz stimuli at approximately 1.6 mN and 2.4 mN, respectively. In the field of tactile sensation, it is well known that tactile sensitivity is high to a stimulus of about 200 Hz, and our results are consistent with this knowledge.

(Tactile experiment 5b: Crossfield effect)
We have conducted a verification to evaluate the perceptual threshold of laser plasma for shock waves under the preload of ultrasonic vibrational tactile stimuli weaker than the perceptual threshold. There are two possible effects of ultrasound on laser haptics. One is a masking effect that improves the perceptual threshold of the laser plasma, and the other is a stochastic effect that reduces it.

Nine subjects participated in this study (mean 21.6 years, 4 females, 5 males). Subjects were asked to use the index finger of their right hand to touch the femtosecond laser-induced plasma. The laser output power was set to 0.05, 0.10, or 0.15 W. The modulation frequency of the ultrasonic waves was 200 Hz or 50 Hz to stimulate the PC channel and RA channel, respectively. There are 24 trials per subject. In each trial, the subject was asked if he had touched up to 10 plasma dots and felt something on his index finger. The combination of laser output and ultrasonic frequency was randomized and each laser output and each ultrasonic frequency was repeated at least 4 times in each trial. The ultrasonic stimulus was adjusted to be just below the perceptible force for each frequency and subject. Subjects wore blindfolds to exclude visual information and headphones to exclude auditory information to reproduce white noise.

The experiment was carried out using a preferred embodiment based on the system configuration shown in FIG. 1 (referred to herein as "system D"). System D combines the ultrasonic transducer array system of System C with an optical circuit system similar to that of System A.

As shown in FIG. 17, the optical circuit system configuration of the system D includes a femtosecond laser light source 310 manufactured by Coherent. The femtosecond laser light source 310 has a center wavelength of 800 nm, a repetition frequency of 1 kHz, and a pulse energy of 1 to 2 MJ. The femtosecond laser light source 310 is configured to emit a laser pulse of 40 fs. System D is further equipped with SLM (XB267 LC-SLM) manufactured by Hamamatsu Photonics. This SLM has a resolution of 768 × 768 pixels, a pixel size of 20 × 20 μm ² , and a response time of 100 ms. This SLM is configured to generate a Fourier CGH used for parallel optical access. CGH is derived from the Optimal Rotation Angle (ORA) method. System D includes a setup that includes a Canon GM-1010 as the galvano scanner unit 330 and a 3D position scanner that uses the Optotune EL-10-30 as the varifocal lens unit. These devices are operated by applications created using C ++. While workspace is 2X2x2cm ^3, by using an angular range larger lens extensible the optical scanner can be enlarged.

System D's ultrasonic phased array can roughly generate tactile images (spatial resolution is only 16 mm, twice the wavelength). However, the generated tactile image can cover a large area (about 30 cm) and the radiation pressure is sufficiently strong (16 mN). The femtosecond laser system of System D can accurately generate tactile images (spatial resolution 1 μm). However, the generated tactile image can only cover a small area (up to 2 cm). The overlapping area of these lasers and the ultrasonic haptics workspace is 2x2x2 cm ³ .

System D is controlled using a PC and all programs are coded in C ++. The PC is connected to an ultrasonic phased array, an SLM, a galvano scanner unit, and a varifocal lens unit. To monitor the interaction, connect the microscope camera to the system via a USB link to the PC. The ultrasonic phased array, the galvano scanner unit, and the varifocal lens unit each move along different threads and synchronize when a new drawing pattern is input. The user input is captured at 60 Hz and the SLM is connected to the computer as an external display.

In the optical system, the PC directly sets the coordinates and controls the drive mirror, the lens, and the SLM. In the acoustic system, the PC transmits data to the FPGA, including the coordinates of the focal position and the output. When the FPGA receives the data, it calculates an appropriate time delay for each oscillator based on the equations (13) and (15), and generates a drive signal. The drive signal is sent to the transducer via the amplifier. Modification of the time delay calculation algorithm changes the distribution of the acoustic potential field. The output changes depending on the pulse width modulation (PWM) control of the driving force.

The results are shown in FIG. 18 (d). Here, "laser only" is the same as in FIG. 18 (a). The results show that ultrasonic fields weaker than the perceptual threshold affect the perception of laser shock waves. The 50% perceptual threshold of laser haptics with imperceptible ultrasonic preload is about 0.15 W. This is about five times the "laser only" approach. (See FIG. 18 (d), which compares the trend lines corresponding to 200 Hz and 50 Hz with the trend lines corresponding to "laser only").

The results demonstrate that the two fields may overlap and that the combination of fields has a synergistic effect on tactile perception. In addition, the results support a masking effect, that is, ultrasonic waves reduce human sensitivity to laser plasma. The sound field affects the tactile sensation of laser haptics. This means that ultrasonic preload makes laser haptics less surprising and less painful. Superposition in this area also provides advantages such as multi-resolution tactile images.

For aerial interaction, the volmetric display has two requirements. Volmetric displays should be safe and accessible. Experiments have shown that a system configured as System D can provide safe and accessible tactile interactions.

(Tactile Application 1) Tactile Interface According to another embodiment of the present invention, an interactive user interface is provided. Experiments have shown that the effects of human contact on plasma are detectable or can cause content changes. The femtosecond laser-induced plasma produces a shock wave that is safe to contact and becomes brighter when it comes into contact with an object. FIG. 14 (c) shows the effect of the interaction between the heart and the object generated in the air. FIG. 14 (f) shows a light spot that turns into a "jewel" after contact with the ring. FIG. 14 (g) shows the direct interaction between the bright spot and the finger. Therefore, plasma-based aerial displays can be transformed into interactive aerial display systems by adding cameras or other sensors.

Sensors, such as cameras or photodetectors, can be used to detect interactions between the plasma and the user. Tactile sensations can also be used, for example, to generate aerial checkboxes. FIG. 10B shows the interaction between the user and the aerial image. The femtosecond laser field is used to generate the aerial button graphic 12. When the user 10 touches the aerial button graphic 12, the user 10 can sense the shock wave 14 caused by the contact with the plasma acting as haptic feedback. The contact also causes a change in the brightness of the plasma. This can be detected by a camera or other optical sensor coupled to the processor. This detected change can be registered as an indication that the aerial button graphic has been selected. The control system can control the laser field to produce different aerial button graphics 16 that visually indicate that the aerial button graphic has been selected as ancillary visual feedback.

(Tactile application 2: Multi-resolution haptics for VR)
Other distributions of sound fields that can be generated according to the present invention include sound fields having any shape, including any 3D shape. For example, one or more ultrasonic phased arrays surrounding a workspace can be used to generate standing waves of various shapes to provide a sound field with any shape. According to the present embodiment, a desired three-dimensional ultrasonic distribution can be generated by ultrasonic calculation holography using a plurality of phased arrays using the formula (2), and a desired CGH can be formed. When generating the standing wave by using a plurality of arrays, CGH U _r to be generated by each phased array is dependent on its spatial position relative to other phased array. For each phased array, in order to obtain a phased array of U _h, it should rotate the CGH U _r according to the relative position of the phased array. The desired 3D ultrasonic distribution is finally obtained by superimposing the 3D ultrasonic distributions provided by each ultrasonic phased array.

ここで、Ｕ_ｒは、式（１）によって与えられるレーザ焦点位置を示し、ｔは期間を示し、ｐはレーザ強度を示す。 The laser-guided tactile image is given by the combination of the SLM image and the galvano scanner unit. The haptic image Hi is the sum of the time series of focal points and is given by Eq. (16).

Here, _Ur indicates the laser focal position given by the equation (1), t indicates the period, and p indicates the laser intensity.

According to other embodiments of the present invention, multiple resolution haptics for virtual reality are provided. The sound field can be used to present simple three-dimensional tactile images and general tactile regions. Lightfields can be used for detailed tactile images. In AR / VR, the perimeter (single) of an object can be represented by ultrasonic waves, and the internal structure and / or display (details) can be represented by a laser.

FIG. 23 shows a series of photographs of an augmented reality system used to present the location of a tumor in a virtual three-dimensional model of the heart. The AR marker 504 is used to match the coordinates between the camera view and the 3D object. The low resolution tactile image from the sound field is used as a general guide to point to part of the 3D model. High-resolution tactile images from plasma are used to accurately represent the internal structure of the 3D model of interest. When a participant puts his finger in the virtual 3D model generated by System D, he first feels the outer tactile image corresponding to the perimeter of the virtual model. Participants then feel the laser plasma tactile sensation inside the virtual model. This plasma serves as an indicator to an accurate point (eg, a tumor in an organ, a pointer to a three-dimensional tactile map, etc.). This application extends traditional ultrasonic haptics with resolution and various tactile feedback patterns.

(Tactile application 3: Aerial Braille alphabet)
Traditional Braille alphabet displays are made of pin actuator arrays or other contact displays. Conventional ultrasonic or air jet tactile displays cannot produce accurate, high-resolution tactile images. According to another embodiment of the present invention, an aerial Braille system is provided. System D is programmable to represent a small and accurate tactile image at any location in the air. These tactile images can be a collection of dots and dashes that can be used to generate a braille display. A series of generated plasma images representing alphanumeric characters from the Braille alphabet and their corresponding CGH are shown. Blind subjects no longer need to look for Braille sentences. A camera system integrated with System D can identify the position of a blind subject's finger. The sound field is used to present a general area of detailed tactile images. In addition, the sound field can be shaped to guide the visually impaired finger to a general area for detailed tactile images. Therefore, a blind subject can easily find a general area of a detailed tactile image. It will change the interaction with the Braille alphabet from "touch" to "come".

(Audio generation)
Plasma emits not only light and palpable shock waves, but also sound waves in the air. Plasma produces audible sound as a series of shock waves. We hypothesized that each plasma spot could be an ideal point source with flat frequency characteristics. Therefore, each plasma spot can be a speaker.

ここで、rは点音源の位置からの距離であり、tは時間であり、ｐ０は単位距離での音圧であり、ｋは波数であり、ωは音の角周波数である。空間分布に焦点を合わせるために、時間成分ｅ^−ｊωｔは計算で省略可能である。相対圧力値が解析に十分であるので、ｐ_０の値は１に等しいと仮定される。 The sound pressure pb (r) of a single-point sound source can be expressed by the following equation.

Here, r is the distance from the position of the point sound source, t is the time, p0 is the sound pressure at a unit distance, k is the wave number, and ω is the angular frequency of the sound. In order to focus on the spatial distribution, the time component e ^−jωt can be omitted in the calculation. It is assumed that the value of _{p 0} is equal to 1 because the relative pressure value is sufficient for the analysis.

We also hypothesized that multiple plasma spots generated at the same time could form a speaker array. A simulator for graphically designing the sound field generated by the plasma speaker array was developed based on Equation 17. The simulator has a graphical user interface as shown in FIG. 30 (a). This allows the user to enter the position of the sound source by selecting a point on the grid. Then, the sound wave radiated from the sound source is calculated, and the result is displayed as directivity. That is, the heat map shown in FIGS. 30 (b) to 30 (c) or the heat map shown in FIGS. 30 (d) to 30 (e).

ここで、ｒ_ｎは、目標位置（ｘ；ｙ；ｚ）とｎ番目のプラズマスポットとの間の距離である。総エネルギーは実際にはＮ個のプラズマドットの間で分割されたが、それら全てに対してｐ_０＝１を使用し、且つ、計算において相対値を得た。 As mentioned above, the distortion due to the optical Kerr effect changes the focal shape and results in filamentation. This effect can affect the directivity of the speaker array and must be taken into account when generating multiple plasma sources in free space. The filamentization effect was calculated as the sum of the plasma spots distributed along the elongated focal point. This model was formulated as follows.

Here, r _n is the target position is the distance between (x; z; y) and n-th plasma spot. The total energy was actually divided among the N plasma dots, but p ₀ = 1 was used for all of them and the relative values were obtained in the calculation.

We conducted a series of experiments to investigate the acoustic properties of aerial plasma induced by femtosecond pulsed lasers. In the experiment, the emitted sound is recorded by a microphone system that records at 192 kHz and 24 bits. Each microphone is a monaural microphone.

For each of the audio experiments 1-6, one or more of the following preferred embodiments were used.

FIG. 31 shows an exemplary embodiment of the system 600 for producing a multipoint plasma speaker according to the present invention. The system 600 includes a system controller 601, a femtosecond laser light source 610, two spatial light modulators 620, 625, and optical lenses 642, 644, and 646. The lens 646 functions as an objective lens. The system 600 can be used to generate a plasma sound source 690 at any position in the three-dimensional space. Beam size, polarization, and power are adjustable. For example, a polarizing beam splitter (PBS) can be used for such adjustments.

A preferred embodiment based on the optical system settings shown in FIG. 31 (referred to as "system E" in the present specification) is as follows.

System E includes a coherent femtosecond laser light source having a center wavelength of 800 nm, a repetition frequency of 1 kHz, and a pulse energy of 0.4 to 7 mJ.

System E further includes a system controller with a PC running the Windows® operating system. All programs are coded in C ++, which controls the operation of DMD SLMs.

System E includes two DMD SLMs, DLP4500, manufactured by Texas Instruments. They have a resolution of 1190x712, a frame rate of 4 kHz, and a pixel size of ^{7.6x7.6 μm 2.} The DMD SLM is controlled via the USB interface. The first DMD SLM is used to compensate for the wavelength dispersion. All mirror pixels of the first DMD SLM are uniformly controlled. The second DMD SLM is used for frequency modulation and parallel access for placing multiple plasma speakers in the air. The mirror pixels of the second DMD SLM are individually switched using pulse width modulation.

With respect to the system E, sound having a frequency up to 1 kHz (repetition frequency of the laser light source) can be emitted. Frequencies below 1 kHz are generated by subtracting extra laser pulses. Since the frame rate of the SLM is 4 kHz, it is possible to switch whether or not to control each laser pulse and send it to the target point. Fluctuation and frequency ranges can be improved by using faster laser sources and DMDs.

The lenses 642 and 644 have a focal length of 100 mm, and the lens 646 has a focal length of 40 mm.

FIG. 32 shows an exemplary embodiment of the system 700 for producing a multipoint plasma speaker according to the present invention. The system 700 includes a system controller 701, a femtosecond laser light source 710, a spatial light modulator 720, and an optical lens 742 that functions as an objective lens. The system 700 can be used to generate a plurality of plasma sound sources 790 at arbitrary positions in the three-dimensional space. Beam size, polarization, and power can be adjusted by using PBS.

A preferred embodiment (hereinafter referred to as “system F”) based on the optical system setup shown in FIG. 32 will be described.

System F includes a femtosecond laser light source manufactured by Coherent. A femtosecond laser light source manufactured by Coherent has a center wavelength of 800 nm, a repetition frequency of 1 kHz, and a pulse energy of 0.4 to 7 mJ.

System F further includes a system controller including a PC running the Windows® operating system. All programs are coded in C ++, which controls the operation of SLM.

System F includes LCOS-SLM manufactured by Hamamatsu Photonics. The LCOS-SLM has a resolution of 768 × 768, a frame rate of 10 Hz, and a pixel size of ^{20 × 20 μm 2.} The LCOS-SLM is controlled via a USB interface and a video graphic array (GVA) display interface.

With respect to the system F, sound having a frequency up to 1 kHz (repetition frequency of the laser source) can be emitted. Lower frequencies can be generated by subtracting excess laser pulses. Since the frame rate of the SLM is 10 Hz, a burst wave of a maximum of 5 Hz (10 ms in the case of silence) and 10 ms in the case of sound radiation of 1 kHz can be generated. Fluctuation and frequency resolution can be improved by using faster laser sources and SLMs.

The lens 742 has a focal length of 200 mm.

FIG. 33 shows an exemplary embodiment of the system 800 for producing a multipoint plasma speaker according to the present invention. The system 800 includes a system controller 801, a femtosecond laser light source 810, an SLM 820, an optical mirror and lenses 842, 844, and 846, a galvano scanner unit 830, and a microlens array 880. The microlens array functions as an objective lens.

A preferred embodiment (hereinafter referred to as “system G”) based on the optical system setup shown in FIG. 33 will be described.

System G includes a femtosecond laser light source manufactured by Coherent. This femtosecond laser source has a center wavelength of 800 nm, a repetition frequency of 1 kHz, pulse energy up to 2 mJ, and a pulse width adjustable between 30 fs and 100 fs.

System G further includes LCOS-SLM manufactured by Hamamatsu Photonics. The device can only modulate phases greater than 2 radians and has a resolution of 768 pixels x 768 pixels, ^{a pixel size of 20 x 20 μm 2} , and a response time of 100 ms.

System G further includes a galvano scanner unit. It includes a scanning head unit (Canon GH-315) driven by a scanner control board (Canon GB-501). The scanning head unit (Canon GH-315) has a beam diameter of 10 to 14 mm, a scanning angle of ± 0.17 rad, an error of less than 5 μrad, and a resolution of 20 bits. The scanning head covers an area of at least about 10x10 mm ^2. The scanner control board (Canon GB-501) controls the scan head unit and the laser unit to direct the laser beam to any coordinate in the XY plane. It has a standard PCI bus for interfacing with a PC and receives instructions from PC commands.

System G further comprises a system controller including a C ++ coded PC in which all programs running the Windows® operating system control the operation of the SLM, galvano scanner unit, and varifocal lens unit. .. The galvano scanner unit and the varifocal lens unit run along different threads and synchronize when a new drawing pattern is received. User input may be captured at 20 Hz. The control system further includes a USB microscope used to monitor the interaction between the optics and the display medium.

System G further includes a microlens array. The lens size of the microlens array is 4x4 mm ² , and the focal length is 38.24 mm.

The system G can create a plurality of sound sources at arbitrary positions. The SLM820 divides a single laser beam 812 into multiple beams 814 by CGH, and the galvanometer mirror 830 directs these beams to multiple focal points. Finally, the Microlens Array 880 focuses these beams into a sound source.

FIG. 34 shows an exemplary embodiment of a system 900 for producing a multipoint plasma speaker according to the present invention. System 900 includes a modulator 980 manufactured in place of SLM. The modulator is a microlens array covered with a print mask.

The manufactured modulator is a passive modulator that can be manufactured by printing a grayscale pattern on a transparent film using an inkjet printer. The laser pulse beam is scanned on a modulator manufactured using a galvano mirror, and the focal point is modulated by the energy absorption of a grayscale pattern. This method takes advantage of the high spatial resolution of inkjet printers.

A preferred embodiment (hereinafter referred to as “system H”) based on the optical system setup shown in FIG. 34 will be described.

The system H is similar to the system G except that the SLM 830 is replaced with an optical mirror 940 and the microlens array 880 is replaced with a manufactured modulator 980.

System H includes a manufactured modulator with a resolution of 300 dpi.

The galvanometer mirror 930 directs the laser beam 912 along a trajectory programmed through the grayscale pattern of the manufactured modulator 980. The laser beam 912 is attenuated by a grayscale pattern, and then the microlens array focuses the laser beam. Grayscale patterns are used to modulate the amplitude of the laser beam. The grayscale pattern can be derived according to the audio data for playback (a phonorecord-like method), or a code in which the galvanometer mirror directs the laser beam to a specific position in the coding pattern to produce a specific sound. It can be a conversion pattern (a method like a piano).

(Audio Experiment 1: Laser Power vs Sound Volume)
We conducted experiments to evaluate the relationship between plasma generation and the radiation that results from the speaker. The experiment determined the volume corresponding to the energy level. The experiment was performed using System A configured at 30 fs for a time average power output range of 0.05 to 1.60 W. The experiments were performed under energy per pulse in the range of 0.16 to 1.6 mJ. The experimental results are shown in FIG. 25 (a). The phase is the relative angle between the propagation direction of the laser beam 5 and the position of the microphone with respect to the plasma spot 2, as shown in FIG. 25 (b). The vertical axis shows the amplitude of the sound wave on a linear scale (not decibels). Brighter plasma spots tend to be accompanied by louder sounds.

(Audio experiment 2: Polar characteristics)
We conducted experiments to evaluate the polarity characteristics of a single sound source in order to consider the effect of laser filamentization on sonic applications. The polarity of a sound source is a measure of the directivity of a sound source at a short distance, whereas the directivity is generally a measure of the directivity of a sound source at a long distance. In this experiment and in the experiments below, the audio data was measured at short distances and the sound waves could interfere with each other over longer distances. Nevertheless, the terms polarity and directivity are similar for plots that indicate angles with respect to amplitude and are used interchangeably herein.

Directivity is determined by the placement of the sound source. Therefore, the difference between these devices in terms of directivity is how freely they can generate sound sources in the air.

We have experimented with three different focal length lenses (f = 40, 100 and 300 mm). The femtosecond laser light source is configured to have a pulse width of 30 fs, a light source output of 6.62 W, and a repeating pulse of 1 kHz (6.62 mJ / pulse).

FIG. 26 shows the polarity characteristics of a single focal point at various focal lengths for a laser beam propagating from left to right. Experiments have shown that filamentation of short focal length lenses is shorter than that of long focal length lenses. Experiments have also shown that filamentization has a directivity of sound radiation. It emits stronger sound in the vertical direction than in the parallel direction. This graph shows the filamentization characteristics that become more pronounced as the focal length increases.

Interference between sound waves emitted from sound sources creates a complex sound pressure distribution around them. A two-dimensional spatial map of sound pressure is used to describe these sources. However, when viewed from a distance, these sound sources look like a single directional sound source. Therefore, the angle, that is, the polarity characteristic, sufficiently represents the characteristic of a single sound source.

(Audio Experiment 3: Pulse Width vs. Sound Volume)
We conducted an experiment to investigate the relationship between the laser output and the resulting speaker volume for each pulse width setting. Experiments were performed using System A with pulse widths set to 30 fs and 100 fs and a laser output power range of 0.05 to 1.60 W. The generated sound was captured using the same microphone used in Audio Experiment 1.

The experimental results are shown in FIG. A plasma generated from a pulse width of 100 fs emits a slightly weaker sound than a plasma generated from a pulse width of 30 fs, but the difference is minimal under the power of the same laser source.

(Audio Experiment 4: Frequency Domain Characteristics)
We analyzed the frequency characteristics of the measured sound using the Fast Fourier Transform (FFT). 25 (a)-(b) show exemplary sonic shapes and frequency characteristics of a single focus. FIG. 25 (a) shows the recorded waveform in the time domain, and FIG. 25 (b) shows the calculated frequency spectrum. There are sharp peaks at 1 kHz intervals, which is expected because the repetition frequency of the femtosecond laser source is 1 kHz. Sharp pulses that repeat at 1 ms intervals in the time domain are converted into a wide spectrum sampled by a comb function at 1 kHz intervals in the frequency domain.

(Audio Experiment 5: Polarity Characteristics of Voxels Addressed Simultaneously by Microlens Arrays)
The inventors conducted experiments to determine whether multiple plasma sound sources could be generated in free space at the same time. Simultaneous addressing is important for the spatial distribution of the sound field, the polarity characteristics of the sound source, and the generation of directional speaker functionality. Simultaneously addressed plasma speakers have a smaller maximum amount of speakers than a single plasma speaker because energy is distributed between them. In this experiment, multiple sound sources were generated simultaneously from the microlens array.

The microlens array has a separate small lens. A microlens array is generally a glass plate having a surface having recesses arranged in a grid pattern. Each recess acts as a lens. In this experiment, a microlens array with a ^{4 x 4 mm 2 lens was used.}

When a microlens array is used as an objective, a single laser beam is split by a small lens and multi-access is achieved without SLM. For example, if the diameter of the laser beam is about 8 mm, the laser beam can pass through a group of four adjacent lenses, resulting in the formation of a focal point. FIG. 36A shows a microlens array 20 having a lens 22. The laser beam passes through the lens group indicated by the dark lens 24, which divides the laser beam into four beams. When the SLM is used with a microlens array, any lens on the microlens array is available for focal formation (see, eg, FIG. 35 (b)). As shown in FIG. 36 (b), the dark lens 24 shows a different irradiation pattern with respect to the laser beam.

The purpose of this experiment was to investigate the polar characteristics of multiple sound sources generated at the same time and to verify the simulator and hologram generator to compare the results. In order to consider the difference in spatially distributed plasma, an experiment was conducted using a ^{microlens array (4 × 4 mm 2) having a lens focal length of 38.24 mm.} The sound was recorded with monaural microphones located at various positions from -90 ° to 90 °. As shown in FIG. 36 (e), the laser pulse beam passes through the four lenses 24 and is divided into four laser pulse beams 21 that cause a plasma emission effect at the four focal points 26. Inductively coupled plasmas generate interacting sound waves to form a sound pressure distribution. It depends on the distribution of plasma. See FIGS. 1 and 2. 36 (c) and 36 (d) show the theoretical sound pressure distribution corresponding to the plasma distribution induced by the microlens array having the irradiation pattern shown in FIGS. 35 and 36, respectively. And 36 (b).

The result of the system using the microlens array as a modulator is shown in FIG. FIG. 44 shows the directivity characteristics of the 10 kHz, 20 kHz, 30 kHz, 40 kHz, 50 kHz, 60 kHz, 70 kHz, 80 kHz, and 90 kHz components. These measurements show the basic characteristics of frequency-dependent interference between laser-generated sources.

The possible plasma distribution is limited by the spacing of the microlenses. A continuous objective instead of a microlens array can generate sound sources at any arrangement and spacing.

(Audio Experiment 6: Polar Characteristics of Simultaneously Addressed Voxels by SLM)
The present inventors conducted experiments to evaluate the polar characteristics of a plurality of sound sources simultaneously generated from SLM and hologram. The purpose of this experiment was to compare the polar coordinate characteristics of a plurality of sound sources generated by using the holography technique used in the above-mentioned experiment in which the spatial position of the plasma was fixed by the microlens array described in the audio experiment 5. This experiment also aimed to verify the directivity / distribution simulator developed for sound field estimation and design. A focal length of 30 mm was used in this experiment. Sound was recorded using a monaural microphone located 90 ° towards the optical path. The plasma distribution from the SLM was circulated by changing the CGH in 10 ° steps. FIG. 32 shows the results and simulation. As a result, it was confirmed that the plasma distribution using SLM and CGH can be generated, and that the measured distribution verifies the simulation result.

System F was used for this experiment.

Two and four sources were generated at 0.5 mm intervals, but were too short for effective interference. This can be improved by using a larger diameter objective lens. As another method for increasing the spacing between sound sources, a microlens array can be used in combination with LCOS-SLM.

(Audio application)

(Audio application 1: 3D aerial audio speaker (spatial audio))
The laser-induced plasma can be manipulated and distributed at any position in the three-dimensional space. The plasma guided to the focal point of the ultrashort pulse laser generates an impulse-like shock wave. The shock wave is modulated and can be adapted to the spatial acoustic design. In this way, the plasma can be placed as an audio speaker at any position in the three-dimensional (3D) configuration in the real world. This spatial control feature extends the range of applications that can control sound distribution. The generation of 3D audio, along with the visual display, plays an important role in immersing users in communication and entertainment.

According to an exemplary embodiment of the present invention, the present inventors who provide a plasma-based aerial speaker have studied a method for controlling volume. Amplitude control is performed by adjusting the intensity (ie, power) of the laser pulse. For laser pulses of constant intensity, DMD SLM, LC-SLM, LCOS-SLM, manufactured modulators, or beam shutters can be used to attenuate the intensity of the laser pulse. Since the DMD SLM can control the number of pixels that are turned on, it can be used to control the amount of reflected laser pulses of the incident laser pulses. Since the LC type SLM can generate a plurality of focal points from a single laser pulse beam, it can be used to reduce the laser intensity by dividing the laser pulse beam. Grayscale glass can be used to attenuate the laser intensity because the amount of transmitted laser pulses among the incident laser pulses can be controlled. Grayscale glass is a manufactured modulator with pixel areas printed with various grayscale watermark images. The beam shutter can also attenuate the intensity of the laser pulse. Since the beam shutter can reduce the cross-sectional area of the laser pulse beam, it can be used to control the amount of transmitted laser among the incident lasers.

We have verified a method of frequency modulation. Frequency control is performed by adjusting the repetition rate of the laser pulse. For fixed frequency laser pulses, some pulses can be removed. This can be achieved by sending a laser pulse to the sound source position at the intended frequency. For example, if a 1 kHz pulsed laser source is used and a 500 Hz tone is desired, then only half of the pulses generated by the laser source should be directed to the source position.
In this way, half, one-third, one-fourth, etc. of the repetition frequency can be generated. A DMD SLM and / or galvano scanner can be used to subtract excess laser pulses from the series of pulses coming from the laser source. For example, the DMD SLM can be controlled to reflect only specific laser pulses from the pulse train. The galvano scanner unit can be controlled to direct only a specific laser pulse from the pulse train to the target position. As shown in FIG. 37 (c), the galvano scanner unit 85 is used to halve the frequency generated by alternately directing the laser beam pulses to the target position 82 and the end position 86. Since the galvano scanner unit is slower than the DMD SLM, the DMD SLM is preferred. A beam chopper or shutter can also be used to block a particular pulse from the pulse train. Faster laser sources and SLMs can be used to extend the range of frequencies of sound that can be generated.

Fourier computer-generated holography (CGH) can also be used to control the number of laser pulses sent to the source position per second by varying the distribution pattern. However, this requires an SLM with a high scanning speed.

Position control is performed by generating the focal point of the laser pulse beam at the target position. A three-dimensional scanner (eg, a combination of a galvano scanner and a varifocal lens) can be used. The galvano scanner unit scans the direction of the laser pulse beam in the X and Y directions in the lateral direction. The varifocal lens unit changes the focal length in the Z-axis direction. Also, Fourier computer-generated holography (CGH) can be used to place sound sources in free space.

Directivity control is performed by arranging one or more sound sources so that the radiated sounds interfere with each other so that the resulting wavefront is directed at the target. The plasma generated by the femtosecond laser pulse radiates electromagnetic waves (radio frequencies and light known as supercontinues) as well as broadband waves over a very wide frequency range, including sound waves (audible and ultrasonic waves). Waves with a wide spectrum coexist, but the frequency band can be controlled separately. When multiple plasma spots are spatially arranged in free space, it is possible to generate a "phased array" of light, electromagnetic waves, sound, and / or ultrasonic waves.

Although the phases of the wave sources are the same, the directivity of a specific frequency wave can be controlled by the arrangement of the wave source (laser focus) in order to create a specific interference wave pattern. The spatial position and distance between the plasma spots determine the time (phase) difference. Therefore, it is not a "phased array" in itself, but a "spatial array".

Computational phase modulation allows for complex placement of focal points in the air. A particular spatial array arrangement can be designed to control the interference pattern and to shape the wavefront to obtain directional control of the radiated sound.

The LC type SLM uses the derived CGH to generate multiple coherent focal points. In addition, the DMD SLM can control the amplitude of the laser pulse beam so that the resulting shock waves can interfere with each other. The CGH can be derived using a simulator for designing the sound wave distribution. After determining the placement of the sound source to produce the desired wave surface, the CGH is calculated for the LCSLM or DMD SLM based on the position of the sound source.

Microlens arrays can also be used. The individual small lenses divide a single laser beam into, for example, four sound sources at intervals of the microlens array. When spaced sufficiently apart, the four sound sources can generate sounds that interfere with each other to form a particular sound distribution pattern.

Longer filamentation emits stronger sound waves perpendicular to the direction of laser propagation. This makes it possible to determine a preferred laser direction for effectively delivering the sound wave to the target point.

The present inventors have investigated a method for reproducing audio data at a target position. One way to produce sound and voice is to use amplitude modulation. As shown in FIG. 40, a beam shutter can be used to control the power of individual pulses. To reproduce the 44.1 kHz sampling data, a minimum of 44.1 kHz is required for both the pulsed laser light source and the shutter.

This can also be achieved by using the harmonics of the impulse sound (eg, by using the configurations shown in FIGS. 38 and 39, or by using an ultrafast SLM). Using a microphone capable of measuring up to 100 kHz, it was confirmed that a wave having a frequency up to at least 96 kHz was emitted. Although multiple frequency components are emitted at the same time, the target frequency can be selected by controlling the directivity of each frequency. Pure tones can be difficult, but at least most extra frequencies can be removed in the target direction.

(Audio Application 2: Spatial Audio Virtual Reality)
Aerial plasma speakers can turn ordinary objects into audio media. The aerial plasma source can be superimposed on or adjacent to real-world objects. Therefore, any real-world object can be a sound source. For example, an aerial plasma sound source can be placed next to the mouth of the toy figurine to make the toy figurine appear to be a sound source. Spatial plasma can be manually relocated or automatically relocated by utilizing a camera system that can scan and map the environment around the plasma speaker workspace. You can also.

(Audio application 3: Spatial speaker: Aerial speaker without reality)
The spatial position and intensity of the plasma spot can be changed by changing the CGH. According to an exemplary embodiment of the invention, the laser-induced plasma can be generated anywhere and modulated to radiate audible waves as a point sound source such as a conventional speaker. The plasma can be induced at multiple points to function as a set of traditional surround sound speakers. For example, a plurality of laser-induced plasma spots are arranged indoors in a conventional surround sound configuration, and each plasma spot is modulated by a different audio signal.

According to an exemplary embodiment of the invention, the laser plasma speaker is configured to behave like a parametric directional speaker. Directivity is achieved by arranging multiple sound sources. The directivity is variable.

(Audio application 4: Spatial speaker array: Directional speaker)
The ultrasonic super-directional speaker can be simulated with a laser sound source. The laser focus emits ultrasonic waves. These ultrasonic waves are modulated based on audio data. The non-linearity of air demodulates these modulated ultrasonic waves to audible sound.

Alternatively, the use of SLM and computational phase modulation allows for complex placement of focal points in the air. A particular spatial array arrangement can be designed to control the interference pattern and shape the wavefront to obtain directional control of the radiated sound.

As shown in FIG. 41 (d), the focus can be placed to direct the sound wave to a particular target. As shown in FIG. 43, the focal point 62 causes the movement and orientation of the virtual object so that the sound 64 produced by the plasma induced at the focal point 62 appears to be radiated with respect to the movement and orientation of the virtual object. It can be continuously rearranged for tracking.

(Improvement of secure application)
Aerial plasma generation requires an instantaneous laser output of petawatts per square centimeter. Optical circuits need to be carefully developed and handled. When high intensity lasers are utilized, ionization can occur along the optical circuit. This limits the available laser power, as damage to the optics must be avoided. In addition, since plasma generation is a non-linear phenomenon, care must be taken when handling it. These issues need to be carefully considered to ensure the safety of the application. By using an SLM with higher reflection efficiency and increasing the time average laser power, a larger number of simultaneously addressed voxels can be generated.

To avoid unwanted ionization, an objective lens is used to focus low-power, high-intensity laser pulses at specific points in space to generate the plasma. When using an objective lens, there is a limit to the size of the workspace. The workspace is determined by the angular range of the galvano mirror and the depth range of the varifocal lens. However, the angular range of the galvano mirror depends on the aperture of the objective lens. An objective lens with a larger aperture allows for a larger angular range of the galvano scanner in the lateral direction, i.e. XY scanning.

According to another embodiment of the invention, safety can be further improved by increasing the scanning speed as a precautionary measure to prevent skin damage or to minimize discomfort.

In an exemplary embodiment, the volumetric display scans three-dimensional space very quickly. Therefore, no significant damage can occur because it does not stay at a particular point in space for a long period of time.

According to another embodiment of the invention, the safety of the interactive haptics is to adjust the target position of the plasma to avoid the possibility of serious damage by preventing further contact at the same point. Can be further improved by.

In an exemplary embodiment, a camera or sensor system can be used to monitor user activity in and around the workspace where plasma spots can be induced. As a precaution, the plasma voxel may be disabled or rearranged within 17 ms (single frame) after the camera or sensor system detects contact with the plasma voxel. This cutoff time is sufficiently shorter than the harmful exposure time of 2000 ms [ Yoichi Ochiai, Kota Kumagai, Takayuki Hoshi, Jun Rekimoto, Satoshi Hasegawa, and Yoshio Hayasaki. 2015. Fairy lights in femtoseconds: Aerial and volumetric graphics rendered by In ACM SIGGRAPH 2015 Emerging Technologies, ACM, New York, NY, USA, SIGGRAPH '15. DOI: http://dx.doi.org/10.1145/2782782.2792492 ]. As the number and energy of laser source iterations increases, faster camera recognition systems and faster scanning systems are needed.

(Immersive Virtual Reality Application-Composite Audiovisual Tactile System)
It is possible to combine visual, tactile, and audio applications. The 3D image generated in the air appears to speak to the user. The user can interact with these images via touch operations. The control rate for visual applications is 30 fps, which is a sufficient value for tactile applications. This rate is significantly lower than that of the audio application, so the visual application does not interfere with the audio application.

According to an exemplary embodiment of the present invention, there is provided a method of realizing a functional aerial audio speaker utilizing a femtosecond laser-induced plasma. Since the plasma induced in the focal point of the ultrashort pulse laser generates an impulse-like shock wave, the focal point can be distributed at any position in the three-dimensional space. The position of the focal point can be changed dynamically. Speakers generated in the air using a 7W femtosecond laser source have a wide frequency response in the range of 1kHz to 96kHz.

The experimental examples, experimental data, tables, graphs, plots, photographs, figures, and processing and / or operating parameters (eg, values and / or ranges) described herein are some of the possible operations of the present invention. It is intended to illustrate the conditions. The disclosed systems and methods are disclosed and are not intended to limit the scope of operating conditions for other embodiments of the methods and systems disclosed herein. In addition, the experiments, experimental data, computational data, tables, graphs, plots, photographs, figures, and other data disclosed herein feature effective embodiments of the disclosed systems and methods. Demonstrates various situations in which one or more desired results can be produced. Such operating plans and desired results are not limited to, for example, specific values of operating parameters, conditions, or results shown in tables, graphs, plots, diagrams, or photographs. Also includes the appropriate range to include or extend to them. Accordingly, the values disclosed herein include a range of values between any of the values listed or shown in tables, graphs, plots, figures, photographs and the like. In addition, the values disclosed herein are in tables, graphs, plots, figures, photographs, etc., as demonstrated by other values described or displayed in tables, graphs, plots, figures, photographs, etc. Includes a range of values either above or below listed. Also, the data disclosed herein establish one or more effective operating ranges and / or one or more desired results for a particular embodiment, but all such operating ranges. It does not have to be operational with. In addition, other embodiments of the disclosed systems and methods operate in other modes of operation and / or include exemplary experiments, experimental data, tables, graphs, plots, photographs, figures, and other data. Produces results other than those shown and explained with reference.

Other systems, settings, and parameters may be used in other implementations, which may provide the same or different results. Many modifications are possible and can be envisioned within the scope of this disclosure.

Having illustrated and described specific embodiments of the invention, it will be apparent to those skilled in the art that various other modifications and modifications can be made without departing from the spirit and scope of the invention. Accordingly, it is intended that all such modifications and modifications within the scope of the present invention are covered by the appended claims.

Claims (21)

Equipped with a femtosecond light source that produces a laser pulse beam
Equipped with a processor that produces computer-generated holograms
A spatial light modulator that modulates the laser pulse beam according to the computer-generated hologram.
A three-dimensional scanner optically connected to the spatial light modulator is provided.
The three-dimensional scanner directs a modulated laser pulse beam modulated by the spatial light modulator to one or more focal points in the air.
A lens for focusing the modulated laser pulse beam is provided .
The modulated laser pulse beam produces a palpable light field at one or more focal points.
Ru comprises an ultrasonic phased array to generate an acoustic field palpable around the one or more focal points,
Plasma generator. The tactile light field comprises a tactile image pattern.
The plasma generator according to claim 1. Equipped with a femtosecond light source that produces a laser pulse beam
Equipped with a processor that produces computer-generated holograms
A spatial light modulator that modulates the laser pulse beam according to the computer-generated hologram.
A three-dimensional scanner optically connected to the spatial light modulator is provided.
The three-dimensional scanner directs a modulated laser pulse beam modulated by the spatial light modulator to one or more focal points in the air.
A microlens array for focusing the modulated laser pulse beam.
Plasma generator. The modulated laser pulse beam induces a light emitting effect at one or more focal points.
The plasma generator according to any one of claims 1 to 3. A sensor for detecting a change in the brightness of the light emitting effect is provided.
The plasma generator according to claim 4. The three-dimensional scanner includes a galvano scanner and a varifocal lens.
The plasma generator according to any one of claims 1 to 5. The modulated laser pulse beam produces a palpable acoustic field at one or more focal points.
The plasma generating apparatus according to any one of claims 1 to 6. Equipped with a sensor to detect the position of an object,
The plasma generator according to any one of claims 1 to 7. An amplitude modulator that changes the intensity of the laser pulse beam according to an audio signal.
The plasma generator according to any one of claims 1 to 8. With steps to generate a femtosecond laser pulse beam
With steps to generate computer-generated holograms
A step of generating one or more modulated laser pulse beams by modulating the femtosecond laser pulse beam according to the computer-generated hologram.
A step of directing the one or more laser pulse beams to one or more focal points in the air .
It said one or more focus, by being disposed at a position corresponding to the computer generated hologram, that generates a directional acoustic wave,
Plasma generation method. A step of inducing a luminescence effect at one or more foci by focusing one or more modulated laser pulse beams at one or more foci.
The plasma generation method according to claim 10. A step of detecting a change in the brightness of the light emitting effect is provided.
The plasma generation method according to claim 11. Includes steps to generate a palpable light field at one or more focal points,
The plasma generation method according to any one of claims 10 to 12. Generates ultrasonic radiation pressure at least one of one or more foci and one or more foci.
The plasma generation method according to claim 13. It comprises a step of inducing the luminescence effect at a rate sufficient to generate an afterimage.
The plasma generation method according to claim 11. A step of determining a new combination of one or more focal points in the vicinity of one or more focal points.
The plasma generation method according to claim 11. The step comprises using the detected change as an input selection.
The plasma generation method according to claim 12. A step of deriving a light emitting effect that generates sound waves by directing one or more modulated laser pulse beams at one or more focal points.
The plasma generation method according to any one of claims 10 to 17. The sound wave is in either an audible frequency band or an ultrasonic frequency band.
The plasma generation method according to claim 18. A step of adjusting the intensity of the femtosecond laser pulse beam according to an audio signal.
The plasma generation method according to any one of claims 10 to 19. A step of weakening the femtosecond laser pulse beam.
The plasma generation method according to any one of claims 10 to 20.

Images