RU213521U1 - The device of an artificial vision system of the "dragonfly eye" type - Google Patents

Abstract

The utility model can find application in surveillance devices, robotics, devices for target detection, positioning, recognition and collision-free navigation of ground and aerospace vehicles, endoscopy instruments, digital cameras that mimic hemispherical compound compound eyes known in biology.

A device for an artificial vision system of the “dragonfly eye” type is proposed, consisting of a matrix of identical dielectric lenses evenly spaced on a thin hemispherical surface and receiving devices, each of which is located at the focus of the lens, while the receiving devices are configured to supply signals to a processing device that differs the fact that the lenses are made in the form of mesodimensional dielectric particles with a characteristic size of at least λ, where λ is the wavelength of the radiation used, with a relative refractive index of the material lying in the range from about 1.4 to 1.7, forming on their outer boundary with the opposite side of the incident radiation of the region with increased radiation intensity with transverse dimensions of the order of λ/3 - λ/4, and the receiving devices are located directly on the shadow surface of the particle in this region with increased radiation intensity.

The technical result consists in the development of a device for an artificial vision system of the "dragonfly eye" type, which has a high sensitivity of the receiving devices and a small thickness of the microlens - photodetector system. 2 ill.

Description

The utility model can be used in surveillance devices, robotics, devices for target detection, positioning, recognition and collision-free navigation of ground and aerospace vehicles, endoscopy instruments, digital cameras that mimic hemispherical compound compound eyes known in biology.

It is known that in arthropods, for example, in dragonflies, ants, bark beetles, flies, evolution has created a surprisingly complex class of imaging systems with a wide-angle field of view, low aberrations, high motion definition and infinite depth of field - compound eyes [A. Sergeev, A. Blagodatsky. Insects and bionics: mysteries of the visual apparatus // Priroda, No. 1, 2015; Guide to the physiology of the sense organs of insects / Ed. G. A. Mazokhina-Porshnyakova. M., 1983; Warrant, E. &Nillson, D.-E. Invertebrate Vision Ch. 1 (Cambridge Univ. Press, 2006); Dudley, R. The Biomechanics of Insect Flight: Form, Function, Evolution Ch. 5 (Princeton Univ. Press, 2000); Floreano, D., Zufferey, J.-C., Srinivasan, M. V. & Ellington, C. Flying Insects and Robot Ch. 10 (Springer, 2009).]. They are strongly convex hemispheres, divided into many thousands of small lenses - facets with the number of lenses from several hundred to several tens of thousands. The more facets, the higher the resolution of the optical system. Facet sizes typically range from about 10λ to 100λ, where λ is the wavelength of the radiation used. The lens (facet) has a very limited angle of view and "sees" only that tiny section of the object in front of the eyes, to which the continuation of the optical axis of this lens is directed. Since the lenses are closely adjacent to each other, and at the same time their axes diverge in a radial manner, the compound eye covers the object as a whole, and the image of the object turns out to be mosaic (that is, composed of many separate pieces). The consequence of this is that the compound eye can accurately and simultaneously render images of multiple objects in the field of view,

Thus, the main purpose of the facet is to collect electromagnetic radiation in a narrow cone and transfer it to the sensitive element. In technology, lenses and horn or reflector antennas can act as analogues of such devices. In this case, for example, the horn can have a larger opening cross section of a round shape or rectangular, or hexagonal, or arbitrary shape and with a maximum opening of about 5λ to 100λ, and the minimum opening is determined by the size of the sensitive area of the photodetector. Such a vision system allows you to see various objects simultaneously in all directions.

Microlens arrays are one of the main components of the next generation 3D imaging system because it has some good optical properties such as extremely large viewing angles, low aberrations and distortions, high temporal resolution and infinite depth of field.

It is known that, unlike vision systems based on fisheye lenses, the compound eye vision system has an almost infinite depth of field, i.e. this system allows you to see everything around, and everything, both close and far, with the ability to detect fast movements of objects [ZahraM.Bagheri, StevenD. Wiederman, BenjaminS. Cazzolato, Steven Grainger and David C. O'Carroll. Properties of neuronal facilitation that improve target tracking in natural pursuit simulations // The Royal Society Interface, Published: 06 July 2015, https://doi.org/10.1098/rsif.2015.0083].

A vision system based on an artificial "dragonfly eye" is known [Jun Tanida, Tomoya Kumagai, Kenji Yamada, Shigehiro Miyatake, Kouichi Ishida, Takashi Morimoto, Noriyuki Kondou, Daisuke Miyazaki, and Yoshiki Ichioka. Thin observation module by bound optics (TOMBO): concept and experimental verification // Applied Optics, 40(11), P.1806-P.1813], consisting of a matrix of identical microlenses located on a plane, a CCD camera and a screen system, located between the lenses and the CCD camera.

As a microlens, a dielectric lens with a diameter approximately equal to 500λ (at a wavelength of 500 nm) with a focal length of 1300λ, made of sheet Nippon glass, was used.

The disadvantage of the device is a small viewing angle due to the placement of the microlens array on a flat surface, the low sensitivity of the receiving devices due to the inefficient interaction of the receiving element with optical radiation in the lens focus area, the large thickness of the microlens system - the receiving device due to the large value of the focal length of the microlens.

Known hemispherical camera for compound eyes (SCECAM), which is similar to the eye of a fruit fly and consists of three consecutive subsystems: hemispherical compound eye, optical relay system and CMOS image sensor [Chengyong Shi, Yuanyuan Wang, Chenyang Liu, Taisheng Wang, Hongxin Zhang , Wuxia Liao, Zhijun Xu, Weixing Yu. SCECam: a spherical compound eye camera for fast location and recognition of objects at a large field of view // Optics Express, v.25, N 26, 25 Dec. 2017, 32333].

By introducing an intermediate optical relay system, the curved focal plane after the hemispherical compound eye can be converted and projected onto the flat focal plane of the image sensor.

A hemispherical array of 4400 microlenses with a diameter of about 500 μm (about 1000λ) and a focal length of about 1.28 mm, made of a material with a refractive index of N=1.49, was located on a hemisphere with a diameter of 40 mm. Beam width of each lens is 2.4 degrees. The physical size of the SCECam is 40 mm × 40 mm × 80 mm.

The advantage of the vision system is a large viewing angle (up to 122.4°) with 4400 lenses and receiving elements, which allows you to detect and find fast moving objects at a very high speed.

Known omnidirectional camera, implemented by superimposing CMOS image sensors on the surface of a spherical structure [Hossein Afshari, Laurent Jacques, Luigi Bagnato, Alexandre Schmid, Pierre Vandergheynst, Yusuf Leblebici. The PANOPTIC Camera: A Plenoptic Sensor with Real-Time Omnidirectional Capability // J. Sign Process 2013, 70, pp. 305-328, DOI 10.1007/s11265-012-0668-4], consisting of identical image sensors located on a hemisphere. The diameter of the hemisphere, on which about 30 receiving elements are placed, is approximately 129 mm.

A vision system is known [Hansong Zeng, Hassan Borteh, and Yi Zhao. A Bioispired 3D Artifical Compound Eye with Focus-Tunable Single Lenses // 15th International Conference on Miniaturized Systems for Chemistry and Life Sciences October 2-6, 2011, Seattle, Washington, USA], consisting of many liquid microlenses located on a hemispherical surface and enclosed in compatible membranes, a flexible base diaphragm, an intelligent microfluidic system and radiation receiving elements.

The microlens array is made from widely available PDMS material using a standard soft lithography process. The microlenses are filled with a refractive substance - mineral oil (SigmaAldrich®, refractive index equal to 1.33, at a wavelength of λ=532 nm). The focusing power of a single lens is controlled by adjusting the pressure that drives the refractive medium.

The advantage of the device is the ability to quickly adjust the focal length of the microlenses.

The disadvantage of hemispherical vision systems is the low sensitivity of the receiving devices, due to the inefficiency of the interaction of the receiving element with optical radiation in the focus area of the lens, the complexity of placing and aligning the receiving matrix in the lens foci, the large thickness of the microlens - receiving device system.

As a prototype, a hemispherical device for imaging the optical wavelength range was chosen, consisting of a matrix of identical dielectric convex lenses on a cylindrical base evenly spaced on a thin hemispherical surface and photodetectors (silicon photodetector (photodiode)), each of which is located at the focus of the lens, while the signals from receiving devices were fed into the processing device [YoungMinSong, YizhuXie, ViktorMalyarchuk, JianliangXiao, InhwaJung, Ki-JoongChoi, ZhuangjianLiu, HyunsungPark, ChaofengLu, Rak-HwanKim, RuiLi, KennethB. Crozier, Yonggang Huang & John A. Rogers. Digital cameras with designs inspired by the arthropod eye // Nature, 2 MAY 2013, Vol 497, 95-99, doi:10.1038/nature12083.].

The device used about 180 convex dielectric lenses with a diameter of 0.4 mm made of elastomeric poly(dimethylsiloxane) (PDMS, refractive index n < 1.43) on a cylindrical base 0.4 mm high with a single element beamwidth of about 9.7 degrees and evenly spaced on a hemispherical thin surface with a radius of 6.96 mm. Silicon photodiodes were located at the focus of lenses with a focal length of 1.35 mm, the signals from which were fed into the processing device.

The advantage of the device is a wide field of view (about 160 degrees) and the infinite depth of the depicted space.

The disadvantage of the device is the low sensitivity of the receiving devices, due to the inefficiency of the interaction of the receiving element with optical radiation in the focus area of the lens, the complexity of placing and aligning the photodetector matrix in the lens foci, the large thickness of the microlens-receiving device system.

The objective of the claimed utility model is to eliminate these shortcomings, namely, the development of a vision system device similar to the "dragonfly eye", having a high sensitivity of the receiving devices, a small thickness of the microlens - photodetector system.

The problem is solved by the fact that the device of the vision system, similar to the “dragonfly eye”, consisting of a matrix of identical dielectric lenses evenly spaced on a thin hemispherical surface and receiving devices, each of which is located at the focus of the lens, while the signals from the receiving devices were fed into the processing device, what is new is that the lenses are made in the form of mesodimensional dielectric particles with a characteristic size of at least λ, where λ is the wavelength of the radiation used, with a relative refractive index of the material lying in the range from about 1.4 to 1.7, forming on their outer boundary on the opposite side of the incident radiation, there are regions with increased radiation intensity with transverse dimensions of the order of λ/3 - λ/4, and receiving devices are located directly on the shadow surface of the particle in this region with increased radiation intensity.

The utility model is illustrated by drawings.

Figure 1 shows a diagram of the device of the vision system similar to the "eye of a dragonfly".

On FIG. Figure 2 shows examples of the formation on the outer boundary of a dielectric mesodimensional particle on the opposite side of the incident radiation of a region with increased radiation intensity with transverse dimensions of the order of λ / 3 - λ / 4 for particles with a characteristic size of 3λ of spherical shape (a), cylindrical shape (b), cuboid (c), circular cone (d), truncated cone (e), pyramid (f).

Designations: 1 - mesodimensional dielectric particle, 2 - photodetector, 3 - processing device, 4 - region with increased radiation intensity with transverse dimensions of the order of λ/3 - λ/4, 5 - hemispherical surface.

It is known that the fundamental Rayleigh criterion for the resolution of optical systems is that the minimum size of a distinguishable object is somewhat less than the wavelength of the radiation used and is fundamentally limited by the diffraction of this radiation [Born M., Wolf E. Fundamentals of optics. - M.: Mir, 1978]. The impossibility of focusing light in free space into a spot with sizes less than a certain diffraction limit also follows from a relation like the Heisenberg uncertainty relation [Minin I.V., Minin O.V. Experimental verification 3D subwavelength resolution beyond the diffraction limit with zoneplate in millimeter wave // Microwave and Optical Technology Letters, Vol. 56, no. 10, October 2014, 2436-2439].

By overcoming the diffraction limit refers to the focusing of radiation into a spot with a size smaller than that of the Airy spot [Born M., Wolf E. Fundamentals of optics. - M.: Mir, 1978].

There are various ways to overcome the diffraction limit in optics, for example, using the “photonic nanojet” effect (for example, see A.Heifetz et al. Experimental confirmation of backscattering enhancement induced by a photonic jet // Appl.Phys.Lett., 89, 221118 (2006)). The transverse size of a photonic nanojet is 1/3 ... 1/4 of the radiation wavelength, which is less than the diffraction limit of a classical lens.

The term photonic jet was introduced in 2004 by [Z.G. Chen, A. Taflove, and V. backman. Photonic nano jet enhancement of back scattering of light by nano particles: a potential novel visible-light ultramicroscopy technique // Optics Express, vol. 12, pp. 1214-1220, Apr 2004] to describe a focusing region with spatial dimensions less than the diffraction limit and occurring near a dielectric cylinder or sphere when illuminated by optical radiation. It was pointed out that the field enhancement observed in a photonic jet cannot be predicted from the geometrical optics approximation.

At the same time, it is possible to form local areas of electromagnetic energy concentration near the surface of mesodimensional dielectric particles using particles of various shapes, for example, in the form of a sphere, a truncated sphere, a polygonal prism, a polygonal pyramid, a cuboid, a circular cone, a cylinder, etc. when they are irradiated with an electromagnetic wave with a plane wave front, etc. [I.V.Mininand O.V.Minin. Diffractive optics and nano photonics: Resolution below the diffraction limit, Springer, 2016 http://www.springer.com/us/book/9783319242514#aboutBook; I.V. Minin, O.V. Minin. Photonics of Isolated Dielectric Particles of Arbitrary Three-Dimensional Shape - a New Direction in Optical Information Technologies // Vestnik NGU. Series: Information Technologies, 2014, No. 4, P. 4-10; Minin I.V., Minin O.V., Kharitoshin N.A. Localized high field enhancements from hemispherical 3D mesoscale dielectric particles in the reflection mode // 16th International Conference of Young Specialists on Micro/Nanotechnologies and Electron Devices June 29 - July 3, 2015; V. Pacheco-Pena, M. Beruete, I. V. Minin, O. V. Minin. Terajets produced by 3D dielectric cuboids // Appl. Phys. Lett. V.105, 084102 (2014)].

The operation of the device is as follows. Electromagnetic radiation illuminates the mesoscale dielectric particle 1, which forms on its shadow side a region with increased radiation intensity with transverse dimensions of the order of λ/3 - λ/44. transverse dimensions of the order of λ/3 - λ/4 4 placed photodetector elements 2, such as photodiodes or avalanche photodiodes. The signals from the photodetector elements enter the processing device 3. The photodetector elements 2, for example, can be placed on a hemispherical surface and directly attached to the shadow surface of the mesoscale particles 1. The overall imaging follows from the point sampling by the receiving element 2 of the images formed on each mesosize particle 1. Thus, each ommatidium contributes one pixel to a different area of the resulting image. Each mesoscale particle 1 creates its own small image of the object (dot). The mesoscale particle 1 has a very limited angle of view and "sees" only that tiny area of the object in front of the eyes, to which the continuation of its optical axis is directed. Since the mesoscale particles 1 are tightly adjacent to each other on a hemispherical surface, while their axes diverge in a radial manner, the hemispherical vision system covers the object as a whole, and the image of the object turns out to be mosaic (that is, composed of many separate pieces).

As a result of the studies, it was found that dielectric mesosized particles form on their shadow side a region with increased radiation intensity with transverse dimensions of the order of λ/3 - λ/4 with a characteristic particle size from λ to 1000λ, where λ is the wavelength of the radiation used.

When the refractive index of the material of a mesoscale particle is less than about 1.4, the transverse size of the local region of the field concentration becomes on the order of or more than the diffraction limit and does not provide a significant increase in the intensity of the electromagnetic field at its boundary. When the refractive index of the material of a mesoscale particle is approximately more than 1.7, the local concentration of the electromagnetic field occurs inside the particle and cannot be used to increase the sensitivity of the receiving device.

Increasing the sensitivity of the photodetector element is achieved due to more efficient interaction of the electromagnetic field in the area of radiation focusing with the photodetector. This is achieved due to the localization of the electromagnetic field in a transverse dimension of the order of λ/3 - λ/4.

It has been established that, for example, for a mesoscale particle with a characteristic size of the order of the radiation wavelength, the light intensity directly at the particle boundary exceeds the incident intensity by about 7–8 times, for a particle with characteristic dimensions of the order of two wavelengths, by about 20 times. For larger particles, this ratio increases even more.

Compared to lenses, the effect of increasing the intensity of electromagnetic radiation in the focus area is 1.5-2 times, since for an ideal lens the transverse size of the focus area is λ/2.

The small thickness of the microlens-photodetector system is ensured by the fact that the radiation focusing area is located on the shadow side of the particle, i.e. focal length tends to 0.

The production of mesoscale dielectric particles located on a thin spherical surface is possible, for example, by casting and curing a polymer in a precision aluminum mold with micromachining [Young Min Song, Yizhu Xie, Viktor Malyarchuk, Jianliang Xiao, InhwaJung, Ki-Joong Choi, Zhuangjian Liu, Hyunsung Park, Chaofeng Lu, Rak-HwanKim, RuiLi, KennethB. Crozier, Yonggang Huang & John A. Rogers. Digital cameras with designs inspired by the arthropod eye // NATURE, 2 MAY 2013, VOL 497, 95, doi:10.1038/nature12083.], micromachining methods based on a femtosecond laser, and the thermomechanical bending method [H. W. Liu, F. Chen, Q. Yang, P. B. Qu, S. G. He, X. H. Wang, J. H. Si, and X. Hou, Fabrication of bioinspired omnidirectional and gapless microlens array for wide field-of-view detections // Appl. Phys. Lett. 100, 133701 (2012); Z. Deng, F. Chen, Q. Yang, H. Bian, G. Du, J. Yong, C. Shan, and X. Hou, Dragonfly-Eye-Inspired Artificial Compound Eyes with Sophisticated Imaging // Adv. Funct. mater. 26, 1995–2001 (2016).], soft lithography methods [M. J. Wang, T. S. Wang, H. H. Shen, J. L. Zhao, Z. Y. Zhang, J. L. Du, and W. X. Yu, Subtle control on hierarchic reflow for the simple and massive fabrication of biomimetic compound eye arrays in polymers for imaging at a large field of view / / J. Mater. Chem. C mater. Opt. electron. Devices 4, 108–112 (2016)] followed by a thermal embossing process [H. W. Liu, F. Chen, Q. Yang, P. B. Qu, S. G. He, X. H. Wang, J. H. Si, and X. Hou Fabrication of bioinspired omnidirectional and gapless microlens array for wide field-of-view detections // Appl. Phys. Lett. 100, 133701 (2012); W. K. Kuo, G. F. Kuo, S. Y. Lin, and H. H. Yu, Fabrication and characterization of artificial miniaturized insect compound eyes for imaging // Bioinspir. Biomim.10(5), 056010 (2015)], using ultraviolet radiation [J. Chen, J. Cheng, D. Zhang, and S.-C. Chen, Precision UV imprinting system for parallel fabrication of large-area micro-lens arrays on non-planar surfaces // Precis. Eng. 44, 70–74 (2016).] and other well-known methods, which are reviewed, for example, in [Wei Yuan, Li Hua Li, Wing Bun Lee and Chang Yuen Chan. Fabrication of Microlens Array and Its Application: A Review // J. Mech. Eng. (2018) 31:16 https://doi.org/10.1186/s10033-018-0204-y].

Various materials can be used as the material of mesosized particles, for example, SiO ₂ with a refractive index of 1.538 at a wavelength of 0.7 μm, polyester, with a refractive index of 1.59 at a wavelength of 0.532 μm, various types of glasses, glass ceramics, quartz, polymethyl methacrylate, polystyrene, polycarbonates [Handbook of the designer opto-mechan. devices / Ed. V.A. Panov. - L.: Mashinostroenie, 1980.] with relative refractive indices of the material lying in the range from 1.4 to 1.7. Other polymers and materials can be used to ensure the transmission of radiation in the desired range.

Hemispherical arrays of photodetectors and methods for their manufacture are known, for example, using an elastomeric membrane with pneumatic pumping, a method for creating curved monolithic silicon structures based on microstructuring of a monolithic silicon matrix using a deep reactive ion etching process; manufacturing method on a flexible silicon substrate using CCD technology, on which the structures of receiving elements are mosaically deposited with the smallest possible size, etc. [Heung Cho Ko, Mark P Stoykovich, Jizhou Song, Viktor Malyarchuk, Won Mook Choi, Chang-Jae Yu, Joseph B Geddes 3rd, Jianliang Xiao, Shuodao Wang, Yonggang Huang, John A Rogers. A hemispherical electronic eye camera based on compressible silicon optoelectronics // Nature 454, 748–753 (2008); InhwaJ unga, Jianliang Xiao, Viktor Malyarchuka , Chaofeng Lub, Ming Lic, Zhuangjian Liu, Jongseung Yoon, Yonggan Huangc, and John A. Rogers. Dynamically tunable hemispherical electronic eye camera system with adjustable zoom capability. //Proc. Natl Acad. sci. USA 108, 1788–1793 (2011); Hung, P. J., Jeong, K., Liu, G. L. & Lee, L. P. Microfabricated suspensions for electrical connections on the tunable elastomer membrane, Appl. Phys. Lett. 85, 6051–6053 (2004), https://doi.org/10.1063/1.1835553; Jeong, K., Kim, J. & Lee, L. P. Biologically inspired artificial compound eyes // Science 312, 557–561 (2006); Dinyari, R., Rim, S.-B., Huang, K., Catrysse, P. B. & Peumans, P. Curving monolithic silicon for nonplanar focal plane array applications // Appl. Phys. Lett. 92, 191114 (2008); Xu, X., Davanco, M., Qi, X. F. & Forrest, S. R. Direct transfer patterning on three dimensionally deformed surfaces at micrometer resolutions and its application to hemispherical focal plane detector arrays // Org. electron. 9, 1122–1127 (2008)].

The useful effect achieved in such a vision system is expressed in the development of a vision system device similar to the “dragonfly eye”, which has a high sensitivity of receiving devices, a small thickness of the microlens - photodetector system.

Claims (1)

The device of the vision system, similar to the “dragonfly eye”, consisting of a matrix of identical dielectric lenses evenly spaced on a thin hemispherical surface, and receivers, each of which is located at the focus of the lens, while the receivers are configured to supply signals to a processing device that differs the fact that the lenses are made in the form of mesodimensional dielectric particles with a characteristic size of at least λ, where λ is the wavelength of the radiation used, with a relative refractive index of the material lying in the range from about 1.4 to 1.7, forming on their outer boundary with on the opposite side of the incident radiation, a region with increased radiation intensity, with transverse dimensions of the order of λ / 3 - λ / 4, and the receiving devices are located directly on the shadow surface of the particle in this region with increased radiation intensity.

Images