CN110231021B - Ripple sensor, ripple reconstruction method and application thereof - Google Patents
Ripple sensor, ripple reconstruction method and application thereof Download PDFInfo
- Publication number
- CN110231021B CN110231021B CN201810180409.4A CN201810180409A CN110231021B CN 110231021 B CN110231021 B CN 110231021B CN 201810180409 A CN201810180409 A CN 201810180409A CN 110231021 B CN110231021 B CN 110231021B
- Authority
- CN
- China
- Prior art keywords
- imaging
- sampling
- light
- ripple
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
- G01C11/04—Interpretation of pictures
- G01C11/30—Interpretation of pictures by triangulation
- G01C11/34—Aerial triangulation
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Image Processing (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a ripple sensor, a ripple reconstruction method and application thereof, wherein the ripple sensor comprises a sampling array, a sampling unit and a control unit, wherein the sampling array is used for sampling refracted rays passing through ripples; the reflecting plate comprises a light receiving surface, and the light receiving surface is used for receiving the sampled refracted light and forming a sampling light spot; the first imaging element is used for imaging the light receiving surface to acquire distribution information of the sampling light spots; the control element is used for reconstructing the ripple distribution according to the distribution information of the sampling light spots, the relative position information of the reflecting plate and the first imaging element, the position information of the array unit in the sampling array corresponding to the sampling light spots and the direction information of the incident light corresponding to the sampled refracted light; by adopting the reflecting plate as a light spot imaging element for sampling refraction light, the staggered light spot phenomenon of transmission type imaging is avoided, the underwater application depth is increased, and the imaging correction effect in the water surface to air imaging application is improved.
Description
Technical Field
The invention belongs to the technical field of underwater aerial imaging, and particularly relates to a ripple sensor, a ripple reconstruction method and application thereof.
Background
For the observation of the target on the water surface, real-time observation can be performed by establishing a monitoring device on the land on the water surface or applying a periscope under water according to the reflection principle of light. However, these observation methods have poor concealment and are liable to reveal the position information of the monitor, and thus have limited application in the fields such as frontier defense.
Correspondingly, direct observation of targets above the water surface from underwater presents some huge challenges, as shown in fig. 7, the ripple on the water surface causes random scattering interference to incident light, and when the ripple is smaller and the water surface is relatively calm, the interference is negligible (see fig. 8(a) when the water surface is a still plane, the image of the checkerboard in the air is shot from underwater), and when the ripple is more severe, the image is distorted and distorted seriously, and particularly, under the condition of strong waves on the sea, the image is completely distorted and the targets cannot be identified (see fig. 8(b) when the water surface has ripples, the checkerboard image in the air is shot from underwater).
The moire sensor was proposed in 2014 by Marina alternan et al in israel, which, with reference to the Shack-Hartmann wavefront sensor concept of atmospheric adaptive optics, samples and measures the moire distribution using an array of apertures or microlenses and reconstructs the moire distribution from the measurement information. The ripple sensor is suitable for outdoor environment, has strong anti-interference capability and great application potential in marine environment, becomes a potential way for reconstructing and correcting the underwater aerial imaging ripple of the ocean, and is concerned by all parties. However, due to the structural design of the existing ripple sensor, the current ripple sensor still has an unsatisfactory correction to the empty imaging under water, for example, (1) referring to fig. 9 and 10, the ripple sensor in the prior art adopts a transmission type ground glass plate scattering plate as a light spot attachment imaging device, the upper and lower surfaces of a scatterer form dislocation light spots, the light spot identification positioning error is large, and the reflection, scattering and absorption of the scatterer have obvious consumption to the light energy, thereby reducing the available underwater depth of the sensor and reducing the concealment and the practicability of the sensor; (2) the reconstruction accuracy of the water surface ripple is not high, and the reconstruction error is large especially when the number of spatial sampling points is small.
Disclosure of Invention
One of the objects of the present invention is to provide a ripple sensor for improving the reconstruction accuracy of water surface ripples and improving the correction effect in underwater space imaging applications, the ripple sensor comprising:
a sampling array for sampling the refracted rays passing through the corrugations;
a reflector plate comprising a light receiving face for receiving the sampled refracted light and forming a sampling spot;
a first imaging element for imaging the light receiving surface to acquire distribution information of the sampling spot;
and the control element is used for reconstructing the ripple distribution according to the distribution information of the sampling light spots, the relative position information of the reflecting plate and the first imaging element, the position information of the array unit in the sampling array corresponding to the sampling light spots and the direction information of the incident light corresponding to the sampled refracted light.
In one embodiment, the sampling array is an aperture array or a microlens array.
In an embodiment, the control element is specifically configured to calculate position information of the sampling light spot according to distribution information of the sampling light spot and relative position information of the reflector and the first imaging element; and the number of the first and second groups,
calculating the direction information of the sampled refracted ray according to the position information of the sampling light spot and the position information of the array unit in the sampling array corresponding to the sampling light spot; and the number of the first and second groups,
and calculating a normal vector corresponding to a ripple sampling point according to the direction information of the sampled refracted ray and the direction information of the incident ray corresponding to the sampled refracted ray, and further reconstructing ripple distribution.
In one embodiment, the control element is further configured to calculate, according to longitude and latitude information and sampling time information corresponding to the sampled refracted ray, direction information of the incident ray corresponding to the sampled refracted ray; and/or the control element is further used for determining the corresponding relation between the sampling light spots and the array units in the sampling array according to the longitude and latitude information and the sampling time information corresponding to the sampled refraction light rays.
In one embodiment, the control element is further configured to obtain ripple distributions at a plurality of historical moments within a same regional target time period; and the number of the first and second groups,
and calculating the ripple distribution of any time in the target time period of the region according to the ripple distribution of the plurality of historical times and the time series model.
The invention also provides an underwater air imaging system, comprising:
a ripple sensor as described above; and the number of the first and second groups,
a second imaging element for imaging an imaging target, the second imaging element being arranged to image imaging light of the imaging target through ripples through which the sampled refracted light passes;
and the image correction element is used for correcting the imaging pattern according to the position information of the second imaging element, the reconstruction moire and the position information of each correction unit in the imaging pattern.
In one embodiment, the method further comprises:
and the third imaging element is used for imaging the imaging target, the third imaging element is arranged for enabling the imaging light of the imaging target to pass through the ripples passed by the sampled refraction light, and the imaging angles of the second imaging element and the third imaging element on the imaging target are different.
In one embodiment of the present invention, the first and second electrodes are,
the image correction element is specifically used for calculating the direction information of the corresponding imaging light rays of the second imaging element and the third imaging element in the water according to the position information of the second imaging element and the third imaging element and the position information of each correction unit in the imaging pattern; and the number of the first and second groups,
according to the direction information of the corresponding imaging light rays of the second imaging element and the third imaging element in the water and the reconstruction ripple, calculating the direction information of the corresponding imaging light rays of the second imaging element and the third imaging element in the air, and thus performing three-dimensional correction on the imaging pattern; and/or the presence of a gas in the gas,
the correction units are pixels.
The invention also provides a ripple reconstruction method, which comprises the following steps:
sampling the refracted rays passing through the corrugations;
receiving the sampled refracted light by using a light receiving surface of the reflecting plate and forming a sampling light spot;
imaging the light receiving surface to acquire distribution information of the sampling light spots;
and reconstructing the ripple distribution according to the distribution information of the sampling light spots, the relative position information of the reflecting plate and the first imaging element, the position information of the array units in the sampling array corresponding to the sampling light spots, and the direction information of the incident light corresponding to the sampled refracted light.
The invention also provides an underwater air imaging method, which comprises the following steps:
sampling the refracted rays passing through the corrugations;
receiving the sampled refracted light by using a light receiving surface of the reflecting plate and forming a sampling light spot;
imaging the light receiving surface to acquire distribution information of the sampling light spots;
reconstructing the ripple distribution according to the distribution information of the sampling light spots, the relative position information of the reflecting plate and the first imaging element, the position information of array units in the sampling array corresponding to the sampling light spots, and the direction information of incident light corresponding to the sampled refracted light;
imaging an imaging target, wherein imaging light rays of the imaging target pass through ripples through which the sampled refracted light rays pass;
and correcting the imaging pattern according to the position information of the imaging element, the reconstruction moire and the position information of each correction unit in the imaging pattern.
According to the ripple sensor provided by the invention, the reflector plate is used as a light spot imaging element for sampling refracted light, so that the staggered light spot phenomenon of transmission type imaging is avoided, the light energy loss of a transmission type scatterer is avoided, the underwater application depth is increased, the problem of reconstruction errors caused by few sampling points is indirectly solved, and the imaging correction effect in the water surface to air imaging application is improved.
Drawings
FIG. 1 is a schematic structural diagram of a ripple sensor according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a water-to-air imaging system according to an embodiment of the present invention;
FIG. 3 is a coordinate system established from sampled refracted rays in accordance with one embodiment of the present invention;
FIG. 4 is a diagram of a world coordinate system, a camera coordinate system, an image coordinate system, and an image pixel coordinate system according to one embodiment of the present invention;
FIG. 5 is a flowchart of a method for reconstructing a ripple in accordance with an embodiment of the present invention;
FIG. 6 is a flow chart of a method for underwater aerial imaging in accordance with an embodiment of the present invention;
FIG. 7 is a schematic diagram of underwater aerial imaging;
FIG. 8 is a comparison graph of the influence of water surface ripples on imaging results during underwater aerial imaging;
FIG. 9 is a schematic diagram of a prior art corrugated sensor using a scatterer as a light spot imaging element;
fig. 10 is a schematic diagram of spot dislocation generated when ground glass (scattering glass) is used for underwater imaging in the prior art.
Detailed Description
The present application will now be described in detail with reference to the particular embodiments thereof as illustrated in the accompanying drawings. These embodiments are not intended to limit the present application, and structural, methodological, or functional changes made by those skilled in the art according to these embodiments are included in the scope of the present application.
Referring to fig. 1, an embodiment of a ripple sensor 10 of the present invention is described, in which the ripple sensor 10 includes a sampling array 101, a reflective plate 102, a first imaging element 103, and a control element (not shown).
The sampling array 101 is used to sample the refracted rays that pass through the corrugations. It should be noted that in the embodiment of the present invention, the ripple sensor 10 is usually disposed below the water surface to sample the refracted light from the air entering the water through the ripples, but in some other embodiments, the ripple sensor 10 may be disposed in the air to sample the refracted light from the water entering the air through the ripples; alternatively, the corrugated sensor 10 may be disposed on one side of any two adjacent optically transmissive media.
In some embodiments, the sampling array 101 may be an aperture array or a microlens array.
The reflective plate 102 includes a light receiving surface 1021 for receiving the sampled refracted light and forming a sampling spot. By replacing the transmission type scattering glass with the non-transmission type reflecting plate 102, the sampled refraction light only forms light spots on the light receiving surface 1021 of the reflecting plate 102, so that the staggered light spot phenomenon of transmission type imaging is avoided, the light energy loss of the transmission type scattering body is avoided, and the underwater application depth is increased.
In some embodiments, the reflective plate 102 may be made of a reflective material, or a reflective layer may be formed on the surface of a transparent or opaque material.
The first imaging element 103 is used to image the light receiving surface 1021 to obtain distribution information of the sampling spot. In one embodiment, the first imaging element 103 may be a camera with an optical imaging function.
The control element is used for reconstructing the ripple distribution according to the distribution information of the sampling light spots, the relative position information of the reflector plate 102 and the first imaging element 103, the position information of the array unit in the sampling array 101 corresponding to the sampling light spots, and the direction information of the incident light corresponding to the sampled refracted light. For example, when the sampling array 101 is an aperture array, the array unit here refers to a plurality of light-transmitting holes in the aperture array.
Specifically, the control element is used for calculating the position information of the sampling light spot according to the distribution information of the sampling light spot and the relative position information of the reflector plate 102 and the first imaging element 103; calculating the direction information of the sampled refracted ray according to the position information of the sampling light spot and the position information of the array unit in the sampling array 101 corresponding to the sampling light spot; and calculating a normal vector corresponding to the ripple sampling point according to the direction information of the sampled refracted ray and the direction information of the incident ray corresponding to the sampled refracted ray, and further reconstructing ripple distribution.
In an embodiment, the control element may determine the corresponding relationship between the sampling spot and the array unit in the sampling array 101 according to the longitude and latitude information corresponding to the sampled refracted light and the sampling time information.
In a specific determination process, referring to fig. 3, for example, the solar altitude angle y may be introduced,namely the included angle between the sunlight direction and the ground plane; and the solar azimuth angle phi is the included angle between the projection of the sunlight direction vector on the ground plane and the true south direction. The two angles can be determined according to the longitude and latitude information corresponding to the sampled refracted ray and the sampling time, for example, by using the existing software: virtual astronomical gymnasium Stellarium. Based on these two angles, a coordinate system is established in which the x-axis is the true south direction, the y-axis is the true east direction, and the z-axis is numerically up. If a point A is taken in the incident direction of the sunlight, the projection is B on the horizontal ground plane, C on the X-axis and D on the Y-axis, the cosine vector of the incident light direction under the xyz coordinate system is (a), (B) and (D)
Sin γ), the location coordinates (x) of the array elements in the sample array 101 0 ，y 0 ，z 0 ) Sampling spot coordinates (x) 1 ，y 1 ，z 1 ) Satisfies the following conditions:
z 0 -z 1 ＝t sinγ
therefore, from the known coordinates of the sampling spot and the coordinates of the array elements in the sampling array 101, the correspondence of the sampling spot to the array elements in the sampling array 101 can be determined. And the direction information of each sampled refracted ray can be determined one by one according to the corresponding relation between the sampling light spots and the array units in the sampling array 101.
In an embodiment, the control element may further calculate the direction information of the incident light corresponding to the sampled refracted light according to the longitude and latitude information corresponding to the sampled refracted light and the sampling time information.
In the specific determination process, since the distance from the sun to the earth is approximately infinity, the incident light rays can be considered as parallel light, that is, a plurality of incident light rays all have the same incident angle, and the incident angle can be uniquely determined by longitude and latitude information and sampling time information corresponding to the sampled refracted light rays.
In the case of incident rays having the same angle of incidence, the cause of the deviation in the direction of each sampled refracted ray can only be determined as the different planes of incidence of the corrugations with which the ray is incident. Therefore, according to the direction information of the sampled refracted ray and the direction information of the incident ray corresponding to the sampled refracted ray, the normal vector of the ripple sampling point corresponding to each sampled refracted ray can be calculated, and then the ripple distribution is reconstructed.
In this embodiment, the control element is further configured to obtain ripple distributions at a plurality of historical time instants in the same regional target time period, and calculate a ripple distribution at an arbitrary time instant in the regional target time period according to the ripple distributions at the plurality of historical time instants and the time series model. Since the water surface ripple has continuity in time, the distribution change of the ripple in a target time period in a region can be obtained through a time series model. In some embodiments, according to fluid dynamics, wave equation modeling can be firstly carried out on water surface ripple distribution to obtain a mathematical model of wave height of the ripple distribution and time variation of a gradient function, then the mathematical model is combined with a time sequence model for measuring gradient, and the most effective solution of mathematical model parameters is carried out by using a group intelligence algorithm; and continuously adding ripple distribution measurement information of the next historical moment in the solving process, and performing parameter iteration, evaluation, elimination and updating to finally enable the obtained ripple distribution result to approach the real ripple distribution.
In this embodiment, the control element may be an integrated circuit including a Microcontroller (MCU). As is well known to those skilled in the art, a microcontroller may include a Central Processing Unit (CPU), a Read-only memory (ROM), a Random Access Memory (RAM), a timing module, a digital-to-analog conversion module (a/DConverter), and several input/output ports. Of course, the control element may also be an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA). The control element may be integrated inside the ripple sensor 10; or in the form of separate modules, without limiting the physical structural relationship with each other, in data/information communication with the associated unit in the above-described ripple sensor 10.
Referring to fig. 2, an embodiment of the underwater air imaging system of the present invention is described. In this embodiment, the underwater air imaging system includes a moire sensor 10, a second imaging element 20, and an image correction element (not shown).
For the detailed structure and the operation principle of the ripple sensor 10, please refer to the above embodiments, which are not described in detail in this embodiment.
A second imaging element 20 is used to image the imaged object, the second imaging element 20 being arranged to image the imaged object with the image light passing through the corrugations of the refracted light sampled by the corrugation sensor 10. The image correction element is used for correcting the imaging pattern according to the position information of the second imaging element 20, the reconstructed moire in the moire sensor 10 and the position information of each correction unit in the imaging pattern.
The imaging light of the second imaging element 20 is set to pass through the corrugations through which the refracted light sampled by the corrugation sensor 10 passes, and since the corrugation distribution of the sheet of corrugations can be reconstructed by the corrugation sensor 10, the imaging distortion due to the fluctuation of the corrugations can be corrected according to the corrugation distribution. In the present embodiment, the image correction element is configured to first calculate the direction information of the imaging light in the water according to the position information of the second imaging element 20 and the position information of each correction unit in the imaging pattern; and calculating the direction information of the imaging light in the air according to the direction information of the imaging light in the water and the reconstructed ripples, thereby correcting the imaging pattern.
In an embodiment, the second imaging element 20 may be a camera with an optical imaging function, and the correction process may be performed by taking a pixel as a basic unit as the correction unit.
With reference to fig. 4, in a specific pattern correction process, for example, a spatial position coordinate system, i.e., a world coordinate system (O), is defined with reference to the optical axis direction w -x w y w z w ) (ii) a Camera coordinate system (O) c -x c y c z c ) Wherein, the optical lens of the camera is the origin of coordinates, x c y c Two coordinate axes parallel to two coordinate axes of the camera imager, z c The direction is determined by the right-hand criterion; image coordinate system (O) p -x p y p ) The origin of which is the intersection of the optical axis and the plane of the imaging pattern, x p y p The two coordinate axes are parallel to two adjacent right-angle sides of the imaging pattern; image pixel coordinate system (O) pix -x pix y pix ) The pixel coordinate system is established by taking the upper left corner of the imaging pattern as the origin and the increasing directions of the rows and the columns as two axes.
The coordinate conversion of each coordinate system is described below.
First, from the world coordinate system to the camera coordinate system:
let the coordinate of a point in the world coordinate system be P W ＝[x w ，y w ，z w ] T The coordinate of the point in the camera coordinate system is P c ＝[x c ，y c ，z c ] T Then there is
Where R is the orthogonal rotation matrix:
t is the translation matrix:
T＝[t x t y t z ] T (formula 3)
Therefore, the external parameters of 6 cameras are needed for determining R and T, and when the relative poses of the world coordinate system and the camera coordinate system are determined, the R and T are uniquely determined.
From camera coordinate system to image coordinate system:
based on the triangle similarity principle, the method can be obtained
Conversion to matrix form
From image coordinate system to image pixel coordinate system:
wherein s is x Denotes the number of pixels in unit mm in the x-axis direction of the image pixel coordinate system, s y Denotes the number of pixels in unit mm in the y-axis direction of the image pixel coordinate system, x 0 ,y 0 Representing the coordinates of the center of the projection plane in the image pixel coordinate system.
Written in matrix form
From the world coordinate system to the image pixel coordinate system:
note the book
Respectively representing the equivalent focal lengths of the focal lengths in the x-axis and y-axis directions of the image pixel coordinate system, and combining the formulas (1), (5), (7) and (8)
f x ，f y ，x 0 ，y 0 For the camera internal known parameters, since the coordinate information of the imaging pattern in the image pixel coordinate system is known, the coordinate information of each pixel (i.e., the correction unit) in the imaging pattern can be converted into world coordinate system information through the above-described process.
Further, the direction of the refraction light in water corresponding to each pixel in the imaging pattern can be determined according to the obtained world coordinate system information, so that the intersection point of each refraction light and the reconstructed corrugation of the corrugation sensor 10 is obtained, and the direction information of the reverse incidence light is calculated according to the law of refraction; a virtual target plane is drawn up, the intersection point of the reverse incident ray and the virtual target plane can be regarded as the real position of each point on the imaging target, and the imaging pattern is corrected.
It should be noted that the above embodiments are only exemplary to describe the image correction process of the present invention, and in practical applications, reference may also be made to use other camera calibration methods to achieve the same image correction effect, and a description thereof is not further provided.
As a preferred embodiment, to achieve correction of the imaging pattern in three dimensions, the underwater air imaging system further comprises a third imaging element 30, the third imaging element 30 also being used to image the imaging target. The third imaging element 30 is disposed such that imaging light of an imaging target passes through corrugations through which refracted light sampled by the corrugation sensor 10 passes, and the imaging angles of the imaging target by the second imaging element 20 and the third imaging element 30 are different.
Correspondingly, the image correction element is used for calculating the direction information of the corresponding imaging light rays of the second imaging element 20 and the third imaging element 30 in the water according to the position information of the second imaging element 20 and the third imaging element 30 and the position information of each correction unit in the imaging pattern; and calculating the direction information of the corresponding imaging light rays of the second imaging element 20 and the third imaging element 30 in the air according to the direction information of the corresponding imaging light rays of the second imaging element 20 and the third imaging element 30 in the water and the reconstruction ripple, thereby performing three-dimensional correction on the imaging pattern. Here, by using the principle of binocular stereo vision, the three-dimensional information is acquired by the trigonometric principle, and since the positional relationship of the second imaging element 20 and the third imaging element 30 is known, the three-dimensional size and the three-dimensional coordinates of the object in the field of view common to the two imaging elements can be obtained.
In an embodiment, the third imaging element 30 may be a camera with an optical imaging function, and the correction process may be performed by taking a pixel as a basic unit as the correction unit.
Referring to fig. 5, an embodiment of the method for reconstructing a ripple according to the present invention will be described. In this embodiment, the method comprises:
s11, sampling the refracted ray passing through the ripple.
In one embodiment, the refracted light rays passing through the corrugations may be sampled by a sampling array, which may be an array of apertures or a microlens array.
And S12, receiving the sampled refracted light rays by using the light receiving surface of the reflecting plate and forming a sampling light spot.
The non-transmission type reflecting plate replaces transmission type scattering glass, and sampled refraction light only forms light spots on a light receiving surface of the reflecting plate, so that the staggered light spot phenomenon of transmission type imaging is avoided, the light energy loss of a transmission type scattering body is avoided, and the underwater application depth is increased.
And S13, imaging the light receiving surface to acquire the distribution information of the sampling light spot.
In one embodiment, the light receiving surface may be imaged by a first imaging element, and the first imaging element may be a camera having an optical imaging function.
And S14, reconstructing the ripple distribution according to the distribution information of the sampling light spots, the relative position information of the reflecting plate and the first imaging element, the position information of the array unit in the sampling array corresponding to the sampling light spots, and the direction information of the incident light corresponding to the sampled refracted light.
In one embodiment, the position information of the sampling light spot can be calculated by the control element according to the distribution information of the sampling light spot and the relative position information of the reflector and the first imaging element; calculating the direction information of the sampled refracted ray according to the position information of the sampling light spot and the position information of the array unit in the sampling array corresponding to the sampling light spot; and calculating a normal vector corresponding to a ripple sampling point according to the direction information of the sampled refraction light and the direction information of the incident light corresponding to the sampled refraction light, and further reconstructing ripple distribution.
In an embodiment, the control element may determine a corresponding relationship between the sampling spot and the array unit in the sampling array according to longitude and latitude information corresponding to the sampled refracted light and sampling time information.
In an embodiment, the control element may further calculate the direction information of the incident light corresponding to the sampled refracted light according to the longitude and latitude information corresponding to the sampled refracted light and the sampling time information.
In the specific determination process, since the distance from the sun to the earth is approximately infinity, the incident light rays can be considered as parallel light, that is, a plurality of incident light rays all have the same incident angle, and the incident angle can be uniquely determined by longitude and latitude information and sampling time information corresponding to the sampled refracted light rays.
In the case of incident rays having the same angle of incidence, the cause of the deviation in the direction of each sampled refracted ray can only be determined as the different planes of incidence of the corrugations with which the ray is incident. Therefore, according to the direction information of the sampled refracted ray and the direction information of the incident ray corresponding to the sampled refracted ray, the normal vector of the ripple sampling point corresponding to each sampled refracted ray can be calculated, and then the ripple distribution is reconstructed.
In an embodiment, the control element is further configured to obtain ripple distributions at multiple historical times in the same regional target time period, and calculate a ripple distribution at any time in the regional target time period according to the ripple distributions at the multiple historical times and the time series model. Because the water surface ripple has continuity in time, the distribution change of the ripple in a target time period in a region can be obtained through a time series model. In some embodiments, according to fluid dynamics, wave equation modeling can be firstly carried out on water surface ripple distribution to obtain a mathematical model of the wave height of the ripple distribution and the change of a gradient function along with time, then the mathematical model is combined with a time sequence model for measuring the gradient, and the most effective solution of mathematical model parameters is carried out by utilizing a group intelligence algorithm; and continuously adding ripple distribution measurement information of the next historical moment in the solving process, and performing parameter iteration, evaluation, elimination and updating to finally enable the obtained ripple distribution result to approach the real ripple distribution.
Referring to fig. 6, an embodiment of the underwater air imaging method of the present invention is described. In this embodiment, the method comprises:
s21, sampling the refracted ray passing through the ripple.
In one embodiment, the refracted light rays passing through the corrugations may be sampled by a sampling array, which may be an array of apertures or a microlens array.
And S22, receiving the sampled refracted light rays by using the light receiving surface of the reflecting plate and forming a sampling light spot.
The non-transmission type reflecting plate replaces transmission type scattering glass, and sampled refraction light only forms light spots on a light receiving surface of the reflecting plate, so that the staggered light spot phenomenon of transmission type imaging is avoided, the light energy loss of a transmission type scattering body is avoided, and the underwater application depth is increased.
And S23, imaging the light receiving surface to acquire distribution information of the sampling light spots.
In an embodiment, the light receiving surface may be imaged by a first imaging element, and the first imaging element may be a camera having an optical imaging function.
And S24, reconstructing the ripple distribution according to the distribution information of the sampling light spots, the relative position information of the reflecting plate and the first imaging element, the position information of the array unit in the sampling array corresponding to the sampling light spots, and the direction information of the incident light corresponding to the sampled refracted light.
In one embodiment, the position information of the sampling light spot can be calculated by the control element according to the distribution information of the sampling light spot and the relative position information of the reflector and the first imaging element; calculating the direction information of the sampled refracted ray according to the position information of the sampling light spot and the position information of the array unit in the sampling array corresponding to the sampling light spot; and calculating a normal vector corresponding to the ripple sampling point according to the direction information of the sampled refracted ray and the direction information of the incident ray corresponding to the sampled refracted ray, and further reconstructing ripple distribution.
In an embodiment, the control element may determine a corresponding relationship between the sampling spot and the array unit in the sampling array according to longitude and latitude information corresponding to the sampled refracted light and sampling time information.
In an embodiment, the control element may further calculate the direction information of the incident light corresponding to the sampled refracted light according to the longitude and latitude information corresponding to the sampled refracted light and the sampling time information.
In the specific determination process, since the distance from the sun to the earth is approximately infinity, the incident light may be considered as parallel light, that is, a plurality of incident light have the same incident angle, and the incident angle may be uniquely determined by the longitude and latitude information and the sampling time information corresponding to the sampled refracted light.
In the case of incident rays having the same angle of incidence, the cause of the deviation in the direction of each sampled refracted ray can only be determined as the different planes of incidence of the corrugations with which the ray is incident. Therefore, according to the direction information of the sampled refracted ray and the direction information of the incident ray corresponding to the sampled refracted ray, the normal vector of the ripple sampling point corresponding to each sampled refracted ray can be calculated, and then the ripple distribution is reconstructed.
The control element is also used for acquiring the ripple distribution of a plurality of historical moments in the same region target time period, and calculating the ripple distribution of any moment in the region target time period according to the ripple distribution of the plurality of historical moments and the time series model. Because the water surface ripple has continuity in time, the distribution change of the ripple in a target time period in a region can be obtained through a time series model. In some embodiments, according to fluid dynamics, wave equation modeling can be firstly carried out on water surface ripple distribution to obtain a mathematical model of wave height of the ripple distribution and time variation of a gradient function, then the mathematical model is combined with a time sequence model for measuring gradient, and the most effective solution of mathematical model parameters is carried out by using a group intelligence algorithm; and continuously adding ripple distribution measurement information of the next historical moment in the solving process, and performing parameter iteration, evaluation, elimination and updating to finally enable the obtained ripple distribution result to approach the real ripple distribution.
And S25, imaging the imaging target, wherein the imaging light ray of the imaging target passes through the ripples passed by the sampled refraction light ray.
In an embodiment, the imaging object may be imaged by a second imaging element arranged to image a moire through which imaging light of the imaging object passes through refracted light sampled by the moire sensor. The imaging light of the imaging element is set to pass through the ripples passed by the refracted light sampled by the ripple sensor, and the ripple distribution of the ripples of the sheet can be reconstructed by the ripple sensor, so that the imaging distortion caused by the fluctuation of the ripples can be corrected according to the ripple distribution.
And S26, correcting the imaging pattern according to the position information of the imaging element, the reconstruction moire and the position information of each correction unit in the imaging pattern.
In one embodiment, the imaging pattern may be corrected by an image correction element, and the image correction element may calculate the direction information of the imaging light in the water, for example, according to the position information of the imaging element and the position information of each correction unit in the imaging pattern; and calculating the direction information of the imaging light in the air according to the direction information of the imaging light in the water and the reconstructed ripples, thereby correcting the imaging pattern.
In still another embodiment, the step S25 may be performed by setting two second imaging elements and two third imaging elements having different imaging angles to the imaging target to respectively image the imaging target. The image correction element is used for calculating the direction information of the corresponding imaging light rays of the second imaging element and the third imaging element in water according to the position information of the second imaging element and the third imaging element and the position information of each correction unit in the imaging pattern; and calculating the direction information of the corresponding imaging light rays of the second imaging element and the third imaging element in the air according to the direction information of the corresponding imaging light rays of the second imaging element and the third imaging element in the water and the reconstruction ripple, thereby performing three-dimensional correction on the imaging pattern. Here, the three-dimensional information is acquired by the trigonometric principle using the principle of binocular stereo vision, and since the positional relationship of the second imaging element and the third imaging element is known, the three-dimensional size and the three-dimensional coordinates of the object in the field of view common to the two imaging elements can be obtained.
The invention has the following beneficial effects through the above embodiment:
by adopting the reflecting plate as a light spot imaging element for sampling refraction light, the staggered light spot phenomenon of transmission type imaging is avoided, the light energy loss of a transmission type scatterer is avoided, the underwater application depth is increased, the problem of reconstruction errors caused by few sampling points is indirectly solved, and the imaging correction effect in the water surface to air imaging application is improved.
It should be understood that although the specification describes embodiments, not every embodiment includes only a single embodiment, and such description is for clarity purposes only, and it will be appreciated by those skilled in the art that the specification as a whole may be appropriately combined to form other embodiments as will be apparent to those skilled in the art.
The above-listed detailed description is only a specific description of possible embodiments of the present application, and they are not intended to limit the scope of the present application, and equivalent embodiments or modifications made without departing from the technical spirit of the present application should be included in the scope of the present application.
Claims (10)
1. A ripple sensor, comprising:
a sampling array for sampling the refracted rays passing through the corrugations;
a reflector plate comprising a light receiving face for receiving the sampled refracted light and forming a sampling spot;
a first imaging element for imaging the light receiving surface to acquire distribution information of the sampling spot;
and the control element is used for reconstructing the ripple distribution according to the distribution information of the sampling light spots, the relative position information of the reflecting plate and the first imaging element, the position information of the array unit in the sampling array corresponding to the sampling light spots and the direction information of the incident light corresponding to the sampled refracted light.
2. The ripple sensor of claim 1, wherein the sampling array is an array of apertures or a microlens array.
3. The ripple sensor of claim 1, wherein the control element is specifically configured to calculate position information of the sampling spot based on distribution information of the sampling spot and relative position information of the reflective plate and the first imaging element; and (c) a second step of,
calculating the direction information of the sampled refracted ray according to the position information of the sampling light spot and the position information of the array unit in the sampling array corresponding to the sampling light spot; and (c) a second step of,
and calculating a normal vector corresponding to a ripple sampling point according to the direction information of the sampled refracted ray and the direction information of the incident ray corresponding to the sampled refracted ray, and further reconstructing ripple distribution.
4. The ripple sensor of claim 1 or 3, wherein the control element is further configured to calculate the direction information of the incident ray corresponding to the sampled refracted ray according to the longitude and latitude information corresponding to the sampled refracted ray and the sampling time information; and/or the control element is further used for determining the corresponding relation between the sampling light spots and the array units in the sampling array according to the longitude and latitude information and the sampling time information corresponding to the sampled refraction light rays.
5. The ripple sensor of claim 1, wherein the control element is further configured to obtain a ripple distribution at a plurality of historical times within a target time period of the same region; and the number of the first and second groups,
and calculating the ripple distribution of any time in the target time period of the region according to the ripple distribution of the plurality of historical times and the time series model.
6. An underwater aerial imaging system, comprising:
a ripple sensor of any one of claims 1 to 5; and (c) a second step of,
a second imaging element for imaging an imaging target, the second imaging element being arranged to image imaging light of the imaging target through ripples through which the sampled refracted light passes;
and the image correction element is used for correcting the imaging pattern according to the position information of the second imaging element, the reconstruction moire and the position information of each correction unit in the imaging pattern.
7. The underwater aerial imaging system of claim 6 further comprising:
and the third imaging element is used for imaging the imaging target, the third imaging element is arranged to enable imaging light rays of the imaging target to pass through ripples passed by the sampled refraction light rays, and the imaging angles of the second imaging element and the third imaging element to the imaging target are different.
8. Underwater air imaging system according to claim 7,
the image correction element is specifically used for calculating the direction information of the corresponding imaging light rays of the second imaging element and the third imaging element in the water according to the position information of the second imaging element and the third imaging element and the position information of each correction unit in the imaging pattern; and the number of the first and second groups,
according to the direction information of the corresponding imaging light rays of the second imaging element and the third imaging element in the water and the reconstruction ripple, calculating the direction information of the corresponding imaging light rays of the second imaging element and the third imaging element in the air, and thus performing three-dimensional correction on the imaging pattern; and/or the presence of a gas in the gas,
the correction units are pixels.
9. A method of moire reconstruction, comprising:
sampling the refracted rays passing through the corrugations;
receiving the sampled refracted light by using a light receiving surface of the reflecting plate and forming a sampling light spot;
imaging the light receiving surface to acquire distribution information of the sampling light spots;
and reconstructing the ripple distribution according to the distribution information of the sampling light spots, the relative position information of the reflecting plate and the first imaging element, the position information of the array units in the sampling array corresponding to the sampling light spots, and the direction information of the incident light corresponding to the sampled refracted light.
10. An underwater aerial imaging method, comprising:
sampling the refracted rays passing through the corrugations;
receiving the sampled refracted light by using a light receiving surface of the reflecting plate and forming a sampling light spot;
imaging the light receiving surface to acquire distribution information of the sampling light spots;
reconstructing the ripple distribution according to the distribution information of the sampling light spots, the relative position information of the reflecting plate and the first imaging element, the position information of array units in the sampling array corresponding to the sampling light spots, and the direction information of incident light corresponding to the sampled refracted light;
imaging an imaging target, wherein imaging light rays of the imaging target pass through ripples through which the sampled refracted light rays pass;
and correcting the imaging pattern according to the position information of the imaging element, the reconstruction moire and the position information of each correction unit in the imaging pattern.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810180409.4A CN110231021B (en) | 2018-03-05 | 2018-03-05 | Ripple sensor, ripple reconstruction method and application thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810180409.4A CN110231021B (en) | 2018-03-05 | 2018-03-05 | Ripple sensor, ripple reconstruction method and application thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110231021A CN110231021A (en) | 2019-09-13 |
| CN110231021B true CN110231021B (en) | 2022-09-20 |
Family
ID=67861626
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810180409.4A Active CN110231021B (en) | 2018-03-05 | 2018-03-05 | Ripple sensor, ripple reconstruction method and application thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110231021B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111277735B (en) * | 2019-11-04 | 2021-07-16 | 苏州臻迪智能科技有限公司 | Underwater cloud deck, underwater imaging system and underwater robot |
| CN110907132B (en) * | 2019-12-13 | 2022-06-07 | 中国人民解放军军事科学院国防科技创新研究院 | Wave direction detection method, system, equipment and medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107063117A (en) * | 2017-03-15 | 2017-08-18 | 上海大学 | Underwater laser synchronous scanning range of triangle imaging system and method based on optical field imaging |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7023954B2 (en) * | 2003-09-29 | 2006-04-04 | Jordan Valley Applied Radiation Ltd. | Optical alignment of X-ray microanalyzers |
| CN102331795B (en) * | 2011-08-26 | 2013-08-14 | 浙江中控太阳能技术有限公司 | Method for controlling sunlight reflecting device to automatically track sun based on facula identification |
| CN104596638B (en) * | 2015-02-05 | 2017-01-25 | 中国工程物理研究院应用电子学研究所 | High-resolution multi-wavelength laser intensity distribution detector and measurement method implemented by same |
-
2018
- 2018-03-05 CN CN201810180409.4A patent/CN110231021B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107063117A (en) * | 2017-03-15 | 2017-08-18 | 上海大学 | Underwater laser synchronous scanning range of triangle imaging system and method based on optical field imaging |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110231021A (en) | 2019-09-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10929966B2 (en) | System and method for imaging underwater environments using fluid lensing | |
| CN103913148B (en) | Space flight TDI CCD camera full link numerical value emulation method | |
| CN111537003B (en) | Starlight atmospheric refraction measurement correction method based on refraction surface collineation | |
| CN106949836B (en) | Device and method for calibrating same-side target position of stereoscopic camera | |
| CN104764445B (en) | Object point coordinate determination method and device under water | |
| CN106679676B (en) | A kind of monoscopic multifunctional optical sensor and implementation method | |
| Jordt | Underwater 3D reconstruction based on physical models for refraction and underwater light propagation | |
| CN107016210B (en) | Imaging quality simulation method of Fresnel diffraction optical system | |
| CN105716581B (en) | Underwater object point coordinate determination method and device based on double medium camera works | |
| CN104089627A (en) | Absolute radiometric calibration method for satellite staring planar array CCD camera | |
| CN110231021B (en) | Ripple sensor, ripple reconstruction method and application thereof | |
| CN105182362B (en) | A kind of natural water surface polarization remote sensing imaging simulation method | |
| CN108830811A (en) | A kind of aviation image real-time correction method that flight parameter is combined with camera internal reference | |
| CN106023193A (en) | Array camera observation method for detecting structure surface in turbid media | |
| CN112857356A (en) | Unmanned aerial vehicle water body environment investigation and air route generation method | |
| Xue et al. | Airborne optical polarization imaging for observation of submarine Kelvin wakes on the sea surface: Imaging chain and simulation | |
| CN111307140B (en) | Atmospheric polarized light orientation method used under cloudy weather condition | |
| CN108917633A (en) | Fuel assembly deformation detection system based on underwater binocular vision | |
| Lin et al. | 3D point cloud capture method for underwater structures in turbid environment | |
| CN110231020A (en) | Ripple sensor, ripple method for reconstructing and its application | |
| CN108318458B (en) | Method for measuring outdoor typical feature pBRDF (binary RDF) suitable for different weather conditions | |
| CN112991532A (en) | Underwater high-precision three-dimensional reconstruction method based on photometric stereo method and laser triangulation method | |
| CN113432611B (en) | Orientation device and method based on all-sky-domain atmospheric polarization mode imaging | |
| CN106530351B (en) | A kind of method for positioning mass center obtained based on image sensor pixel internal quantum efficiency | |
| CN114119768A (en) | Stichopus japonicus activity monitoring method based on multi-vision technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |