JP7372589B2 - Underwater laser viewing device - Google Patents

Description

The present invention relates to an underwater laser viewing device, and more particularly to an underwater laser viewing device that can obtain an image of an object even when the presence of the object included in a photographed image is unclear. .

When conducting marine surveys or marine construction, it is necessary to visually check the conditions of the seabed and the conditions of objects installed on the seabed. However, in turbid water with extremely low transparency, even if normal lighting is turned on, the light will be scattered, making it impossible to visually recognize the object from the captured image.

Therefore, as described in Patent Document 1, a pulsed laser beam is irradiated toward an object in turbid water, and the moment the pulsed laser beam reflects off the object and returns, the shutter is opened and the image is taken. A gate camera designed to do this has already been developed.

Japanese Patent Application Publication No. 7-72250

However, even when using a gate camera such as the one described in Patent Document 1, when you want to visually recognize an object at a deeper position or from a more distant position, the amount of light and contrast may vary. There was a problem in that the existence of the target object included in the photographed image was unclear due to insufficient number of lenses, and the target object could not be visually recognized.

The present invention was devised in view of such problems, and is an underwater laser visual recognition method that can obtain an image of an object even when the presence of the object included in the photographed image is unclear. The purpose is to provide equipment.

According to the present invention, there is provided an underwater laser viewing device mounted on an underwater device that can be placed at a position remote from an object in turbid water, the laser oscillating a pulsed laser beam toward the object. a light irradiation device; an imaging device including a shutter that can be opened and closed in accordance with the reflected light from the object; and a plurality of images taken by the imaging device while controlling the laser light irradiation device and the imaging device. a processing device for storing, the processing device searches for whether or not the target object is included in the photographed image, and extracts feature points from the search image that is determined to include the target object. , configured to collate the feature points extracted from the plurality of search images to determine a correspondence relationship, align the feature points, integrate the search images, and obtain an integrated image including the target object. Further, the processing device divides the photographed image into a plurality of regions, performs fast Fourier transform on each region to create a reconstructed image, and extracts an outline of the object from the reconstructed image. is configured to search whether or not the target object is included in the captured image, and configured to set the size of the area based on the wavelength of the result of fast Fourier transform of the entire captured image. An underwater laser viewing device is provided.

The processing device is configured to create the reconstructed image using any one of the maximum value, average value, standard deviation, average value/maximum value, or first-order differential value of the fast Fourier transform. Good too.

According to the method for acquiring an image of an object in turbid water and the underwater laser viewing device according to the present invention described above, the amount of light and contrast are insufficient, and the existence of the object included in the photographed image becomes unclear, making it difficult to identify the object. Even if an object cannot be visually recognized, the feature points can be extracted from the photographed image (search image) that includes the object, and the search images can be integrated while aligning the feature points. A visible image can be obtained.

BRIEF DESCRIPTION OF THE DRAWINGS It is explanatory drawing which shows the underwater laser visual recognition apparatus based on one Embodiment of this invention, (a) is a usage state, (b) is a schematic block diagram. This is an example of images taken in turbid water. (a) is an image taken at a distance of 5 m, (b) is an image taken at a distance of 11 m, (c) is an image taken at a distance of 13 m, and (d) is an image taken at a distance of 15 m. , is shown. FIG. 2 is a flow diagram showing a method for acquiring an image of an object in turbid water according to an embodiment of the present invention. It is an explanatory diagram showing the result of fast Fourier transform of the whole photographed image at a distance of 15 m, (a) is an FFT image, and (b) is a brightness distribution diagram. This is a brightness distribution map obtained by separating the brightness distribution map shown in FIG. 4(b) into vertical and horizontal directions, where (a) is a horizontal brightness distribution map, and (b) is a vertical brightness distribution map. . FIG. 3 is an explanatory diagram showing an image reconstruction process, in which (a) is a conceptual diagram and (b) is a reconstructed image using standard deviation. It is explanatory drawing which shows the modification of an image reconstruction process, (a) is a 1st modification, (b) is a 2nd modification. It is explanatory drawing which shows the modification of an image reconstruction process, (a) is a 3rd modification, (b) is a 4th modification. It is an explanatory diagram showing a contour extraction process, (a) is an image in which the contour of the object is extracted from the reconstructed image using the standard deviation, and (b) is an image showing only the extracted contour. .

Embodiments of the present invention will be described below with reference to FIGS. 1(a) to 9(b). Here, FIG. 1 is an explanatory diagram showing an underwater laser viewing device according to an embodiment of the present invention, in which (a) is a usage state and (b) is a schematic configuration diagram.

The underwater laser viewing device 1 according to an embodiment of the present invention can be placed at a position away from an object X in turbid water, for example, as shown in FIGS. 1(a) and 1(b). An imaging device that is installed in the underwater equipment 2 and includes a laser beam irradiation device 3 that emits pulsed laser light toward the object X, and a shutter 41 that can be opened and closed according to the reflected light from the object X. 4, and a processing device 5 that controls the laser beam irradiation device 3 and the imaging device 4 and stores a plurality of images taken by the imaging device 4.

The underwater device 2 is a structure that can be submerged while maintaining an internal watertight state. The underwater device 2 may be, for example, a box-shaped housing or an unmanned underwater vehicle (UUV). Further, the underwater device 2 may be of a towed type, a floating type, or an autonomous navigation type.

The laser beam irradiation device 3 is a device that oscillates a pulsed laser beam toward an object X in turbid water. Suspended particles in turbid water cause various types of scattered light to be emitted by the laser beam. Therefore, if all the reflected light of the laser beam is received, the scattered light of the suspended particles will also be included, and the reflected light of the object X will be buried.

Therefore, in this embodiment, the shutter 41 is momentarily switched to an open state in accordance with the timing of receiving the reflected light from the object X, and the shutter 41 is used to allow the light to pass through the imaging device 4. In this way, by using the shutter 41 that can be opened and closed at high speed, scattered light from suspended particles can be removed by keeping the shutter 41 closed.

Specifically, the laser beam irradiation device 3 includes a distance measurement sensor (not shown), and measures the distance to the object X while processing the timing at which the reflected light from the object X reaches the imaging device 4. It is calculated by device 5. A synchronizing signal is transmitted to the shutter 41 from the laser beam irradiation device 3 simultaneously with the oscillation of the laser beam. The timing for switching the shutter 41 to the open state is transmitted from the processing device 5 to the shutter 41 .

The imaging device 4 is a high-sensitivity camera that includes, for example, an image intensifier that is an optical amplifier including a focus lens, a light receiving element, a scanning device, and the like. Note that the imaging device 4 may be a camera having another configuration as long as it is capable of acquiring an image based on reflected light of a laser beam.

The processing device 5 is, for example, a computer equipped with a CPU (Central Processing Unit), a memory, a hard disk, and the like. The processing device 5 controls the operations of the laser beam irradiation device 3 and the imaging device 4 based on a built-in program, and stores images taken by the imaging device 4. The processing device 5 also performs a process of searching for the target object X from the unclear photographed image and acquiring an image in which the presence of the target object X can be visually recognized. Furthermore, the processing device 5 may include a monitor that displays images.

The underwater laser viewing device 1 described above is mounted on an underwater device 2 and submerged in water, for example, as shown in FIG. 1(a). For example, if the object X is on the seabed, the underwater device 2 needs to be submerged at a depth close to the seabed. Generally, as the depth of water increases, the amount of sunlight decreases, and turbidity tends to increase near the ocean floor due to disturbances, so it is necessary to get closer to the object X and take an image.

However, in reality, it may be difficult to approach the object X or it may be difficult to dive the underwater device 2 to near the seabed. For example, if the object X is an object that swings due to tidal currents, there is a possibility that the underwater device 2 will collide with the object X. In addition, if there are obstacles such as rocks or marine plants on the seabed where object However, the control of the underwater equipment 2 becomes complicated.

Therefore, a method is desired that allows photographing the object X from a position as far away as possible. Here, FIG. 2 shows an example of images taken in turbid water, where (a) is an image taken at a distance of 5 m, (b) is an image taken at a distance of 11 m, (c) is an image taken at a distance of 13 m, and (d ) indicates an image taken at a distance of 15 m.

The object X is a test object prepared for the test, and is, for example, a rectangular whiteboard. This whiteboard displays a plurality of vertical lines, horizontal lines, numbers, symbols, etc. on its surface, and has light emitters attached to the four corners. The object X is installed on the seabed, and representative images are shown in FIGS. 2(a) to 2(d), which are taken while gradually moving the underwater laser viewing device 1 away from the object X. ing.

Referring to FIGS. 2(a) to 2(d), the presence of object X becomes unclear as the distance from object X increases, and the presence of object However, the existence of the object X cannot be visually recognized in a photographed image taken 15 meters away.

Therefore, in this embodiment, based on the method for acquiring an image of an object in turbid water shown in FIG. There is. Here, FIG. 3 is a flow diagram showing a method for acquiring an image of an object in turbid water according to an embodiment of the present invention.

In the image acquisition method according to the present embodiment, the underwater laser viewing device 1 is placed at a position away from the object X, and the camera of the imaging device 4 is directed in a direction where the object It is determined whether object X can be recognized from the image, feature points are extracted from the photographed images in which the presence of object This is to obtain an image in which the presence of X is visible.

Specifically, as shown in FIG. 3, the image acquisition method according to the present embodiment includes a photographing process Step 1, an image selection process Step 2, a noise removal process Step 3, a search process Step 4, an extraction process Step 5, The process includes an integration confirmation process Step 6, a verification process Step 7, an integration process Step 8, a set number of sheets confirmation process Step 9, and a sharpening process Step 10.

The photographing process Step 1 is a process of acquiring a plurality of photographed images that may include the target object X from positions distant from the target object X in the turbid water. The plurality of captured images are captured from positions separated from the object X by approximately the same distance. Specifically, such a captured image is captured by the underwater laser viewing device 1 described above.

The image selection step Step 2 is a step of selecting an image for searching for the target object X from a plurality of captured images. One photographed image is arbitrarily selected from a plurality of photographed images taken from positions separated from the object X by approximately the same distance. This process is repeated until the set number of sheets to be accumulated is reached.

The noise removal step Step 3 is a step of removing noise from the selected image. For example, noise in the selected image is removed using a common method such as a median filter. The median filter is well known as a filter that leaves edge information among filters based on order statistics. After the noise removal step Step 3, sharpening processing such as level correction and PSF (Point Spread Function) correction may be performed.

The search step Step 4 is a step of searching whether or not the target object X is included in the selected image. Specifically, the search step Step 4 includes an image reconstruction step Step 41 of dividing the selected image into a plurality of regions α and performing fast Fourier transform (FFT) on each region to create a reconstructed image; The process includes a contour extraction step Step 42 in which the contour β of the object is extracted from the reconstructed image, and an object confirmation step Step 43 in which it is confirmed whether the object X is included in the reconstructed image.

Here, it is assumed that the image acquisition method according to this embodiment is applied to the image taken at a distance of 15 m shown in FIG. 2(d). FIG. 4 is an explanatory diagram showing the results of fast Fourier transform of the entire photographed image at a distance of 15 m, where (a) is an FFT image and (b) is a brightness distribution diagram. FIG. 5 is a brightness distribution map obtained by separating the brightness distribution map shown in FIG. 4(b) into vertical and horizontal directions, where (a) is a horizontal brightness distribution map, and (b) is a vertical brightness distribution map. Figure. FIG. 6 is an explanatory diagram showing the image reconstruction process, in which (a) is a conceptual diagram and (b) is a reconstructed image using standard deviation.

In the image reconstruction step Step 41, the size of the area α, which is a unit for fast Fourier transform (FFT), is set based on the wavelength λ of the result of fast Fourier transform (FFT) of the entire captured image. To calculate this wavelength λ, first, as shown in FIG. 4(a), the entire selected image is subjected to fast Fourier transform (FFT).

When this image is expressed in a brightness distribution diagram, it is displayed as shown in FIG. 4(b). The luminance distribution diagram shown in FIG. 4(b) is separated into vertical and horizontal directions and is displayed in FIGS. 5(a) and 5(b). In addition, in FIGS. 4(b) to 5(b), "vertical direction" means the vertical direction of the selected image, "horizontal direction" means the horizontal direction of the selected image, and "intensity" means the intensity of brightness. It means.

The wavelength λ of one period including the peak value is determined while considering both the vertical and horizontal luminance distributions shown in FIGS. 5(a) and 5(b). This wavelength λ means a blur in the luminance distribution, that is, the width of the edge of the object X. Note that the wavelength λ may be determined by the processing device 5 based on conditions such as a threshold value, or may be determined visually by an operator.

In this embodiment, the wavelength λ is determined to be, for example, 16 dots. Therefore, the area α is set to a size of 16 dots×16 dots. In this manner, by determining the wavelength λ to be a power of 2, fast Fourier transform (FFT) of the region α can be easily performed. For example, the size of the area α may be 8×8 or 32×32.

FIG. 6A shows a selected image virtually displayed as dots. The number of dots in the selected image is, for example, 16 bits, 1200 (vertical direction) x 1800 (horizontal direction). Note that FIG. 6(a) is a simplified display of 1200×1800 dots. The area α is displayed as a range of 16 dots (vertical direction)×16 dots (horizontal direction), for example, as shown filled in gray in FIG. 6(a). The entire area of the selected image is subjected to Fast Fourier Transform (FFT) while sliding in the horizontal and vertical directions, using a 16×16 area α as one unit of Fast Fourier Transform (FFT).

The reconstructed image shown in FIG. 6(b) is obtained by subjecting the region α to Fast Fourier Transform (FFT) and replacing the region α with the value of its standard deviation. Note that the reconstructed image can also be created using values other than standard deviation. Here, FIG. 7 is an explanatory diagram showing a modification of the image reconstruction process, in which (a) is a first modification and (b) is a second modification. FIG. 8 is an explanatory diagram showing a modification of the image reconstruction process, in which (a) is a third modification and (b) is a fourth modification.

The first modified example shown in FIG. 7(a) is a reconstructed image using the maximum value of fast Fourier transform (FFT). Further, the second modified example shown in FIG. 7(b) is a reconstructed image using the average value of fast Fourier transform (FFT). The third modified example shown in FIG. 8(a) is a reconstructed image using the average value/maximum value of fast Fourier transform (FFT). Further, the fourth modified example shown in FIG. 8(b) is a reconstructed image using a first-order differential value (difference) of fast Fourier transform (FFT).

In this way, the reconstructed image is replaced using any one of the standard deviation, maximum value, average value, average value/maximum value, or first-order differential value of the result of fast Fourier transform (FFT) of the area α. It can be reconfigured by

The contour extraction step Step 42 is, for example, a step of calculating the contour β of an object that may be the object X included in the image, based on the brightness threshold of the reconstructed image. The contour extraction method is not limited to this method, and may be edge detection using first-order differentiation, or edge detection using the difference in matching frequency with the object X whose shape has been memorized in advance. It may be foreign object detection or edge detection using principal component analysis.

Here, FIG. 9 is an explanatory diagram showing the contour extraction process, where (a) is an image in which the contour of the object has been extracted from the reconstructed image using the standard deviation, and (b) is only the extracted contour. This is an image showing. By the contour extraction method described above, the contour β of the object included in the reconstructed image can be extracted as shown by the black line in FIG. 9(a). Thereafter, if necessary, only the shape of the outline β of the object may be extracted as shown in FIG. 9(b).

The object confirmation step Step 43 is a step of checking whether the contour β of the object extracted in the contour extraction step Step 42 is the object X or not. Whether or not the extracted object contour β is the object X can be determined based on the matching rate with the contour shape of the object X stored in advance. Such confirmation may be performed by the processing device 5, or may be performed visually by an operator.

If it is determined that the object X is included in the recreated image (Y), the process moves to step 5 of the extraction process, and if it is determined that the object ), the process returns to the image selection process Step 2 in order to exclude the image and check whether the object X is included in the next photographed image.

The extraction step Step 5 is a step of extracting feature points from the search image that is determined to include the target object X. Specifically, after extracting the edges of the object X using an edge extraction filter (for example, a differential filter, etc.), corners are detected, and Extract feature points.

Note that the method for extracting feature points is not limited to the method described above; for example, feature points may be extracted based on shape features, or feature points may be extracted based on texture features. Alternatively, feature points may be extracted based on high-order local autocorrelation features.

The integration possibility checking step Step 6 is a step of checking whether the search image from which feature points have been extracted is the first image. If it is the first image (Y), there is no image to be integrated, so the process returns to step 2 of the image selection step in order to obtain a search image from which the next feature point has been extracted. If it is not the first image (N), there is an image to be integrated, so the process moves to step 7 of the matching process.

The matching step Step 7 is a step of matching feature points extracted from a plurality of search images to determine a correspondence relationship. In the two search images, feature points are compared based on the degree of matching of indexes (vectors) of the feature points. At this time, in one search image, matching may be performed in order of feature points having the highest luminance intensity. In this matching process Step 7, for example, feature points on the upper left of object X correspond to each other, feature points on the upper right of object X correspond to each other, feature points on the lower left of object X correspond to each other, feature points on the lower right of object X correspond to each other. This is the process of verifying that the

The integration step Step 8 is a step of aligning the feature points and integrating the search images to create an integrated image. Since the correspondence between the feature points has been grasped in the matching process Step 7, search images can be integrated by aligning the corresponding feature points so that they match. Through such processing, it is possible to integrate the images while adjusting the orientation of the images so that the objects X overlap.

In this way, by matching the feature points and creating an integrated image, it is possible to obtain an integrated image in which the object X can be visually recognized from a captured image with insufficient light amount or contrast. In addition, even if the object X is swinging in the sea or the underwater equipment 2 equipped with the underwater laser viewing device 1 is moving or swinging slowly, the integration allows the object X to be visually recognized. Image can be obtained.

The set number of images confirmation step Step 9 is a step of checking whether the accumulated number of images has reached the set number of images. Although the set number of sheets is arbitrary, it may be, for example, about 16 to 32 sheets. If the accumulated number of images has not reached the set number (N), the process returns to image selection step Step 2 in order to obtain the next image to be accumulated. Further, when the accumulated number of images reaches the set number (Y), the process moves to the sharpening process Step 10.

The sharpening step Step 10 is a step of sharpening the integrated image obtained in the integrating step Step 8. Specifically, the integrated image is sharpened using general techniques such as level correction and PSF (Point Spread Function) correction. Note that the sharpening step Step 10 can be omitted as necessary, or noise removal processing may be performed before the sharpening processing.

It goes without saying that the present invention is not limited to the embodiments described above, and that various changes can be made without departing from the spirit of the present invention.

1 Underwater laser viewing device 2 Underwater equipment 3 Laser light irradiation device 4 Imaging device 5 Processing device 41 Shutter Step 1 Photographing process Step 2 Image selection process Step 3 Noise removal process Step 4 Search process Step 41 Image reconstruction process Step 42 Contour extraction process Step 43 Object confirmation process Step 5 Extraction process Step 6 Integration availability confirmation process Step 7 Verification process Step 8 Integration process Step 9 Set number of sheets confirmation process Step 10 Sharpening process X Object α Area β Contour

Claims (2)

An underwater laser visual recognition device mounted on an underwater device that can be placed at a distance from an object in turbid water,
a laser beam irradiation device that oscillates a pulsed laser beam toward the target;
an imaging device equipped with a shutter that can be opened and closed according to the reflected light from the object;
a processing device that controls the laser light irradiation device and the imaging device and stores a plurality of images taken by the imaging device,
The processing device searches for whether or not the target object is included in the photographed image, extracts feature points from the search images determined to include the target object, and extracts feature points from the plurality of search images. It is configured to collate the feature points to determine a correspondence relationship, align the feature points, integrate the search image, and obtain an integrated image including the target object;
Furthermore, the processing device divides the photographed image into a plurality of regions, performs fast Fourier transform on each region to create a reconstructed image, and extracts the outline of the object from the reconstructed image, thereby processing the photographed image. configured to search whether or not the target object is included in the image, and configured to set the size of the area based on the wavelength of the result of fast Fourier transform of the entire captured image;
An underwater laser viewing device characterized by: The processing device is configured to create the reconstructed image using any one of the maximum value, average value, standard deviation, average value/maximum value, or first-order differential value of the fast Fourier transform. , An underwater laser viewing device according to claim 1 .