JP2011181984A - Image processing method, image processing apparatus, and underwater inspection device with the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method and apparatus, capable of correcting deflection of an image using only image information and stabilizing the image to improve the operability of an underwater inspection device, and to provide the underwater inspection device mounted with the image processing apparatus. <P>SOLUTION: An image processing method comprises the steps of: computing the amount of image deflection by image correlation operation for an acquired image at a time to be displayed and an acquired image at the last time; computing a moving average for the amount of image deflection for a certain time established beforehand; moving the acquired image at the last time by pixels corresponding to the moving average for the amount of image deflection to compute a corrected image; and displaying the corrected image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing method, an image processing apparatus, and an underwater inspection apparatus equipped with the same.

  Regarding the captured image processing of a camera mounted on a conventional moving body, first, a technique for stabilizing the visibility of a camera that a human wears on the head is known (for example, see Patent Document 1).

  Secondly, a technique is known in which a motion vector is detected from a captured image, and image shake is suppressed using only the captured image (see, for example, Patent Document 2).

  On the other hand, a technique of mounting a motion detection sensor in an underwater inspection apparatus and steering in an arbitrary direction is known (see, for example, Patent Document 3).

JP 2000-97637 A JP-A-5-219420 JP 2006-224863 A

  Patent Document 1 relates to a device that detects the movement using a gyro sensor mounted on a device that stabilizes the visibility of a camera that a human wears on the head. The technique of Patent Document 2 detects motion by image correlation processing. As described above, Patent Document 1 and Patent Document 2 are techniques for detecting image shake and correcting the amount of shake in order to stabilize the image.

  However, the underwater inspection apparatus that is the subject of the present invention not only shakes the apparatus, but also may intentionally change the attitude of the apparatus, so it is necessary to cause the image to follow in the opposite direction. Therefore, the techniques described in Patent Literature 1 and Patent Literature 2 cannot be applied to the underwater inspection apparatus.

  Patent Document 3 is an apparatus that controls the attitude of an underwater inspection apparatus by detecting the attitude using an internal sensor such as a gyro and performing feedback control, and is not a technique that stabilizes only an image.

  An object of the present invention is to provide an image processing method, an image processing apparatus, and an underwater inspection apparatus equipped with the image processing method, which can correct image shake using only image information, stabilize the image, and improve the operability of the underwater inspection apparatus. Is to provide.

  The present invention calculates an image shake amount calculated by image correlation calculation of an acquired image at a time to be displayed and an acquired image at the immediately preceding time, calculates a moving average of image shake amounts for a preset time, and calculates the previous time The acquired image is moved by a pixel corresponding to the moving average of the image shake amount, a corrected image is calculated, and the corrected image is displayed.

  According to the present invention, it is possible to correct image shake by using only image information and to stabilize the image, and the operability of the underwater inspection apparatus is improved.

It is a figure which shows apparatus arrangement | positioning of the underwater inspection apparatus in the underwater inspection work by Example 1. FIG. 1 is a diagram illustrating a configuration of an inspection vehicle in Embodiment 1. FIG. 6 is a diagram illustrating a definition of an image shake amount in Embodiment 1. FIG. 6 is a diagram illustrating a calculation image of an image shake amount in Embodiment 1. FIG. 6 is a diagram illustrating an image of a display image range in Embodiment 1. FIG. FIG. 6 is a diagram illustrating a flow of image processing in the first embodiment. It is a figure which shows the structure of the underwater camera in Example 2. FIG.

  The present invention relates to an image processing method for processing an image taken by a photographing device mounted on an underwater moving body, an image processing device, and an underwater inspection device equipped with the same, and more particularly, an underwater inspection device for inspecting an inside of a nuclear reactor, an atom The present invention relates to an image processing method and an image processing apparatus suitable for a swimming-type underwater inspection apparatus that visually inspects in-furnace equipment such as a jet pump in addition to a shroud and a pressure vessel in the furnace, and an underwater inspection apparatus equipped with the image processing apparatus.

  The configuration and operation of the underwater inspection apparatus of this embodiment will be described with reference to FIGS. The underwater inspection apparatus of the present embodiment is an apparatus used for defect inspection in a nuclear reactor, particularly for visual inspection of structures.

First, a device arrangement during an underwater inspection operation using the underwater inspection apparatus according to the present embodiment will be described with reference to FIG.
FIG. 1 is an equipment layout diagram during an underwater inspection operation using the underwater inspection apparatus of the present embodiment. In the nuclear reactor 1, there are structures such as a shroud 2, an upper lattice plate 3, a core support plate 4, and a shroud support 5. In addition, an operation floor 6 that is a work space is provided in the upper part of the nuclear reactor 1, and a fuel changer 7 is also provided in the upper part.

  The inspection vehicle 8 that has entered the nuclear reactor 1 is connected to the control device 10 via the vehicle cable 9. The control device 10 supplies power for moving the inspection vehicle 8 in water and navigating it, and performs video communication in order to perform a visual inspection at the inspection target location. In addition, a display device 11 is connected to the control device 10 and displays an image captured by an imaging means mounted on the inspection vehicle 8. Further, a controller 12 is connected to the control device 10 and is operated by a vehicle operator 13a. On the fuel changer 7, the operation assistant 13b runs the vehicle cable 9.

  Inside the control device 10, a display image is displayed for displaying on the display device 11 a position control means for controlling the movement of the position of the inspection vehicle 8 and an image picked up by the image pickup means mounted on the inspection vehicle 9. Image display processing means to be generated.

  Next, the configuration of the inspection vehicle 8 used in the underwater inspection apparatus of this embodiment will be described with reference to FIG. FIG. 2 is a bird's-eye view showing the configuration of the inspection vehicle 8 used in the underwater inspection apparatus.

  The inspection vehicle 8 is equipped with a camera unit 20 as an imaging means at the front. The camera unit 20 houses a camera and illumination. The vehicle operator 13a shown in FIG. 1 is configured to be able to operate the movement or the like of the inspection vehicle 8 while confirming the image from the camera unit 20.

  The inspection vehicle 8 is equipped with a drive mechanism for moving in water. This drive mechanism is equipped with a lifting thruster 21 for moving up and down, a propulsion thruster 22 for moving forward and backward, and translational turning thrusters 23a and 23b for moving and turning left and right. The translation turning thrusters 23a and 23b are rotated in the same direction when moving left and right, and rotated in the opposite direction when turning.

Next, the concept of the image processing method by the underwater inspection apparatus of the present embodiment will be described with reference to FIGS.
FIG. 3 shows the definition of the image shake amount. FIG. 3 is an image in which an image captured by the camera unit 20 is taken into the control device 10. In the captured image 30, it is assumed that the feature point 31 at time T _i-1 has moved to the position of the feature point 32 at the next time T _i . The movement amount (a, b) at this time is defined as an image shake amount 33. This image shake amount 33 is calculated by image correlation processing described later, and (a, b) indicates the shake amounts in the X direction and Y direction of the image. FIG. 3 illustrates changes in feature points between two images. Then, when the image shake amount is considered in a video which is a set of a plurality of continuous images, the feature points are continuous shakes. The cause of the shake of the feature point can be divided into a direction change intended by the operator and an unintended shake. Of these, the direction change intended by the operator has a low frequency, and the unintended shake is considered to have a high frequency.

FIG. 4 shows an image of a system in which only the high-frequency component of the image shake is removed and only the direction change intended by the operator is reflected in the image display. In FIG. 4, the image shake amount change 40 has a large vibration (shake width) as shown by the solid line. Here, a moving time 41 of image shake amounts of 3 samples, for example, a preset time is calculated. As a result of this processing, the high frequency component is removed, and only the change intended by the operator remains. Compared with the amount of change 42 from time T _i-1 to T _i when the moving average is not performed, the amount of change when the moving average is performed is small. This amount is used as the image correction amount 43 at time T _i . The image correction amount 43 is calculated independently in a and b in FIG. 3, that is, in the X direction and the Y direction, and the values are A and B, respectively.

FIG. 5 is a diagram showing an image of the display image range. The display image 51 at the time T _i is displayed by being shifted by the image shift amount 52 in the X direction and the image shift amount 53 in the Y direction with respect to the display image 50 at the time T _i−1 . Note that there is no change in the display size in either case of time T _i-1 or time T _i . For this reason, a portion having no image information is generated by the amount of shifting the display image. Therefore, a blank area 54 is inserted at time T _i .

  Next, the processing content of the image display method by the underwater inspection apparatus of a present Example is demonstrated using FIG. These processes are executed by the image display processing means of the control device 10 shown in FIG. 1, and are flowcharts for embodying the methods of FIGS.

After starting the process in step S00, a time constant N is input in step S01. This time constant indicates the width of the moving average shown in FIG. 4 and is specifically the number of moving average samples. Next, in step S02, initial image processing at time T ₀ is performed. In step S03, an initial photographed image is read. In step S04, distortion caused by the camera lens is corrected. In step S05, the initial image is stored. Next, in step S06, an initial image correlation within a range determined by the time constant N is calculated. Specifically, a subroutine for calculating the amount of image blur from step S07 to step S08 is called and repeated calculation is performed. In step S08, first, an image at the current time T _i is acquired in step S09. Specifically, the image at the current time is captured in step S10, distortion is corrected in step S11, and the current image is stored in step S12. Next, an image at the previous time T _i-1 is read in step S13, and an image correlation calculation is performed in step S14. As a result, in step S15, the image shake amount (a, b) shown in FIG. 3 is determined. In step S16, the image shake amount (a, b) is stored.

Next, the time is incremented in step S17, and image correction processing is performed in step S18. Here, the image correction processing in step S18 will be described. First, in order to calculate the image shake amount at the current time, step 19 calls subroutine step S08, and performs the processing from step S09 to step S16. Next, in step S20, the past image shake amount stored in step S16 is read. What is read here is the result of the time from i-N to i-1 with respect to the current time i. For example, when the time constant N = 3, step S20 reads the result of time from i-3 to i-1. However, when the current time is T ₁ or T ₂ , the number of acquired images in the past is less than the time constant (N = 3). Therefore, if the current time is T ₁ or T ₂ , step S20 is skipped.

Step S21 calculates the moving average (A, B) of the image shake amount using the result of step S20, and step S22 shifts the image at the previous time T _{i-1 by} the number of pixels of (A, B). This step is based on the method shown in FIG. Step S23 displays the result as a corrected image. At this time, as shown in FIG. 5, a blank area is inserted and displayed at the edge of the screen. Finally, loop determination is performed in the next step S24. If there is no end input from the operator in step S25, the process jumps from step S26 to step S27, and the processes from step S17 to step S23 are repeated. If there is an end input from the operator, end processing is performed in step S28.

  As described above, in this embodiment, the image shake amount calculated by the image correlation calculation (S14) between the acquired image at the time to be displayed and the acquired image at the immediately preceding time is calculated, and the movement of the image shake amount for a preset time is calculated. An average (S21) is calculated, and the corrected image is calculated by moving the acquired image at the immediately preceding time by a pixel corresponding to the moving average of the image shake amount (S22), and the corrected image is displayed (S23). By using the moving average (S21) of the image shake amount, the high-frequency component of the image shake amount is removed, and only the change intended by the operator can be reflected in the image display. For this reason, it is possible to correct the shake of the image using only the image information and to follow the intended change in posture. As a result, it is possible to stabilize the captured image mounted on the moving body, and improve operability during movement.

  Moreover, it becomes possible to stabilize the image of the camera mounted in the underwater inspection apparatus, and the inspection efficiency in visual inspection can be improved.

  Next, the configuration and operation of the underwater inspection apparatus of this embodiment will be described with reference to FIG. Compared with FIG. 1, the arrangement of the equipment during the underwater inspection work using the underwater inspection apparatus according to the present embodiment is that the underwater camera 70 is used instead of the inspection vehicle 8, and the cable is included instead of the vehicle cable. The difference is that the holding jig 71 is used. Other configurations, image processing methods, and display methods are the same as those in FIG. The underwater camera 70 includes a camera unit 72 and lamp units 73a and 73b. The control device 10 performs image processing of the video captured by the camera unit 72 and displays it on the display device 11.

  The present embodiment is intended to achieve the same purpose as the first embodiment. However, unlike a migration type inspection vehicle 8, a device on which a camera is mounted uses a holding jig with a small up and down method. Therefore, the shake that is not desired by the operator is mainly a horizontal shake. In this case, when the image shake amount (FIG. 4) is calculated, the same effect can be obtained by correcting only the horizontal direction of the image, that is, the X direction. Therefore, the calculation time required for the image processing is shortened and the responsiveness is improved. It becomes possible.

  As described above, according to the present embodiment, it is possible to correct the shake of the image using only the image information and to follow the intended change in posture. As a result, it is possible to stabilize the captured image mounted on the moving body, and improve operability during movement.

  Moreover, it becomes possible to stabilize the image of the camera mounted in the underwater inspection apparatus, and the inspection efficiency in visual inspection can be improved.

DESCRIPTION OF SYMBOLS 1 Reactor 2 Shroud 3 Upper lattice plate 4 Core support plate 5 Shroud support 6 Operation floor 7 Fuel change device 8 Vehicle for inspection 9 Vehicle cable 10 Control device 11 Display device 12 Controller 20 Camera unit 21 Lifting thruster 22 Propulsion thruster 23a, 23b Translational turning thruster 30 Captured image 31 Feature point 32 at time T _i-1 Feature point 33 at time T _i Image shake amount 40 Change in image shake amount 41 Moving average 42 of image shake amount From time T _i-1 T image correction amount in the change amount 43 times T _i to _i 50 time T _i-1 shift amount 54 times the shift amount 53 Y direction of the image of the display image 52 X-direction of the image in the display image 51 time T _i in T _i Blank area 70 underwater camera 71 holding jig 72 camera 73a, 73b knit

Claims (5)

An image processing method for displaying a corrected image using an acquired image taken by an imaging means mounted on a moving body,
Calculate the image shake amount calculated by the image correlation calculation of the acquired image at the time to be displayed and the acquired image at the previous time,
Calculate the moving average of the image shake amount for a preset time,
An image processing method comprising: calculating a corrected image by moving an acquired image at the immediately preceding time by a pixel corresponding to a moving average of the image shake amount, and displaying the corrected image. The image processing method according to claim 1,
The time for calculating the moving average is a time traced back by a time constant. The image processing method according to claim 1,
An image processing method, wherein the correction image includes inserting a blank area into a data blank area corresponding to a moving average of the image shake amount generated by moving an acquired image at the immediately preceding time. An image processing apparatus that displays a corrected image using an acquired image captured by an imaging unit mounted on a moving body,
Means for calculating an image shake amount calculated by image correlation processing of the acquired image at the time to be displayed and the acquired image at the immediately preceding time;
Means for calculating a moving average of the image shake amount for a preset time;
Means for calculating a corrected image by moving the acquired image at the immediately preceding time by a pixel corresponding to a moving average of the image shake amount;
An image processing apparatus comprising means for displaying the corrected image. An inspection vehicle having a driving mechanism capable of migrating in three dimensions and an imaging means capable of visually recognizing a structure in water, and an image with a preset image display size cut out from an image captured by the imaging means, and an image obtained by correcting shake An underwater inspection apparatus having image display processing means for displaying,
The image display processing means calculates an image shake amount calculated by image correlation processing of an acquired image at a display time and an acquired image at the immediately preceding time;
Means for calculating a moving average of the image shake amount for a preset time;
Means for calculating a corrected image by moving the acquired image at the immediately preceding time by a pixel corresponding to a moving average of the image shake amount;
An underwater inspection apparatus comprising means for displaying the corrected image.

Images