JP2010066963A - Image processing method, image processor, and underwater inspection device with the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing method, an image processor, and an underwater inspection device with the image processor, for correcting the deflection of an image by using only image information, and for stabilizing the image, and for improving the operability of the underwater inspection device. <P>SOLUTION: A display image 34 at a certain time (t) exists in a photographic image 33a at the time (t). The display image 34 is positioned at the center of the photographic image 33a in an initial state, and then updated by the following methods. A section 35 highly correlated with the display image 33a at the time (t) in a photographic image 33b at a time (t+1) is retrieved by image correlation processing. An image positioned between the image region 35 similar to the display image 34 at the time (t) and an image 36 located at the center of the photographic image at a time (t+1) is displayed as a display image 37 so as to shift from the section 35 highly correlated with the display image at the time (t) to the central image 36 of the photographic image 33b at the time (t+1) by a preset time constant. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing method, an image processing apparatus, and an underwater inspection apparatus equipped with the image processing method for processing an image captured by an imaging apparatus mounted on an underwater mobile body, and more particularly, an underwater inspection apparatus for inspecting the inside of a nuclear reactor, particularly The present invention relates to an image processing method, an image processing apparatus, and an underwater inspection apparatus equipped with the image processing method suitable for use in a swimming-type underwater inspection apparatus that visually inspects in-reactor equipment such as a jet pump in addition to a shroud and pressure vessel in a nuclear reactor .

  Regarding the captured image processing of a camera mounted on a conventional moving body, firstly, one that stabilizes the visibility of a camera that a human wears on the head is known (for example, see Patent Document 1).

  Secondly, there is known a technique that detects a motion vector from a photographed image and suppresses image blur using only the photographed image (see, for example, Patent Document 2).

  On the other hand, there is known a device in which a motion detection sensor is mounted on an underwater inspection apparatus and steered in an arbitrary direction (for example, see Patent Document 3).

JP 2000-97637 A JP-A-5-219420 JP 2006-224863 A

  The thing of patent document 1 is related with the apparatus which stabilizes the visibility of the camera with which a human body mounts to a head, and detects a motion using the mounted gyro sensor. Moreover, in the thing of patent document 2, a motion is detected by the correlation process of an image. As described above, the devices described in Patent Document 1 and Patent Document 2 detect image shake and correct only the amount of shake in order to stabilize the image.

  However, the underwater inspection apparatus that is the subject of the present invention not only shakes the apparatus, but also may intentionally change the attitude of the apparatus, so it is necessary to follow the image in the opposite direction. The technique described in Patent Document 2 cannot be applied to an underwater inspection apparatus.

  Also, what is described in Patent Document 3 is an apparatus that controls the attitude of the underwater inspection apparatus by detecting the attitude using an internal sensor such as a gyro and performing feedback control, and stabilizes only the image. It is not a thing.

  An object of the present invention is to provide an image processing method, an image processing apparatus, and an underwater inspection apparatus equipped with the image processing method, which can correct image shake using only image information, stabilize the image, and improve the operability of the underwater inspection apparatus. Is to provide.

(1) In order to achieve the above object, the present invention is an image display method for cutting out an image having a preset image display size from a photographed image photographed by photographing means mounted on a moving body and displaying the clipped image. In the captured image at the time to be displayed, an image located between an image region similar to the display image immediately before the time and the image having the image display size at the center of the captured image at the time Is displayed as a display image.
With this method, it is possible to correct the shake of the image and stabilize the image using only the image information.

  (2) In the above (1), preferably, using the immediately preceding display image as a template, an image correlation calculation is performed in the captured image at the time, and a region where the correlation coefficient is equal to or greater than a preset threshold value The search is performed to obtain an image region similar to the display image immediately before the time.

  (3) In the above (1), preferably, a feature region including a feature point is searched from a photographed image immediately before the time, and an image is captured in the photographed image at the time using the feature region as a template. A correlation calculation is performed, a template movement area where the correlation coefficient is equal to or greater than a preset threshold is searched, a relative position of the template movement area with respect to the template is set as an image shake, and a display image immediately before that time is captured The image shake is added to the position in the image to define the cutout position at the time, and the image at the cutout position in the captured image at the time is used as the display image at the time.

  (4) In the above (1), preferably, the center of the display image is the center of the image area similar to the display image immediately before the time in the captured image at the time and the center of the captured image at the time. The distance from the center of the image region similar to the display image immediately before the time in the photographed image at the time is located on a line segment formed from the center of the image of the image display size in the predetermined threshold. The restriction is as follows.

(5) Further, in order to achieve the above object, the present invention includes an imaging unit mounted on a moving body, and an image display control unit that cuts out an image having a preset image display size from an image captured by the imaging unit. An image display device having image display means for displaying an image cut out by the image display control means, wherein the display image is an image similar to the display image immediately before the time in the captured image at the time Image display processing means for calculating an image positioned between the area and the image having the image display size at the center of the photographed image at the time is provided.
With this configuration, it is possible to correct the shake of the image and stabilize the image using only the image information.

(6) Furthermore, in order to achieve the above-mentioned object, the present invention is based on an inspection vehicle having a driving mechanism capable of moving in three dimensions and an imaging unit capable of visually recognizing a structure in water, and an image captured by the imaging unit. An underwater inspection apparatus having an image display control means for cutting out an image having a preset image display size, and an image display means for displaying an image cut out by the image display control means, wherein the image display processing means In the captured image at the time, the image is calculated as an image located between the image area similar to the display image immediately before the time and the image of the image display size at the center of the captured image at the time. Is.
With such a configuration, it is possible to correct the shake of the image using only the image information, stabilize the image, and improve the operability of the underwater inspection apparatus.

  According to the present invention, it is possible to correct image shake by using only image information and to stabilize the image, and the operability of the underwater inspection apparatus is improved.

  Hereinafter, the configuration and operation of an underwater inspection apparatus according to an embodiment of the present invention will be described with reference to FIGS. The underwater inspection apparatus according to the present embodiment is used for defect inspection in a nuclear reactor, particularly for visual inspection of a structure.

First, with reference to FIG. 1, an equipment arrangement during an underwater inspection work using the underwater inspection apparatus according to the present embodiment will be described.
FIG. 1 is an equipment layout diagram for underwater inspection work using an underwater inspection apparatus according to an embodiment of the present invention.

  In the nuclear reactor 1, there are structures such as a shroud 2, an upper grid plate 3, a core support plate 4, and a shroud support 5, and pipes such as a PLR pipe 6 are connected. In addition, an operation floor 7 that is a work space is provided in the upper part of the nuclear reactor 1, and a fuel changer 8 is provided in the upper part.

  The inspection vehicle 9 that has entered the nuclear reactor 1 is connected to the control device 11 via the vehicle cable 10. The control device 11 supplies power for moving the inspection vehicle 9 in the water and navigating it, and performs video communication in order to perform a visual inspection at the inspection target location. In addition, a display device 12 is connected to the control device 11 and displays an image of an imaging means mounted as an imaging means on the inspection vehicle. Further, a controller 13 is connected to the control device 11 and is operated by the vehicle operator 14a. On the fuel changer 8, the operation assistant 14b runs the vehicle cable 10.

  The control device 11 includes a position control means for controlling the position movement of the inspection vehicle 9 and a display for displaying on the display device 12 an image picked up by the image pickup means mounted on the inspection vehicle 9. Image display processing means for generating an image.

Next, the configuration of the inspection vehicle used in the underwater inspection apparatus according to the present embodiment will be described with reference to FIG.
FIG. 2 is a cross-sectional perspective view showing the configuration of the inspection vehicle used in the underwater inspection apparatus according to the embodiment of the present invention.

  The inspection vehicle 9 is equipped with a camera 20 with a wide-angle lens and an illumination 21 as imaging means at the front. The camera 20 and the illumination 21 are housed inside a transparent dome 22. The vehicle operator 14a shown in FIG. 1 is configured to be able to operate the movement of the inspection vehicle 9 while confirming the image of the camera 20.

  Further, a propulsion thruster 24 for moving back and forth, a turning screw 26 for turning left and right, and a balance weight 28 for adjusting the pitch posture are mounted. The propulsion thruster 24 is driven by the ROV drive motor 23a. The turning screw 26 is driven by the ROV driving motor 23b. The balance weight 28 is driven by the ROV driving motor 23c. The balance weight 28 transmits the power of the ROV driving motor 23 c via a plurality of gears 27. The turning screw 26 transmits the power of the ROV driving motor 23 b via the bevel gear 25.

Next, the concept of the image display method by the underwater inspection apparatus according to the present embodiment will be described with reference to FIG.
FIG. 3 is an explanatory diagram of the concept of the image display method by the underwater inspection apparatus according to the embodiment of the present invention.

  The image example shown in FIG. 3 is an image obtained by enlarging a part of the shroud 2 in the nuclear reactor 1. Then, the welding line 31 at the boundary between the shroud inner wall 30a and the leg 30b located at the bottom of the shroud 2 is photographed by the camera 20 of the inspection vehicle 9 shown in FIG. 2, and displayed on the display device 12 shown in FIG. The image to be visually inspected is shown from the image to be displayed.

  It is assumed that there is a display image 34 at time (t) in the captured image 33a at a certain time (t). This display image 34 is positioned in the center of the captured image 33a in the initial state, and is subsequently updated by the method described below. It is assumed that the captured image 33b at time (t + 1) appears at a different position from the captured image at time (t). This is an event that occurs when the posture of the inspection vehicle 9 changes from the position at time (t) to the upper right in the state shown in the figure, as indicated by an arrow A1. The change in posture may be caused by an unintended disturbance or intentionally changed in direction. An unintentional disturbance is a flow of water inside the nuclear reactor 1. In the case of intentionally changing the direction, when inspecting the weld line 31, for example, the inspection vehicle 9 is moved in the horizontal direction (X-axis direction). However, even in such a case, when the small inspection vehicle 9 is used, since the weight is light, the movement may be larger than the intended movement amount. Here, whether the change in posture is due to unintentional disturbance or intentional change of direction is intended if the posture changes for a long time. Interpreted as a change of direction.

  In the present embodiment, when there is such a change in posture beyond the originally planned position, the control device 11 shown in FIG. 1 displays a display image corresponding to the posture change as described below. The following process is performed.

  Here, in the captured image 33b at time (t + 1), a portion 35 having a high correlation with the display image 33a at time (t) is searched by image correlation processing. The center image 36 of the captured image 33b at time (t + 1) indicates an image in which the inspection vehicle 9 is directly facing the front.

  Therefore, processing is performed to follow the center image 36 of the captured image 33b at time (t + 1) from the portion 35 having a high correlation with the display image at time (t) with a preset time constant. The image to be displayed is a display image 37 at time (t + 1) shown in the figure. Specifically, the maximum rate of change from the display image 33a at time (t) to the highly correlated portion 35 is set to an amount obtained by dividing the number of pixels of the display image by an arbitrary value. For example, the number of pixels is 480, and an arbitrary value is divided by 30 (fps), which is a video update cycle (frame rate), to obtain 16 pixels / frame.

  For example, it is assumed that the deviation amount in the X-axis direction between the highly correlated portion 35 and the center image 36 is ΔX1. For example, the shift amount ΔX1 is set to several tens of pixels. As described above, when the central image 36 moves significantly with respect to the display image at the time (t), the display image 37 at the time (t + 1) has a ΔX2 (ΔX2 ( Only <ΔX1) is moved. The movement amount ΔX2 is 16 pixels / frame in the above example. The same applies to the Z-axis direction.

  Thereby, for example, when the visual inspection is performed along the weld line 31, if the state of the display image 34 at the time (t) shifts to the display image 37 at the time (t + 1), how far the visual inspection is performed. Although it becomes unknown and the inspection accuracy decreases, the inspection accuracy can be improved by displaying the display image 37 at time (t + 1).

  Thus, even when there is a posture change due to an unintended disturbance, for example, an image shifted by 16 pixels / frame is displayed on the display image 34 at time (t), so that the display image is substantially retained. State. Further, even when the posture change is intentionally performed and the posture change amount is too large, for example, an image shifted by 16 pixels / frame is displayed with respect to the display image 34 at time (t). The display image in the posture change direction can be obtained.

Next, processing contents of the image display method by the underwater inspection apparatus according to the present embodiment will be described with reference to FIG. These processes are executed by the image display processing means of the control device 11 shown in FIG.
FIG. 4 is a flowchart showing the processing contents of the image display method by the underwater inspection apparatus according to the embodiment of the present invention.

  After the start of processing, initial image processing is performed in step S00. First, in step S01, an initial image is captured. Next, in step S02, distortion included in the camera lens is corrected. Next, in step S03, an initial effective image is displayed. In step S03A, a display image position (for example, 640 × 480 pixels) is defined at the center of the captured image. Next, in step S03B, the image is cut out and displayed in step S03C. In step S04, template registration is performed. In step S04A, the feature amount in the photographed image is extracted. In step S04B, the surrounding window size is defined, and registered in step S04C. Here, as a feature amount extraction method, for example, a method described in Japanese Patent Laid-Open No. 5-219420 is used. The window size is the same as the display image, for example, 640 × 480 pixels.

  Next, the basic image processing of step S10 is entered. First, as basic image processing, a captured image is acquired in step S11, and distortion correction is performed in step S12. Next, the process proceeds to image blur correction processing in step S20. First, in step S21, a registered template is acquired. Next, in step S22, a shake amount calculation process is executed. First, in step S22A, image correlation processing between the captured image and the template is performed. Here, in step S22B, the shake amount is defined as the center distance between the portion of the captured image where the image correlation coefficient of the template is equal to or greater than a preset threshold and the position of the display image at the previous time. As a result, in step S22C, the center of the display image is updated to one obtained by adding the shake amount to the display image center position at the previous time. Further, in step S23, the display image is cut out and displayed in step S24.

  Subsequently, the process proceeds to a posture change tracking process in step S30. First, in step S31, a follow-up amount calculation process is performed. Here, in step S31A, a vector in the direction from the center of the display image to the center of the captured image is obtained and set as the follow-up direction. In step S31B, the distance between the center of the display image and the center of the captured image is calculated and set as the center shift amount (Δmax). Further, in step S31C, the follow-up restriction amount (Δlim) is obtained as a half value (pixel / fps) of the image size. Based on these calculation results, a follow-up amount is obtained in step S31D. Here, in step S32, it is defined as the smaller value of the center shift amount (Δmax) and the tracking limit amount (Δlim). In step S33, the obtained follow-up amount is added to the display image center position, and the display image center position is updated. In step S34, the resulting display image is cut out and displayed.

  Finally, the process proceeds to template processing in step S40. In step S41, the display image is registered as a template.

  Further, the process proceeds to a loop process in step S50. If the process is not ended in step S51, the process returns to FIG. 4A, and a series of process loops from the image basic process is repeated.

  As described above, according to the present embodiment, it is possible to correct the shake of the image using only the image information and to follow the intended change in posture. As a result, it is possible to stabilize the captured image mounted on the moving body, and improve operability during movement.

  Moreover, it becomes possible to stabilize the image of the camera mounted in the underwater inspection apparatus, and the inspection efficiency in visual inspection can be improved.

  Next, the configuration and operation of an underwater inspection apparatus according to another embodiment of the present invention will be described with reference to FIGS. 5 and 6. The equipment arrangement during the underwater inspection work using the underwater inspection apparatus according to the present embodiment is the same as that shown in FIG. The configuration of the inspection vehicle used in the underwater inspection apparatus according to the present embodiment is the same as that shown in FIG.

  FIG. 5 is an explanatory view of the concept of an image display method by the underwater inspection apparatus according to another embodiment of the present invention. FIG. 6 is a flowchart showing the processing contents of the image display method by the underwater inspection apparatus according to another embodiment of the present invention.

  This embodiment is intended to achieve the same object as the above-described embodiment, but is suitable when there is no feature point in the display image and it is difficult to obtain image correlation.

  First, the concept of the image processing method will be described with reference to FIG. The image example shown in FIG. 5 is the same as that of the first embodiment.

  First, the display image 42 at the time (t) is defined in the captured image 41a at a certain time (t). The display image 42 is located in the center of the captured image 41a in the initial state. However, in this example, although there is an image of the weld line in the display image, it is an image on the line and image correlation cannot be calculated. Thus, when there is no feature point in the display image 42, the feature point 43a in time (t) is searched from areas other than the display image 42, and it registers as a temporary template. Next, it is assumed that the captured image 41b at time (t + 1) appears at a different position from the captured image 42 at time (t). Here, a portion 43b having a high correlation with the temporary template including the feature point at the time (t) is searched for in the captured image 41b at the time (t + 1). Next, the positions 44 and 43b where the temporary template 43a including the feature point at time (t) appeared are searched by image correlation processing. From the result, the amount of movement of the temporary template is calculated, and the amount is added to the position of the display image 42 at time (t) to obtain the central image at time (t + 1).

  Next, processing contents of the image display method by the underwater inspection apparatus according to the present embodiment will be described.

  The overall processing content is the same as that shown in FIG. 4, but the content of the image blur processing in step S20 is different. FIG. 6 shows the contents of image blur correction processing. In FIG. 6, compared with FIG. 4, steps S26 to S28 and S22 'are added between step S21 and step S22.

  First, in step S21, a template, that is, a display image at the previous time is acquired, and in step S26, it is determined whether or not there is a feature point in the template. If there is a feature point here, the amount of shake is calculated and the display image is cut out and displayed in steps S22 to S24 as in FIG.

  If there is no feature point in the template, the feature point is extracted from the entire photographed image at the previous time in step S27. In step S28, the region including the feature point is registered as a temporary template. The temporary template need not be the size of the display image and can be set arbitrarily. For example, when the display image is 640 × 480, the temporary template is 100 × 100.

  Next, in step S22 ', the shake amount is calculated. Here, in step S22A ', an area having a high correlation with the temporary template is searched from the photographed image at the current time. Also in this case, as in the example shown in FIG. 4, a portion where the image correlation coefficient is equal to or greater than a preset threshold is searched. Thereby, the movement amount of the position of the temporary template is obtained. Here, in step S22B ′, the shake amount is defined as the center distance between the portion of the captured image where the image correlation coefficient of the temporary template is equal to or greater than a preset threshold and the position of the display image at the previous time. . As a result, in step S22C, the center of the display image is updated to one obtained by adding the shake amount to the display image center position at the previous time. Further, in step S23, the display image is cut out and displayed in step S24.

  As described above, according to the present embodiment, it is possible to correct the shake of the image using only the image information and to follow the intended change in posture. As a result, it is possible to stabilize the captured image mounted on the moving body, and improve operability during movement.

  Moreover, it becomes possible to stabilize the image of the camera mounted in the underwater inspection apparatus, and the inspection efficiency in visual inspection can be improved.

Further, even when there are no feature points in the display image, it is possible to calculate the image correlation and correct the shake of the image. However, when there is no feature amount in the entire captured image, it cannot be applied, but using a wide-angle lens as an imaging unit can widen the field of view and avoid it.

It is an equipment layout at the time of underwater inspection work using the underwater inspection device by one embodiment of the present invention. It is a section perspective view showing the composition of the inspection vehicle used for the underwater inspection device by one embodiment of the present invention. It is explanatory drawing of the concept of the image display method by the underwater inspection apparatus by one Embodiment of this invention. It is a flowchart which shows the processing content of the image display method by the underwater inspection apparatus by one Embodiment of this invention. It is explanatory drawing of the concept of the image display method by the underwater inspection apparatus by other embodiment of this invention. It is a flowchart which shows the processing content of the image display method by the underwater inspection apparatus by other embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Reactor 2 ... Shroud 3 ... Upper lattice board 4 ... Core support plate 5 ... Shroud support 6 ... PLR piping 7 ... Operation floor 8 ... Fuel changer 9 ... Inspection vehicle 20 ... Camera 21 with wide angle lens ... Illumination 22 ... Transparent dome 23a, 23b, 23c ... ROV drive motor 24 ... propulsion thruster 25 ... bevel gear 26 ... turning screw 27 ... gear box 28 ... balancer

Claims (6)

An image display method for cutting out an image having a preset image display size from a photographed image photographed by photographing means mounted on a moving body, and displaying the clipped image,
In the captured image at the time to be displayed, an image located between the image area similar to the display image immediately before the time and the image of the image display size at the center of the captured image at the time is displayed. An image display method characterized by displaying as an image. The image display method according to claim 1,
Using the immediately preceding display image as a template, image correlation calculation is performed in the captured image at the time, searching for a region where the correlation coefficient is equal to or greater than a preset threshold, and similar to the display image immediately before the time An image display method characterized in that an image area is formed. The image display method according to claim 1,
Search for a feature region including a feature point from the captured image immediately before the time,
Using the feature area as a template, image correlation calculation is performed in the captured image at the time, a template movement area having a correlation coefficient equal to or greater than a preset threshold is searched, and the template movement area with respect to the template The relative position of
The image shake is added to the position in the captured image of the display image immediately before the time to define the cutout position at the time, and the image at the cutout position in the captured image at the time is the display image at the time An image display method characterized by: The image display method according to claim 1,
The center of the display image is composed of the center of the image area similar to the display image immediately before the time in the captured image at the time and the center of the image with the image display size at the center of the captured image at the time. Located on the line,
An image display method characterized in that a distance from the center of an image area similar to a display image immediately before the time in a captured image at the time is limited to a predetermined threshold value or less. Image capturing means mounted on a moving body, image display control means for cutting out an image having a preset image display size from an image captured by the image capturing means, and image display means for displaying an image cut out by the image display control means An image display device comprising:
An image positioned between the image area similar to the display image immediately before the time and the image of the image display size at the center of the captured image at the time in the captured image at the time. An image display apparatus comprising image display processing means for calculating as follows. An inspection vehicle having a driving mechanism capable of migrating in three dimensions and an imaging means capable of visually recognizing a structure in water, an image display control means for cutting out an image of a preset image display size from the image taken by the imaging means, An underwater inspection apparatus having image display means for displaying an image cut out by the image display control means,
The image display processing means is positioned between an image region similar to the display image immediately before the time and an image of the image display size at the center of the captured image at the time in the captured image at the time. An underwater inspection apparatus characterized by being calculated as an image to be processed.

Images