JP2019106075A - Inspection image synthesizing device, inspection image synthesizing system, inspection image synthesizing method and program - Google Patents

Abstract

To provide an inspection image synthesizing device, an inspection image synthesizing system, an inspection image synthesizing method, and a program that can specify, in a shorter time, an overlapping portion of a plurality of images photographed for visual inspection.SOLUTION: An inspection image synthesizing device includes: an image information acquisition unit that acquires a photographed image and photographing position information from a photographing device that scans a surface of an inspection object and photographs at constant time intervals; a movement amount calculation unit that calculates, from the photographing position information for each of images adjacent in time series, a movement amount between the images; a search range setting unit that calculates a range of overlapping pixels between the images adjacent in time series based on the movement amount, and sets a range as a search range in template matching; and an image synthesizing unit that performs template matching in the search range to specify the range of overlapping pixels between the images, and synthesizes a panoramic image in which the specified range of pixels is overlapped.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an inspection image combining device, an inspection image combining system, an inspection image combining method, and a program.

As a method of visually inspecting the inspection object in the place where the worker can not enter, make the mobile photographing device scan the surface of the inspection object at the corresponding place and make the photographing, and use the obtained image for visual inspection There is a way to do that. In this manner, a plurality of obtained images may be manually overlaid to generate a panoramic image. In particular, in visual inspection, in general, it is required to confirm an object with high resolution, so the angle of view becomes narrow, and a defect or the like may be photographed across a plurality of images. In such a case, a panoramic image may be manually generated in order to obtain an overview of a flaw or the like.
Further, in Patent Document 1, the SN ratio is achieved by locally aligning a plurality of frames of a video containing substantially independent noise with respect to each frame obtained by photographing an inspection target. It is stated that the high frame is generated.

Patent No. 5602530

  Manually generating a panoramic image is a burden for the operator, and preferably, the generation of the panoramic image can be automated. On the other hand, images obtained by visual inspection are generally poor in color tone difference, so it is difficult or time-consuming to identify overlapping portions of a plurality of images. It is preferable that overlapping portions of a plurality of images captured for visual inspection can be identified in a shorter time.

  Further, Patent Document 1 does not disclose a specific method of detecting an overlapping portion of a plurality of images when the imaging apparatus moves and performs imaging. For example, although Patent Document 1 describes that a frame is divided into local regions of the same shape and registration is performed, it is not described which local region and which local region are to be aligned. .

  The present invention provides an inspection image combining device, an inspection image combining system, an inspection image combining method, and a program capable of specifying an overlapping portion of a plurality of images captured for visual inspection in a shorter time.

  According to the first aspect of the present invention, the inspection image synthesizing device acquires the photographed image and the photographing position information from the photographing device which scans the surface of the inspection object and photographs at predetermined time intervals. An acquisition unit, a movement amount calculation unit calculating a movement amount between the images based on shooting position information of each of the images adjacent in time series, and moving a range of overlapping pixels between the images adjacent in the time series A search range setting unit that calculates based on the amount and sets it as a search range in template matching and performs template matching in the search range to specify the range of overlapping pixels between the images, and overlaps the specified range of pixels And an image combining unit that combines the combined panoramic images.

  The search range setting unit may set the search range having a width corresponding to the variation of the movement amount.

  The search range setting unit may set the search range having a width corresponding to the variation of the movement amount in each of the vertical direction and the horizontal direction of the pixel of the image.

  The search range setting unit may calculate a width according to the variation of the movement amount from time-series images captured by the imaging device in the past.

  The search range setting unit may calculate a width according to the variation of the movement amount from operation test data of the imaging device.

  The search range setting unit sets the difference between the maximum value or the minimum value of the moving amount in the plurality of actual data of the moving amount and the standard moving amount of the imaging device as the maximum variation of the moving amount. The width of the search range may be calculated according to the maximum value of the variation of.

  The search range setting unit sets a value obtained by adding a margin to the magnitude of the variation of the moving amount obtained from the actual data of the moving amount as the maximum value of the variation of the moving amount, the maximum of the variation of the moving amount. The width of the search range according to the value may be calculated.

  The search range setting unit may calculate the width of the search range according to the maximum value of the variation of the movement amount, with 3σ in the frequency distribution of the movement amount as the maximum value of the variation of the movement amount.

  The search range setting unit may calculate a width corresponding to the variation of the movement amount based on scan locus information set to cause the imaging device to scan the surface of the inspection object.

  According to the second aspect of the present invention, the inspection image synthesizing system scans the surface of the inspection object and performs imaging at predetermined time intervals, an image captured by the imaging device, and imaging position information An image information acquisition unit to be acquired, a movement amount calculation unit calculating a movement amount between the images from the photographing position information for each of the images adjacent in time series, and pixels overlapping between the images adjacent in the time series A search range setting unit which calculates a range based on the movement amount and sets it as a search range in template matching, and performs template matching in the search range to specify the range of overlapping pixels between the images, and specifies the specified pixels And an image synthesizing unit for synthesizing a panoramic image in which the ranges of.

  According to a third aspect of the present invention, there is provided an inspection image combining method which acquires a photographed image and photographing position information from a photographing device which scans a surface of a test object and photographs at predetermined time intervals. From the photographing position information of each of the images adjacent in time series, calculating the movement amount between the images, and calculating the range of overlapping pixels between the images adjacent in the time series based on the movement amount Setting as a search range in template matching, and performing template matching in the search range to specify a range of overlapping pixels between the images, and combining a panoramic image in which the specified range of pixels is superimposed. ,including.

  According to the fourth aspect of the present invention, the program causes the computer to acquire a photographed image and photographing position information from a photographing device that scans the surface of the inspection object and performs photographing at predetermined time intervals. From the photographing position information of each of the images adjacent in time series, calculating the movement amount between the images, and calculating the range of overlapping pixels between the images adjacent in the time series based on the movement amount Setting as a search range in template matching, and performing template matching in the search range to specify a range of overlapping pixels between the images, and combining a panoramic image in which the specified range of pixels is superimposed. , Is a program to execute.

  According to the inspection image synthesizing device, the inspection image synthesizing system, the inspection image synthesizing method, and the program described above, the overlapping portion of a plurality of images photographed for visual inspection can be specified in a short time.

It is a schematic block diagram which shows the function structure of the test | inspection image synthetic | combination system which concerns on one Embodiment of this invention. It is a figure which shows the example of the scanning pattern of the imaging device which concerns on the embodiment. It is a figure which shows the 2nd example of the scanning pattern of the imaging device concerning the embodiment. It is a figure which shows the 3rd example of the scanning pattern of the imaging device concerning the embodiment. It is a figure which shows the example of the time series of the image which the imaging device which concerns on the embodiment image | photographed. It is a figure which shows the example of the movement amount of the imaging device concerning the embodiment. It is a figure showing the example of superposition of the picture concerning the embodiment. It is a figure which shows the example of the superposition of the image in case the amount of movement of the imaging device which concerns on the embodiment has a variation. It is a figure which shows the example of a setting of the template in the template matching which concerns on the same embodiment. It is a figure which shows the example of a setting of the search range in the template matching which concerns on the embodiment. It is a flowchart which shows the example of the procedure of the test | inspection using the test | inspection image synthetic | combination system which concerns on the embodiment. It is a schematic block diagram showing composition of a computer concerning at least one embodiment.

Hereinafter, embodiments of the present invention will be described, but the following embodiments do not limit the invention according to the claims. Moreover, not all combinations of features described in the embodiments are essential to the solution of the invention.
FIG. 1 is a schematic block diagram showing a functional configuration of an inspection image combining system according to an embodiment of the present invention. As shown in the figure, the inspection image combining system 1 includes an imaging device 100 and an inspection image combining device 200. The inspection image synthesizing device 200 includes a communication unit 210, a display unit 220, an operation input unit 230, a storage unit 280, and a control unit 290. The control unit 290 includes a movement amount calculation unit 291, a search range setting unit 292, and an image combining unit 293.

The inspection image synthesis system 1 provides a photographed image of the surface of the inspection object for visual inspection (VT) of the inspection object.
The imaging device 100 scans the surface of the inspection object and performs imaging at fixed time intervals.
FIG. 2 is a view showing an example of a scanning pattern of the imaging device 100. As shown in FIG. In FIG. 2, a scanning pattern of the imaging device 100 on the surface of the inspection object 900 is indicated by a line L11. Also, a flaw C11 on the surface of the inspection object 900 is shown.

In the scanning pattern of FIG. 2, the imaging device 100 moves downward in the drawing while reciprocating the surface of the inspection object 900 to the left and right of the drawing. The imaging device 100 performs imaging on each of the forward pass and the return pass.
However, the scanning pattern of the imaging device 100 is not limited to a specific one. As a scanning pattern of the imaging device 100, various scanning patterns capable of imaging the entire visual inspection target range can be used.

FIG. 3 is a diagram showing a second example of the scan pattern of the imaging device 100. As shown in FIG. In FIG. 3, the inspection object 900 has a shape in which a cylinder, a truncated cone, and a rectangular parallelepiped are combined. The imaging apparatus 100 scans the inspection object 900 while rotating in the circumferential direction of the inspection object from the upper side of the inspection object 900 as indicated by the arrow, and moves downward each time it makes a round and scans in the circumferential direction repeat.
The scanning performed by the imaging device 100 in this manner can also scan an inspection object having a circular shape, such as a cylinder and a truncated cone.

FIG. 4 is a view showing a third example of the scan pattern of the imaging device 100. As shown in FIG. In FIG. 4, a scanning pattern of the imaging device 100 on the surface of the inspection object 900 is shown by a solid line and a broken line. The image when the imaging device 100 moves along the solid line is used for visual inspection. On the other hand, the image when the imaging device 100 moves along the broken line is not used for visual inspection.
Here, it is conceivable that the operation of the imaging device 100 differs between the forward pass and the return path due to backlash or the like of the gear of the drive unit of the imaging device 100. In this case, the influence of backlash or the like can be reduced by using only the image captured in one direction by the imaging device 100.

The inspection image synthesizing device 200 generates a panoramic image in which a plurality of images obtained by photographing the surface of the inspection object by the photographing device 100 are connected. The panoramic image referred to here is an image (corresponding to an image showing a wider range of the inspection object) equivalent to one taken at a wider angle of view than the actual angle of view of the photographing device 100.
The inspection image synthesizing device 200 is configured using, for example, a computer such as a personal computer (PC) or an EWS (engineering workstation).

The communication unit 210 communicates with other devices. In particular, the communication unit 210 receives imaging data from the imaging device 100. The shooting data includes an image shot by the shooting device 100 as image data, and also includes shooting position information of the image (position information of the shooting device 100 when the image is shot). The communication unit 210 corresponds to an example of an image information acquisition unit.
The shooting position information here may be any information that indicates the amount of movement of the shooting apparatus 100 between shooting and shooting. For example, the imaging position information may be integrated movement distance information of the imaging device 100. Alternatively, the photographing position information may indicate the position of the photographing apparatus 100 at the time of photographing by three-dimensional coordinate data or two-dimensional coordinate data.

The display unit 220 has a display screen such as a liquid crystal panel or a light emitting diode (LED) panel, and displays various images. For example, the display unit 220 displays a panoramic image of the inspection object generated by the inspection image synthesizing device 200.
The operation input unit 230 is configured using an input device such as a keyboard and a mouse, for example, and receives a user operation. For example, when the display unit 220 is displaying a panoramic image of the inspection object, the operation input unit 230 instructs the user to change the magnification of the image and the user to change (move) the display range. You may be asked to

The storage unit 280 stores various data. For example, the storage unit 280 stores shooting data received by the communication unit 210 in time series.
The storage unit 280 is configured using a storage device included in the inspection image combining device 200.
The control unit 290 controls each unit of the inspection image synthesis device 200 to perform various processes. The control unit 290 is configured such that a CPU (Central Processing Unit, central processing unit) included in the inspection image combining device 200 reads out a program from the storage unit 280 and executes it.

The movement amount calculation unit 291 calculates the movement amount between the images based on the photographing position information of each image from the photographing device 100 adjacent in time series. In particular, the movement amount calculation unit 291 calculates the movement amount by the number of pixels.
For example, the movement amount calculation unit 291 calculates the distance between the photographing positions (that is, the movement amount of the photographing apparatus 100) by calculating the difference between the photographing position information of each image adjacent in time series. Then, the movement amount calculation unit 291 converts the calculated distance into the number of pixels by multiplying the calculated distance by the number of pixels per unit distance in the photographed image of the photographing device 100.

FIG. 5 is a diagram illustrating an example of a time series of images captured by the imaging device 100. The horizontal axis of FIG. 5 shows time. In FIG. 5, the shooting time of each image is indicated by a frame number. The frame referred to here is an image obtained by one photographing of the photographing device 100 (therefore, an individual image photographed by the photographing device 100).
Among the images shown in FIG. 5, a part of the flaw (a flaw C11 in FIG. 2) appears in each of the frame numbers t + ω, t + (ω + 1), and t + (ω + 2). In the scanning pattern shown in FIG. 2, the image of FIG. 5 is photographed while the photographing device 100 moves from left to right constantly in the vertical direction of the drawing. As a result, when the image is viewed in the order of frame numbers t + ω, t + (ω + 1), t + (ω + 2), the flaw image is constant in height (vertical direction in the figure) and moves from the right to the center to the left of the figure. doing.

FIG. 6 is a diagram illustrating an example of the movement amount of the imaging device 100.
In FIG. 6, among the images in FIG. 5, the image of frame number t + ω and the image of frame number t + (ω + 1) are shown aligned vertically with the left and right ends aligned. In the following description, an image of frame number t + ω is referred to as a first image, and an image of frame number t + (ω + 1) is referred to as a second image.

The point P11 in the first image and the point P12 in the second image indicate the same place on the surface of the inspection object. The first image and the second image are not shifted in the vertical direction of the figure. The distance d corresponds to the movement amount of the photographing apparatus 100 from the photographing of the first image to the photographing of the second image. The movement amount calculation unit 291 calculates the distance d from the number of pixels p in the image captured by the imaging device 100.
After shifting the second image rightward by the number of pixels p from the state of FIG. 6, the second image is moved upward to align the upper and lower ends of the first image with the upper and lower ends of the second image, A panoramic image of these two images can be obtained.

The search range setting unit 292 calculates, based on the movement amount calculated by the movement amount calculation unit 291, the range of pixels adjacent to each other in time series and overlapping between the images from the imaging device 100. The search range setting unit 292 sets the calculated range of pixels as a search range in template matching performed by the image combining unit 293.
FIG. 7 shows an example of image superposition. In the example of FIG. 7, as described with reference to FIG. 6, the first image and the second image are shifted by the number of pixels p to the left and right, aligned vertically, and superimposed.

  If the imaging apparatus 100 always moves from the left to the right in FIG. 2 by a distance D between imaging and imaging, and if the imaging apparatus 100 does not shift in the vertical direction in FIG. A panoramic image (for one horizontal line) can be obtained by shifting and superposing the number of pixels p in the direction. In this case, the range of pixels to be superimposed is uniquely determined, and it is not necessary to perform image matching.

However, in practice, it is difficult to always move the imaging device 100 at a constant speed, and the amount of movement of the imaging device 100 between imaging and imaging varies. Therefore, the search range setting unit 292 sets a search range having a width corresponding to the variation of the movement amount of the imaging device 100.
In addition, usually, when the photographing apparatus 100 moves, not only the variation of the movement amount occurs in the planned movement direction (horizontal direction in FIG. 2), but also the direction perpendicular to the planned movement direction (vertical direction in FIG. 2) Misalignment (a shift from 0 occurs). Therefore, the search range setting unit 292 sets a search range having a width corresponding to the variation of the movement amount in each of the vertical direction and the horizontal direction of the pixel of the image.

FIG. 8 is a view showing an example of superposition of images in the case where the movement amount of the imaging device 100 has variation. In FIG. 8, the position of the image (first image in the example of FIG. 7) positioned on the left side of the drawing in superposition is indicated by the position q1. In relation to the position q1, three positions q2, q2 'and q2''are shown as the positions of the image (the second image in the example of FIG. 7) positioned on the right side of the figure in the superposition.
Hereinafter, an image positioned on the left side of the drawing in superposition will be referred to as a left image, and an image positioned on the right side of the drawing in superposition will be referred to as a right image.

The pixel number p _h, in which the search range setting section 292 indicates a standard amount of movement of the imaging device 100 assumed in the number of pixels. In the example of FIG. 8, the photographing apparatus 100 between the imaging and imaging is a standard to move to the right in Figure by the number of pixels p _h.
Number of pixels Delta] p _h, the search range setting unit 292 is assumed, the maximum value of the variation of the amount of movement planned moving direction (lateral direction in the drawing). In the example of FIG. 8, the photographing apparatus 100 is assumed to move to the right in FIG. Within the range of the pixel number p _h -Δp _h until p _{h +} Δp _h between the imaging and imaging.
The number of pixels Δp _v is the maximum value of the variation of the movement amount in the direction (vertical direction in the drawing) perpendicular to the planned movement direction, which the search range setting unit 292 assumes. In the example of FIG. 8, it is assumed that the imaging apparatus 100 shifts between the imaging and the imaging within the range of the number of pixels Δp _{V in} any of the upper and lower sides of the diagram.

Position q2 is, the photographing apparatus 100 is moved to the right in FIG. Is a scheduled movement direction by the number of pixels p _h, when no shift in the vertical direction and the position of the right image. Position q2 'is photographing apparatus 100 is moved by the number of pixels p _h -Δp _h in the right direction in the figure, when displaced by the number of pixels Delta] p _v upward in the figure, the position of the right image. Position q2 '' includes a photographing apparatus 100 is moved by the number of pixels p _h -Δp _h in the right direction in the figure, when displaced by the number of pixels Delta] p _v upward in the figure, the position of the right image.
The positions q2 ′ and q2 ′ ′ indicate the positions where there is the maximum possible distortion both at the top and bottom and at the left and right of the figure. An area A11, which is an overlapping portion of the positions q1, q2 ′ and q2 ′ ′, indicates a portion where two images necessarily overlap within the assumed range.

FIG. 9 is a diagram showing an example of setting of a template in template matching. In the example of FIG. 9, the search range setting unit 292 sets an area A11 in which the left image necessarily overlaps the right image within the assumed range of the variation of the movement amount of the imaging device 100 as a template in template matching.
Assuming that the number of pixels of the left image in the horizontal direction in FIG. 9 is p _x , the maximum value of the horizontal movement amount of the imaging device 100 within the assumed range of the search range setting unit 292 p _n + Δp _n (standard movement amount + _p x -p _n + Δp _n of the maximum value) was subtracted from _{p x} of variation, the number of pixels in the horizontal direction of the template (area A11).
Further, assuming that the number of pixels of the left image in the vertical direction in FIG. 9 is p _y , twice the Δp _n which is the maximum value of the vertical movement amount of the imaging device 100 within the assumed range of the search range setting unit 292 is subtracted The calculated p _y −2Δp _n is the number of pixels in the vertical direction of the template (area A11). If the photographing apparatus 100 is shifted upward, it is subtracted twice the Delta] p _n assuming both be shifted downward.

  Template matching here is a pattern matching method for searching for the appearance position of a pattern of pixel values of a template within a search range. Specifically, pattern matching is performed according to the following procedure.

(1) A part (area A11 in the example of FIGS. 8-8) of one of the two images to be matched (left image in the examples of FIGS. 8-8) is set as a template.
(2) A part of the other of the two images to be matched (the right image in the example of FIGS. 8 to 8) is set as the search range.
(3) Within the search range, search for a partial image whose pixel value most closely matches the template. As the evaluation score of the degree of coincidence of pixel values, for example, a score capable of evaluating the magnitude of the difference between pixel values, such as the root mean square of differences between pixel values at each pixel, is calculated. Search for a partial image that is evaluated to have the smallest difference magnitude.

  FIG. 10 is a diagram showing an example of setting a search range in template matching. The search range setting unit 292 sets the search range to a possible range (which is an image or a partial image obtained by photographing the same portion of the surface of the inspection object) overlapping the template (the area A11 in FIG. 9) in the right image. Set In the example of FIG. 10, the search range setting unit 292 sets the area A12 as a search range.

The minimum value of the lateral movement amount of the photographing apparatus 100 within the assumed range of the search range setting unit 292 p _n −Δp _n (standard movement amount−variation maximum value) The pixel number of the search range (area A12) in the horizontal direction is obtained by subtracting p _x- (p _n -Δp _n ) from the pixel number p _{x of x} .
In the vertical direction in FIG. 10, the entire right image (number of pixels p _y ) is set as the search range.
As area A11 is shown in area A12 in FIG. 10, in template matching, a partial image of the same shape, direction, and size as the template is set in the search range, and the evaluation score of the degree of coincidence of pixel values is calculated. Do. Among all the partial images that can be set, the position of the partial image with the smallest evaluation score is detected.
In addition, the method of performing template matching here is not limited to a specific method.

In order to set the template illustrated in FIG. 9 and the search range illustrated in FIG. 10, the standard amount of movement of the photographing device 100 (number of pixels p _{h in} the example of FIGS. 8 to 8) and movement of the photographing device 100 maximum value of the variation amount (in the example of FIG. 8 to 8 Delta] p _h and Delta] p v number of _pixels) must be set.
Among them, the standard movement amount of the photographing apparatus 100 is calculated by multiplying the movement speed set in the photographing apparatus 100 by the photographing cycle (time interval between photographing and photographing) and converting it into the number of pixels. It can.

On the other hand, it is conceivable to obtain the maximum value of the variation of the movement amount of the imaging device 100 based on the result data of the operation of the imaging device 100 in any way.
For example, the search range setting unit 292 or a person calculates the movement amount of the imaging device 100 for each photographing from the time-series images photographed by the photographing device 100 in the past, and the calculated movement amount shows the maximum value of variation of movement amounts. May be set. As a time-series image captured by the imaging device 100 in the past, it is conceivable to use an image captured at the time of inspection in the past, or an image captured at the time of testing or adjustment in a factory or the like.
Alternatively, if the movement amount between photographing and photographing is recorded in the operation test of the photographing device 100, the maximum value of the variation of the movement amount of the photographing device 100 can be set from the operation test data.

If several movement data of the photographing device 100 are obtained, the search range setting unit 292 or a person detects the maximum value or the minimum value of the movement amount, and the obtained maximum value or the minimum value The difference between this and the standard movement amount may be used as the maximum value of the variation of the movement amount of the imaging device 100. Alternatively, the search range setting unit 292 or a person may use a value obtained by adding a margin (a margin) to the size of the variation obtained from the actual data of the moving amount as the maximum value of the variation of the moving amount of the imaging device 100. You may
Alternatively, the search range setting unit 292 or a person determines the frequency distribution of the movement amount of the imaging device 100, calculates 3σ, and uses it as the maximum value of the movement amount variation, etc. The value may be determined by a probabilistic method.

In addition, when the search range setting unit 292 or a person calculates the maximum value of the variation of the movement amount of the imaging device 100, movement is performed based on the scanning locus information set to cause the imaging device 100 to scan the surface of the inspection object The maximum value of the variation of the amount may be set.
For example, in the scanning pattern of FIG. 2, when the imaging device 100 moves from left to right in the drawing and when it moves from right to left, the moving direction of the image shown in the image is reversed. For example, in each of the images of frame numbers t + ω, t + (ω + 1), and t + (ω + 2) in FIG. 5, the image of the flaw being captured moves from the right to the left of the image. When the photographing device 100 moves from the right to the left in the figure, the movement of the image of this flaw is reversed. Further, the image when the imaging device 100 moves in the vertical direction in FIG. 2 is not used for visual inspection. When the moving amount of the imaging device 100 is calculated based on the time series of the image captured in the past by the imaging device 100, the image and the scanning locus information are matched to grasp the moving direction in the image, and the moving amount of the imaging device 100 You may ask for

Further, in the case of the scanning pattern of FIG. 3, the length of one portion of the truncated cone is longer than that of the cylindrical portion of the inspection object 900, and one portion is closer to the lower side of the portion of the truncated cone. The length of the As described above, when the moving amount of the imaging device 100 differs depending on the portion of the inspection object 900, it is conceivable to associate the image with the position of the imaging device 100 with reference to the scanning locus information.
In the case of the scanning pattern of FIG. 4, the image when the imaging device 100 moves along the broken line is not used for visual inspection. Also in this case, it is conceivable to associate the image with the position of the imaging device 100 with reference to the scanning locus information.

The image combining unit 293 performs template matching in the search range set by the search range setting unit 292, and specifies the range of overlapping pixels between the images adjacent in time series. Then, the image combining unit 293 combines a panoramic image in which the specified range of pixels is superimposed.
Here, as a method of image matching, there is also image matching by feature point extraction other than template matching. However, an image obtained by photographing for visual inspection is generally poor in tone difference, and it is difficult to extract an appropriate feature point. From this, it is expected that in the image matching for the image taken for visual inspection, the accuracy of the pattern matching is better if the template matching is performed than the image matching by the feature point extraction.

In template matching, the larger the difference between the size of the search range and the size of the template, the larger the number of partial images that can be taken within the search range, and it takes time for processing.
On the other hand, in the inspection image synthesizing device 200, the search range setting unit 292 sets a template and sets a portion that may overlap with the template as the search range, thereby reducing the number of partial images that can be taken within the search range. Processing time is relatively short.

Next, an inspection procedure using the inspection image synthesis system 1 will be described.
FIG. 11 is a flow chart showing an example of an inspection procedure using the inspection image synthesis system 1.
In the inspection procedure of FIG. 11, the imaging device 100 performs imaging of the surface of the inspection object (step S101).
Next, the inspection image synthesizing device 200 performs panoramic synthesis of the image of the surface of the inspection object by the photographing device 100, and generates an entire image of the visual inspection object range of the surface of the inspection object (step S102).

The inspection worker performs a visual inspection using the entire image generated by the inspection image synthesizing device 200 (step S103).
Then, the inspection worker determines whether or not an instruction has been given by visual inspection (step S104). The instruction here is an abnormal point such as a shape change or a flaw.
If it is determined that there is no instruction (step S104: NO), the inspection worker reports the inspection result (step S121).
After step S121, the process of FIG. 11 ends.

On the other hand, when it is determined that there is an instruction (step S104: instruction is present), the inspection worker enlarges a portion corresponding to the instruction in the entire image generated by the inspection image combining device 200, and performs visual inspection in more detail. Perform (step S111).
Then, the inspection worker reports the inspection result (step S112), and determines and implements a countermeasure for the instruction such as inspection by another method, evaluation of soundness or repair work (step S113).
After step S113, the process of FIG. 11 ends.

The inspection procedure of FIG. 11 is that the inspection image synthesizing device 200 performs panoramic combination of the images to generate the entire image of the visual inspection target range, so that the entire image can be obtained without causing a large burden on the inspection worker. It is established on the premise.
In the inspection procedure of FIG. 11, the inspection worker can easily grasp the shape of the inspection object by obtaining the entire image of the visual inspection target range. In addition, when the inspection object has an instruction, the position of the instruction on the inspection object can be easily grasped. In addition, even when the instruction is relatively large and does not fit within the angle of view of the imaging device 100, the inspection worker can easily grasp the entire shape of the instruction by referring to the entire image of the visual inspection target range.

  Further, since the inspection image combining device 200 generates the entire image of the visual inspection target range, the angle of view of each image captured by the imaging device 100 may be relatively small. In this regard, the imaging device 100 can perform imaging with high resolution. The imaging device 100 performs imaging with high resolution, and the inspection image combining device 200 performs panoramic synthesis of the image to generate the entire image of the visual inspection target range, thereby obtaining the entire image of high resolution. If an instruction is found in the visual inspection, the inspection worker can magnify the corresponding portion of the entire image to perform a more detailed visual inspection.

In a general inspection that does not use the inspection image synthesis system 1, a visual inspection is performed using the image of the inspection object, and if an instruction is found, the magnification of imaging is increased and the relevant portion is rephotographed and visual inspection is performed again. It is carried out. On the other hand, in the inspection using the inspection image synthesis system 1, it is not necessary to rephotograph the inspection object as described above. In this respect, by using the inspection image synthesis system 1, the inspection can be efficiently performed.
Further, in a general inspection that does not use the inspection image synthesis system 1, the inspection worker refers to the image captured by the imaging device in real time to confirm the presence or absence of an instruction. On the other hand, in the inspection using the inspection image synthesis system 1, the inspection worker may check the presence or absence of the instruction with reference to the entire image of the visual inspection target range, and there is no time constraint due to the real time property.
Moreover, in the general inspection which does not use the inspection image synthesizing system 1, the image to which the inspection worker refers changes in time series. On the other hand, in the inspection using the inspection image synthesis system 1, since the inspector refers to the entire image of the visual inspection target range, the images do not change in time series. In this respect, in the inspection using the inspection image synthesis system 1, the inspection accuracy can be improved. In particular, in the inspection using the inspection image synthesis system 1, the visual inspection can be performed not with a moving image but with a still image. In addition, since the image does not change in time series, the observation time (time for performing a visual inspection) is not limited, and as a result, the possibility of missing an abnormality or the like can be reduced.

  As described above, the communication unit 210 acquires the photographed image and the photographing position information from the photographing device 100 which scans the surface of the inspection object and photographs at a constant time interval. The movement amount calculation unit 291 calculates the movement amount between the images from the photographing position information of each image adjacent in time series. The search range setting unit 292 calculates the range of overlapping pixels between the images adjacent in time series based on the movement amount of the imaging device 100, and sets the range as a search range in template matching. The image combining unit 293 performs template matching in the search range to specify the range of overlapping pixels between images, and combines a panoramic image in which the specified range of pixels is superimposed.

The search range setting unit 292 can narrow the search range by setting the search range based on the movement amount of the imaging device 100. In this respect, according to the inspection image synthesizing device 200, the overlapping portion of a plurality of images captured for visual inspection can be specified in a shorter time.
In addition, the image synthesizing unit 293 synthesizes the panoramic image to generate the entire image of the visual inspection target range, whereby the inspection worker grasps the entire positional relationship and efficiently performs visual inspection with high accuracy. It can be carried out.

Further, the search range setting unit 292 sets a search range having a width corresponding to the variation of the movement amount of the imaging device 100.
Thus, the search range setting unit 292 can narrow the search range to such an extent that template matching can be performed even if the movement amount of the imaging device 100 varies. From this point of view, according to the inspection image synthesizing device 200, it is possible to achieve both the securing of the accuracy of the template matching and the reduction of the processing time.

Further, the search range setting unit 292 sets a search range having a width corresponding to the variation of the movement amount in each of the vertical direction and the horizontal direction of the pixel of the image.
Thereby, in the inspection image synthesizing device 200, in addition to the variation in the movement amount of the imaging device 100 in the movement direction of the imaging device 100 planned, the deviation in the vertical direction to the movement direction of the imaging device 100 scheduled Then, the search range can be narrowed.

Further, the search range setting unit 292 calculates the width of the search range according to the variation of the movement amount of the imaging device 100 from the time-series images captured by the imaging device 100 in the past.
Thus, the search range setting unit 292 can set the search range based on the results of the operation of the imaging device 100. In this regard, the search range setting unit 292 can set the search range with high accuracy.

Further, the search range setting unit 292 calculates the width of the search range according to the variation of the movement amount of the imaging device 100 from the operation test data of the imaging device 100.
Thus, the search range setting unit 292 can set the search range based on the results of the operation of the imaging device 100. In this regard, the search range setting unit 292 can set the search range with high accuracy.

In addition, the search range setting unit 292 sets the difference between the maximum value or the minimum value of the movement amounts in the plurality of actual data of movement amounts and the standard movement amount of the imaging device 100 as the maximum movement amount variation. The width of the search range according to the maximum value of the variation is calculated.
Thus, the search range setting unit 292 can set the width of the search range with high accuracy in accordance with the maximum value or the minimum value of the variation of the movement amount.

Further, the search range setting unit 292 sets a value obtained by adding a margin to the size of the variation of the moving amount obtained from the actual data of the moving amount as the maximum value of the variation of the moving amount as the maximum value of the variation of the moving amount. Calculate the width of the search range according to.
By calculating the search range setting unit 292 by adding a margin to the maximum value of the variation of the movement amount, the possibility of underestimating the magnitude of the variation of the movement amount can be reduced.

Further, the search range setting unit 292 calculates the width of the search range according to the maximum value of the variation of the movement amount, with 3σ in the frequency distribution of the movement amount as the maximum value of the variation of the movement amount.
The width of the search range can be set with high accuracy in that the search range setting unit 292 calculates the magnitude of the variation of the movement amount based on the probabilistic method.

Further, the search range setting unit 292 calculates the width of the search range according to the variation of the movement amount of the imaging device 100 based on the scanning locus information set to cause the imaging device 100 to scan the surface of the inspection object. Do.
As a result, the search range setting unit 292 can obtain the variation of the movement amount of the imaging device 100 corresponding to various operation patterns in the scanning performed by the imaging device 100, and based on this, the search range can be set. it can. The search range setting unit 292 can set the search range with high accuracy, in that a large amount of data for obtaining the variation of the movement amount of the imaging device 100 can be obtained.

FIG. 12 is a schematic block diagram showing the configuration of a computer according to at least one embodiment.
The computer 10 includes a CPU 11, a main storage 12, an auxiliary storage 13, and an interface 14.
The inspection image synthesizing device 200 described above is mounted on the computer 10. The operation of each unit of the control unit 290 described above is stored in the auxiliary storage device 13 in the form of a program. The CPU 11 reads a program from the auxiliary storage device 13 and develops it in the main storage device 12, and executes the above processing according to the program. Further, the CPU 11 secures a storage area corresponding to the storage unit 280 described above in the main storage device according to the program.

Note that a program for realizing all or part of functions of the control unit 290 is recorded in a computer readable recording medium, and the computer system reads the program recorded in the recording medium and executes each part. Processing may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices.
The "computer system" also includes a homepage providing environment (or display environment) if the WWW system is used.
The “computer-readable recording medium” is a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, a hard disk built in a computer system, or a storage device such as a solid state drive (SSD). It means that. The program may be for realizing a part of the functions described above, or may be realized in combination with the program already recorded in the computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and design changes and the like within the scope of the present invention are also included.

DESCRIPTION OF SYMBOLS 1 inspection image synthesizing system 100 photographing device 200 inspection image synthesizing device 210 communication unit 220 display unit 230 operation input unit 280 storage unit 290 control unit 291 movement amount calculating unit 292 search range setting unit 293 image synthesizing unit

Claims (12)

An image information acquisition unit that acquires a photographed image and photographing position information from a photographing device that scans the surface of the inspection object and photographs at constant time intervals;
A movement amount calculation unit that calculates the movement amount between the images from the photographing position information for each of the images adjacent in time series;
A search range setting unit that calculates a range of overlapping pixels between the images adjacent in time series based on the movement amount, and sets the range as a search range in template matching;
An image combining unit that performs template matching in the search range to specify a range of overlapping pixels between the images, and combines a panoramic image in which the specified range of pixels are superimposed;
An inspection image synthesizing apparatus comprising: The search range setting unit sets the search range having a width corresponding to the variation of the movement amount.
The inspection image synthesizing device according to claim 1. The search range setting unit sets the search range having a width corresponding to the variation of the movement amount in each of the vertical direction and the horizontal direction of the pixel of the image.
The inspection image synthesizing device according to claim 2. The search range setting unit calculates a width according to the variation of the movement amount from time-series images captured by the imaging device in the past.
The inspection image synthesizing device according to claim 2 or 3. The search range setting unit calculates a width according to the variation of the movement amount from operation test data of the imaging device.
The inspection image synthesizing device according to any one of claims 2 to 4. The search range setting unit sets the difference between the maximum value or the minimum value of the moving amount in the plurality of actual data of the moving amount and the standard moving amount of the imaging device as the maximum variation of the moving amount. Calculate the width according to the maximum value of the
The inspection image synthesizing device according to claim 4 or 5. The search range setting unit sets a value obtained by adding a margin to the magnitude of the variation of the moving amount obtained from the actual data of the moving amount as the maximum value of the variation of the moving amount, the maximum of the variation of the moving amount. Calculate the width of search range according to the value,
The inspection image synthesizing device according to claim 4 or 5. The search range setting unit calculates the width of the search range according to the maximum value of the variation of the movement amount, with 3σ in the frequency distribution of the movement amount as the maximum value of the variation of the movement amount.
The inspection image synthesizing device according to claim 4 or 5. The search range setting unit calculates a width according to the variation of the movement amount based on scanning locus information set to cause the imaging device to scan the surface of the inspection object.
The inspection image synthesizing device according to any one of claims 2 to 8. An imaging device for scanning the surface of the inspection object and performing imaging at fixed time intervals;
An image information acquisition unit that acquires an image captured by the imaging device and imaging position information;
A movement amount calculation unit that calculates the movement amount between the images from the photographing position information for each of the images adjacent in time series;
A search range setting unit that calculates a range of overlapping pixels between the images adjacent in time series based on the movement amount, and sets the range as a search range in template matching;
An image combining unit that performs template matching in the search range to specify a range of overlapping pixels between the images, and combines a panoramic image in which the specified range of pixels are superimposed;
An examination image combining system comprising: Acquiring a photographed image and photographing position information from a photographing device which scans the surface of the inspection object and photographs at constant time intervals;
Calculating the amount of movement between the images from the photographing position information of each of the images adjacent in time series;
Calculating a range of overlapping pixels between the images adjacent in time series based on the movement amount, and setting the range as a search range in template matching;
Template matching is performed in the search range to specify a range of overlapping pixels between the images, and combining a panoramic image in which the specified range of pixels is superimposed;
An inspection image combining method including: On the computer
Acquiring a photographed image and photographing position information from a photographing device which scans the surface of the inspection object and photographs at constant time intervals;
Calculating the amount of movement between the images from the photographing position information of each of the images adjacent in time series;
Calculating a range of overlapping pixels between the images adjacent in time series based on the movement amount, and setting the range as a search range in template matching;
Template matching is performed in the search range to specify a range of overlapping pixels between the images, and combining a panoramic image in which the specified range of pixels is superimposed;
A program to run a program.

Images