JP5745128B2 - Image processing device - Google Patents

Description

The present invention relates to an image processing apparatus and program for processing an image obtained by imaging blades periodically arranged in a jet engine.

Conventionally, in order to inspect a blade in a jet engine, the blade is observed using an observation jig such as an endoscope. For example, Patent Document 1 describes a method of detecting a blade defect by continuously imaging a blade and comparing two consecutive images. Patent Document 2 describes a method of detecting a blade defect based on a feature amount of a known defect pattern.

US Patent Application Publication No. 2004/183900 JP 2007-163723 A

Patent Document 1 does not describe the position of the blade in the two consecutive images (how the blade looks). In order to extract a defect by a simple method of obtaining a difference between images, it is desired that the positions of the blades in the image are substantially the same. However, in order to make the positions of the blades in the image substantially the same, it is necessary to match the rotation of the blades with the timing of imaging, which makes the control complicated. On the other hand, when the positions of the blades in the image are different, a method of extracting the blade pattern from each of the two images by pattern matching using a known blade pattern and comparing the extracted blade patterns can be considered. However, this method requires complicated image processing.

Further, in the method described in Patent Document 2, since a feature amount of a known defect pattern is used, a defect that does not match the defect pattern may be missed.

The present invention has been made in view of the above-described problems, and an object thereof is to provide an image processing apparatus and a program capable of detecting a blade defect regardless of the type of defect by a simple method. .

  The present invention has been made to solve the above-described problems. For each image of a plurality of images obtained by capturing blades periodically arranged in a jet engine and rotating the blades, a template image and An image comparison unit that compares the images by looking at the correlation, an image selection unit that selects a part of the images based on the result of the image comparison by the image comparison unit, from a plurality of images obtained by imaging the blade; A difference extraction unit that extracts a difference from the template image for each image with respect to a part of the images selected by the image selection unit.

  The image comparison unit calculates a correlation value with the template image for each of a plurality of images obtained by imaging the blade, and the image selection unit is based on the correlation value calculated by the image comparison unit. Preferably, the partial image is selected from a plurality of images obtained by imaging the blade.

Also have the template image further template image registration unit for registering the template image registration unit, from among the plurality of images, it is preferable to register an image in which there is an instruction of the user as the template image .

  Further, it is preferable that a display signal generation unit that outputs a display signal to the display unit is further included, and the display signal generation unit generates a display signal for displaying the template image on the display unit.

  Further, the difference extraction unit converts the partial image selected by the image selection unit and the template image into grayscale images, respectively, and the grayscale image of the partial image and the template image It is preferable to extract a difference from the grayscale image.

According to the present invention, the special control for adjusting the rotation of the blade and the timing of the imaging by selecting a part of the images based on the result of the image comparison with the template image from the images of the imaging of the blade. The image of the blade can be acquired by a simple method. By extracting the difference between the image acquired in this way and the template image, it is possible to detect a blade defect regardless of the type of defect.

1 is a block diagram showing a configuration of a blade inspection system according to a first embodiment of the present invention. It is a block diagram showing composition of an endoscope apparatus with which a blade inspection system by a 1st embodiment of the present invention is provided. It is a block diagram which shows the structure of the blade test | inspection system (modification) by the 1st Embodiment of this invention. It is a block diagram which shows the structure of the blade test | inspection system (modification) by the 1st Embodiment of this invention. It is a block diagram which shows the structure of PC with which the braid | blade test | inspection system (modification) by the 1st Embodiment of this invention is provided. It is a reference figure which shows the screen of the blade recording software in the 1st Embodiment of this invention. It is a reference figure which shows the screen of the blade recording software in the 1st Embodiment of this invention. It is a reference diagram showing a directory structure in the memory card in the first embodiment of the present invention. It is a reference figure which shows the preservation | save folder list | wrist in the 1st Embodiment of this invention. It is a reference diagram showing an image file list in the first embodiment of the present invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 1st Embodiment of this invention. It is a graph which shows the time-dependent change of the correlation value in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 1st Embodiment of this invention. It is a reference figure which shows the screen of the blade recording software in the 2nd Embodiment of this invention. It is a reference figure which shows the screen of the blade recording software in the 2nd Embodiment of this invention. It is a reference figure which shows the screen of the blade recording software in the 2nd Embodiment of this invention. It is a reference figure which shows the screen of the blade recording software in the 2nd Embodiment of this invention. It is a reference figure which shows the screen of the blade recording software in the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 2nd Embodiment of this invention. It is a reference figure for demonstrating the defect extraction process in the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 2nd Embodiment of this invention. It is a flowchart which shows the procedure of the operation | movement by the blade recording software in the 2nd Embodiment of this invention. It is a reference figure for demonstrating the defect designation | designated process in the 2nd Embodiment of this invention. It is a reference figure which shows the screen of the blade recording software in the 2nd Embodiment of this invention. It is a reference figure which shows the screen of the blade recording software in the 2nd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
First, a first embodiment of the present invention will be described. FIG. 1 shows a configuration of a blade inspection system according to the present embodiment. In the jet engine 1, a plurality of turbine blades 10 (or compressor blades), which are inspection objects, are periodically arranged at predetermined intervals. Further, a turning tool 2 that rotates the turbine blade 10 in the rotation direction A at a predetermined speed is connected to the jet engine 1. In the present embodiment, the turbine blade 10 is always rotated while an image of the turbine blade 10 is captured.

In the present embodiment, an endoscope apparatus 3 (corresponding to the image processing apparatus of the present invention) is used to acquire an image of the turbine blade 10. An endoscope insertion portion 20 of the endoscope device 3 is inserted inside the jet engine 1, and an image of the rotating turbine blade 10 is captured by the endoscope insertion portion 20. Further, the endoscope apparatus 3 stores blade recording software for recording an image obtained by imaging the turbine blade 10 at a desired angle.

FIG. 2 shows the configuration of the endoscope apparatus 3. The endoscope apparatus 3 includes an endoscope insertion unit 20, an endoscope apparatus main body 21, a monitor 22, and a remote controller (remote controller) 23. An imaging optical system 30a and an imaging element 30b are built in the distal end of the endoscope insertion unit 20. The endoscope apparatus body 21 includes an image signal processing unit (CCU) 31, a light source 32, a bending control unit 33, and a control computer 34.

In the endoscope insertion unit 20, the imaging optical system 30a collects light from the subject and forms a subject image on the imaging surface of the imaging element 30b. The image sensor 30b photoelectrically converts the subject image to generate an image signal. The imaging signal output from the imaging element 30 b is input to the image signal processing device 31.

In the endoscope apparatus body 21, the image signal processing device 31 converts the image signal from the image sensor 30 b into a video signal such as an NTSC signal and supplies it to the control computer 34, and further, as an analog video output if necessary. Output to the outside.

The light source 32 is connected to the distal end of the endoscope insertion portion 20 through an optical fiber or the like, and can irradiate light to the outside. The bending control unit 33 is connected to the distal end of the endoscope insertion unit 20 and can bend the distal end up and down and left and right. The light source 32 and the bending control unit 33 are controlled by the control computer 34.

The control computer 34 includes a RAM 34a, a ROM 34b, a CPU 34c, a network I / F 34d, an RS232C I / F 34e, and a card I / F 34f as external interfaces. The RAM 34a is used for temporarily storing data such as image information necessary for software operation. A series of software for controlling the endoscope apparatus 3 is stored in the ROM 34b, and blade recording software described later is also stored in the ROM 34b. The CPU 34c executes various control operations and the like using the data stored in the RAM 34a according to the software instruction code stored in the ROM 34b.

The network I / F 34d is an interface for connecting to an external PC through a LAN cable, and can develop video information output from the image signal processing device 31 to the external PC. The RS232C I / F 34e is an interface for connecting to the remote controller 23, and various operations of the endoscope apparatus 3 can be controlled by the user operating the remote controller 23. The card I / F 34f can freely attach and detach various memory cards 50 that are recording media. When the memory card 50 is attached, data such as image information stored in the memory card 50 can be taken in or recorded in the memory card 50 under the control of the CPU 34c.

As a modification of the configuration of the blade inspection system according to the present embodiment, the configuration shown in FIG. 3 may be used. In this modification, the video terminal cable 4 and the video capture card 5 are connected to the endoscope apparatus 3, whereby the video captured by the endoscope apparatus 3 is displayed on the PC 6 (corresponding to the image processing apparatus of the present invention). Can also be incorporated into The PC 6 is illustrated as a notebook PC in FIG. 3, but may be a desktop PC or the like. The PC 6 stores blade recording software for recording an image of the turbine blade 10 taken at a desired angle.

Further, in FIG. 3, the video terminal cable 4 and the video capture card 5 are used for capturing video to the PC 6, but a LAN cable 7 may be used as shown in FIG. The endoscope apparatus 3 includes a network I / F 34d that can develop the captured video on the LAN network. Then, the video can be captured by the PC 6 through the LAN cable 7.

FIG. 5 shows the configuration of the PC 6. The PC 6 includes a PC main body 24 and a monitor 25. A control computer 35 is built in the PC main body 24. The control computer 35 includes a RAM 35a, an HDD (hard disk drive) 35b, a CPU 35c, and a network I / F 35d and a USB I / F 35e as external interfaces. The control computer 35 is connected to the monitor 25, and video information, a software screen, and the like are displayed on the monitor 25.

The RAM 35a is used for temporarily storing data such as image information necessary for software operation. A series of software is stored in the HDD 35b to control the endoscope apparatus, and blade recording software is also stored in the HDD 35b. In the present embodiment, the storage folder for storing the image of the turbine blade 10 is set in the HDD 35b. The CPU 35c executes various control operations and the like using the data stored in the RAM 35a in accordance with the software instruction code stored in the HDD 35b.

The network I / F 35d is an interface for connecting the endoscope apparatus 3 and the PC 6 by the LAN cable 7, and can input the video information output from the endoscope apparatus 3 to the PC 6 into the PC 6. The USB I / F 35 e is an interface for connecting the endoscope apparatus 3 and the PC 6 by the video capture card 5, and can input video information output from the endoscope apparatus 3 as analog video to the PC 6.

The blade inspection system shown in FIGS. 3 and 4 can obtain the same effects as the blade inspection system shown in FIG. In particular, the blade inspection system shown in FIGS. 3 and 4 is effective when the performance of the endoscope apparatus is inferior to that of the PC and the operation speed of the endoscope apparatus is not sufficient.

Next, the screen of the blade recording software will be described. FIG. 6 shows a main window of the blade recording software. A main window 600 shown in FIG. 6 is displayed when the user starts the blade recording software.

The display of the main window 600 is performed according to control by the CPU 34c. The CPU 34 c generates a graphic image signal (display signal) for displaying the main window 600 and outputs it to the monitor 22. When the video captured by the endoscope apparatus 3 (hereinafter referred to as “endoscopic video”) is displayed in a superimposed manner on the main window 600, the CPU 34c graphically displays the image data captured from the image signal processing apparatus 31. A process of superimposing on the image signal is performed, and the processed signal (display signal) is output to the monitor 22.

When updating the GUI display state on the main window 600, the CPU 34c generates a graphic image signal corresponding to the updated main window 600 and performs the same processing as described above. Processing related to display of windows other than the main window 600 is the same as described above. Hereinafter, the process in which the CPU 34c generates a graphic image signal for displaying (including updating) the main window 600 and the like is referred to as the process for displaying the main window 600 and the like.

The user can browse the endoscope video and save the image file by operating the main window 600 via the remote controller 23 using a GUI (graphical user interface) function. Hereinafter, functions of various GUIs will be described.

In the upper part of the main window 600, a [Preview image] box 601, a [Template image] box 602, and a [Record image] box 603 are arranged.

[Preview image] box 601 is a box for displaying an endoscopic video. When the turbine tool 10 is rotated by the turning tool 2, when an “preview start” button 610, which will be described later, is pressed, an endoscopic image (image of the rotating turbine blade 10) is displayed in real time. The Thus, the [Preview Image] box 601 allows the user to view the endoscopic video. Hereinafter, displaying an endoscopic image in the [preview image] box 601 is referred to as a preview.

[Template image] box 602 is a box for displaying a template image. When a “template registration” button 612, which will be described later, is pressed, an image of one frame captured at that timing among images of each frame constituting the endoscope video is displayed in a template image box 602. Displayed as an image. The template image is an image serving as a reference when displaying a record image to be described later.

[Record image] box 603 is a box for displaying a record image to be described later. After a “record start” button 613 to be described later is pressed, images having high correlation with the template image (hereinafter referred to as record images) are sequentially displayed among the images of the frames constituting the endoscope video. The record images displayed in the [Record Image] box 603 are sequentially stored in the storage folder in the memory card 50 as an image file.

The image file saved here is hereinafter referred to as a record image file. Further, storing the record image file sequentially in the storage folder in the memory card 50 is hereinafter referred to as a record. Details of the storage folder will be described later.

[Preview] button 610 is a button for starting display of the endoscope video in [Preview image] box 601. The “Preview Stop” button 611 is a button for stopping display of the endoscope video in the “Preview Image” box 601.

[Register template] button 612 is a button for registering a desired image as a template image. When the “template registration” button 612 is pressed, an image of one frame captured at that timing among images of each frame constituting the endoscopic video is displayed as a template image in the “template image” box 602. Is displayed. Further, the image for one frame is recorded in the RAM 34a as a template image.

[Record start] button 613 is a button for starting a record. When the [record start] button 613 is pressed, a value in a “record number” box 620 described later is reset to zero. Then, the comparison between the endoscope video and the template image is performed every frame, and among the images of each frame constituting the endoscope video, a record image for one frame having a high correlation with the template image is displayed as [record image. ] Boxes 603 are sequentially displayed. Further, the displayed record images are sequentially stored in a storage folder in the memory card 50 as an image file.

More specifically, when the position and angle of the turbine blade 10 in the endoscopic image are the same as the position and angle of the turbine blade 10 in the template image (in a simplified manner, in other words, in the endoscopic image The image of one frame (when the appearance of the turbine blade 10 in the template image and the turbine blade 10 in the template image are the same) is displayed and stored.

The “record stop” button 614 is a button for stopping the record. The “browse image” button 615 is a button for browsing the image file stored in the storage folder in the memory card 50. When the [browse image] button 615 is pressed, an [browse image] window described later is displayed. While the [browse image] window is displayed, the main window 600 is in a state in which the operation by the user is invalid.

[Record number] box 620 is a box for displaying the current number of stored record image files (hereinafter referred to as record number). However, image files of template images are not counted. Further, as described above, when the [record start] button 613 is pressed, the value in the [record number] box 620 is reset to zero.

The [maximum record] number box 621 is a box for displaying the maximum number of record image files (hereinafter referred to as the maximum number of records). If the number of records reaches the same value as the maximum number of records, the record is automatically terminated. In the [maximum record] number box 621, an arbitrary maximum number of records can be input. For example, by inputting the number of blades for one turn of the turbine blade 10 in the [maximum record] number box 621, the necessary number of image files of the turbine blade 10 can be stored.

[Exit] button 630 is a button for ending the blade recording software. When the [END] button 630 is pressed, the main window 600 is not displayed and the operation of the blade recording software is ended.

FIG. 7 shows an [image browsing] window of the blade recording software. The [browse image] window 700 shown in FIG. 7 is displayed when the [browse image] button 615 of the main window 600 is pressed, as described above.

The user can browse the record image file by operating the [browse image] window 700 via the remote controller 23 using the GUI function. Hereinafter, functions of various GUIs will be described.

A “browsing image” box 701 is a box for displaying a record image file. When a “<< previous” button 710 or a “next >>” button 711 (to be described later) is pressed or the selection of the “date and time selection” box 724 is changed, the record image file displayed in the “browsing image” box 701 Switches. A “browsing image” box 701 allows the user to browse the record image file. Hereinafter, the record image displayed in the [browsing image] box 701 is referred to as a browsing image, and the image file is referred to as a browsing image file.

[<< Previous] button 710 is a button for switching browsing images. When the “<< previous” button 710 is pressed, the image file is one smaller than the image file No (image file number) of the image file displayed in the “browsing image” box 701 in the image file list described later. An image file with No is displayed. Along with this, the image file name displayed in the [image file name] box 720 described later is also switched.

[Next >>] button 711 is also a button for switching the browse image. When the [Next >>] button 711 is pressed, an image file having an image file number one larger than the image file number of the image file displayed in the [browsing image] box 701 in the image file list to be described later. Is displayed. Along with this, the image file name displayed in the [image file name] box 720 described later is also switched.

[Image file name] box 720 is a box for displaying the file name of the browse image file. When the [<< Previous] button 710 or the [Next >>] button 711 is pressed or the selection in the [Date / time selection] box 724 is changed, the display of the image file name in the [Image file name] box 720 is switched. .

[Image file number] box 721 is a box for displaying the number of image files in an image file list to be described later. When the selection in the [Select date / time] box 724 is changed, the display of the number of image files in the [Number of image files] box 721 is switched.

[Save Date / Time] box 722 is a box for displaying the save date / time of the browse image file. When the [<< Previous] button 710 or the [Next >>] button 711 is pressed or the selection of the [Date / time selection] box 724 is changed, the storage date / time of the image file in the [Save date / time] box 722 is displayed. Switch.

[Date / time selection] box 724 is a box for switching browse images. The record start date and time of the save folder list, which will be described later, is displayed in a list format in a [date and time selection] box 724. When the selection of the record start date / time in the [Date / Time Selection] box 724 is changed, the record image file stored in the storage folder having the selected record start date / time is displayed in the [Browse Image] box 701. Accordingly, the display of the image file name in the [Image File Name] box 720 and the number of image files in the [Image File Number] box 721 are also switched.

[Close] button 730 is a button for ending the browsing of the image. When the [Close] button 730 is pressed, the [View Image] window 701 is hidden and the main window 600 is returned to the operated state.

Next, the directory structure in the memory card 50 will be described with reference to FIGS. As shown in FIG. 8, the directory immediately below the memory card 50 is composed of a plurality of storage folders 800. A storage folder 800 is a folder in which record image files are stored. The record start date and time becomes the folder name of the storage folder 800. For example, if the record start date is “2007/12/26 21:32:21”, the folder name is “20071226-213221”.

A directory immediately under each storage folder is composed of a plurality of record image files 810. The record image file names are “001.jpg”, “002.jpg”, “003.jpg”,... In the order in which the record image files are stored. However, the file name of the template image file is “Temp.jpg”.

In addition, a storage folder list and an image file list are created in an image browsing process described later.

The save folder list is a list of save folders. As shown in FIG. 9, the save folder list is composed of a save folder number (save folder number), a record start date and time, and a folder name. Numbers 1, 2, 3,... Are assigned to the storage folder No. in the order in which the storage folders are created.

The image file list is a list of record image files stored in each storage folder. As shown in FIG. 10, the image file list includes an image file number, a file save date and time, and a file name. Numbers 1, 2, 3,... Are assigned to the image file No in the order in which the files are stored. However, the last image file No. is assigned only to the template image.

Next, the operation flow of the blade recording software will be described with reference to FIG. In step SA, the user activates the blade recording software. At this time, based on the activation instruction of the blade recording software input to the remote controller 23, the CPU 34c reads the blade recording software stored in the ROM 34b into the RAM 34a and starts an operation according to the blade recording software. In step SB, the CPU 34c performs a process for displaying the main window.

In step SC, the CPU 34c performs an initialization process. The initialization process is a process of setting initial states of various GUIs in the main window and setting initial values of various data recorded in the RAM 34a. Details of the initialization process will be described later. In step SD, the CPU 34c performs a preview process. The preview process is a process for starting and stopping a preview. Details of the preview process will be described later.

In step SE, the CPU 34c performs template registration processing. The template registration process is a process of displaying a template image in the [Template Image] box and further recording the template image in the RAM 34a. Details of the template registration process will be described later. In step SF, the CPU 34c performs record processing. Record processing is processing for starting and stopping records. Details of the record processing will be described later.

In step SG, the CPU 34c performs image browsing processing. The image browsing process is a process performed by the user for browsing the record image file. Details of the image browsing process will be described later. In step SH, the process branches depending on whether the user presses the [END] button. If the user presses the [Finish] button, the process proceeds to step SI. If the user does not press the [END] button, the process proceeds to step SD. In step SI, the CPU 34c hides the main window and ends the operation of the blade recording software.

Next, the flow of the initialization process (step SC) will be described with reference to FIG. In step SC1, the CPU 34c sets all of the [Preview Stop] button, [Register Template] button, [Record Start] button, and [Record Stop] button to be in an invalid state. Hereinafter, a state in which a GUI such as a button is invalid for a user operation (for example, gray state) is simply described as invalid, and a state in which a user operation is valid is simply described as valid.

In step SC2, the CPU 34c records the number of records R as 0 and the maximum number of records Rm as Ri in the RAM 34a. Ri is an initial value of the maximum number of records Rm, and a predetermined value as Ri is recorded in the RAM 34a. In Step SC3, the CPU 34c performs a process for displaying the record number R (= 0) in the [Record Number] box. In step SC4, the CPU 34c performs a process for displaying the maximum record number Rm in the [maximum record number] box.

In step SC5, the CPU 34c sets all the preview flag, record flag, and save flag to OFF and records them in the RAM 34a. The preview flag is a flag indicating whether or not the current state is a preview state. The record flag is a flag indicating whether or not the current state is being recorded. The save flag is a flag indicating whether or not to save a buffer image (to be described later) as a record image file in a record. Hereinafter, all the flags used during the operation of the blade recording software take ON or OFF values. When the process of step SC5 ends, the process proceeds to step SD.

Next, the flow of the preview process in step SD will be described with reference to FIG. In step SD1, the CPU 34c confirms whether or not the [preview start] button has been pressed by the user. If the [preview start] button is pressed, the process proceeds to step SD2. If the [preview start] button is not pressed, the process proceeds to step SD4.

In step SD2, the CPU 34c disables the [Preview Start] button, enables the [Preview Stop] button, and enables the [Template Registration] button. In step SD3, the CPU 34c sets the preview flag to ON and records it in the RAM 34a.

In step SD4, the CPU 34c confirms whether the preview flag recorded in the RAM 34a is ON. If the preview flag is ON, the process proceeds to step SD5. If the preview flag is OFF, the process proceeds to step SD8.

In step SD5, the CPU 34c captures an image (image signal) for one frame from the image signal processing device 31 as a frame image. Note that at a time before step SD5, the image sensor 30b generates an image signal for one frame, and the image signal processing device 31 converts the image signal into a video signal to generate an image for one frame. Yes.

In step SD6, the CPU 34c records the frame image captured in step SD5 in the RAM 34a. The frame image recorded in the RAM 34a is overwritten every time the CPU 34c captures the frame image. In step SD7, the CPU 34c performs a process for displaying the frame image captured in step SD5 in the [Preview Image] box.

In step SD8, the CPU 34c confirms whether or not the [Preview Stop] button has been pressed by the user. If the [Preview Stop] button is pressed, the process proceeds to step SD9. If the [Preview Stop] button is not pressed, the process proceeds to Step SE.

In step SD9, the CPU 34c validates the [Preview Start] button, invalidates the [Preview Stop] button, and invalidates the [Template Registration] button. In step SD10, the CPU 34c sets the preview flag to OFF and records it in the RAM 34a. When the process of step SD10 ends, the process proceeds to step SE.

Next, the flow of template registration processing in step SE will be described with reference to FIG. In step SE1, the CPU 34c checks whether or not the [template registration] button has been pressed by the user. If the [template registration] button is pressed, the process proceeds to step SE2. If the [template registration] button is not pressed, the process proceeds to step SF.

In step SE2, the CPU 34c records the frame image recorded in the RAM 34a as a template image in the RAM 34a. The template image recorded in the RAM 34a is overwritten every time the [template registration] button is pressed. In step SE3, the CPU 34c performs processing for displaying the frame image recorded in the RAM 34a in the [template image] box. Specifically, the CPU 34c performs a process of superimposing the frame image recorded in the RAM 34a on the graphic image signal, and outputs the processed signal (display signal) to the monitor 22.

From the above, it can be seen that the processing of steps SE2 to SE3 is processing for registering a frame image captured at the timing when the [template registration] button is pressed as a template image. In step SE4, the CPU 34c validates the [record start] button. When the process of step SE4 ends, the process proceeds to step SF.

Next, the flow of record processing in step SF will be described with reference to FIG. In step SF1, the CPU 34c confirms whether or not the [record start] button has been pressed by the user. If the [record start] button is pressed, the process proceeds to step SF2, and if the [record start] button is not pressed, the process proceeds to step SF9.

In step SF2, the CPU 34c disables the [stop preview] button, disables the [template registration] button, disables the [record start] button, enables the [record stop] button, disables the [view image] button, [maximum record] Disable the number box. In step SF3, the CPU 34c sets the number of records R, the correlation value C, the maximum correlation value Cm, the correlation value buffer Cb, and the correlation value status Sc to 0 (R = 0, C = 0, Cm = 0, Cb = 0, Sc = 0) and recorded in the RAM 34a. Details of the correlation value C, the maximum correlation value Cm, the correlation value buffer Cb, and the correlation value status Cd will be described later.

In step SF4, the CPU 34c performs a process for displaying the record number R (= 0) in the [Record Number] box. In step SF5, the CPU 34c acquires the maximum record number Rm input in the [maximum record number] box and records it in the RAM 34a. In step SF6, the CPU 34c creates a storage folder in the memory card 50. At this time, the date and time when the [record start] button was pressed by the user becomes the folder name of the storage folder as it is.

In step SF7, the CPU 34c saves the template image recorded in the RAM 34a in step SE2 of FIG. 14 as an image file (hereinafter referred to as a template image file) in a save folder in the memory card 50. At this time, the template image file name is “Temp.jpg”. In step SF8, the CPU 34c sets the record flag to ON and records it in the RAM 34a.

In step SF9, the CPU 34c confirms whether the record flag recorded in the RAM 34a is ON. If the record flag is ON, the process proceeds to step SF10. If the record flag is OFF, the process proceeds to step SF18.

In step SF10, the CPU 34c calculates a correlation value between the template image and the frame image, and executes a correlation process for determining the timing for storing the record image based on the correlation value. Details of the correlation processing will be described later. In step SF11, the CPU 34c checks whether or not the save flag recorded in the RAM 34a is ON. If the save flag is ON, the process proceeds to step SF12. If the save flag is OFF, the process proceeds to step SF18.

In step SF12, the CPU 34c performs a process for displaying the record image recorded in the RAM 34a in the [record image] box during the correlation process in step SF10. In step SF13, the CPU 34c increments the record number R by 1 (R + 1 is substituted for R) and records it in the RAM 34a. In step SF14, the CPU 34c performs a process for displaying the record number R in the [record image] box.

In step SF15, the CPU 34c saves the record image recorded in the RAM 34a during the correlation process in step SF10 as an image file in a save folder. In step SF16, the CPU 34c sets the save flag to OFF and records it in the RAM 34a. In step SF17, the CPU 34c checks whether or not the record number R is equal to or greater than the maximum record number Rm (R ≧ Rm). If the record number R is greater than or equal to the maximum record number Rm, the process proceeds to step SF19. If the record number R is less than the maximum record number Rm, the process proceeds to step SF18.

In step SF18, the CPU 34c checks whether or not the [record stop] button has been pressed by the user. If the [record stop] button is pressed, the process proceeds to step SF19. If the [record stop] button is not pressed, the process proceeds to step SG.

In step SF19, the CPU 34c enables the [Preview stop] button, enables the [Register template] button, enables the [Record start] button, disables the [Record stop] button, enables the [View image] button, [Max record] Enable the number box. In step SF20, the CPU 34c sets the record flag to OFF and records it in the RAM 34a. When the process of step SF20 ends, the process proceeds to step SG.

Next, the flow of correlation processing in step SF10 will be described with reference to FIG. The correspondence between the correlation process shown in FIG. 16 and the actual change of the correlation value will be described later with reference to FIG. In step SF100, the CPU 34c acquires the luminance value (brightness value) of each pixel of the template image and the frame image recorded in the RAM 34a. Here, for example, the luminance value of the pixel represented by the luminance of each component of RGB is calculated using the following equation (1).
Y = 0.299 × R + 0.587 × G + 0.114 × B (1)

In step SF101, the CPU 34c calculates a correlation value C between the template image and the frame image recorded in the RAM 34a. Details of the correlation value C will be described below. Assuming that the luminance values at the pixel position (x, y) of two images are f1 (x, y) and f2 (x, y), the average luminance values of the two images are respectively expressed by Equation (2) and ( 3) It is expressed by the formula. However, X and Y are the numbers of pixels in the x and y directions, respectively, and Size is the total number of pixels (Size = X × Y).

Furthermore, the standard deviations of the two images are expressed by equations (4) and (5), respectively.

Further, the covariance of the two images is expressed by equation (6).

The correlation value C between the two images is expressed by equation (7). This correlation value C serves as a scale indicating whether or not two images are similar. Generally, the correlation value is close to 1 if they are similar, and tends to approach 0 if they are not similar.

When obtaining the correlation value after thinning out the image size, in the above formulas, when calculating the sum of x and y, the number of increments of x and y is changed, and the total number of pixels Size is changed. Good. For example, when the correlation value is obtained after thinning the image size to ¼, the number of steps for increasing x and y is set to 4, and the total number of pixels Size is set as Size = (X × Y) / (4 × 4). That's fine. If it is desired to increase the speed of the correlation processing, it is effective to use a thinning-out processing because the calculation amount can be reduced.

In step SF102, the CPU 34c checks whether the correlation value status Sc is 0 (Sc = 0). The correlation value status Sc is the status of the correlation value C. The correlation value status Sc takes a value from 0 to 2. When the correlation value status Sc is 0, it is an initial state, when it is 1, it is a state until the CPU 34c finds a frame image to be saved as a record image, and when it is 2, the frame image that the CPU 34c should save as a record image. It is in the state where it was found. If the correlation value status Sc is 0, the process proceeds to step SF103, and if the correlation value status Sc is not 0, the process proceeds to step SF105.

In step SF103, the CPU 34c confirms whether the correlation value C is larger than the correlation value threshold Ct (C> Ct) and the correlation value buffer Cb is equal to or smaller than the correlation value threshold Ct (Cb ≦ Ct). The correlation value threshold value Ct is a threshold value of the correlation value C, and a predetermined value is recorded in the RAM 34a as the correlation value threshold value Ct. The correlation value status Sc changes depending on what value the correlation value C takes compared to the correlation value threshold Ct. What value is set as the correlation value threshold Ct will be described later. The correlation value buffer Cb is a value in a buffer provided in the RAM 34a in order to hold the correlation value C previously calculated by the CPU 34c. If C> Ct and Cb ≦ Ct in step SF103, the process proceeds to step SF104. If C ≦ Ct or Cb> Ct, the process proceeds to step SF105.

In step SF104, the CPU 34c records the correlation value status Sc as 1 (Sc = 1) in the RAM 34a. In step SF105, the CPU 34c checks whether or not the correlation value status Sc is 1 (Sc = 1). When the correlation value status Sc is 1, the process proceeds to step SF106, and when the correlation value status Sc is not 1, the process proceeds to step SF110.

In step SF106, the CPU 34c confirms whether or not the correlation value C is larger than the maximum correlation value Cm (C> Cm). The maximum correlation value Cm is a buffer value for holding the maximum value of the correlation value C. If C> Cm, the process proceeds to step SF107. If C ≦ Cm, the process proceeds to step SF108.

In step SF107, the CPU 34c records the maximum correlation value Cm as the correlation value C (Cm = C) in the RAM 34a. In step SF109, the CPU 34c records the frame image as a buffer image in the RAM 34a. The buffer image recorded in the RAM 34a is overwritten every time the process of step SF109 is executed. The buffer image refers to the image in the buffer provided in the RAM 34a for temporarily holding the frame image until the CPU 34c can confirm that the frame image is a record image (an image highly correlated with the template image). That is.

In step SF108, the CPU 34c sets the correlation value status Sc to 2 (Sc = 2) and records it in the RAM 34a. In step SF110, the CPU 34c checks whether or not the correlation value status Sc is 2 (Sc = 2). If the correlation value status Sc is 2, the process proceeds to step SF111. If the correlation value status Sc is not 2, the process proceeds to step SF116.

In step SF111, the CPU 34c confirms whether the correlation value C is smaller than the correlation value threshold Ct (C <Ct) and the correlation value buffer Cb is equal to or greater than the correlation value threshold Ct (Cb ≧ Ct). If C <Ct and Cb ≧ Ct, the process proceeds to step SF112. If C ≧ Ct or Cb <Ct, the process proceeds to step SF116.

In step SF112, the CPU 34c records the correlation value status Sc as 0 (Sc = 0) in the RAM 34a. In step SF113, the CPU 34c records the maximum correlation value Cm as 0 (Cm = 0) in the RAM 34a. In step SF114, the CPU 34c records the buffer image as a record image in the RAM 34a. The record image recorded in the RAM 34a is overwritten every time the process of step SF114 is executed. In step SF115, the CPU 34c sets the save flag to ON and records it in the RAM 34a.

In step SF116, the CPU 34c records the correlation value buffer Cb as the correlation value C (Cb = C) in the RAM 34a. When the process of step SF116 ends, the process proceeds to step SF11.

FIG. 17 is a graph showing the change with time of the correlation value C. Hereinafter, the details of the record processing and the correlation processing will be described with reference to FIG.

The horizontal axis of the graph shown in FIG. 17 is time, and the vertical axis is the correlation value C calculated by the CPU 34c in step SF101. In the correlation value C, a maximum value and a minimum value appear periodically. The region where the correlation value C shows the maximum value indicates the correlation value between the template image and the blade image. The region where the correlation value C shows a minimum value indicates the correlation value between the template image and the image of the blade background (such as the inner wall of the jet engine). The correlation value threshold value Ct is set so as to be a substantially intermediate value between the two, and is recorded in the RAM 34a.

First, the correlation value status Sc is 0 (Sc = tc) from the timing when the user presses the [record start] button (t = 0) to the timing (t = t1) when the correlation value C becomes larger than the correlation value threshold Ct. 0). Subsequently, the correlation value status Sc is 1 (Sc = 1) from t = t1 to the timing (t = t2) at which the correlation value C shows the maximum value. During this time, the maximum correlation value Cm is sequentially updated to the correlation value C (Cm = C: step SF107), and the frame image is sequentially recorded in the RAM 34a as a buffer image (step SF109).

Subsequently, the correlation value status Sc is 2 (Sc = 2) from t = t2 to the timing when the correlation value C becomes smaller than the correlation value threshold Ct (t = t3). During this time, the maximum correlation value Cm is not updated and remains constant, and the frame image is not recorded as a buffer image in the RAM 34a.

Subsequently, at t = t3, the correlation value status Sc becomes 0 (Sc = 0) again (step SF112), and the buffer image is recorded as a record image in the RAM 34a (step SF114). The buffer image at this time is a frame image at a timing when the correlation value C shows a maximum at t = t2. Thereafter, the frame images at the timing (t = t4, t5, t6,...) At which the correlation value C is maximum are sequentially stored as record images until the user presses the [Stop Record] button.

Next, the flow of the image browsing process in step SG will be described with reference to FIG. In step SG1, the CPU 34c confirms whether or not the [browse image] button is pressed by the user. If the [view image] button is pressed, the process proceeds to step SG2. If the [view image] button is not pressed, the process proceeds to step SH.

In step SG2, the CPU 34c performs processing for displaying an [image browsing] window. As described above, while the [browse image] window is displayed, the main window is in an invalid operation by the user. In step SG3, the CPU 34c performs an initialization process. The initialization process is a process of setting initial states of various GUIs in the [image browsing] window and setting initial values of various data recorded in the RAM 34a. Details of the initialization process will be described later.

In step SG4, the CPU 34c performs a date / time selection process. The date and time selection process is a process in which the CPU 34c detects that the user has changed the selection of the record start date and time in the [Date and time selection] box, and changes the image displayed in the [Browse image] box. Details of the date and time selection process will be described later.

In step SG5, the CPU 34c performs image selection processing. The image selection process is a process in which the CPU 34c detects that the user presses the [<< Previous] button and the [Next >>] button, and changes the image displayed in the [Browse image] box. Details of the image selection process will be described later.

In step SG6, the CPU 34c confirms whether or not the [Close] button has been pressed by the user. If the [Close] button is pressed, the process proceeds to step SG7. If the [Close] button is not pressed, the process proceeds to step SG4. In step SG7, the CPU 34c performs processing for hiding the [image browsing] window. When the process of step SG7 ends, the process proceeds to step SH.

Next, the flow of the initialization process in step SG3 will be described with reference to FIG. In step SG300, the CPU 34c creates a save folder list. In step SG301, the CPU 34c records the created save folder list in the RAM 34a. The save folder list recorded in the RAM 34a is overwritten every time the save folder list is created.

In step SG302, the CPU 34c creates an image file list in the save folder whose save folder number is 1 in the save folder list. In step SG303, the CPU 34c records the created image file list in the RAM 34a. The image folder list recorded in the RAM 34a is overwritten every time an image folder list is created.

In step SG304, the CPU 34c performs a process for displaying all record start dates and times in the [date and time selection] box in the save folder list. In step SG305, the CPU 34c performs processing for highlighting the record start date / time when the save folder number is 1 in the save folder list among the record start dates / times displayed in a list format in the [Select date / time] box.

In step SG306, the CPU 34c performs processing for displaying the image file having the image file number 1 in the image file list in the [browsing image] box. In step SG307, the CPU 34c performs processing for displaying the image file name having the image file number 1 in the image file list in the [image file name] box.

In step SG308, the CPU 34c performs processing for displaying the number of image files in the storage folder whose storage folder number is 1 in the storage folder list in the [Image File Number] box. In step SG309, the CPU 34c performs processing for displaying the save date and time of the image file whose image file number is 1 in the image file list in the [Save Date and Time] box. When the process of step SG309 ends, the process proceeds to step SG4.

Next, the flow of date and time selection processing in step SG4 will be described with reference to FIG. In step SG400, the CPU 34c checks whether or not the selection of the record start date / time in the [Date / time selection] box has been changed by the user. If the selection of the record start date / time has been changed, the process proceeds to step SG401. If the selection of the record start date / time has not been changed, the process proceeds to step SG5.

In step SG401, the CPU 34c obtains from the save folder list the save folder number of the save folder having the record start date and time selected by the user in the [Date and time selection] box. Let F be the folder number acquired at this time. In step SG402, the CPU 34c creates an image file list in the storage folder whose storage folder number is F in the storage folder list. In step SG403, the CPU 34c records the image file list created in step SG402 in the RAM 34a. The image file list recorded in the RAM 34a is overwritten every time an image file list is created.

In step SG404, the CPU 34c performs processing for displaying an image file having an image file number of 1 in the image file list in the [browsing image] box. In step SG405, the CPU 34c performs processing for displaying an image file name having an image file number 1 in the image file list in the [image file name] box.

In step SG406, the CPU 34c performs processing for displaying the number of image files in the storage folder having the folder number 1 in the storage folder list in the [Image File Number] box. In step SG407, the CPU 34c performs processing for displaying the save date and time of the image file whose image file number is 1 in the image file list in the [Save Date and Time] box. When the process of step SG407 ends, the process proceeds to step SG5.

Next, the flow of image selection processing in step SG5 will be described using FIG. In step SG500, the CPU 34c confirms whether or not the [<< previous] button has been pressed by the user. If the “<< previous” button is pressed, the process proceeds to step SG501. If the “<< previous” button is not pressed, the process proceeds to step SG504.

In step SG501, the CPU 34c puts an image file having an image file number one before the image file No. of the image file displayed in the current [browsing image] box in the [browsing image] box in the image file list. Process to display. In step SG502, the CPU 34c performs processing for displaying in the [image file name] box the image file name of the image file having the previous image file No in the image file list.

In step SG503, the CPU 34c performs processing for displaying the save date / time of the image file having the previous image file No in the image file list in the [Save Date / Time] box. In step SG504, the CPU 34c confirms whether or not the [Next >>] button has been pressed by the user. If the [Next >>] button is pressed, the process proceeds to Step SG505. If the [Next >>] button is not pressed, the process proceeds to Step SG6.

In step SG505, the CPU 34c puts an image file having an image file number one after the image file No. of the image file displayed in the current [browsing image] box in the [browsing image] box in the image file list. Process to display. In step SG506, the CPU 34c performs processing for displaying in the [image file name] box the image file name of the image file having the next image file No in the image file list.

In step SG507, the CPU 34c performs processing for displaying the save date / time of the image file having the next image file No in the image file list in the [Save Date / Time] box. When the process of step SG507 ends, the process proceeds to step SG6.

As a modification of the present embodiment, means for identifying the individual jet engine 1 is provided, and the maximum number of turbine blades 10 for each jet engine 1 is stored in the endoscope device 3 so that the identified jet engine 1 The corresponding maximum number may be used when the blade recording software operates. As means for identifying the individual jet engine 1, for example, a barcode or IC tag is attached to the jet engine 1, a reader such as a barcode reader or IC tag reader is connected to the endoscope device 3, and the bar is read by the reader. What is necessary is just to read the identification information of the jet engine 1 from a code or an IC tag.

According to the present embodiment, the following effects can be obtained. In this embodiment, a turbine image in a frame image is selected by selecting a part of the frame image based on a correlation value that is a result of image comparison with a template image from a plurality of frame images obtained by imaging the turbine blade. A frame image can be acquired when the position and angle of the blade are the same as the position and angle of the turbine blade in the template image. For this reason, it is possible to acquire an image of the turbine blade by a simple method without requiring special control for adjusting the rotation of the turbine blade and the timing of imaging.

It is possible to inspect the turbine blade using the image of the turbine blade acquired by the method shown in the present embodiment. In particular, the turbine blade can be inspected in real time by displaying the record image in the record image box 603 of FIG. Further, by saving the record image as an image file in a recording medium, the time and place where the inspection can be performed can be extended. Furthermore, when the image of the turbine blade is saved, if the endoscope video is saved as a moving image file as it is, the file size becomes large, but as in this embodiment, some of the endoscope videos By storing the frame image as a still image file, it is possible to store the turbine blade image necessary for the inspection while preventing the recording capacity of the recording medium from being compressed.

In addition, by obtaining a frame image when the position and angle of the turbine blade in the frame image are the same as the position and angle of the turbine blade in the template image with reference to the template image, the user can obtain the A frame image captured in a state suitable for inspection can be acquired, and inspection can be performed efficiently. Further, by using the template image selected from the frame images, the change over time of the correlation value shown in FIG. 17 becomes clear, so that the accuracy of acquiring the frame image captured in a desired state can be improved. it can. Furthermore, by displaying the template image selected from the frame images, it is possible to make the user confirm whether or not the state of the turbine blade in the acquired frame image is suitable for the user.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment, only the record image file browsing function is mounted in the [image browsing] window of the flade recording software. However, the [image browsing] window in this embodiment has a record image file browsing function. As well as blade defect extraction and stereo measurement functions.

FIG. 22 shows an [image browsing] window in the present embodiment. 22 differs from the [image browsing] window 700 (FIG. 7) in the first embodiment in that a [defect inspection] group box 2201 is located on the right side of the [image browsing] window 2200. It is a point that is arranged. In the [Defect Inspection] group box 2201, various GUIs for performing defect extraction and stereo measurement are arranged. In the following, the case where the browse image 2202 is a pair of left and right images captured through a stereo optical adapter capable of forming two subject images related to the same subject will be mainly described. The stereo optical adapter is attached to the distal end of the endoscope insertion unit 20. Hereinafter, an image displayed on the left side is referred to as a left image, and an image displayed on the right side is referred to as a right image.

Hereinafter, functions of various GUIs in the [Defect Extraction] group box 2201 will be described. [Defect extraction] check box 2210 is a check box for performing defect extraction processing on browse image 2202. When the user checks the [Defect Extraction] check box 2210, a defect outline 2230 is superimposed on the browse image 2202 as shown in FIG. Details of the defect extraction processing will be described later.

[Luminance threshold value] bar 2211 is a bar for setting a luminance threshold value, which is one of inspection parameters for defect extraction processing described later. The luminance threshold is used when binarizing the browse image 2202 in the defect extraction process. [Area threshold value] bar 2212 is a bar for setting an area threshold value which is one of inspection parameters for defect extraction processing described later. The area threshold is used when a small blob (particle) in the browse image is removed in the defect extraction process. [Luminance selection] radio button 2213 is a radio button for setting the type of luminance value, which is one of inspection parameters for defect extraction processing described later. The luminance value is used when the image is converted into a grayscale image in the defect extraction process.

[Stereo measurement] check box 2220 is a check box for performing stereo measurement to be described later on browse image 2202. When the user selects the [Stereo Measurement] check box 2220 with the [Defect Extraction] check box 2210 checked, as shown in FIG. 24, the measurement area line 2240 is superimposed on the browsing image 2202, and the defect is displayed. A stereo measurement is possible for the defect extracted by the extraction process.

The measurement area line 2240 is a boundary line of an area where stereo measurement can be performed in the browsing image 2202, and is displayed as a pair of left and right rectangular lines. Then, as shown in FIG. 25, when the user moves the cursor 2250 to the defect outline 2230 superimposed on the left image of the browsing image 2202, and designates the defect outline 2230 by left clicking or the like, FIG. As described above, the defect outline 2230 is surrounded by the defect rectangular line 2260, the measurement point 2270 is displayed on the left image, and the matching point 2271 is displayed on the right image. Details of the defect rectangular line, the measurement point, and the matching point will be described later. Furthermore, the result of stereo measurement for the designated defect is displayed in a [Measurement Result] box 2222 described later.

[Environmental data] button 2221 in FIG. 22 is a button for selecting environmental data. The environmental data is data used when performing stereo measurement, and includes data for correcting optical distortion of the stereo optical adapter. The environmental data is the same as that described in JP-A-2001-275934.

When the [Environmental Data] button 2221 is pressed, a file selection dialog (not shown) is opened. In this file selection dialog, the user selects environment data. The environmental data selected at this time is data corresponding to the stereo optical adapter used when capturing an image. Then, the [stereo measurement] check box 2220 switches from the invalid state to the valid state, and the [stereo measurement] check box 2220 can be checked.

Note that the [Stereo Measurement] check box and the [Environmental Data] button are always disabled when the browse image is not a stereo measurement image (a pair of left and right images), and the stereo measurement cannot be performed. .

[Measurement result] box 2222 is a box for displaying the measurement result. There are five measurement results: distance, width 1, 2, circumference length, and area. Details of the measurement result will be described later.

Next, the flow of image browsing processing in this embodiment will be described with reference to FIG. The contents of the initialization process in step SG3a, the date selection process in step SG4a, and the image selection process in step SG5a shown in FIG. 27 are different from the flow of the image browsing process in the first embodiment (FIG. 18). Moreover, it is also 1st Embodiment that the defect extraction process of step SG8, the stereo measurement pre-process of step SG9, and the defect designation | designated process of step SG10 are added between step SG5a and step SG6 shown in FIG. This is different from the flow of image browsing processing in FIG. Only the differences from the image browsing process flow (FIG. 18) in the first embodiment will be described below.

In step SG3a, the CPU 34c performs an initialization process. Details of the initialization process will be described later. In step SG4a, the CPU 34c performs date / time selection processing. Details of the date and time selection process will be described later. In step SG5a, the CPU 34c performs image selection processing. Details of the image selection process will be described later.

In step SG8, the CPU 34c performs defect extraction processing. The defect extraction process is a process of performing a process of extracting a defect on the browse image based on the set inspection parameter and displaying the extracted defect superimposed on the browse image. Details of the defect extraction processing will be described later.

In step SG9, the CPU 34c performs stereo measurement preprocessing. The stereo measurement pre-process is a process for correcting a browse image based on selected environment data so that the browse image can be stereo-measured. Details of the stereo measurement pre-processing will be described later.

In step SG10, the CPU 34c performs defect designation processing. The defect designation process is a process in which the CPU 34c detects that the user has designated the defect superimposed on the browse image and displays the measurement result of the defect size in the [Measurement Result] box. Details of the defect designation process will be described later.

Next, the flow of the initialization process in step SG3a will be described with reference to FIG. The point that step SG310 and step SG311 are added after step SG309 shown in FIG. 28 is different from the flow (FIG. 19) of the initialization process in the first embodiment. Only differences from the initialization processing flow (FIG. 19) in the first embodiment will be described below.

In step SG310, the CPU 34c invalidates the [stereo measurement] check box. In step SG311, the CPU 34c records the defect extraction flag in the RAM 34a with the defect extraction flag set to OFF and the stereo measurement flag set to OFF. The defect extraction flag is a flag indicating whether or not to perform defect extraction processing. The stereo measurement flag is a flag indicating whether or not stereo measurement preprocessing is performed.

Next, the flow of date and time selection processing in step SG4a will be described with reference to FIG. The point that step SG408 is added after step SG407 shown in FIG. 29 is different from the flow of date and time selection processing in the first embodiment (FIG. 20). Only the differences from the flow of the date / time selection process (FIG. 20) in the first embodiment will be described below.

In step SG408, the CPU 34c sets the defect extraction flag and the stereo measurement flag to ON and records them in the RAM 34a. In step SG408, the defect extraction flag and the stereo measurement flag are set to ON respectively. In step SG400, when the selection of the record start date / time in the [Date / time selection] box is changed, the browse image is changed, so that the defect is detected. This is because it is necessary to perform extraction processing and stereo measurement preprocessing again. In step SG404, when the image file is displayed in the [browsing image] box, all the measurement area lines and the like that are already superimposed are hidden.

Next, the flow of image selection processing in step SG5 will be described using FIG. Step SG508 is added after step SG503 shown in FIG. 30, and step SG509 is added after step SG507, which is different from the flow of image selection processing in the first embodiment (FIG. 21). Only differences from the image selection processing flow (FIG. 21) in the first embodiment will be described below.

In step SG508, the CPU 34c sets the defect extraction flag and the stereo measurement flag to ON and records them in the RAM 34a. In step SG509, the CPU 34c sets the defect extraction flag and the stereo measurement flag to ON and records them in the RAM 34a. In steps SG508 and SG509, the defect extraction flag and the stereo measurement flag are turned ON because the browse image is changed when the [<< Previous] button and [Next >>] button are pressed in steps SG500 and SG504. This is because the defect extraction process and the stereo measurement pre-process need to be performed again. In steps SG501 and SG505, when the image file is displayed in the [browsing image] box, all the measurement area lines and the like that are already superimposed are hidden.

Next, the flow of the defect extraction process in step SG8 will be described using FIG. 31 and FIG. In step SG800, the CPU 34c confirms whether or not the [Defect extraction] check box is checked. If the [Defect Extraction] check box is checked, the process proceeds to Step SG801. If the [Defect Extraction] check box is not checked, the process proceeds to Step SG802.

In step SG801, the CPU 34c acquires the luminance threshold Yt from the [luminance threshold] bar, the area threshold At from the [area threshold] bar, and the luminance selection S from the [luminance selection] radio button, and records them in the RAM 34a. In step SG802, the CPU 34c confirms whether or not the user has instructed to check the [Defect Extraction] check box. If there is an instruction to check the [Defect Extraction] check box, the process proceeds to Step SG803. If there is no instruction to check the [Defect Extraction] check box, the process proceeds to Step SG9.

In step SG803, as in step SG801, the CPU 34c acquires the luminance threshold Yt from the [luminance threshold] bar, the area threshold At from the [area threshold] bar, and the luminance selection S from the [luminance selection] radio button, and records them in the RAM 34a. To do. In addition, the CPU 34c performs processing for checking the [Defect Extraction] check box.

In step SG804, the CPU 34c sets the luminance threshold Yt, area threshold At, and luminance selection S acquired in step SG803 as the previous luminance threshold Ytl, previous area threshold Atl, and previous luminance selection S1, respectively, and records them in the RAM 34a. The previous luminance threshold value Ytl, the previous area threshold value Atl, and the previous luminance selection S1 are the temporary values of the luminance threshold value Yt, the area threshold value At, and the luminance selection S that were used when the previous defect extraction process was performed. Is recorded. In step SG805, the CPU 34c sets the defect extraction flag to ON and records it in the RAM 34a.

In step SG806, the CPU 34c checks whether the luminance threshold Yt, the area threshold At, and the luminance selection S are all equal to the previous luminance threshold Ytl, the previous area threshold Atl, and the previous luminance selection S1, respectively. The process of step SG806 is a process for confirming whether or not the inspection parameter used when the defect extraction process was performed last time has been changed by the user. If all the inspection parameters are the same as the previous time, the process proceeds to step SG808. If one or more inspection parameters are different from the previous time, the process proceeds to step SG807. In step SG807, the CPU 34c sets the defect extraction flag to ON and records it in the RAM 34a.

In step SG808, the CPU 34c checks whether the defect extraction flag is ON. If the defect extraction flag is ON, the process proceeds to step SG809. If the defect extraction flag is OFF, the process proceeds to step SG821.

Hereinafter, steps SG809 to SG818 will be described using FIG. 33 as appropriate. In step SG809, the CPU 34c acquires the image data of the template image file and the browse image file stored in the storage folder and records them in the RAM 34a. This image data refers to the RGB luminance value of each pixel of the image.

In step SG810, the CPU 34c converts the acquired two pieces of image data into a grayscale image based on the luminance selection S recorded in the RAM 34a in step SG801 or step SG803. When the luminance selection S is “Gray”, the luminance value Y of each pixel of the grayscale image is calculated from the RGB luminance value of each pixel of the image data by the following equation (8).
Y = 0.299 × R + 0.587 × G + 0.114 × B (8)
Further, when the luminance selection S is any one of “R”, “G”, and “B”, the luminance values of R, G, and B of each pixel of the image data are directly applied to the pixels of the grayscale image. The luminance value Y is obtained.

In step SG811, the CPU 34c creates an image (hereinafter referred to as a difference image) obtained by taking the difference between the two grayscale images created in step SG810. FIG. 33 shows a state in which a difference image 3310 is created by taking a difference between the grayscale image 3300 of the template image and the grayscale image 3301 of the browsing image.

In step SG812, the CPU 34c binarizes the difference image based on the luminance threshold value Yt recorded in the RAM 34a and creates a binary image. FIG. 33 shows how a binary image 3320 is created by binarizing the difference image 3310.

In step SG813, the CPU 34c performs expansion / contraction processing on the created binary image to remove small noise. In step SG814, the CPU 34c performs a labeling process on the binary image from which noise has been removed in step SG813, and extracts blobs (particles). In step SG815, the CPU 34c removes the blob having an area smaller than the area threshold At recorded in the RAM 34a from the image from which noise has been removed in step SG814. FIG. 33 shows a state where small blobs are removed from the binary image 3320.

In step SG816, the CPU 34c extracts the outline of the remaining blob as a defect outline from the binary image from which the small blob is removed in step SG814. FIG. 33 shows a blob outline 3330 extracted. In step SG817, the CPU 34c records the coordinates of the defect outline extracted in step SG816 in the RAM 34a.

In step SG818, the CPU 34c performs a process of superimposing and displaying the defect outline on the browse image based on the coordinates of the defect outline recorded in the RAM 34a. FIG. 33 shows a state in which a defect outline 3330 is superimposed on the browse image 3340. In step SG819, the CPU 34c sets the luminance threshold Yt, area threshold At, and luminance selection S recorded in the RAM 34a in step SG801 or step SG803 as the previous luminance threshold Ytl, previous area threshold Atl, and previous luminance selection S1, respectively. , Recorded in the RAM 34a.

In step SG820, the CPU 34c sets the defect extraction flag to OFF and records it in the RAM 34a. In step SG821, the CPU 34c confirms whether or not the user has instructed to remove the check from the “defect extraction” check box. If there is an instruction to remove the check from the [defect extraction] check box, the process proceeds to step SG822. If there is no instruction to remove the check from the [defect extraction] check box, the process proceeds to step SG9.

In step SG822, the CPU 34c performs processing for hiding the defect outline displayed in the browse image based on the coordinates of the defect outline recorded in the RAM 34a in step SG817. In addition, the CPU 34c performs processing for removing the check from the [Defect extraction] check box. When the process of step SG822 ends, the process proceeds to step SG9.

Next, the flow of the stereo measurement preprocessing in step SG9 will be described with reference to FIG. In step SG900, the CPU 34c confirms whether or not the [environment data] button has been pressed by the user. If the [environmental data] button is pressed, the process proceeds to step SG901. If the [environmental data] button is not pressed, the process proceeds to step SG905.

In step SG901, the CPU 34c performs processing for displaying an [open file] dialog (not shown). In step SG902, the CPU 34c checks whether or not environmental data has been selected by the user in the [Open File] dialog. If the environmental data is selected, the process proceeds to step SG903. If the environmental data is not selected, the process proceeds to step SG905.

In step SG903, the CPU 34c records the selected environment data in the RAM 34a. The environmental data recorded in the RAM 34a is overwritten every time environmental data is selected. In step SG904, the CPU 34c enables the [stereo measurement] check box. In step SG905, the CPU 34c confirms whether or not the [stereo measurement] check box is checked. If the [stereo measurement] check box is checked, the process proceeds to step SG908. If the [stereo measurement] check box is not checked, the process proceeds to step SG906.

In step SG906, the CPU 34c confirms whether or not the user gives an instruction to check the [stereo measurement] check box. If there is an instruction to check the [stereo measurement] check box, the process proceeds to step SG907. If there is no instruction to check the [stereo measurement] check box, the process proceeds to step SG10.

In step SG907, the CPU 34c sets the stereo measurement flag to ON and records it in the RAM 34a. Further, the CPU 34c performs processing for checking a [stereo measurement] check box. In step SG908, the CPU 34c checks whether the stereo measurement flag is ON. If the stereo measurement flag is ON, the process proceeds to step SG909. If the stereo measurement flag is OFF, the process proceeds to step SG10.

In step SG909, the CPU 34c performs a process of superimposing and displaying the measurement area line on the browse image based on the coordinates of the measurement area line recorded in the RAM 34a. The coordinates of the measurement area line are recorded in the RAM 34a as part of the environmental data.

In step SG910, the CPU 34c acquires the image data of the browse image file saved in the save folder and records it in the RAM 34a. In step SG911, the CPU 34c corrects the image data acquired in step SG910. The correction process performed in step SG911 is the same as that described in JP-A-10-248806.

In step SG912, the CPU 34c records the image data corrected in step SG911 in the RAM 34a as corrected image data. The corrected image data recorded in the RAM 34a is overwritten every time corrected image data is created. In step SG913, the CPU 34c sets the stereo measurement flag to OFF and records it in the RAM 34a.

In step SG914, the CPU 34c confirms whether the user has instructed to remove the check from the [stereo measurement] check box. If there is an instruction to remove the check from the [stereo measurement] check box, the process proceeds to step SG915. If there is no instruction to remove the check from the [stereo measurement] check box, the process proceeds to step SG10. In step SG915, the CPU 34c performs processing for hiding the measurement area line displayed in the browse image based on the coordinates of the measurement area line recorded in the RAM 34a. Further, the CPU 34c performs processing for removing the check from the [stereo measurement] check box. When the process of step SG915 ends, the process proceeds to step SG10.

Next, the flow of the defect designation process in step SG10 will be described using FIG. In step SG1000, the CPU 34c confirms whether or not the [stereo measurement] check box is checked. If the [stereo measurement] check box is checked, the process proceeds to step SG1001, and if the [stereo measurement] check box is not checked, the process proceeds to step SG6.

In step SG1001, the CPU 34c confirms whether or not the defect contour displayed in the left measurement region of the browse image is designated by the user. If the defect contour is designated by the user, the process proceeds to step SG1002, and if the defect contour is not designated by the user, the process proceeds to step SG6.

In step SG1002, the CPU 34c performs a process of hiding the defective rectangular line, the measurement point, and the matching point that are already superimposed on the browse image. In step SG1003, the CPU 34c performs a process of superimposing and displaying a defective rectangular line on the browse image. The defect rectangular line is a rectangular line displayed around the defect area line designated by the user, and indicates a defect contour line currently designated by the user.

Hereinafter, steps SG1004 to SG1009 will be described with reference to FIG. In step SG1004, the CPU 34c calculates measurement point coordinates based on the coordinates of the defect contour line currently specified by the user recorded in the RAM 34a. A measurement point is a point used when measuring the size of a defect. As shown in FIGS. 36A and 36B, the measurement points 3610 are located at equal intervals on the defect outline 3600.

In step SG1005, the CPU 34c calculates matching point coordinates in the right measurement area corresponding to the measurement point coordinates in the left measurement area, based on the image data of the browsing image. More specifically, the CPU 34c executes pattern matching processing based on the measurement point coordinates, and calculates the coordinates of matching points that are corresponding points of the left and right two images. The pattern matching processing method is the same as that described in Japanese Patent Application Laid-Open No. 2004-49638.

In step SG1006, the CPU 34c calculates the spatial point coordinates (three-dimensional coordinates in the actual space) of each measurement point based on the measurement point coordinates and matching point coordinates calculated in steps SG1004 to SG1005. The calculation method of the spatial point coordinates is the same as that described in Japanese Patent Application Laid-Open No. 2004-49638.

In step SG1007, the CPU 34c calculates a measurement result based on the spatial point coordinates calculated in step SG1006. The measurement results include five types of defect distances, widths 1 and 2, peripheral length, and area.

The distance is an average value of coordinates in the depth direction of all spatial points. As shown in FIGS. 36C and 36D, the width 1 is a spatial distance between measurement points closest to the intersection point of the equivalent ellipse 3620 obtained from all the measurement point coordinates and the major axis 3621 thereof. is there. The width 2 is a spatial distance between measurement points closest to the intersection point between the equivalent ellipse 3620 and its minor axis 3622 as shown in FIGS. 36 (c) and 36 (d). The equivalent ellipse is an ellipse that can be approximated from a plurality of coordinates. The perimeter is the sum of the spatial point distances 3630 of all adjacent measurement points as shown in FIG. The area is a spatial area of a region 3640 surrounded by all adjacent measurement points as shown in FIG.

In step SG1008, the CPU 34c performs a process of superimposing and displaying the measurement points in the left measurement area of the browsing image and displaying the matching points in the right measurement area. In step SG1009, the CPU 34c performs processing for displaying the measurement result calculated in step SG1007 in the [Measurement Result] box. When the process of step SG1009 ends, the process proceeds to step SG6.

In the present embodiment, a browsing image captured using a stereo optical adapter is used. However, for defect extraction processing, a browsing image captured using an optical adapter other than the stereo optical adapter can also be used. . In an [image browsing] window 3700 shown in FIG. 37, a browsing image 3701 obtained by capturing one subject image formed by the optical adapter is displayed. When the user checks the [Defect Extraction] check box 3710, a defect outline 3720 is superimposed on the browse image 3701 as shown in FIG.

In the present embodiment, as shown in FIG. 23, the defect outline 2230 is displayed at the position corresponding to the defect extracted by the defect extraction process in the browse image 2202, but the position of the defect can be clearly shown. For example, it is not necessary to display a line. For example, a graphic such as an arrow may be displayed at a position corresponding to the defect, or a character such as “defect” may be displayed.

According to this embodiment, by extracting the difference between the browse image and the template image, it is possible to extract a blade defect regardless of the type of defect. Furthermore, by displaying a defect outline or the like superimposed on the extracted defect, the user can easily recognize the position of the defect.

Further, the size of the defect can be known by performing measurement on the extracted defect. Furthermore, the defect size which a user desires can be known by performing a measurement with respect to the defect designated by designating a defect outline among the defects displayed on the browsing image. Furthermore, the defect extraction process is performed using the browsing image captured using the stereo optical adapter, and the three-dimensional size of the defect can be known by executing stereo measurement based on the extracted defect.

As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the above-described embodiments, and includes design changes and the like without departing from the gist of the present invention. .

DESCRIPTION OF SYMBOLS 1 ... Jet engine, 2 ... Turning tool, 3 ... Endoscope apparatus, 4 ... Video terminal cable, 5 ... Video capture card, 6 ... PC, 7 ... LAN Cable 10, turbine blade 20, endoscope insertion section 21, endoscope apparatus main body 22, 25 monitor (display section) 23 remote control (input section) 24 ... PC body, 30a ... imaging optical system, 30b ... imaging device, 31 ... image signal processing device, 32 ... light source, 33 ... bending control unit, 34, 35. ..Control computer, 34a, 35a... RAM, 34b... ROM, 34c, 35c... CPU (image comparison unit, image selection unit, difference extraction unit, measurement unit), 34d, 35d. Network I / F, 34e ... RS2 2C I / F, 34f ··· card I / F, 35b ··· HDD, 35e ··· USB I / F

Claims (5)

An image comparison unit that compares images by looking at a correlation with a template image for each image of a plurality of images obtained by rotating the blades periodically arranged in the jet engine, and
An image selection unit that selects a part of images based on a result of image comparison by the image comparison unit from a plurality of images obtained by imaging the blade;
A difference extraction unit that extracts a difference from the template image for each image for a part of the images selected by the image selection unit;
An image processing apparatus comprising: The image comparison unit calculates a correlation value with the template image for each of a plurality of images obtained by imaging the blade;
2. The image according to claim 1, wherein the image selection unit selects the partial image from a plurality of images obtained by imaging the blade based on the correlation value calculated by the image comparison unit. Processing equipment. A template image registration unit for registering the template image;
The template image registration unit, the image processing device according to the plurality of images, to Claim 2, an image in which there is an instruction of the user and registers as the template image. A display signal generator for outputting a display signal to the display;
4. The image processing apparatus according to claim 3, wherein the display signal generation unit generates a display signal for causing the display unit to display the template image.   The difference extraction unit converts each of the partial image selected by the image selection unit and the template image into a grayscale image, and converts the grayscale image of the partial image and the grayscale of the template image. 2. The image processing apparatus according to claim 1, wherein a difference is extracted from the scale image.