JP5602449B2 - Endoscope device - Google Patents

Description

  The present invention relates to an endoscope apparatus having a measurement function. The present invention also relates to a program for operating an endoscope apparatus.

  In gas turbines mainly used for aircraft, the inside becomes extremely hot, and defects (burning) such as burning and discoloration may occur on the surface of the turbine blade. The size (dimension) of this defect is one of the conditions for judging blade replacement, and its inspection is extremely important. For inspection of the blade, an endoscope apparatus having a measurement function is used. In this blade inspection, the endoscope apparatus measures the area of the defect based on an image obtained by imaging the defect (hereinafter referred to as a measurement image), and displays the measurement result (see, for example, Patent Document 1). After confirming the measurement results, the user needs to replace the blade if the area is large, because there is a problem with the defect. If the area is small, the user determines that there is no problem with the defect and the blade does not need to be replaced. It was.

JP 2008-206956 A

  The area of the defect is one of the indicators for determining whether or not there is a problem with the defect, but it may cause an erroneous determination. For example, even if the area is small, if there is a large defect in the erosion of the blade, it may be determined that the blade needs to be replaced. In this case, in the image obtained by imaging the blade, for example, the brightness and color of the defect are often greatly different from the colors in the peripheral part. Therefore, it is effective to determine the defect based on the image characteristics such as brightness and color. It is. Therefore, it is possible to make a determination in consideration of both the area and the feature of the image.

  The present invention has been made in view of the above-described problems, and is an endoscope apparatus capable of notifying a user of information for assisting determination when the user visually observes the subject and makes a determination regarding the subject. The purpose is to provide a program.

An endoscope apparatus according to an embodiment of the present invention includes an imaging unit that images a subject and generates image data, a reference point setting unit that sets two reference points in an image based on the image data, Based on two reference points, a reference line setting unit that sets a plurality of reference lines that bisect a feature region in the image, and an extraction that sets extraction points for extracting the features of the image on the reference line A feature of the image at the extraction point is calculated based on a point setting unit, a measurement unit that calculates spatial coordinates of the reference point or the extraction point, performs measurement on the subject, and information on the image at the extraction point. Stored in the storage unit together with the generation unit that generates the feature information to be shown, the storage unit that stores the feature information corresponding to the plurality of reference lines and the measurement result by the measurement unit, and the image . Previous A display unit for results switchably displayed for each reference line of measurement by the feature information and the measurement unit corresponding to a plurality of reference lines, the said characteristic information determines whether satisfies a predetermined criterion 1 The extraction point setting unit sets a plurality of the extraction points on the reference line, and the first determination unit sets the extraction set in the first part of the reference line. Determining whether a difference between the feature information at a point and the feature information at the extraction point set at a second part different from the first part of the reference line satisfies a predetermined criterion It is.

  In the endoscope apparatus of the present invention, the extraction point setting unit sets a plurality of extraction points, the generation unit generates the feature information for each of the extraction points, and the display unit includes: The feature information for each position of the plurality of extraction points is displayed.

  In the endoscope apparatus of the present invention, the reference line setting unit sets a plurality of reference lines, and the extraction point setting unit sets a plurality of extraction points on each of the plurality of reference lines. The display unit displays the feature information for each position of the plurality of extraction points so as to be switchable for each reference line.

  The endoscope apparatus according to the present invention further includes a first determination unit that determines whether or not the feature information satisfies a predetermined criterion.

  In the endoscope apparatus of the present invention, the extraction point setting unit sets a plurality of extraction points on the reference line, and the first determination unit is set to a first part of the reference line. Whether the difference between the feature information at the extraction point and the feature information at the extraction point set in a second part different from the first part of the reference line satisfies a predetermined criterion. It is characterized by determining.

  In the endoscope apparatus of the present invention, the display unit further displays a result of determination by the first determination unit.

  In the endoscope apparatus of the present invention, the extraction point setting unit sets a plurality of the extraction points, and the measurement unit calculates a spatial coordinate of the plurality of extraction points, for each of the extraction points. The measurement on the subject is performed, and the display unit displays the measurement result for each of the positions of the plurality of extraction points.

  In the endoscope apparatus of the present invention, the reference line setting unit sets a plurality of reference lines, and the extraction point setting unit sets a plurality of extraction points on each of the plurality of reference lines. The display unit displays the measurement result for each position of the plurality of extraction points so as to be switched for each reference line.

  The endoscope apparatus according to the present invention further includes a second determination unit that determines whether or not the measurement result satisfies a predetermined criterion.

  In the endoscope apparatus of the present invention, the extraction point setting unit sets a plurality of extraction points on the reference line, and the second determination unit is set to a first part of the reference line. Whether the difference between the measurement result at the extraction point and the measurement result at the extraction point set in the second part different from the first part of the reference line satisfies a predetermined criterion. It is characterized by determining.

  In the endoscope apparatus of the present invention, the display unit further displays a result of determination by the second determination unit.

  In the endoscope apparatus of the present invention, the generation unit generates the feature information based on luminance information of the image at the extraction point.

  In the endoscope apparatus of the present invention, the generating unit generates the feature information based on color information of the image at the extraction point.

  The present invention also provides an endoscope apparatus that includes an imaging unit that captures an image of a subject and generates image data, and a display unit that displays the image. In the image based on the image data, two reference points are provided. Setting a reference line that bisects the feature region in the image based on the two reference points, and setting an extraction point for extracting the feature of the image on the reference line A step of calculating a spatial coordinate of the reference point or the extraction point and measuring the subject, and a feature indicating the characteristics of the image at the extraction point based on information on the image at the extraction point It is a program for executing a step of generating information and a step of displaying the feature information and a result of measurement by the measurement unit together with the image.

  According to the present invention, the feature information at the extraction point set on the reference line that bisects the feature region in the image and the measurement result at the reference point or the extraction point are displayed. It is possible to notify the user of information for assisting the determination when making a determination regarding the subject.

1 is a block diagram showing a configuration of an endoscope apparatus according to a first embodiment of the present invention. It is a block diagram which shows the structure of the measurement process part with which the endoscope apparatus by the 1st Embodiment of this invention is provided. It is a reference figure which shows the measurement screen in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the measurement in the 1st Embodiment of this invention. It is a reference diagram showing a reference point, a reference line, and an intermediate point in the first embodiment of the present invention. It is a reference figure which shows the depth profile in the 1st Embodiment of this invention. It is a reference diagram showing a luminance profile in the first embodiment of the present invention. It is a reference figure which shows the depth profile in the 1st Embodiment of this invention. It is a reference diagram showing a luminance profile in the first embodiment of the present invention. It is a flowchart which shows the procedure of the depth profile calculation process in the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the brightness | luminance profile calculation process in the 1st Embodiment of this invention. It is a reference figure which shows the measurement screen in the 1st Embodiment of this invention. It is a reference figure showing a standard point, a standard line, and a reference point in the first embodiment of the present invention. It is a reference figure which shows the measurement screen in the 1st Embodiment of this invention. It is a block diagram which shows the structure of the measurement process part with which the endoscope apparatus by the 2nd Embodiment of this invention is provided. It is a flowchart which shows the procedure of the measurement in the 2nd Embodiment of this invention. It is a reference diagram showing a reference point, a reference line, a reference point, and a reference area in the second embodiment of the present invention. It is a reference figure which shows the measurement screen in the 2nd Embodiment of this invention. It is a reference figure showing the standard point in the 2nd embodiment of the present invention, a standard line, and a reference point. It is a reference figure which shows the measurement screen in the 2nd Embodiment of this invention. It is a reference figure for demonstrating how to obtain | require the three-dimensional coordinate of the measurement point by stereo measurement.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Below, the measurement function of a defect is demonstrated by making into an example the case where the burning of a turbine blade is made into a measuring object.

(First embodiment)
First, a first embodiment of the present invention will be described. FIG. 1 shows the configuration of the endoscope apparatus according to the present embodiment. As shown in FIG. 1, an endoscope apparatus 1 includes an endoscope 2, a control unit 3, a remote controller 4 (input device), a liquid crystal monitor 5, optical adapters 7a, 7b, and 7c, and an endoscope. The mirror unit 8, the camera control unit 9, and the control unit 10 are configured.

  An endoscope 2 (electronic endoscope) that captures an image of a measurement object and generates an imaging signal includes an elongated insertion unit 20. The insertion portion 20 is configured by connecting, in order from the distal end side, a rigid distal end portion 21, a bending portion 22 that can be bent vertically and horizontally, and a flexible tube portion 23 having flexibility. A proximal end portion of the insertion portion 20 is connected to the endoscope unit 8. For example, the distal end portion 21 may be a variety of optical adapters such as a stereo optical adapter 7a, 7b (hereinafter referred to as a stereo optical adapter) having two observation fields or a normal observation optical adapter 7c having only one observation field. It is configured to be detachable by screwing.

  The control unit 3 includes an endoscope unit 8, a camera control unit (hereinafter referred to as CCU) 9 as an image processing means, and a control unit 10 as a control device. The endoscope unit 8 includes a light source device that supplies illumination light necessary for observation, and a bending device that bends the bending portion 22 that constitutes the insertion portion 20. The CCU 9 receives an imaging signal output from the solid-state imaging device 2 a built in the distal end portion 21 of the insertion unit 20, converts this into a video signal such as an NTSC signal, and supplies the video signal to the control unit 10. The solid-state imaging device 2a photoelectrically converts a subject image formed through an optical adapter and generates an imaging signal.

  The control unit 10 includes an audio signal processing circuit 11, a video signal processing circuit 12, a ROM 13, a RAM 14, a PC card interface (hereinafter referred to as PC card I / F) 15, and a USB interface (hereinafter referred to as USB). (Described as I / F) 16, an RS-232C interface (hereinafter referred to as RS-232C I / F) 17, and a measurement processing unit 18.

  An audio signal collected by the microphone 34, an audio signal obtained by reproducing a recording medium such as a memory card, or an audio signal generated by the measurement processing unit 18 is supplied to the audio signal processing circuit 11. The video signal processing circuit 12 generates a video signal from the CCU 9 under the control of the measurement processing unit 18 in order to display a composite image obtained by synthesizing the endoscopic image supplied from the CCU 9 and the graphic operation menu. A process of combining with a graphic image signal for an operation menu or the like is performed. In addition, the video signal processing circuit 12 performs a predetermined process on the combined video signal and supplies it to the liquid crystal monitor 5 in order to display the video on the screen of the liquid crystal monitor 5.

  Further, the video signal processing circuit 12 outputs image data based on the video signal from the CCU 9 to the measurement processing unit 18. Since a stereo optical adapter is attached to the distal end portion 21 at the time of measurement, the image based on the image data from the video signal processing circuit 12 includes a plurality of subject images related to the same subject as a measurement target. In this embodiment, as an example, a pair of left and right subject images are included.

  The PC card I / F 15 can freely attach and detach a memory card (recording medium) such as the PCMCIA memory card 32 and the flash memory card 33. By mounting the memory card, the control processing information, the image information, the optical data, etc. stored in the memory card are taken in according to the control of the measurement processing unit 18, or the control processing information, the image information, the optical data, etc. Can be recorded on a memory card.

  The USB I / F 16 is an interface for electrically connecting the control unit 3 and a personal computer (PC) 31. By electrically connecting the control unit 3 and the PC 31 via the USB I / F 16, various control instructions such as an endoscope image display instruction and image processing at the time of measurement can be performed on the PC 31 side. It becomes possible. Also, various processing information and data can be input / output between the control unit 3 and the PC 31.

  The RS-232C I / F 17 is connected to the CCU 9 and the endoscope unit 8, and is connected to the remote controller 4 for controlling and operating the CCU 9 and the endoscope unit 8. When the user operates the remote controller 4, communication necessary for controlling the operation of the CCU 9 and the endoscope unit 8 is performed based on the operation content.

  The measurement processing unit 18 acquires image data from the video signal processing circuit 12 by executing a program stored in the ROM 13, and executes measurement processing based on the image data. The RAM 14 is used as a work area for temporary storage of data by the measurement processing unit 18. FIG. 2 shows the configuration of the measurement processing unit 18. As shown in FIG. 2, the measurement processing unit 18 includes a control unit 18a, a reference point designation unit 18b, a reference line calculation unit 18c, a coordinate calculation unit 18d, profile calculation units 18e and 18f, and a defect diagnosis unit 18g. And a storage unit 18h.

  The control unit 18a controls each unit in the measurement processing unit 18. The control unit 18 a also has a function of generating a graphic image signal for displaying a measurement result, an operation menu, and the like on the liquid crystal monitor 5 and outputting the graphic image signal to the video signal processing circuit 12.

  The reference point specifying unit 18b specifies a reference point on the measurement object based on a signal input from the remote controller 4 or the PC 31 (input unit). When the user inputs a desired reference point while viewing the image of the measurement object displayed on the liquid crystal monitor 5, the coordinates are calculated by the reference point designating unit 18b. In the following description, it is assumed that the user operates the remote controller 4, but the same applies when the user operates the PC 31. In the present embodiment, two reference points are set on the image.

  The reference line calculation unit 18c sets a reference line passing through the two reference points specified by the reference point specifying unit 18b, and calculates a straight line formula for determining the reference line. In this specification, the two-dimensional coordinates on the image displayed on the liquid crystal monitor 5 are described as image coordinates, and the three-dimensional coordinates in the actual space are described as space coordinates. The coordinate calculation unit 18d sets a reference point (extraction point) that serves as a reference position for extracting feature information (luminance in this embodiment) indicating the feature of an image used for defect diagnosis, and calculates the image coordinates. In this embodiment, a reference point is set on the reference line.

  The profile calculation unit 18e calculates spatial coordinates at a position based on the reference point, and calculates a depth position corresponding to the distance to the subject (object distance). This depth position is the measurement result. In addition, the profile calculation unit 18e generates a depth profile in which each position of the reference point set on the reference line is associated with the depth position at each position. The depth profile indicates the surface shape of the subject including the defect.

  The profile calculation unit 18f generates luminance information that is image feature information at a position based on the reference point. In addition, the profile calculation unit 18f generates a luminance profile in which each position of the reference point set on the reference line is associated with the luminance information at each position. The luminance profile indicates the luminance distribution on the surface of the subject including the defect.

  The defect diagnosis unit 18g performs defect diagnosis for diagnosing whether or not there is a problem with the defect by determining whether or not the depth profile and the luminance profile satisfy predetermined criteria. The storage unit 18h stores various types of information processed in the measurement processing unit 18. Information stored in the storage unit 18h is appropriately read out by the control unit 18a and output to each unit.

Next, how to obtain the three-dimensional coordinates of the measurement point by stereo measurement will be described with reference to FIG. The three-dimensional coordinates (X, Y, Z) of the measurement target point 60 are calculated by the following formulas (1) to (3) by the triangulation method for the images picked up by the left and right optical systems. Is done. However, the coordinates of the measurement point 61 and the corresponding point 62 (the point on the right image corresponding to the measurement point 61 on the left image) on the left and right images subjected to distortion correction are respectively (X _L , Y _L ), ( X _R , Y _R ), the distance between the left and right optical centers 63 and 64 is D, the focal length is F, and t = D / (X _L −X _R ).
X = t × X _R + D / 2 (1)
Y = t × Y _R (2)
Z = t × F (3)

  When the coordinates of the measurement point 61 and the corresponding point 62 are determined as described above, the three-dimensional coordinates of the measurement target point 60 are obtained using the parameters D and F. By obtaining three-dimensional coordinates of several points, various measurements such as the distance between two points, the distance between two points and the distance between one point, area, depth, and surface shape can be performed. It is also possible to obtain the distance (object distance) from the left optical center 63 or the right optical center 64 to the subject. In order to perform the above stereo measurement, optical data indicating the characteristics of the optical system including the tip 21 and the stereo optical adapter is required. The details of the optical data are described in, for example, Japanese Patent Application Laid-Open No. 2004-49638, and the description thereof is omitted.

  Next, the measurement screen in this embodiment will be described. In this embodiment, a defect is measured by stereo measurement. In stereo measurement, since the measurement object is imaged with the stereo optical adapter attached to the distal end portion 21 of the endoscope 2, an image of the measurement object is displayed in a pair on the measurement screen.

  FIG. 3 shows a measurement screen before the start of measurement. As measurement information, a left image of the measurement object is displayed on the left screen 700, and a right image of the measurement object is displayed on the right screen 710. Further, in the area on the measurement screen excluding the left screen 700 and the right screen 710, as other measurement information, optical adapter name information 720, time information 721, message information 722, icons 723a, 723b, 723c, 723d, 723e, and A zoom window 724 is displayed.

  Both the optical adapter name information 720 and the time information 721 are information indicating measurement conditions. The optical adapter name information 720 is character information indicating the name of the optical adapter currently used. The time information 721 is character information indicating the current date and time. The message information 722 includes character information indicating an operation instruction to the user and character information indicating the coordinates of a reference point that is one of the measurement conditions.

  The icons 723a to 723e constitute an operation menu for the user to input an operation instruction such as switching of the measurement mode or clearing of the measurement result. When the user operates the remote controller 4 to move the cursor 725 over any of the icons 723 a to 723 e and perform an operation such as a click, a signal corresponding to the operation is input to the measurement processing unit 18. The control unit 18a recognizes an operation instruction from the user based on the signal and controls the measurement process. In the zoom window 724, an enlarged image of the measurement object located around the cursor 725 is displayed.

  Next, the measurement procedure in this embodiment will be described. FIG. 4 shows a measurement procedure. In step SA, the user designates two reference points by operating the remote controller 4 on the measurement screen displayed on the liquid crystal monitor 5. While the user designates the reference point, the reference point designating unit 18b detects the cursor movement instruction based on the signal from the remote controller 4, and calculates the image coordinates after the cursor movement. The control unit 18 a generates a graphic image signal for displaying a cursor at the image coordinates calculated by the reference point designating unit 18 b and outputs the graphic image signal to the video signal processing circuit 12. As a result, the cursor moves on the measurement screen in response to a cursor movement instruction from the user.

  When the user inputs an instruction to designate a reference point, the reference point designating unit 18b recognizes the image coordinates of the current cursor as the image coordinates of the reference point. When the reference point is designated, the control unit 18a generates a graphic image signal for displaying the reference point at the image coordinates of the reference point, and outputs the graphic image signal to the video signal processing circuit 12. As a result, the reference point is displayed on the left screen.

  In step SB, the reference line calculation unit 18c calculates a reference line that passes through the two reference points specified in step SA. As shown in FIG. 5A, the user designates the reference points 100 a and 100 b so as to be located around the defect 110. A straight line 120 connecting the reference points 100a and 100b is the reference line of this embodiment. The reference line is a straight line that divides the defect 110 into two. It is preferable that the reference line passes near the center of the defect 110.

  In step SC, the coordinate calculation unit 18d calculates image coordinates of a plurality of reference points located on the reference line. As shown in FIG. 5B, a plurality of reference points 130 are set between two reference points on the reference line. A reference point may be set at the position of two reference points. In step SD, the coordinate calculation unit 18d calculates the image coordinates of the intermediate point located between the two reference points. As shown in FIG. 5C, an intermediate point 140 is set on a reference line passing through the two reference points.

  In step SE, the profile calculation unit 18e calculates a depth profile. As described above, the depth profile is a profile in which each position of the reference point is associated with the depth position (object distance) at each position, and indicates the defect and the surface shape around the defect. A method for calculating the depth profile will be described later.

  FIG. 6 shows an example of a depth profile. The horizontal axis is the position on the image, the left end is one of the two reference points, the right end is the other of the two reference points, and the center is the position of the intermediate point. The vertical axis is the depth position (Z). The profile 200 is graph data of a depth profile calculated by the profile calculation unit 18e. The shape of the profile 200 in FIG. 6 shows the surface shape of a defect at a point located between two reference points, that is, a reference point. In the graph of FIG. 6, since the depth position at the midpoint position is larger than the depth position at the two reference point positions, it can be seen that the vicinity of the center of the defect is recessed.

  In step SF, the profile calculation unit 18f calculates a luminance profile. As described above, the luminance profile is a profile in which each position of the reference point is associated with the luminance information at each position, and indicates the luminance distribution of the defect and the surrounding surface. A method for calculating the luminance profile will be described later.

  FIG. 7 shows an example of a luminance profile. The horizontal axis is the position on the image, the left end is one of the two reference points, the right end is the other of the two reference points, and the center is the position of the intermediate point. The vertical axis represents the luminance value. The profile 210 is graph data of a luminance profile calculated by the profile calculation unit 18f. The shape of the profile 210 in FIG. 7 shows the luminance distribution of the surface of the defect at a point located between two reference points, that is, a reference point. In the graph of FIG. 7, it can be seen that the brightness near the center of the defect is dark because the brightness at the position of the intermediate point is smaller than the brightness at the position of the two reference points.

  In step SG, the defect diagnosis unit 18g performs defect diagnosis based on the depth profile and the luminance profile. As illustrated in FIG. 8, the defect diagnosis unit 18 g calculates the average of data around the two reference points in the depth profile (data in the areas 220 and 221) and the data around the intermediate point (data in the area 222). And the difference between each average is calculated. If this difference is greater than or equal to a predetermined value, the defect diagnosis unit 18g considers that the vicinity of the center of the defect is greatly recessed and sets the diagnosis result to NG. Further, if this difference is less than a predetermined value, the defect diagnosis unit 18g considers that the surface shape of the defect is smooth, and sets the diagnosis result to OK.

  As shown in FIG. 9, the defect diagnosis unit 18g also calculates the average of the data around the two reference points (data in the areas 230 and 231) in the luminance profile and the data around the intermediate point (in the area 232). Data) average, and the difference between each average is calculated. If this difference is equal to or greater than a predetermined value, the defect diagnosis unit 18g considers that the vicinity of the center of the defect is dark, that is, dark, and sets the diagnosis result to NG. Further, if this difference is less than the predetermined value, the defect diagnosis unit 18g considers that the luminance distribution on the surface of the defect is smooth, and sets the diagnosis result to OK. The defect diagnosis unit 18g individually obtains a diagnosis result based on each of the depth profile and the luminance profile as described above. The defect diagnosis unit 18g may obtain an integrated diagnosis result based on both the depth profile and the luminance profile. For example, the defect diagnosis unit 18g may obtain the result of logical sum of OK and NG as an integrated diagnosis result, with the diagnosis result being OK as 0, NG as 1.

  In step SH, the control unit 18 a generates a graphic image signal for displaying the reference line and the intermediate point on the left screen, and outputs the graphic image signal to the video signal processing circuit 12. As a result, the reference line and the intermediate point are displayed on the left screen. Note that step SH is not necessarily executed after step SG, and may be executed after step SD. In step SI, the control unit 18a generates a graphic image signal for displaying the result window on the right screen and outputs the graphic image signal to the video signal processing circuit 12. As a result, a result window is displayed on the right screen. On the result window, a depth profile, a luminance profile, and a diagnosis result are displayed. Details of the contents displayed in the result window will be described later.

  Next, the procedure of step SE (depth profile calculation process) will be described. FIG. 10 shows the procedure of step SE. In step SE1, the image coordinates of the reference point calculated by the coordinate calculation unit 18d are input to the profile calculation unit 18e. In step SE2, the profile calculation unit 18e calculates the image coordinates of the matching point in the right screen corresponding to the reference point in the left screen. More specifically, the profile calculation unit 18e performs pattern matching processing based on the image coordinates of the reference points, and calculates the image coordinates of matching points that are corresponding points of the left and right two images. The pattern matching processing method is the same as that described in Japanese Patent Application Laid-Open No. 2004-49638.

  In step SE3, the profile calculation unit 18e calculates the spatial coordinates based on the image coordinates of the reference point in the left screen and the image coordinates of the matching point in the right screen. The method for calculating the spatial coordinates is the same as the method described with reference to FIG. The Z coordinate of this spatial coordinate is the depth position (object distance). In step SE4, the profile calculation unit 18e generates a depth profile in which each position of the reference point is associated with the depth position at each position. In step SE5, the profile calculation unit 18e outputs the depth profile to the control unit 18a.

Next, the procedure of step SF (depth profile calculation process) will be described. FIG. 11 shows the procedure of step SF. In step SF1, the image coordinates of the reference point calculated by the coordinate calculation unit 18d are input to the profile calculation unit 18f. In step SF2, the profile calculation unit 18f acquires luminance information of each pixel on the reference point from the image data. At this time, the luminance value Y of each pixel is calculated from the RGB component values of each pixel by the following equation (1). This luminance value Y is luminance information.
Y = 0.299 × R + 0.587 × G + 0.114 × B (1)

  In step SF3, the profile calculation unit 18f generates a luminance profile in which each position of the reference point is associated with the luminance information at each position. In step SF4, the profile calculation unit 18f outputs the luminance profile to the control unit 18a.

  Next, a display method of the defect measurement result and the diagnosis result will be described. FIG. 12 shows a measurement screen when a defect measurement result and a diagnosis result are displayed. On the left screen, two reference points (×), an intermediate point (●), and a reference line are displayed. A result window 300 is displayed on the right screen. In the upper part of the result window 300, a depth profile graph 310, a difference (Differ) between the average of the peripheral data of the two reference points and the average of the peripheral data of the intermediate point, and a diagnosis result (Result) are displayed. In the lower portion of the result window 300, a luminance profile graph 320, a difference (Differ) between the average of the peripheral data of the two reference points and the average of the peripheral data of the intermediate point, and a diagnosis result (Result) are displayed. The user can check whether there is a problem with the defect based on the measurement result and the diagnosis result of the defect.

Next, a first modification of the present embodiment will be described. In the above description, luminance information is used as feature information when performing defect diagnosis. In this modification, color information is used. As color information, an xy value used in the xy chromaticity diagram is used. Relational expressions between RGB values and xy values are as shown in the following equations (2) and (3).
x = 0.60 × R−0.28 × G−0.32 × B (2)
y = 0.20 × R−0.52 × G−0.31 × B (3)

  Then, the distance between coordinates in the xy chromaticity diagram is used as a value representing the difference in color information. If the average color information of the area around the two reference points is xa, ya and the color information of the area around the intermediate point is xb, yb, the difference D between the two color information is expressed by the following equation (4). . This modification is effective when there is little difference in luminance between the inside and around the defect and there is only a color difference.

  Next, a second modification of the present embodiment will be described. In the above description, the depth profile and the luminance profile on one straight line (reference line) are calculated. In this modification, the depth profile and the luminance profile on a plurality of straight lines (reference line) are calculated at once. be able to.

  First, the user designates the first reference point. At this time, as shown in FIG. 13A, it is desirable that the user designates the reference point 400 so as to be located at the center of the defect 410. When the user inputs an instruction to designate a reference point, the reference point designating unit 18b recognizes the current image coordinates of the cursor 420 as the image coordinates of the first reference point.

  Subsequently, when the user moves the cursor, as shown in FIG. 13B, a circle 430 (reference circle) whose radius is the distance between the reference point and the cursor is displayed. At this time, the reference point designating unit 18b calculates a reference circle based on the image coordinates of the reference point and the image coordinates of the cursor. The control unit 18 a generates a graphic image signal for displaying the reference circle on the left screen and outputs the graphic image signal to the video signal processing circuit 12. As a result, the reference circle is displayed on the left screen.

  Subsequently, the user designates a second reference point. At this time, as shown in FIG. 13C, the user desirably designates the reference point 440 so as to be located outside the defect. When the user inputs an instruction to designate a reference point, the reference point designation unit 18b recognizes the image coordinates of the current cursor as the image coordinates of the second reference point.

  As shown in FIG. 13D, when a reference point is designated, an even number of points 450 (reference circle constituent points) arranged at a predetermined interval on the reference circle are set. The coordinate calculation unit 18d calculates the image coordinates of the reference circle composing point on the reference circle. In FIG. 13D, there are eight reference circle composing points, but more or fewer reference circle composing points may be set. Subsequently, as shown in FIG. 13 (e), a plurality of straight lines 460 (reference lines) connecting the reference circle composing points at positions symmetrical to each other around the first reference point are set. The reference line calculation unit 18c calculates a reference line that connects reference circle composing points that are symmetrical to each other around the first reference point. Each reference line is a straight line that divides the defect area into two. As described above, the user can set a plurality of reference lines with a simple operation.

  A plurality of reference points are set on these reference lines, and a depth profile and a luminance profile are calculated for each reference line by the processing described above. The depth profile and the luminance profile are stored in the storage unit 18h in association with the reference line (in association with information for identifying the reference line).

  FIG. 14A shows a measurement screen when the defect measurement result and the diagnosis result are displayed. On the left screen, two reference points (x), reference circles, reference lines, reference circle composing points (●), and reference circle composing point numbers are displayed. The numbers of the reference circle composing points to be displayed are 1 and 1 'if they are the numbers of the reference circle composing points constituting the reference line of number 1. If the number of the reference circle composing point constituting the reference line of number 2 is 2 and 2 '.

  A result window 500 is displayed on the right screen. A difference from the result window 300 of FIG. 12 is that a display switching button 510 is arranged at the top of the window. In the result window 500, the depth profile and the luminance profile on the reference line 1-1 ′ are displayed as an initial setting. As shown in FIG. 14B, the user moves the cursor, and the display switching button 510 is displayed. By pressing, the displayed reference line number is switched, and the depth profile and the luminance profile on the reference line 2-2 ′ are displayed. When the user presses the triangle button on the right side of the display switching button 510, the displayed reference line number is incremented by one. When the user presses the triangle button on the left side of the display switching button 510, the reference line number displayed is one. Returning, the depth profile and the luminance profile corresponding to the updated reference line are displayed.

  Based on the signal from the remote controller 4, the control unit 18 a recognizes the reference line designated by the user by operating the display switching button 510. Subsequently, the control unit 18a reads the depth profile and the luminance profile corresponding to the reference line designated by the user from the storage unit 18h, generates a graphic image signal for displaying them, and outputs the graphic image signal to the video signal processing circuit 12. . Thereby, the depth profile and the luminance profile at the reference line designated by the user are displayed.

  As described above, according to the present embodiment, since the difference in luminance information and the measurement result at the reference point set on the reference line that bisects the defect area are displayed, the user visually observes the measurement object. The user can be notified of information that assists the determination when determining whether there is a problem with the defect. This embodiment is effective when diagnosing a defect having a difference in image characteristics between the inside and outside of the defect area, such as burning.

  Further, by displaying the brightness profile, it is possible to notify the user of detailed information regarding the defect state (surface brightness). Furthermore, by displaying the luminance profile for each reference line in a switchable manner, the user can make a detailed determination of a defect while comparing the luminance profiles.

  Further, it is determined whether or not there is a problem with the defect by determining whether or not the feature information of the image satisfies a predetermined criterion, specifically, whether or not the difference in luminance information is equal to or greater than a predetermined value. This can be done based on objective criteria. Furthermore, by displaying the result of this determination, the user can be notified of the objective determination result for the defect.

  Further, by displaying the depth profile as a measurement result, it is possible to notify the user of detailed information regarding the defect state (surface shape). Furthermore, by displaying the depth profile for each reference line in a switchable manner, the user can make a detailed determination of the defect while comparing the depth profiles.

  In addition, it is objective to determine whether or not there is a problem with the defect by determining whether or not the measurement result satisfies a predetermined criterion, specifically, whether or not the difference in depth position is greater than or equal to a predetermined value. Can be done based on various criteria. Furthermore, by displaying the result of this determination, the user can be notified of the objective determination result for the defect.

  Also, by generating luminance information as image feature information, it is possible to obtain information suitable for determining a defect having a characteristic in luminance (difference in luminance between the inside and outside of the defect). . Also, by generating color information as image feature information, it is possible to obtain information suitable for making a determination about a defect having a color characteristic (a color difference between the inside and the outside of the defect). . In the present embodiment, luminance information or color information is used as image feature information. However, the feature information is not limited to the luminance information or the color information as long as the feature information is reflected.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. The configuration of the endoscope apparatus according to the present embodiment is as shown in FIG. 1, but the configuration of the measurement processing unit 18 is different. FIG. 15 shows the configuration of the measurement processing unit 18 of the present embodiment. As shown in FIG. 15, the measurement processing unit 18 includes a control unit 18a, a reference point designation unit 18b, a reference line calculation unit 18c, a coordinate calculation unit 18d, a defect diagnosis unit 18g, a storage unit 18h, and a defect size. The calculation unit 18i and the luminance acquisition unit 18j are configured. The configuration shown in FIG. 15 is different from the configuration shown in FIG. 2 in that the profile calculation units 18e and 18f are deleted and a defect size calculation unit 18i and a luminance acquisition unit 18j are added. The defect size calculation unit 18i calculates the spatial coordinates of the reference point and calculates the size of the defect. The luminance acquisition unit 18j acquires luminance information at the reference point from the image data.

  Next, the measurement procedure in this embodiment will be described. FIG. 16 shows a measurement procedure. In step Sa, the user designates two reference points by operating the remote controller 4 on the measurement screen displayed on the liquid crystal monitor 5. The process at this time is the same as the process in step SA of FIG.

  In step Sb, the reference line calculation unit 18c calculates a reference line that passes through the two reference points specified in step Sa. As shown in FIG. 17A, the user designates the reference points 600 a and 600 b so as to be located on the edge of the defect 610. A straight line connecting the reference points 600a and 600b is the reference line of this embodiment. The reference line 620 is a straight line that divides the defect 610 into two. It is preferable that the reference line 620 passes near the center of the defect 610.

  In step Sc, the coordinate calculation unit 18d calculates image coordinates of a plurality of reference points located on the reference line. More specifically, as shown in FIG. 17B, the coordinate calculation unit 18d displays images of reference points 630a and 630b that are outside the defect area by a predetermined distance from each of the two reference points on the reference line. The coordinates are calculated, and further the image coordinates of the reference point 630c located in the middle of the two reference points are calculated. Subsequently, as illustrated in FIG. 17C, the coordinate calculation unit 18 d has images of reference areas 640 a, 640 b, and 640 c that are rectangular image processing ranges for calculating luminance information with each reference point as the center. Calculate the coordinates. Further, the coordinate calculation unit 18d sets a plurality of reference points in each reference area.

  In step Sd, the defect size calculation unit 18i calculates the size of the defect. The size of the defect in the present embodiment is the width of the defect, and is a spatial distance (distance between two points) between two spatial coordinates corresponding to the two reference points specified in step Sa. In step Se, the luminance acquisition unit 18j acquires luminance information of each pixel on the reference point in the reference area from the image data.

In step Sf, the defect diagnosis unit 18g performs defect diagnosis based on the luminance information acquired in step Se. More specifically, the defect diagnosis unit 18g compares the luminance information of each reference area and diagnoses the defect based on the comparison result. The average luminance information of the reference areas outside the defective area (reference areas 640a and 640b in FIG. 17C) is assumed to be Ya, and the luminance information of the reference area inside the defective area (reference area 640c in FIG. 17C). If Yb is Yb, the luminance difference D that is the difference between the two luminance information is expressed by the following equation (4).
D = | Ya−Yb | (4)

  If the luminance difference D is equal to or greater than a predetermined value, the defect diagnosis unit 18g considers that the luminance difference between the inside and the periphery of the defect is large and sets the diagnosis result to NG. Further, if the above-described luminance difference D is less than the predetermined value, the defect diagnosis unit 18g considers that the luminance difference is small and sets the diagnosis result to OK.

  In step Sg, the control unit 18 a generates a graphic image signal for displaying a reference line on the left screen and outputs the graphic image signal to the video signal processing circuit 12. As a result, the reference line is displayed on the left screen. In step Sh, the control unit 18a generates a graphic image signal for displaying the result window on the right screen and outputs the graphic image signal to the video signal processing circuit 12. As a result, a result window is displayed on the right screen. On the result window, the measurement result and diagnosis result of the defect are displayed.

  FIG. 18 shows a measurement screen when the defect measurement result and the diagnosis result are displayed. On the left screen, two reference points (x) and a reference line are displayed. A result window 800 is displayed on the right screen. An image of the defect is displayed at the top of the result window 800. Further, in the lower part of the result window 800, in addition to the distance between two points (W) that is a defect measurement result, a difference in luminance information (Differ) and a diagnosis result (Result) are displayed. The user can check whether there is a problem with the defect based on the measurement result and the diagnosis result of the defect.

  Next, a modification of this embodiment will be described. In the above, a single straight line (reference line) was obtained to calculate the size and luminance difference of the defect, but in this modification, a plurality of straight lines (reference line) were obtained to determine the defect size and the luminance difference. It can be calculated at once.

  As in the modification in the first embodiment, when the user designates the reference point 900 located almost in the center of the defect area in FIG. 19, the distance between the reference point 900 and the cursor 910 is set with the reference point 900 as the center. A reference circle 920 as a radius is displayed. Subsequently, the user designates a second reference point. At this time, unlike the modification in the first embodiment, as shown in FIG. 19, it is desirable that the user designates the reference point 930 so as to be positioned on the edge of the defect. That is, it is desirable that the user designates the reference point 930 so that the reference circle matches the defect edge as much as possible.

  When two reference points are designated, an even number of reference circle composing points 940 arranged at predetermined intervals on the reference circle are set. At this time, the coordinate calculation unit 18d calculates the image coordinates of the reference circle composing point on the reference circle. In addition, a plurality of reference lines 950 connecting the reference circle composing points that are symmetrical to each other around the reference point 900 are set. The reference line calculation unit 18c calculates a reference line that connects reference circle composing points that are symmetrical to each other around the first reference point. Each reference line is a straight line that divides the defect area into two. As described above, the user can set a plurality of reference lines with a simple operation.

  Subsequently, although not shown, a reference point that is outside the defect area by a predetermined distance from the reference circle composing point is set on each reference line. At this time, the coordinate calculation unit 18d calculates the image coordinates of the reference point. Subsequently, the coordinate calculation unit 18d calculates the image coordinates of the reference area centered on the reference point and the reference point, and sets a plurality of reference points in each reference area. Based on the plurality of reference points, a luminance difference is calculated for each reference line by the processing described above. A spatial distance between two spatial coordinates corresponding to the two reference points is calculated as a measurement result. The measurement result and the luminance difference are stored in the storage unit 18h in association with the reference line (in association with information for identifying the reference line).

  FIG. 20 shows a measurement screen when a defect measurement result and a diagnosis result are displayed. On the left screen, two reference points (x), reference circles, reference lines, reference circle composing points (●), and reference circle composing point numbers are displayed. A result window 1000 is displayed on the right screen. A difference from the result window 800 of FIG. 18 is that a display switching button 1010 is arranged at the center of the window. The function of the display switching button 1010 is the same as the function of the display switching button 510 in FIG.

  Based on the signal from the remote controller 4, the control unit 18 a recognizes the reference line designated by the user by operating the display switching button 1010. Subsequently, the control unit 18a reads out the measurement result and the luminance difference corresponding to the reference line designated by the user from the storage unit 18h, generates a graphic image signal for displaying them, and outputs it to the video signal processing circuit 12. . Thereby, the measurement result and the luminance difference corresponding to the reference line designated by the user are displayed.

  As described above, according to the present embodiment, as in the first embodiment, when the user visually checks the measurement object and determines whether there is a problem with the defect, the information that assists the determination is obtained. The user can be notified. Further, by displaying the measurement result and the luminance difference for each reference line in a switchable manner, the user can make a detailed determination of the defect while comparing the measurement results and the luminance difference.

  In this embodiment, luminance information is used as image feature information. However, the color information described above can also be used, and the feature information may be information reflecting the feature of the defect. It is not limited to information or color information.

  As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the above-described embodiments, and includes design changes and the like without departing from the gist of the present invention. .

  DESCRIPTION OF SYMBOLS 1 ... Endoscope apparatus, 2a ... Solid-state image sensor (imaging part), 4 ... Remote controller (input device), 5 ... Liquid crystal monitor (display part), 12 ... Video signal processing Circuit (imaging unit), 18 ... measurement processing unit, 18a ... control unit, 18b ... reference point designation unit (reference point setting unit), 18c ... reference line calculation unit (reference line setting unit) , 18d ... coordinate calculation unit (extraction point setting unit), 18e ... profile calculation unit (measurement unit), 18f ... profile calculation unit (generation unit), 18g ... defect diagnosis unit (determination unit) , 18h ... storage unit, 18i ... defect size calculation unit (measurement unit), 18j ... luminance acquisition unit (generation unit)

Claims (4)

An imaging unit that images a subject and generates image data;
A reference point setting unit for setting two reference points in the image based on the image data;
A reference line setting unit that sets a plurality of reference lines that bisect a feature region in the image based on the two reference points;
An extraction point setting unit for setting an extraction point for extracting the feature of the image on the reference line;
A measurement unit that calculates spatial coordinates of the reference point or the extraction point and performs measurement on the subject;
Based on the information of the image at the extraction point, a generation unit that generates feature information indicating the feature of the image at the extraction point;
A storage unit for storing the feature information corresponding to the plurality of reference lines and a measurement result by the measurement unit;
A display unit that is stored in the storage unit together with the image, and that displays the feature information corresponding to the plurality of reference lines and a measurement result by the measurement unit in a switchable manner for each reference line ;
A first determination unit that determines whether or not the feature information satisfies a predetermined criterion;
With
The extraction point setting unit sets a plurality of the extraction points on the reference line,
The first determination unit is configured to set the feature information at the extraction point set in the first part of the reference line and the second part of the reference line different from the first part. An endoscope apparatus, wherein it is determined whether or not a difference from the feature information at an extraction point satisfies a predetermined criterion.   The endoscope apparatus according to claim 1, further comprising a second determination unit that determines whether or not a result of the measurement satisfies a predetermined criterion. The extraction point setting unit sets a plurality of the extraction points on the reference line,
The second determination unit is set to a second result different from the first part of the reference line and a result of the measurement at the extraction point set to the first part of the reference line. The endoscope apparatus according to claim 2, wherein it is determined whether or not a difference from the measurement result at the extraction point satisfies a predetermined criterion. The endoscope apparatus according to claim 3 , wherein the display unit further displays a result of determination by the first determination unit or a result of determination by the second determination unit.

Images