JP2010279539A - Diagnosis supporting apparatus, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To recognize for which section of a subject the image has been photographed in the display of an endoscopic image. <P>SOLUTION: A background image B is formed by composing two or more images having an overlapped region with each other among images which have been photographed by an endoscope 10 up to the present point. Feature points which are present in both of the present image P(ts), which has been photographed by the endoscope 10 at the present point ts, and a reference image P(tb), which is one of the two or more images constituting the background image B, are detected in each of the present image P(ts) and the reference image P(tb). Based on the shift of the position Qs of the feature point in the present image from the position Qb of the feature point on the reference image, the position L(ts) of the endoscope at the present point on the background image B is acquired. A position displaying image 30 which expresses the acquired position L(ts) of the endoscope on the background image B is formed, and is displayed on the screen of a display 5 together with the image which has been photographed by the endoscope at the present point. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention is an invention suitable for use in the medical field, and relates to an apparatus and method for supporting image diagnosis based on an image taken by an endoscope, and a computer program.

  2. Description of the Related Art Endoscopes are widely used that have an optical system built into the main body and can be viewed inside by inserting it into the body. Such endoscopes include a scope-type endoscope having a slender shape and a capsule-like endoscope (capsule endoscope). In addition to observation, there are some that can perform some degree of surgery and sample collection.

  And, for example, in Patent Document 1, in order to reduce the burden of observing an endoscopic image by a user, when providing a large number of images obtained for a long time with a capsule endoscope for image diagnosis, It has been proposed to provide each image as a single developed image that is not an individual image but connected to each other.

  Further, in Patent Document 2, when a plurality of images obtained by photographing with a plurality of optical systems incorporated in an endoscope are displayed side by side on a monitor, two or more optical systems with overlapping photographing ranges are displayed. It has been proposed to improve the viewing performance of endoscopic images by preventing overlapping areas between images taken in step 1 from being displayed in an overlapping manner.

JP 2007-236700 A JP 2000-325307 A

  Since an endoscope is usually used for imaging in a body cavity or a thin tube, there is a problem that it is difficult to determine which part of the subject is captured only by the captured image.

  The present invention provides an improved diagnosis support apparatus, method, and program so that a doctor performing diagnosis can recognize which part of an object is captured by an image captured by an endoscope. The purpose is to do.

  The diagnosis support apparatus according to the present invention includes a background image generation unit that generates a background image by combining two or more images having regions overlapping each other among images captured up to the present time by an endoscope, A feature point existing in both of the current image captured at the present time by the mirror and one of two or more reference images constituting the background image is detected in each of the current image and the one reference image. Obtains the current position of the endoscope on the background image based on the feature point detection means and the displacement of the detected feature point position on the current image from the detected feature point position on the reference image A position display image for generating a position display image representing the position of the endoscope acquired by the endoscope position acquisition unit on the background image generated by the background image generation unit And forming means, characterized in that an image display means for displaying on the screen together with the image captured at the moment by the endoscope position indication image generated by the position display image generating means.

  Here, the term “current time” refers to the time when the image is taken when the position of the endoscope at the time of photographing is to be acquired and displayed together with the image taken by the endoscope.

  In addition, the images captured up to the present time include images captured at the current time, that is, current images.

  In addition, when the endoscope is a scope type, the position of the endoscope means, for example, the position of the distal end portion where an optical system or the like is provided. When the endoscope is a capsule endoscope, for example, the capsule Refers to the position of the body.

  In addition, the position display image may be an image that shows where the position of the endoscope is in the background image, and does not necessarily have to be a picture of the endoscope itself written on the background image. .

  Further, “two or more images having an overlapping area between images” means two or more images each having an area overlapping with at least one of the other images.

  “Generate two or more images to generate a background image” may be to generate a background image for all areas represented by each image taken up to the present time, or to be in all areas. A background image may be generated only for a partial region. However, it is necessary to include at least a background portion corresponding to the current position of the endoscope.

  Further, the image that is displayed on the screen and is currently captured by the endoscope may be the current image or an image that is captured by a different camera.

  In the above apparatus, the position acquisition means acquires a shift in the position of the endoscope at the current time from the position of the endoscope at the time of shooting the one reference image based on the shift in the position of the feature point. The position of the endoscope may be acquired.

  The position display image generation means may generate a position display image by combining an endoscope picture with the acquired endoscope position on a background image.

  The position display image generating means is provided with a detecting means for detecting the operation status of the accessory of the endoscope, and the position display image generating means is a position that further represents the current state of the accessory on the background image based on the detected operation status of the accessory. A display image may be generated.

  The image display means has a recording means for recording the position display image generated by the position display image generation means in association with the current image, and the image display means has taken the position display image recorded in the recording means at the present time by the endoscope You may further have a function to display on a screen with an image.

  A position display image comprising recording means for recording the background image generated by the background image generation means and the position of the endoscope acquired by the endoscope position acquisition means in association with the image captured at the present time by the endoscope; The generation unit may further have a function of generating a position display image using the background image recorded in the recording unit and the position of the endoscope.

  The diagnosis support method according to the present invention includes a step of generating a background image by combining two or more images having regions that overlap each other among images captured up to the present time by an endoscope, and an endoscope at the present time. A step of detecting a feature point existing in both of the captured current image and one of two or more reference images constituting the background image in each of the current image and the one reference image, on the reference image Acquiring the current position of the endoscope on the background image based on the deviation of the position of the detected feature point on the current image from the position of the detected feature point in A step of generating a position display image representing the position of the endoscope, and a step of displaying the generated position display image on the screen together with the image currently captured by the endoscope It is an feature.

  In the above method, the step of acquiring the position of the endoscope is based on the shift in the position of the feature point, and the shift in the position of the endoscope at the current time from the position of the endoscope at the time of shooting the one reference image. The position of the endoscope at the present time may be acquired by acquiring.

  The diagnosis support program of the present invention is a program that causes a computer to function as background image generation means, feature point detection means, endoscope position acquisition means, position display image generation means, and image display means in the diagnosis support apparatus. is there. This program is recorded on a recording medium such as a CD-ROM or DVD, or is recorded in a downloadable state on a built-in disk or network storage of a server computer and provided to the user.

  According to the diagnosis support apparatus, method, and program of the present invention, a background image is generated by synthesizing two or more images having a region where the images overlap among images captured up to the present time by an endoscope. A feature point that is present in both the current image captured at the present time by the endoscope and one of two or more reference images that constitute the background image is detected in each of the current image and the one reference image. The current position of the endoscope on the background image is acquired on the background image based on the deviation of the position of the detected feature point on the current image from the position of the detected feature point on the reference image. A position display image representing the acquired position of the endoscope is generated, and the generated position display image is displayed on the screen together with the image currently captured by the endoscope. Therefore, a doctor performing diagnosis refers to a position display image when performing image diagnosis or the like using an image captured by an endoscope, so that which part of the subject is captured by the endoscopic image. You can recognize if you are.

1 is a schematic configuration diagram showing an embodiment of a diagnosis support apparatus of the present invention. The perspective view which shows the structure of the front-end | tip part of an endoscope The figure for demonstrating the process which produces | generates a background image using a side image The figure which shows the relationship between the photographing range of the front camera and the side camera of the endoscope The figure for demonstrating the process which produces | generates a background image using a side image and a front image The figure which shows an example of the feature point detected from the reference | standard image The figure which shows an example of the feature point detected from the present image Diagram showing movement vector of endoscope The figure which shows an example of the position display image produced | generated by the position display image production | generation process The figure which shows an example of the screen displayed on a display by a diagnostic assistance apparatus The figure which shows the example of arrangement | positioning of the stereo camera provided in the front-end | tip part of an endoscope The figure which shows the example of arrangement | positioning of the stereo camera provided in the front-end | tip part of an endoscope The figure which shows an example of the sensor which detects the operation condition of an accessory The figure which shows an example of the position display image which represented the state of the accessory further

  Embodiments of a diagnosis support apparatus and method and a diagnosis support program according to the present invention will be described below in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of an endoscopic image diagnostic system. As shown in the figure, this system is inserted into a body cavity of a subject and provides an image for diagnosis based on an endoscope 10 that images an observation site and an image acquired by the endoscope 10 And a diagnosis support apparatus 1 for performing the above.

  The endoscope 10 includes an insertion unit 11 that is inserted into a body cavity of a subject, a switch that receives a predetermined operation instruction by an operator, and an operation unit 12 that is provided integrally with the insertion unit 11. A cable 13 for connecting the operation unit 12 and the diagnosis support apparatus 1 is provided.

  As shown in the perspective view of FIG. 2, an illumination window 11b that illuminates the photographing range with light from a light source is provided at the distal end of the insertion portion 11 so that the optical axis coincides with or is parallel to the axial direction of the insertion portion 11. In addition to the high-resolution front camera Cf and the side cameras Cu, Cd, Cr, Cl provided on the top, bottom, left, and right sides that run perpendicular to the direction, the forceps outlet 11c connected to the forceps port 14, washing water and air A nozzle 11a and the like for ejecting and sucking the air are provided.

  The operation unit 12 includes a first operation switch 12a that receives an air / water supply operation instruction, a second operation switch 12b that receives a suction operation instruction, an operation button 12c that is a shutter button that receives an imaging timing, A rotation controller 12d for receiving an instruction of the direction of the distal end portion of the insertion portion 11.

  The diagnosis support apparatus 1 has a diagnosis support program installed in a computer. The computer may be a workstation or a personal computer directly operated by a doctor who performs diagnosis, or may be a server computer connected to them via a network. The diagnosis support program is stored and distributed on a recording medium such as a DVD or CD-ROM, and is installed in the computer from the recording medium. Alternatively, it is stored in a storage device of a server computer connected to a network or a network storage in a state where it can be accessed from the outside, and downloaded and installed on a computer used by a doctor upon request.

  The diagnosis support apparatus 1 includes a CPU 2, a memory 3, and a hard disk 4 as a standard workstation configuration. The diagnosis support apparatus 1 is connected to a display 5 and an input device such as a mouse 6.

  In addition to the image data sequentially acquired by the endoscope 10, the hard disk 4 includes data (for example, position display image data, background image data) obtained by the CPU 2 performing various processes to be described later on the image data. , Endoscope position data) is stored.

  The memory 3 stores a diagnosis support program. The diagnosis support program defines a background image generation process, a feature point detection process, an endoscope position acquisition process, a position display image generation process, an image display process, and an image recording process as processes to be executed by the CPU 2. Then, when the CPU 2 executes these processes according to the program, the general-purpose workstation can perform background image generation means, feature point detection means, endoscope position acquisition means, position display image generation means, image display means, and image recording. It will function as a means (recording means).

  Next, specific processing performed by the diagnosis support apparatus 1 in order to provide a diagnostic image will be described. First, the diagnosis support apparatus 1 executes background image generation processing. In the background image generation process, the diagnosis support apparatus 1 first joins (synthesizes) images acquired by the side cameras Cu, Cd, Cr, and Cl from the start of imaging with the endoscope 10 to the present time. A background image B is generated. Specifically, as shown in FIG. 3, images Pu (t0), Pd (t0), Pr (t0), and Pl (t0) acquired by the side cameras Cu, Cd, Cr, and Cl at the photographing time t0. To create a developed image P (t0). Further, the developed images P (t1), P (t2),..., P (tk) are created by the same processing at the other shooting times t1, t2,. Next, the developed images P (t0) to P (tk) are further connected to generate a background image B. In this background image B generation process, blood vessel patterns and organ structure patterns in each image are extracted, and the images are joined based on the extraction result. The background image thus generated is an image obtained by capturing the inner wall of the luminal organ from one direction.

  Here, the case where the background image B is generated using the images acquired by the side cameras Cu, Cd, Cr, and Cl (hereinafter referred to as the side images) has been described. However, the image acquired by the front camera Cf ( Hereinafter, it may be generated using a front image). Specifically, it is assumed that the luminal organ has a cylindrical structure, and the optical axis of the front camera Cf substantially coincides with the core line of the luminal organ. By performing a geometric conversion process assigned to the top, a developed image is created from the front images acquired at each time point, and the developed images are further connected to generate the background image B.

  However, the normal side camera usually has its optical axis close to the inner wall of the luminal organ, and the side image acquired by the side camera tends to have less geometric distortion than the front image. For image generation, it is preferable to use the side image in preference to the front image. On the other hand, as shown in FIG. 4, with the side camera alone, the visual field range of the luminal organ in the direction of the core line direction is limited to r1, so If the image of the portion (field of view range r2) is further used, a wider range of background images can be generated. FIG. 5 shows a background image B generated by further joining the developed image Pf (tk) of the front image at the shooting time tk to the background image of FIG. 3 generated only from the side images at the shooting time t1 to tk.

  In addition, here, the case where the background image is generated by synthesizing all the images captured by the endoscope 10 up to the present time (by the side camera and / or the front camera) has been described. Is, for example, two or more images including an area where adjacent images overlap each other including the current image taken at the present time (for example, images of the past N frames including the current image, where N is set to 2 or more Changeable) may be combined to generate a background image, or a background image may be generated by combining two or more images taken before the current time and having an overlapping area between adjacent images. May be.

  Note that when generating one background image, it is sufficient if there is an overlapping area between images so that missing areas do not occur. Therefore, the captured images used for creating the background image do not need to have consecutive shooting timings. The images may be taken at time intervals set so as to have overlapping regions.

  The background image generation process may be performed at a predetermined time interval, or whenever the endoscope movement vector from the previous background image generation process point exceeds a predetermined amount. Alternatively, it may be performed at a timing manually designated by an operator using an input device such as the mouse 6.

  Hereinafter, each captured image adopted as the background image is expressed as a reference image. Since the angle of view at the time of shooting is constant, the relative positional relationship of the endoscope in one reference image is fixed. Therefore, in the entire background image, the endoscope corresponding to each reference image constituting the background image The position (position in the background image) can be specified.

  Next, a specific example of the feature point detection process performed by the diagnosis support apparatus 1 will be described with reference to FIGS. In the feature point detection process, the diagnosis support apparatus 1 first extracts a blood vessel pattern from the current image Pl (ts) photographed at the present time by the endoscope 10 and detects a blood vessel branch point Qs as a feature point. Furthermore, a blood vessel pattern is extracted from one reference image Pl (tb) of two or more reference images that are images constituting the background image, and a feature point Qb corresponding to the feature point Qs is determined based on the extracted blood vessel pattern. To detect.

  Here, the corresponding feature point Qb is detected by determining a block area (displayed as a dotted rectangular area in the figure) of a predetermined size including the feature point Qs in the current image Pl (ts), and its pixel value. The block region having the distribution characteristic of the pixel value that most closely matches the distribution characteristic may be detected from the reference image Pl (tb). Specifically, a block of a predetermined size is scanned on the reference image Pl (tb), and a difference square sum of each pixel of the block region and each pixel of the block region determined in the current image Pl (ts) is calculated. A block region having the minimum value is detected (see JP-A-2009-104284).

  In addition, the corresponding feature point is detected from the reference image Pl (tb) by detecting a block region having a blood vessel pattern that matches the blood vessel pattern in the block region determined to include the feature point Qs in the current image Pl (ts). Qb may be detected. Here, for blood vessel pattern matching (detection of matching blood vessel patterns), the personal authentication algorithm based on the blood vessel pattern described in Japanese Patent Application Laid-Open No. 2008-097109, the personal authentication algorithm based on the fingerprint pattern described in Japanese Patent Application Laid-Open No. You can do that. A plurality of adjacent reference images may be used as the reference image for detecting the corresponding feature point Qb.

  Here, the blood vessel pattern is extracted by performing a predetermined matrix operation on the target image to generate a spectral image in which the blood vessel pattern included in the image is emphasized, and the color information of the generated spectral image Is binarized (monochrome), and a binarized image in which a blood vessel appears as a black region is used as a blood vessel pattern. In addition, as described in JP-A-2003-93336, the spectral image is obtained by converting the RGB color sensitivity characteristics into numerical data and the spectral characteristics of a specific narrowband bandpass as numerical data. It is formed by obtaining a relationship with an object as matrix data (coefficient set) and performing an operation based on the matrix data on the RGB signal of the target endoscopic image.

  Note that the reference image Pl (tb) has only to have an area overlapping with the current image Pl (ts) and the position of the endoscope 10 is known at the photographing time tb, and the current image Pl ( It is not necessary that the images are taken continuously in time with ts).

Next, endoscope position acquisition processing performed by the diagnosis support apparatus 1 will be described with reference to FIGS. In the endoscope position acquisition process, the diagnosis support apparatus 1 first shifts the detected feature point position Qs on the current image Pl (ts) from the feature point position Qb on the reference image Pl (tb). Qs-Qb (= -W) is acquired. Then, on the contrary from the positional deviation of the feature points, a deviation of the position of the endoscope on those images, that is, a movement vector W (wx, wy) is obtained. Furthermore, the endoscope movement vector W (wx, wy) on the images is converted into the endoscope movement vector V (vx, vy) in real space by the following equation (1). The endoscope at the current time ts on the background image is added to the endoscope position L (tb) at the reference time tb by the endoscope movement vector V (vx, vy) obtained by the above calculation. Get the position L (ts).
V = A x W x (d / d0) (1)
Here, d is the subject distance of the current image, d0 is a reference value of the subject distance, and A is a conversion coefficient between the inter-pixel distance and the real space distance when the subject distance is d0. The subject distance d may be converted as a predetermined fixed value, or may be derived from the result of autofocus.

  If the photographing magnification β can be changed, the conversion coefficient A corresponding to the value of each photographing magnification β is stored in advance, and the above calculation is performed based on this. In addition, when the distortion characteristics of the photographing lens in the camera are stored in advance and the geometric distortion of the image due to the distortion characteristics of the lens is corrected based on this, the position acquisition processing of the endoscope is performed. Good.

  Subsequently, the diagnosis support apparatus 1 generates a position display image 30 representing the current position L (ts) of the endoscope acquired by the position acquisition process on the background image B generated by the background generation process. Display image generation processing is performed. For example, as shown in FIG. 9, a position display image 30 is generated by combining an endoscope picture E with the endoscope position L (ts) acquired in the position acquisition process on the background image B. The position display image 30 thus generated is a CG image as if the endoscope in the body cavity of the subject is viewed from the outside, and the subject can be viewed from all directions by setting / changing the viewpoint position. It is possible to generate a position display image in which the endoscope in the body cavity is viewed from above.

  It should be noted that the position display image 30 may be an image that shows where the position of the endoscope is in the background image, and is not necessarily limited to an image in which the picture E of the endoscope itself is written on the background image B. For example, an area corresponding to the imaging range of the endoscope is displayed on the background image B in a color different from other areas, or a predetermined mark is attached to the position of the endoscope on the background image B. Including things.

  Further, in the position display image 30 shown in FIG. 9, in addition to the picture E of the endoscope, the photographing range Er (ts) by the front camera is displayed in front of the picture E of the endoscope. When displaying the shooting range Er (ts) by the front camera in the position display image 30, the rotation amount of the rotation controller 12d by the operator is detected, and the bending of the distal end portion of the insertion portion 11 corresponding to the rotation amount is detected. It is better if the angle is further acquired and the photographing range Er (ts) is adjusted based on the angle. Furthermore, it is preferable to adjust the shooting range Er (ts) so that the image in the shooting range Er (ts) matches the front image actually acquired by the front camera at the current time ts.

  Subsequently, the diagnosis support apparatus 1 performs image display processing for displaying the generated position display image 30 on the screen of the display 5 together with the current front image acquired by the front camera Cf at the current time ts. For example, as shown in FIG. 10, a front image 7, a CG bird's-eye image 8 from above, and a CG bird's-eye image 9 from right are displayed on the screen of the display 5.

  Further, the diagnosis support apparatus 1 performs image recording processing for recording the position display image 30 on the hard disk 4 in association with the current front image acquired by the front camera Cf at the current time ts. Thereby, the position display image 30 and the current front image recorded in the hard disk 4 can be displayed again on the screen of the display 5 at the time of diagnosis by recording.

  The diagnosis support apparatus 1 records the parameters necessary for creating the position display image 30 in the hard disk 4 in place of recording the position display image 30 itself in the image recording process, so that the position display image 30 is recorded. When the image is re-displayed, a position display image may be generated using the parameters and displayed on the screen of the display 5 together with the current front image. Here, as parameters necessary for creating the position display image, for example, the background image (entire version) at each time t, the position of the endoscope (relative position with respect to the background image), tip inclination data (by operator operation) Bending angle), and operation information of accessories.

  According to the diagnosis support apparatus and program of the present embodiment, a doctor who makes a diagnosis refers to a position display image when performing image diagnosis or the like using an image photographed by an endoscope. Since it is possible to recognize which part of the subject is captured by the image, diagnosis can be performed efficiently.

  In the above-described embodiment, each camera provided in the endoscope 10 captures a different field of view range, and the image of the image acquired by each camera is captured in a plane, on the plane. As described above, the position of the endoscope is acquired based on the positional deviation of the feature point of the endoscope 10, and the optical axis is arranged on the side surface of the distal end portion of the endoscope 10 as shown in FIGS. Are parallel and two cameras Cd1 and Cd2 (Cd1 'and Cd2') are installed side by side so that the overlapping of the field of view is large, stereo images acquired by these cameras By analyzing the image, the distance to each tissue site on the inner wall of the hollow organ and the inclination of the tissue surface can be derived, so that the positional relationship between the living tissue and the endoscope can be simulated more accurately. For example, the inclination of the distal end portion of the endoscope 10 can be added to the position display image by using the change in the inclination of the camera portion of the endoscope and the surface of the biological tissue. It is also possible to generate a three-dimensional background image by reconstructing the structure of the luminal organ three-dimensionally.

  In the above embodiment, the sensor further includes a sensor (detection means) for detecting the operation status of the accessory of the endoscope 10, and the accessory detected by the sensor in the position display image generation process in the diagnosis support apparatus 1 On the basis of the operation status, a position display image that further represents the state of the accessory at the present time may be generated on the background image. For example, as shown in FIG. 13, a probe 15 having an optical or magnetic scale (mark) is provided with a sensor 14a for detecting the scale. Is inserted into the body, the length a of the probe 15 protruding from the distal end portion of the endoscope is obtained from the insertion length of the probe 15 detected by the sensor 14a, and based on that, the distal end of the endoscope is obtained. The state of the probe 15 extended with the acquired length a is expressed by CG and displayed on the position display image. When inserting a probe with a scissor tip, install a sensor that detects the control amount of the operation part that opens and closes the scissors, so that the scissors can be opened and closed from the control amount of the operation part. The amount is detected, and the opening / closing state is expressed by CG and displayed on the position display image. FIG. 14 is a diagram showing an example of a position display image that further represents a state M of the probe having a scissor tip.

  In addition, a system that can measure the insertion length in real time by installing a scale (optical or magnetic) in the endoscope insertion section and installing a sensor that reads the scale in the endoscope insertion section is possible. In this case, when creating the background image, the reference image may be selected for each predetermined insertion length interval. Display image data can also be created by associating the background image and each captured image with the insertion length data.

  Further, in the above-described embodiment, the case where the branch point of the blood vessel pattern is detected as the feature point Qs in the feature point detection processing has been described as an example. However, the present invention is not limited to this. Change of taste), shadow pattern due to surface irregularities, or pattern pattern of staining result with endoscopic examination medicine (JP 2008-273900) etc., so as to detect characteristic points (feature points) Also good. In the feature point detection process, the detection performance can be improved by applying various image processing techniques such as differentiation, binarization, and density range normalization.

  In the above-described embodiment, in the feature point detection process, the feature points existing in both the current image and one of two or more reference images that are images constituting the background image are converted into the current image and the reference image. Although the case of detecting in each case has been described, feature points existing in both the current image and the background image may be detected in each of the current image and the background image. In that case, in the endoscope position acquisition process performed by the diagnosis support apparatus 1, the current position ts on the background image is based on the deviation of the position of the feature point on the current image from the position of the feature point on the background image. The position L (ts) of the endoscope is acquired.

1 Diagnosis support device 2 CPU
3 Memory 4 Hard disk 5 Display
10 Endoscope
12 Operation unit
14 Forceps opening
Cf front camera
Cu, Cd, Cr, Cl Side camera
B Background image
E Endoscope picture
P (ts) Current image
P (td) reference image
Qs, Qd feature points
15 Probe (accessory)

Claims (9)

A background image generation means for generating a background image by combining two or more images having regions overlapping with each other among images captured up to the present time by an endoscope;
A feature point present in both of the current image captured at the present time by the endoscope and one of two or more reference images that constitute the background image is represented by the current image and the one reference. Feature point detection means for detecting each of the images;
Based on the deviation of the position of the detected feature point on the current image from the position of the detected feature point on the reference image, the current position of the endoscope on the background image Endoscope position acquisition means for acquiring a position;
Position display image generation means for generating a position display image representing the position of the endoscope acquired by the endoscope position acquisition means on the background image generated by the background image generation means;
A diagnostic support apparatus, comprising: an image display unit configured to display the position display image generated by the position display image generation unit on a screen together with the image captured at the present time by the endoscope.   The position acquisition means acquires a shift in the position of the endoscope at the current time from the position of the endoscope at the time of shooting the one reference image based on the shift in the position of the feature point. The diagnosis support apparatus according to claim 1, wherein the position of the endoscope is acquired.   The position display image generating unit generates the position display image by combining an endoscope picture with the acquired endoscope position on the background image. 3. The diagnosis support apparatus according to 1 or 2. Comprising detection means for detecting the operation status of the accessory of the endoscope,
The position display image generating means generates the position display image further representing the state of the accessory at the present time on the background image based on the detected operation state of the accessory. The diagnosis support apparatus according to any one of claims 1 to 3. Recording means for recording the generated position display image in association with the current image;
The image display means further has a function of displaying a position display image recorded in the recording means on the screen together with an image photographed at the present time by the endoscope. The diagnosis support apparatus according to any one of 1 to 4. Recording means for recording the generated background image and the acquired position of the endoscope in association with the image captured at the present time by the endoscope;
2. The position display image generating means further has a function of generating the position display image using a background image recorded in the recording means and a position of an endoscope. 5. The diagnosis support device according to any one of items 1 to 4. A step of generating a background image by synthesizing two or more images having a region where the images overlap among images captured up to the present time by an endoscope;
A feature point existing in both of the current image captured at the present time by the endoscope and one of two or more reference images that constitute the background image is defined as the current image and the one reference. Detecting in each of the images,
Based on the deviation of the position of the detected feature point on the current image from the position of the detected feature point on the reference image, the current position of the endoscope on the background image Obtaining a position;
Generating a position display image representing the position of the acquired endoscope on the generated background image;
And a method for displaying the generated position display image on the screen together with the image photographed at the present time by the endoscope.   The step of acquiring the position of the endoscope is based on the deviation of the position of the feature point, and the position of the endoscope at the current time from the position of the endoscope at the time of photographing the one reference image. The diagnosis support method according to claim 7, wherein the position of the endoscope is acquired by acquiring a deviation. Computer
A background image generation means for generating a background image by combining two or more images having regions overlapping with each other among images captured up to the present time by an endoscope;
A feature point present in both of the current image captured at the present time by the endoscope and one of two or more reference images that constitute the background image is represented by the current image and the one reference. Feature point detection means for detecting each of the images;
Based on the deviation of the position of the detected feature point on the current image from the position of the detected feature point on the reference image, the current position of the endoscope on the background image Endoscope position acquisition means for acquiring a position;
Position display image generation means for generating a position display image representing the position of the endoscope acquired by the endoscope position acquisition means on the background image generated by the background image generation means;
A diagnosis support program for causing a position display image generated by the position display image generation means to function as an image display means for displaying on a screen together with the image photographed at the present time by the endoscope.