JP4624802B2 - Endoscope apparatus and program thereof - Google Patents

Description

  The present invention relates to an endoscope apparatus in which an imaging element is provided in a distal end portion of an insertion portion, and the distal end portion is curved to control an imaging direction.

2. Description of the Related Art Conventionally, there is an endoscope apparatus that includes an imaging element in a distal end portion of an insertion portion.
For example, Patent Document 1 describes an endoscope apparatus that includes a plurality of imaging elements in a distal end portion of an insertion portion and can obtain a continuous panoramic image with a wide field of view.

In addition, there is an endoscope apparatus that includes an imaging element in a distal end portion of an insertion portion, and that the distal end portion is curved to control an imaging direction. Some of such endoscope apparatuses can detect a moving object that has entered an imaging range during imaging by a background difference method. In addition, the moving object detection by the background difference method registers a background image in the imaging range in advance, takes the difference between the background image and the moving object image that has entered the imaging range during imaging, and detects the difference as a moving object. It is a method. For example, Patent Document 2 describes an image processing apparatus that detects a moving object using this background difference method. Patent Document 3 describes a moving body tracking device that automatically detects a moving body in a captured image and automatically tracks the moving body using a background difference method.
JP 2000-325306 PR JP 2004-23373 PR JP 2001-285695 A

  By the way, in the above-described endoscope apparatus capable of detecting a moving body, it is assumed that the bending position of the distal end portion of the insertion portion is used in a fixed state. For example, when it is desired to track the tip of the moving body in accordance with the movement of the moving body, the application is difficult. In addition, when moving object detection is performed by the background difference method, it is necessary to re-register the background image when the distal end portion of the insertion portion is curved, but it is extremely inefficient to register the background image every time the distal end portion is curved, Further, since the moving object enters the imaging range during the moving object detection, it is technically difficult to extract only the background.

In view of the above circumstances, an object of the present invention is to provide an apparatus and a program capable of detecting a moving object without re-capturing and registering a background image even when the distal end portion of the insertion portion is curved.

In order to achieve the above object, an endoscope apparatus according to a first aspect of the present invention includes an imaging unit that acquires a captured image, a bending control unit that curves a distal end portion of an endoscope insertion portion , and the distal end portion. A curved position acquisition unit that acquires the position of the tip in a curved state, a panoramic image generation unit that combines the plurality of captured images acquired by the imaging unit to create a panoramic image, and the curved position A predetermined region centered on a point obtained by linearly interpolating imaging center points of the plurality of captured images in the panoramic image created by the panoramic image creation unit corresponding to the position of the tip acquired by the obtaining unit ; a reference image extracting unit for extracting as a reference image, the reference image extracted by the reference image extracting unit, by the imaging means at a position of the tip portion when the reference image is extracted The difference between the obtained captured image and a moving object detection means for detecting a moving object.

  The present invention is not limited to these apparatuses, and can be configured as a method and program for detecting moving objects.

  According to the present invention, even when the distal end of the insertion section is curved, for example, when the distal end of the insertion section is tracked in accordance with the movement of a moving body that has entered the imaging range, the moving body detection is performed without capturing and registering a background image again. It can be performed.

  Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a configuration diagram of an endoscope system including an endoscope apparatus according to Embodiment 1 of the present invention.
In this figure, this system has, as main components, an endoscope unit 2 to which a scope (endoscope insertion portion) 1 provided with an image sensor for imaging a target object in a distal end portion is attached, and an endoscope unit 2. A camera control unit (hereinafter referred to as “CCU”) 3 that converts a signal from the video signal into a video signal, and a remote controller 4 that includes a plurality of switches and enables control according to switch operation (for example, bending control of the scope tip). The endoscope apparatus 5 that controls the operation of the entire system, the video according to the video signal from the endoscope apparatus 5 (such as an image captured by the scope 1), and the information on the moving object detected by the operation described later And a video output device 6 that outputs (displays) and the like.

The endoscope apparatus 5 includes a system control unit 7, a video signal processing circuit 8, and an RS-232C interface 9.
The system control unit 7 includes a ROM 10 in which a control program for controlling the operation of the entire system including the apparatus 5 is recorded, and the entire system including the apparatus 5 by reading and executing the control program. CPU 11 for controlling the operation of the CPU 11, and a RAM 12 for temporarily storing a control program executed by the CPU 11 and securing a work area for the CPU 11. The bending control of the scope tip is performed according to the switch operation of the remote controller 4 or by a control program executed by the CPU 11. That is, the bending control of the scope tip can be performed manually or automatically.

  The video signal processing circuit 8 has an interface function of performing predetermined signal processing on the video signal from the CCU 3 and outputting it to the video output device 7. The RS-232C interface 9 is an interface for enabling transmission / reception of signals between the apparatus 5 and an external device (in the present embodiment, the endoscope unit 2, the CCU 3, and the remote controller 4). Here, the remote controller 4 and the RS-232C interface 9 are connected by, for example, a cable.

  In this endoscope apparatus 5, the ROM 10, CPU 11, RAM 12, video signal processing circuit 8, and RS-232C interface 9 are each connected to the bus 13, so that signals (data) can be transmitted / received to / from each other. Yes.

Next, the moving body detection operation performed by the endoscope apparatus 5 of the present system will be described with reference to FIGS.
FIG. 2 is a functional block diagram of the endoscope apparatus 5 according to the moving object detection operation according to the present embodiment. 3 to 5 are diagrams for explaining a process of imaging a plurality of backgrounds. FIG. 6 is a diagram for explaining the outline of the planar projective transformation. FIGS. 7 to 10 are diagrams for explaining processing for creating a single wide-field panoramic image by pasting together a plurality of stored background images. FIGS. 11 to 12 are diagrams for explaining processing from extracting a reference image from a panoramic image to performing moving object detection.

  First, as shown in FIG. 2, in this operation, first, in order to obtain a plurality of background images BK necessary for creating a panoramic image that covers the curved range of the scope tip, the block BL01 is obtained by the imaging device of the scope 1. A process of imaging a plurality of backgrounds is performed. Specifically, as shown in FIG. 3, first, the scope distal end is curved to the scope curved position (curved position of the scope distal end) SP1, and the background image BK1 obtained by imaging the background 16 within the imaging range R1. Save. Subsequently, the scope distal end is bent to the scope bending position SP2, and the background image BK2 obtained by imaging the background 16 in the imaging range R2 is stored. At this time, an overlapping portion (colored portion 14 in the figure) is provided in a predetermined range between the imaging ranges R1 and R2. Subsequently, the scope distal end is curved to the scope range SP3, and the background image BK3 obtained by imaging the background 16 in the imaging range R3 is stored. Also at this time, an overlapping portion (colored portion 15 in the figure) is provided in a predetermined range between the imaging ranges R2 and R3. Similarly, as shown in FIG. 4, the scope tip end portion is sequentially curved so that an overlapping portion is provided in a predetermined range between adjacent imaging ranges, and the background covered by the scope tip end curvature range is covered. Imaging is performed for all 16 ranges.

  FIG. 5 is a diagram showing a flowchart relating to processing performed by such a block BL01, that is, processing for imaging a plurality of backgrounds. As shown in the figure, in this process, first, n = 1 is set (step (hereinafter simply referred to as “S”) 1), and then the scope distal end is bent to the scope bending position SPn (S2). 16 is imaged within the imaging range Rn (S3), and the obtained background image BKn is stored (S4). However, in S3, when n ≧ 2, the background 16 is imaged in the imaging range Rn so that the adjacent imaging ranges have an overlapping portion in a predetermined range. Subsequently, it is determined whether or not the background image has been saved at all necessary scope bending positions (S5). If the determination result is Yes, this flow is terminated, and if it is No, n is incremented. Then, the process returns to S2, and the above-described processing is repeated for the next scope bending position SPn.

  When the background images obtained at all necessary scope bending positions are stored in this way, the block BL02 shown in FIG. 2 combines the plurality of stored background images, and the block BL03 displays the combined background image. One wide-field panoramic image is created from the image.

  Here, an example of a technique for creating a single wide-field panoramic image by pasting a plurality of images will be described. In general, geometric transformation between images is used for combining images. There are various geometric transformations. Here, a case where planar projection transformation is used will be described as an example. Even if the target background is a distant view or a close view, when only a planar object is shown, planar projective transformation is established between the respective background images. An outline of the planar projective transformation will be described with reference to FIG. As shown in the figure, when the point M on the plane P is observed from two different viewpoints VP1, VP2, the coordinate transformation between the points IP1, IP2 on these image planes S1, S2 is linear, This is called planar projective transformation. A plane projective transformation formula between the point IP1 (x1, y1, 1) on the image plane S1 and the point IP2 (x2, y2, 1) on the image plane S2 is defined by the following formula (1). .

However, k is a non-zero number.
Once this conversion matrix is obtained, the image taken from the viewpoint VP1 can be converted in image coordinates to create an image as if viewed from another viewpoint VP2.

  How to calculate this transformation matrix is a problem for image pasting, but by finding a predetermined number or more of feature points (corresponding points) that can be associated with each other with respect to overlapping portions between images. This conversion matrix can be calculated.

  In this embodiment, this conversion matrix is calculated as follows. First, as shown in FIG. 7, an image captured in a state where the scope tip is not curved is selected as a reference image from a plurality of stored background images, and this is used as a first image 17. Note that all the images are combined as an observation image observed from the viewpoint of the reference image. Subsequently, as illustrated in FIG. 8, a background image that overlaps with the first image 17 is selected from the stored background images, and this is set as the second image 18. Then, as shown in FIG. 9, matching is performed between the first image 17 and the second image 18, and a predetermined number or more of feature points (black dots in the figure) are extracted in the overlapping portion 19, and the feature points By performing the association, the parameters m1 to m9 of the planar projective transformation of Expression (1) are obtained, and the transformation matrix can be calculated. Using this transformation matrix, the first image 17 and the second image 18 are pasted as observation images observed from the viewpoint of the reference image. When the pasting is completed, the pasted image 20 is set as a new first image. Hereinafter, by pasting the remaining background images in the same manner, one wide-view panoramic image observed from the viewpoint of the reference image can be created.

  FIG. 10 is a flowchart relating to processing performed by the blocks BL02 and BL03 shown in FIG. 2, that is, processing for creating a single wide-field panoramic image by pasting a plurality of stored background images. FIG. As shown in the figure, in this process, first, an image captured in a state where the scope tip is not curved is selected as a reference image from a plurality of stored background images (S11). Is the first image (S12). Subsequently, an image having an overlapping portion with the first image is selected from the stored background images, and is set as the second image (S13). Subsequently, matching is performed between the first image and the second image (S14), and a predetermined number or more of feature points are extracted in the overlapping portion between them (S15), and the features of the first image and the second image, respectively. A transformation matrix is calculated from the position of the point (S16). That is, by associating the feature points in the overlapping portion, the parameters m1 to m9 of the planar projective transformation of Expression (1) are obtained to calculate the transformation matrix. Subsequently, using the transformation matrix, the first image and the second image are pasted as an observation image observed from the viewpoint of the reference image (S17). When the pasting is completed, it is then determined whether or not the pasting of all the saved background images has been completed (S18). If the result of the determination is Yes, the creation of the panoramic image is completed. (S19), this flow ends. On the other hand, when S18 is No, the image pasted in S17 is set as a new first image, the process proceeds to S13, and the above-described processing is repeated for the remaining background images.

  In this way, when the creation of the panoramic image is completed, the moving object can be detected. When the moving object detection is started, the block BL06 shown in FIG. 2 acquires the current scope curved position (assumed to be SPc), and the block BL04 is a panoramic image based on the scope curved position SPc. A reference image is extracted from.

  Here, an example of a technique for extracting a reference image from the panoramic image will be described with reference to FIG. As shown in the figure, in the panoramic image PI obtained by pasting a plurality of background images, the imaging center points in the respective background images are designated as C1 to C9 (black circles in the figure), and two points C4 and C5 are included. The corresponding scope bending positions are SP4 and SP5. In this case, for example, when the current scope bending position SPc is exactly in the middle of SP4 and SP5, a predetermined region 21 centering on the center point CM (white circle in the figure) of C4 and C5 is extracted as a reference image. That is, a predetermined area centered on a point obtained by linear interpolation of the imaging center point in the panoramic image corresponding to the scope bending position is extracted as a reference image. It is desirable that the reference image extracted at this time be slightly larger than the imaging range of the scope 1.

  When the reference image is extracted from the panoramic image based on the scope bending position SPc in this way, the block BL05 shown in FIG. 2 corrects the reference image by plane projective transformation. Specifically, planar projective transformation is performed so that the extracted reference image is an image observed from the viewpoint of the scope bending position SPc. When this is finished, the block BL07 captures an image at the scope curved position SPc to obtain a captured image, and the block BL08 captures the reference image corrected (planar projective transformation) by the block BL05 obtained by the block BL07. However, the size, position, and distortion of the reference image are corrected by matching so as to match the (captured image when the moving object is not captured). More specifically, the correction by matching performed in the block BL08 is, for example, nine regions (in a grid pattern) for matching in an image to be corrected (a reference image corrected in the block BL05 in this embodiment). Regions) are determined in advance and compared with images to be compared in each region (captured images obtained in the block BL07 in this embodiment), and the matching results derived as a result (region matching degree, deviation) The image is corrected (distortion, enlargement / reduction, etc.) using the averaged data as matching data. That is, correction is performed so that the image to be corrected matches the image to be compared when it is assumed that the moving image is not included in the captured image.

  However, since correction by matching is performed and moving object detection is performed, moving object detection is performed when the background of the image to which correction by matching is appropriately performed is shown in the captured image, that is, basically in the captured image. This should be done only when the moving object is relatively small. If there is a moving object that covers the entire captured image, the background itself will hardly appear, so correction by matching is performed. Avoid moving object detection. Alternatively, in this case, the entire captured image may be regarded as a moving object.

  When correction by matching in block BL08 is completed in this way, block BL09 takes the difference between the reference image corrected by block BL08 and the captured image obtained by block BL07, and block BL10 Is detected as a moving object.

  FIG. 12 is a diagram showing a flowchart relating to processing performed by the blocks BL04 to BL10 shown in FIG. 2, that is, processing for actually detecting a moving object. As shown in the figure, in this process, first, the current scope curved position SPc is read (S31), a reference image is extracted from the panoramic image based on the scope curved position SPc (S32), and the reference image is extracted. Correction is performed by plane projective transformation (S33). Subsequently, a captured image is acquired by imaging at the scope bending position SPc (S34), the reference image corrected in S33 is corrected by matching with the captured image acquired in S34 (S35), and acquired in S34. The difference between the captured image and the reference image corrected in S35 is taken (S36), and the difference is detected as a moving object (S37). Subsequently, it is determined whether or not the moving object detection operation is to be ended (S38). If the determination result is Yes, the present flow is ended. If the determination result is No, the process returns to S31 and the above-described processing is repeated.

  As described above, according to the present embodiment, a panoramic image created from a plurality of background images obtained by imaging a background can be changed in viewpoint, enlarged / small, and rotated by plane projective transformation. Even if the scope state (curved position, rotation angle, zoom state, etc.) being detected is different from that during background imaging, an appropriate reference image can be extracted according to the scope state. Therefore, by creating a panoramic image in advance before moving object detection in this way, even when the scope tip is curved, it is possible to detect a moving object without capturing and registering a background image again. A flexible moving object detection that is not affected is possible.

  In the above-described first embodiment, the planar projection conversion is performed so that the reference image becomes an image observed from the viewpoint of the captured image, and the converted reference image is corrected by matching with the captured image, and the correction is performed. The moving object is detected by taking the difference between the later reference image and the captured image. However, in this embodiment, the moving image is detected so that the captured image becomes an image observed from the viewpoint of the reference image. Projective conversion is performed, the captured image after the conversion is corrected by matching with a reference image, and a moving object is detected by taking a difference between the corrected captured image and the reference image.

  FIG. 13 is a functional block diagram of the endoscope apparatus 5 according to the moving object detection operation according to the present embodiment. In the figure, blocks BL01 to BL04, blocks BL06 to BL07, and block BL10 are the same as those shown in FIG.

  The block BL11 corrects the captured image acquired in the block BL07 by plane projective transformation. That is, planar projection conversion of the captured image is performed based on the scope curved position corresponding to the viewpoint of the reference image so that the captured image becomes an image observed from the viewpoint of the reference image extracted in the block BL04. In the block BL12, the size, position, and distortion of the captured image are corrected so that the captured image corrected in the block BL11 (provided that the moving object is not captured) matches the reference image. Correct by matching. The block BL13 takes a difference between the captured image corrected by the block BL12 and the reference image extracted by the block BL04, and the block BL10 detects the difference as a moving object.

  FIG. 14 is a diagram showing a flowchart related to the processing performed by the blocks BL04, BL06 to BL07, BL11 to BL13, and the block BL10 shown in FIG. 13, that is, the processing for actually detecting a moving object. In the figure, in S41 to S42, the same processing as S31 to S32 in FIG. 12 is performed. Subsequently, a captured image is obtained by imaging at the scope bending position SPc (S43), the captured image is corrected by plane projective transformation (S44), and the captured image corrected in S44 is extracted in S42. Correction is performed by matching with the image (S45), and the difference between the corrected captured image and the reference image extracted in S42 is calculated (S46). In subsequent S47 to 48, the same processing as S37 to 38 in FIG. 12 is performed.

  As described above, this embodiment can provide the same effects as those of the first embodiment.

  In the present embodiment, after the moving object detection in the first embodiment is performed, the position of the detected moving object is further acquired, and the distal end of the scope is bent so that the position becomes the imaging center of the scope 1. The scope tip is curved and tracked according to the movement.

  FIG. 15 is a functional block diagram of the endoscope apparatus 5 according to the moving object detection operation according to the present embodiment. As shown in the figure, blocks BL14 and BL15 are further added to the configuration shown in FIG. The block BL14 acquires the position of the moving object detected in the block BL10 in the panoramic image created in the block BL03. The block BL15 curves the scope tip so that the imaging center of the scope 1 is the position of the moving object in the panoramic image acquired in the block BL14. Thereby, the scope tip can be tracked by the moving object.

  FIGS. 16, 17, and 18 are diagrams for explaining an example of a technique for tracking the distal end portion of the scope to the moving object. First, as shown in FIG. 16, the moving object 23 is detected by taking the difference between the captured image 22 obtained in the block BL07 and the reference image corrected in the block BL08, and the reference corrected in the block BL08. The position MP1 of the moving object 23 in the reference image 24 extracted from the panoramic image PI in the block BL04, which is the source of the image, is acquired. And the moving body position MP2 in the panoramic image PI corresponding to the moving body position MP1 is acquired. The moving object position MP2 can be calculated by storing which region of the panoramic image PI is extracted as a reference image. Subsequently, as shown in FIG. 17, the scope curve positions corresponding to the imaging center points C2, C3, C5, and C6 at the time of background imaging (when acquiring the background image) are in the vicinity of the moving object position MP2 in the panoramic image PI. Let SP2, SP3, SP5, and SP6. Then, as shown in FIG. 18, the relative positional relationship of the moving object position MP2 with respect to the imaging center points C2, C3, C5, and C6 is obtained by linear interpolation, and the scope bending position SPM corresponding to the moving object position MP2 is determined as the scope bending. The positions SP2, SP3, SP5, and SP6 are calculated by linear interpolation, and the scope tip is bent to the scope bending position SPM. Thereby, the imaging center of the scope 1 can be moved to the moving object position MP2, and the distal end of the scope can be tracked by the moving object 23.

  FIG. 19 is a diagram illustrating a flowchart relating to a process of tracking such a distal end portion of the scope with a moving object. In this process, first, when a moving object is detected as described in the first embodiment (S51), the position of the moving object in the reference image extracted from the panoramic image is acquired in the block BL04 used for detecting the moving object. (S52). Subsequently, the position of the moving object in the panoramic image is acquired (S53), the corresponding scope bending position is calculated from the moving object position in the panoramic image (S54), and the scope tip is placed at the calculated scope bending position. Curve. (S55). Subsequently, it is determined whether or not the moving object detection operation has ended (S56). If the determination result is Yes, this flow ends. If the determination result is No, the flow returns to S51 to repeat the above-described processing.

  As described above, according to the present embodiment, it is possible to further track the distal end portion of the scope to the moving body in accordance with the movement of the moving body.

  In the present embodiment, after the moving object detection in the second embodiment is performed, the position of the detected moving object is further acquired, and the distal end of the scope is bent so that the position becomes the imaging center of the scope 1. The scope tip is curved and tracked according to the movement.

  FIG. 20 is a functional block diagram of the endoscope apparatus 5 according to the moving object detection operation according to the present embodiment. As shown in the figure, blocks BL14 and BL15 are further added to the configuration shown in FIG. Blocks BL14 and 15 are as described in the third embodiment. Further, the process of tracking the distal end of the scope to the moving object is as described in the third embodiment.

As described above, this embodiment can provide the same effects as those of the third embodiment.
Although the present invention has been described in detail above, the present invention is not limited to the above-described embodiment, and it is needless to say that various improvements and changes may be made without departing from the gist of the present invention.

1 is a configuration diagram of an endoscope system including an endoscope apparatus according to Embodiment 1. FIG. 1 is a functional block diagram of an endoscope apparatus according to a moving object detection operation according to Embodiment 1. FIG. It is a 1st figure for demonstrating the process which images two or more backgrounds. It is a 2nd figure for demonstrating the process which images two or more backgrounds. It is a figure which shows the flowchart which concerns on the process which images two or more backgrounds. It is a figure for demonstrating the outline | summary of planar projection conversion. It is the 1st figure for explaining processing which pastes up a plurality of background images and creates one panoramic image of a wide field of view. It is the 2nd figure for explaining processing which pastes up a plurality of background images and creates one panoramic image of a wide field of view. FIG. 11 is a third diagram for explaining a process of creating a single wide-field panoramic image by pasting a plurality of background images. It is a figure which shows the flowchart which concerns on the process which pastes together several background images and produces | generates one sheet of wide-field panoramic images. It is a figure for demonstrating the method of extracting a reference image from a panoramic image. It is a figure which shows the flowchart which concerns on the process which actually detects a moving body based on Example 1. FIG. 10 is a functional block diagram of an endoscope apparatus related to a moving object detection operation according to Embodiment 2. FIG. It is a figure which shows the flowchart which concerns on the process which actually detects a moving body based on Example 2. FIG. FIG. 10 is a functional block diagram of an endoscope apparatus according to a moving object detection operation according to a third embodiment. It is a 1st figure for demonstrating an example of the method of tracking a scope tip to a moving body. It is the 2nd figure for demonstrating an example of the method of tracking a scope tip to a moving body. It is a 3rd figure for demonstrating an example of the method of tracking the scope front-end | tip to a moving body. It is a figure which shows the flowchart which concerns on the process which tracks a scope tip to a moving body. FIG. 10 is a functional block diagram of an endoscope apparatus related to a moving object detection operation according to a fourth embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Scope 2 Endoscope unit 3 Camera control unit 4 Remote control 5 Endoscope apparatus 6 Video output device 7 System control part 8 Video signal processing circuit 9 RS-232C interface 10 ROM
11 CPU
12 RAM
13 Buses 14 and 15 Overlapping part 16 Background 17 First image 18 Second image 19 Overlapping part 20 Pasted image 21 Predetermined area 22 Captured image 23 Moving object 24 Reference image

Claims (6)

An imaging means for acquiring a captured image;
Bending control means for bending the distal end portion of the endoscope insertion portion;
Curved position acquisition means for acquiring the position of the tip in a state where the tip is curved ;
Panorama image creation means for creating a panoramic image by pasting together a plurality of the captured images acquired by the imaging means;
Centered on a point obtained by linearly interpolating the imaging center points of the plurality of captured images in the panoramic image created by the panoramic image creation unit corresponding to the position of the tip acquired by the curved position acquisition unit. Reference image extraction means for extracting a predetermined area as a reference image;
Moving object detection means for detecting, as a moving object, a difference between the reference image extracted by the reference image extraction means and the captured image acquired by the imaging means at the position of the tip when the reference image is extracted. An endoscope apparatus comprising the endoscope apparatus. The panoramic image creation unit is configured to paste a plurality of the captured images obtained by the imaging unit at a plurality of different positions on the tip , and having an overlapping portion between the captured images. The endoscope apparatus according to claim 1, wherein the panoramic image is created. The moving object detection means includes
Conversion means for planar projective transformation of the reference image so that the reference image extracted by the reference image extraction means becomes an image observed from the viewpoint of the position of the tip when the reference image is extracted;
The plane projective transformation is performed so that the reference image obtained by plane projection conversion by the conversion unit matches the captured image acquired by the imaging unit at the position of the tip when the reference image is extracted. Correction means for correcting the reference image;
Have
The difference between the reference image corrected by the correction unit and the captured image acquired by the imaging unit at the position of the tip when the reference image is extracted is detected as a moving object. The endoscope apparatus according to 1 or 2. The moving object detection means includes
The captured image acquired by the imaging unit at the position of the tip when the reference image is extracted by the reference image extracting unit is an image observed from the same viewpoint as the viewpoint of the reference image. Conversion means for plane projective conversion of the captured image;
Correction means for correcting the captured image that has undergone planar projection conversion so that the reference image and the captured image that has undergone planar projection conversion by the converting means match;
Have
The endoscope apparatus according to claim 1, wherein a difference between the reference image and the captured image corrected by the correction unit is detected as a moving object. Moving object position calculating means for calculating the position of the moving object detected by the moving object detecting means in the panoramic image;
Depending on the position calculated by the moving object position calculating means, and the corresponding curved position calculating means for calculating the position of the tip portion,
Further comprising
The bending control means causes the tip to track the moving body by bending the tip to the position of the tip calculated by the bending position calculation means. The endoscope apparatus according to claim 1. An image acquisition function for acquiring a captured image obtained by the imaging means;
A bending control function for bending the distal end portion of the endoscope insertion portion;
A curved position acquisition function for acquiring the position of the tip in a state where the tip is curved ;
A panoramic image creation function for creating a panoramic image by pasting together a plurality of the captured images acquired by the image acquisition function;
Centered on a point obtained by linearly interpolating the imaging center points of the plurality of captured images in the panoramic image created by the panoramic image creation function corresponding to the position of the tip obtained by the curved position obtaining function. A reference image extraction function for extracting a predetermined area as a reference image;
A moving object detecting function for detecting, as a moving object, a difference between the reference image extracted by the reference image extracting function and a captured image captured at the position of the tip when the reference image is extracted. program.