JP2004358095A - Apparatus and method for analyzing endoscope inserting form - Google Patents

Apparatus and method for analyzing endoscope inserting form Download PDF

Info

Publication number
JP2004358095A
JP2004358095A JP2003162844A JP2003162844A JP2004358095A JP 2004358095 A JP2004358095 A JP 2004358095A JP 2003162844 A JP2003162844 A JP 2003162844A JP 2003162844 A JP2003162844 A JP 2003162844A JP 2004358095 A JP2004358095 A JP 2004358095A
Authority
JP
Japan
Prior art keywords
shape
endoscope
insertion
step
insertion portion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2003162844A
Other languages
Japanese (ja)
Other versions
JP2004358095A5 (en
JP4274854B2 (en
Inventor
Katsumi Hirakawa
克己 平川
Original Assignee
Olympus Corp
オリンパス株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Olympus Corp, オリンパス株式会社 filed Critical Olympus Corp
Priority to JP2003162844A priority Critical patent/JP4274854B2/en
Priority claimed from EP03809860A external-priority patent/EP1504712B1/en
Publication of JP2004358095A publication Critical patent/JP2004358095A/en
Publication of JP2004358095A5 publication Critical patent/JP2004358095A5/ja
Application granted granted Critical
Publication of JP4274854B2 publication Critical patent/JP4274854B2/en
Application status is Active legal-status Critical
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Detecting, measuring or recording for diagnostic purposes; Identification of persons
    • A61B5/06Devices, other than using radiation, for detecting or locating foreign bodies ; determining position of probes within or on the body of the patient
    • A61B5/061Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body
    • A61B5/062Determining position of a probe within the body employing means separate from the probe, e.g. sensing internal probe position employing impedance electrodes on the surface of the body using magnetic field
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002Operational features of endoscopes
    • A61B1/00004Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009Operational features of endoscopes characterised by electronic signal processing of image signals
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002Operational features of endoscopes
    • A61B1/00043Operational features of endoscopes provided with signal output arrangements
    • A61B1/00045Display arrangement
    • A61B1/0005Display arrangement for multiple images
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002Operational features of endoscopes
    • A61B1/00043Operational features of endoscopes provided with signal output arrangements
    • A61B1/00055Operational features of endoscopes provided with signal output arrangements for alerting the user
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/005Flexible endoscopes
    • A61B1/0051Flexible endoscopes with controlled bending of insertion part

Abstract

An object of the present invention is to issue a warning when an insertion section forms a loop in an object during an endoscopic examination or when an observation site of the object extends, and to provide shape information, an insertion method, and an operation according to the insertion operation. There is a demand for an endoscope image processing device that displays instruction information together.
An electronic endoscope having an insertion portion inserted into a body cavity, a shape observation device for observing and detecting the shape of the electronic endoscope insertion portion, and an endoscope insertion portion detected by the shape observation device A plurality of shapes of the endoscope insertion portion based on the results of the analysis by the shape analysis means for analyzing the shape of the endoscope insertion portion based on the stored shapes of the plurality of endoscope insertion portions. An endoscope insertion shape analysis device including a PC 15 having information providing means for providing information on the shape of the endoscope.
[Selection diagram] Fig. 1

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an endoscope insertion shape analysis apparatus that analyzes an insertion shape of an endoscope insertion section in a body cavity and displays the insertion shape information as an image to improve the operability of inserting the endoscope.
[0002]
[Prior art]
In general, an electronic endoscope has an elongated and flexible insertion portion provided with an electronic imaging element at the tip, and this insertion portion is inserted from the outside into a lumen in a body cavity which is a subject, and the insertion portion has a distal end. Observing the inside of a lumen from an image reproduced on a monitor screen based on an image signal generated and generated by the electronic imaging device, and collecting a living tissue with a treatment forceps inserted from a forceps channel provided in an insertion portion And therapeutic treatments.
[0003]
By the way, when inserting and observing an endoscope in a curved lumen such as a large intestine or a small intestine, the position of the endoscope insertion portion is inserted into the lumen of the lumen. Or, if it becomes clear what shape the endoscope insertion portion has, the operability of the observation treatment by the endoscope will be improved.
[0004]
For this reason, there has been proposed an endoscope shape detection device that can use any lumen without harming the position at which the insertion portion of the endoscope is inserted into the lumen, the insertion shape, and the like to a human body as a subject. .
[0005]
For example, a source coil composed of a plurality of first coils, a sense coil composed of a plurality of second coils for detecting magnetic fields from the plurality of first coils of the source coil, and any one of the source coil and the sense coil And a shape estimating means for estimating the shape of the endoscope insertion portion based on the magnetic field of the source coil detected by the sense coil. There is an endoscope shape detecting device that detects a loop shape from the insertion portion shape estimated by the shape estimating means and generates a warning when the loop shape is detected. reference).
[0006]
[Patent Document 1]
JP-A-2000-175861 (columns 0011 to 0049, FIGS. 1 to 10).
[0007]
[Problems to be solved by the invention]
Conventionally, the operator's consciousness in the examination of a subject by an endoscope is mainly focused on an endoscope image generated by imaging a site to be observed in a lumen, and a shape detection device of an endoscope insertion unit is used as necessary. In general, a style in which an insertion part shape image generated and displayed is viewed.
[0008]
In this endoscopy, when the shape of the insertion portion forms a loop, an endoscope having an endoscope insertion support function that detects that the loop has been formed, issues a warning, and alerts the operator. A mirror shape detecting device has been proposed in Patent Document 1 mentioned above.
[0009]
Further, in order to improve the insertability, it is desired to provide information according to the movement of the endoscope insertion portion to be actually inserted, or to provide information indicating an endoscope insertion method, an operation instruction, and the like. ing.
[0010]
The present invention has been made in view of such a demand, and analyzes an endoscope insertion section shape, and provides an endoscope insertion shape analysis capable of providing information leading to an improvement in endoscope insertability. It is intended to provide a device.
[0011]
[Means for Solving the Problems]
An endoscope insertion shape analyzing apparatus according to the present invention includes an endoscope having an insertion portion to be inserted into a body cavity, a shape detection means for detecting a shape of the endoscope insertion portion, and an inner shape detected by the shape detection means. Shape storage means for storing a plurality of shapes of the endoscope insertion portion, and shape analysis means for analyzing the shape of the endoscope insertion portion based on the shapes of the plurality of endoscope insertion portions stored in the shape storage means. And information providing means for providing information on the shape of the endoscope insertion section in accordance with the result of the analysis by the shape analyzing means.
[0012]
An endoscope insertion shape analysis device of the present invention includes an endoscope having an insertion portion to be inserted into a body cavity, shape detection means for detecting the shape of the endoscope insertion portion, and detection by the shape detection means. It is characterized by comprising a shape analyzing means for analyzing a shape of the endoscope insertion portion, and an information providing means for providing information on an endoscope operation in accordance with a result of the analysis by the shape analyzing means.
[0013]
An endoscope insertion shape analysis method according to the present invention includes an endoscope having an insertion portion to be inserted into a body cavity, a shape detection step of detecting a shape of the endoscope insertion portion, and a shape detection step. A shape storage step of storing a plurality of shapes of the endoscope insertion section; and a shape analysis for analyzing a shape of the endoscope insertion section based on the shapes of the plurality of endoscope insertion sections stored in the shape storage step. And an information providing step of providing information on the shape of the endoscope insertion section in accordance with a result of the analysis by the shape analysis step.
[0014]
An endoscope insertion shape analysis method according to the present invention includes an endoscope having an insertion portion to be inserted into a body cavity, a shape detection step of detecting a shape of the endoscope insertion portion, and a shape detection step. It is characterized by comprising a shape analyzing step of analyzing the shape of the endoscope insertion section, and an information providing step of providing information on an endoscope operation in accordance with a result of the analysis by the shape analyzing step.
[0015]
An endoscope insertion shape analysis apparatus and an endoscope insertion shape analysis method of the present invention are configured to display relevant information of an analysis result when an insertion shape of an endoscope insertion portion is analyzed, thereby providing an operator as a doctor. Can accurately grasp the insertion shape of the endoscope and provide operation information, thereby improving the efficiency of the examination with the endoscope and greatly reducing discomfort to the patient.
[0016]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. A first embodiment of the endoscope insertion shape analyzing apparatus according to the present invention will be described with reference to FIGS.
[0017]
FIG. 1 is a block diagram showing the overall configuration of an electronic endoscope system using an image processing device for performing an endoscope insertion shape analysis according to the present invention, and FIG. 2 is an image processing device of the electronic endoscope system according to the present invention. FIG. 3 is an explanatory diagram illustrating an insertion position coordinate system in FIG. 3, and FIG. 3 is an explanatory diagram illustrating a data structure of insertion position detection data generated by the endoscope insertion shape observation device of the electronic endoscope system according to the present invention. 5 is a flowchart for explaining a processing operation of an endoscope image, insertion shape data, and the like according to the first embodiment of the image processing device of the electronic endoscope system according to the present invention, and FIG. 5 is an electronic endoscope according to the present invention. FIG. 6 is a flowchart for explaining an extension detection operation of a subject, which is the first embodiment of the image processing apparatus of the system. FIG. 6 is an image processing apparatus of the electronic endoscope system according to the present invention. It is an explanatory view illustrating a display screen that.
[0018]
First, an electronic endoscope system 1 using an image processing device for analyzing an endoscope insertion shape according to a first embodiment of the present invention will be described with reference to FIG.
[0019]
The electronic endoscope system 1 includes an endoscope device 2, an endoscope insertion shape observation device 3, and an image processing device 4.
[0020]
The endoscope device 2 includes an electronic endoscope 12, a video processor 10, a light source device 11, and an observation monitor 14.
[0021]
Although not shown, the electronic endoscope 12 is provided with an electronic image pickup device at a distal end of an elongated insertion portion to be inserted into a lumen of a body cavity which is a subject. In addition to generating and outputting an imaged video signal of the observation site in the inside, observation light for illuminating the observation site in the lumen is irradiated by a light guide provided in the insertion portion.
[0022]
Further, a bending portion is provided at a distal end portion of the insertion portion of the electronic endoscope 12, and a bending operation can be performed from an operation portion provided at a base end of the insertion portion.
[0023]
Furthermore, a release switch 12a is provided on the operation unit of the electronic endoscope 12, and a cable for controlling the drive of an electronic image pickup device with the video processor 10 and transmitting / receiving an image pickup image signal generated and imaged. And a light guide cable or the like for guiding observation light from the light source device 11 to the light guide.
[0024]
Although not shown, the electronic endoscope 12 is provided with a detection function for detecting the insertion position and the shape of the insertion section in the lumen. This insertion position and shape detection function are not shown, and a plurality of source coils provided at predetermined intervals in an insertion portion of the endoscope and a plurality of sense coils provided in the endoscope insertion shape observation device 3 are provided. It comprises a sense coil unit 19 having coils.
[0025]
The video processor 10 controls the driving of the electronic image pickup device of the electronic endoscope 12 and performs predetermined signal processing on a video signal of a moving image captured and generated by photoelectric conversion by the electronic image pickup device to generate a luminance signal and a color signal. , Or an RGB signal or the like.
[0026]
The Y / C signal or the RGB signal including the luminance signal and the chrominance signal generated by the video processor 10 is directly output to the observation monitor 14 and the image processing device 15.
[0027]
When the release switch 12a is operated, an instruction to output a still image of a captured image is possible.
[0028]
Although not shown, the video processor 10 has a function of inputting examination information relating to endoscopy.
[0029]
The light source device 11 includes a lamp as an illumination light source, a lighting circuit of the lamp, and the like, supplies illumination light projected when the lamp is turned on to the light guide of the electronic endoscope 12, and supplies a light from the distal end of the insertion portion to the lumen. Is projected to the observation site.
[0030]
The observation monitor 14 displays an endoscope image based on the Y / C signal or the RGB signal generated by the video processor 10.
[0031]
The endoscope insertion shape observation device 3 is a peripheral device of the endoscope device 2, and includes a sense coil unit 19 that detects a magnetic field from a source coil provided in the electronic endoscope 12, and a sense coil unit 19. A shape processing device 13 for estimating the shape of the endoscope insertion portion based on the detected magnetic field, and a monitor (display) 13b for displaying the shape of the endoscope insertion portion estimated by the shape processing device 13 I have.
[0032]
The shape processing device 13 outputs a drive signal for driving the source coil to the electronic endoscope 12 to generate a magnetic field in the source coil, and based on the detection signal from the sense coil unit 19 for detecting the magnetic field, The position coordinate data of the coil is calculated, and the shape of the endoscope insertion section is estimated from the calculated position coordinate data. In addition, it generates an insertion portion shape image signal for displaying the estimated shape of the endoscope insertion portion on the monitor 13b, and outputs three-dimensional coordinate information indicating the shape of the endoscope insertion portion to be output to the image processing device 4. And insert shape data such as a shape display attribute are generated.
[0033]
Note that the endoscope insertion shape observation device 3 does not show shape display attributes such as a rotation angle and an enlargement / reduction ratio of an insertion portion shape image which is generated by the shape processing device 13 and displayed on the monitor 13b. It can be changed by inputting an instruction from the operation panel.
[0034]
The inserted shape data generated by the shape processing device 13 can be output to the image processing device 4.
[0035]
The endoscope insertion shape observation device 3 continuously outputs the insertion shape data to the image processing device 4 during the endoscopic examination, and simultaneously operates the release switch 12 a provided in the electronic endoscope 12. It is also possible to output only the insertion shape data at the time of operation.
[0036]
The image processing apparatus 4 reproduces and displays a personal computer (hereinafter, simply referred to as a PC) 15, a mouse 16 and a keyboard 17 for inputting various instructions to the PC 15, and various information data and image information processed by the PC 15. Display 18.
[0037]
Further, the PC 15 has a communication port 15a for taking in the insertion shape data output from the communication port 13a of the shape processing device 13 of the endoscope insertion shape observation device 3, and a communication port of the video processor 10 of the endoscope device 2. A communication port 15b for receiving endoscopic inspection information output from the endoscope 10a; and a moving image input board 15c for converting a moving image signal generated by the video processor 10 of the endoscope 2 into predetermined compressed image data. are doing.
[0038]
That is, the video signal of the moving image generated by the video processor 10 is input to the moving image input board 15c of the image processing device 4, and the video signal of the moving image is converted into predetermined compressed moving image video signal data, for example, The image data is converted into MJPEG format compressed image data and stored in a recording device (not shown) of the PC 15.
[0039]
Generally, before the endoscope examination is started, examination information related to the endoscope examination is input from the video processor 10, and characters or characters are input to the observation monitor 14 based on the inputted examination information data. In addition to being displayed in the form of numbers, the inspection information data can be transmitted and recorded to the image processing apparatus 4 from the communication port 10a through the communication port 15b.
[0040]
The test information includes, for example, the patient's name, date of birth, gender, age, patient code, test date and time, and the like.
[0041]
That is, the image processing device 4 is connected to the video processor 10 as necessary, and receives and stores various information data from the video processor 10.
[0042]
Generation of insertion shape data in the endoscope insertion shape observation device 3 in the electronic endoscope system 1 having such a configuration will be described with reference to FIGS. 2 and 3.
[0043]
The endoscope insertion shape observation device 3 includes M source coils incorporated in the insertion portion of the electronic endoscope 12 for each frame of an imaged video signal generated by the electronic imaging device of the electronic endoscope 12. Is generated. Based on the insert shape data, an insert shape image is generated and displayed on the display 13b, and the insert shape data is output and supplied to the image processing device 4.
[0044]
As shown in FIG. 2, the coordinate system of the source coil detected by the endoscope insertion shape observation device 3 is such that M insertion coils for the insertion shape estimation are built in the insertion portion of the electronic endoscope 12. And the three-dimensional coordinates (X) of the i-th source coil (i = 0, 1,.j i, Yj i, Zj i).
[0045]
In the structure of the insertion shape data indicating the coordinate system of the source coil detected by the endoscope insertion shape observation device 3, as shown in FIG. 3, data relating to one frame is transmitted as one packet. One packet includes data such as the acquisition time of insertion shape data, display attribute information, attached information, and source coil coordinates. In the source coil coordinate data, the three-dimensional coordinates of the source coil built in the insertion section of the electronic endoscope 12 are arranged in order from the tip of the insertion section to the operation section provided at the base end of the insertion section. It is to be arranged. It is assumed that the coordinates of the source coil outside the detection range of the endoscope insertion shape observation device 3 are set to predetermined constants.
[0046]
Next, the inspection information and the endoscope image from the video processor 10 of the endoscope device 2 in the image processing device 4 and the insertion shape data from the shape processing device 13 of the endoscope insertion shape observation device 3 are output. The acquisition recording processing operation and the detection of the extension of the large intestine as the subject will be described with reference to FIGS.
[0047]
This processing operation is realized by developing and driving the inspection application provided in the image processing apparatus 4 on the PC 15.
[0048]
When starting the endoscopic examination, the video processor 10 inputs examination information, and the PC 15 of the image processing apparatus 4 starts an examination application. When the examination application is started, the examination window 100 and the endoscope image window 105 shown in FIG. 6 are displayed on the display 18.
[0049]
When the inspection application is developed and driven on the PC 15 of the image processing apparatus 4 and the inspection window 100 is displayed on the display 18, the PC 15 determines in step S1 the inspection information and the endoscope image data from the video processor 10 The mode is set to receive and store the insertion shape data from the shape processing device 13 of the endoscope insertion shape observation device 3.
[0050]
Next, in step S2, the PC 15 determines whether or not the operator operates the mouse 16 or the keyboard 17 to turn on the examination start button (noted as Start Exam.) 101 displayed in the examination window 100. I do. The process waits until the inspection start button 101 is turned on, and when it is turned on, step S3 and subsequent steps are executed.
[0051]
In step S3, the PC 15 opens the communication port 15a and starts communication with the shape processing device 13 of the endoscope insertion shape observation device 3, and in step S4 opens the communication port 15b and starts communication with the video processor 10. I do.
[0052]
In step S5, the PC 15 transmits a test information acquisition command to the video processor 10 from the communication port 15b to the communication port 10a of the video processor 10, and the video processor 10 that has received the test information acquisition command transmits the test information to the video processor 10. Send it to PC15.
[0053]
In step S5, the PC 15 records the inspection information transmitted from the video processor 10 in a recording device (not shown) in step S6.
[0054]
Next, in step S7, the PC 15 transmits an insertion shape data acquisition command from the communication port 15a to the communication port 13a of the shape processing device 13, and the shape processing device 13, which has received the insertion shape data acquisition command, The transmission output of the shape data is started. This transmission is continued until the communication between the PC 15 and the shape processing device 13 ends and the communication port 15a is closed.
[0055]
In step S8, the PC 15 receives the inserted shape data transmitted and output from the shape processing device 13 in step S7 and associates the inserted shape data with the inspection information recorded and stored in step S6. (Hereinafter, referred to as an insertion shape file).
[0056]
Next, in step S9, the PC 15 converts the moving image video signal input from the video processor 10 to the moving image input board 15c into compressed image data in MJPEG format, and associates it with the inspection information recorded and stored in step S6. A file is stored in a hard disk (not shown) of the PC 15 (hereinafter, referred to as an image file), and the moving image input to the moving image input board 15c is viewed in an endoscope image window 105 shown in FIG. It is displayed in the mirror image area 106.
[0057]
Next, in step S10, the PC 15 executes a warning process after step S21 shown in FIG. 5, and when this warning process ends, in step S11, the PC 15 issues an inspection end button (End Exam. It is determined whether or not the inspection end button 102 has been operated. If it is determined that the inspection end button 102 has not been operated, the process returns to step S8. If it is determined that the inspection end button 102 has been operated, the communication port 15a and 15b are closed to terminate the communication of information data between the shape processing device 13 and the video processor 10.
[0058]
The warning process in step S10 will be described with reference to FIG. The warning process in step S10 is an extension detection process of the large intestine, which is the subject. If the insertion length of the insertion portion of the endoscope in the large intestine is extended while the distal end of the insertion portion of the electronic endoscope 12 is almost stopped, it is considered that the large intestine is extended. The extension of the large intestine is detected by detecting the insertion length of the insertion portion.
[0059]
In step S21, the PC 15 determines whether or not the insertion shape data of the frame preceding the insertion shape data of the current frame acquired and recorded in step S8 has been acquired, and determines that the insertion shape data of the previous frame has not been acquired. Then, the processing shifts to the processing after step S11.
[0060]
If it is determined in step S21 that the insertion shape data of the previous frame has been acquired, the PC 15 moves the insertion end of the electronic endoscope 12 from the insertion shape data of the previous frame and the current frame in step S22. The quantity dif is calculated from Equation 1.
[0061]
(Equation 1)
The movement amount dif calculated by Expression 1 uses only m pieces of data from the tip of the insertion section. The calculation of the movement amount dif is not limited to Expression 1, and for example, the Euclidean distance may be used.
[0062]
The moving distance of the distal end of the insertion portion, which is the moving amount dif calculated by Expression 1 in step S22, is compared with a predetermined threshold in step S23. If the moving distance of the moving amount dif is larger than the predetermined threshold, the distal end of the insertion portion is If it is determined that the insertion movement is being performed, the process proceeds to step S11 and the subsequent steps. If it is smaller than the threshold value, the distal end of the insertion portion is determined to be in the stopped state, and the steps after step S24 are executed.
[0063]
In step S24, the PC 15 obtains the length of the insertion portion of the electronic endoscope 12 of the current frame detected by the shape processing device 13 of the insertion shape observation device 3. The length of the insertion portion of the electronic endoscope 12 is calculated, for example, based on the number of valid source coil coordinate data of the insertion shape data generated by the shape processing device 13.
[0064]
Next, in step S25, the PC 15 compares the length of the insertion portion of the current frame calculated in step S24 with the length of the endoscope insertion portion of the frame before the current frame. As a result of the comparison in step S25, if the length of the endoscope insertion portion of the current frame is longer than the length of the endoscope insertion portion of the previous frame, the insertion portion is determined to be in the insertion movement, and the process proceeds to step S11 and subsequent steps. When the transition is made and the length of the endoscope insertion portion of the current frame is shorter than the length of the endoscope insertion portion of the previous frame, the insertion portion is determined to be in the stopped or pulled-out state, and step S26 and subsequent steps are executed.
[0065]
The PC 15 determines in step S25 that the length of the insertion portion of the current frame is shorter than the length of the insertion portion in the previous frame, and that extension of the large intestine has not occurred. In step S26, the insertion portion of the endoscope is determined. It is determined whether the end is stopped and the length of the endoscope insertion portion of the current frame is longer than that of the previous frame for a predetermined number of frames.
[0066]
In this step S26, when the distal end of the insertion portion of the electronic endoscope is in a stopped state, it is determined that the colon extension has not occurred unless the increase state of the length of the insertion portion of the previous frame continues for a predetermined number of frames. When the process of step S11 and subsequent steps is executed, and the insertion portion of the electronic endoscope is in a stopped state, and the length of the insertion portion continues to increase by a predetermined number of frames, it is determined that extension of the large intestine has occurred. Is executed.
[0067]
In step S27, the PC 15 generates a warning signal to warn that extension of the large intestine has occurred, and generates a warning sound to provide information on the shape of the endoscope insertion portion, and generates an alarm window, A warning message is displayed on the warning display unit 110 of the control unit 100. The warning display is not only characters, but also a graphic display such as an icon on the warning graphic display unit 111 or a blinking display of the warning display.
[0068]
As described above, the image processing apparatus of the present invention uses the plurality of insertion shape data of the current frame and the previous frame to change the shape of the endoscope insertion section according to the movement of the endoscope insertion section that is actually inserted. Information can be provided, and the extension of the observation site, such as the large intestine, can be quickly and easily detected and determined, and the operator can be quickly alerted and recognized.
[0069]
In the image processing device 4, only the warning of detection of the extension of the large intestine has been described based on the insertion shape data from the insertion shape observation device 3, but a viewer that allows browsing of the endoscope image and the insertion shape data is described. It is also possible to detect the extension of the large intestine with respect to the specified insertion shape data.
[0070]
Next, a second embodiment of the original / image processing apparatus 4 of the electronic endoscope system 1 will be described with reference to FIGS.
[0071]
The configuration of the electronic endoscope system 1 of the second embodiment is the same as that of the above-described first embodiment, and is different from the endoscope device 2 and the endoscope insertion shape observation device 3 in the image processing device 4. Are basically the same.
[0072]
The difference between the second embodiment and the first embodiment described above is that the processing method of the warning process in step S10 by the image processing device 4 is different. The processing after step S41 shown in FIG. 7 is executed.
[0073]
In step S41, the PC 15 determines whether the insertion shape data of the frame preceding the insertion shape data of the current frame acquired and recorded in step S8 has been acquired, and determines that the insertion shape data of the previous frame has not been acquired. Then, the processing shifts to the processing after step S11.
[0074]
If it is determined in step S41 that the insertion shape data of the previous frame has been acquired, the PC 15 moves the insertion end of the electronic endoscope 12 from the insertion shape data of the previous frame and the current frame in step S42. The quantity dif1 is calculated from Equation 2.
[0075]
(Equation 2)
The movement amount dif1 calculated by Expression 2 uses only m1 data from the tip of the insertion portion. The calculation of the movement amount dif1 is not limited to Expression 2, but may be, for example, a Euclidean distance.
[0076]
The moving distance of the distal end of the insertion portion, which is the moving amount dif1 calculated in step S42, is compared with a predetermined threshold value in step S43. If the moving distance of the moving amount dif1 is larger than the predetermined threshold value, If it is determined that the insertion movement is being performed, the process proceeds to step S11 and the subsequent steps. If it is smaller than the threshold value, the distal end of the insertion portion is determined to be in the stopped state, and the steps after step S44 are executed.
[0077]
In step S44, the PC 15 calculates the amount of movement dif2 of the insertion unit of the electronic endoscope 12 from the operation unit side from Equation 3 based on the insertion shape data of the previous frame and the current frame, as in step S42.
[0078]
(Equation 3)
The movement distance of the insertion unit on the operation unit side, which is the movement amount dif2 calculated by Expression 3 in step S44, is compared with a predetermined threshold value in step S45. If the movement distance that is the movement amount dif2 is smaller than the predetermined threshold value, the insertion is performed. The operation unit side of the unit is determined to be stopped, and the process proceeds to step S11 and subsequent steps. If the operation unit side of the insertion unit is larger than the threshold value, the operation unit side of the insertion unit is determined to be moving, and step S46 and subsequent steps are executed.
[0079]
Next, in step S46, the PC 15 uses the ith coil coordinate of the j-th frame as the current frame and the j-th frame as the previous frame as shown in FIG. Angle Ang formed by the direction of the insertion portion based on the direction of the movement and the coordinates of the i-th and i-th eleventh coils of the j-1 frame which is the previous frame.j 1Ask for.
[0080]
(Equation 4)
That is, (Xj-1 i, Yj-1 i, Zj-1 i) And (Xj-1 i-1, Yj-1 i-1, Zj-1 i-1) Is the direction of the insertion portion, and (X)j-1 i, Yj-1 i, Zj-1 i) And (Xj i, Yj i, Zj i) Indicates the direction of movement of the distal end of the insertion portion. Here, the coil number given in advance is used for i.
[0081]
Next, in step S47, the PC 15 determines the angle Ang determined in step S46.j 1Is compared with a predetermined threshold value to determine whether the insertion section of the electronic endoscope 12 is inserted in the direction of the insertion section. Angle Ang between the moving direction of the tip of the insertion portion and the direction of the tip of the insertion portionj 1Is smaller than the predetermined threshold value, it is determined that the operator is performing the insertion operation of the electronic endoscope 12, and steps S48 and subsequent steps are executed. It is determined that the insertion operation has not been performed, and the processing shifts to the processing after step S11.
[0082]
If it is determined in step S47 that the insertion operation of the electronic endoscope 12 has been performed, the PC 15 determines in step S48 whether the state of performing the insertion operation of the electronic endoscope 12 continues for a predetermined number of frames. If it is determined that the predetermined number of frames has not been continued, the process proceeds to the processing after step S11. If it is determined that the predetermined number of frames has been continued, it is determined that the large intestine has expanded, In step S49, a warning is issued as provision of information on the shape of the endoscope insertion section. As the warning method, for example, a warning sound may be generated by the PC 15 or a warning message may be displayed on the warning display unit 110 of the inspection window shown in FIG. The warning display may be not only characters but also figures such as icons, and the character figures may be displayed by blinking.
[0083]
Thereby, it becomes easy to detect extension of the large intestine which may cause discomfort to the patient who is the subject.
[0084]
In addition, when the image processing device 4 receives the insertion shape data from the insertion shape observation device 3, only the detection and warning of the extension of the large intestine has been described. However, it is possible to view the endoscope image and the insertion shape data. It is also possible to prepare a viewer and detect extension of the large intestine with respect to the specified insertion shape data.
[0085]
Next, a third embodiment of the image processing apparatus of the electronic endoscope system according to the present invention will be described with reference to FIGS.
[0086]
The configuration of the electronic endoscope system 1 according to the third embodiment is the same as that of the above-described first embodiment, and is different from the endoscope device 2 and the endoscope insertion shape observation device 3 in the image processing device 4. Are basically the same.
[0087]
The difference of the third embodiment from the first embodiment described above is the detection processing when the insertion operation is performed in a state where a certain angle (angle) is given to the distal end of the insertion section of the electronic endoscope 12. . If the insertion operation of the endoscope is performed without observing the insertion direction of the endoscope insertion portion in a state where the end of the endoscope insertion portion is provided with an angle, the patient of the subject will be uncomfortable. This is to warn of an insertion operation in a state where an angle is given to the endoscope insertion tip.
[0088]
The detection of the insertion operation in a state where the end of the endoscope insertion portion is given an angle is realized as an inspection application provided in the image processing device 4 as in the first embodiment. When the inspection application is activated on the PC 15, an inspection window 100 shown in FIG. 6 is displayed on the display 18, and the operation of the inspection application is the same as the operation in steps S1 to S12. However, the warning process executed in step S10 is the process after step S61 shown in FIG.
[0089]
In step S61, the PC 15 determines whether or not the insertion shape data of the frame before the insertion shape data of the current frame acquired and recorded in step S8 has been acquired, and determines that the insertion shape data of the previous frame has not been acquired. Then, the processing shifts to the processing after step S11.
[0090]
If it is determined in step S61 that the insertion shape data of the previous frame has been acquired, the PC 15 determines in step S62 the angle of the tip of the insertion portion of the electronic endoscope 12 from the insertion shape data of the previous frame and the current frame. Angj 1Ask for. This angle Angj 1As shown in FIG. 10, three consecutive points of the coordinates of the m source coils from the distal end of the endoscope insertion portion are sequentially selected from the end of the endoscope insertion portion by Expression 5 to obtain the angle Ang.j 1Ask for. It should be noted that the larger the angle considered when the starting points of Vec3 and Vec4 are aligned, the greater the bending of the distal end of the insertion portion.
[0091]
(Equation 5)
Next, in step S63, the PC 15 determines the angles Ang determined in step S62.j 1Is compared with a predetermined threshold value to determine the angle Ang.j 1If any of the values is larger than the predetermined threshold, it is determined that the end of the endoscope insertion portion is angled, and the steps after step S64 are executed, and the angle Ang larger than the predetermined threshold is determined.j 1If it is determined that there is no value indicating the value of (i), the processing shifts to the processing after step S11.
[0092]
Angle Ang indicating a value larger than a predetermined threshold value in step S63j 1Exists, and it is determined that the angle is given to the distal end of the endoscope insertion portion, the PC 15 determines in step S64 that the electronic frame of the current frame detected by the shape processing device 13 of the insertion shape observation device 3 The length of the insertion portion of the endoscope 12 is obtained. The length of the insertion portion of the electronic endoscope 12 is calculated, for example, based on the number of valid source coil coordinate data of the insertion shape data generated by the shape processing device 13.
[0093]
Next, in step S65, the PC 15 compares the length of the insertion portion of the current frame calculated in step S64 with the length of the endoscope insertion portion of the frame before the current frame. As a result of the comparison in step S65, if the length of the endoscope insertion portion of the current frame is longer than the length of the endoscope insertion portion of the previous frame, it is determined that the insertion portion is being inserted and moved deeper into the organ, and step S66 If the length of the endoscope insertion portion of the current frame is shorter than the length of the endoscope insertion portion of the previous frame, the insertion portion is determined to be stopped or the insertion portion is pulled out, and the process proceeds to step S11 and thereafter. Is executed.
[0094]
In step S66, the PC 15 determines whether the increase in the length of the endoscope insertion portion has continued for a predetermined number of frames.
[0095]
If it is determined in step S66 that the state of increase in the length of the insertion portion has not continued for the predetermined number of frames, the process proceeds to step S11 and the subsequent steps. Generates a warning signal to warn that the endoscope insertion section is inserted with an angle given to the tip, provides information on the shape of the endoscope insertion section, and generates a warning sound Or a warning message is displayed on the warning display section 110 of the inspection window 100 in FIG. The warning display is not only characters but also a graphic display such as an icon or blinking display.
[0096]
With this, it is possible to provide information on the shape of the endoscope insertion section according to the movement of the endoscope insertion section that is actually inserted, based on the plurality of insertion shape data of the current frame and the previous frame, and It is possible to detect and warn an insertion operation in a state where an angle is given to the distal end of the endoscope insertion portion, which causes discomfort. In addition, as for the insertion operation warning in a state where the angle is given, it is also possible to prepare a viewer that allows browsing of the image and the shape data, and perform the same warning for the specified insertion shape data.
[0097]
Next, a fourth embodiment of the endoscope insertion shape analyzing apparatus according to the present invention will be described with reference to FIGS.
[0098]
The configuration of the electronic endoscope system 1 according to the fourth embodiment is the same as that of the above-described first embodiment, and is different from the endoscope device 2 and the endoscope insertion shape observation device 3 in the image processing device 4. Is basically the same as the method of processing the endoscope image and the insertion shape data.
[0099]
The fourth embodiment is a detection process in a case where an insertion operation is performed in a state where different angles are given to the distal end of the insertion section of the different electronic endoscope 12 than the third embodiment described above.
[0100]
The detection of the insertion operation in a state where the end of the endoscope insertion section is given an angle is realized as an inspection application provided in the image processing device 4 as in the third embodiment. When this inspection application is activated on the PC 15, an inspection window 100 shown in FIG. 6 is displayed on the display 18, and the operation of this inspection application is the same as that in steps S1 to S12. However, the processing executed in step S10 is the processing after step S81 shown in FIG.
[0101]
In step S81, the PC 15 determines whether the insertion shape data in the frame preceding the insertion shape data of the current frame acquired and recorded in step S8 has been acquired, and determines that the insertion shape data of the previous frame has not been acquired. Then, the processing shifts to the processing after step S11.
[0102]
If it is determined in step S81 that the insertion shape data of the previous frame has been acquired, the PC 15 determines in step S82 the angle of the tip of the insertion portion of the electronic endoscope 12 from the insertion shape data of the previous frame and the current frame. Angj 1As shown in FIG. 10, three consecutive points are sequentially selected from the coordinates of the m source coils from the distal end of the endoscope insertion portion by using Expression 5, and an angle Ang is selected.j 1Ask for.
[0103]
Next, in step S83, the PC 15 determines the angles Ang obtained in step S82.j 1Is compared with a predetermined threshold value to determine the angle Ang.j 1If any of the values is larger than the predetermined threshold, it is determined that the angle is given to the distal end of the endoscope insertion section, and the steps from step S84 are executed, and the angle Ang larger than the predetermined threshold is executed.j 1If it is determined that there is no value indicating the value of (i), the processing shifts to the processing after step S11.
[0104]
Angle Ang indicating a value larger than a predetermined threshold value in step S83j 1Exists, and it is determined that the end of the endoscope insertion section is angled, the PC 15 determines in step S84 the operation of the insertion section of the electronic endoscope 12 from the insertion shape data of the previous frame and the current frame. As shown in FIG. 8 according to Equation 4, the direction of the movement of the unit and the direction of the direction of the insertion unit operation unit side are the i-th of the j-th frame which is the current frame and the j-1 frame which is the previous frame. Angle Ang formed by the direction of movement of the insertion unit operation unit using the coil coordinates and the direction of the insertion unit operation unit based on the i-th and i-th eleventh coil coordinates of the j-1 frame, which is the previous frame.j 1Ask for.
[0105]
That is, (Xj-1 i, Yj-1 i, Zj-1 i) And (Xj-1 i-1, Yj-1 i-1, Zj-1 i-1) Is the direction of the insertion portion, and (X)j-1 i, Yj-1 i, Zj-1 i) And (Xj i, Yj i, Zj i) Indicates the direction of movement of the distal end of the insertion portion. Here, the coil number given in advance is used for i.
[0106]
Next, in step S85, the PC 15 determines the angle Ang determined in step S84.j 1It is determined whether the operation unit of the insertion unit of the electronic endoscope 12 has been inserted from a predetermined threshold value. Angle Ang between the movement direction of the insertion section operation section and the movement direction of the insertion section operation sectionj 1Is smaller than the predetermined threshold, it is determined that the operator is performing an insertion operation on the operation unit of the insertion unit of the electronic endoscope 12, and steps S86 and subsequent steps are executed. Then, it is determined that the insertion operation of the insertion section operation section of the electronic endoscope 12 has not been performed, and the processing shifts to the processing after step S11.
[0107]
If it is determined in step S85 that the insertion section operation section of the electronic endoscope 12 is being inserted, the PC 15 determines in step S86 that the current frame is the j-th frame as shown in FIG. The movement direction of the tip of the insertion portion is determined by using the i-th coil coordinate of the j-1 frame which is the previous frame, and the first and i-th eleventh coil coordinates of the j-1 frame which is the previous frame. Angle Ang formed by the direction of the insertion portionj 1Ask for.
[0108]
That is, (Xj-1 i, Yj-1 i, Zj-1 i) And (Xj-1 i-1, Yj-1 i-1, Zj-1 i-1) Is the direction of the insertion portion, and (X)j-1 i, Yj-1 i, Zj-1 i) And (Xj i, Yj i, Zj i) Indicates the direction of movement of the distal end of the insertion portion. Here, the coil number given in advance is used for i.
[0109]
Next, in step S87, the PC 15 determines the angle Ang obtained in step S86.j 1Then, it is determined whether the distal end of the insertion section of the electronic endoscope 12 is inserted in another direction different from the observation direction. The determination of the direction of movement of the distal end of the insertion portion is based on the angle Ang with the direction of the distal end of the insertion portion.j 1Is larger than the predetermined threshold, it is determined that the distal end of the insertion section of the endoscope is inserted in a direction different from the observation direction. If it is determined in this step S87 that the end of the insertion portion of the endoscope is inserted in a direction different from the observation direction, steps S88 and subsequent steps are executed, and it is determined that the insertion end of the endoscope is inserted in the observation direction. If it is determined, the process proceeds to the processing after step S11.
[0110]
If it is determined in step S87 that the distal end of the insertion portion of the endoscope is inserted in a direction different from the observation direction, the PC 15 determines in step S88 that the insertion operation in another direction different from the observation direction is performed for a predetermined number of frames. It is determined whether or not the operation has continued for a predetermined number of frames. If it is determined that the operation has not continued for a predetermined number of frames, the process proceeds to step S11 and the subsequent steps. Then, it is determined that the insertion operation is being performed in a state where an angle is given to the distal end of the insertion portion, and a warning is generated in step S89. As the warning method, for example, a warning sound may be generated by the PC 15 or a warning message may be displayed on the warning display unit 110 of the inspection window 100 shown in FIG. The warning display may be not only characters but also figures such as icons, and the character figures may be displayed by blinking.
[0111]
Thereby, it becomes easy to detect insertion in a state where an angle that causes discomfort of the patient as a subject is given.
[0112]
In addition, when the image processing device 4 receives the insertion shape data from the insertion shape observation device 3, only the insertion warning in which an angle is given to the distal end of the insertion portion has been described with respect to the detection warning. It is also possible to prepare a viewer that allows browsing of the shape data, and issue a similar warning to the specified inserted shape data.
[0113]
Next, a fifth embodiment of the endoscope insertion shape analyzing apparatus according to the present invention will be described with reference to FIGS.
[0114]
The configuration of the electronic endoscope system 1 of the fifth embodiment is the same as that of the above-described first embodiment, and is different from the endoscope device 2 and the endoscope insertion shape observation device 3 in the image processing device 4. The processing method of the endoscope image and the insertion shape data is basically the same.
[0115]
In the fifth embodiment, for example, in a large intestine endoscope, a loop may be formed in the process of inserting the insertion portion of the endoscope into the large intestine. This loop is called an α loop, an N loop, a γ loop, or the like depending on its shape.
[0116]
For the purpose of reducing the discomfort given to the patient who is the subject by such a loop and improving the insertion property of the insertion section, an operation of releasing the loop of the endoscope insertion section and performing linearization is performed.
[0117]
When a loop is formed during the insertion operation of the endoscope insertion section, the loop state is recognized and displayed, and the method of linearizing the endoscope insertion section is displayed. The purpose of the present invention is to improve the insertability of the medical device, shorten the time required for endoscopy, and reduce the discomfort of the patient.
[0118]
The detection of the loop formed at the end of the endoscope insertion section is realized as an inspection application provided in the image processing device 4 as in the first embodiment. When this inspection application is activated on the PC 15, an inspection window 100 shown in FIG. 6 is displayed on the display 18, and the operation of this inspection application is the same as that in steps S1 to S12. However, the processing executed in step S10 is the processing after step S101 shown in FIG.
[0119]
Here, as an example of recognizing the loop state of the endoscope insertion section and performing the endoscope operation for linearizing in the fifth embodiment, (1) whether the loop is clockwise or counterclockwise . The direction of rotation of the loop is considered to be the direction along the operation unit side from the distal end of the endoscope insertion unit. (2) Which of the loop-forming portion of the endoscope insertion portion, the distal end side and the operation portion side is closer to the viewpoint side? Are used.
[0120]
In addition, since the endoscope insertion shape can be considered as a curve, a curve feature extraction technique can be used for loop recognition and rotation direction determination.
[0121]
Therefore, in the description of this embodiment, for simplicity, the inserted shape is considered as a two-dimensional curve projected on the X and Y axes, and, for example, a P-type Fourier descriptor is used. The P-type Fourier descriptor is described in IEICE Transactions Vol. j67-A No. 3 is detailed.
[0122]
In this method, first, a curve is divided into Vn (Vn is a positive number of segments) line segments, end points of the line segments are displayed in a complex form, a total curvature function at each point is defined, and a total curvature function is defined. Is a characteristic of the curve obtained by performing Fourier transform on.
[0123]
Also, the characteristics of the clockwise and counterclockwise circular loop-shaped P-type Fourier descriptors are stored in advance, and the power spectrum of each shape is represented by Cq (k), (k = 0,..., Vn− 1). q = 0 is clockwise, and q = 1 is counterclockwise.
[0124]
In step S101, the PC 15 determines a loop formation candidate. When a circular loop is formed, since the source coil is close to the source coil, a portion where the distance between the two source coils is smaller than a predetermined threshold is set as a loop formation candidate.
[0125]
Next, in step S102, the PC 15 divides the loop forming candidate curve into Vn line segments, and obtains power spectra C (k), (k = 0,..., Vn-1) using the P-type Fourier descriptor. . The end points of the Vn line segments are (Px0, Py0, Pz0),..., (PxVn, PyVn, PzVn) from the tip end side of the insertion portion. In the P-type Fourier descriptor, only the xy components are used.
[0126]
Next, in step S103, the PC 15 obtains a Euclidean distance between the power spectrum Cq (k) and the power spectrum C (k). The Euclidean distance obtained in step S103 is compared with a predetermined threshold value in step S104. If the obtained Euclidean distance is smaller than the threshold value, it is determined that a loop is formed. In this loop determination, if both C0 (k) and C1 (k) are smaller than the threshold value, it is determined that the rotation direction is smaller. If there is no loop, the processing shifts to the processing after step S11.
[0127]
If it is determined in step S104 that there is a loop, the PC 15 determines in step S105 (PxVn, PyVn, PzVn), (PxVn-1, PyVn-1, PzVn-1), (PxVn-2, PyVn-2, The plane S on which the loop is formed is determined using PzVn-2). Next, in step S106, a normal vector of the plane S is obtained, and an angle θ formed with a visual line vector indicating a direction in which the endoscope insertion portion is observed is obtained. Next, in step S107, the coordinates of (Px0, Py0, Pz0) on the distal end side of the insertion portion are substituted into the equation of the plane S, and is it located on the viewpoint side or on the opposite side to the plane S? decide.
[0128]
The method of determining whether the plane S is on the viewpoint side or the opposite side to the viewpoint is as follows: (1) If the angle θ is greater than 90 degrees and the substitution result is positive, the viewpoint side (see FIG. 13); 2) When the angle θ is larger than 90 degrees and the substitution result is negative, the opposite side to the viewpoint (see FIG. 13); (3) When the angle θ is 90 degrees or less and the substitution result is positive, the opposite side to the viewpoint. (See FIG. 14), (4) If the angle θ is 90 degrees or less and the result of the substitution is negative, it is on the viewpoint side (see FIG. 14).
[0129]
In step S108, the operation method of the endoscope, that is, the loop release method is determined based on the relationship between the loop direction, the insertion portion tip, and the plane S in step S107. An example of the criterion of the loop release method is as follows.
[0130]
(1) If the tip is counterclockwise and the tip is on the opposite side to the viewpoint, rotate the endoscope clockwise (see FIG. 15). (2) If the tip is clockwise and the tip is on the viewpoint, rotate clockwise. Rotate the endoscope (see FIG. 16). (3) If the tip is clockwise and the tip is on the opposite side to the viewpoint, rotate the endoscope counterclockwise (see FIG. 17). When the distal end is clockwise, the endoscope is rotated counterclockwise (see FIG. 18).
[0131]
Next, in step S109, the PC 15 displays the loop release method set in step S108 on the operation method display unit 111 of the examination window 100 as provision of endoscope operation information (information for prompting operation).
[0132]
As shown in FIG. 6, the display of the operation method display unit 111 is displayed as a figure or an icon indicating the endoscope rotation operation clockwise or counterclockwise with an arrow or the like, or an operation explanatory word, or Alternatively, a voice instruction may be used.
[0133]
In the image processing device 4, the release operation display when receiving the insertion shape data from the insertion shape observation device 3 has been described. However, a viewer capable of browsing the endoscope image and the insertion shape data is prepared. Similar display can be performed for the specified insertion shape data.
[0134]
As described above, the insertion shape is recognized, the state of the loop is recognized and displayed, and the insertion operation is presented, thereby improving the insertion performance of the endoscope and shortening the time of the endoscopic examination, and reducing the pain given to the patient. Become.
[0135]
Note that the image processing device 4 for realizing the above-described first to fifth embodiments may be one of the functions constituting the electronic endoscope system 1 or may be mounted on the endoscope insertion shape observation device 3. good. Alternatively, an integrated function of the insertion shape observation device 3 and the electronic endoscope system 1 may be provided.
[0136]
Further, in the above-described first to fifth embodiments, the mechanism for estimating the shape of the endoscope insertion section includes one of a source coil group for generating magnetism and a sense coil group for detecting magnetism. Was described in the example of mounting on the endoscope insertion part and estimating the shape by the magnetism detected by the sense coil group.However, for example, using an optical fiber that causes transmission loss due to bending, this optical fiber is Provided in the endoscope insertion section, a method of detecting bending of the insertion section according to the change in the amount of light transmitted through the optical fiber, or applying special ink to a flexible board, and the resistance value changes in proportion to the bending of the board Other detection methods such as utilizing characteristics may be used.
[0137]
[Appendix]
According to the embodiment of the present invention described in detail above, the following configuration can be obtained.
[0138]
(Supplementary Note 1) An endoscope having an insertion portion to be inserted into a body cavity,
Shape detection means for detecting the shape of the endoscope insertion portion,
Shape storage means for storing a plurality of shapes of the endoscope insertion portion detected by the shape detection means,
Based on the shape of the plurality of endoscope insertion sections stored in the shape storage means, shape analysis means for analyzing the shape of the endoscope insertion section,
Information providing means for providing information on the shape of the endoscope insertion portion, according to the result of the analysis by the shape analysis means,
An endoscope insertion shape analyzing apparatus comprising:
[0139]
(Supplementary Note 2) An endoscope having an insertion portion to be inserted into a body cavity,
Shape detection means for detecting the shape of the endoscope insertion portion,
Shape analysis means for analyzing the shape of the endoscope insertion portion detected by the shape detection means,
Information providing means for providing information on the endoscope operation, according to the result of the analysis by the shape analysis means,
An endoscope insertion shape analyzing apparatus comprising:
[0140]
(Supplementary Note 3) An endoscope having an insertion portion to be inserted into a body cavity,
A shape detection step of detecting the shape of the endoscope insertion unit,
A shape storing step of storing a plurality of shapes of the endoscope insertion portion detected in the shape detecting step;
A shape analysis step of analyzing the shape of the endoscope insertion portion based on the shapes of the insertion portions of the plurality of endoscopes stored in the shape storage step;
According to the result of the analysis by the shape analysis step, an information providing step of providing information on the shape of the endoscope insertion portion,
An endoscope insertion shape analysis method characterized by comprising:
[0141]
(Supplementary Note 4) An endoscope having an insertion portion to be inserted into a body cavity,
A shape detection step of detecting the shape of the endoscope insertion unit,
A shape analysis step of analyzing the shape of the endoscope insertion section detected in the shape detection step,
An information providing step of providing information on an endoscope operation according to a result of the analysis by the shape analysis step,
An endoscope insertion shape analysis method characterized by comprising:
[0142]
(Supplementary Note 5) The shape detecting means includes a plurality of magnetic field generating means for generating a magnetic field, a plurality of magnetic field detecting means for detecting a magnetic field of the magnetic field generating means, and the magnetic field generating means or the magnetic field detecting means arranged in an insertion portion. ,
3. The endoscope insertion shape analysis device according to claim 1, further comprising: a shape estimation unit configured to estimate a shape of the insertion portion based on a detection result of the magnetic field detection unit.
[0143]
(Supplementary Note 6) The shape detection means includes a plurality of means for detecting a physical quantity of a specific portion of the insertion portion, and a shape estimation means for estimating a shape of the insertion portion based on a result of the detected physical quantity and an arrangement position. The endoscope insertion shape analyzing apparatus according to any one of appendix 1 and appendix 2, characterized by comprising:
[0144]
(Supplementary note 7) The endoscope insertion shape analyzing apparatus according to supplementary note 6, wherein the means for detecting the physical quantity of the specific portion of the insertion portion includes a sensor that detects a plurality of distortions.
[0145]
(Supplementary Note 8) The endoscope insertion shape analyzing apparatus according to supplementary note 6, wherein the means for detecting the physical quantity of the specific portion of the insertion section includes a sensor that detects a plurality of pressures.
[0146]
(Supplementary Note 9) The endoscope insertion shape analyzing apparatus according to Supplementary Note 6, wherein the means for detecting the physical quantity of the specific portion of the insertion portion includes a sensor that detects a plurality of displacements.
[0147]
(Supplementary Note 10) The shape analysis means includes: means for calculating a movement amount of a specific portion of the insertion portion; and means for estimating a state of the insertion portion from a movement amount of the specific portion of the insertion portion. 2. The endoscope insertion shape analysis device according to attachment 1, wherein
[0148]
(Supplementary Note 11) The endoscope insertion shape analyzing apparatus according to supplementary note 2, wherein the shape analysis unit includes a unit that detects a loop shape of the insertion unit.
[0149]
(Supplementary Note 12) The endoscope insertion shape analyzing apparatus according to supplementary note 1, wherein the information providing unit displays information on a result of the analysis unit on a display device.
[0150]
(Supplementary Note 13) The endoscope insertion shape analyzing apparatus according to any one of Supplementary note 1 or 12, wherein the information providing unit presents information of a result of the analysis unit by sound or voice.
[0151]
(Supplementary note 14) The endoscope insertion shape analyzing apparatus according to supplementary note 2, wherein the operation information providing unit provides information on a loop release method according to a result of the analysis unit.
[0152]
(Supplementary note 15) The endoscope insertion shape analyzing apparatus according to Supplementary note 2 or 14, wherein the operation information providing unit displays information on an endoscope operation on a display device according to a result of the analysis unit. .
[0153]
(Supplementary note 16) The supplementary note 2, Supplementary note 14, or Supplementary note 15, wherein the operation information providing unit presents information on the endoscope operation by sound or voice according to the result of the analysis unit. Endoscope insertion shape analyzer.
[0154]
(Supplementary Note 17) The endoscope insertion shape analyzing apparatus according to any one of Supplementary note 12 and Supplementary note 15, wherein the information displayed on the display device is a character and / or a graphic.
[0155]
【The invention's effect】
The endoscope insertion shape analysis device and the endoscope insertion shape analysis method of the present invention can provide information that leads to an improvement in endoscope insertability when analyzing the insertion shape of the endoscope insertion section. This has the effect of shortening the endoscopy time and reducing discomfort to the patient.
[Brief description of the drawings]
FIG. 1 is a block diagram showing the overall configuration of an electronic endoscope system using an image processing device for performing an endoscope insertion shape analysis according to the present invention.
FIG. 2 is an explanatory diagram illustrating an insertion position coordinate system in the image processing device of the electronic endoscope system according to the present invention.
FIG. 3 is an explanatory diagram illustrating a data structure of insertion position detection data generated by the endoscope insertion shape observation device of the electronic endoscope system according to the present invention.
FIG. 4 is a flowchart illustrating the processing operation of an endoscope image, insertion shape data, and the like, which is the first embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 5 is a flowchart illustrating an extension detection operation of the subject, which is the first embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 6 is an explanatory diagram illustrating a display screen displayed on a display at the time of endoscopic examination in the image processing device of the electronic endoscope system according to the present invention.
FIG. 7 is a flowchart illustrating an angle detection operation of an endoscope tip in the second embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 8 is an explanatory diagram illustrating an angle between a direction of an endoscope insertion unit and a movement direction of the insertion unit in a second embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 9 is a flowchart for explaining an angle detection and a warning operation of an endoscope distal end portion in the third embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 10 is an explanatory diagram illustrating angle detection of a distal end portion of an endoscope in a third embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 11 is a flowchart illustrating an angle detection and a warning operation of an endoscope distal end portion in a fourth embodiment of the image processing apparatus of the electronic endoscope system according to the present invention.
FIG. 12 is a flowchart illustrating a loop detection and loop release method display operation of an endoscope insertion unit in a fifth embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 13 is an explanatory diagram illustrating a state of a loop of an endoscope insertion unit in a fifth embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 14 is an explanatory diagram illustrating a state of a loop of an endoscope insertion unit in a fifth embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 15 is an explanatory diagram illustrating a method of releasing a loop of an endoscope insertion unit in a fifth embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 16 is an explanatory diagram illustrating a method of releasing a loop of an endoscope insertion unit in a fifth embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 17 is an explanatory diagram illustrating a method of releasing a loop of an endoscope insertion unit in a fifth embodiment of the image processing device of the electronic endoscope system according to the present invention.
FIG. 18 is an explanatory diagram illustrating a method of releasing a loop of an endoscope insertion unit in a fifth embodiment of the image processing device of the electronic endoscope system according to the present invention.
[Explanation of symbols]
1. Electronic endoscope system
2. Endoscope device
3: Endoscope insertion shape observation device
4: Image processing device
10 Video processor
11 Light source device
12 ... Electronic endoscope
13. Shape processing device
14. Observation monitor
15. Personal computer (PC)
16 ... Mouse
17 ... Keyboard
18 ... Display

Claims (4)

  1. An endoscope having an insertion portion to be inserted into a body cavity,
    Shape detection means for detecting the shape of the endoscope insertion portion,
    Shape storage means for storing a plurality of shapes of the endoscope insertion portion detected by the shape detection means,
    Based on the shape of the plurality of endoscope insertion sections stored in the shape storage means, a shape analysis means for analyzing the shape of the endoscope insertion section,
    Information providing means for providing information on the shape of the endoscope insertion portion, according to the result of the analysis by the shape analysis means,
    An endoscope insertion shape analyzing apparatus comprising:
  2. An endoscope having an insertion portion to be inserted into a body cavity,
    Shape detection means for detecting the shape of the endoscope insertion portion,
    Shape analysis means for analyzing the shape of the endoscope insertion portion detected by the shape detection means,
    Information providing means for providing information on the endoscope operation, according to the result of the analysis by the shape analysis means,
    An endoscope insertion shape analyzing apparatus comprising:
  3. An endoscope having an insertion portion to be inserted into a body cavity,
    A shape detection step of detecting the shape of the endoscope insertion unit,
    A shape storage step of storing a plurality of shapes of the endoscope insertion portion detected in the shape detection step,
    Based on the shapes of the plurality of endoscope insertion sections stored in the shape storage step, a shape analysis step of analyzing the shape of the endoscope insertion section,
    According to the result of the analysis by the shape analysis step, an information providing step of providing information on the shape of the endoscope insertion portion,
    An endoscope insertion shape analysis method characterized by comprising:
  4. An endoscope having an insertion portion to be inserted into a body cavity,
    A shape detection step of detecting the shape of the endoscope insertion unit,
    A shape analysis step of analyzing the shape of the endoscope insertion section detected in the shape detection step,
    An information providing step of providing information on an endoscope operation according to a result of the analysis by the shape analysis step,
    An endoscope insertion shape analysis method characterized by comprising:
JP2003162844A 2003-06-06 2003-06-06 Endoscope insertion shape analyzer Active JP4274854B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2003162844A JP4274854B2 (en) 2003-06-06 2003-06-06 Endoscope insertion shape analyzer

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
JP2003162844A JP4274854B2 (en) 2003-06-06 2003-06-06 Endoscope insertion shape analyzer
EP03809860A EP1504712B1 (en) 2002-10-29 2003-10-29 Endoscope information processor and processing method
DE60330335A DE60330335D1 (en) 2002-10-29 2003-10-29 Endoscopic information processor and processing method
PCT/JP2003/013829 WO2004039249A1 (en) 2002-10-29 2003-10-29 Endoscope information processor and processing method
US10/515,668 US8211010B2 (en) 2002-10-29 2003-10-29 Endoscope information processor and processing method

Publications (3)

Publication Number Publication Date
JP2004358095A true JP2004358095A (en) 2004-12-24
JP2004358095A5 JP2004358095A5 (en) 2007-08-09
JP4274854B2 JP4274854B2 (en) 2009-06-10

Family

ID=34054869

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2003162844A Active JP4274854B2 (en) 2003-06-06 2003-06-06 Endoscope insertion shape analyzer

Country Status (1)

Country Link
JP (1) JP4274854B2 (en)

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006288752A (en) * 2005-04-11 2006-10-26 Olympus Medical Systems Corp Endoscope insertion shape analyzer and endoscope insertion shape analysis method
WO2007023631A1 (en) 2005-08-25 2007-03-01 Olympus Medical Systems Corp. Device for analyzing endoscope insertion shape and system for analyzing endoscope insertion shape
JP2007054401A (en) * 2005-08-25 2007-03-08 Olympus Medical Systems Corp Apparatus for analyzing shape into which endoscope is inserted
JP2007135783A (en) * 2005-11-16 2007-06-07 Olympus Medical Systems Corp Insertion monitoring apparatus
EP1872705A1 (en) * 2005-04-18 2008-01-02 Olympus Medical Systems Corp. Endoscope-shape detection device
WO2008032478A1 (en) * 2006-09-12 2008-03-20 Olympus Medical Systems Corp. Endoscope device
WO2008059636A1 (en) 2006-11-13 2008-05-22 Olympus Medical Systems Corp. Endoscope insertion shape analyzing system, and observation system for living body
JP2008119260A (en) * 2006-11-13 2008-05-29 Olympus Medical Systems Corp Living body observation system
JP2008119259A (en) * 2006-11-13 2008-05-29 Olympus Medical Systems Corp Endoscope insertion shape analysis system
JP2008136628A (en) * 2006-11-30 2008-06-19 Olympus Medical Systems Corp Endoscope insertion shape analyzing system
KR100871035B1 (en) * 2006-11-07 2008-11-27 엔이씨 도낀 가부시끼가이샤 Lead frame, method of manufacturing a face-down terminal solid electrolytic capacitor using the lead frame, and face-down terminal solid electrolytic capacitor manufactured by the method
WO2009044580A1 (en) * 2007-10-02 2009-04-09 Olympus Medical Systems Corp. Endoscope-shaped analyzer
JP2009082625A (en) * 2007-10-02 2009-04-23 Olympus Medical Systems Corp Shape analysis device of endoscope
JP2009090023A (en) * 2007-10-11 2009-04-30 Olympus Medical Systems Corp Endoscope-shaped analyzer
JP2014064917A (en) * 2007-01-29 2014-04-17 Intuitive Surgical Operations Inc System for controlling instrument using shape sensor
US20150351608A1 (en) * 2013-01-10 2015-12-10 Ohio University Method and device for evaluating a colonoscopy procedure
WO2016098251A1 (en) * 2014-12-19 2016-06-23 オリンパス株式会社 Insertion and removal support device and insertion and removal support method
JPWO2016098253A1 (en) * 2014-12-19 2017-10-12 オリンパス株式会社 Insertion / removal support device and insertion / removal support method
JPWO2016098254A1 (en) * 2014-12-19 2017-10-12 オリンパス株式会社 Insertion / removal support device and insertion / removal support method
WO2018069992A1 (en) * 2016-10-12 2018-04-19 オリンパス株式会社 Insertion system
WO2018220797A1 (en) * 2017-06-01 2018-12-06 オリンパス株式会社 Flexible tube insertion assistance device and flexible tube insertion device
WO2019155617A1 (en) * 2018-02-09 2019-08-15 オリンパス株式会社 Endoscope system, endoscope control device, operation method for endoscope system, and storage medium storing endoscope control program

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE112014007268B4 (en) * 2014-12-19 2019-02-07 Olympus Corporation Insertion / removal support apparatus and insertion / removal support method
WO2018116446A1 (en) * 2016-12-22 2018-06-28 オリンパス株式会社 Flexible tube inserting device
WO2018131134A1 (en) * 2017-01-13 2018-07-19 オリンパス株式会社 Flexible tube insertion device

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4656988B2 (en) * 2005-04-11 2011-03-23 オリンパスメディカルシステムズ株式会社 Endoscope insertion shape analysis apparatus and endoscope insertion shape analysis method
JP2006288752A (en) * 2005-04-11 2006-10-26 Olympus Medical Systems Corp Endoscope insertion shape analyzer and endoscope insertion shape analysis method
EP1872705A1 (en) * 2005-04-18 2008-01-02 Olympus Medical Systems Corp. Endoscope-shape detection device
US9125560B2 (en) 2005-04-18 2015-09-08 Olympus Corporation Endoscope shape detecting apparatus
EP1872705A4 (en) * 2005-04-18 2009-12-02 Olympus Medical Systems Corp Endoscope-shape detection device
JP2007054401A (en) * 2005-08-25 2007-03-08 Olympus Medical Systems Corp Apparatus for analyzing shape into which endoscope is inserted
WO2007023631A1 (en) 2005-08-25 2007-03-01 Olympus Medical Systems Corp. Device for analyzing endoscope insertion shape and system for analyzing endoscope insertion shape
JP2007135783A (en) * 2005-11-16 2007-06-07 Olympus Medical Systems Corp Insertion monitoring apparatus
WO2008032478A1 (en) * 2006-09-12 2008-03-20 Olympus Medical Systems Corp. Endoscope device
KR100871035B1 (en) * 2006-11-07 2008-11-27 엔이씨 도낀 가부시끼가이샤 Lead frame, method of manufacturing a face-down terminal solid electrolytic capacitor using the lead frame, and face-down terminal solid electrolytic capacitor manufactured by the method
JP2008119260A (en) * 2006-11-13 2008-05-29 Olympus Medical Systems Corp Living body observation system
WO2008059636A1 (en) 2006-11-13 2008-05-22 Olympus Medical Systems Corp. Endoscope insertion shape analyzing system, and observation system for living body
US8251890B2 (en) 2006-11-13 2012-08-28 Olympus Medical Systems Corp. Endoscope insertion shape analysis system and biological observation system
JP2008119259A (en) * 2006-11-13 2008-05-29 Olympus Medical Systems Corp Endoscope insertion shape analysis system
JP2008136628A (en) * 2006-11-30 2008-06-19 Olympus Medical Systems Corp Endoscope insertion shape analyzing system
JP2014064917A (en) * 2007-01-29 2014-04-17 Intuitive Surgical Operations Inc System for controlling instrument using shape sensor
US9737198B2 (en) 2007-01-29 2017-08-22 Intuitive Surgical Operations, Inc. System for controlling an instrument using shape sensors
WO2009044580A1 (en) * 2007-10-02 2009-04-09 Olympus Medical Systems Corp. Endoscope-shaped analyzer
JP2009082625A (en) * 2007-10-02 2009-04-23 Olympus Medical Systems Corp Shape analysis device of endoscope
US8545397B2 (en) 2007-10-02 2013-10-01 Olympus Medical Systems Corp. Endoscope shape analysis apparatus
JP2009090023A (en) * 2007-10-11 2009-04-30 Olympus Medical Systems Corp Endoscope-shaped analyzer
US20150351608A1 (en) * 2013-01-10 2015-12-10 Ohio University Method and device for evaluating a colonoscopy procedure
WO2016098251A1 (en) * 2014-12-19 2016-06-23 オリンパス株式会社 Insertion and removal support device and insertion and removal support method
JPWO2016098253A1 (en) * 2014-12-19 2017-10-12 オリンパス株式会社 Insertion / removal support device and insertion / removal support method
JPWO2016098254A1 (en) * 2014-12-19 2017-10-12 オリンパス株式会社 Insertion / removal support device and insertion / removal support method
JPWO2016098251A1 (en) * 2014-12-19 2017-10-19 オリンパス株式会社 Insertion / removal support device and insertion / removal support method
WO2018069992A1 (en) * 2016-10-12 2018-04-19 オリンパス株式会社 Insertion system
WO2018220797A1 (en) * 2017-06-01 2018-12-06 オリンパス株式会社 Flexible tube insertion assistance device and flexible tube insertion device
WO2019155617A1 (en) * 2018-02-09 2019-08-15 オリンパス株式会社 Endoscope system, endoscope control device, operation method for endoscope system, and storage medium storing endoscope control program

Also Published As

Publication number Publication date
JP4274854B2 (en) 2009-06-10

Similar Documents

Publication Publication Date Title
US6319201B1 (en) Imaging device and associated method
CN101322640B (en) Endoscope system
US5871446A (en) Ultrasonic medical system and associated method
EP0975257B1 (en) Endoscopic system
JP4740212B2 (en) System and method for displaying a data stream
US5776050A (en) Anatomical visualization system
JP5774596B2 (en) visual tracking / annotation of clinically important anatomical landmarks for surgical intervention
JP4899068B2 (en) Medical image observation support device
US8248413B2 (en) Visual navigation system for endoscopic surgery
US20060170765A1 (en) Insertion support system for producing imaginary endoscopic image and supporting insertion of bronchoscope
JP5972865B2 (en) System for displaying in-vivo image portion and method for operating the same
JP2006518623A (en) Intravascular imaging
US8617058B2 (en) Displaying image data from a scanner capsule
US6589163B2 (en) Endoscope shape detecting apparatus wherein form detecting processing is controlled according to connection state of magnetic field generating means
US7824328B2 (en) Method and apparatus for tracking a surgical instrument during surgery
JP4585048B2 (en) Endoscope system
US7195587B2 (en) Apparatus and method using it for detecting and displaying form of insertion part of endoscope inserted into body cavity
JP4009639B2 (en) Endoscope device, endoscope device navigation method, endoscope image display method, and endoscope image display program
US8248414B2 (en) Multi-dimensional navigation of endoscopic video
EP2946733A1 (en) Device for assisting laparoscopic surgery
JP3660887B2 (en) Surgery support device
JP4436966B2 (en) Endoscope tutorial system
US20130038707A1 (en) Apparatus and Method for Using Augmented Reality Vision System in Surgical Procedures
US20040230094A1 (en) Remote operation support system and method
US6295368B1 (en) Endoscopic image processing system capable of estimating absolute shape of object entity

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20060602

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20070621

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20070911

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20071106

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20080430

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20080630

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20081118

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20090119

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20090210

A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20090303

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120313

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120313

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130313

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20140313

Year of fee payment: 5

S531 Written request for registration of change of domicile

Free format text: JAPANESE INTERMEDIATE CODE: R313531

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250