JP7084991B2 - Force information calculation device, endoscope system and force information calculation method - Google Patents

Description

The present invention relates to a force information calculation device for calculating the magnitude of a force acting on a flexible member, an endoscope system, and a force information calculation method.

In recent years, endoscopes having a flexible insertion portion have been widely used in the medical and industrial fields. In the medical field, endoscopes are widely used for organ observation, therapeutic measures using treatment tools, surgical operations under endoscopic observation, and the like.

When the insertion portion of the endoscope is inserted into a lumen such as the large intestine, the insertion portion is inserted into the lumen while in contact with the inner wall of the lumen, and is deformed according to the force received from the lumen. If the insertion portion is inserted with an unnecessarily large force while the insertion portion is in contact with the inner wall of the lumen, there is a risk of damaging an organ or the like. In order to prevent this, it is preferable to be able to acquire information on the force acting on the insertion portion.

The force acting on the insertion portion can be detected, for example, by providing a sensor such as a strain gauge on the insertion portion. However, if it is attempted to detect a component in a plurality of directions for a force acting on the insertion portion or to detect a force acting on the insertion portion at a plurality of positions, the number of sensors will increase. As a result, the number of wirings for the sensor is increased, and the flexibility of the insertion portion is reduced.

Japanese Patent Publication No. 2013-94337 describes the direction of the external force applied to the tip of the insertion part by using a bending sensor using an optical fiber, which has almost no effect on the size and hardness of the insertion part. A tubular insertion device that detects size is disclosed.

However, in order to prevent damage to organs and the like, it is necessary to acquire information on the force acting not only on the tip of the insertion portion but also on a place other than the tip of the insertion portion. With the tubular insertion device disclosed in Japanese Patent Publication No. 2013-94337, it is difficult to obtain information on the force acting on a place other than the tip of the insertion part or a plurality of places in the insertion part.

The above problem is to acquire information on the force acting on the insertion part of an endoscope used in the industrial field, or to acquire information on the force acting on a flexible member other than the insertion part such as a catheter. This also applies when you do.

Therefore, an object of the present invention is to provide a force information calculation device, an endoscope system, and a force information calculation method capable of calculating information on the magnitude of a force acting on an arbitrary position on a flexible member. do.

The force information calculation device of one aspect of the present invention is an insertion portion of an endoscope, and is shape information having a correspondence relationship with a flexible member having an elongated shape and the shape of the flexible member. A shape sensor unit that acquires a plurality of first shape information which is shape information at each of a plurality of measurement points arranged at intervals along the longitudinal direction of the flexible member in the flexible member. A plurality of second shape information which is shape information at each of a plurality of calculation points defined at intervals along the longitudinal direction in the flexible member based on the plurality of first shape information. A shape information acquisition unit for acquiring a shape information acquisition unit, a usage frequency information acquisition unit for acquiring the number of times the flexible member has been used, a first constant set including a plurality of first constants and a fourth constant, and the use. The storage unit for storing the value of the number of times, the multiplication of each of the plurality of second shape information and the corresponding ones of the plurality of first constants, and the plurality of second shape information. Obtaining the sum of a plurality of terms obtained by multiplying the corresponding ones of the plurality of first constants, and adding the term obtained by multiplying the value of the number of times of use and the fourth constant to the sum. A force information calculation unit that calculates first force information including information on the magnitude of the force in at least one direction acting on the first detection position in the flexible member by an operation including. It is equipped with.
The force information calculation device of another aspect of the present invention is an insertion portion of an endoscope, and is a flexible member having an elongated shape and shape information having a correspondence relationship with the shape of the flexible member. , A shape sensor unit that acquires a plurality of first shape information which is shape information at each of a plurality of measurement points arranged at intervals along the longitudinal direction of the flexible member in the flexible member. And a plurality of second shapes, which are shape information at each of the plurality of calculation points defined at intervals along the longitudinal direction in the flexible member based on the plurality of first shape information. A shape information acquisition unit for acquiring shape information, a usage frequency information acquisition unit for acquiring the number of times the flexible member has been used, a first constant set including a plurality of first constants and a fourth constant, and a set of first constants. The flexibility is achieved by an operation including a storage unit for storing the value of the number of times of use and a multiplication of each of the plurality of second shape information and the corresponding ones of the plurality of first constants. A force information calculation unit for calculating a first force information including information on the magnitude of a force in at least one predetermined direction acting on the first detection position in the member is provided, and the force information calculation unit is a neural unit. It has an input layer, an intermediate layer, and an output layer constituting the network, and the plurality of second shape information and the value of the number of times of use are input to the input layer, and the calculation in the force information calculation unit is performed. In the intermediate layer, a plurality of second shapes by an operation including multiplication of the plurality of second shape information and the corresponding ones of the plurality of first constants and multiplication of the value of the number of times of use and the fourth constant. An operation including a first operation for calculating one term and a multiplication of a plurality of second terms having a correspondence relationship with the plurality of first terms and the corresponding ones of the plurality of first constants in the intermediate layer. A plurality of fourth terms by a second operation for calculating a plurality of third terms by means of an operation including multiplication of the plurality of third terms and the corresponding ones of the plurality of first constants in the output layer. The first force information includes the third operation for calculating the above, and the first force information is information composed of the plurality of fourth terms.

Further, the endoscope system according to one aspect of the present invention includes a force information calculation device and an endoscope having the flexible member as the insertion portion to be inserted into the subject.

Further, in the force information calculation method of one aspect of the present invention, the shape sensor unit is shape information having a correspondence relationship with the shape of a flexible member having an elongated shape and being an insertion portion of an endoscope. A step of acquiring a plurality of first shape information which is shape information at each of a plurality of measurement points arranged at intervals from each other along the longitudinal direction of the flexible member in the flexible member, and shape information. Based on the plurality of first shape information, the acquisition unit is the shape information at each of the plurality of calculation points defined at intervals along the longitudinal direction in the flexible member. 2. The step of acquiring the shape information, the step of acquiring the number of times of use of the flexible member by the usage count information acquisition unit, and the storage unit including a plurality of first constants and fourth constants . The step of storing the constant set of 1 and the value of the number of times of use, and the force information calculation unit correspond to each of the plurality of second shape information and each of the plurality of first constants. To obtain the sum of a plurality of terms obtained by the multiplication of, the plurality of second shape information, and the multiplication of the corresponding ones of the plurality of first constants, the value of the number of times of use, and the fourth. A first containing information on the magnitude of the force in at least one direction acting on the first detection position in the flexible member by an operation that includes adding a term obtained by multiplication with a constant to the sum . It includes a step of calculating the force information of 1.

It is explanatory drawing which shows the schematic structure of the endoscope system which concerns on 1st Embodiment of this invention. It is a functional block diagram which shows the structure of the calculation apparatus main body part of the force information calculation apparatus which concerns on 1st Embodiment of this invention. It is explanatory drawing which shows an example of the hardware composition of the calculation apparatus main body part of the force information calculation apparatus which concerns on 1st Embodiment of this invention. It is explanatory drawing for demonstrating the measurement point and the calculation point in 1st Embodiment of this invention. It is explanatory drawing which shows the example of the 1st detection position and 2nd detection position in 1st Embodiment of this invention. It is a flowchart which shows the force information calculation method which concerns on 1st Embodiment of this invention. It is a functional block diagram which shows the structure of the calculation apparatus main body part in the 1st modification of 1st Embodiment of this invention. It is a functional block diagram which shows the structure of the calculation apparatus main body part in the 2nd modification of the 1st Embodiment of this invention. It is a functional block diagram which shows the structure of the calculation apparatus main body part of the force information calculation apparatus which concerns on the 2nd Embodiment of this invention. It is a functional block diagram which shows the structure of the calculation apparatus main body part in the 1st modification of the 2nd Embodiment of this invention. It is a functional block diagram which shows the structure of the calculation apparatus main body part in the 2nd modification of the 2nd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
(Configuration of endoscope system)
First, the schematic configuration of the endoscope system according to the first embodiment of the present invention will be described. The endoscope system 100 according to the present embodiment is a medical endoscope system. FIG. 1 is an explanatory diagram showing a schematic configuration of the endoscope system 100. As shown in FIG. 1, the endoscope system 100 includes an endoscope 1, a main body device 2, and a display unit 3. The endoscope 1 and the display unit 3 are connected to the main body device 2. The display unit 3 is connected to the main body device 2.

The endoscope 1 has an insertion unit 21 to be inserted into the subject and an operation unit 22 connected to the base end of the insertion unit 21. The insertion portion 21 is a flexible member having an elongated shape. A light source unit and an image pickup device are provided at the tip of the insertion unit 21. The light source unit is composed of a light emitting element such as an LED, and generates illumination light to be applied to the subject.

The reflected light of the subject irradiated with the illumination light is taken into the image sensor through an observation window (not shown). The image pickup device is composed of, for example, a CCD or CMOS, and photoelectrically converts the reflected light of the subject to generate an image pickup signal. The image pickup signal generated by the image pickup element is converted from an analog signal to a digital signal and then output to the main body device 2.

The main body device 2 is configured as a video processor, and includes a control unit 2A that controls the endoscope 1 and performs predetermined image processing on an image pickup signal. The predetermined image processing includes, for example, image adjustment such as gain adjustment, white balance adjustment, gamma correction, contour enhancement correction, and enlargement / reduction adjustment. The main unit 2 outputs an image pickup signal (hereinafter, referred to as an image pickup image) to which a predetermined image processing has been performed to the display unit 3. The display unit 3 is composed of a monitor device or the like, and displays the captured image output from the main body device 2 on the screen.

The endoscope system 100 further includes a force information calculation device 10 according to the present embodiment. The force information calculation device 10 includes a flexible member having an elongated shape, a shape sensor unit, and a calculation device main body unit 11. In the present embodiment, the flexible member is the insertion portion 21 of the endoscope 1.

Further, in the present embodiment, the shape sensor unit is an endoscope insertion shape observation device 5 for observing the insertion state of the insertion unit 21 of the endoscope 1. The endoscope insertion shape observation device 5 includes an observation device main body 51, a receiving antenna 52, and a display unit 53. The receiving antenna 52 and the display unit 53 are connected to the observation device main unit 51. The observation device main body 51 is connected to the main body device 2.

The insertion portion 21 of the endoscope 1 is provided with a plurality of source coils 23 arranged at intervals along the longitudinal direction of the insertion portion 21. The receiving antenna 52 receives a plurality of magnetic fields generated by a plurality of source coils 23 provided in the insertion portion 21 of the endoscope 1. The observation device main body 51 includes a control unit 51A that performs a process of generating an insertion shape image of the insertion unit 21. The control unit 51A acquires shape information at each of a plurality of points (hereinafter referred to as measurement points) defined by the plurality of source coils 23, and based on the acquired shape information, the insertion shape image of the insertion unit 21. To generate. In the present embodiment, the control unit 51A acquires information having a correspondence relationship with the shape of the insertion unit 21, specifically, the coordinates of the measurement point, as the shape information. The coordinates of the measurement point can be calculated based on the magnetic field received by the receiving antenna 52.

Further, the observation device main body 51 outputs the insertion shape image generated by the control unit 51A to the display unit 53. The display unit 53 is composed of a monitor device or the like, and displays an inserted shape image output from the observation device main body 51 on the screen.

The calculation device main body 11 is connected to the observation device main body 51. The observation device main body 51 outputs the shape information acquired by the control unit 51A to the calculation device main body 11. The shape information output by the observation device main body 51 is shape information at each of the plurality of measurement points. Hereinafter, the shape information output by the observation device main body 51 is referred to as the first shape information. In the present embodiment, the first shape information includes information on the coordinates of the measurement points.

The force information calculation device 10 further includes a notification means 18 connected to the calculation device main body 11. The notification means 18 will be described later.

(Structure of the main body of the calculation device)
Next, with reference to FIG. 2, the configuration of the calculation device main body 11 of the force information calculation device 10 will be described. FIG. 2 is a functional block diagram showing the configuration of the calculation device main body 11. The calculation device main body 11 includes a shape information acquisition unit 12, a storage unit 13, a force information calculation unit 14, and a force information transmission unit 15.

The first shape information output by the observation device main body 51 is input to the shape information acquisition unit 12. The shape information acquisition unit 12 is shape information at each of a plurality of calculation points arranged at intervals along the longitudinal direction of the insertion unit 21 in the insertion unit 21 based on the plurality of first shape information. Acquire the second shape information. A plurality of calculation points will be described later.

The storage unit 13 stores a plurality of constants used in the force information calculation unit 14. The force information calculation unit 14 is configured to be able to read out a plurality of constants stored in the storage unit 13, and is arbitrary in the insertion unit 21 based on the plurality of second shape information and the plurality of constants. Calculate information that has a correspondence with the force acting on the position. The force information transmission unit 15 transmits a signal having a correspondence relationship with the information calculated by the force information calculation unit 14 to an external device or the like of the calculation device main body unit 11. The details of the operations of the force information calculation unit 14 and the force information transmission unit 15 will be described later.

Here, the hardware configuration of the calculation device main body 11 will be described with reference to FIG. FIG. 3 is an explanatory diagram showing an example of the hardware configuration of the calculation device main body 11. In the example shown in FIG. 3, the calculation device main body 11 is composed of a processor 11A, a memory 11B, a storage device 11C, and an input / output unit 11D.

The processor 11A is used to execute at least a part of the functions of the shape information acquisition unit 12, the force information calculation unit 14, and the force information transmission unit 15. The processor 11A is configured by, for example, an FPGA (Field Programmable Gate Array). At least a part of the shape information acquisition unit 12, the force information calculation unit 14, and the force information transmission unit 15 may be configured as a circuit block in the FPGA.

The memory 11B is composed of a rewritable volatile storage element such as a RAM. The storage device 11C is composed of a rewritable non-volatile storage device such as a flash memory or a magnetic disk device. The storage unit 13 is composed of at least one of the memory 11B and the storage device 11C. The plurality of constants stored in the storage unit 13 may be configured to be stored in the storage device 11C, or may be read from another storage device (not shown) and stored in the memory 11B. good.

The input / output unit 11D is used to transmit / receive a signal between the calculation device main unit 11 and the outside.

The processor 11A may be configured by a central processing unit (hereinafter referred to as a CPU). In this case, at least a part of the functions of the shape information acquisition unit 12, the force information calculation unit 14, and the force information transmission unit 15 are realized by the CPU reading a program from the storage device 11C or another storage device (not shown) and executing the program. May be done.

Further, the hardware configuration of the calculation device main body 11 is not limited to the example shown in FIG. For example, each of the shape information acquisition unit 12, the force information calculation unit 14, and the force information transmission unit 15 may be configured as separate electronic circuits. Alternatively, the calculation device main body 11 and the main body 2 or the observation device main body 51 may be configured as an integrated device. Also in this case, at least a part of the shape information acquisition unit 12, the force information calculation unit 14, and the force information transmission unit 15 is configured as a circuit block in the FPGA provided in the main body device 2 or the observation device main body unit 51. Alternatively, at least a part of the functions of the shape information acquisition unit 12, the force information calculation unit 14, and the force information transmission unit 15 may be performed by the CPU provided in the main unit 2 or the observation device main unit 51 by the main unit 2 or the observation unit. It may be realized by reading and executing a program from a storage device provided in the device main body 51 or another storage device (not shown).

(Measurement points and calculation points)
Next, measurement points and calculation points will be described with reference to FIGS. 1, 2, and 4. FIG. 4 is an explanatory diagram for explaining the measurement point and the calculation point, and shows a part of the insertion portion 21. As described above, the measurement point is a point defined by the source coil 23 provided in the insertion portion 21. The measurement point may be, for example, a point corresponding to the center position of the source coil 23. Since the plurality of source coils 23 are arranged at intervals in the insertion portion 21 along the longitudinal direction of the insertion portion 21, a plurality of measurement points are also arranged in the insertion portion 21 in the longitudinal direction of the insertion portion 21. They are arranged so as to be spaced apart from each other. In FIG. 4, a plurality of measurement points are indicated by points with the symbol Ps.

Further, the measurement points Ps can be represented as points in a Cartesian coordinate system represented by an X-axis, a Y-axis, and a Z-axis that are orthogonal to each other with an arbitrary point as the origin. In the present embodiment, the first shape information includes the information of the coordinates (Psx, Psy, Psz) of the measurement point Ps. Note that Psx is the value of the X coordinate, Psy is the value of the Y coordinate, and Psz is the value of the Z coordinate.

Further, the plurality of calculation points are points defined by the shape information acquisition unit 12 in order to acquire the second shape information. The positions of the plurality of calculation points may all coincide with the positions of the measurement points Ps. Alternatively, at least a part of the plurality of calculation points may not coincide with the measurement points Ps. In FIG. 4, a plurality of calculation points are indicated by points with the symbol Pc. FIG. 4 shows an example in which a part of the plurality of calculation points Pc is located at a position different from the measurement point Ps in the longitudinal direction of the insertion portion 21.

In FIG. 4, for convenience, the measurement points Ps and the calculation points Pc located at the same position in the longitudinal direction of the insertion portion 21 are drawn separated from each other in the direction orthogonal to the longitudinal direction of the insertion portion 21. However, the measurement point Ps and the calculation point Pc are actually at the same position.

The calculation point Pc can be represented as a point in the Cartesian coordinate system, similarly to the measurement point Ps. In the present embodiment, the second shape information includes the information of the coordinates (Pcx, Pcy, Pcz) of the calculation point Pc. In addition, Pcx is the value of the X coordinate, Pcy is the value of the Y coordinate, and Pcz is the value of the Z coordinate.

Hereinafter, a second method of acquiring shape information will be described. The shape information acquisition unit 12 acquires the second shape information at the calculation point Pc from the first shape information at each of the plurality of measurement points Ps. When the positions of the plurality of calculation points Pc all match the positions of the measurement points Ps, the shape information acquisition unit 12 sets the coordinates of the measurement points Ps to the coordinates of the calculation points Pc, for example, so that the calculation points Pc can be set to the coordinates of the calculation points Pc. Acquire the second shape information including the coordinate information.

Further, when at least a part of the plurality of calculation points Pc is located at a position different from the measurement points Ps, the shape information acquisition unit 12 obtains, for example, a spline interpolation function from the plurality of measurement points Ps and calculates from the obtained functions. By calculating the coordinates of the point Pc, the second shape information including the information of the coordinates of the calculated point Pc is acquired.

The second shape information may further include information on the curvature of the insertion portion 21 at the calculation point Pc. The shape information acquisition unit 12 calculates the curvature of the insertion unit 21 at the calculation point Pc from the coordinates of the plurality of measurement points Ps or the coordinates of the plurality of calculation points Pc, so that the curvature of the insertion unit 21 at the calculation point Pc is calculated. The second shape information including the information of is acquired.

(Operation of force information calculation unit)
Next, the operation of the force information calculation unit 14 will be described with reference to FIG. The force information calculation unit 14 is based on a plurality of second shape information and a plurality of constants stored in the storage unit 13, and information on the magnitude of the force acting on an arbitrary position in the insertion unit 21 in at least one direction. Calculate force information including. In this embodiment, at least one direction is a first direction, a second direction, and a third direction orthogonal to each other. The force information is the first information which is the information of the magnitude of the first direction, the second information which is the information of the magnitude of the force in the second direction, and the magnitude of the force in the third direction. It includes a third piece of information, which is information.

As described above, the second shape information includes the information of the coordinates (Pcx, Pcy, Pcz) of the calculation point Pc. Here, the direction parallel to the X axis is referred to as the X direction, the direction parallel to the Y axis is referred to as the Y direction, and the direction parallel to the Z axis is referred to as the Z direction. In the present embodiment, the first direction is the X direction, the second direction is the Y direction, and the third direction is the Z direction.

Further, in the present embodiment, the force information calculation unit 14 calculates the force information at each of the plurality of positions in the insertion unit 21. The plurality of positions may or may not coincide with the positions of the plurality of calculation points Pc. Further, the number of the plurality of positions may or may not match the number of the plurality of calculation points Pc.

Here, an arbitrary position in the insertion portion 21 is referred to as a first detection position, and a position different from the first detection position in the insertion portion 21 is referred to as a second detection position. FIG. 5 shows an example of the first detection position P1 and the second detection position P2. Hereinafter, a method of calculating the first force information which is the force information at the first detection position P1 and a method of calculating the second force information which is the force information at the second detection position P2 will be described.

First, a method of calculating the first force information will be described. In the present embodiment, the force information calculation unit 14 applies a force F1 (see FIG. 5) acting on the first detection position P1 based on n second shape information (n is an integer of 2 or more). calculate. Hereinafter, the n second shape information will be referred to as the first to nth second shape information for convenience. In the present embodiment, the force information calculation unit 14 uses the following equations (1) to (3) to form the X-direction component F1x of the force F1, the Y-direction component F1y of the force F1, and the force F1. The component F1z in the Z direction is calculated.

F1x = A11 / X1 + A12 / X2 + A13 / X3
＋ ・ ・ ・ ＋ A1n ・ Xn ＋ C1… (1)
F1y = A21 / X1 + A22 / X2 + A23 / X3
＋ ・ ・ ・ ＋ A2n ・ Xn ＋ C2… (2)
F1z = A31 / X1 + A32 / X2 + A33 / X3
＋ ・ ・ ・ ＋ A3n ・ Xn ＋ C3… (3)
In the formulas (1) to (3), Xn represents the nth second shape information, A1n, A2n, and A3n are constants for multiplying Xn, respectively, and C1, C2, and C3 are constant terms. Is.

Xn may be a three-dimensional column vector including three coordinate values of the calculation point Pc corresponding to the nth second shape information as elements. In this case, A1n, A2n, and A3n are three-dimensional row vectors including three constants for multiplying each of the three coordinate values of Xn as elements. In the present embodiment, for convenience, a row vector including a constant as an element is also referred to as a constant. Alternatively, Xn may be a value calculated by substituting the three coordinate values of the calculation point Pc into a predetermined formula. In this case, A1n, A2n, and A3n are constants, respectively.

The magnitude and direction of the force F1 can be calculated from F1x, F1y, F1z. Further, F1x also calculates the magnitude of the force acting on the first detection position P1 in the X direction, and F1y calculates the magnitude of the force acting on the first detection position P1 in the Y direction. F1z is also a calculation of the magnitude of the force acting on the first detection position P1 in the Z direction. The force information calculation unit 14 may use the magnitude and direction of the force F1 as the first force information, or may use F1x, F1y, and F1z as the first force information.

In the present embodiment, when calculating the first force information, the second shape information, that is, the three coordinate values of the calculation point Pc or the value calculated according to a predetermined rule from the three coordinate values is multiplied. The constant is called the first constant. The storage unit 13 stores a first set of constants including a plurality of first constants. In the present embodiment, the first constant set includes A1n, A2n, and A3n as the first constant. Further, the first constant set further includes constant terms C1, C2, and C3.

Here, the information of F1x is referred to as the first information, the information of F1y is referred to as the second information, and the information of F1z is referred to as the third information. The plurality of first constants include a plurality of constants for calculating the first information, a plurality of constants for calculating the second information, and a plurality of constants for calculating the third information. Includes.

Next, a method of calculating the second force information will be described. In the present embodiment, the force information calculation unit 14 has a force F2 acting on the second detection position P2 based on n second shape information as well as the first force information (see FIG. 5). Is calculated. In the present embodiment, the force information calculation unit 14 uses the following equations (4) to (6) to form the X-direction component F2x of the force F2, the Y-direction component F2y of the force F2, and the force F2. The component F2z in the Z direction is calculated.

F2x = A41 / X1 + A42 / X2 + A43 / X3
＋ ・ ・ ・ ＋ A4n ・ Xn ＋ C4… (4)
F2y = A51 / X1 + A52 / X2 + A53 / X3
＋ ・ ・ ・ ＋ A5n ・ Xn ＋ C5… (5)
F2z = A61 / X1 + A62 / X2 + A63 / X3
＋ ・ ・ ・ ＋ A6n ・ Xn ＋ C6… (6)
In the formulas (4) to (6), A4n, A5n, and A6n are constants for multiplying Xn, respectively, and C1, C2, and C3 are constant terms. Similar to A1n, A2n, and A3n, A4n, A5n, and A6n may be three-dimensional row vectors including three constants for multiplying each of the three coordinate values of Xn as elements, or are constants. There may be.

The magnitude and direction of the force F2 can be calculated from F2x, F2y, F2z. Further, F2x also calculates the magnitude of the force acting on the second detection position P2 in the X direction, and F2y calculates the magnitude of the force acting on the second detection position P2 in the Y direction. F2z is also a calculation of the magnitude of the force acting on the second detection position P2 in the Z direction. The force information calculation unit 14 may use the magnitude and direction of the force F2 as the second force information, or may use F2x, F2y, and F2z as the second force information.

In the present embodiment, when calculating the second force information, the second shape information, that is, the three coordinate values of the calculation point Pc or the value calculated according to a predetermined rule from the three coordinate values is multiplied. The constant is called the second constant. The storage unit 13 stores a second set of constants including a plurality of second constants. In the present embodiment, the second constant set includes A4n, A5n, and A6n as the second constant. The second set of constants further includes the constant terms C4, C5 and C6.

Here, the information of F2x is referred to as the first information, the information of F2y is referred to as the second information, and the information of F2z is referred to as the third information. The plurality of second constants include a plurality of constants for calculating the first information, a plurality of constants for calculating the second information, and a plurality of constants for calculating the third information. Includes.

The calculation method of F1x, F1y, F1z, F2x, F2y, and F2z is not limited to the examples shown in the equations (1) to (6). For example, each of the formulas (1) to (6) may contain a second-order or higher-order term such as Xn ² , or may include an interaction term such as Xm · Xn. Note that m is an integer of 1 or more different from n, and is a number that satisfies the relationship of, for example, m = n-1.

Up to this point, a method of calculating the first force information at the first detection position P1 and the second force information at the second detection position P2 has been described. The above description also applies to the calculation of force information at any detection position P3 (see FIG. 5) other than the first and second detection positions P1 and P2.

Further, up to this point, the case where the second shape information includes only the information of the coordinates of the calculation point Pc has been described as an example. As described above, the second shape information may include information on the curvature of the insertion portion 21 at the calculation point Pc. In this case, Xn may be a four-dimensional column vector including the three coordinate values of the calculation point Pc corresponding to the nth second shape information and the curvature of the insertion portion 21 as elements. In this case, A1n, A2n, A3n, A4n, A5n, and A6n each have three constants for multiplying each of the three coordinate values of Xn and one constant for multiplying the curvature of the insertion portion 21 of Xn. It is a four-dimensional row vector including as. The first constant set and the second constant set each include a constant for multiplying the curvature of the insertion portion 21.

(Calculation method of the first and second constants)
Next, a method of calculating the first and second constants will be described. The first constant is calculated, for example, as follows. First, the second shape information when the insertion portion 21 has a certain shape is acquired, and the force acting on the first detection position P1 (hereinafter referred to as the first force) is acquired. The first force may be obtained by an experiment using a sensor or the like, or may be obtained by a simulation. In the simulation, the first force can be obtained, for example, by solving simultaneous equations defined by the balance of moments at a plurality of positions in the insertion portion 21 by using a convergence operation.

Next, the shape of the insertion portion 21 is changed. The acquisition of the second shape information and the first force described above is performed every time the shape of the insertion portion 21 is changed. As a result, a plurality of sets of the second shape information and the first force are prepared. Next, the first constants A1n, A2n, and A3n are calculated by substituting a plurality of sets of the second shape information and the first force into the equations (1) to (3). The component in the X direction of the first force is substituted into F1x in the equation (1), the component in the Y direction of the first force is substituted into F1y in the equation (2), and F1z in the equation (3) is substituted. Is substituted with the Z-direction component of the first force. The first constants A1n, A2n, and A3n can be calculated by, for example, the least squares method. The constant terms C1, C2, and C3 of the equations (1) to (3) are calculated at the same time as the first constants A1n, A2n, and A3n.

The second constant is calculated, for example, as follows. First, the second shape information when the insertion portion 21 has a certain shape is acquired, and the force acting on the second detection position P2 (hereinafter referred to as the second force) is acquired. The second force acquisition method is the same as the first force acquisition method.

Next, the shape of the insertion portion 21 is changed. The acquisition of the second shape information and the second force described above is performed every time the shape of the insertion portion 21 is changed. As a result, a plurality of sets of the second shape information and the second force are prepared. Next, the second constants A4n, A5n, and A6n are calculated by substituting a plurality of sets of the second shape information and the second force into the equations (4) to (6). The component in the X direction of the second force is substituted into F2x in the equation (4), the component in the Y direction of the second force is substituted into F2y in the equation (5), and F2z in the equation (6) is substituted. Is substituted with the Z-direction component of the second force. The second constants A4n, A5n, and A6n can be calculated by, for example, the least squares method. The constant terms C4, C5, and C6 of the equations (4) to (6) are calculated at the same time as the second constants A4n, A5n, and A6n.

When calculating the first and second constants, it is necessary to carry out an experiment or simulation while changing the shape of the insertion portion 21 in order to obtain the first and second forces. Therefore, the first and second constants are calculated before the use of the endoscope 1. Further, the first constant set including the calculated first constants and the second constant set including the calculated second constants are stored in the storage unit 13 before the use of the endoscope 1. Will be done.

(Force information calculation method)
Next, a force information calculation method according to the present embodiment will be described with reference to FIGS. 1, 2, and 6. FIG. 6 is a flowchart showing a force information calculation method according to the present embodiment. The force information calculation method according to the present embodiment includes steps S11, S12, S13, and S14 shown in FIG. Steps S11, S12, S13, and S14 are executed in this order.

Step S11 is a step in which the storage unit 13 stores a plurality of constants. In the present embodiment, the storage unit 13 stores, as a plurality of constants, a first constant set including a plurality of first constants and a second constant set including a plurality of second constants.

Step S12 is a step in which the shape sensor unit, that is, the endoscope insertion shape observation device 5, acquires the first shape information. Step S13 is a step in which the shape information acquisition unit 12 acquires the second shape information. Step S14 is a step in which the force information calculation unit 14 calculates force information. In the present embodiment, the force information calculation unit 14 calculates the first and second force information as the force information.

Note that step S11 is executed before the use of the endoscope 1. Steps S12 to S14 are repeatedly executed when the endoscope 1 is used.

(Operation of force information transmission unit)
Next, the operation of the force information transmitting unit 15 will be described with reference to FIGS. 1 and 2. As described above, the force information transmission unit 15 transmits a signal having a correspondence relationship with the force information calculated by the force information calculation unit 14 to, for example, a notification means 18 such as an external device of the calculation device main body unit 11. When the notification means 18 is a display unit configured by a monitoring device or the like, the force information transmission unit 15 may output a signal of information on the magnitude of the force acting on a predetermined detection position to the display unit. The display unit displays information on the magnitude of the output force on the screen.

Further, the force information transmission unit 15 compares the magnitude of the force acting on the predetermined detection position with the predetermined threshold value, and notifies that the magnitude of the force is equal to or greater than the predetermined threshold value (hereinafter, notification signal). ) May be output to the notification means 18. When the notifying means 18 is a display unit, the display unit may display a warning on the screen based on the notification signal. Further, when the notification unit 18 is an alarm unit composed of a speaker or the like, the alarm unit may emit a warning sound based on the notification signal.

Further, the force information transmission unit 15 may output a signal of information on the magnitude of the force acting on a predetermined detection position or a notification signal to the observation device main body unit 51. In this case, the control unit 51A of the observation device main body 51 may synthesize information on the magnitude of the force, a warning, etc. into the inserted shape image, or may display the magnitude of the force on the screen of the display unit 53. Information, warnings, etc. may be output to the display unit 53. The control unit 51A may compare the magnitude of the force acting on the predetermined detection position with the predetermined threshold value.

Similarly, the force information transmission unit 15 may output a signal of information on the magnitude of the force acting on a predetermined detection position or a notification signal to the main body device 2. In this case, the control unit 2A of the main unit 2 may synthesize the force magnitude information, a warning, etc. into the captured image, or the force magnitude information or the force magnitude information to be displayed on the screen of the display unit 3. A warning or the like may be output to the display unit 53. The magnitude of the force acting on the predetermined detection position and the predetermined threshold value may be compared with each other in the control unit 2A.

Further, the main body device 2 may have a subject influence determination unit for determining the influence on the subject based on the captured image and the information on the magnitude of the force.

Further, the endoscope system 100 may include an insertion robot that supports at least a part of the insertion / removal operation of the insertion portion 21 of the endoscope 1. The insertion robot is provided with a plurality of arms for supporting the insertion unit 21 and the operation unit 22 of the endoscope 1, a mechanism for inserting and removing the insertion unit 21 inserted into the subject, and the operation unit 22. A mechanism for operating the operation knob is provided. The force information transmission unit 15 outputs a signal having a correspondence relationship with the force information calculated by the force information calculation unit 14 to the insertion robot control device. The insertion robot control device performs the insertion / removal operation of the insertion unit 21 based on the above signal, the insertion shape image or the first shape information output by the observation device main body 51, the captured image output by the main body 2, and the like. Control the insertion robot so that it can be done smoothly.

(Action and effect)
Next, the actions and effects of the force information calculation device 10, the endoscope system 100, and the force information calculation method according to the present embodiment will be described. In the present embodiment, the force information calculation unit 14 includes information on the magnitude of the force acting on the first detection position P1 in the insertion unit 21 of the endoscope 1 which is a flexible member. The first force information is calculated. In the present embodiment, the first detection position P1 is not limited to the tip of the insertion portion 21, and can be arbitrarily set. Thereby, according to the present embodiment, it is possible to calculate information on the magnitude of the force acting on an arbitrary position.

Further, in the present embodiment, the force information calculation unit 14 corresponds to the operations shown in the equations (1) to (3), that is, the plurality of second shape information and the plurality of first constants. The first force information is calculated by an operation including multiplication of each other. Further, in the present embodiment, the above calculation is obtained by multiplying the corresponding ones of the plurality of second shape information and the plurality of first constants as shown in the equations (1) to (3). It includes finding the sum of multiple terms. As described above, in the present embodiment, the first force information is obtained by multiplication and addition. As a result, according to the present embodiment, the load for calculating the first force information can be reduced as compared with the case where the first force information is calculated by the simulation using the convergence calculation or the like. , The calculation time of the first force information can be shortened. Further, according to the present embodiment, the first force information can be calculated in a substantially constant time regardless of the position of the first detection position P1 and the shape of the insertion portion 21. From these facts, according to this embodiment, it becomes possible to calculate the first force information in real time when the endoscope 1 is used.

Further, in the present embodiment, the first constant is stored in the storage unit 13 in advance. As described above, the calculation of the first constant is performed by preparing a plurality of sets of the second shape information and the first force acting on the first detection position P1 while changing the shape of the insertion portion 21. Will be done. The larger the number of this set, the better the calculation accuracy of the first force information. In the present embodiment, since the calculation of the first constant can be performed other than when the endoscope 1 is used, it is easy to increase the number of pairs of the second shape information and the first force. Is. Thereby, according to the present embodiment, the calculation accuracy of the first force information can be improved.

Further, in the present embodiment, the force information calculation unit 14 acts on a second detection position P2 different from the first detection position P1 in the insertion unit 21 of the endoscope 1 which is a flexible member. A second force information including information on the magnitude of the force in one direction is calculated. In the present embodiment, the position and number of the second detection position P2 can be arbitrarily set as long as the requirement that the second detection position P2 is different from the first detection position P1 is satisfied. Thereby, according to the present embodiment, it is possible to calculate information on the magnitude of the force acting on a plurality of positions.

Further, in the present embodiment, similarly to the first force information, the load for calculating the second force information can be reduced, and the calculation time of the second force information can be shortened. .. As a result, according to the present embodiment, even when the number of force information to be calculated is large, the load for calculating a plurality of force information is increased as compared with the case where a plurality of force information is calculated by simulation. It can be reduced and the calculation time of a plurality of force information can be shortened. This makes it possible to calculate a plurality of force information in real time when the endoscope 1 is used according to the present embodiment.

(First modification)
Next, a first modification of the present embodiment will be described with reference to FIG. 7. FIG. 7 is a functional block diagram showing the configuration of the calculation device main body 11 in the first modification. In the first modification, the calculation device main body 11 includes a rigidity information acquisition unit 16 in addition to the shape information acquisition unit 12, the storage unit 13, the force information calculation unit 14, and the force information transmission unit 15. The rigidity information acquisition unit 16 acquires rigidity values such as bending rigidity of the insertion unit 21 (see FIG. 1) of the endoscope 1 which is a flexible member, and outputs the acquired rigidity values to the storage unit 13. The storage unit 13 stores the rigidity value output by the rigidity information acquisition unit 16. The force information calculation unit 14 is configured to be able to read out the rigidity value stored in the storage unit 13.

The rigidity value can be measured, for example, by using a sensor or the like before using the endoscope 1. The rigidity information acquisition unit 16 may acquire the rigidity value directly from the measuring instrument that measures the rigidity value. Alternatively, the rigidity information acquisition unit 16 may acquire the rigidity value via a user interface unit (not shown) connected to the calculation device main body unit 11. In this case, the user inputs the measurement result of the rigidity value to the user interface unit.

Further, in the first modification, the first constant set and the second constant set stored in the storage unit 13 each include a third constant. The force information calculation unit 14 multiplies the rigidity value by the third constant of the first constant set, and the term obtained by this multiplication (hereinafter, referred to as the first rigidity value term) is a plurality of second. The first force information is calculated by adding to the sum of the plurality of terms obtained by multiplying the shape information of 2 and the corresponding ones of the plurality of first constants. Specifically, for example, the force information calculation unit 14 adds a first rigidity value term to each of the equations (1) to (3), so that the force F1 acting on the first detection position is in the X direction. The component F1x, the component F1y in the Y direction of the force F1, and the component F1z in the Z direction of the force F1 are calculated.

Similarly, the force information calculation unit 14 multiplies the rigidity value by the third constant of the second constant set, and the term obtained by this multiplication (hereinafter referred to as the second rigidity value term) is obtained. The second force information is calculated by adding to the sum of the plurality of terms obtained by multiplying the plurality of second shape information and the plurality of corresponding ones of the second constants. Specifically, for example, the force information calculation unit 14 adds a second rigidity value term to each of the equations (4) to (6), so that the force F2 acting on the second detection position is in the X direction. The component F2x, the component F2y in the Y direction of the force F2, and the component F2z in the Z direction of the force F2 are calculated.

The third constant of the first constant set can be calculated at the same time as the first constant when calculating the first constant. Specifically, for example, while changing the shape of the insertion portion 21 of the endoscope 1, a plurality of sets of the second shape information and the first force and the rigidity value acting on the first detection position are prepared. By substituting the second shape information and a plurality of sets of the first force and the stiffness value into the equation in which the first stiffness value term is added to each of the equations (1) to (3), the first constant And the third constant are calculated.

Similarly, the third constant of the second constant set can be calculated at the same time as the second constant when calculating the second constant. Specifically, for example, while changing the shape of the insertion portion 21 of the endoscope 1, a plurality of sets of the second shape information and the second force and the rigidity value acting on the second detection position are prepared. By substituting the second shape information and the plurality of sets of the second force and the stiffness value into the equation in which the second stiffness value term is added to each of the equations (4) to (6), the second constant And the third constant are calculated.

The rigidity value of the insertion portion 21 of the endoscope 1 varies to some extent from product to product. According to the first modification, for example, when the endoscope 1 is replaced and used, the force information can be calculated accurately by using the rigidity value of the insertion portion 21 of the endoscope 1 to be used. can.

(Second modification)
Next, a second modification of the present embodiment will be described with reference to FIG. FIG. 8 is a functional block diagram showing the configuration of the calculation device main body 11 in the second modification. The configuration of the calculation device main body 11 in the second modification is different from the first modification in the following points. In the second modification, the calculation device main body 11 is used in addition to the shape information acquisition unit 12, the storage unit 13, the force information calculation unit 14, the force information transmission unit 15, and the rigidity information acquisition unit 16. It is equipped with 17. The usage count information acquisition unit 17 acquires the usage count of the insertion unit 21 (see FIG. 1) of the endoscope 1 which is a flexible member, and outputs the acquired usage count value to the storage unit 13. The storage unit 13 stores the value of the number of times of use output by the number of times of use information acquisition unit 17. The force information calculation unit 14 is configured to be able to read out the value of the number of times of use stored in the storage unit 13.

For example, the main body device 2 (see FIG. 1) is configured to be able to store the number of times the endoscope 1 has been used. The usage count information acquisition unit 17 is configured to be able to acquire the usage count value of the endoscope 1 stored in the main body device 2.

Further, in the second modification, the first constant set and the second constant set stored in the storage unit 13 each include a fourth constant. The force information calculation unit 14 multiplies the rigidity value by the fourth constant of the first constant set, and the term obtained by this multiplication (hereinafter, referred to as the first number of times of use term) is a plurality of second. The first force information is calculated by adding to the sum of the plurality of terms obtained by multiplying the shape information of 2 and the corresponding ones of the plurality of first constants. Specifically, for example, the force information calculation unit 14 acts on the first detection position by adding the first rigidity value term and the first usage frequency term to each of the equations (1) to (3). The component F1x in the X direction of the force F1 to be applied, the component F1y in the Y direction of the force F1, and the component F1z in the Z direction of the force F1 are calculated.

Similarly, the force information calculation unit 14 multiplies the value of the number of times of use by the fourth constant of the second set of constants, and the term obtained by this multiplication (hereinafter referred to as the second number of times of use term) is used. , The second force information is calculated by adding to the sum of the plurality of terms obtained by multiplying each of the plurality of second shape information by the corresponding second constant. Specifically, for example, the force information calculation unit 14 acts on the second detection position by adding the second rigidity value term and the second usage frequency term to each of the equations (4) to (6). The component F2x in the X direction of the force F2, the component F2y in the Y direction of the force F2, and the component F2z in the Z direction of the force F2 are calculated.

The fourth constant of the first constant set can be calculated at the same time as the first constant when calculating the first constant. Specifically, for example, while changing the shape of the insertion portion 21 of the endoscope 1, a plurality of first force, rigidity value, and number of times of use value acting on the second shape information and the first detection position. Prepare a set, and apply a plurality of sets of the second shape information, the first force, the rigidity value, and the number of times of use to the first rigidity value term and the first one in each of the equations (1) to (3). The first constant, the third constant, and the fourth constant are calculated by substituting into the equation to which the frequency of use term is added.

Similarly, the fourth constant of the second constant set can be calculated at the same time as the second constant when calculating the second constant. Specifically, for example, while changing the shape of the insertion portion 21 of the endoscope 1, a plurality of second force, rigidity value, and number of times of use value acting on the second shape information and the second detection position. Prepare a set, and apply a plurality of sets of the second shape information, the second force, the rigidity value, and the number of times of use to the second rigidity value term and the second set in each of the equations (4) to (6). The second constant, the third constant, and the fourth constant are calculated by substituting into the equation to which the frequency of use term is added.

When the same endoscope 1 is continuously used, the rigidity value of the insertion portion 21 of the endoscope 1 changes depending on the number of times of use. According to the second modification, the force information can be calculated accurately even when the rigidity value of the insertion portion 21 changes depending on the number of times the endoscope 1 is used.

[Second Embodiment]
(Structure of force information calculation unit)
Next, the force information calculation device according to the second embodiment of the present invention will be described. First, with reference to FIG. 9, the configuration of the force information calculation device 10 according to the present embodiment will be described as being different from the first embodiment. FIG. 9 is a functional block diagram showing the configuration of the calculation device main body 11 of the force information calculation device 10 according to the present embodiment. In the present embodiment, the calculation device main body 11 includes a force information calculation unit 140 instead of the force information calculation unit 14 in the first embodiment. Like the force information calculation unit 14, the force information calculation unit 140 is configured to be able to read out a plurality of constants stored in the storage unit 13, and the plurality of constants and the shape information acquisition unit 12 output the plurality of constants. Based on the plurality of second shape information, information having a correspondence relationship with the force acting on an arbitrary position in the insertion portion 21 (see FIG. 1) of the endoscope 1 is calculated.

As shown in FIG. 9, the force information calculation unit 140 has an input layer 141, an intermediate layer 142, and an output layer 143 that form a neural network. The input layer 141, the intermediate layer 142, and the output layer 143 are arranged in this order from the shape information acquisition unit 12 side. Further, the input layer 141, the intermediate layer 142, and the output layer 143 each include a plurality of units. In FIG. 9, the units are represented by circles. Note that FIG. 9 schematically shows the configurations of the input layer 141, the intermediate layer 142, and the output layer 143, and the number of units is not limited to the example shown in FIG.

The intermediate layer 142 may include a plurality of layers. In FIG. 9, a plurality of units arranged in a row in the vertical direction in FIG. 9 form one layer. In the example shown in FIG. 9, the intermediate layer 142 includes three layers. Hereinafter, the layer closest to the input layer 141 side in FIG. 9 is referred to as a first layer, the layer adjacent to the right side of the first layer in FIG. 9 is referred to as a second layer, and the right side of the second layer in FIG. The layer adjacent to is called the third layer. The number of layers included in the intermediate layer 142 is not limited to the example shown in FIG.

(Operation of force information calculation unit)
Next, the operation of the force information calculation unit 140 will be described. In the present embodiment, the plurality of second shape information is input to the input layer 141. The plurality of second shape information is input to different units included in the input layer 141, respectively.

In the intermediate layer 142, the first operation for calculating a plurality of first terms is performed in the first layer. Specifically, a plurality of second shape information input to the plurality of units of the input layer 141 is input to any one unit constituting the first layer. In this unit, for example, the first term is calculated by multiplying the corresponding ones of the plurality of second shape information and the plurality of constants and obtaining the sum of the plurality of terms obtained by the multiplication.

In the intermediate layer 142, in the second layer and the third layer, a second operation for calculating a plurality of third terms is performed using a plurality of second terms having a correspondence relationship with the plurality of first terms. In the second layer, the plurality of first terms themselves are used as the plurality of second terms. In the third layer, as the plurality of second terms, a plurality of terms calculated by the calculation using the first term in the second layer (hereinafter referred to as the initial third term) are used. The plurality of third terms are calculated in the third layer.

Specifically, a plurality of first terms are input to any one unit constituting the second layer. In this unit, for example, the initial third term is calculated by multiplying the corresponding ones of the plurality of first terms and the plurality of constants and obtaining the sum of the plurality of terms obtained by the multiplication.

Further, a plurality of initial third terms are input to any one unit constituting the third layer. In this unit, for example, the third term is calculated by multiplying a plurality of initial third terms and corresponding ones of a plurality of constants and obtaining the sum of the plurality of terms obtained by the multiplication.

In the output layer 143, a third operation for calculating a plurality of fourth terms is performed. Specifically, a plurality of third terms are input to any one unit constituting the output layer 143. In this unit, for example, the fourth term is calculated by multiplying the corresponding ones of a plurality of third terms and a plurality of constants and obtaining the sum of the plurality of terms obtained by the multiplication.

In the present embodiment, the first force information is information composed of a plurality of fourth terms calculated by the output layer 143. Specifically, for example, the output layer 143 has a plurality of fourth terms, that is, a component in the X direction of the force acting on the first detection position and a component in the Y direction of the force acting on the first detection position. , The Z-direction component of the force acting on the first detection position is calculated. The force information calculation unit 140 may use the magnitude and direction of the force acting on the first detection position calculated from the X-direction component, the Y-direction component, and the Z-direction component as the first information. The above-mentioned component in the X direction, the component in the Y direction, and the Z direction may be used as the first force information.

Further, in the present embodiment, when calculating the first force information, the second shape information, the first term, the second term (the first term or the initial third term), and the third term are multiplied. The constant is called the first constant.

Next, a method of calculating the second force information in the present embodiment will be described. The method for calculating the second force information in the present embodiment is basically the same as the method for calculating the first force information. In the present embodiment, the second force information is information composed of a plurality of fourth terms calculated by the output layer 143. Specifically, for example, the output layer 143 has a plurality of fourth terms, that is, a component in the X direction of the force acting on the second detection position and a component in the Y direction of the force acting on the second detection position. , The Z-direction component of the force acting on the second detection position is calculated. The force information calculation unit 140 may use the magnitude and direction of the force acting on the second detection position calculated from the above-mentioned component in the X direction, the component in the Y direction, and the component in the Z direction as the second information. The above-mentioned component in the X direction, the component in the Y direction, and the Z direction may be used as the second force information.

In the present embodiment, when calculating the second force information, a constant for multiplying the second shape information, the first term, the second term (the first term or the initial third term), and the third term is used. , The second constant.

The calculation method of the first term, the initial third term, the third term and the fourth term is not limited to the above example. For example, the first term may be calculated by adding a constant term to the sum of the plurality of terms obtained by multiplying the plurality of second shape information and the corresponding ones of the plurality of constants. , The above sum or the sum plus a constant term may be assigned to the activation function. As the activation function, for example, a sigmoid function or a ReLU function is used. The initial terms 3, 3 and 4 may also be calculated by the same method as the calculation method of the above-mentioned first term.

(Calculation method of the first and second constants)
Next, a method of calculating the first and second constants in the present embodiment will be described. In the present embodiment, the first constant and the second constant are calculated by machine learning such as deep learning. The first constant is calculated, for example, as follows. First, the second shape information when the insertion portion 21 has a certain shape is acquired, and the first force acting on the first detection position is acquired. The method of acquiring the first force is the same as that of the first embodiment.

Next, the shape of the insertion portion 21 is changed. The acquisition of the second shape information and the first force described above is performed every time the shape of the insertion portion 21 is changed. As a result, a plurality of sets of the second shape information and the first force are prepared. Next, machine learning is executed using a plurality of sets of the second shape information and the first force as teacher data. As a result, a plurality of first constants are calculated.

The second constant is calculated, for example, as follows. First, the second shape information when the insertion portion 21 has a certain shape is acquired, and the second force acting on the second detection position is acquired. The second force acquisition method is the same as that of the first embodiment.

Next, the shape of the insertion portion 21 is changed. The acquisition of the second shape information and the second force described above is performed every time the shape of the insertion portion 21 is changed. As a result, a plurality of sets of the second shape information and the second force are prepared. Next, machine learning is executed using a plurality of sets of the second shape information and the second force as teacher data. As a result, a plurality of second constants are calculated.

(Action and effect)
Next, the actions and effects peculiar to the present embodiment will be described. In the present embodiment, the first and second force information is calculated by the force information calculation unit 140 having the input layer 141, the intermediate layer 142, and the output layer 143 constituting the neural network. As a result, according to the present embodiment, the load for calculating the first and second force information can be reduced as compared with the case where the first and second force information is calculated by simulation. , The calculation time of the first and second force information can be shortened. Further, according to the present embodiment, the first and second force information can be calculated in a substantially constant time regardless of the positions of the first and second detection positions and the shape of the insertion portion 21. .. From these facts, according to this embodiment, it becomes possible to calculate the first and second force information in real time when the endoscope 1 is used.

Further, in the present embodiment, as in the first embodiment, the first and second constants are stored in the storage unit 13 in advance. As described above, in the calculation of the first constant, while changing the shape of the insertion portion 21, a plurality of sets of the second shape information and the first force are prepared, and the second shape information and the first force are calculated. It is done by executing machine learning using multiple sets of. As teacher data. Further, in the calculation of the second constant, while changing the shape of the insertion portion 21, a plurality of sets of the second shape information and the second force are prepared, and the second shape information and the plurality of second forces are calculated. It is done by executing machine learning using the set as teacher data. In the present embodiment, since the calculation of the first and second constants can be performed other than when the endoscope 1 is used, it is easy to increase the number of teacher data. Thereby, according to the present embodiment, the calculation accuracy of the first and second force information can be improved.

(First modification)
Next, with reference to FIG. 10, a first modification of the present embodiment will be described. FIG. 10 is a functional block diagram showing the configuration of the calculation device main body 11 in the first modification. In the first modification, the calculation device main body 11 includes a rigidity information acquisition unit 16 in addition to the shape information acquisition unit 12, the storage unit 13, the force information calculation unit 140, and the force information transmission unit 15. The function of the rigidity information acquisition unit 16 is the same as the function described in the first modification of the first embodiment.

In the first modification, the storage unit 13 stores the rigidity value acquired and output by the rigidity information acquisition unit 16. The force information calculation unit 140 is configured to be able to read out the rigidity value stored in the storage unit 13.

Further, in the first modification, the first constant set and the second constant set stored in the storage unit 13 each include a third constant.

Further, in the first modification, in the input layer 141 of the force information calculation unit 140, in addition to the plurality of second shape information, the rigidity value acquired by the rigidity information acquisition unit 16 and stored in the storage unit 13 is stored in the input layer 141. Entered. As described above, in the intermediate layer 142, the first operation for calculating the plurality of first terms is performed in the first layer. In the first modification, a plurality of second shape information and rigidity values are input to any one unit constituting the first layer. When calculating the first force information, in this unit, for example, the corresponding ones of the plurality of second shape information and the plurality of first constants are multiplied, and the rigidity value and the first constant set are calculated. Multiply by a third constant. Then, the first term is calculated by obtaining the sum of the plurality of terms obtained by multiplication. Further, when calculating the second force information, in this unit, for example, the corresponding ones of the plurality of second shape information and the plurality of second constants are multiplied, and the rigidity value and the second constant are calculated. Multiply by the third constant of the set. Then, the first term is calculated by obtaining the sum of the plurality of terms obtained by multiplication.

The third constant of the first constant set can be calculated at the same time as the first constant when calculating the first constant. Specifically, for example, while changing the shape of the insertion portion 21 of the endoscope 1, a plurality of sets of the second shape information and the first force and the rigidity value acting on the first detection position are prepared. By executing machine learning using these plurality of sets as teacher data, the first constant and the third constant are calculated.

Similarly, the third constant of the second constant set can be calculated at the same time as the second constant when calculating the second constant. Specifically, for example, while changing the shape of the insertion portion 21 of the endoscope 1, a plurality of sets of the second shape information and the second force and the rigidity value acting on the second detection position are prepared. By executing machine learning using these plurality of sets as teacher data, a second constant and a third constant are calculated.

(Second modification)
Next, a second modification of the present embodiment will be described with reference to FIG. FIG. 11 is a functional block diagram showing the configuration of the calculation device main body 11 in the second modification. The configuration of the calculation device main body 11 in the second modification is different from the first modification in the following points. In the second modification, the calculation device main body 11 is used in addition to the shape information acquisition unit 12, the storage unit 13, the force information calculation unit 140, the force information transmission unit 15, and the rigidity information acquisition unit 16. It is equipped with 17. The function of the usage count information acquisition unit 17 is the same as the function described in the second modification of the first embodiment.

In the second modification, the storage unit 13 stores the value of the number of times of use acquired and output by the number of times of use information acquisition unit 17. The force information calculation unit 140 is configured to be able to read out the value of the number of times of use stored in the storage unit 13.

Further, in the second modification, the first constant set and the second constant set stored in the storage unit 13 each include a fourth constant.

Further, in the second modification, in addition to the plurality of second shape information and rigidity values, the input layer 141 of the force information calculation unit 140 is acquired by the usage frequency information acquisition unit 17 and stored in the storage unit 13. The value of the number of uses is entered. As described above, in the intermediate layer 142, the first operation for calculating the plurality of first terms is performed in the first layer. In the second modification, a plurality of second shape information, rigidity values, and usage count values are input to any one unit constituting the first layer. When calculating the first force information, in this unit, for example, the corresponding ones of the plurality of second shape information and the plurality of first constants are multiplied, and the rigidity value and the first constant set are calculated. It is multiplied by the third constant, and further, the value of the number of times of use is multiplied by the fourth constant of the first constant set. Then, the first term is calculated by obtaining the sum of the plurality of terms obtained by multiplication. Further, when calculating the second force information, in this unit, for example, the corresponding ones of the plurality of second shape information and the plurality of second constants are multiplied, and the rigidity value and the second constant are calculated. It multiplies the third constant of the set, and further multiplies the value of the number of uses by the fourth constant of the second constant set. Then, the first term is calculated by obtaining the sum of the plurality of terms obtained by multiplication.

The fourth constant of the first constant set can be calculated at the same time as the first constant when calculating the first constant. Specifically, for example, while changing the shape of the insertion portion 21 of the endoscope 1, a plurality of first force, rigidity value, and number of times of use value acting on the second shape information and the first detection position. A first constant and a third constant are calculated by preparing a set and executing machine learning using the plurality of sets as teacher data.

Similarly, the third constant of the second constant set can be calculated at the same time as the second constant when calculating the second constant. Specifically, for example, while changing the shape of the insertion portion 21 of the endoscope 1, a plurality of second force, rigidity value, and number of times of use value acting on the second shape information and the second detection position. By preparing a set and executing machine learning using this set as teacher data, the second constant and the third constant are calculated.

Other configurations, actions and effects in this embodiment are the same as in the first embodiment.

The present invention is not limited to the above-described embodiment, and various changes, modifications, and the like can be made without changing the gist of the present invention. For example, the force information calculation device of the present invention can be applied not only to a medical endoscope system but also to an industrial endoscope system.

Further, the shape sensor unit in the present invention is not limited to the endoscope insertion shape observation device 5, and acquires the first shape information of each of a plurality of measurement points by using a sensor such as an optical fiber sensor. May be good.

Further, in the second modification of the first embodiment and the second modification of the second embodiment, the rigidity information acquisition unit 16 may not be provided.

Claims (11)

A flexible member that is the insertion part of the endoscope and has an elongated shape,
Shape information corresponding to the shape of the flexible member, at each of a plurality of measurement points arranged at intervals along the longitudinal direction of the flexible member in the flexible member. A shape sensor unit that acquires a plurality of first shape information, which is shape information, and
A plurality of second shape information which is shape information at each of a plurality of calculation points defined at intervals along the longitudinal direction in the flexible member based on the plurality of first shape information. The shape information acquisition unit to acquire
The number-of-use information acquisition unit for acquiring the number of times the flexible member has been used,
A storage unit for storing a first constant set including a plurality of first constants and a fourth constant, and the value of the number of times of use .
The multiplication of each of the plurality of second shape information and the corresponding ones of the plurality of first constants, and the corresponding ones of the plurality of second shape information and the plurality of first constants. The flexibility is calculated by an operation including finding the sum of a plurality of terms obtained by multiplying of the above and adding the term obtained by multiplying the value of the number of times of use and the fourth constant to the sum. A force information calculation unit that calculates the first force information including information on the magnitude of the force in at least one predetermined direction acting on the first detection position in the sex member.
A force information calculation device characterized by being equipped with. The storage unit further stores a second set of constants including a plurality of second constants.
The force information calculation unit is different from the first detection position in the flexible member by an operation including multiplication of the plurality of second shape information and the corresponding ones of the plurality of second constants. The force information calculation device according to claim 1, wherein the second force information including information on the magnitude of the force in at least one predetermined direction acting on the second detection position is calculated. Further, a rigidity information acquisition unit for acquiring the rigidity value of the flexible member is provided.
The storage unit stores the rigidity value and stores the rigidity value.
The first set of constants further comprises a third constant.
The force information according to claim 1 , wherein the calculation in the force information calculation unit further includes adding a term obtained by multiplying the rigidity value and the third constant to the sum. Calculation device. A flexible member that is the insertion part of the endoscope and has an elongated shape,
Shape information corresponding to the shape of the flexible member, at each of a plurality of measurement points arranged at intervals along the longitudinal direction of the flexible member in the flexible member. A shape sensor unit that acquires a plurality of first shape information, which is shape information, and
A plurality of second shape information which is shape information at each of a plurality of calculation points defined at intervals along the longitudinal direction in the flexible member based on the plurality of first shape information. The shape information acquisition unit to acquire
The number-of-use information acquisition unit for acquiring the number of times the flexible member has been used,
A storage unit for storing a first constant set including a plurality of first constants and a fourth constant, and the value of the number of times of use.
At least one predetermined position acting on the first detection position in the flexible member by an operation including multiplication of each of the plurality of second shape information and the corresponding ones of the plurality of first constants. A force information calculation unit that calculates the first force information including information on the magnitude of the force in the direction of
Equipped with
The force information calculation unit has an input layer, an intermediate layer, and an output layer constituting the neural network.
The plurality of second shape information and the value of the number of times of use are input to the input layer.
In the intermediate layer, the calculation in the force information calculation unit is a multiplication of the plurality of second shape information and the corresponding ones of the plurality of first constants, a value of the number of times of use, and the fourth constant. A first operation for calculating a plurality of first terms by an operation including multiplication with , and a plurality of second terms and a plurality of first terms having a correspondence relationship with the plurality of first terms in the intermediate layer. A second operation that calculates a plurality of third terms by an operation including multiplication of corresponding constants, and a pair of corresponding ones of the plurality of third terms and the plurality of first constants in the output layer. Including a third operation that calculates a plurality of fourth terms by an operation including multiplication of
The force information calculation device, characterized in that the first force information is information composed of the plurality of fourth terms. Further, a rigidity information acquisition unit for acquiring the rigidity value of the flexible member is provided.
The storage unit stores the rigidity value and stores the rigidity value.
The first set of constants further comprises a third constant.
The stiffness value is input to the input layer and
The force information calculation device according to claim 4 , wherein the first calculation further includes a product of the rigidity value and the third constant. The at least one predetermined direction is a first direction, a second direction, and a third direction orthogonal to each other.
The first force information includes a first information which is information on the magnitude of the force in the first direction, a second information which is information on the magnitude of the force in the second direction, and the first information. Including the third information which is the information of the magnitude of the force in the three directions,
The plurality of first constants include a plurality of constants for calculating the first information, a plurality of constants for calculating the second information, and a plurality of for calculating the third information. The force information calculation device according to claim 1 or 4 , wherein the force information calculation device includes the constants of. The first shape information includes information on the coordinates of the measurement point.
The force information calculation device according to claim 1 or 4 , wherein the second shape information includes information on the coordinates of the calculation point. The force information calculation device according to claim 1 or 4 , wherein the second shape information includes information on the curvature of the flexible member at the calculation point. The force information calculation device according to claim 1 or 4 , wherein at least a part of the plurality of measurement points and at least a part of the plurality of calculation points are located at different positions from each other. The force information calculation device according to claim 1 or 4 , and an endoscope having the flexible member as the insertion portion to be inserted into the subject.
An endoscope system characterized by being equipped with. The shape sensor portion is shape information corresponding to the shape of the flexible member which is the insertion portion of the endoscope and has an elongated shape, and is the shape information corresponding to the shape of the flexible member in the flexible member in the longitudinal direction of the flexible member. A step of acquiring a plurality of first shape information which is shape information at each of a plurality of measurement points arranged at intervals along the line, and a step of acquiring the plurality of first shape information.
A plurality of shape information acquisition units are shape information at each of a plurality of calculation points defined at intervals along the longitudinal direction in the flexible member based on the plurality of first shape information. The step to acquire the second shape information of
A step in which the usage count information acquisition unit acquires the usage count of the flexible member, and
A step in which the storage unit stores a first constant set including a plurality of first constants and a fourth constant, and the value of the number of times of use .
The force information calculation unit multiplies each of the plurality of second shape information and the corresponding ones of the plurality of first constants, and the plurality of second shape information and the plurality of first constants. Includes finding the sum of a plurality of terms obtained by multiplying the corresponding constants, and adding the term obtained by multiplying the value of the number of times of use to the fourth constant to the sum. A step of calculating a first force information including information on the magnitude of the force in at least one predetermined direction acting on the first detection position in the flexible member by calculation.
A force information calculation method characterized by including.

Images