JP6694544B1 - Micro work robot - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a micro-working robot capable of tightly, accurately, and rapidly cooperatingly controlling an imaging unit, an operating body such as a treatment tool, and an arm holding the operating body. An image pickup unit that has at least one lens optical unit for observing a subject in an object and obtains image data by picking up an observation image formed by light passing through the lens optical unit, and is capable of contacting the subject. An arm that holds a large operating body so that it can be driven at intervals smaller than the resolution of the observed image, a drive controller that controls the drive of the operating body and the arm, an imaging controller that controls the imaging unit, and a drive control And a control unit that controls the image pickup control unit in cooperation with each other. [Selection diagram] Figure 1

Description

  The present invention relates to a micro working robot including an operating body, an arm section that holds the operating body, and an imaging section.

  In recent years, various surgical methods using a micro-work robot capable of fine operation have been proposed. An example of such a micro-working robot is the robot assembly described in Patent Document 1. The robot assembly of Patent Document 1 has a configuration including at least one medical device having a frame and a joint device, and can control the medical device easily and accurately without impairing safety and allows intuitive operation. In addition, it is durable and can be regularly maintained without impairing sterility and reliability. Further, by connecting a video camera or a microscope to the support member to which the medical device is attached, it is possible to frame the end portion of the medical device or the like.

Japanese Patent Publication No. 2018-534100

In surgery, the size and position of the target range may change or the position of the operating tool such as a treatment tool may change according to the progress thereof. In addition, the treatment tool may be made to stand by once or the waiting treatment tool may be returned to the periphery of the target portion. Furthermore, in addition to these, it is conceivable that the relationship between the shooting range of the image pickup unit and the operating body changes sequentially. In addition, in recent years, the number of scenes in which surgery is performed by a micro-working robot using high-definition images with resolutions equivalent to 4K and 8K is increasing. Therefore, in the micro-work robot, the imaging unit, the operating tool such as the treatment tool, and the arm section that holds the operating tool are used in response to such various changes even when a high-definition image is used. There is a growing demand for close, accurate, and swift collaboration.
Here, the resolution equivalent to 4K means that the number of pixels in one frame is 3840 × 2160 and about 8.3 million or more, and the resolution equivalent to 8K means that the number of pixels in one frame is approximately 7680 × 4320. It means a resolution of 33 million or more.

  Therefore, the present invention provides a micro-working robot capable of tightly, accurately, and quickly and cooperatively controlling an imaging unit, an operating tool such as a treatment tool, and an arm section holding the operating tool. With the goal.

In order to solve the above problems, a micro-working robot of the present invention has at least one lens optical unit for observing a subject in an object, and captures an observation image formed by light passing through the lens optical unit. An imaging unit that obtains image data, an operation unit that can contact a subject, and an arm unit that holds the operation unit at a narrower interval than the resolution of the observed image, and a drive control unit that controls the operation of the operation unit and the arm unit. It is characterized by including an imaging control unit that controls the imaging unit and a control unit that controls the drive control unit and the imaging control unit in cooperation.
Here, it is preferable that the resolution of the observed image includes a resolution corresponding to 4K or 8K.

  In the micro work robot of the present invention, it is preferable that the control unit includes a coordinate system matching unit that associates the coordinate system of the imaging unit with the coordinate system of the operating body.

In the micro work robot of the present invention, the position is specified in the coordinate system of the imaging unit and the shared origin whose position is specified in the coordinate system of the operating body, and the coordinate system matching unit is based on the shared origin. It is preferable that the coordinate system of the imaging unit and the coordinate system of the operating body are made common.
It is preferable that the coordinate system matching unit associates the coordinate system of the imaging unit with the coordinate system of the operating body by using image data including an image of the marker whose position is specified by the coordinate system of the operating body. In addition, the coordinate system matching unit defines the direction of the coordinate axis that defines the position in the plane whose normal is the direction along the image capturing axis of the image capturing unit in the coordinate system of the image capturing unit and the horizontal plane of the operating body in the coordinate system of the operating body. It is preferable to generate a control signal for moving at least one of the position of the imaging unit and the reference position of the operating body so that the direction of the coordinate axis defining the inner position is aligned.

  In the micro-working robot of the present invention, the control unit has an imaging status information generation unit that generates imaging status information of the imaging unit, and a drive signal that generates a control signal to the drive control unit by inputting a signal based on the imaging status information. And a generator. It is preferable that the imaging status information includes focus, magnification, observation field of view, and operation state information.

In the micro work robot of the present invention, the control unit includes a drive status information generation unit that generates drive status information of the arm unit and the operating body,
It is preferable to include an imaging signal generation unit that receives a signal based on the drive status information and generates a control signal to the imaging control unit. The drive status information preferably includes position information and operation state information.

  In the micro-working robot of the present invention, the drive signal generation unit controls the out-of-range movement control for moving the operating body to the outside of the observation visual field of the imaging unit based on the signal related to the observation visual field of the imaging unit input from the imaging status information generation unit. If the drive control unit determines that the operation control unit performs the operation, the stop control signal for stopping the out-of-range movement control is generated before the operating body is located outside the observation visual field, and the stop control signal is transmitted to the drive control unit. It is preferable to output to.

  In the micro work robot of the present invention, when the drive signal generation unit determines that the operating body is located outside the observation visual field of the imaging unit based on the signal related to the observation visual field of the imaging unit input from the imaging status information generation unit. Preferably generates a proximity control signal for operating body proximity control that controls the arm portion so that the operating body is located within the range of the observation visual field, and outputs the proximity control signal to the drive control unit.

  In the micro work robot of the present invention, the drive signal generation unit controls the arm unit so as to move the operating body to the standby position set within the range of the observation visual field of the imaging unit based on the input imaging status information. It is preferable to generate a standby position control signal for position movement control and output the standby position control signal to the drive control unit.

  In the micro-working robot of the present invention, the imaging signal generation unit is based on a signal input from the drive status information generation unit, and the focus control for focusing the lens optical unit to a predetermined position defined with the position of the operating body as a reference. It is preferable to generate a focus control signal for outputting the focus control signal to the imaging control unit.

  In the micro-working robot of the present invention, the image pickup unit has a plurality of lens optical units, and the image pickup signal generation unit has a plurality of lens optical units based on the signal input from the drive status information generation unit with the position of the operating body as a reference. It is preferable to generate an optical axis movement control signal for controlling the optical axis movement that moves the respective optical axes to a predetermined direction, and output the optical axis movement control signal to the imaging control unit.

  In the micro-working robot of the present invention, the image pickup signal generation unit performs the out-of-range movement control for moving the operating body to the outside of the observation visual field of the image pickup unit based on the signal input from the drive status information generation unit. If it is determined that the operating body is located outside the range of the observation visual field, the imaging range that controls the imaging unit so that the operating body is located within the range of the observation visual field when the out-of-range movement control is completed. It is preferable to generate an imaging range control signal for movement / enlargement control and output the imaging range control signal to the imaging control unit.

  In the micro-working robot of the present invention, the imaging signal generation unit observes the operation body when it determines that the operation body is located outside the observation field of view of the imaging unit based on the signal input from the drive status information generation unit. It is preferable to generate an image pickup range control signal for image pickup range movement / enlargement control for controlling the image pickup unit so as to be positioned within the range of the visual field, and output the image pickup range control signal to the image pickup control unit.

  The micro-working robot of the present invention preferably further comprises a subject measuring unit that measures the position of the subject and a subject coordinate system setting unit that sets the coordinate system of the subject based on the measurement result of the subject measuring unit. The subject measuring section preferably measures the position of the subject by inputting the image data. Further, it is preferable that the imaging unit has a plurality of lens optical units, and the subject measuring unit measures the position of the subject based on a plurality of image data obtained from the plurality of lens optical units.

  In the micro work robot of the present invention, it is preferable that the coordinate system matching unit associates the coordinate system of the subject with the coordinate system of the imaging unit by inputting a signal indicating the coordinate system of the subject.

  In the micro-working robot of the present invention, it is preferable that the coordinate system matching unit inputs the signal indicating the coordinate system of the subject and associates the coordinate system of the subject with the coordinate system of the operating body.

  In the micro-working robot of the present invention, it is preferable that the coordinate system matching unit calculates the position of the target designated as a part of the observation image in the coordinate system of the subject. It is preferable that the control unit includes an imaging signal generation unit that generates a control signal to the imaging control unit based on an input signal from the coordinate system matching unit. The imaging signal generation unit controls the observation visual field of the imaging unit based on the position of the target in the coordinate system of the subject so that the target is located at a predetermined position in the observation visual field of the imaging unit. It is preferable to generate a signal and output the view control signal to the imaging control unit.

  In the micro-working robot of the present invention, the imaging signal generation unit generates a focus control signal for focus control for focusing the lens optical unit on the target position based on the position of the target in the coordinate system of the subject, It is preferable to output the control signal to the imaging control unit.

  In the micro-working robot of the present invention, the image pickup unit has a plurality of lens optical units, and the image pickup signal generation unit determines the direction of the optical axis of each of the plurality of lens optical units with reference to the position of the target in the coordinate system of the subject. It is possible to generate an optical axis movement control signal for optical axis movement control for setting and moving the optical axes of a plurality of lens optical sections in the set direction, and outputting the optical axis movement control signal to the imaging control section. preferable.

  In the micro-working robot of the present invention, the control unit further includes a drive status information generation unit that generates drive status information, and the imaging signal generation unit receives a signal based on the drive status information as an input and outputs a control signal to the imaging control unit. Is preferably generated. The imaging signal generation unit, based on the input drive status information, when it determines that the operating body is located outside the observation visual field of the imaging unit, performs imaging so that the operating body and the target are located within the observation visual field. It is preferable to generate an image pickup range control signal for controlling the image pickup range movement / enlargement and output the image pickup range control signal to the image pickup control unit.

  In the micro work robot of the present invention, when the image pickup signal generation unit determines that the drive control unit performs the out-of-range movement control for moving the operating body to the outside of the observation visual field of the image pickup unit based on the input drive status information. Includes an imaging range movement / control that controls the imaging unit so that the operating body is located within the observation visual field together with the target when the out-of-range movement control is completed before the operating body is located outside the observation visual field. It is preferable to generate an imaging range control signal for enlargement control and output the imaging range control signal to the imaging control unit.

  In the micro-working robot of the present invention, it is preferable that the drive signal generation unit generate a control signal to the drive control unit based on the input signal from the coordinate system matching unit. It is preferable that the coordinate system matching unit sets the prohibited area in which the operating body should not be located, based on the coordinate system of the subject. The drive signal generation unit, when the drive control unit performs the prohibited area movement control for moving the operating body into the prohibited area, outputs the recovery control signal for the recovery control for moving the operating body to the outside of the prohibited area. It is preferable to generate and output the recovery control signal to the drive control unit.

  In the micro work robot of the present invention, when the drive signal generation unit determines that there is a prohibited area movement input for moving the operating body to the prohibited area, the driving body moves before the operating body reaches the prohibited area. It is preferable to generate a stop control signal for stop control for stopping, and output the stop control signal to the drive control unit.

  In the micro work robot of the present invention, when the drive signal generation unit determines that the operation body passes through the prohibition area, the operation body passes through the prohibition area before reaching the prohibition area. It is preferable to calculate a new movement route that does not occur, generate an avoidance control signal for avoidance control for moving the operating body on the new movement route, and output the avoidance control signal to the drive control unit.

  In the micro work robot of the present invention, when the drive signal generation unit determines that there is the prohibited area movement input for moving the operation body to the prohibited area, the drive signal generation unit is generated based on the operator operating the operation body and the image data. It is preferable to generate a control signal for informing control that informs at least one of the observers of the display image that the prohibited area movement input has been made.

  In the micro-working robot of the present invention, the coordinate system matching unit receives the signal indicating the coordinate system of the subject as an input, associates the coordinate system of the subject with the coordinate system of the imaging unit, and is designated as a part of the observation image. The target signal position is calculated in the coordinate system of the subject, and the drive signal generation unit calculates the standby position of the operating body based on the position of the target in the coordinate system of the subject, so that the operating body moves to the standby position. It is preferable to generate a standby position control signal for controlling the standby position movement for controlling the arm portion and output the standby position control signal to the drive control unit.

  In the micro work robot of the present invention, it is preferable that the drive signal generation unit outputs the control signal to the drive control unit on condition that the permission signal is input.

  In the micro-working robot of the present invention, it is preferable that the imaging signal generation unit outputs the control signal to the imaging control unit on condition that the permission signal is input.

  According to the present invention, it is possible to provide a micro-working robot capable of tightly, accurately, and promptly and cooperatively controlling an imaging unit, an operating tool such as a treatment tool, and an arm section holding the operating tool. You can

It is a block diagram which shows the structure of the micro working robot which concerns on embodiment of this invention. It is a figure showing an example of a coordinate system of a manipulator, a coordinate system of an image pick-up part, and a photographic subject coordinate system.

Hereinafter, a micro-working robot according to an embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing the configuration of a micro work robot 100 according to this embodiment. As illustrated in FIG. 1, the micro work robot 100 includes an imaging control unit 11, an imaging unit 12, a drive control unit 21, an operating body 22, an arm unit 23, a control unit 30, and the like.

  The imaging control unit 11 is a circuit that controls driving of the imaging unit 12. The imaging unit 12 has at least one lens optical unit 13 for observing a subject in an object to be worked by the micro-working robot 100. When the micro-working robot 100 performs an operation on an affected part in the patient's body, the patient is an object and the affected part in the patient's body is a subject. The lens optical unit 13 has at least one lens. The imaging unit 12 has a solid-state imaging device (not shown), and light passing through the lens of the lens optical unit 13 connects an observation image on the imaging surface of the solid-state imaging device to generate image data. This observed image preferably has a resolution equivalent to 4K or 8K. The imaging control unit 11 receives the focus control signal to perform focus control for focusing the lens optical unit on the position of the target, and also receives the optical axis movement control signal to reference the position of the operating body as a reference. Optical axis movement control is performed to move the respective optical axes of the optical unit to a predetermined direction.

  The drive control unit 21 controls the drive of a treatment tool or other operating body 22 for treating a diseased part (subject) and an arm 23. The operating body 22 is arranged so as to be able to contact the subject, and is held by the arm portion 23 so that it can be driven at an interval narrower than the resolution of the observed image, that is, an interval finer than the resolution.

  The control unit 30 includes a drive signal generation unit 31, a drive status information generation unit 32, a coordinate system matching unit 33, an image pickup signal generation unit 34, and an image pickup status information generation unit 35, and the drive control unit 21 and the image pickup control unit 11 are included. And control so that and drive in cooperation with each other. The drive signal generation unit 31, the drive status information generation unit 32, the coordinate system matching unit 33, the imaging signal generation unit 34, and the imaging status information generation unit 35 may be configured as independent arithmetic circuits, or one operation circuit. It may be configured by an arithmetic circuit.

The drive status information generation unit 32 generates drive status information on the operating body 22 and the arm unit 23. The imaging signal generation unit 34 receives a signal based on the drive status information as an input, and generates a control signal to the imaging control unit 11 based on an input signal from the coordinate system matching unit 33. The drive status information includes, for example, position information and an operation state of each of the operating body 22 and the arm portion 23.
The coordinate system matching unit 33 associates the coordinate system of the imaging unit 12 with the coordinate system of the operating body 22.

  The imaging status information generation unit 35 generates imaging status information of the imaging unit 12 based on the input signal from the imaging signal generation unit 34 and the input signal from the imaging unit 12. The drive signal generation unit 31 receives a signal based on the imaging status information as an input, and generates a control signal to the drive control unit 21 based on an input signal from the coordinate system matching unit 33. The imaging status information includes, for example, focus information including a focused state, magnification, observation field of view or imaging range, and operation state information in the lens optical unit 13.

  As described above, in the micro work robot 100, a control signal for controlling the image pickup unit 12 and one of the operating body 22 and the arm unit 23 (status information) is generated, and the image pickup unit 12 causes the control signal to be generated. Imaging and driving of the operating body 22 and the arm portion 23 are controlled in cooperation with each other.

  The following (a) to (c) are examples of controlling the operating body 22 and the arm unit 23 based on the imaging status information of the image capturing unit 12 in the cooperative control of the image capturing unit 12 and the operating body 22 and the arm unit 23. ) Is mentioned.

  (A) The drive signal generation unit 31 controls the out-of-range movement to move the operating body 22 out of the observation field of view of the image capturing unit 12 based on the signal related to the observation field of view of the image capturing unit 12 input from the image capturing status information generation unit 35. If the drive control unit 21 determines that the operation control unit 21 performs the operation, the stop control signal for stopping the out-of-range movement control is generated before the operating body 22 is located outside the range of the observation visual field, and this stop control signal is output. Output to the drive control unit 21. As a result, the operating body 22 is controlled so as to stop moving before being located outside the range of the observation visual field and continue to be located within the range of the observation visual field.

  (B) When the drive signal generation unit 31 determines that the operating body 22 is located outside the observation visual field of the imaging unit 12 based on the signal related to the observation visual field of the imaging unit 12 input from the imaging status information generation unit 35. Generates a proximity control signal for operating body proximity control that controls the arm portion 23 so that the operating body 22 is located within the range of the observation visual field, and outputs this proximity control signal to the drive control unit 21. Thereby, even if the operating body 22 is temporarily located outside the range of the observation visual field, it can be controlled to return to the inside of the observation range.

  (C) The drive signal generation unit 31 arms so that the operating body 22 moves to the standby position set within the range of the observation visual field of the imaging unit 12 based on the imaging status information input from the imaging status information generation unit 35. A standby position control signal for controlling the standby position movement that controls the unit 23 is generated, and the standby position control signal is output to the drive control unit 21.

  On the other hand, as an example of controlling the image pickup unit 12 based on the drive status information of the operation body 22 and the arm unit 23 in the cooperative control of the image pickup unit 12 and the operation body 22 and the arm unit 23, the following (d) to (G) is mentioned.

  (D) The image pickup signal generation unit 34 performs focus control for focusing the lens optical unit 13 on a predetermined position defined based on the position of the operating body 22 based on the signal input from the drive status information generation unit 32. The focus control signal is generated, and the focus control signal is output to the imaging control unit 11. As a result, the in-focus state can be maintained even if the position of the operating body 22 changes.

  (E) In the configuration in which the image pickup unit 12 includes the plurality of lens optical units 13, the image pickup signal generation unit 34 uses the signals input from the drive status information generation unit 32 to set the plurality of lenses based on the position of the operating body 22. An optical axis movement control signal for controlling the optical axis movement for moving the respective optical axes of the optical section 13 to a predetermined direction is generated, and the optical axis movement control signal is output to the imaging control section 11. Thereby, even if the position of the operating body 22 is changed, the operating body 22 can be positioned within the observation range of the plurality of lens optical units 13. Furthermore, for example, when two lens optical units 13 are provided, it becomes possible to set a desired convergence angle with the position of the operating body 22 as a reference.

  (F) In the imaging signal generation unit 34, the drive control unit 21 performs out-of-range movement control for moving the operating body 22 outside the observation field of view of the imaging unit 12 based on the signal input from the driving status information generation unit 32. If it is determined that the operating body 22 is located outside the observation visual field, the imaging unit 12 is controlled so that the operating body 22 is located within the observation visual field when the out-of-range movement control is completed. An imaging range control signal for imaging range movement / enlargement control is generated, and the imaging range control signal is output to the imaging control unit 11. As a result, the operating body 22 is controlled so as to be continuously positioned within the range of the observation visual field.

  (G) When the imaging signal generation unit 34 determines that the operating body 22 is located outside the observation field of view of the imaging unit 12 based on the signal input from the drive status information generating unit 32, the operating body 22 observes. An imaging range control signal for imaging range movement / enlargement control for controlling the imaging section 12 so as to be positioned within the range of the visual field is generated, and this imaging range control signal is output to the imaging control section 11. Thereby, even if the operating body 22 is temporarily located outside the range of the observation visual field, it can be controlled so as to be again located within the observation visual field by enlarging the observation visual field.

  In the micro work robot 100, the imaging unit 12 and the operating body 22 have coordinate systems independent of each other. Here, the position and the like of the arm portion 23 are determined by the coordinate system related to the operating body 22. Further, the subject coordinate system can be set based on the measurement result of the subject measuring unit 44.

  As the coordinate system, for example, as shown in FIG. 2, the operating body 22 has a coordinate system X1-Y1-Z1, the imaging unit 12 has a coordinate system X2-Y2-Z2, and the coordinate system X3 on the subject S side. -Y3-Z3 is set. In the example illustrated in FIG. 2, the two distal end portions 23a and 23b of the arm portion 23 are provided with the two treatment tools 22a and 22b as the operating body 22, respectively, and the imaging unit 12 includes the two lens optical portions 13a. An image of the subject S of the target is captured via 13b. Here, the two lens optical units 13a and 13b have optical axes AX1 and AX2, respectively, and the imaging unit 12 has the subject S along the observation axis AX0 (the optical axes of the entire two lens optical units 13a and 13b). The observation range including is imaged.

  As shown in FIG. 2, in the coordinate system of the operating body 22, the Z1 direction is set along the vertical direction, and the X1-Y1 plane including the X1 direction and the Y1 direction is orthogonal to the Z1 axis. In the coordinate system of the imaging unit 12, the Z2 direction is set along the observation axis AX0, and the X2-Y2 plane including the X2 direction and the Y2 direction is orthogonal to the Z2 axis. In the coordinate system of the subject S, the Z3 direction is set along the vertical direction, and the X3-Y3 plane including the X3 direction and the Y3 direction is orthogonal to the Z3 axis. The X1 direction in the coordinate system of the operating body 22 is set to be parallel to the X2 direction in the coordinate system of the imaging unit 12. The coordinate system of the operating body 22 and the coordinate system of the imaging unit 12 share the origin O, and the shared origin O is set on the subject S, for example.

  Each coordinate system is associated by the coordinate system matching unit 33. When the resolution of the observation image by the imaging unit 12 is equivalent to 4K or 8K, the coordinate system of the imaging unit 12, the coordinate system of the operating body 22, and the object coordinate system are generated with high accuracy corresponding to the resolution of the observation image. Therefore, the coordinate system matching is also performed with high accuracy. Examples of the association of the coordinates include the following methods (1) to (3).

  (1) A common origin is set by the coordinate system of the imaging unit 12 and the coordinate system of the operating body 22. This origin is the shared origin O in the example shown in FIG. 2, and the position is specified by the coordinate system of the imaging unit 12 (X2-Y2-Z2 in the case of FIG. 2) and the coordinate system of the operating body 22 (see FIG. In the case of 2, the position is specified by X1-Y1-Z1). The coordinate system matching unit 33 makes the coordinate system of the imaging unit 12 and the coordinate system of the operating body 22 common based on this shared origin.

  (2) The coordinate system matching unit 33 associates the coordinate system of the imaging unit 12 with the coordinate system of the operating body 22 by using the image data including the image of the marker whose position is specified by the coordinate system of the operating body 22. To do.

  (3) The coordinate system matching unit 33 uses the image pickup axis of the image pickup unit 12 in the coordinate system of the image pickup unit 12, for example, the optical axis of one or more whole lens optical units (observation axis AX0 in the case of FIG. 2). To control so that the direction of the coordinate axis that defines the position in the plane whose normal is the direction along the line and the direction of the coordinate axis that defines the position of the operation body 22 in the horizontal plane in the coordinate system of the operation body 22 are aligned. Generate the control signal of. In the example shown in FIG. 2, the X2 direction defining the plane X2-Y2 plane having the normal along the observation axis AX0 and the X1 direction of the coordinate axes X1-Y1-Z1 of the operating body 22 are aligned. There is. The control signal for aligning the coordinate axes is output to the drive signal generation unit 31 and the image pickup signal generation unit 34, and by moving at least one of the position of the image pickup unit 12 and the reference position of the operating body 22, the directions of the two coordinate axes. Align.

  As shown in FIG. 1, the micro work robot 100 includes a drive input unit 41, a control input unit 42, a subject coordinate system setting unit 43, a subject measuring unit 44, and an imaging input unit 45. These are configured by a touch panel, a keyboard, and other input devices. For example, a common touch panel may be used depending on the selection of a menu, or may be configured by independent input devices.

  The drive input unit 41 is provided to set and input drive conditions for the operating body 22 and the arm unit 23, and the input signal is output to the drive signal generation unit 31. The image capturing input unit 45 is provided to set and input the image capturing conditions of the image capturing unit 12 including the lens optical unit 13, and the input signal is output to the image capturing signal generating unit 34. The setting by the imaging input unit 45 includes an instruction to switch the plurality of lens optical units 13.

  The control input unit 42 is provided to set and input control conditions such as association and alignment of the coordinate system of the image capturing unit 12 and the coordinate system of the operating body 22, and cooperative control of the image capturing control unit 11 and the drive control unit 21. The input signal is output to the drive signal generation unit 31, the coordinate system matching unit 33, and the imaging signal generation unit 34.

  The imaging status information generated by the imaging status information generating unit 35 and the data of the image captured by the imaging unit 12 (image data) are input to the subject measuring unit 44. The subject measuring unit 44 measures the position of the subject in the image data based on the imaging status information and the image data, that is, with the image data as an input, and sets a control signal for setting the subject coordinates in the subject coordinate system setting. It is output to the unit 43. Upon receiving this control signal, the subject coordinate system setting unit 43 sets the coordinate system of the subject based on the measurement result of the subject measuring unit 44, and sends the set coordinate system information to the coordinate system matching unit 33. When the imaging unit 12 is provided with a plurality of lens optical units 13, the position of the subject is measured based on a plurality of image data obtained from the plurality of lens optical units 13.

  Regarding the coordinate system of the subject, the coordinate system matching unit 33 receives the signal indicating the coordinate system of the subject received from the subject coordinate system setting unit 43 as input, and according to the control signal from the control input unit 42, the coordinate system of the subject, The association with the coordinate system of the imaging unit 12 and / or the coordinate system of the operating body 22 is performed. Further, the coordinate system matching unit 33 calculates the position of the target designated as a part of the observation image obtained by the imaging unit 12 in the coordinate system of the subject. The calculation result is output to the drive signal generation unit 31 and the image pickup signal generation unit 34. Here, the target is, for example, a mark or the like provided on the operating body 22 or the arm unit 23, or the like associated with a pixel at a predetermined position of the solid-state imaging device included in the imaging unit 12.

  The imaging signal generation unit 34 that has received the calculation result in the coordinate system matching unit 33 uses the imaging unit 12 so that the target is located at a predetermined position in the observation visual field of the imaging unit 12 with the position of the target in the coordinate system of the subject as a reference. A visual field control signal for visual field adjustment control for controlling the observation visual field is generated, and the visual field control signal is output to the imaging control unit 11.

  Further, the image pickup signal generation unit 34 generates a focus control signal for focus control for focusing the lens optical unit 13 on the target position with reference to the position of the target in the coordinate system of the subject, and outputs this focus control signal. It outputs to the imaging control unit 11.

  In the configuration in which the plurality of lens optical units 13 are provided in the image pickup unit 12, the image pickup signal generation unit 34 sets the directions of the respective optical axes of the plurality of lens optical units 13 on the basis of the position of the target in the coordinate system of the subject. Then, an optical axis movement control signal for optical axis movement control for moving the respective optical axes of the plurality of lens optical sections 13 in the set direction is generated, and this optical axis movement control signal is sent to the imaging control section 11. Output.

  When the imaging signal generation unit 34 determines that the operating body 22 is located outside the observation visual field of the imaging unit 12 based on the input drive status information, the operating body 22 and the target are located within the observation visual field range. The imaging range control signal for controlling the imaging range movement / enlargement for controlling the imaging section 12 is generated, and the imaging range control signal is output to the imaging control section 11.

  When the drive control unit 21 determines that the imaging signal generation unit 34 performs the out-of-range movement control for moving the operating body 22 to the outside of the observation field of view of the imaging unit 12 based on the input drive status information, the operation is performed. Before the body 22 is located outside the range of the observation field of view, when the out-of-range movement control is completed, the operation body 22 is controlled with the target so that the imaging unit 12 is located within the range of the observation field of view. An imaging range control signal for control is generated, and this imaging range control signal is output to the imaging control unit 11.

  In the micro work robot 100, a prohibited area in which the operating body 22 should not be located is set. The prohibited area is set automatically based on the coordinate system of the subject or by the user's designation by the control input unit 42. The prohibited area is, for example, an area in the subject and its surrounding area that cannot be allowed to come into contact with the operating body 22 or approach within a predetermined distance. In addition, the operation body 22 and an area of a predetermined distance around the operation body 22 may be set as the prohibited area. Further, it is preferable that the area set as the standby position of the operating body 22 is also a prohibited area in a status other than the standby state.

  When the drive control unit 21 performs the prohibited area movement control for moving the operating body 22 into the prohibited area, the drive signal generation unit 31 performs recovery control for moving the operating body 22 to the outside of the prohibited area. A recovery control signal is generated and this recovery control signal is output to the drive control unit 21.

  In addition, when the drive signal generation unit 31 determines that the input to the drive input unit 41 or the control input unit 42 is an input regarding the prohibited area movement for moving the operating body 22 to the prohibited area (prohibited area movement input). For example, a stop control signal for stopping control for stopping the movement of the operating body 22 is generated before the operating body 22 reaches the prohibited area, and the stop control signal is output to the drive control unit 21.

  Further, when the drive signal generation unit 31 determines that the input to the drive input unit 41 or the control input unit 42 is an input regarding the prohibition region passage in which the operating body 22 passes through the prohibition region (prohibition region passage input). Before the operating body 22 reaches the prohibited area, a new moving route that does not pass through the prohibited area is calculated, and an avoidance control signal for avoiding control for moving the operating body 22 on the new moving route is generated. , And outputs this avoidance control signal to the drive control unit 21.

  In addition, when the drive signal generation unit 31 determines that the input to the drive input unit 41 or the control input unit 42 includes a prohibited area movement input for moving the operation body 22 to the prohibited area, the drive signal generation unit 31 operates the operation body 22. A control signal for informing control is generated to inform at least one of the operator who performs the operation and the observer of the display image generated based on the image data that there is a prohibited area movement input. By this notification control, for example, notification information can be displayed on a screen (not shown) displaying an observation image, vibration can be generated in a portion held by the operator, and a notification sound can be generated.

  The coordinate system matching unit 33 receives the signal indicating the coordinate system of the subject as an input, associates the coordinate system of the subject with the coordinate system of the imaging unit 12, and determines the position of the target designated as a part of the observed image. Of the target position in the coordinate system of the subject as a reference, the drive signal generation unit 31 calculates the standby position of the operating body 22, and the arm unit 23 moves the operating body 22 to the standby position. A standby position control signal for controlling the standby position movement is generated, and this standby position control signal is output to the drive control unit 21.

  When the operating body 22 is in the standby signal, the operating body 22 cannot be driven until a permission signal for moving the operating body 22 is input, and the drive signal generating unit 31 and the imaging signal generating unit 34 On condition that the permission signal is input, the control signal is output to the drive control unit 21 and the image pickup control unit 11, respectively, and the driving of the operation body 22 and the arm unit 23 and the image pickup by the image pickup unit 12 are restarted.

  When a plurality of lens optical units 13 are provided, an operator or a person concerned may manually perform the same method as a commercially available lens for a single-lens reflex camera. Alternatively, the operator can operate the exchange device by operating the imaging input unit 45 to switch the lens optical unit. When switching is performed using the exchange device, the information of the currently selected lens optical unit 13 is output to the imaging status information generation unit 35, and the optical characteristics of the currently selected lens optical unit 13, such as the focal length, A drive signal is generated in the drive signal generation unit 31 according to the observation range. Therefore, based on the control signal corresponding to the optical characteristic of the exchanged lens optical unit 13, the operation body 22 and the arm unit 23 are cooperatively controlled.

The micro-working robot 100 according to the present embodiment is configured as described above, and thus the imaging unit 12, the operating body 22, and the arm unit 23 that holds the operating body 22 are tightly, accurately, and quickly linked. It becomes possible to control.
Although the present invention has been described with reference to the above embodiments, the present invention is not limited to the above embodiments, and improvements or changes can be made within the scope of the object of improvement or the idea of the present invention.

  As described above, the micro-working robot according to the present invention is useful in that it can tightly, accurately, and swiftly control the imaging unit, the operating body, and the arm unit in cooperation with each other.

11 image pickup control section 12 image pickup section 13, 13a, 13b lens optical section 21 drive control section 22, 22a, 22b operating body 23 arm section 23a, 23b tip section 30 control section 31 drive signal generation section 32 drive status information generation section 33 coordinates System matching unit 34 Imaging signal generation unit 35 Imaging status information generation unit 41 Drive input unit 42 Control input unit 43 Subject coordinate system setting unit 44 Subject measuring unit 45 Imaging input unit 100 Micro work robot X1-Y1-Z1 Coordinate system of operating body X2-Y2-Z2 Imaging system coordinate system X3-Y3-Z3 Subject coordinate system AX0 Observation axes AX1, AX2 Optical axis O Origin (shared origin)
S subject

Claims (27)

An imaging unit that has at least one lens optical unit for observing a subject in an object, and obtains image data by capturing an observation image formed by light passing through the lens optical unit;
An arm portion that holds the operating body that can come into contact with the subject so as to be drivable at an interval narrower than the resolution of the observation image;
A drive control unit that controls the drive of the operating body and the arm unit;
An imaging control unit that controls the imaging unit;
A control unit that controls the drive control unit and the imaging control unit in cooperation with each other ;
The control unit includes a coordinate system matching unit that associates the coordinate system of the imaging unit with the coordinate system of the operating body,
The coordinate system matching unit,
In the coordinate system of the image capturing unit, the direction of the coordinate axis that defines the position in the plane whose normal is the direction along the image capturing axis of the image capturing unit, and the position in the horizontal plane of the operating body in the coordinate system of the operating body. defined as the direction of the coordinate axes are aligned to, micro-working robot characterized that you generate a control signal for moving at least one of the reference position of the position and the operation of the imaging unit.   An imaging unit that has at least one lens optical unit for observing a subject in an object, and obtains image data by capturing an observation image formed by light passing through the lens optical unit;
  An arm portion that holds the operating body that can come into contact with the subject so as to be drivable at an interval narrower than the resolution of the observation image;
  A drive control unit that controls the drive of the operating body and the arm unit;
  An imaging control unit that controls the imaging unit;
  A control unit that controls the drive control unit and the imaging control unit in cooperation with each other;
  The control unit is
    An imaging status information generation unit that generates imaging status information of the imaging unit;
    A drive signal generation unit that receives a signal based on the imaging status information and generates a control signal to the drive control unit,
A micro-working robot comprising:   The micro-working robot according to claim 2, wherein the imaging status information includes focus, magnification, observation field of view, and operation state information. An imaging unit that has at least one lens optical unit for observing a subject in an object, and obtains image data by capturing an observation image formed by light passing through the lens optical unit;
An arm portion that holds the operating body that can come into contact with the subject so as to be drivable at an interval narrower than the resolution of the observation image;
A drive control unit that controls the drive of the operating body and the arm unit;
An imaging control unit that controls the imaging unit;
A control unit that controls the drive control unit and the imaging control unit in cooperation with each other;
The control unit is
A drive status information generation unit that generates drive status information of the arm unit and the operating body;
An image pickup signal generation unit that receives a signal based on the drive status information and generates a control signal to the image pickup control unit,
A micro-working robot comprising: The micro work robot according to claim 4 , wherein the drive status information includes position information and operation state information. The drive signal generation unit,
Based on a signal related to the observation field of view of the imaging unit input from the imaging status information generation unit,
When it is determined that the drive control unit performs the out-of-range movement control for moving the operating body to the outside of the observation visual field,
Before the operating body is positioned outside the range of the observation field, and generates a stop control signal for stopping the outside movement control, wherein said stop control signal to claim 2 to be output to the drive control unit Micro working robot. The drive signal generation unit,
Based on a signal related to the observation field of view of the imaging unit input from the imaging status information generation unit,
When it is determined that the operating body is located outside the range of the observation visual field,
3. The proximity control signal for operating body proximity control for controlling the arm unit so that the operating body is located within the range of the observation visual field is generated, and the proximity control signal is output to the drive control unit. Micro-working robot described in. The drive signal generation unit,
Based on the input imaging status information,
A standby position control signal is generated for standby position movement control for controlling the arm unit to move the operating body to a standby position set within a range of the observation field of the image capturing unit, and the standby position control signal is generated. micro working robot according to claim 2 to be output to the drive control unit. The imaging signal generation unit,
Based on the signal input from the drive status information generator,
Claim 4 generates a focus control signal for focusing control for focusing the lens optic portion in a predetermined position defined the position of the operating body as a reference, and outputs the focus control signal to the imaging control unit Micro-working robot described in. The imaging unit has a plurality of lens optical units,
The imaging signal generation unit,
Based on the signal input from the drive status information generator,
An optical axis movement control signal is generated for optical axis movement control for moving the optical axis directions of the plurality of lens optical units to a predetermined direction based on the position of the operating body, and the optical axis movement control is performed. The micro work robot according to claim 4 , wherein a signal is output to the imaging control unit. The imaging signal generation unit,
Based on the signal input from the drive status information generator,
When it is determined that the drive control unit performs the out-of-range movement control for moving the operating body to the outside of the observation field of view of the imaging unit,
Before the operating body is located outside the range of the observation visual field, the imaging range movement for controlling the imaging unit so that the operating body is located within the range of the observation visual field when the movement control outside the range is completed. 5. The micro work robot according to claim 4 , wherein an imaging range control signal for enlargement control is generated, and the imaging range control signal is output to the imaging control unit. The imaging signal generation unit,
Based on the signal input from the drive status information generator,
When it is determined that the operating body is located outside the observation field of view of the imaging unit,
An imaging range control signal for imaging range movement / enlargement control that controls the imaging unit so that the operating body is located within the range of the observation visual field is generated, and the imaging range control signal is output to the imaging control unit. The micro-working robot according to claim 4 . An imaging unit that has at least one lens optical unit for observing a subject in an object, and obtains image data by capturing an observation image formed by light passing through the lens optical unit;
An arm portion that holds the operating body that can come into contact with the subject so as to be drivable at an interval narrower than the resolution of the observation image;
A drive control unit that controls the drive of the operating body and the arm unit;
An imaging control unit that controls the imaging unit;
A control unit that controls the drive control unit and the imaging control unit in cooperation with each other;
The control unit includes a coordinate system matching unit that associates the coordinate system of the imaging unit with the coordinate system of the operating body,
A subject measuring unit for measuring the position of the subject,
A subject coordinate system setting unit that sets the coordinate system of the subject based on the measurement result of the subject measuring unit;
Further equipped with,
The coordinate system matching unit receives a signal indicating the coordinate system of the subject as an input and associates the coordinate system of the subject with the coordinate system of the imaging unit,
The coordinate system matching unit calculates the position of the target designated as a part of the observation image in the coordinate system of the subject,
The control unit is
A micro-working robot comprising: an imaging signal generation unit that generates a control signal to the imaging control unit based on an input signal from the coordinate system matching unit. The imaging signal generation unit,
So that the target is located at a predetermined position in the observation field of view of the imaging unit,
With reference to the position of the target in the coordinate system of the object, the claims produces a field control signal for the field adjustment control for controlling the observation field of view of the imaging unit, and outputs the field control signal to the imaging control unit 13. The micro work robot according to 13 . The imaging signal generation unit,
With a position of the target in the coordinate system of the subject as a reference, a focus control signal for focus control that focuses the lens optical unit on the position of the target is generated, and the focus control signal is output to the imaging control unit. The micro work robot according to claim 13 . The imaging unit has a plurality of lens optical units,
The imaging signal generation unit,
The orientations of the optical axes of the plurality of lens optical units are set with reference to the position of the target in the coordinate system of the subject, and the optical axes of the plurality of lens optical units are moved to the set orientations. 14. The micro work robot according to claim 13 , wherein an optical axis movement control signal for optical axis movement control is generated, and the optical axis movement control signal is output to the imaging control unit. The control unit is
Further comprising a drive status information generation unit for generating drive status information of the operating body,
The micro-working robot according to claim 13 , wherein the imaging signal generation unit receives the signal based on the drive status information as an input and generates the control signal. The imaging signal generation unit,
Based on the input drive status information,
When it is determined that the operating body is located outside the observation field of view of the imaging unit,
An imaging range control signal for imaging range movement / enlargement control for controlling the imaging unit so that the operating body and the target are located within the range of the observation visual field is generated, and the imaging range control signal is used for the imaging control. The micro-working robot according to claim 17 , wherein the micro-working robot outputs the data to a unit. The imaging signal generation unit,
Based on the input drive status information,
When it is determined that the drive control unit performs the out-of-range movement control for moving the operating body to the outside of the observation field of view of the imaging unit,
Before the operating body is located outside the range of the observation visual field, the operating body is controlled so that the operating body is located within the range of the observation visual field together with the target when the movement control outside the range is completed. The micro work robot according to claim 17 , wherein an imaging range control signal for imaging range movement / enlargement control is generated, and the imaging range control signal is output to the imaging control unit. An imaging unit that has at least one lens optical unit for observing a subject in an object, and obtains image data by capturing an observation image formed by light passing through the lens optical unit;
An arm portion that holds the operating body that can come into contact with the subject so as to be drivable at an interval narrower than the resolution of the observation image;
A drive control unit that controls the drive of the operating body and the arm unit;
An imaging control unit that controls the imaging unit;
A control unit that controls the drive control unit and the imaging control unit in cooperation with each other;
The control unit includes a coordinate system matching unit that associates the coordinate system of the imaging unit with the coordinate system of the operating body,
A subject measuring unit for measuring the position of the subject,
A subject coordinate system setting unit that sets the coordinate system of the subject based on the measurement result of the subject measuring unit;
Further equipped with,
The coordinate system matching unit receives a signal indicating the coordinate system of the subject as an input and associates the coordinate system of the subject with the coordinate system of the operating body,
A drive signal generation unit that generates a control signal to the drive control unit based on an input signal from the coordinate system matching unit;
The coordinate system matching unit,
A micro-working robot, wherein a prohibited area in which the operating body should not be located is set based on the coordinate system of the subject. The drive signal generation unit,
When the drive control unit performs a prohibited area movement control for moving the operating body into the prohibited area, a recovery control signal for recovery control for moving the operating body outside the prohibited area is generated. The micro-working robot according to claim 20 , wherein the recovery control signal is output to the drive control unit. The drive signal generation unit,
When it is determined that there is a prohibited area movement input for moving the operating body to the prohibited area,
Before the operating body reaches the prohibited area, according to claim 20 for generating a stop control signal for the stop control for stopping the movement of the operating member, and outputs the suspension control signal to the drive control unit Micro working robot. The drive signal generation unit,
When it is determined that the operation body has the prohibition area passing input that passes through the prohibition area,
Before the operating body reaches the prohibited area, a new moving route that does not pass through the prohibited area is calculated, and an avoidance control signal for avoiding control for moving the operating body on the new moving route is generated. The micro-working robot according to claim 20 , wherein the avoidance control signal is output to the drive control unit. The drive signal generation unit,
When it is determined that there is a prohibited area movement input for moving the operating body to the prohibited area,
Claim for generating a control signal for the notification control indicating that there is the prohibited area movement input to at least one of the viewer's operator and the image data in the basis of the generated display image operating the operating body 20 Micro-working robot described in. An imaging unit that has at least one lens optical unit for observing a subject in an object, and obtains image data by capturing an observation image formed by light passing through the lens optical unit;
An arm portion that holds the operating body that can come into contact with the subject so as to be drivable at an interval narrower than the resolution of the observation image;
A drive control unit that controls the drive of the operating body and the arm unit;
An imaging control unit that controls the imaging unit;
A control unit that controls the drive control unit and the imaging control unit in cooperation with each other;
The control unit includes a coordinate system matching unit that associates the coordinate system of the imaging unit with the coordinate system of the operating body,
A subject measuring unit for measuring the position of the subject,
A subject coordinate system setting unit that sets the coordinate system of the subject based on the measurement result of the subject measuring unit;
Further equipped with,
The coordinate system matching unit receives a signal indicating the coordinate system of the subject as an input and associates the coordinate system of the subject with the coordinate system of the operating body,
A drive signal generation unit that generates a control signal to the drive control unit based on an input signal from the coordinate system matching unit;
The coordinate system matching unit receives the signal indicating the coordinate system of the subject as an input, associates the coordinate system of the subject with the coordinate system of the imaging unit, and determines the target designated as a part of the observation image. Calculate the position in the coordinate system of the subject,
The drive signal generation unit,
Based on the position of the target in the coordinate system of the subject, calculate the standby position of the operating body,
And wherein the operating body generates a waiting position control signal for the standby position movement control for controlling the arm portion to move to the standby position, and outputs the waiting position control signal to the drive control unit micro work robot to be. The drive signal generation unit outputs the control signal to the drive control unit on condition that a permission signal is input. Any one of claims 7, 8 and 21 to 25. Micro work robot described. The image pickup signal generation unit outputs the control signal to the image pickup control unit on condition that a permission signal is input. 10. The claim 9, claim 12, claim 14 to claim 16, and claim 14. Item 19. The micro work robot according to any one of items 18 .