JP2023139442A - Image processing system and image processing method - Google Patents

Abstract

To provide an image processing system and an image processing method capable of properly focusing on a surgery target in a case of executing a surgical operation on the surgery target by using a plurality of surgical instruments.SOLUTION: The image processing system comprises: an acquisition processing unit which acquires a captured image from an image-capturing unit which captures an image of a surgery target; a detection processing unit which detects identification mediums provided respectively on a plurality of surgical instruments, in the captured image acquired by the acquisition processing unit; and a control processing unit which controls the imaging unit so as to focus on a position corresponding to a first surgical instrument identified based on priority order corresponding to the respective identification mediums detected by the detection processing unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing system and an image processing method.

BACKGROUND ART Conventionally, when an operator performs a surgery while displaying a captured image of a region to be operated on a display, a system is known that appropriately displays the region (for example, see Patent Document 1).

Japanese Patent Application Publication No. 2015-228955

With conventional technology, when using a specific surgical instrument, it is possible to detect the position of the surgical instrument and focus on the surgical target area, but when multiple surgical instruments are used, the surgical target can be focused on depending on the surgical instrument. It is difficult to properly focus on this area.

An object of the present invention is to provide an image processing system and an image processing method that are capable of appropriately focusing a surgical target when performing surgery on the surgical target using a plurality of surgical instruments.

An image processing system according to one aspect of the present invention includes: an acquisition processing unit that acquires a captured image from an imaging unit that images a surgical target; a detection processing unit that detects an identification medium provided in the detection processing unit; and a detection processing unit that detects an identification medium provided in the detection processing unit; and a control processing unit that controls the imaging unit so that the imaging unit is in focus.

An image processing system according to one aspect of the present invention includes: an acquisition processing unit that acquires a captured image from an imaging unit that images a surgical target; a detection processing unit that detects an identification medium provided therein; and a detection processing unit that detects a predetermined identification medium set in advance on the first surgical instrument from the predetermined position of the first surgical instrument specified based on the identification medium detected by the detection processing unit. and a control processing unit that controls the imaging unit so that the imaging unit is focused at a position separated by a set distance.

An image processing method according to another aspect of the present invention includes an acquisition step in which one or more processors acquire a captured image from an imaging unit that images a surgical target, and a plurality of captured images acquired in the acquisition step. a detection step of detecting identification media provided on each of the plurality of surgical instruments; and a first surgical instrument identified based on a priority order corresponding to each of the plurality of identification media detected in the detection step. and a control step of controlling the imaging section so that the imaging section is focused on a position.

An image processing method according to another aspect of the present invention includes an acquisition step in which one or more processors acquire a captured image from an imaging unit that images a surgical target; a detection step of detecting an identification medium provided at a predetermined position of the instrument; and a detection step of detecting an identification medium provided at a predetermined position of the instrument; The image processing method includes a control step of controlling the imaging unit so that the imaging unit is focused on a position separated by a set distance.

According to the present invention, it is possible to provide an image processing system and an image processing method that can appropriately focus on a surgical target when performing surgery on the surgical target using a plurality of surgical instruments.

FIG. 1 is a functional block diagram showing the configuration of an image processing system according to an embodiment of the present invention. FIG. 2A is a diagram illustrating an example of a surgical instrument according to an embodiment of the present invention. FIG. 2B is a diagram illustrating an example of a surgical instrument according to an embodiment of the present invention. FIG. 2C is a diagram illustrating an example of a surgical instrument according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of appliance information according to Embodiment 1 of the present invention. FIG. 4 is a diagram showing an example of a display screen according to an embodiment of the present invention. FIG. 5 is a diagram showing an example of a display screen according to an embodiment of the present invention. FIG. 6 is a diagram showing an example of a display screen according to an embodiment of the present invention. FIG. 7 is a diagram showing an example of a display screen according to an embodiment of the present invention. FIG. 8 is a flowchart for explaining an example of the procedure of image control processing executed in the image processing system according to the first embodiment of the present invention. FIG. 9 is a diagram showing an example of a surgical instrument according to Embodiment 2 of the present invention. FIG. 10 is a diagram showing an example of appliance information according to Embodiment 2 of the present invention. FIG. 11 is a flowchart for explaining an example of the procedure of image control processing executed in the image processing system according to Embodiment 2 of the present invention.

Embodiments of the present invention will be described below with reference to the accompanying drawings to provide an understanding of the present invention. Note that the following embodiments are examples of embodying the present invention, and do not limit the technical scope of the present invention.

[Embodiment 1]
[Image processing system 10]
As shown in FIG. 1, an image processing system 10 according to Embodiment 1 of the present invention includes an image processing device 1, an imaging section 2, a display 3, and a plurality of surgical instruments. The image processing device 1, the imaging unit 2, and the display 3 can communicate via a communication network N1 such as a LAN, the Internet, a WAN, or a public telephone line.

The image processing system 10 is introduced into a surgical facility such as a hospital. The image processing device 1 is installed inside or outside the hospital, and the imaging unit 2 and display 3 are installed in an operating room within the hospital. In addition, a plurality of surgical instruments used by surgeons such as doctors are arranged in the operating room. In FIG. 1, three surgical instruments A, B, and C are illustrated. For example, in an operating room, the imaging unit 2 images a surgical target (such as a patient's surgical site), and the display 3 displays the captured image of the imaging unit 2.

The imaging unit 2 is a digital camera equipped with an autofocus function. The imaging unit 2 may be configured with a well-known camera. There may be one imaging unit 2, or there may be multiple imaging units.

The surgeon performs the surgery using surgical instruments while viewing the photographed image displayed on the display 3, as necessary. The image processing device 1 controls the operations of the imaging section 2 and the display 3. For example, the image processing device 1 executes processing to focus the imaging unit 2 on the surgical target (focus processing). The image processing device 1 supports the surgeon's surgery by performing the focus processing.

In this embodiment, the image processing device 1 alone corresponds to the image processing system according to the present invention, but the image processing system according to the present invention is configured to include the image processing device 1, the imaging section 2, and the display 3. Good too.

The surgical instrument according to this embodiment is provided with an identification medium that can identify the surgical instrument. FIG. 2A shows a surgical instrument A that is a laser scalpel, FIG. 2B shows a surgical instrument B that is a tweezers, and FIG. 2C shows a surgical instrument C that is a forceps. A marker M1 is provided at the distal end of the surgical instrument A, a marker M2 is provided at the distal end of the surgical instrument B, and a marker M3 is provided at the distal end of the surgical instrument C. Marker M1 is associated with identification information of surgical instrument A (instrument name, instrument ID, etc.), and marker M2 is associated with identification information of surgical instrument B (instrument name, instrument ID, etc.). The marker M3 is associated with identification information of the surgical instrument C (instrument name, instrument ID, etc.). The markers M1, M2, M3 may be composed of stickers of different colors (red, blue, green, etc.). The identification medium is not limited to a marker, but may be a one-dimensional code or a two-dimensional code.

[Image processing device 1]
As shown in FIG. 1, the image processing device 1 is a server that includes a control section 11, a storage section 12, an operation display section 13, a communication section 14, and the like. Note that the image processing apparatus 1 is not limited to one computer, but may be a computer system in which a plurality of computers work together. Furthermore, various types of processing executed by the image processing device 1 may be executed in a distributed manner by one or more processors.

The communication unit 14 connects the image processing device 1 to the communication network N1 by wire or wirelessly, and performs data communication with external devices such as the imaging unit 2 and the display 3 via the communication network N1 according to a predetermined communication protocol. It is a communication interface for executing.

The operation display unit 13 is a user interface that includes a display unit such as a liquid crystal display or an organic EL display that displays various information, and an operation unit such as a mouse, keyboard, or touch panel that receives operations.

The storage unit 12 is a nonvolatile storage unit such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that stores various information. Specifically, the storage unit 12 stores data such as appliance information D1. FIG. 3 is a diagram showing an example of the appliance information D1.

As shown in FIG. 3, the instrument information D1 includes information such as the corresponding "instrument name," "marker," "color," and "priority order" for each surgical instrument. The instrument name is the name (identification information) of the surgical instrument. The marker is the type of marker (marker name, marker ID) provided on the surgical instrument. The color is an example of identification information for identifying the marker. The priority order is information indicating the priority of surgical instruments. For example, the priority order is used when determining which surgical instrument should be prioritized and focused when a plurality of surgical instruments are included in the captured image of the imaging unit 2. In FIG. 3, as an example, surgical instrument A is set as the "first" priority, surgical instrument B is set as the "second" priority, and surgical instrument C is set as the "third" priority. has been done. For example, the surgeon registers the priority order in advance according to the content (step) of the surgery and the type of surgical instrument. For example, the surgeon registers the instrument information D1 in advance on the operation display section 13 of the image processing device 1 before performing the surgery.

In addition, as another embodiment, part or all of the appliance information D1 may be stored in another server that is accessible from the image processing device 1 via the communication network N1. In this case, the control unit 11 of the image processing device 1 may acquire the information from the other server and execute various processes such as image control processing (see FIG. 8), which will be described later.

Further, the storage unit 12 stores a control program such as an image processing program for causing the control unit 11 to execute image control processing (see FIG. 8), which will be described later. For example, the image control program is recorded non-temporarily on a computer-readable recording medium such as a CD or DVD, and is read by a reading device (not shown) such as a CD drive or a DVD drive included in the image processing device 1. and stored in the storage unit 12.

The control unit 11 includes control devices such as a CPU, ROM, and RAM. The CPU is a processor that executes various types of arithmetic processing. The ROM is a non-volatile storage unit that stores in advance control programs such as a BIOS and an OS for causing the CPU to execute various arithmetic processes. The RAM is a volatile or nonvolatile storage unit that stores various information, and is used as a temporary storage memory (work area) for various processes executed by the CPU. The control unit 11 controls the image processing apparatus 1 by executing various control programs stored in the ROM or the storage unit 12 in advance on the CPU.

Specifically, the control unit 11 includes various processing units such as an acquisition processing unit 111, a detection processing unit 112, a specific processing unit 113, and a focus control processing unit 114, as shown in FIG. The control unit 11 functions as the various processing units by causing the CPU to execute various processes according to the image control program. Further, a part or all of the processing section may be constituted by an electronic circuit. Note that the image control program may be a program for causing a plurality of processors to function as the processing units.

The acquisition processing unit 111 acquires a captured image from the imaging unit 2 that images the surgical target. For example, when a surgery is started, the control unit 11 outputs an imaging start instruction to the imaging unit 2. Upon acquiring the imaging start instruction, the imaging unit 2 starts imaging a predetermined imaging range (operative field). The imaging unit 2 images a shooting range at a predetermined frame rate and transmits image data of the captured image to the control unit 11. The acquisition processing unit 111 sequentially acquires the captured images from the imaging unit 2.

When the acquisition processing unit 111 acquires the captured image, the control unit 11 causes the display 3 to display the captured image. This allows the surgeon to perform the surgery while viewing the surgical target (for example, the patient's surgical site) displayed on the display 3.

The detection processing unit 112 detects markers provided on each of the plurality of surgical instruments in the captured image acquired by the acquisition processing unit 111. Furthermore, the detection processing unit 112 recognizes a marker in the captured image, and detects a surgical instrument based on the recognized marker. FIG. 4 shows an example of a screen (captured image) displayed on the display 3. For example, in the example shown in FIG. 4, the detection processing unit 112 recognizes the red color of the marker M1, detects the surgical instrument A based on the instrument name associated with the marker M1 (see FIG. 3), and detects the blue color of the marker M2. The surgical instrument B is detected based on the instrument name recognized and associated with the marker M2 (see FIG. 3), and the surgical instrument B is detected based on the instrument name recognized and associated with the marker M3 (see FIG. 3) by recognizing the green color of the marker M3. Detect surgical instrument C. In this manner, the detection processing unit 112 is able to detect at least one marker in the captured image and detect the surgical instrument corresponding to the detected marker.

The identification processing unit 113 identifies the surgical instrument to be focused (an example of the first surgical instrument of the present invention) based on the priority order corresponding to each of the plurality of markers detected by the detection processing unit 112. In the example shown in FIG. 4, since the detection processing unit 112 detects the surgical instruments A, B, and C, the identification processing unit 113 refers to the instrument information D1 (see FIG. 3) and detects the surgical instruments A, B, and C. Among them, the surgical instrument with the highest priority (here, "surgical instrument A" with the "first" priority) is specified.

For example, in the example shown in FIG. 5, since the detection processing unit 112 detects surgical instruments B and C, the identification processing unit 113 refers to the instrument information D1 (see FIG. 3) and selects one of the surgical instruments B and C. The surgical instrument with the highest priority (here, "surgical instrument B" with the "second" priority) is identified.

The focus control processing unit 114 controls the imaging unit 2 so that the imaging unit 2 focuses on the position corresponding to the surgical instrument specified by the identification processing unit 113. For example, when the identification processing unit 113 identifies the surgical instrument A, the focus control processing unit 114 focuses the imaging unit 2 on the position of the marker M1 of the surgical instrument A, as shown in FIG. For example, when the identification processing unit 113 identifies the surgical instrument B, the focus control processing unit 114 focuses the imaging unit 2 on the position of the marker M2 of the surgical instrument B, as shown in FIG.

In this way, the focus control processing section 114 causes the imaging section 2 to focus on the position corresponding to the surgical instrument specified based on the priority order corresponding to each of the plurality of surgical instruments detected by the detection processing section 112. The imaging unit 2 is controlled as follows.

[Image control processing of Embodiment 1]
Image control processing executed in the image processing system 10 according to the first embodiment will be described below with reference to FIG. 8. Specifically, in this embodiment, the image control process is executed by the control unit 11 of the image processing device 1.

Note that the present invention can be regarded as an invention of an image processing method that executes one or more steps included in the image control processing. Furthermore, one or more steps included in the image control processing described here may be omitted as appropriate. Note that the steps in the image control processing may be executed in a different order as long as similar effects are produced. Furthermore, although the case where the control unit 11 executes each step in the image control processing will be described as an example, an image processing method in which one or more processors execute each step in the image control processing in a distributed manner Other embodiments are also possible.

First, in step S1, the control unit 11 determines whether a captured image has been acquired from the imaging unit 2. When the control unit 11 acquires the captured image from the imaging unit 2 (S1: Yes), the control unit 11 shifts the process to step S2. Furthermore, upon acquiring the captured image, the control unit 11 causes the display 3 to display the captured image. The control unit 11 waits until a captured image is acquired from the imaging unit 2 (S1: No).

Next, in step S2, the control unit 11 determines whether a marker is detected in the captured image. When the control unit 11 detects a marker in the captured image (S2: Yes), the control unit 11 moves the process to step S3. When the control unit 11 does not detect a marker in the captured image, the control unit 11 moves the process to step S1 (S2: No). As shown in FIGS. 4 and 5, when a marker is included in the captured image, the control unit 11 detects the marker. The control unit 11 recognizes, for example, the color of the marker.

Next, in step S3, the control unit 11 determines whether a plurality of markers have been detected. When the control unit 11 determines that there are a plurality of detected markers (S3: Yes), the control unit 11 moves the process to step S4. On the other hand, if the control unit 11 determines that the number of detected markers is one (S3: No), the control unit 11 shifts the process to step S31. In the example shown in FIGS. 4 and 5, the control unit 11 determines that a plurality of markers are detected.

In step S4, the control unit 11 acquires the priority order associated with each of the plurality of detected markers. Specifically, the control unit 11 specifies the marker based on the color of the detected marker, and refers to the appliance information D1 (see FIG. 3) to obtain the priority order associated with the specified marker. In the example shown in FIG. 4, the control unit 11 recognizes the red color of the marker M1 and acquires the priority "1" associated with the marker M1, and recognizes the blue color of the marker M2 and acquires the priority "1" associated with the marker M2. The priority level "2" is acquired, and the green color of the marker M3 is recognized to acquire the priority level "3" associated with the marker M3.

Further, in the example shown in FIG. 5, the control unit 11 recognizes the blue color of the marker M2 and obtains the priority "2" associated with the marker M2, and recognizes the green color of the marker M3 and obtains the priority "2" associated with the marker M3. Get priority "3".

Next, in step S5, the control section 11 controls the imaging section 2 so that the imaging section 2 is focused on the position corresponding to the surgical instrument having the highest priority. For example, in the example shown in FIG. 4, the control unit 11 focuses the imaging unit 2 on the position of the marker M1 of the surgical instrument A having the first priority (see FIG. 6). For example, in the example shown in FIG. 5, the control unit 11 focuses the imaging unit 2 on the position of the marker M2 of the surgical instrument B having the second priority (see FIG. 7). After step S5, the control unit 11 moves the process to step S6.

On the other hand, in step S31, the control unit 11 controls the imaging unit 2 so that the imaging unit 2 is focused on the position corresponding to the surgical instrument corresponding to the one detected marker. After step S31, the control unit 11 moves the process to step S6.

In step S6, the control unit 11 determines whether the surgery has ended. When the surgery is completed (S6: Yes), the control unit 11 ends the image control process. On the other hand, if the surgery has not been completed (S6: No), the control unit 11 returns the process to step S1 and repeats the above-described process. In this manner, the control unit 11 performs a focus process to focus the imaging unit 2 at a predetermined position while detecting the surgical instrument in the captured image during the surgery.

As described above, the image processing device 1 according to the present embodiment acquires a captured image from the imaging unit 2 that images a surgical target, and in the acquired captured image, identifies the identification provided on each of a plurality of surgical instruments. Detect multiple surgical instruments based on media (markers). The image processing device 1 also controls the imaging unit 2 so that the imaging unit 2 is focused on a position corresponding to the first surgical instrument specified based on the priority order corresponding to each of the plurality of detected surgical instruments. Control.

Thereby, when performing surgery on a surgical target using a plurality of surgical instruments, it becomes possible to appropriately focus on the surgical target. Furthermore, by setting priorities for each of the plurality of surgical instruments according to the surgical process, for example, the focus of the imaging unit 2 can be controlled according to the surgical process.

[Embodiment 2]
An image processing system 10 according to a second embodiment of the present invention will be described. Below, description of the same configuration as the image processing system 10 according to the first embodiment will be omitted.

Here, for example, when the focus of the imaging unit 2 is adjusted to the position of a marker of a surgical instrument, the focus may be slightly shifted from the position of the surgical target (surgical site). For example, as shown in FIG. 9, when the focus of the imaging unit 2 is adjusted to the position of the marker M1 of the surgical instrument A, the focus shifts by a distance L1 from the position X1 of the surgical object.

Therefore, the image processing system 10 according to the second embodiment is configured to focus the imaging unit 2 on the position X1 of the surgical target.

Specifically, in the image processing system 10 according to the second embodiment, the detection processing unit 112 detects an identification medium (marker) provided at a predetermined position of the surgical instrument in the captured image acquired by the acquisition processing unit 111. To detect. Further, the focus control processing unit 114 captures an image at a position that is a predetermined distance away from the predetermined position of the surgical instrument specified based on the identification medium detected by the detection processing unit 112. The imaging section 2 is controlled so that the imaging section 2 is in focus.

Specifically, the storage unit 12 stores data of appliance information D2 shown in FIG. FIG. 10 is a diagram showing an example of the appliance information D2.

As shown in FIG. 10, the instrument information D2 includes information such as the corresponding "instrument name," "marker," "color," and "setting distance" for each surgical instrument. The instrument name, the marker, and the color are the same as the instrument information D1 shown in FIG. 3. The set distance is the distance from the position of the marker provided on the surgical instrument to the tip of the surgical instrument. The tip of the surgical instrument corresponds to the location of the surgical target. For example, a set distance L1 (see FIG. 9) is set for surgical instrument A, a set distance L2 is set for surgical instrument B, and a set distance L3 is set for surgical instrument C. The set distance may be a distance to a position further beyond the tip of the surgical instrument. Further, the set distance may be set to a different value for each surgical instrument, or may be set to the same value.

The focus control processing unit 114 refers to the instrument information D2, obtains a set distance associated with the marker detected by the detection processing unit 112, and moves the imaging unit to a position that is the set distance away from the position of the marker of the surgical instrument. Focus on 2.

[Image control processing of Embodiment 2]
Image control processing executed in the image processing system 10 according to the second embodiment will be described below with reference to FIG. 11. Below, description of the same processing as the image control processing (see FIG. 8) according to the first embodiment will be omitted.

When the control unit 11 detects the marker in the captured image (S22: Yes), in step S23, the control unit 11 identifies the marker based on the color of the detected marker, and refers to the instrument information D2 (see FIG. 10) to identify the marker. Gets the set distance associated with a marker.

Next, in step S24, the control unit 11 calculates the focal position of the imaging unit 2. For example, the control unit 11 calculates the distance from the imaging unit 2 to the marker using a focal length of a lens, a distance measuring sensor, and the like. Then, the control unit 11 adds the set distance acquired in step S23 to the calculated distance from the imaging unit 2 to the marker, and sets the calculated value as the focal position.

Next, in step S25, the control unit 11 focuses the imaging unit 2 on the calculated focal position.

In this way, the image processing device 1 according to the second embodiment acquires a captured image from the imaging unit 2 that images the surgical target, and uses an identification medium (marker) provided at a predetermined position of a surgical instrument in the acquired captured image. ) is detected. Further, the image processing device 1 (focus control processing unit 114) moves the first surgical instrument a preset distance away from the predetermined position of the first surgical instrument specified based on the detected identification medium. The imaging unit 2 is controlled so that the imaging unit 2 is focused at the position. Thereby, the focus of the imaging unit 2 can be adjusted to the tip of the surgical instrument, so that the position X1 (see FIG. 9) of the surgical target can be focused.

[Other embodiments]
The present invention is not limited to the first and second embodiments described above. Other embodiments of the present invention will be described below.

For example, when surgical instrument B is in focus at a position corresponding to marker M2, if the surgeon pulls surgical instrument B largely toward the user (toward the imaging unit 2 side), the focal point will move significantly toward the user. . For example, the above problem occurs when the surgeon pulls the thread forward with tweezers (surgical instrument B). In such a case, the control unit 11 may be configured not to perform focus processing in order to prevent the focal position of the imaging unit 2 from moving toward the front. Specifically, when the focal length of the lens with respect to the detected marker is out of a predetermined range, the control unit 11 excludes the marker from the control target for focusing the imaging unit 2 . The predetermined range is set to a range that is farther than a position that is a predetermined distance away from the imaging unit 2. This can prevent the focal position from changing significantly when the surgical instrument is pulled toward the user.

Further, for example, the detection processing unit 112 may be able to recognize the presence of the marker but may not be able to determine the type of marker due to the influence of dirt on the marker, orientation, distance to the lens, and the like. In this case, it becomes difficult to appropriately focus the imaging section 2. Therefore, as another embodiment, the distance at which the imaging unit 2 is focused (focal length) may be set in advance. For example, the focal length is set to a distance that is longer than the position of the surgical instrument (the position of the marker) when performing a normal surgery. When the control unit 11 determines that the type of marker cannot be determined, the control unit 11 focuses on a position having a preset focal length.

As a result, when it is impossible to focus on the marker because the surgical instrument is too close to the lens of the imaging unit 2, the focus can be adjusted to a position close to the position expected to be a little ahead of the surgical instrument. . Therefore, it becomes easier to confirm the vicinity of the position of the surgical target (surgical site). Note that the focus position may be a central position within the display screen or a preset position.

In this way, the focus control processing section 114 may adjust the focus of the imaging section 2 to a position having a preset focal length when the type of marker detected by the detection processing section 112 cannot be determined. .

Furthermore, for example, the detection processing unit 112 may not be able to recognize the marker itself. For example, if there is no image of a surgical instrument in the captured image, the detection processing unit 112 cannot recognize the marker. In this case, the control unit 11 focuses the imaging unit 2 on the entire imaging range in order to make it easier to understand the entire imaging range. For example, the focus control processing unit 114 further narrows down the aperture of the lens to widen the depth of field. This widens the depth of field of the imaging unit 2, so it is possible to widen the in-focus range.

However, if you close down the lens aperture while keeping the exposure the same, the shutter speed will slow down. There is no problem as long as the shutter speed is slow within the range that can be used as an image, but depending on the shooting conditions at the time, the shutter speed may drop to a range that cannot be used as an image. Therefore, by controlling the screen so that it becomes bright, the shutter speed does not drop. Furthermore, if the imaging unit 2 is equipped with an ND filter, the value of the ND filter is reduced. This makes it possible to brighten the entire screen without reducing image quality, making it possible to increase the shutter speed or avoid decreasing it. The lighting may also be brighter if the lighting system can be controlled. Furthermore, if the shutter speed cannot be increased even with these measures, or if these measures are not available, the sensitivity of the imaging section 2 may be increased. This allows you to increase the shutter speed without changing the exposure.

In this way, when the detection processing section 112 cannot detect a marker, the focus control processing section 114 widens (increases) the depth of field of the imaging section 2. Further, when the marker cannot be detected, the control unit 11 further narrows down the aperture of the lens of the imaging unit 2, checks the shutter speed, and checks whether it is lower than a preset value. When the shutter speed is lower than a preset value, the control unit 11 reduces the value of the ND filter, controls the illumination system to brighten the surgical field, or increases the sensitivity of the camera.

Furthermore, for example, during a surgical procedure, the surgical site often advances from laparotomy and thoracotomy to deep parts of the body. As the surgery progresses, the position of the surgical instrument gradually changes from the front position to the far position when viewed from the imaging unit 2 side. Here, when a plurality of surgical instruments are present in the surgical field, there are cases where it is desired to focus the imaging unit 2 on the tip of the surgical instrument being operated at the farthest position among them. In this case, if the focus of the imaging unit 2 is automatically set in advance to the marker located at the farthest distance, the effort of selecting the priority order shown in the first embodiment can be saved.

Further, depending on the type of surgery and the situation, it may be necessary to focus the imaging unit 2 on the surgical instrument closest to the user. In this case, it is effective to automatically focus the imaging unit 2 on the marker located at the closest position.

Therefore, the focus control processing section 114 selects the surgical instrument that is the farthest from the imaging section 2 or the distance from the imaging section 2 among the plurality of surgical instruments corresponding to each of the plurality of markers detected by the detection processing section 112. The imaging section 2 may be controlled so that the imaging section 2 is focused on a position corresponding to the surgical instrument closest to the surgical instrument.

As another embodiment, when the detection processing section 112 detects a plurality of markers, the focus control processing section 114 may switch the marker to be controlled to focus the imaging section 2 according to instructions from the operator. . For example, when a surgeon operates a foot pedal installed in the operating room, operates an operation switch installed on a surgical instrument, performs a touch operation on the display 3, or utters a predetermined voice (keyword). In this case, the focus control processing unit 114 switches the marker to be controlled.

As another embodiment, the control unit 11 may be configured to be able to change the color of the marker provided on each surgical instrument. For example, when the marker is composed of a light emitting element, the control unit 11 changes the color of the light emitted from the marker of each surgical instrument. For example, the priority of red is set (fixed) to "first", the priority of blue to "second", and the priority of green to "third". In this case, if it is desired to change the priority of the surgical instrument A provided with the marker M1 from the first to the second, the control unit 11 changes the luminescent color of the marker M1 from "red" to "blue". Just change it. In this way, by changing the emitted light color, it becomes possible to easily change the priority order of surgical instruments. Note that the identification information of the marker is not limited to the color, but may be the blinking frequency. In this way, the present invention may be configured such that the marker is composed of a light emitting element, and the luminescent color or blinking frequency of each of the plurality of markers is mutually switched.

By the way, once the focal point of the imaging unit 2 is at the position of the surgical target (surgical site), there are cases where the focal point does not want to be moved (where the focal position is desired to be fixed). Therefore, as another embodiment, the focus control processing section 114 may fix the focus of the imaging section 2 in accordance with an instruction from the operator after adjusting the focus of the imaging section 2 to a predetermined position. The operator's instructions include, for example, operation of the foot pedal, operation of an operation switch provided on the surgical instrument, touch operation of the display 3, and utterance of a keyword. This can prevent the focus of the imaging unit 2 from changing frequently in response to movement of the surgical instrument. Note that the operator may issue an instruction to cancel the fixed focal position. In another embodiment, the focus control processing unit 114 may fix the focus of the imaging unit 2 until a predetermined time elapses after focusing the imaging unit 2 at a predetermined position. For example, the focus control processing unit 114 focuses the imaging unit 2 at a predetermined position, fixes the focus of the imaging unit 2 for a predetermined time, and adjusts the focus again according to the position of the surgical instrument after a predetermined time has elapsed.

As another embodiment, the imaging unit 2 may include each processing unit of the image processing device 1. In other words, the image processing device 1 and the imaging section 2 may be an integrated device.

The present invention is not limited to the field of surgery, but can be applied to various fields. That is, the image processing system of the present invention can be expressed as follows.

(Configuration 1)
an acquisition processing unit that acquires a captured image from an imaging unit that captures an image of a shooting target;
a detection processing unit that detects a plurality of identification media in the captured image acquired by the acquisition processing unit;
Control for controlling the imaging unit so that the imaging unit focuses on a position corresponding to the identification medium specified based on the priority order corresponding to each of the plurality of identification media detected by the detection processing unit. a processing section;
An image processing system comprising:

(Configuration 2)
an acquisition processing unit that acquires a captured image from an imaging unit that captures an image of a shooting target;
a detection processing unit that detects an identification medium in the captured image acquired by the acquisition processing unit;
a control processing unit that controls the imaging unit so that the imaging unit focuses on a position that is a preset distance away from the identification medium from a predetermined position of the identification medium detected by the detection processing unit; ,
An image processing system comprising:

1: Image processing device 2: Imaging section 3: Display 10: Image processing system 11: Control section 12: Storage section 13: Operation display section 14: Communication section 111: Acquisition processing section 112: Detection processing section 113: Specification processing section 114 : Focus control processing unit A : Surgical instrument B : Surgical instrument C : Surgical instrument D1 : Instrument information D2 : Instrument information L1 : Setting distance L2 : Setting distance L3 : Setting distance M1 : Marker M2 : Marker M3 : Marker

Claims (12)

an acquisition processing unit that acquires a captured image from an imaging unit that captures an image of a surgical target;
a detection processing unit that detects an identification medium provided on each of a plurality of surgical instruments in the captured image acquired by the acquisition processing unit;
controlling the imaging unit so that the imaging unit focuses on a position corresponding to a first surgical instrument specified based on a priority order corresponding to each of the plurality of identification media detected by the detection processing unit; a control processing unit;
An image processing system comprising: an acquisition processing unit that acquires a captured image from an imaging unit that captures an image of a surgical target;
a detection processing unit that detects an identification medium provided at a predetermined position of a surgical instrument in the captured image acquired by the acquisition processing unit;
The focus of the imaging unit is located at a position that is a predetermined distance away from the predetermined position of the first surgical instrument that is specified based on the identification medium detected by the detection processing unit. a control processing unit that controls the imaging unit to match the imaging unit;
An image processing system comprising: A distance from the identification medium to the tip of the surgical instrument is set in advance for each of the plurality of surgical instruments;
The image processing system according to claim 2. If a focal length of the imaging unit with respect to the identification medium detected by the detection processing unit is outside a predetermined range, excluding the identification medium from a control target for focusing the imaging unit;
The image processing system according to any one of claims 1 to 3. The control processing unit adjusts the focus of the imaging unit to a preset focal length position when the type of the identification medium detected by the detection processing unit cannot be determined.
The image processing system according to any one of claims 1 to 4. When the detection processing unit cannot detect the identification medium, the control processing unit increases the depth of field of the imaging unit.
The image processing system according to any one of claims 1 to 5. The control processing section is configured to select a surgical instrument that is the farthest from the imaging section or a distance from the imaging section among the plurality of surgical instruments corresponding to each of the plurality of identification media detected by the detection processing section. controlling the imaging unit so that the imaging unit is focused on a position corresponding to the surgical instrument closest to the imaging unit;
The image processing system according to any one of claims 1 to 6. The control processing unit switches a marker to be controlled to focus the imaging unit according to an instruction from a surgeon.
The image processing system according to any one of claims 1 to 7. The identification medium is composed of a light emitting element,
mutually exchanging the luminescent colors or blinking frequencies of the plurality of identification media;
The image processing system according to any one of claims 1 to 8. The control processing section fixes the focal point position according to an instruction from an operator when the focal point of the imaging section is adjusted to a position corresponding to the first surgical instrument.
The image processing system according to any one of claims 1 to 9. one or more processors,
an acquisition step of acquiring a captured image from an imaging unit that images the surgical target;
a detection step of detecting an identification medium provided on each of a plurality of surgical instruments in the captured image obtained in the acquisition step;
Control for controlling the imaging unit so that the imaging unit focuses on a position corresponding to a first surgical instrument specified based on a priority order corresponding to each of the plurality of identification media detected in the detection step. step and
An image processing method that performs one or more processors,
an acquisition step of acquiring a captured image from an imaging unit that images the surgical target;
a detection step of detecting an identification medium provided at a predetermined position of a surgical instrument in the captured image acquired in the acquisition step;
The imaging section focuses on a position that is a preset distance away from the first surgical instrument from the predetermined position of the first surgical instrument that is identified based on the identification medium detected in the detection step. a control step of controlling the imaging unit so as to
An image processing method that performs