EP3701540A1 - Surgery system, information processing apparatus, and information processing method - Google Patents

Surgery system, information processing apparatus, and information processing method

Info

Publication number
EP3701540A1
EP3701540A1 EP18816268.9A EP18816268A EP3701540A1 EP 3701540 A1 EP3701540 A1 EP 3701540A1 EP 18816268 A EP18816268 A EP 18816268A EP 3701540 A1 EP3701540 A1 EP 3701540A1
Authority
EP
European Patent Office
Prior art keywords
surgery
displayed
display
icon
control signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP18816268.9A
Other languages
German (de)
English (en)
French (fr)
Inventor
Takahito Nakano
Takeshi Miyai
Keisuke UYAMA
Daisuke Tsuru
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Publication of EP3701540A1 publication Critical patent/EP3701540A1/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/40—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the management of medical equipment or devices, e.g. scheduling maintenance or upgrades
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00006—Operational features of endoscopes characterised by electronic signal processing of control signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000094—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope extracting biological structures
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A61B1/0005—Display arrangement combining images e.g. side-by-side, superimposed or tiled
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/25—User interfaces for surgical systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/361—Image-producing devices, e.g. surgical cameras
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/20—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation

Definitions

  • the present disclosure relates to a surgery system, an information processing apparatus, and an information processing method.
  • devices such as a camera including an endoscope apparatus or the like, a recorder, a monitor, and a light are installed in a surgery room, and surgery is performed by using such devices.
  • GUI graphical user interface
  • the present disclosure proposes a novel and improved surgery system, information processing apparatus, and information processing method that are capable of controlling a target device more easily.
  • a medical operation system comprising an information processing apparatus, including processing circuitry configured to (1) cause a map of an operating room to be displayed on a display, the map including an icon representing a device located in the operating room or accessible from the operating room, (2) receive, via a user operation on the displayed map, designation information representing a designation of a change in at least one of an input source, an output destination, and an internal setting for the device, (3) generate a control signal to control the device based on the designation information, and (4) cause the generated control signal to be transmitted to the device.
  • processing circuitry configured to (1) cause a map of an operating room to be displayed on a display, the map including an icon representing a device located in the operating room or accessible from the operating room, (2) receive, via a user operation on the displayed map, designation information representing a designation of a change in at least one of an input source, an output destination, and an internal setting for the device, (3) generate a control signal to control the device based on the designation information, and (4) cause the generated control signal to be transmitted to the device.
  • a method of controlling a device located in a medical operating room and accessible during a medical procedure in the medical operating room comprising: (1) displaying a map of the medical operating room, the map including an icon representing the device; (2) receiving, via a user operation on the displayed map, a designation of a change in one of an input source, an output destination, and an internal setting for the device; (3) generating a control signal to control the designated change, based on the received user operation; and (4) causing the generated control signal to be transmitted to the device.
  • an image processing apparatus comprising processing circuitry configured to (1) display a map of an operating room, the map including an icon representing a device located in the operating room or useable during a medical procedure in the medical operating room, (2) receive, via a user operation on the displayed map, a designation of a change in one of an input source, an output destination, and an internal setting for the device, and (3) generate a control signal to control the designated change, based on the received user operation; and a transmitter configured to transmit, to the device, the generated control signal.
  • FIG. 1 is a diagram illustrating a schematic configuration of a surgery system 1000 according to an embodiment of the present disclosure.
  • FIG. 2 is a block diagram illustrating a configuration example of an information processing apparatus 1 according to the embodiment.
  • FIG. 3 is an explanatory diagram illustrating an example of a screen that a display control unit 14 causes a display unit 40 to display.
  • FIG. 4 is a flowchart illustrating an operation example of an information processing apparatus 1 according to the embodiment.
  • FIG. 5 is an explanatory diagram illustrating a first specific example according to the embodiment.
  • FIG. 6 is an explanatory diagram illustrating the first specific example according to the embodiment.
  • FIG. 7 is an explanatory diagram illustrating the first specific example according to the embodiment.
  • FIG. 8 is an explanatory diagram illustrating the first specific example according to the embodiment.
  • FIG. 9 is an explanatory diagram illustrating a second specific example according to the embodiment.
  • FIG. 10 is an explanatory diagram illustrating the second specific example according to the embodiment.
  • FIG. 11 is an explanatory diagram illustrating the second specific example according to the embodiment.
  • FIG. 12 is an explanatory diagram illustrating the second specific example according to the embodiment.
  • FIG. 13 is an explanatory diagram illustrating the second specific example according to the embodiment.
  • FIG. 14 is an explanatory diagram illustrating a third specific example according to the embodiment.
  • FIG. 15 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 16 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 17 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 10 is an explanatory diagram illustrating the second specific example according to the embodiment.
  • FIG. 11 is an explanatory diagram illustrating the second specific example according to the embodiment.
  • FIG. 12 is an explanatory diagram
  • FIG. 18 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 19 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 20 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 21 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 22 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 23 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 24 is an explanatory diagram illustrating a fourth specific example according to the embodiment.
  • FIG. 25 is an explanatory diagram illustrating the fourth specific example according to the embodiment.
  • FIG. 26 is an explanatory diagram illustrating the fourth specific example according to the embodiment.
  • FIG. 20 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 21 is an explanatory diagram illustrating the third specific example according to the embodiment.
  • FIG. 22 is an explanatory diagram illustrating
  • FIG. 27 is an explanatory diagram illustrating the fourth specific example according to the embodiment.
  • FIG. 28 is an explanatory diagram illustrating a fifth specific example according to the embodiment.
  • FIG. 29 is an explanatory diagram illustrating a sixth specific example according to the embodiment.
  • FIG. 30 is an explanatory diagram illustrating the sixth specific example according to the embodiment.
  • FIG. 31 is an explanatory diagram illustrating the sixth specific example according to the embodiment.
  • FIG. 32 is an explanatory diagram illustrating the sixth specific example according to the embodiment.
  • FIG. 33 is an explanatory diagram illustrating the sixth specific example according to the embodiment.
  • FIG. 34 is an explanatory diagram illustrating the sixth specific example according to the embodiment.
  • FIG. 35 is an explanatory diagram illustrating the sixth specific example according to the embodiment.
  • FIG. 36 is an explanatory diagram illustrating a hardware configuration example.
  • FIG. 1 is a diagram illustrating a schematic configuration of a surgery system 1000 according to an embodiment of the present disclosure.
  • the surgery system 1000 includes an information processing apparatus 1, a plurality of surgery devices 2, and a network 5.
  • the information processing apparatus 1 is connected to the plurality of surgery devices 2 via the network 5 as illustrated in FIG. 1.
  • the information processing apparatus 1 is an information processing apparatus for integrally handling and controlling the plurality of surgery devices 2.
  • the information processing apparatus 1 may be a personal computer (PC) or a tablet terminal. Note that, a detailed functional configuration of the information processing apparatus 1 will be described later with reference to FIG. 2.
  • the surgery devices 2 may be devices present in or outside of a surgery room.
  • the surgery devices 2 may be used during surgery. Note that, although FIG. 1 illustrates three surgery devices 2A to 2C as the surgery devices 2, the number of surgery devices 2 is not specifically limited.
  • the surgery room there may be various devices that are to be used during surgery.
  • a device installed outside of the surgery room is used during surgery.
  • a recorder that stores an image captured in advance is installed in a room other than the surgery room, and the image stored in the recorder is seen during surgery by using a monitor installed in the surgery room.
  • the image is not limited to a still image.
  • the word “image” is used as an expression including a moving image.
  • the surgery devices 2 illustrated in FIG. 1 may include a surgery device present in a surgery room and a surgery device present outside of the surgery room.
  • the surgery device present outside of the surgery room is a device located accessible from the surgery room.
  • the surgery devices 2 may include a camera (imaging apparatus), a recorder (storage apparatus), a monitor (display apparatus), a light, and various other devices that may be used during surgery.
  • a camera imaging apparatus
  • a recorder storage apparatus
  • a monitor display apparatus
  • various other devices that may be used during surgery.
  • the camera that is an example of the surgery devices 2 is a transmission device having a function (example of transmission function) of capturing an image and transmitting the image to other surgery devices 2 and the information processing apparatus 1.
  • the surgery devices 2 may include a ceiling camera that is installed on a ceiling in a surgery room to capture situations in the whole surgery room, a surgical field camera that is installed on the ceiling in the surgery room to capture a place near hands of a surgeon, and a camera such as an endoscope or a microscope for observing an inside of a body cavity of a patient, for example.
  • the recorder that is an example of the surgery devices 2 stores images captured by the cameras during surgery, images captured in advance, or the like.
  • the recorder that is an example of the surgery devices 2 may store a computed tomography (CT) image captured during an examination carried out before surgery.
  • CT computed tomography
  • the recorder that is an example of the surgery devices 2 is a transmission/reception device having a function (example of reception function) of receiving images from other surgery devices 2 and a function (example of transmission function) of transmitting images to other surgery devices 2 and the information processing apparatus 1.
  • the monitor that is an example of the surgery devices 2 displays an image captured by the camera during surgery, an image stored in the recorder, or the like.
  • the monitor that is an example of the surgery devices 2 is a reception device having a function (example of reception function) of receiving images from other surgery devices 2.
  • the monitor that is an example of the surgery devices 2 may also have a function of dividing a display area and simultaneously displaying a plurality of images.
  • the network 5 is a wired or wireless communication path through which information is transmitted from devices connected to the network 5.
  • the network 5 may be a local area network (LAN) or the like established in a hospital including the insides and outsides of surgery rooms.
  • the network 5 is not limited thereto.
  • the network 5 may include a public network such as the Internet, a telephone network, and a satellite communication network, various LANs including Ethernet (registered trademark), a wide area network (WAN), and the like.
  • the network 5 may include a dedicated network such as an internet protocol-virtual private network (IP-VPN).
  • FIG. 1 illustrates the example in which the information processing apparatus 1 and the surgery devices 2 are connected via the network 5, one or all of the surgery devices 2 may be directly connected to the information processing apparatus 1.
  • the information processing apparatus 1 has a function of carrying out control regarding the surgery devices 2, and provides a GUI for operating the surgery devices 2. Detailed configuration of the information processing apparatus 1 will be described later.
  • the information processing apparatus 1 may display a layout screen that shows arrangement of the surgery devices 2, and receive user operation performed on the layout screen.
  • the layout screen is a map of an operating room (a surgery room).
  • the user operations provide designation information representing a designation of a change in at least one of an input source, an output destination, and an internal setting for the device. Such user operations that provide the designation information will be described in more detail below.
  • the information processing apparatus 1 may carry out control regarding the surgery devices 2 on the basis of user operation performed on the layout screen.
  • the arrangement of the surgery devices 2 may be changed in accordance with contents and procedures of surgery. According to such a configuration, the user is capable of intuitively operating a desired surgery device 2.
  • the user is capable of distinguishing a plurality of surgery devices 2 and operating a desired surgery device 2 even in the case where the surgery system 1000 includes the plurality of surgery devices 2 of the same type.
  • FIG. 2 is a block diagram illustrating a configuration example of the information processing apparatus 1 according to the present embodiment.
  • the information processing apparatus 1 includes a control unit 10, an operation unit 30, a display unit 40, a storage unit 50, and a communication unit 60. Note that, hereinafter, an overall configuration of the information processing apparatus 1 will be described first, and then a detailed configuration of the control unit 10 will be described.
  • the control unit 10 functions as an arithmetic processing apparatus and a control apparatus to control entire operation in the information processing apparatus 1 in accordance with various kinds of programs.
  • the control unit 10 according to the embodiment functions as an arrangement information control unit 12, a display control unit 14, and a device control unit 16 as illustrated in FIG. 2. Note that, functions of the control unit 10 as the arrangement information control unit 12, the display control unit 14, and the device control unit 16 will be described later.
  • the operation unit 30 receives operation performed by a user (user operation).
  • the operation unit 30 receives user operation performed on various kinds of screens that the display control unit 14 (to be described later) causes the display unit 40 to display.
  • the operation unit 30 may be implemented by a mouse, a keyboard, a touchscreen, a button, a switch, a lever, a dial, or the like.
  • the display unit 40 displays various kinds of screens under the control of the display control unit 14 (to be described later). Note that, although the operation unit 30 and the display unit 40 are illustrated as separate structural elements in the example illustrated in FIG. 2, the information processing apparatus 1 may include a touchscreen display having the functions of the operation unit 30 and the functions of the display unit 40.
  • the storage unit 50 stores data such as programs and parameters to be used by the control unit 10 executing the above-described functions.
  • the storage unit 50 may store data for the display control unit 14 to cause the various kinds of screens to be displayed.
  • the storage unit 50 may store arrangement information regarding arrangement of the surgery devices 2, information regarding functions of the surgery devices 2, various kinds of icons, and the like.
  • the communication unit 60 is a communication interface that mediates communication between the information processing apparatus 1 and other apparatuses.
  • the communication unit 60 supports any wireless or wired communication protocol, and directly establishes communication connection with the other apparatuses or establishes communication connection with the other apparatuses via the network 5 described above with reference to FIG. 1, for example.
  • the communication unit 60 may transmit control signals generated by the device control unit 16 (to be described later) to surgery devices 2.
  • the communication unit 60 may receive, from the surgery devices 2, information regarding statuses of the surgery devices 2, information acquired by the surgery devices 2 (such as images that are being captured), information stored in the surgery devices 2 (such as images captured in advance), and the like.
  • control unit 10 The overall configuration example of the information processing apparatus 1 has been described above. Next, details of the functions of the control unit 10 as the arrangement information control unit 12, the display control unit 14, and the device control unit 16 will be described.
  • the arrangement information control unit 12 provides arrangement information related to arrangement of the surgery devices 2 to the display control unit 14.
  • the arrangement information may include information of positions, attitudes, and the like of the surgery devices 2, for example.
  • the information of the position of the surgery device 2 may include information indicating in which room the surgery device 2 is present, and may further include information indicating at which position in the room the surgery device 2 is present.
  • the arrangement information control unit 12 may generate arrangement information on the basis of user operation received through the operation unit 30.
  • the arrangement information control unit 12 may automatically generate the arrangement information.
  • the arrangement information control unit 12 may recognize positions and attitudes of respective surgery devices 2 among the surgery devices 2 and generate the arrangement information on the basis of an image received from a camera (such as the surgical field camera or the like) installed at a higher perspective position.
  • the arrangement information control unit 12 may generate arrangement information on the basis of the beacons.
  • the arrangement information control unit 12 may cause the storage unit 50 to store the generated arrangement information. Subsequently, the arrangement information control unit 12 may select a piece of arrangement information corresponding to current arrangement of the surgery devices 2 from a plurality of pieces of arrangement information that have been generated in advance and that are stored in the storage unit 50. For example, such selection may be carried out on the basis of user operation. For example, the selection may be carried out in accordance with arrangement instructions corresponding to respective procedures or respective doctors. This configuration enables reduction in load related to generation of arrangement information.
  • arrangement information may be generated by another apparatus.
  • the arrangement information control unit 12 may provide the display control unit 14 with arrangement information received by the communication unit 60 from the another apparatus.
  • the display control unit 14 causes the display unit 40 to display various kinds of screens.
  • the wording “screen” means one display set displayed on the display unit 40.
  • the display control unit 14 may cause the display unit 40 to simultaneously display a plurality of images.
  • one screen may include another screen.
  • FIG. 3 is an explanatory diagram illustrating an example of the screen that the display control unit 14 causes the display unit 40 to display.
  • the display unit 40 is divided into four display areas including a display area V100, a display area V200, a display area V300, and a display area V400.
  • the display area V100 displays nothing.
  • the display control unit 14 may cause various kinds of screens to be displayed in the display area V100 on the basis of user operation performed on a screen displayed in another display area.
  • the display control unit 14 can cause a setting screen to be displayed in the display area V100 on the basis of user operation performed on a layout screen (to be described later) displayed in the display area V200.
  • the setting screen is used for configuring settings of a surgery device 2. A specific example of such display control will be described later with reference to FIG. 9 to FIG. 12 and the like.
  • the display control unit 14 may cause a preview screen to be displayed on the basis of user operation performed on the layout screen (to be described later) displayed in the display area V200 or a device list screen (to be described later) displayed in the display area V300.
  • the preview screen includes an image transmitted from a surgery device 2. A specific example of such display control will be described later with reference to FIG. 14 to FIG. 23 and the like.
  • the display control unit 14 causes a layout screen to be displayed in the display area V200.
  • the layout screen shows arrangement of surgery devices 2 present in a surgery room.
  • the display control unit 14 may cause the layout screen to be displayed on the basis of arrangement information acquired from the arrangement information control unit 12. Note that, it is only necessary for the layout screen that the display control unit 14 causes to be displayed, to be a screen showing arrangement of at least one of the plurality of surgery devices 2 connected to the information processing apparatus 1.
  • the display control unit 14 may cause icons V211 to V213, V221, V231 to V233, and V241 to V242 that correspond to surgery devices 2 to be displayed in the layout screen.
  • the display control unit 14 may cause respective texts to be displayed near the icons.
  • the respective texts indicate types of the surgery devices 2 corresponding to the icons.
  • the icons V211 to V213 are icons corresponding to respective cameras that are examples of the surgery devices 2.
  • the icon V221 is an icon corresponding to a recorder that is an example of the surgery devices 2.
  • the icons V231 to V233 are icons corresponding to respective monitors that are examples of the surgery devices 2.
  • the icons V241 to V242 are icons corresponding to other types of respective surgery devices 2.
  • the display control unit 14 may cause the icons corresponding to the surgery devices 2 to be displayed on the basis of positions and attitudes of the surgery devices 2 in a real space (in the surgery room in the example illustrated in FIG. 3).
  • the display control unit 14 causes the icons corresponding to the surgery devices 2 to be displayed at positions in the layout screen in accordance with positions of the surgery devices 2 in the real space.
  • the display control unit 14 causes the icons corresponding to the surgery devices 2 to be displayed with attitudes in the layout screen in accordance with attitudes (such as angles) of the surgery devices 2 in the real space.
  • the display control unit 14 causes the device list screen to be displayed in the display area V300.
  • the device list screen includes icons V311 to V313, V321 to V323, and V331 to V333 corresponding to the surgery devices 2 present in or outside of the surgery room.
  • icons corresponding to the surgery devices 2 present outside of the surgery room are displayed in the device list screen such that the icons are classified by functions of the surgery devices 2.
  • the display control unit 14 may display icons corresponding to surgery devices 2 on the basis of functions of the surgery devices 2. For example, the display control unit 14 may cause icons to be displayed such that the icons are displayed with colors, patterns, or shapes in accordance with functions of surgery devices 2 corresponding to the respective icons. Such a configuration enables the user to easily recognize the functions of the surgery devices 2 corresponding to the icons.
  • the display control unit 14 may cause an icon to be displayed on the basis of whether or not a surgery device 2 has a transmission function of transmitting predetermined information (such as images) to other surgery devices 2.
  • the display control unit 14 may cause an icon to be displayed on the basis of whether or not a surgery device 2 has a reception function of receiving predetermined information (such as images) from other surgery devices 2.
  • icons corresponding to surgery devices 2 having the transmission function of transmitting images, icons corresponding to surgery devices 2 having the reception function of receiving images, and icons corresponding to surgery devices 2 having both the transmission function and the reception function are displayed in a distinguishable manner.
  • a hatch pattern of the icons V211 to V213 and V311 to V313 indicates that the respective icons correspond to surgery devices 2 having the transmission function of transmitting images.
  • a hatch pattern of the icons V221 and V321 to V323 indicates that the respective icons correspond to surgery devices 2 having both the transmission function of transmitting images and the reception function of receiving images.
  • a hatch pattern of the icons V231 to V233 and V331 to V333 indicates that the respective icons correspond to surgery devices 2 having the reception function of receiving images. Note that, in the example illustrated in FIG. 3, surgery devices 2 corresponding to icons V241 and V242 do not have any of the transmission function and the reception function.
  • the device control unit 16 controls connection between the surgery devices 2 having the transmission function and the surgery devices 2 having the reception function on the basis of user operation. Accordingly, display of the icons that help distinguishing whether or not the respective surgery devices 2 have the transmission function and the reception function, is convenient for the user.
  • the display area V400 is placed above the display area V200.
  • the display area V400 displays a tab V401 and a tab V402.
  • the display control unit 14 may switch display related to a screen displayed in the display area V200 in accordance with user operation performed on the tab V401 or the tab V402. Note that, in the example illustrated in FIG. 3, the tab V401 is selected.
  • the display control unit 14 may cause a plurality of layout screens to be displayed.
  • the display control unit 14 may switch display of a layout screen to be displayed in the display area V200 in accordance with user operation performed on the tab V401 or the tab V402.
  • a layout screen displayed in the display area V200 when the tab V401 is selected shows arrangement of surgery devices 2 differently from a layout screen displayed in the display area V200 when the tab V402 is selected.
  • a specific example of such display control will be described later with reference to FIG. 24 to FIG. 25.
  • the device control unit 16 generates control signals for carrying out control regarding the surgery devices 2. For example, the device control unit 16 may generate the control signals on the basis of user operation performed on various kinds of screens caused to be displayed by the display control unit 14.
  • the device control unit 16 may generate the control signals on the basis of user operation performed on the layout screen. For example, the device control unit 16 may generate a control signal for carrying out control regarding a surgery device 2 corresponding to an icon displayed on the layout screen in the case where user operation is performed on the icon.
  • the layout screen shows arrangement of surgery devices 2. Therefore, the user is capable of easily recognizing correspondence between the surgery devices 2 and respective icons. Therefore, such a configuration enables a user to easily recognize an icon corresponding to a surgery device 2 that the user wants to control, and intuitively operate it.
  • the device control unit 16 may generate a control signal on the basis of user operation performed on a plurality of icons caused to be displayed by the display control unit 14.
  • the plurality of icons all of the icons may be included in the layout screen or the device list screen.
  • the icons may be a combination of icons included in the layout screen and icons included in the device list screen.
  • the device control unit 16 may generate a control signal for controlling connection between the plurality of surgery devices 2 (hereinafter, also referred to as inter-device connection) on the basis of user operation performed on the plurality of icons.
  • the device control unit 16 may generate a control signal for controlling connection between a surgery device 2 having the function of transmitting images (hereinafter, also referred to as a first device) and a surgery device 2 having the function of receiving images (hereinafter, also referred to as a second device).
  • a control signal for controlling the connection between the first device and the second device may be generated on the basis of user operation performed on an icon corresponding to the first device and an icon corresponding to the second device.
  • a control signal may be transmitted to at least any one of the first device or the second device by using the communication unit 60, for example.
  • control signal for controlling inter-device connection may be generated on the basis of other user operation.
  • the device control unit 16 may generate the control signal for controlling inter-device connection such that the first device transmits an image to the second device on the basis of user operation performed on the preview screen and the layout screen that are caused to be displayed by the display control unit 14.
  • the control signal for controlling inter-device connection such that the first device transmits an image to the second device on the basis of user operation performed on the preview screen including the image to be transmitted from the first device and the icon corresponding to the second device included in the layout screen.
  • the device control unit 16 may generate a control signal regarding settings of a surgery device 2 caused to be displayed by the display control unit 14, on the basis of user operation performed on a setting screen for configuring the settings of the surgery device 2.
  • the generated control signal is transmitted to the surgery device 2 by using the communication unit 60.
  • control signals generated by the device control unit 16 are not limited thereto.
  • a specific example regarding the above-described control signals and a specific example of other control performed by the device control unit 16 will be described later together with specific examples of display control with reference to FIG. 5 to FIG. 35.
  • FIG. 4 is a flowchart illustrating the operation example of the information processing apparatus 1 according to the present embodiment.
  • the display control unit 14 acquires arrangement information from the arrangement information control unit 12 (S102). Next, the display control unit 14 causes the display unit 40 to display a layout screen showing arrangement of surgery devices 2 on the basis of the arrangement information acquired in Step S102 (S104).
  • the display control unit 14 controls the display such that the display is updated on the basis of the user operation (S106), and the device control unit 16 generates a control signal for controlling a surgery device 2 on the basis of the user operation (S108).
  • the communication unit 60 transmits the control signal generated in Step S108 to the surgery device 2 (S110).
  • Step S106 to Step S110 in FIG. 4 may vary in accordance with user operation.
  • the processes in Step S106 and Step S108 may be performed only in the case where user operation is performed.
  • One of the processes or both of the processes may be performed in accordance with user operation.
  • the process in Step S110 may be performed in the case where the process in Step S108 is performed.
  • FIG. 5 to FIG. 8 are explanatory diagrams illustrating the first specific example according to the present embodiment.
  • FIG. 5 illustrates a state in which drag operation (user operation for moving a mouse while pressing a button of the mouse) is performed on the icon V211 included in the layout screen by using a mouse cursor V902 in the state illustrated in FIG. 3.
  • the surgery device 2 corresponding to the icon V211 is a camera having the function of transmitting images.
  • the surgery device 2 corresponding to the icon V211 may be referred to as a first device.
  • a drag icon (also referred to as a ghost) V711 corresponding to the icon V211 is displayed.
  • the drag icon V711 is displayed such that the drag icon V711 follows movement of the mouse cursor V902 while performing the drag operation.
  • FIG. 6 illustrates a state in which the mouse cursor V902 is moved in the state illustrated in FIG. 5 and drag operation is performed such that the drag icon V711 overlaps the icon V232.
  • the surgery device 2 corresponding to the icon V232 is a monitor having the function of receiving images.
  • the surgery device 2 corresponding to the icon V232 may be referred to as a second device.
  • the second device further has a function of dividing the display area and simultaneously displaying a plurality of images. More specifically, the second device is capable of dividing the display area into four areas and simultaneously displaying a maximum of four images.
  • the display control unit 14 causes a display area selection screen V810 to be displayed.
  • the display area selection screen V810 is a pop-up screen for selecting a display position of an image transmitted from the first device to the second device in the second device.
  • the display area selection screen V810 includes display positions in the second device, in other words, divided area V811 to V814 corresponding to divided display areas.
  • FIG. 8 illustrates a state in which selection operation (for example, user operation for pressing a button of the mouse at an appropriate mouse position) is performed on the divided area V814 included in the display area selection screen V810 in the state illustrated in FIG. 7.
  • the device control unit 16 may generate a control signal for controlling inter-device connection such that the first device transmits an image to the second device, on the basis of the above-described series of user operation performed on the icon V211 corresponding to the first device and the icon V232 corresponding to the second device.
  • Such a configuration enables the user to intuitively perform operation for the inter-device connection.
  • the device control unit 16 may generate a control signal for carrying out control regarding a display position in the second device on the basis of the selection operation.
  • the control signal may be a control signal for designating a display position corresponding to the selection operation.
  • the device control unit 16 may generate a control signal for controlling the inter-device connection such that the first device transmits an image to the second device at a time when the drop operation is performed in the state illustrated in FIG. 6, for example. ⁇ 4-2.
  • Second specific example> an example in which user operation is performed for carrying out control regarding settings of a surgery device 2 will be described as a second specific example with reference to FIG. 9 to FIG. 13.
  • FIG. 9 to FIG. 13 are explanatory diagrams illustrating the second specific example according to the present embodiment.
  • FIG. 9 illustrates a state in which drag operation is performed on the icon V241 included in the layout screen by using the mouse cursor V902 in the state illustrated in FIG. 3, and a drag icon V721 corresponding to the icon V241 is displayed.
  • FIG. 10 illustrates a state in which the mouse cursor V902 is moved in the state illustrated in FIG. 9 and drag operation is performed such that the drag icon V721 enters the display area V100.
  • the state transitions to a state illustrated in FIG. 11.
  • the display control unit 14 causes a setting screen to be displayed in the display region V100.
  • the setting screen is a setting screen of the surgery device 2 corresponding to the icon V241.
  • the icon V110 illustrated in FIG. 11 has appearance similar to an actual operation unit of the surgery device 2 corresponding to the icon V241.
  • the device control unit 16 may generate a control signal for carrying out control regarding a setting of the surgery device 2 corresponding to the icon V241 in accordance with the selection operation.
  • Such a configuration enables the user to perform operation regarding settings by using the setting screen as if the user has directly touched the actual surgery device 2 and performed the operation regarding settings.
  • the setting screen displayed in the display area V100 includes setting interfaces V120, V130, and V140 for configuring settings of the surgery device 2 corresponding to the icon V241.
  • the setting interfaces V120, V130, and V140 respectively include sliders V121, V131, and V141.
  • the user is capable of using the mouse cursor V902 to perform drag operation on the slider V121.
  • the device control unit 16 may generate a control signal for carrying out control regarding settings associated with the setting interface V120.
  • the setting screen may be displayed in the display area V200 as illustrated in FIG. 13.
  • the state may transition to a state illustrated in FIG. 13 in the case where predetermined user operation is performed on the icon V241 included in the layout screen in the state illustrated in FIG. 3.
  • the predetermined user operation is different from user operation for causing the setting screen to be displayed in the display area V100.
  • the predetermined operation may be double click, right click, or the like.
  • the predetermined operation may be long press on the touchscreen.
  • the icon V250 illustrated in FIG. 13 has appearance similar to the actual operation unit of the surgery device 2 corresponding to the icon V241.
  • the device control unit 16 may generate a control signal for carrying out control regarding a setting of the surgery device 2 corresponding to the icon V241 in accordance with the selection operation.
  • Such a configuration enables the user to perform operation regarding the settings of the surgery device 2 by using a larger setting screen.
  • FIG. 14 to FIG. 23 are explanatory diagrams illustrating the third specific example according to the present embodiment.
  • FIG. 14 illustrates a state in which drag operation is performed on the icon V312 included in the device list screen by using the mouse cursor V902 in the state illustrated in FIG. 3, and a drag icon V731 corresponding to the icon V312 is displayed.
  • the surgery device 2 corresponding to the icon V312 has the function of transmitting images.
  • the surgery device 2 corresponding to the icon V312 may be referred to as a first device.
  • the first device is a recorder that stores images captured before surgery.
  • FIG. 15 illustrates a state in which the mouse cursor V902 is moved in the state illustrated in FIG. 14 and drag operation is performed such that the drag icon V731 enters the display area V100.
  • the state transitions to a state illustrated in FIG. 16.
  • the display control unit 14 causes a preview screen to be displayed in the display area V100.
  • the preview screen includes images transmitted from the first device.
  • the preview screen displayed in the display area V100 includes a preview display area V150, a playback position designation interface V160 for designating a playback position, and an image selection screen V170.
  • the playback position designation interface V160 includes a slider V161.
  • the image selection screen V170 includes icons V171 to V176 corresponding to images stored in the first device.
  • the display control unit 14 when selection operation is performed on the icon V172 by using the mouse cursor V902, the display control unit 14 causes an image corresponding to the icon V172 to be displayed in the preview display area V150, and the state transitions to a state illustrated in FIG. 18.
  • FIG. 19 illustrates a state in which drag operation is performed on the slider V161 of the reproduction position designation interface V160 by using the mouse cursor V902 in the state illustrated in FIG. 18.
  • the display control unit 14 causes a frame corresponding to the position of the slider V161 to be displayed in the preview display area V160 among images corresponding to the icon V172 as illustrated in FIG. 19.
  • Such a configuration enables the user to see the image transmitted from the first device in the preview screen, and enables the user to designate a frame in the case where the image is a moving image.
  • FIG. 20 illustrates a state in which drag operation is performed on the preview display area V150 by using the mouse cursor V902 in the state illustrated in FIG. 19, and a drag icon V732 corresponding to the image displayed in the preview display area V150 is displayed.
  • FIG. 21 illustrates a state in which the mouse cursor V902 is moved in the state illustrated in FIG. 19 and drag operation is performed such that the drag icon V732 overlaps the icon V232.
  • the surgery device 2 corresponding to the icon V232 is a monitor having the function of receiving images.
  • the surgery device 2 corresponding to the icon V232 may be referred to as a second device.
  • the second device is capable of dividing the display area into four areas and simultaneously displaying a maximum of four images also in this specific example.
  • the display control unit 14 causes a display area selection screen V820 to be displayed.
  • the display area selection screen V820 is a pop-up screen for selecting a display position of an image transmitted from the first device to the second device in the second device.
  • the display area selection screen V820 includes display positions in the second device, in other words, divided areas V821 to V824 corresponding to divided display areas.
  • FIG. 23 illustrates a state in which selection operation is performed on the divided area V824 included in the display area selection screen V820 in the state illustrated in FIG. 22.
  • the device control unit 16 may generate a control signal for controlling inter-device connection such that the first device transmits an image to the second device, on the basis of the above-described series of user operation performed on the icon V312 corresponding to the first device, the preview display area V150 included in the preview screen, and the icon V232 corresponding to the second device.
  • Such a configuration enables performing intuitive operation for inter-device connection, seeing images displayed in advance on the second device, and designating a frame.
  • the device control unit 16 may generate a control signal for carrying out control regarding a display position in the second device on the basis of the selection operation.
  • the device control unit 16 may generate a control signal for controlling the inter-device connection such that the first device transmits an image to the second device at a time when the drop operation is performed in the state illustrated in FIG. 21.
  • FIG. 24 to FIG. 27 are explanatory diagrams illustrating the fourth specific example according to the present embodiment.
  • FIG. 24 illustrates a state in which the mouse cursor V902 is moved to the position of the tab V402 in the state illustrated in FIG. 3.
  • the display control unit 14 switches the display of the layout screen displayed in the display area V200, and the state transitions to a state illustrated in FIG. 25.
  • the layout screen displayed in the display area V200 shows arrangement of surgery devices 2 that are present in the surgery room but not shown in the layout screen displayed in the display area V200 in the state illustrated in FIG. 24, for example.
  • FIG. 25 illustrates icons V243 to V247 corresponding to the surgery devices 2 that are not illustrated in FIG. 24.
  • the layout screen displayed in FIG. 25 may be a layout screen captured from a viewpoint different from the layout screen illustrated in FIG. 24.
  • the surgery devices 2 corresponding to the icons V243 to V247 displayed in FIG. 25 may be surgery devices 2 installed at heights different from the surgery devices 2 corresponding to the icons included in the layout screen displayed in the display area V200 in the state illustrated in FIG. 24.
  • the surgery devices 2 corresponding to the icons V243 to V247 may be different types of surgery devices 2 from the surgery devices 2 corresponding to the icons included in the layout screen displayed in the display area V200 in the state illustrated in FIG. 24.
  • FIG. 25 illustrates the example of displaying the layout screen showing arrangement of surgery devices 2 present in the surgery room
  • the present technology is not limited thereto. For example, by switching the display of the layout screen in accordance with the tab selection operation, a layout screen showing arrangement of surgery devices 2 present outside of the surgery room may be displayed.
  • the display control unit 14 may switch the display of the layout screen displayed in the display area V200, and the state may transition to a state illustrated in FIG. 27.
  • a device list screen is displayed in the display area V200.
  • the device list screen includes icons V249, V261, and V262 corresponding to surgery devices 2.
  • the surgery devices 2 corresponding to the icons V249, V261, and V262 are present in a surgery room, but it is difficult to display arrangement of the surgery devices 2 corresponding to the icons V249, V261, and V262.
  • Such surgery devices 2 that are present in the surgery room but whose arrangement is difficult to be displayed may be surgery devices 2 present in a wide range, surgery devices 2 that are moved frequently during surgery, or the like.
  • Such a configuration enables performing operation for carrying out control also regarding surgery devices 2 that are present in the surgery room but whose arrangement is difficult to be displayed.
  • the respective layout screens may show arrangement of different surgery devices 2 among a plurality of surgery devices 2 included in the surgery system 1000.
  • the surgery devices 2 shown in the respective layout screens may be classified on the basis of a predetermined standard.
  • FIG. 28 is an explanatory diagram illustrating the fifth specific example according to the present embodiment.
  • FIG. 28 is an example in which the layout screen displayed on the display area V200 includes icons that are simpler than the above-described examples. Note that, in the example illustrated in FIG. 28, icons corresponding to the same surgery devices 2 as the above-described surgery devices 2 are denoted with the same reference signs as the above-described icons in FIG. 3 or the like.
  • the layout screen displayed on the display area V200 shows positions of the surgery devices 2, but does not show attitudes of the surgery devices 2.
  • the icons included in the layout screen displayed in the display area V200 may be displayed not with attitudes corresponding to actual attitudes of the surgery devices 2 but with predetermined attitudes, for example.
  • the layout screen illustrated in FIG. 28 is useful in the case where positions of the surgery devices 2 are not changed during surgery but attitudes of the surgery devices 2 are often changed during surgery.
  • the attitudes are not shown, it is easier to recognize types of the surgery devices 2 corresponding to the respective icons.
  • the icons V231 to V233 corresponding to monitors have functions of showing how the display areas of the monitors corresponding to the respective icons are divided (how many images can be simultaneously displayed at maximum).
  • numerical values representing sizes of the monitors corresponding to the respective icons are displayed near the icons V231 to V233. Such a configuration enables the user to easily recognize information regarding the surgery devices 2.
  • the display control unit 14 may cause an icon corresponding to a surgery device 2 to be displayed on the basis of whether or not the surgery device 2 is in a state capable of transmitting currently capturing images.
  • icons included in the device list screen displayed in the display area V300 are displayed such that states of the icons are distinguishable on the basis of shapes of the icons.
  • the icon V311 indicates that the surgery device 2 corresponding to the icon is in the state capable of transmitting currently capturing images.
  • Such a configuration enables the user to easily recognize whether or not a surgery device 2 corresponding to an icon is in the state capable of transmitting currently capturing images.
  • the display control unit 14 may cause an icon corresponding to a surgery device 2 to be displayed on the basis of whether or not the surgery device 2 is in a state capable of transmitting images that are stored in advance.
  • FIG. 29 to FIG. 35 are explanatory diagrams illustrating the sixth specific example according to the present embodiment.
  • the layout screen showing arrangement of surgery devices 2 present in the surgery room is displayed in the display area V200, and the layout screen showing arrangement of surgery devices 2 present outside of the surgery room is displayed in a display area V500.
  • the display area V500 includes four display areas V510, V520, V530, and V540.
  • a layout screen including icons corresponding to surgery devices 2 present in a different room is displayed in each of the four display areas.
  • the layout screen displayed in the display area V510 includes icons V511 and V512 corresponding to monitors.
  • the layout screen displayed in the display area V520 includes an icon V521 corresponding to a monitor.
  • the layout screen displayed in the display area V530 includes an icon V531 corresponding to a monitor and an icon V532 corresponding to a camera.
  • the layout screen displayed in the display area V540 includes an icon V541 corresponding to a recorder.
  • FIG. 29 simplified icons are used in a way similar to the example illustrated in FIG. 28.
  • icons corresponding to the same surgery devices 2 as the above-described surgery devices 2 are denoted with the same reference signs as the above-described icons in FIG. 28.
  • the display control unit 14 causes three preview screens V610, V620, and V630 to be displayed.
  • the preview screens V610, V620, and V630 respectively include device name display areas V611, V621, and V631 and preview display areas V612, V622, and V632.
  • FIG. 30 illustrates a state in which drag operation is performed on the icon V532 included in a layout screen displayed in the display area V530 by using the mouse cursor V902 in the state illustrated in FIG. 29, and a drag icon V771 corresponding to the icon V532 is displayed.
  • the surgery device 2 corresponding to the icon V532 is a camera having the function of transmitting images.
  • the surgery device 2 corresponding to the icon V532 may be referred to as a first device.
  • FIG. 31 illustrates a state in which the mouse cursor V902 is moved in the state illustrated in FIG. 30 and drag operation is performed such that the drag icon V771 enters the preview screen V610.
  • the state transitions to a state illustrated in FIG. 32.
  • the display control unit 14 causes an image transmitted from the first device to be displayed in the preview display area V612 in the preview screen V610.
  • the display control unit 14 may cause the device name of the first device to be displayed in the device name display area V611 in the preview screen V610.
  • FIG. 33 illustrates a state in which drag operation is performed on the preview display area V612 by using the mouse cursor V902 in the state illustrated in FIG. 32, and a drag icon V772 corresponding to the image displayed in the preview display area V612 is displayed.
  • FIG. 34 illustrates a state in which the mouse cursor V902 is moved in the state illustrated in FIG. 33 and drag operation is performed such that the drag icon V772 overlaps the icon V231.
  • the surgery device 2 corresponding to the icon V231 is a monitor having the function of receiving images.
  • the surgery device 2 corresponding to the icon V231 may be referred to as a second device.
  • the display area of the second device is not divided.
  • the second device is a monitor that displays an image in the entire display area.
  • the device control unit 16 may generate a control signal for controlling the inter-device connection such that the first device transmits an image to the second device when the drop operation is performed in the state illustrated in FIG. 34.
  • FIG. 35 illustrates a state in which the mouse cursor V902 is moved in the state illustrated in FIG. 30 and drag operation is performed such that the drag icon V771 overlaps the icon V231.
  • the device control unit 16 may generate a control signal for controlling the inter-device connection such that the first device transmits an image to the second device when the drop operation is performed in the state illustrated in FIG. 35.
  • FIG. 36 is a block diagram illustrating an example of a hardware configuration of the information processing apparatus 1 according to the embodiment of the present disclosure.
  • the information process performed by the information processing apparatus 1 according to the embodiment of the present disclosure is achieved by operating cooperatively software and hardware.
  • the information processing apparatus 1 includes a central processing unit (CPU) 901, read only memory (ROM) 902, random access memory (RAM) 903, and a host bus 904a.
  • the information processing apparatus 1 includes a bridge 904, an external bus 904b, an interface 905, an input apparatus 906, an output apparatus 907, a storage apparatus 908, a drive 909, a connection port 911, a communication apparatus 913, and a sensor 915.
  • the information processing apparatus 1 may include a processing circuit such as a DSP or an ASIC instead of or in addition to the CPU 901.
  • the CPU 901 functions as an arithmetic processing apparatus and a control apparatus to control entire operation in the information processing apparatus 1 in accordance with various kinds of programs.
  • the CPU 901 may be a microprocessor.
  • the ROM 902 stores programs, arithmetic parameters, and the like used by the CPU 901.
  • the RAM 903 transiently stores programs used when the CPU 901 is executed, various parameters that change as appropriate when executing such programs, and the like.
  • the CPU 901 may be configured as the control unit 10, for example.
  • the CPU 901, the ROM 902, and the RAM 903 are connected to each other through the host bus 904a including a CPU bus and the like.
  • the host bus 904a is connected, via the bridge 904, to the external bus 904b such as a peripheral component interconnect/interface (PCI) bus.
  • PCI peripheral component interconnect/interface
  • the host bus 904a, the bridge 904, and the external bus 904b are not necessarily configured as a separate component. Their functions may be incorporated into in a single bus.
  • the input apparatus 906 is implemented as an apparatus allowing the user to input information, such as a mouse, a keyboard, a touchscreen, a button, a microphone, a switch, and a lever.
  • the input apparatus 906 may be a remote controller using infrared ray or other electric waves, or may be an external connection device such as a cellular phone or a PDA that correspond to operation performed on the information processing apparatus 1, for example.
  • the input apparatus 906 may include an input control circuit or the like that is configured to generate an input signal on the basis of information input by the user using the aforementioned input mechanism and output the generated input signal to the CPU 901.
  • the user of the information processing apparatus 1 is capable of inputting various types of data to the information processing apparatus 1, or instructing the information processing apparatus 1 to perform process operation, by operating the input apparatus 906.
  • the input apparatus 906 may be configured as the operation unit 30, for example.
  • the output apparatus 907 is configured as an apparatus capable of issuing a visual or auditory notification of the acquired information to the user.
  • Examples of such an apparatus include a display apparatus such as a CRT display apparatus, a liquid crystal display apparatus, a plasma display apparatus, an EL display apparatus, or a lamp, a sound output apparatus such as a speaker or a headphone, a printer apparatus, and the like.
  • the output apparatus 907 outputs, for example, results acquired by various processes performed by the information processing apparatus 1.
  • the display apparatus visually displays results acquired by various processes performed by the information processing apparatus 1 in various formats such as text, images, tables, and graphs.
  • the sound output apparatus converts audio signals including reproduced sound data, audio data, and the like into analog signals and audibly outputs the analog signals.
  • the output apparatus 907 may be configured as the display unit 40, for example.
  • the storage apparatus 908 is a data storage apparatus configured as an example of the storage unit of the information processing apparatus 1.
  • the storage apparatus 908 is implemented as a magnetic storage device such as an HDD, a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like.
  • the storage apparatus 908 may include a storage medium, a recording apparatus for recording data on the storage medium, a reading apparatus for reading data from the storage medium, a deletion apparatus for deleting data recorded on the storage medium, and the like.
  • the storage apparatus 908 stores programs and various types of data executed by the CPU 901, various types of data acquired from the outside, and the like.
  • the storage apparatus 908 may be configured as the storage unit 50, for example.
  • the drive 909 is a reader/writer for a storage medium, and is incorporated in or externally attached to the information processing apparatus 1.
  • the drive 909 reads information recorded on a removable recording medium that is mounted such as a magnetic disk, an optical disc, a magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 903.
  • the drive 909 is also capable of writing information to the removable storage medium.
  • the communication port 911 is an interface for connection to an external device, and is, for example, a connection port for connection to an external device capable of transmitting data via a Universal Serial Bus (USB).
  • USB Universal Serial Bus
  • the communication apparatus 913 is, for example, a communication interface configured as a communication device or the like for connection with a network 920.
  • the communication apparatus 913 is, for example, a communication card or the like for a wired or wireless local area network (LAN), long term evolution (LTE), Bluetooth (registered trademark), or wireless USB (WUSB).
  • the communication apparatus 913 may be a router for optical communication, a router for asymmetric digital subscriber line (ADSL), a modem for various communication, or the like.
  • the communication apparatus 913 is capable of transmitting and receiving signals and the like to and from the Internet or other communication devices, for example, in accordance with a predetermined protocol such as TCP/IP or the like.
  • the communication apparatus 913 may be configured as the communication unit 60, for example.
  • the sensor 915 is various sensors such as an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, a sound sensor, a ranging sensor, or a force sensor, for example.
  • the sensor 915 acquires information regarding the state of the information processing apparatus 1 itself such as the attitude and moving speed of the information processing apparatus 1, and acquires information regarding an environment around the information processing apparatus 1 such as brightness and noise around the information processing apparatus 1.
  • the sensor 915 may include a GPS sensor for receiving a GPS signal and measuring the latitude, longitude, and altitude of the apparatus.
  • the network 920 is a wired or wireless communication path through which information is transmitted from apparatuses connected to the network 920.
  • the network 920 may include a public network such as the Internet, a telephone network, and a satellite communication network, various local area networks (LANs) including Ethernet (registered trademark), a wide area network (WAN), and the like.
  • LANs local area networks
  • WAN wide area network
  • the network 920 may include a dedicated network such as an internet protocol-virtual private network (IP-VPN).
  • IP-VPN internet protocol-virtual private network
  • a computer program for implementing each of the functions of the information processing apparatus 1 according to the embodiment of the present disclosure may be created, and may be mounted in a PC or the like.
  • a computer-readable recording medium on which such computer programs are stored may be provided.
  • the recording medium is, for example, a magnetic disc, an optical disc, a magneto-optical disc, a flash memory, or the like.
  • the computer program may be distributed, for example, through a network without using the recording medium.
  • a control signal for controlling inter-device connection between the first device and the second device is transmitted to the first device or the second device.
  • the present technology is not limited thereto.
  • such a control signal may be transmitted to a surgery device 2 (such as a network device) that relays connection between the first device and the second device.
  • the information processing apparatus 1 may have a function of relaying connection between the first device and the second device.
  • the device control unit 16 may transmit such a control signal to a block having such a function.
  • a surgery system including: an information processing apparatus; and a surgery device that is connected to the information processing apparatus, in which the information processing apparatus includes a display control unit configured to cause a layout screen to be displayed, the layout screen showing arrangement of the surgery device, and a device control unit configured to generate a control signal for carrying out control regarding the surgery device.
  • the device control unit generates the control signal on a basis of user operation performed on the layout screen.
  • the surgery system according to (1) or (2) including a plurality of the surgery devices, in which the information processing apparatus is connected to the plurality of surgery devices, and the layout screen shows arrangement of at least one of the plurality of surgery devices.
  • the device control unit generates the control signal for controlling connection between the first device and the second device on a basis of user operation.
  • the first device has a function of transmitting an image
  • the second device has a function of receiving the image
  • the device control unit generates the control signal for controlling the connection such that the first device transmits the image to the second device.
  • the display control unit further causes a preview screen to be displayed on a basis of user operation, the preview screen including the image transmitted from the first device.
  • the device control unit generates the control signal for controlling the connection such that the first device transmits the image to the second device on a basis of user operation performed on the preview screen and the layout screen.
  • the display control unit further causes a display area selection screen to be displayed for selecting a display position of the image transmitted from the first device to the second device, in the second device, and the device control unit generates the control signal for carrying out control regarding the display position in the second device on a basis of user operation performed on the display area selection screen.
  • the surgery system according to any one of (3) to (8) in which the display control unit causes a plurality of the layout screens to be displayed, each of the plurality of layout screens showing arrangement of a different surgery device among the plurality of surgery devices.
  • the plurality of surgery devices include at least one of an endoscope, a microscope, a surgical field camera, or a ceiling camera.
  • the surgery system according to any one of (1) to (12), in which the display control unit further causes a setting screen to be displayed for configuring a setting of the surgery device on a basis of user operation performed on the layout screen, and the device control unit generates the control signal regarding the setting of the surgery device on a basis of user operation performed on the setting screen.
  • the display control unit causes an icon corresponding to the surgery device to be displayed.
  • the display control unit causes the icon corresponding to the surgery device to be displayed at a position in the layout screen in accordance with a position of the surgery device in a real space.
  • the surgery system according to any one of (17) or (18), in which the display control unit causes the icon corresponding to the surgery device to be displayed on a basis of whether or not the surgery device is in a state capable of transmitting an image that is being captured.
  • the display control unit causes the icon corresponding to the surgery device to be displayed on a basis of whether or not the surgery device is in a state capable of transmitting an image that is being captured.
  • An information processing apparatus that is connected to a surgery device, the information processing apparatus including: a display control unit configured to cause a layout screen to be displayed, the layout screen showing arrangement of the surgery device; and a device control unit configured to generate a control signal for carrying out control regarding the surgery device.
  • An information processing method including: causing a layout screen to be displayed, the layout screen showing arrangement of a surgery device; and generating a control signal for carrying out control regarding the surgery device.
  • a medical operation system comprising: an information processing apparatus, including processing circuitry configured to cause a map of an operating room to be displayed on a display, the map including an icon representing a device located in the operating room or accessible from the operating room, receive, via a user operation on the displayed map, designation information representing a designation of a change in at least one of an input source, an output destination, and an internal setting for the device, generate a control signal to control the device based on the designation information, and cause the generated control signal to be transmitted to the device.
  • an information processing apparatus including processing circuitry configured to cause a map of an operating room to be displayed on a display, the map including an icon representing a device located in the operating room or accessible from the operating room, receive, via a user operation on the displayed map, designation information representing a designation of a change in at least one of an input source, an output destination, and an internal setting for the device, generate a control signal to control the device based on the designation information, and cause the generated control signal to be transmitted to the device.
  • the designation information is
  • the processing circuitry is further configured to generate the control signal, which controls a connection between the first device and the second device, when the designation information includes information indicating selection of a first icon representing the first device and a second icon representing the second device.
  • the medical operation system according to (5) wherein the first device is configured to transmit an image, the second device is configured to receive a image, and the processing circuitry is further configured to generate the control signal to control the connections so that the first device transmits the image to the second device.
  • the processing circuitry is further configured to generate the control signal to cause the image to be transmitted from the first device to the second device, based on the designation information generated via the user operation on the map and the preview screen.
  • the medical operation system according to (5) wherein the second device is further configured to simultaneously display a plurality of images, the processing circuitry is further configured to cause a display area selection screen to be displayed, the display area selection screen allowing for selection of a display position, in the second device, of the image transmitted from the first device to the second device, and the processing circuitry is further configured to generate the control signal to perform control regarding the display position in the second device, based on user input performed on the display area selection screen.
  • the processing circuitry is further configured to cause a plurality of maps to be displayed, each of the plurality of maps showing an arrangement of a different surgery device among a plurality of surgery devices.
  • the processing circuitry is further configured to switch display of the plurality of maps, based on a user input operation.
  • the device includes a first surgery device located in a surgery room and a second surgery device located outside of the surgery room.
  • the device is one of an endoscope, a microscope, a surgical field camera, and a ceiling camera.
  • a method of controlling a device located in a medical operating room and accessible during a medical procedure in the medical operating room comprising: displaying a map of the medical operating room, the map including an icon representing the device; receiving, via a user operation on the displayed map, a designation of a change in one of an input source, an output destination, and an internal setting for the device; generating a control signal to control the designated change, based on the received user operation; and causing the generated control signal to be transmitted to the device.
  • An image processing apparatus comprising: processing circuitry configured to display a map of an operating room, the map including an icon representing a device located in the operating room or useable during a medical procedure in the medical operating room, receive, via a user operation on the displayed map, a designation of a change in one of an input source, an output destination, and an internal setting for the device, and generate a control signal to control the designated change, based on the received user operation; and a transmitter configured to transmit, to the device, the generated control signal.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Veterinary Medicine (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Radiology & Medical Imaging (AREA)
  • Pathology (AREA)
  • Optics & Photonics (AREA)
  • Biophysics (AREA)
  • Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Robotics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Endoscopes (AREA)
EP18816268.9A 2017-11-17 2018-11-08 Surgery system, information processing apparatus, and information processing method Pending EP3701540A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017221675A JP7155511B2 (ja) 2017-11-17 2017-11-17 手術システム、情報処理装置、及び情報処理方法
PCT/JP2018/041508 WO2019098120A1 (en) 2017-11-17 2018-11-08 Surgery system, information processing apparatus, and information processing method

Publications (1)

Publication Number Publication Date
EP3701540A1 true EP3701540A1 (en) 2020-09-02

Family

ID=64664354

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18816268.9A Pending EP3701540A1 (en) 2017-11-17 2018-11-08 Surgery system, information processing apparatus, and information processing method

Country Status (5)

Country Link
US (1) US11587670B2 (zh)
EP (1) EP3701540A1 (zh)
JP (1) JP7155511B2 (zh)
CN (1) CN111344802B (zh)
WO (1) WO2019098120A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4120689A1 (en) * 2021-07-12 2023-01-18 Rohde & Schwarz GmbH & Co. KG System and method for configuring the graphical layout of at least one multi-viewer display
WO2022213604A1 (zh) * 2021-11-11 2022-10-13 北京山桐科技有限公司 数字化手术室的数据处理方法及系统
CN114268766B (zh) * 2021-11-11 2022-11-15 北京山桐科技有限公司 数字化手术室的数据处理方法及系统

Family Cites Families (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09289606A (ja) 1996-04-23 1997-11-04 Canon Inc 画像表示装置およびカメラ制御装置
JP2001112775A (ja) * 1999-10-21 2001-04-24 Olympus Optical Co Ltd 手術器具準備システム及びコンピュータ読み取り可能な記憶媒体
JP2002306509A (ja) * 2001-04-10 2002-10-22 Olympus Optical Co Ltd 遠隔手術支援システム
US20030093503A1 (en) * 2001-09-05 2003-05-15 Olympus Optical Co., Ltd. System for controling medical instruments
JP2003190181A (ja) 2001-12-26 2003-07-08 Olympus Optical Co Ltd 制御装置
US20040044295A1 (en) * 2002-08-19 2004-03-04 Orthosoft Inc. Graphical user interface for computer-assisted surgery
JP2006000536A (ja) * 2004-06-21 2006-01-05 Olympus Corp 手術室制御システム
US20060004605A1 (en) 2004-06-21 2006-01-05 Epic Systems Corporation System and method for a comprehensive interactive graphical representation of a health care facility for managing patient care and health care facility resources
US20070268280A1 (en) * 2004-08-23 2007-11-22 Manabu Fujita Image Display Apparatus, Image Display Method, and Image Display Program
US7366934B1 (en) * 2004-09-08 2008-04-29 Stryker Corporation Method of remotely controlling devices for endoscopy
JP4152402B2 (ja) * 2005-06-29 2008-09-17 株式会社日立メディコ 手術支援装置
US20090282350A1 (en) * 2005-09-30 2009-11-12 Panasonic Corporation Display device, external connection device, and screen control method
JP5922491B2 (ja) * 2012-05-15 2016-05-24 ジーイー・メディカル・システムズ・グローバル・テクノロジー・カンパニー・エルエルシー 手術支援装置
US9757507B2 (en) * 2012-11-13 2017-09-12 Karl Storz Imaging, Inc. Configurable control for operating room system
DE102012220672A1 (de) * 2012-11-13 2014-05-15 Trumpf Medizin Systeme Gmbh + Co. Kg Medizinisches Steuerungssystem
US9319483B2 (en) * 2012-11-13 2016-04-19 Karl Storz Imaging, Inc. Asynchronous open task for operating room control system
WO2014128782A1 (ja) * 2013-02-20 2014-08-28 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 携帯情報端末の制御方法及びプログラム
EP2996615B1 (en) * 2013-03-13 2019-01-30 Stryker Corporation System for arranging objects in an operating room in preparation for surgical procedures
WO2015040732A1 (ja) * 2013-09-20 2015-03-26 株式会社日立国際電気 映像監視システム及び映像表示方法
US9999116B2 (en) * 2014-01-10 2018-06-12 Philips Lighting Holding B.V. Tablet-based commissioning tool for addressable lighting
SG11201500538YA (en) * 2014-01-29 2015-09-29 Panasonic Ip Corp America Method for controlling information apparatus and program
US9712948B2 (en) * 2014-04-30 2017-07-18 Avago Technologies General Ip (Singapore) Pte. Ltd. Image triggered pairing
US10854168B2 (en) * 2015-03-30 2020-12-01 Sony Corporation Information processing apparatus, information processing method, and information processing system
WO2017094363A1 (ja) * 2015-11-30 2017-06-08 オリンパス株式会社 集中制御装置
US11314846B1 (en) * 2016-09-29 2022-04-26 OR Link, Inc. Surgical communication and computerization infrastructure

Also Published As

Publication number Publication date
CN111344802B (zh) 2024-04-23
JP7155511B2 (ja) 2022-10-19
US20200279644A1 (en) 2020-09-03
WO2019098120A1 (en) 2019-05-23
CN111344802A (zh) 2020-06-26
US11587670B2 (en) 2023-02-21
JP2019088734A (ja) 2019-06-13

Similar Documents

Publication Publication Date Title
JP2022031339A (ja) 表示方法およびデバイス
US10318093B2 (en) Method and apparatus for message processing
EP2832107B1 (en) Information processing apparatus, information processing method, and program
US9223416B2 (en) Display apparatus, remote controlling apparatus and control method thereof
WO2019098120A1 (en) Surgery system, information processing apparatus, and information processing method
WO2016048024A1 (en) Display apparatus and displaying method thereof
US20150020014A1 (en) Information processing apparatus, information processing method, and program
WO2015043485A1 (zh) 一种显示方法及显示设备
US20130342456A1 (en) Remote control apparatus and control method thereof
US11176747B2 (en) Information processing apparatus and information processing method
US11082670B2 (en) Information processing apparatus, information processing method, and program
EP3190498A1 (en) Information processing device, information processing method, and program
JPWO2017030193A1 (ja) 情報処理装置、情報処理方法、及びプログラム
CN107172471A (zh) 实现在显示设备上运行触摸屏应用的装置和显示设备
US20140380161A1 (en) Information processing apparatus, information processing method, and program
US9548894B2 (en) Proximity based cross-screen experience App framework for use between an industrial automation console server and smart mobile devices
WO2021042622A1 (zh) 思维导图展示方法、装置、存储介质及电子设备
CN106462251A (zh) 显示控制设备、显示控制方法以及程序
WO2016067745A1 (ja) 電子機器およびフィードバック提供方法
JP6977710B2 (ja) 情報処理装置、情報処理方法、およびプログラム
JP2017054251A (ja) 情報処理装置、情報処理方法、およびプログラム
JP2013250819A (ja) 電子機器、画像データの表示制御方法、およびプログラム
JP2017138698A (ja) 情報処理装置、情報処理方法およびプログラム
JP2015111371A (ja) 情報処理装置、情報処理方法およびプログラム
US10373358B2 (en) Edge user interface for augmenting camera viewfinder with information

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: UNKNOWN

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200504

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20210303

RAP3 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SONY GROUP CORPORATION