JP6002273B2 - Medical support system and method - Google Patents

Description

  The present invention relates to a medical support system and method. Specifically, the present invention relates to a system and a method for sharing an image bi-directionally using a network and enabling a conference.

  Thanks to the development of communication technology, it is possible to hold a video conference via a network such as the Internet even if they are far away from each other. For example, when a conference is performed at a medical site, it is common to perform only a voice or display a voice and video (video) (video conference). In addition, it is common practice to transmit and display medical images via a network. However, when discussing the details of image findings during a conference, it is difficult for the other party to understand a specific part without an image, no matter how much anatomical terms are used. In addition, when a medical image is transmitted / received, since it is generally performed with compressed data, the image may be deteriorated. Furthermore, in a remote place such as a foreign country, it may be necessary to exchange with a place where the bandwidth of the network line is different. In such a case, a system that enables a conference is also required. For example, Patent Document 1 discloses a technique in which a doctor inside a medical facility and a doctor outside the medical facility negotiate in real time using medical images.

JP 2008-217294 A

  However, in Patent Document 1, although it is a real-time discussion using a medical image, there is no means for indicating a specific location in the image, and the recognition of the specific location is not correctly shared with each other during the discussion. there is a possibility. Further, the operation information is only in one direction, and it is not disclosed that each other's operation information is shared. When conducting a real-time conference using medical images, it is difficult to recognize a specific part by exchanging only words, unless a specific part of one of the images is accurately shown. There is a risk that it will fall. For this reason, it is desirable to share a specific location recognized by each other in both directions and share other operation information with each other. Also, when conferences are held at remote locations, if the bandwidth of the network line is narrow, medical images are compressed and transmitted and received, which may cause differences in the quality of images shared with each other due to image degradation and the like. There is.

  In view of such a situation, the present invention shares medical images through a network even if they are remote from each other in any line situation, and recognizes a specific location accurately while interactively. The purpose is to provide a medical support system that can communicate with each other.

In order to solve the above problems, the present invention enables one server bidirectionally connected to one terminal and the other server bidirectionally connected to the other terminal to communicate with each other via a network. The medical support system, wherein the one server and the other server conduct a conference between the one terminal and the other terminal by mutually transmitting video and audio, or only audio to each other. and conference processing unit that executes, bidirectionally between the one server and the other server, the image sharing processing to share the medical image which is not different from the process and the image stored in the database of the medical facility , An image storage unit for temporarily storing the shared image shared by the image sharing processing unit, and operation information sent from the one or the other terminal An image for transmitting to the one or the other terminal display information for the one or the other terminal to display the shared image controlled by the control unit based on the control unit that controls the server based on the control unit A display unit, and the one or other terminal displays the shared image based on the display information sent from the one or other image display unit, and displays the operation information of the one or other server. The control unit receives the operation information for displaying the first pointer on the one terminal from the one terminal, and transmits the operation information to the image display unit. Display information for displaying the first pointer on the one terminal based on the operation information is transmitted to the one terminal, and the operation information is transmitted to the other server. Transmitted to the control unit, and in the other server, the control unit, based on the received operation information, has the other position at the same position as the position where the first pointer is displayed on the image display unit. Display information for displaying a second pointer that cannot be operated on the terminal is transmitted to the other terminal, and the other terminal displays the second pointer based on the display information transmitted from the other terminal. It is characterized by doing.

  According to the present invention, medical images that are remote from each other, can share medical images through the network in any network condition, and can exchange bidirectionally while accurately recognizing specific locations from each other. A support system can be provided.

It is the schematic in the case of the two-point conference which concerns on this invention. It is a block diagram of a server concerning one embodiment of the present invention. It is a flowchart of the conference process which concerns on one Embodiment of this invention. It is a flowchart of an image sharing process in case there exists a shared image. It is a flowchart of an image sharing process when there is no shared image. It is the schematic of the image selection which concerns on one Embodiment of this invention. It is the schematic of 2 screen selection which concerns on one Embodiment of this invention. It is the schematic of the pointer display which concerns on one Embodiment of this invention. It is the schematic in the case of the conference between several which concerns on this invention.

  A telemedicine network system according to an embodiment of the medical support system of the present invention will be described with reference to the drawings. In addition, although the Example shown below is a suitable specific example in the telemedicine network system of this invention, and there may be various limitation preferable with respect to treatment, the technical scope of this invention is especially this invention. As long as there is no description which limits these, it is not limited to these aspects. In addition, the constituent elements in the embodiments shown below can be appropriately replaced with existing constituent elements and the like, and various variations including combinations with other existing constituent elements are possible. Therefore, the description of the embodiment described below does not limit the contents of the invention described in the claims.

  First, referring to FIG. 1, a telemedicine network system 1 includes a server 2 (one server) and 3 (the other server) and two servers 2 and 3 that are connected to each other through a network so as to be capable of bidirectional communication. And terminals 4 (one terminal) and 5 (the other terminal) that are bi-directionally connected so as to be communicable. The servers 2 and 3 share all the processing information related to the conference connection and the image sharing according to the present invention bidirectionally. The terminals 4 and 5 may be general-purpose devices that are connected to a network and include a device capable of displaying medical images, a video device such as a web camera capable of a conference, and an audio device such as a microphone. For example, it may be a portable terminal such as a small PC, a mobile phone, a smartphone, or a stationary device such as a desktop PC. Here, in the present embodiment, the conference is a conference that is performed by a plurality of two or more terminals and transmits each other's video and audio to each other, but is an audio-only conference without video. Also good. The medical image is a 2D (2D) or 3D (3D) image used in a medical field such as an MRI image or a CT image, but is not limited to this, and is generally widely used. For example, an image such as JPEG or GIF may be used. The telemedicine network system 1 is provided by so-called cloud computing. Cloud computing here refers to a form in which computing resources such as servers 2 and 3 are provided to external users through a network, including software, hardware, processing performance, storage areas, files, data, and the like. Say.

  Referring to FIG. 2, servers 2 and 3 include conference processing units 6 and 13, image sharing units 7 and 14, image storage units 8 and 15, display units 9 and 16, and control units 10 and 17. , Respectively. The conference processing units 6 and 13 perform conference start processing based on conference start operation information from the terminals 4 and 5. The image sharing units 7 and 14 perform processing for sharing medical images necessary for a conference. The image storage units 8 and 15 temporarily store medical images necessary for the conference shared by the image sharing unit (only during the conference). The display units 9 and 16 send display information such as shared medical images and video images during the conference to the terminals 4 and 5 for display. The control units 10 and 17 control the entire server itself, are connected to medical devices and the like in medical facilities such as hospitals, and are connected to databases 12 and 19 that store medical images, respectively.

  Next, with reference to FIG. 3, the flow of conference processing when a conference is started from the terminal 4 to the terminal 5 will be described. First, operation information for starting conference processing is sent from the terminal 4 to the control unit 10 of the server 2, and the control unit 10 of the server 2 causes the conference processing unit 6 of the server 2 to start conference processing. The conference processing unit 6 starts the conference in step S301. Next, in step S302, a conference request is sent from the conference processing unit 6 of the server 2 to the conference processing unit 13 of the server 3. Upon receiving the conference request, the conference processing unit 13 of the server 3 determines whether or not the conference can be connected in step S303. If no conference is held, the process proceeds to step S304, and the process ends. In the case of having a conference, the process proceeds to step S305, and a connection response is sent to the conference processing unit 6 of the server 2.

  Next, in step S306, the conference processing unit 6 of the server 2 confirms the presence or absence of a video device such as a web camera used during the conference. If there is no video equipment, the process proceeds to step S307, and a conference is performed without displaying video. If there is a video device, the process proceeds to step S308, and a video device connection request is sent to the conference processing unit 13 of the server 3. Upon receiving the video equipment connection request, the conference processing unit 13 of the server 3 confirms the presence or absence of the video equipment in step S309. If there is no video equipment, the process proceeds to step S310, and a conference is performed without displaying video. If there is a video device, the process advances to step S311 to send a video device connection response to the conference processing unit 6 of the server 2.

  Next, the conference processing unit 6 of the server 2 confirms whether or not there is an audio device in step 312. If there is no audio device, the process proceeds to step S313, and the conference is performed without outputting audio. If there is an audio device, the process advances to step S314 to send a connection request for the audio device to the conference processing unit 13 of the server 3. The conference processing unit 13 of the server 3 that has received the connection request for the audio device checks the presence or absence of the audio device in step S315. If there is no audio device, the process proceeds to step S316, and the conference is performed without outputting audio. If there is an audio device, the process proceeds to step S317, and the connection response of the audio device is sent to the conference processing unit 6 of the server 2. After the connection processing of the video equipment and audio equipment is completed, the conference is started in step S318. In the present embodiment, the conference process from the terminals 4 to 5 has been described. However, the conference may be started from the terminals 5 to 4. In that case, the flow is the reverse of the conference process described above. Further, the flow is not limited to this, and the flow may be executed by changing the order, or may be executed simultaneously.

  Next, the flow of image sharing processing will be described with reference to FIGS. First, referring to FIG. 4, a flow in the case where both the servers 2 and 3 hold medical images that need to be shared at the conference will be described. The control unit 10 of the server 2 starts image sharing in step S401. In step S <b> 402, the control unit 10 sends a medical image sharing instruction to be shared at the conference to the image sharing processing unit 7 of the server 2. Next, the image sharing processing unit 7 confirms whether the conference process is completed in step S403. If the conference process is not yet completed, the process proceeds to step S404, and the conference process described above is performed. When the conference process is completed, in step S405, confirmation of the presence / absence of a medical image shared at the conference is sent to the image sharing unit 14 of the server 3. If there is a medical image to be shared in step S406, the process proceeds to step S407, and an image sharing response is sent to the image sharing processing unit 7 of the server 2. The case where there is no medical image to be shared will be described later with reference to FIG.

  Next, in step S <b> 408, the image sharing unit 7 of the server 2 loads the medical image to be shared and temporarily stores it in the image storage unit 8. When temporarily storing in the image storage unit 8 of the server 2, in step S409, the control unit 10 of the server 2 creates a control thumbnail of the medical image to be shared. After step S409 or simultaneously with step S410, the operation information loaded with the image is sent to the image sharing processing unit 14 of the server 3. Upon receiving the operation information sent from the image sharing processing unit 7 of the server 2, the image sharing processing unit 14 of the server 3 loads the medical image shared in step S411 and temporarily stores it in the image storage unit 15 of the server 3. To do. When temporarily storing in the image storage unit 15, the control unit 17 of the server 3 creates a control thumbnail in step S412. When the control thumbnail creation in the server 2 and the server 3 is completed, the medical image sharing process necessary for the conference is completed (step S413), and the servers each have the same medical image.

  Next, referring to FIG. 5, a flow when there is no medical image that needs to be shared at the conference will be described. Steps S501 to S505 are the same as steps S401 to S405 described above, and details thereof are omitted. In step S506, since there is no medical image to share, the process proceeds to step S507, and a request for the medical image to be shared is sent to the image sharing processing unit 7 of the server 2. Upon receiving the request for the medical image to be shared, the image sharing processing unit 7 of the server 2 prepares the medical image to be shared in step S508. At this time, the image sharing processing unit 7 of the server 2 prepares medical images not subjected to processing such as compression processing in the database 12 in the medical facility via the control unit 10. When the preparation of the image to be shared is completed, the image sharing processing unit 7 of the server 2 transmits the medical image to the image sharing processing unit 14 of the server 3 in step 509. At this time, the image is not subjected to compression processing or the like, and the medical image is transmitted as it is. The image sharing processing unit 14 of the server 3 receives the medical image in step S510, and sends a reception completion to the image sharing processing unit 7 of the server 2 in step S511.

  Steps S512 to S517 are the same as steps S408 to S413 described above, and the details are omitted. In the present embodiment, the image sharing process from the servers 2 to 3 has been described. However, the image sharing process from the servers 3 to 2 may be used. In that case, the flow is the reverse of the image sharing process described above. Further, the flow is not limited to this, and the flow may be executed by changing the order, or may be executed simultaneously. The shared medical image is displayed on the terminals 4 and 5 via the display units 9 and 16, respectively. At this time, since the shared medical image is not subjected to processing such as compression processing, when both the terminals 4 and 5 use the same display device, the fidelity of the displayed medical image is the same. .

  In this embodiment, the flow of performing the image sharing process after the conference process has been described. However, the present invention is not limited to this, and may be executed independently at the same time, or the conference process may be executed after the image sharing process. . By executing the conference process and the image sharing process independently at the same time, it is possible to shorten the time from the start of connection by the terminals of the conference parties to the start of the actual conference.

  Next, with reference to FIGS. 6-8, the operation of the medical image to be shared during the conference and the method of sharing information on the operation will be described. First, in FIG. 6, a method for sharing operation information when selecting image 1 from thumbnail images displayed on the terminal 4 will be described. First, the terminal 4 performs an operation of selecting the image 1 from the thumbnail images. Next, operation information for selecting the image 1 from the thumbnail images is sent from the terminal 4 to the server 2. The control unit 10 of the server 2 displays the medical image of the image 1 selected from the thumbnail images on the display unit 9 of the server 2 based on the sent operation information, and the display unit 9 displays the display information on the terminal 4. In the terminal 4, the selected image 1 is displayed on the screen. The control unit 10 of the server 2 sends operation information to the control unit 17 of the server 3 after displaying the image 1 on the display unit 9 or at the same time. The control unit 17 of the server 3 selects the image 1 from the thumbnail images based on the received operation information and displays the image 1 on the display unit 16 of the server 3, and the display unit 16 sends the display information to the terminal 5. Then, the selected image 1 is displayed on the screen.

  Next, a method for sharing operation information in the case of displaying a plurality of screens in which image 2 is displayed in addition to image 1 from thumbnail images displayed on terminal 5 in FIG. 7 will be described. First, in addition to the image 1 displayed on the terminal 5, an image 2 is selected from the thumbnail images, and an operation for selecting an image layout for displaying the image 2 is performed. Next, the operation information of the operation executed on the terminal 5 is sent from the terminal 5 to the control unit 17 of the server 3. At this time, it is assumed that a layout for displaying the image 1 and the image 2 is selected as shown in FIG. Based on the operation information sent from the terminal 5, the control unit 17 displays two medical images side by side on the display unit 16 of the server 3, and the display unit 16 sends the display information to the terminal 5. The selected image 1 and image 2 are displayed on the screen based on the selected layout. The control unit 17 of the server 3 sends the operation information to the control unit 10 of the server 2 after being displayed on the display unit 16 based on the operation information of the terminal 5 or simultaneously. The control unit 10 of the server 2 selects the image 2 from the thumbnail images based on the received operation information, displays two medical images side by side on the display unit 9 of the server 2 according to the image layout, and the display unit 9 The display information is sent to the terminal 4, where the images 1 and 2 are displayed on the screen according to the selected layout. In the present embodiment, the method of displaying two medical images side by side has been described. However, the present invention is not limited to this, and the case where two or more images are displayed side by side is also based on each other's operation information. Implemented using a method of manipulating images.

  Next, a method for sharing operation information when the pointers are displayed on the terminals 4 and 5 in FIG. 8 will be described. First, an operation for displaying the pointer 1 that can be operated by the terminal 4 is performed from the terminal 4 to the control unit 10 of the server 2, and the operation information is transmitted. The control unit 10 of the server 2 displays the pointer 1 on the display unit 9 of the server 2, and the display unit 9 sends the display information to the terminal 4. In the terminal 4, the pointer 1 that can be operated on the terminal 4 is displayed on the screen. Is displayed. The control unit 10 of the server 2 sends operation information to the control unit 17 of the server 3 after displaying the pointer 1 on the display unit 9 or simultaneously with the display. The control unit 17 of the server 3 displays the pointer 1 on the display unit 16 of the server 3 based on the received operation information. The display unit 16 sends the display information to the terminal 5. An inoperable pointer 1 is displayed on the screen. The pointer 1 displayed on the screen of the terminal 5 is displayed at the same position (position) as the pointer 1 displayed on the screen of the terminal 4.

  Next, the control unit 17 of the server 3 displays the pointer 2 that can be operated on the terminal 5 on the display unit 16 after displaying the pointer 1 or at the same time, and the display unit 16 displays the display information on the terminal 5. In the terminal 5, in addition to the pointer 1, the pointer 2 that can be operated on the terminal 5 is displayed. The control unit 17 of the server 3 sends the display information of the pointer 2 to the control unit 10 of the server 2 after displaying the pointer 2 on the display unit 16 or at the same time. The control unit 10 of the server 2 displays the pointer 2 on the display unit 9 of the server 2 based on the received display information. The display unit 9 sends the display information to the terminal 4, and the terminal 4 adds it to the pointer 1. The pointer 2 that cannot be operated on the terminal 4 is displayed on the screen. The pointer 2 displayed on the screen of the terminal 4 is displayed at the same position (position) as the pointer 2 displayed on the screen of the terminal 5.

  At this time, a distinction is made by changing the colors of the pointers 1 and 2 displayed on the screen. For example, on the screen of the terminal 4, the color of the pointer 1 that can be operated on the terminal 4 is displayed in white, and the color of the pointer 2 that cannot be operated is displayed in black. Conversely, on the screen of the terminal 5, the color of the pointer 2 that can be operated on the terminal 5 is displayed in white, and the color of the pointer 1 that cannot be operated is displayed in black. After the mutual pointers are displayed, the pointer 1 or the pointer 2 moves based on the operation information sent from the terminal 4 or the terminal 5. For example, when an operation of moving the pointer 1 that can be operated on the terminal 4 to the left and right is performed, the terminal 4 first sends operation information to the control unit 10 of the server 2. The control unit 10 of the server 2 causes the display unit 9 of the server 2 to display the pointer 1 that moves based on the operation information. The display unit 9 sends the display information to the terminal 4, and the terminal 4 performs the pointer as operated. 1 is displayed by moving left and right. The control unit 10 of the server 2 sends the operation information of the pointer 1 to the control unit 17 of the server 3 after being displayed on the display unit 9 or at the same time. The control unit 17 of the server 3 displays the pointer 1 that moves left and right on the display unit 16 of the server 3 based on the received operation information, and the display unit 16 sends the display information to the terminal 5. 1 is displayed by moving left and right. In this embodiment, the color of the pointer is white and black. However, the color of the pointer displayed on the screen of each terminal is not limited to this. Any color can be used as long as it can be distinguished.

  In the above, the operation of the medical image to be shared during the conference and the method of sharing the operation information have been described. However, without being limited to this method, the operation information performed by the terminal 4 or the terminal 5 is shared with each other via the network, and the operation is performed based on the operation information and displayed on each terminal. It only has to be. Also, the operation of medical images is not limited to this embodiment. For example, brightness / contrast adjustment display, enlargement / reduction display, panning, partial enlargement display, window frame fitting display, pixel size display, actual size display , Black and white reverse display, color palette, RGB component, HSV component, LUT, smoothing, sensitization, edge detection, median filter, threshold, display condition preset, left / right rotation display, up / down reverse display, left / right reverse display, images in the same series Synchronous display, Synchronized image display within multiple series, Angle measurement, Angle measurement against reference line, Length measurement, CT value measurement, Elliptical ROI measurement, Polygon ROI measurement, Arbitrary area ROI measurement, Graphic drawing, Text drawing, Histogram display , Marking, cardiothoracic ratio measurement, drawing overlay deletion, image information display, overlay table / Hide, Display layout setting, Tile mode display, Stack mode display, Multiple series display, Multiple series synchronous display, Thumbnail image display, Scout image display, Reference line display, Cine display, Movie display, Series mode display, Study mode display , Hanging protocol function, mammographic image side-by-side display, mammographic image breast position automatic alignment, mammographic image right-and-left magnifying mirror, various shaping measurements such as cobb angle, FTA, Sharp angle, implant simulation, magnification correction function, and 3D For images, volume rendering, 3D cursor, MPR, CPR, MIP, MinIP, Curved MPR, oblique, fusion, inter-volume calculation, virtual endoscope, volume, distance, etc. measurement, fat measurement, extraction, synthesis, editing ,analysis The operation may include at least one of various workstation functions related to medical images. In this case as well, it is only necessary that the operation information is shared with each other and the same image or the like is displayed between the terminals. After the conference, the images temporarily stored in the image storage units 8 and 15 may be stored as they are, or the images may be deleted.

  Further, in the present embodiment, the conference process, the image sharing process, and the operation of the shared image at the time of the conference between the terminal 4 and the terminal 5 via the servers 2 and 3 have been described. However, without using the terminal 4 and the terminal 5, for example, in each of the servers 2 and 3, a video device such as a web camera capable of conference, an audio device such as a microphone, and a monitor for displaying an image to be shared As long as a display or the like is provided, this embodiment may be applied only between servers.

  Next, with reference to FIG. 9, the flow at the time of performing a conference between three or more terminals will be described. When sharing conferences and medical images with three or more terminals, the communication is performed via a plurality of connection units that are connected to each server and capable of two-way communication. At this time, for example, when the conference process and the image sharing process are performed from the terminal 25, a plurality of the above processes are executed simultaneously. Furthermore, the operation information of the medical image is shared by all the servers via the plurality of connection units, and the same image or the like is displayed on each terminal. Even in this case, the pointer to be displayed may be any number of colors as long as the pointer that can be operated can be distinguished from other pointers, and the number of colors is not limited.

  As described above, according to the present embodiment, it is possible to display an image such as a medical image and to manipulate the image during a conference performed by transmitting each other's video and / or audio to each other. . Accordingly, by sharing operation information with each other, a medical support system is provided that allows medical images to be shared and exchanged in both directions through a network even if they are in remote locations and under any line conditions. Can do.

1 Telemedicine network system 2, 3 Server 4, 5 Terminal 6, 13 Conference processing unit 7, 14 Image sharing processing unit 10, 17 Control unit

Claims (5)

A medical support system in which one server bidirectionally connected to one terminal and the other server bidirectionally connected to the other terminal can communicate with each other via a network,
The one server and the other server are:
A conference processing unit that executes a conference between the one terminal and the other terminal by transmitting each other's video and audio, or only audio;
Bidirectional between the one server and the other server, and the image sharing processing unit for sharing the medical image which is not different from the process and the image stored in the database of the medical facility,
An image storage unit for temporarily storing the shared image shared by the image sharing processing unit;
A control unit that controls its own server based on operation information sent from the one or the other terminal;
An image display unit that transmits to the one or other terminal display information for the one or other terminal to display the shared image whose display is controlled by the control unit;
Each with
The one or other terminal displays the shared image based on the display information sent from the one or other image display unit, transmits the operation information to the control unit of the one or other server,
In the one server, the control unit receives operation information for displaying the first pointer on the one terminal from the one terminal, and sends the operation information to the image display unit based on the operation information. The display information for displaying the first pointer on the one terminal is transmitted to the one terminal, and the operation information is transmitted to the control unit of the other server,
In the other server, based on the received operation information, the control unit cannot operate the second terminal at the same position as the position where the first pointer is displayed on the image display unit. Display information for displaying the pointer is transmitted to the other terminal, and the other terminal displays the second pointer based on the display information transmitted from the other terminal. Medical support system. The control unit of the one server operates the first pointer according to the operation information for operating the first pointer from the one terminal, and transmits the operation information to the control unit of the other server. Send
The medical support system according to claim 1, wherein the control unit of the other server operates the second pointer displayed on the other terminal according to the received operation information. The medical support system according to claim 1 or 2, wherein the first pointer and the second pointer have different colors.   The one image sharing processing unit transmits the shared image from the medical facility database connected to the one server to the other image sharing processing unit when the other image storage unit does not have the shared image. And it preserve | saves in said other image storage part, The medical assistance system of any one of Claims 1-3 characterized by the above-mentioned. The medical support system according to any one of claims 1 to 4, wherein the display of the shared image and the operation of the pointer are simultaneously performed during the conference.

Images