JP7276741B2 - Information sharing system - Google Patents

Description

The present disclosure relates to technology for sharing medical information obtained from medical devices used in surgery and treatment.

Patent Literature 1 below discloses an emergency medical device that uses a plurality of terminals connected via a communication server to realize communication between a person in charge of a hospital where an emergency patient is treated and a pre-registered supervising specialist. A patient care support system is described. In this system, the person in charge of the hospital transmits diagnostic information regarding the emergency patient to the supervising specialist, and the supervising specialist transmits opinion information (hereinafter referred to as a comment) regarding treatment to the person in charge of the hospital. The diagnostic information includes an image of the affected area of the emergency patient, the time at which the image was taken, and the like. Text data input at the terminal is used for the opinion information.

JP 2012-27565 A

However, in the prior art described in Patent Document 1, diagnostic information shared by system users is limited to images of affected areas, and information necessary for judgment is insufficient. In addition, comments exchanged in real time can be important information, but when diagnostic information is played back for review, etc., the correspondence between comments and diagnostic information is unclear, so comments cannot be used effectively. .

One aspect of the present disclosure is to provide a technique for easily sharing system user comments and realizing more effective use.

One aspect of the present disclosure is an information sharing system comprising a server (3) and one or more terminal devices (4).
The server stores a plurality of tagged device information and a plurality of tagged comment information. The tagged device information includes device information acquired from a plurality of medical devices (21) used for patient treatment, the time when the device information was acquired, and information representing the type of device information. This is information associated with one tag information. The tagged comment information is information in which second tag information including information for specifying a comment target is associated with comment information whose comment target is at least a part of device information.

A terminal device is connected to a server via a communication network. The terminal device includes a distribution request unit (43: S520), a main image generation unit (43: S610 to S630), a display generation unit (43: S610 to S620, S640), and a target designation unit (43: S710 to S720 ), a comment input unit (43: S820), and an information generation unit (43: S740 to S800, S840 to S890).

The delivery request unit requests the server to deliver the tagged device information and tagged comment information. The main image generation unit extracts one or more types of device information associated at the same time from the tagged device information acquired from the server in response to a request from the distribution request unit, and displays the extracted device information on one screen. main image data is generated. When a comment target is included in a main image, which is an image displayed by the main image data, the display generation unit generates information corresponding to the comment target from tagged comment information acquired from the server in response to a request from the distribution request unit. Extract comment information. Furthermore, the display generation unit generates comment display data, which is display data for displaying the extracted comment information in association with the comment target in the main image. The target specifying section specifies a comment target in the display image displayed by the main image data. The comment input unit inputs comment information that is information associated with the comment target specified by the target specifying unit. The information generating unit generates tagged comment information by associating the comment information input by the comment input unit with the second tag information including information for specifying the comment target specified by the target specifying unit. and store it in the server.

According to such a configuration, it is possible not only to share a plurality of device information integrated by time with an arbitrary viewer via a terminal device, but also to make various comments on arbitrary comment targets displayed on the screen. Information can be left, and the comment information can also be shared with arbitrary viewers.

As a result, viewers can obtain useful information from comment information that cannot be obtained from device information alone, and by using comment information, they can receive advice, have discussions, etc. , various communication can be realized easily.

1 is a block diagram showing the configuration of an information sharing system; FIG. FIG. 3 is a functional block diagram showing the functions of each part that constitutes the information sharing system; FIG. 4 is an explanatory diagram showing the structure of tagged device information; FIG. 4 is an explanatory diagram showing the structure of tagged comment information; It is a flow chart showing list information generation processing. 9 is a flowchart showing device information generation processing; It is a flow chart showing distribution processing. 4 is a flowchart showing terminal processing; FIG. 10 is an explanatory diagram showing a selection list image; 9 is a flowchart showing display update processing; FIG. 9 is an explanatory diagram showing information read out in display update processing; FIG. 4 is an explanatory diagram showing one example of a layout of a main image; It is a flow chart showing comment provision processing. FIG. 10 is an explanatory diagram showing a display example of comment information; FIG. 10 is an explanatory diagram showing a display example of comment information;

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.
[1. overall structure]
An information sharing system 1 shown in FIG. 1 includes a plurality of information providing devices 2, a server 3, and a plurality of terminal devices 4, which are interconnected via a communication network 5. The information sharing system 1 shown in FIG.

The information providing apparatus 2 is installed in each hospital, for example, and stores integrated information obtained by integrating along the time axis device information obtained from various devices used for surgery and treatment in the server 3 on the communication network 5. .

The terminal device 4 is installed in places such as hospitals and medical colleges where doctors, medical researchers, students, and the like can use it. The terminal device 4 may be installed near the information providing device 2 . The terminal device 4 acquires integrated information from the server 3 via the communication network 5, and executes at least an application for supporting information sharing and communication between users viewing the integrated information using the terminal device 4. . Note that users also include providers who provide integrated information.

The information sharing system 1 is used, for example, for real-time examination and support during surgery, and also for surgical analysis and educational purposes after surgery.
[1-1. Information provision device]
As shown in FIG. 1 , the information providing device 2 includes a device group 21 , a provision processing section 23 and an input section 27 .

The device group 21 includes a plurality of devices 22A-22H, as shown in FIG. Each of the devices 22A to 22H generates data and supplies it to the provision processing unit 23. FIG. In the following description, the device 22 is used to refer to an arbitrary device without distinguishing between the plurality of devices 22A to 22H.

The device 22 is medical equipment, medical equipment, or the like used during surgery or treatment. Specifically, medical equipment includes, for example, biological information monitors, respiratory function monitors, hemodynamic monitors, sedation monitors, anesthesia machines, infusion pumps, syringes, blood purifiers, heart-lung machines, assisted circulation devices, and the like. Examples of medical equipment include surgical navigation systems (hereinafter referred to as surgical navigation), IP cameras, medical gas systems, endoscopes, microscopes, electric scalpels, drills, ultrasonic irradiation devices, and the like. Devices may also include air conditioning systems, isolation systems, and the like.

The data supplied from the device 22 (hereafter referred to as device information) includes measurement values from the device 22, video and still images output from the device 22, device status including the usage status and errors of the device 22, device 22 It includes setting values of itself and setting values related to parameters handled by the device 22 .

The surgery navigation system is a device that acquires the three-dimensional position and orientation of surgical instruments during surgery. Surgical instruments include electric scalpels and forceps, as well as microscopes and the like. As a method of acquiring the three-dimensional position and orientation of the surgical instrument, for example, the method described below can be used. That is, a plurality of markers are attached in advance to specific positions of the surgical instrument, and a vector from a predetermined reference position in the space where the surgery is performed to the specific position on the surgical instrument is specified as the three-dimensional position of the surgical instrument, A posture is specified from the positional relationship between a plurality of markers. Also, the focal point of the microscope may be specified as a three-dimensional position. At this time, the three-dimensional position is specified by associating the patient's diseased part with a three-dimensional image taken in advance by CT, MRI, or the like. CT is an abbreviation for Computed Tomography, and MRI is an abbreviation for Magnetic Resonance Imaging.

Returning to FIG. 1, the provision processing unit 23 includes a microcomputer having a CPU 231 and semiconductor memory such as RAM and ROM (hereinafter referred to as memory 232).
The provision processing unit 23 functionally includes a plurality of providers 24A to 24H and a middleware 25, as shown in FIG. In the following description, the provider 24 is used when referring to any provider without distinguishing among the plurality of providers 24A to 24H.

Each provider 24 is connected to one of the devices 22 respectively. Provider 24 mediates the exchange of information between connected devices 22 .
The middleware 25 is a communication interface that provides unified access means together with the provider 24 to various devices 22 connected to the provider 24 regardless of differences in programming languages and communication protocols. ORiN is used as the middleware 25 here. ORiN is an abbreviation for Open Resource interface for the Network.

The middleware 25 also has at least an information generation function 251 that generates list information and tagged device information T_Idv, and a clock function 252 that provides the current time upon request.

The input unit 27 includes at least one of a keyboard, a mouse, a touch panel, and the like, and is used to input commands and information to the provision processing unit 23 . Note that instead of providing the input unit 27 , a configuration may be adopted in which a command to the provision processing unit 23 is input via one of the terminal devices 4 .

[1-2. server]
The server 3 includes a database 31 and a server processing section 32 .
The database 31 is implemented by an HDD or the like, and has at least a device information area 311, a comment information area 312, and a list information area 313, as shown in FIG. The device information area 311 stores tagged device information T_Idv, the comment information area 312 stores tagged comment information T_Icm, and the list information area 313 stores list information. Also, the tagged device information T_Idv and the tagged comment information T_Icm are collectively referred to as integrated information.

As shown in FIG. 3, tagged device information T_Idv has first tag information T1 and device information Idv.
The device information Idv is information acquired from any device 22 belonging to the device group 21 during surgery, treatment, or examination (hereinafter referred to as treatment, etc.). Data etc. are included.

The first tag information T1 is information related to device information Idv, and includes at least a case identification tag T11, a case information tag T12, a time tag T13, and a data type tag T14.

The case identification tag T11 is used to uniquely identify a series of device information Idv continuously acquired for a certain period in any of the information providing apparatuses 2 when a certain patient undergoes treatment or the like. identification information. The case identification tag T11 may be set so as to identify the information providing apparatus 2 that provided the series of device information Idv.

The case information tag T12 indicates information related to the treatment identified by the case identification tag T11. Specifically, it includes the therapist (for example, the surgeon in the case of surgery), the age and sex of the person to be treated, the name of the disease, the name of the operation, and the like. The information is not limited to this, and may include information that can be used as a search item when used for educational purposes at a later date.

The time tag T13 is information representing the time when the device information Idv was acquired (hereinafter referred to as acquisition time).
The data type tag T14 includes identification information for specifying the device 22 that generated the device information Idv. When a plurality of device information Idv are acquired from one device 22, identification information for specifying which device information Idv is included is also included.

As shown in FIG. 4, tagged comment information T_Icm has second tag information T2 and comment information Icm.
The comment information Icm is information input via the terminal device 4, and specifically includes at least one of text data, voice data, and image data. Note that the image data may include bitmap data representing handwritten characters and drawings.

The second tag information T2 is information related to the comment information Icm, and includes a case identification tag T21, a comment identification tag T22, an input person information tag T23, a comment target tag T24, a time tag T25, a time tag T26, At least a comment type tag T27 is included.

The case identification tag T21 is the same as the case identification tag T11 in the first tag information T1, and is associated with the device information Idv displayed on the terminal device 4 when the comment information Icm was input. An identification tag T11 is used.

The comment identification tag T22 is information for individually identifying the comment input from the terminal device 4. FIG. This information is information for enabling comments on comments.
The input person information tag T23 is information for identifying the input person who input the comment or the terminal device 4 where the input was performed.

The comment target tag T24 is information for specifying the target of the comment. The comment target may include all or part of the device information Idv identified by the data type tag T14. Further, the comment target may include the entire information that does not specify the device information Idv (that is, the entire screen on which the device information Idv is displayed) and the comment specified by the comment identification tag T22. When the device information Idv is a value measured by the device 22 and the measured value is displayed as a number or graph, the part of the device information Idv may include the displayed number, graph, or the like.

The time tag T25 is information representing the acquisition time of the comment target, that is, the time similar to the time tag T13 described above.
The time tag T26 is information representing the time when the comment target was specified when the comment information Icm was input. The time tag T26 represents the current time at which the integrated information is being reproduced (hereinafter referred to as reproduction time). However, when the integrated information is reproduced in real time, that is, when the integrated information is generated and reproduced at the same time, the time tag T26 may have the same value as the time tag T25.

The comment type tag T27 is information representing the format of the comment information Icm, and includes text data, voice data, image data, voice recognition data, motion analysis data, and the like.
Although illustration is omitted, the list information is generated for each series of integrated information identified by the case identification tag T11, and has the case identification tag T11, the case information tag T12, and time information. If the integrated information was generated in the past, the time information is the time required to reproduce all of them, and if the integrated information is generated in real time, the estimated start time and estimated end time of generation. is.

Returning to FIG. 1, the server processing unit 32 includes a microcomputer having a CPU 321 and semiconductor memory such as RAM and ROM (hereinafter referred to as memory 322).
The server processing unit 32 stores the list information and the tagged device information T_Idv provided from the information providing device 2 and the tagged comment information TI_cm provided from the terminal device 4 in the database 31 . The server processing unit 32 also executes at least a distribution AP 33, which is an application (hereinafter referred to as AP) for distributing designated integrated information, according to a request from the terminal device 4. FIG. Details of the distribution AP 33 will be described later.

[1-3. Terminal device]
As shown in FIG. 1 , the terminal device 4 includes a display section 41 , an input section 42 and a terminal processing section 43 . All of the plurality of terminal devices 4 have the same configuration.

The display unit 41 is, for example, a touch panel type liquid crystal display, and displays three-dimensional images such as MPR images, waveform data and numerical values representing measurement results such as biological information, various operation buttons, setting values of each device, and layout values. Displayable. MPR is an abbreviation for Multi Planar Reconstruction.

The input unit 42 is, for example, input devices such as a touch panel, a keyboard, a mouse, a microphone, a camera, and a motion sensor provided on the liquid crystal display described above. The input unit 42 outputs text data, position data, voice data, image data, etc. corresponding to input operations through these input devices to the terminal processing unit 43 .

The terminal processing unit 43 includes a microcomputer having a CPU 431 and semiconductor memory such as RAM and ROM (hereinafter referred to as memory 432). The terminal processing unit 43 executes one or more applications related to integrated information stored in the database 31 of the server 3 . Here, the terminal processing unit 43 executes at least a voice recognition AP 44, a motion analysis AP 45, a display update AP 46, a comment addition AP 47, and a layout change AP 48, as shown in FIG.

The speech recognition AP 44 executes speech recognition processing on the user's speech acquired by a microphone, which is one of the input devices that constitute the input unit 42, and converts the speech, which is a command or text data corresponding to the recognition content, into Recognition data is generated and output together with the speech to be recognized.

The motion analysis AP 45 analyzes an image of a user's motion (for example, motion of hands, face, head, etc.) captured by a camera, which is one of the input devices constituting the input unit 42 . The motion analysis AP 45 also generates motion analysis data, which is commands or text data corresponding to the content of the analysis, and outputs it together with the image to be analyzed.

In other words, the terminal device 4 is provided with the input unit 42, the speech recognition AP 44, and the motion analysis AP 45, thereby realizing non-contact input operations.
The display update AP 46, comment addition AP 47, and layout change AP 48 will be described later.

[2. process]
[2-1. List information generation process]
The information generation function 251 by the provision processing unit 23 is implemented by a list information generation AP and a device information generation AP. First, list information generation processing realized by the provision processing unit 23 executing the list information generation AP will be described with reference to the flowchart of FIG. The list information generation process is started when an instruction to generate list information is input via the input unit 27 .

In S110, the provision processing unit 23 generates identification information for identifying a series of newly generated integrated information (hereinafter referred to as a case), that is, a case identification tag T11. The case identification tag T11 may be, for example, a combination of an identifier that uniquely identifies the information providing device 2 and a serial number that is updated each time a new case occurs in the information providing device 2 .

In subsequent S120, the provision processing unit 23 collects information about the case information tag T12 and time information indicating the scheduled start and end times of acquisition of the device information regarding the case specified by the identification information generated in S110. are obtained through the input unit 27 .

In subsequent S130, the provision processing unit 23 stores the case identification tag T11 generated in S110, the case information tag T12 acquired in S120, and the time information as list information in the memory 232, and stores the list information in the memory 232. to end the process.

[2-2. Device information generation process]
Device information generation processing realized by the provision processing unit 23 executing the device information generation AP will be described with reference to the flowchart of FIG. The device information generation process is executed for each provider 24 when identification information to be used for the case identification tag T11 is specified and a generation start command is input via the input unit 27 of the information providing apparatus 2 .

In S210, the case identification tag T11 and the case information tag T12 are acquired from the list information corresponding to the designated identification information.
In subsequent S220, the provision processing unit 23 determines whether or not the device 22 is connected to the provider 24 to be processed. If the provision processing unit 23 determines that the device 22 is not connected, it waits by repeating the same step, and if it determines that the device 22 is connected, the process proceeds to S230.

In S<b>230 , the provision processing unit 23 acquires device information Idv from the device 22 via the provider 24 . The device information Idv acquired from one device 22 may be of one type or of multiple types.

In subsequent S240, the provision processing unit 23 acquires the current time and generates a time tag T13. The current time may be acquired from the clock function 252 of the microcomputer, or may be time information representing the acquisition time of the device information Idv provided from the device 22 together with the device information Idv.

In subsequent S250, the provision processing unit 23 generates a data type tag T14 representing the type of device information Idv acquired in S230. A data type tag T14 prepared in advance according to the device 22 connected to the provider 24 may be used.

In subsequent S260, the provision processing unit 23 generates tagged device information T_Idv which is the device information Idv in which the first tag information T1 is inserted. The first tag information T1 includes a case identification tag T11, a case information tag T12, a time tag T13, and a data type tag T14, which are obtained and generated in the above process. Further, the provision processing unit 23 stores the generated tagged device information T_Idv in the memory 232 of the information providing apparatus 2 and transmits it to the server 3 .

In subsequent S270, the provision processing unit 23 determines whether or not to end the generation of integrated information. Specifically, the information providing apparatus 2 terminates the process when a command to terminate the generation of integrated information is input via the input unit 27, and when such a command is not input , the process returns to S220 to continue generating integrated information.

[2-2. Delivery process]
Distribution processing realized by the server processing unit 32 executing the distribution AP 33 will be described with reference to the flowchart of FIG. The distribution process is repeatedly executed when the server 3 is activated.

In S310, the server processing unit 32 determines whether or not the list information has been acquired from the information providing apparatus 2. If the list information has been acquired, the process proceeds to S320. to S340.

In S<b>320 , the server processing unit 32 stores the acquired list information in the list information area 313 of the database 31 .
In the following S330, the acquired list information is distributed to all the terminal devices 4, and the process is terminated. Note that the list information may be distributed in response to a request from the terminal device 4. FIG.

In S340, the server processing unit 32 determines whether or not the tagged device information T_Idv has been acquired from the information providing apparatus 2. If the tagged device information T_Idv has been acquired, the process proceeds to S360, and the tagged device information T_Idv is acquired. If the information T_Idv has not been acquired, the process proceeds to S350.

In S350, the server processing unit 32 determines whether or not the tagged comment information T_Icm has been acquired from the terminal device 4. If the tagged comment information T_Icm has been acquired, the process proceeds to S360, and the tagged comment information is acquired. If T_Icm has not been acquired, the process proceeds to S390.

In S<b>360 , the server processing unit 32 stores the acquired tagged device information T_Idv or tagged comment information T_Icm in the device information area 311 or comment information area 312 of the database 31 . At this time, each series of information identified by the case identification tags T11 and T21 may be divided and stored.

In subsequent S370, the server processing unit 32 determines that the case identified by the case identification tag T11 of the acquired tagged device information T_Idv or the case identification tag T21 of the tagged comment information T_Icm (hereinafter referred to as the target case) is being distributed in real time. determine whether there is Real-time distribution means receiving the integrated information of the target case and simultaneously delivering the received integrated information to the terminal device 4 . If the server processing unit 32 determines that the target case is being distributed in real time, the process proceeds to S380, and if it is determined that the target case is not being distributed in real time, the process ends.

In S380, the server processing unit 32 distributes the tagged device information T_Idv or the tagged comment information T_Icm stored in the database 31 in S360 to the terminal device 4 (hereinafter referred to as the target terminal) that is the target of real-time distribution. and terminate the process.

At S390, the server processing unit 32 determines whether or not a distribution request has been received from the terminal device 4. If a distribution request has been received, the process proceeds to S400. Terminate once.

In S400, the server processing unit 32 sets the terminal device 4 that has issued the distribution request as the target terminal.
In subsequent S<b>410 , the server processing unit 32 collects integrated information about the target case from the database 31 . Specifically, it collects all tagged device information T_Idv and tagged comment information T_Icm whose contents of the case identification tags T11 and T21 match the identification information indicated in the distribution request. However, if the request is for real-time distribution and the distribution has not yet started, the integrated information to be collected may not exist in the database 31 .

In subsequent S420, the information collected in S410 is delivered to the target terminal set in S400, and the process is terminated. However, if the distribution request is a request for real-time distribution, a setting indicating that real-time distribution is in progress is made to the target terminal.

[2-3. terminal processing]
Terminal processing executed by the terminal processing unit 43 will be described with reference to the flowchart of FIG. Note that the terminal processing unit 43 executes an information storage process in parallel, apart from the display process. The information storage process is a process of receiving list information from the server 3 or integrated information about cases related to distribution requests from the terminal processing unit 43 and storing the information in a predetermined area of the memory 432 .

The terminal processing is started when there is an input for instructing activation via the input unit 42 .
In S<b>510 , the terminal processing unit 43 causes the display unit 41 to display a selection list screen based on the list information accumulated in the memory 432 . The selection list screen is, as shown in FIG. 9, a screen listing icons of cases identified by the case identification tag T11. At least part of the content of the case information tag T12 of each list information is displayed in the icon. In particular, the scheduled start time and the scheduled end time of the real-time distribution are also displayed. The selection list screen may display a case search display. For example, the information indicated in the case information tag T12, such as the surgical procedure, the surgeon, the name of the disease, etc., is displayed on the selection list screen. It may be configured so that cases can be narrowed down.

The selection list screen is configured so that a case can be selected by selecting an icon on the screen.
In subsequent S520, when the selection of an icon is detected on the selection list screen, the terminal processing unit 43 sends a delivery request indicating identification information (that is, case identification tag T11) specifying the selected case to the server 3. Send to

Since server 3 thereafter transmits integrated information having case identification tags T11 and T21 that match the identification information indicated in the distribution request (hereinafter referred to as integrated information of requested cases), terminal processing unit 43 , the integrated information of the request case is stored in a predetermined area of the memory 432 by the information storage processing described above. However, in the case of non-real-time delivery, the integrated information of the request cases is transmitted all at once, and in the case of real-time delivery, it is transmitted each time new integrated information is generated for the request cases.

In subsequent S530, the terminal processing unit 43 initializes the layout used when displaying the request case. A layout is a combination of the following elements. (1) How to divide the display area on the display screen. (2) How many types of device information are to be displayed simultaneously in each of the divided areas. (3) How to display device information (for example, images, numerical values, graphs, tables, etc.) in each divided area. (4) What kind of comment information to display. (5) How to display the comment information (for example, the size and position of the comment area). The comment information is basically related to the device information to be displayed.

In subsequent S540, the terminal processing unit 43 activates the display update AP46. The display update AP 46 will be described later.
In subsequent S550, the terminal processing unit 43 determines whether or not an end instruction to end the display processing has been input via the input unit 42. If the end instruction has been input, the processing ends and the end instruction is input. If not, the process proceeds to S560.

In S560, the terminal processing unit 43 determines whether or not a change instruction to change the layout has been input via the input unit 42. If a change instruction has been input, the process proceeds to S570, and the change instruction is input. is not input, the process proceeds to S580.

In S570, the terminal processing unit 43 activates the layout change AP 48 and returns the process to S550. When the layout change AP 48 is activated, a layout setting screen is displayed on the display unit 41, and the user of the terminal device 4 uses the layout setting screen to change the layout. The changed layout is stored in the memory 432 and reflected in the display of integrated information on the display unit 41 by display update processing.

Note that changing the layout includes narrowing down the comment information Icm to be displayed using the information indicated in the second tag information T2. For example, only comments by a specific input person may be displayed, or only text data may be displayed.

In S580, the terminal processing unit 43 determines whether or not an instruction requesting comment input has been input via the input unit 42. If the instruction has been input, the process proceeds to S590, and the instruction has been input. If not, the process returns to S550.

In S590, the terminal processing unit 43 activates the comment giving AP 47 and returns the process to S550. The comment giving AP 47 will be described later.
In this processing, S520 corresponds to the distribution requesting section, and S570 corresponds to the comment narrowing-down section and the layout specifying section.

[2-4. Display update process]
The display update processing realized by activating the display update AP 46 in the previous S540 will be described using the flowchart of FIG. Once the display update process is started, it is repeatedly executed at a preset display cycle until an end instruction is input via the input unit 42 .

In the display update process, the device information Idv corresponding to the time S designated via the input unit 42 and the The comment information Icm is displayed on the display screen of the display unit 41 . However, in the operation mode for displaying a still image, the time S is changed according to the input from the input unit 42, and in the operation mode for displaying a moving image, the time S is changed after the initial value is input from the input unit 42. , is automatically updated according to the display cycle.

In S610, the terminal processing unit 43 acquires the time S.
In the following S620, the terminal processing unit 43 retrieves from the memory 432 all the device types and all the comment types to be displayed, the devices to which the time tags T13 and T25 indicating the latest time before the time S are given. Acquire information Idv and comment information Icm. Note that an upper limit of time that can be traced back may be set for the search for the latest time.

For example, as shown in FIG. 11, the memory 432 stores four types of device information identified by A to D and two types of comment information identified by a and c. In FIG. 11, when the time S is 9:00 X milliseconds, the hours and minutes are omitted and displayed as "X" in units of milliseconds. 11 indicates the contents of the time tag T13 attached to the device information Idv of the device type A. In FIG.

When the time S is "100", the device information having the latest time tag T13 before the time S is circled in FIG. That is, for type A, the device information has the time tag T13 of "97". Type B is device information with a time tag T13 of "100". Type C is device information with a time tag T13 of "99". Type D is device information with a time tag T13 of "98".

Also, the comment information having the latest time tag T25 before the time S is the comment information with the time tag T25 of "93" for the type a. Type c means that there is no corresponding comment information within the upper limit time range.

In subsequent S630, the terminal processing unit 43 generates main image data, which is image data for displaying the device information Idv read out in S620 according to the layout set at that time. An image displayed by the main image data is hereinafter referred to as a main image.

As an example of the layout of the main image, for example, as shown in FIG. 12, it may have a monitor area A1, a video area A2, a navigation image area A3, a time area A4, and the like. The monitor area A1 is an area where a graph or the like for monitoring biological information is displayed. In the monitor area A1, biological information is displayed numerically and graphically. The image area A2 is an area in which a moving image of the treatment target captured by a camera or the like is displayed. The navigation image area A3 is an area where the three-dimensional position detected by the surgery navigation is displayed together with the three-dimensional image of the affected area prepared in advance. In the navigation image area A3, three cross-sectional images including the detection position and showing mutually different cross-sections are displayed in a three-view format. The time area A4 is an area for expressing the time range in which the time series of the integrated information exists on the time axis and displaying the time S on the time axis. In the time area A4, along with the time axis, a balloon-shaped display representing the event occurrence time may be displayed.

Returning to FIG. 10, in subsequent S640, the terminal processing unit 43 generates comment display data, which is display data for displaying the comment information Icm read out in S620 together with the main image. Below, the display displayed by comment display data is called comment display. The display position and display format of the comment display on the main image are set in advance according to the comment type of the comment information Icm or the data type of the device information Idv to be commented. Note that there may be a dedicated application for changing the display position and display format of the comment display.

In subsequent S650, the terminal processing unit 43 outputs the main image data generated in S430 and the comment display data generated in S440 to the display unit 41 as display data, thereby displaying the main image and the comment. is displayed on the display screen of the display unit 41, and the process is temporarily terminated.

In this process, S610 to S630 correspond to the main image generator, and S610 to S620 and S640 correspond to the display generator.
[4-2. Comment adding process]
The comment adding process realized by activating the comment adding AP 47 in the previous S590 will be described with reference to the flow chart of FIG.

In S710, the terminal processing unit 43, through the input unit 42, inputs the entire information (hereinafter referred to as "whole") displayed on the screen of the display unit 41 (hereinafter referred to as the display screen) as a comment target. determine whether or not has been performed. The terminal processing unit 43 shifts the process to S740 when determining that all designation inputs have been performed, and shifts the processing to S720 when determining that all designation inputs have not been performed. Note that the terminal processing unit 43 may determine that all specification inputs have been performed, for example, when the event addition button Bev shown in FIG. 12 is clicked.

In S740, the terminal processing unit 43 acquires the acquisition time corresponding to the timing at which all designation inputs are performed to generate a time tag T25, acquires the reproduction time corresponding to the timing, and generates a time tag T26. do. Note that the acquisition time is the time S acquired most recently in S210 of the display processing before the timing at which all designation inputs are performed, or the time tag associated with the device information Idv displayed at that timing. This is the time indicated by T13. The playback time is the current time acquired from the clock function of the microcomputer at the timing when all designation inputs are performed.

In subsequent S750, the terminal processing unit 43 generates a comment target tag T24 indicating that the comment target is "whole", and advances the process to S820.
In S<b>720 , the terminal processing unit 43 determines whether a position input indicating a position on the display screen has been performed via the input unit 42 . The position input may be a position input by a touch panel on the display screen, or a position indicated by a cursor displayed on the display screen operated by a mouse or keyboard. Furthermore, when the navigation image area A3 is included in the main image, it may be the three-dimensional position indicated by the navigation image. Specifically, it is the center position of each image where the auxiliary line intersects, which is shown in each of the three views. If the terminal processing unit 43 determines that the position input has been performed, the process proceeds to S760, and if it determines that the position input has not been performed, the process proceeds to S710.

In S760, the terminal processing unit 43 acquires the acquisition time and the playback time corresponding to the timing of the position input via the input unit 42, and generates time tags T25 and T26 in the same manner as in S740. .

In subsequent S770, the terminal processing unit 43 determines whether or not the specified position, which is the position on the display screen specified by the position input, is an area for displaying the comment. If the terminal processing unit 43 determines that the specified position is an area for displaying comments, the process proceeds to S790, and if it determines that the specified position is not an area for displaying comments, the process proceeds to S780. transition to

In S790, the terminal processing unit 43 removes the comment target tag T24 indicating that the comment information Icm displayed at the specified position is a comment target based on the specified position and the comment display layout at that time. generated, and the process proceeds to S820. At this time, the value of the comment identification tag T22 associated with the comment information Icm displayed at the specified position may be used as the value of the comment target tag T24.

In S780, the terminal processing unit 43 determines whether or not a number representing a measured value or a waveform (hereinafter referred to as a waveform, etc.) is displayed at the specified position, that is, whether or not a waveform, etc., is specified by position input. . If the terminal processing unit 43 determines that the waveform or the like is specified, the process proceeds to S800, and if it determines that the waveform or the like is not specified, the process proceeds to S810.

In S800, the terminal processing unit 43 generates a comment target tag T24 indicating that the waveform or the like specified by the position input is a comment target, and advances the process to S820.
In S810, the terminal processing unit 43 identifies the type of device information Idv displayed at the specified position based on the specified position and the layout of the main image at that time. The type of device information Idv is represented by a data type tag T14 associated with the device information Idv. Further, a comment target tag T24 indicating that the specified type of device information Idv is a comment target is generated, and the process proceeds to S820.

In S800 and S810, the comment target tag T24 may include information indicating the designated position as a relative position within the display area assigned to the comment target. This is information for preventing the relationship between the display and the specified position in the display area from changing when the layout is changed.

In S820, the terminal processing unit 43 determines whether or not a comment has been input via the input unit 42. If there is a comment input, the process proceeds to S840, and if there is no comment input, the process proceeds to S830.

In S830, the terminal processing unit 43 determines whether or not an instruction to cancel comment information input has been input via the input unit 42. If the instruction to cancel has been input, the processing ends, and if the instruction to cancel is received. If there is no input, the process returns to S820.

In S840, the terminal processing unit 43 acquires the case identification tag T11 specifying the integrated information being reproduced, and generates the case identification tag T21.
In subsequent S850, the terminal processing unit 43 generates identification information for uniquely identifying the comment information generated based on the comment input, and generates a comment identification tag T22 using this identification information.

In subsequent S860, the terminal processing unit 43 acquires information identifying the person who entered the comment, and generates the person who entered the comment information tag T23. The information for identifying the comment input person may be input via the input unit 42, or may be a user ID or the like input by the user when the terminal device 4 is activated.

In subsequent S870, the terminal processing unit 43 identifies the type of the input comment from the input device or the like used for inputting the comment, and generates a comment type tag T27. Types of comments include character input via a keyboard, handwriting input via a touch panel, voice input via a microphone, image input via a camera, and the like.

In subsequent S880, the terminal processing unit 43 generates comment information Icm corresponding to the comment type. Specifically, when the comment type is character input or handwriting input, the input text data or image is used as the comment information Icm as it is. When the comment type is voice input, not only voice data but also voice recognition data generated by executing the voice recognition AP 44 is included in the comment information Icm. Similarly, when the comment type is image input, the comment information Icm includes not only image data but also motion analysis data generated by executing the motion analysis AP45.

In subsequent S890, the terminal processing unit 43 generates the tagged comment information T_Icm, adds it to the integrated information stored in the memory 432, and transmits it to the server 3, ending the process. The tagged comment information T_Icm includes the comment information Icm generated in S880, the time tags T25 and T26 generated in S740 and S760, the comment target tag T24 generated in S750 and S790 to S810, and the comment target tag T24 generated in S750 and S790 to S810. It is generated by inserting the case identification tag T21, comment identification tag T22, input person information tag T23, and comment type tag T27 generated in S870.

Note that the tagged comment information T_Icm stored in the memory 432 is immediately reflected in the display on the display unit 41 by the display updating process that is repeatedly executed.
Also, the tagged comment information T_Icm transmitted to the server 3 is stored in the database 31 . Further, when real-time distribution is in progress, this tagged comment information T_Icm is distributed from the server 3 to all the target terminals, and is also reflected in the display on the distributed target terminals.

In this process, S710 to S720 correspond to the target specifying section, S820 corresponds to the comment input section, and S740 to S800 and S840 to S890 correspond to the information generating section.
[5. Display example]
Here, a display example of the comment information Icm by display update processing will be described with reference to FIGS. 14 and 15. FIG.

When the measured value displayed on the monitor area A1 changes greatly, the position is input by touching the part on the screen where the measured value is displayed with a finger or the like, and then a comment is input via a keyboard or the like. Then, as indicated by symbol C1 in FIG. 14, a character comment "comment 1" associated with the parameter to be commented is displayed.

After the position input for designating the image area A2 is performed, a line drawing or characters are input by handwriting on the display screen using an input pen for a touch panel or the like. Then, as indicated by reference numeral C2 in FIG. 14, the handwritten line drawing and characters are superimposed and displayed as they are on the image in the image area A2 to be commented.

When an operation to specify the navigation image area A3 is performed while the MPR image is displayed in the navigation image area A3 in a three-view format, the three-dimensional position specified by the MPR image displayed in the navigation image area A3 is displayed. It is assumed that a position input specifying is performed. Then, it becomes possible to input a comment for the specified three-dimensional position. In this case, the three-dimensional position represented by each center position of the three-view drawing becomes a comment target, and the comment is displayed in the "rice 3" portion indicated by reference numeral C3 in FIG. Also, at another position in the three-view drawing, a comment is displayed in the "rice 4" portion indicated by symbol C4 in FIG. be. Note that the comment may be displayed in a part without an image in the navigation image area A3 instead of being superimposed on the image.

When all specification inputs are performed, for example, as shown in FIG. 15, a comment display area C5 is set at the right corner of the display screen, and comments are displayed in this comment display area C5 in association with the timeline TL. may Also, when the comment target is a comment, the comment may be displayed in an overlapping manner as indicated by reference numerals C51 and C52 in FIG. Also, in the display area C5, it may be possible to display a comment within a predetermined time range centering on a specified arbitrary point, independently of the time of the main screen where the device information is displayed. Further, by specifying a comment displayed in the display area C5 and performing a predetermined operation, the display of the main image may be switched to content related to the specified comment.

[6. effect]
According to the information sharing system 1 detailed above, the following effects are obtained.
(6a) When integrated information is distributed and displayed in real time during surgery or treatment, the status of surgery or treatment can be shared in real time with any person concerned via the terminal device 4 . Furthermore, comments can be left in real time as reminders of advice received during surgery, intentions of device operations and work performed during surgery, and things noticed during surgery.

For example, if the device information displayed on the display screen of the display unit 41 of the terminal device 4 includes a characteristic part that differs from other parts, the device information may be changed, such as medication, air conditioning setting change, and patient posture change. Information that cannot be read from the information can be left as a comment.

(6b) In addition to input that involves operating the device using a touch panel, keyboard, mouse, etc., it is possible to input comments through a microphone through a camera and non-contact comment input using gestures, making it impossible to operate the device. It is also possible to easily leave a comment of a certain practitioner or the like.

(6c) When the device information Idv and the comment information Icm are displayed after the surgery for the purpose of reviewing or educating the surgery or treatment, the viewer may find useful information that cannot be obtained from the device information Idv alone. Information can be obtained from the comment information Icm. At this time, new comments can also be added.

(6d) When entering a comment, individual device information Idv shown on the display screen can be specified, and the comment information Icm is displayed and reproduced in association with the specified device information Idv and time. be done. Therefore, the viewer can clearly grasp the target of the comment.

(6e) Since the entire screen or a specific comment can be selected as a comment target, by using this function, viewers can discuss with each other on the display screen and leave the content of the discussion as a comment. be able to.

(6f) Using the information of the case information tag T12 included in the list information to search for cases, or using the information of the input person information tag T23 included in the tagged comment information T_Icm, for example, a specific person It is possible to extract and display only the comments by. Therefore, necessary information can be efficiently confirmed.

[7. Other embodiments]
Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and various modifications can be made.

(7a) The provision processing unit 23, the server processing unit 32, the terminal processing unit 43, and their techniques described in this disclosure are programmed to perform one or more functions embodied by a computer program. It may also be implemented by a dedicated computer provided by configuring the processor and memory as described above. Alternatively, the provision processing unit 23, the server processing unit 32, the terminal processing unit 43, and their techniques described in the present disclosure can be performed by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits, may be implemented. Alternatively, the provision processing unit 23, the server processing unit 32, the terminal processing unit 43, and their techniques described in the present disclosure may be implemented by a processor and memory programmed to perform one or more functions and one or more It may also be implemented by one or more special purpose computers configured in combination with a processor configured by hardware logic circuitry. Computer programs may also be stored as computer-executable instructions on a computer-readable non-transitional tangible storage medium. The method of realizing the functions of the units included in the provision processing unit 23, the server processing unit 32, and the terminal processing unit 43 does not necessarily include software, and all of the functions are implemented by one or more It may be implemented using hardware.

(7b) A plurality of functions possessed by one component in the above embodiment may be realized by a plurality of components, or a function possessed by one component may be realized by a plurality of components. . Also, a plurality of functions possessed by a plurality of components may be realized by a single component, or a function realized by a plurality of components may be realized by a single component. Also, part of the configuration of the above embodiment may be omitted. Moreover, at least part of the configuration of the above embodiment may be added or replaced with respect to the configuration of the other above embodiment.

(7c) In addition to the information sharing system 1 described above, a program for causing a computer to function as the information providing device 2, the server 3, and the terminal device 4 constituting the information sharing system 1, a semiconductor memory storing this program, etc. The present disclosure can also be implemented in various forms such as a non-transitional tangible recording medium.

REFERENCE SIGNS LIST 1 information sharing system 2 information providing device 3 server 4 terminal device 5 communication network 21 device group 22A to 22H device 23 providing processing unit 24A to 24H provider 25 ... middleware 27 ... input section 31 ... database 32 ... server processing section 41 ... display section 42 ... input section 43 ... terminal processing section 231, 321, 431 ... CPU 232, 322, 432 ... memory, 251... Information generation function, 252... Clock function, 311... Device information area, 312... Comment information area, 313... List information area, 33... Distribution AP, 44... Voice recognition AP, 45... Motion analysis AP, 46... Display update AP, 47 -- Comment addition AP, 48 -- Layout change AP.

Claims (7)

Device information acquired from a plurality of medical devices (21) used for treatment of a patient is associated with first tag information including information representing the time at which the device information was acquired and the type of the device information. a plurality of tagged device information, and a plurality of tagged comment information in which second tag information including information for specifying the comment target is associated with the comment information targeting at least part of the device information as the comment target. a server (3) where is stored;
one or more terminal devices (4) connected to said server via a communication network;
with
The terminal device
a delivery request unit (43: S520) configured to request delivery of the tagged device information and the tagged comment information to the server;
The time at which the device information included in the first tag information of the tagged device information is obtained from the tagged device information about a plurality of medical devices obtained from the server in response to a request from the distribution request unit. A main image configured to extract one or more types of device information associated at the same time based on and to generate main image data, which is image data for displaying the extracted device information on one screen. a generation unit (43: S610 to S630);
According to the tagged comment information acquired from the server in response to a request from the delivery request unit, if the comment target is included in a main image that is an image displayed by the main image data, the comment target is corresponded to. a display generation unit (43: S610) configured to extract the comment information and generate comment display data, which is display data for displaying the extracted comment information in association with the comment target in the main image; ~ S620, S640) and
a target specifying unit (43: S710 to S720) configured to specify the comment target in the display image displayed by the main image data;
a comment input unit (43: S820) configured to input comment information that is information associated with the comment target specified by the target specifying unit;
By associating the comment information input by the comment input unit with the second tag information including information for specifying the comment target specified by the target specifying unit, the tagged comment information is generated. an information generation unit (43: S740 to S800, S840 to S890) configured to generate and store in the server;
Information sharing system with 2. The information sharing system according to claim 1, wherein one of the comment targets specified by the target specifying unit includes a comment displayed together with the main image by the comment display data.
Information sharing system. The information sharing system according to claim 1 or claim 2,
The terminal device
further comprising a comment narrowing unit (43: S570) configured to narrow down the comment information to be displayed using the information indicated in the second tag information,
The information sharing system, wherein the display generation unit extracts the comment information narrowed down by the comment narrowing unit and generates the comment display data. The information sharing system according to claim 3,
The second tag information includes input person information for specifying an input person of the comment information.
Information sharing system. The information sharing system according to any one of claims 1 to 4,
The terminal device
further comprising a layout specifying unit (43: S570) configured to select the device information to be displayed and to specify the arrangement of the device information in the display image;
The information sharing system, wherein the display generation unit generates comment display data for displaying the comment information regarding the device information selected as a display target by the layout designation unit. The information sharing system according to any one of claims 1 to 5,
The information sharing system, wherein the display generation unit generates the comment display data for displaying the comment information in association with a timeline. The information sharing system according to any one of claims 1 to 6,
further comprising one or more information providing devices that generate the tagged device information and provide it to the server via the communication network;
The information sharing system, wherein the first tag information includes information for identifying the information providing apparatus that has provided the tagged device information.

Images