JPH0787210A - Remote consulting system - Google Patents

Abstract

PURPOSE:To provide a consulting system capable of using multimedia information including video information as shared data and holding consultation between remote places as referring to the same output information mutually. CONSTITUTION:When an operating command is issued to share information, a share control part 32 judges whether or not its own station holds the share information, and displays by processing the information by its own station as shown in information flow a 42 when its own station holds the information, and when no share information exists in an opposite station, processed information is transmitted to the opposite station as shown in information flow 43b. When its own station holds no share information, the operating command is sent to the opposite station, and when the operating command or processed share information is received from another station, display processing as processing shown in 38b or information flow 44c is performed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a remote counseling system, and more particularly to a remote counseling system comprising a plurality of terminal devices connected via a broadband ISDN. The present invention relates to a remote consultation system that enables mutual consultations, meetings, meetings, etc. as shared output information of devices.

[0002]

2. Description of the Related Art Conventionally, as an interactive system used between remote places, a plurality of terminal devices are mutually connected as described in, for example, Japanese Unexamined Patent Publication No. 64-41359, "Inter-distance conversation system". When exchanging opinions while sharing the same information, send shared information with a large amount of data, such as image information, to the other party's storage device in advance so that it does not interfere with the conversation on the way. I remember it before I start a conversation (or meeting).

[0003]

In the above-mentioned prior art, since it is necessary to send information having a large amount of data to the other device in advance, the intended conference cannot be started quickly, or a new conference may be required during the conference. There was a problem that required information could not be shared.

Further, since each terminal employs a configuration in which information necessary for the conference is held in its own storage device, when trying to exchange opinions by referring to the same screen, the information from each storage device is exchanged. It is necessary to synchronize the read operations of the above, and particularly when the information to be shared is information that changes in time series such as a moving image or a video, it is difficult to perform accurate synchronization. This problem becomes particularly noticeable when the conference is conducted by communication between three or more parties.

Further, in the above-mentioned prior art, since the first goal is to reduce the amount of data exchanged between terminals during a conference, information having a large amount of data as shared information, for example, a high-definition still image, It is not suitable for applications in fields where video or video information is to be actively handled.

An object of the present invention is to provide a remote consultation system which can handle multimedia information including voice, data, still images and moving images and can exchange such information in real time.

Another object of the present invention is to provide a remote counseling system capable of promptly sharing data as necessary without having to previously send information shared between terminals to a partner device prior to a conference. Especially.

[0008] Another object of the present invention is to make use of multimedia information to enable close communication between remote terminals as if they were in close proximity to each other, which is particularly suitable for application in the medical field. To provide a consultation system.

[0009]

In order to achieve the above object, in a remote consultation system of the present invention, a plurality of terminal devices connected by a broadband ISDN own / own their own information.
Regardless of non-ownership, multimedia data including video information is treated as shared output material between a plurality of terminals, and the same information part as that of the counterpart device can be referred to at the same time.

More specifically, the remote consultation system according to the present invention comprises a plurality of terminal devices (hereinafter referred to as stations) connected via a broadband ISDN communication network.
It has an information storage means, an information output means, and a table means for indicating whether or not a plurality of types of information (hereinafter referred to as shared information) whose output is to be shared with another station is owned by the own station or another station. When there is an operation command from the user of the own station for any of the shared information, the station which executes the operation command and the transmission destination of the processing result are determined by referring to the table means,
The operation command to be executed by another station or the processing result executed by the own station is output to the communication network, and the processing result executed by the own station or the processing result received from another station is output to the output means. Is characterized by.

According to the present invention, when the shared material used for the conference is distributed to each station in advance by using a function such as e-mail or file transfer, the user can share the shared information. When an operation command is input, the operation command is transmitted to other stations via the communication network, and the program processing in response to the operation command is executed for each station to obtain the output of shared information linked with each other.

Further, when the shared information targeted for the operation command is owned only by a specific station, or due to the capacity of the information, the live image being photographed, or the ownership, it is possible to notify each station in advance. In the case of improper distribution, the source station sends the relevant information to other stations via the broadband ISDN,
By outputting the moving image information which is being transmitted or received by each station, the information output synchronized with each other is obtained.

[0013]

According to the present invention, when a plurality of terminal devices constituting the remote consultation system own the same information, the output information can be linked by communicating only the operation information, thus reducing the communication amount. it can.

Further, when the shared information is owned by only a specific terminal device, or when the shared information is generated on the specific terminal side such as the raw image information being photographed, this is handled by the own terminal. By outputting the information on the screen and transmitting it to another terminal in real time, the information interlocked with each other can be used.

According to the present invention, for example, a general practitioner presents a medical record and biochemical test data relating to the diagnosis of his / her patient to a specialist at a remote place, and a still image of a large-capacity radiograph or By sending a moving image of an endoscopic image taken by a gastric camera in real time, a general doctor and a specialist can interact while viewing the same data and images. In this case, by using pointer display and marking writing that are interlocked with each other on the respective terminal screens, it is possible to exchange appropriate instructions regarding indication of the affected area and treatment.

[0016]

FIG. 1 shows the overall configuration of a telemedicine consultation system which is an embodiment of the present invention. Here, two terminal devices (workstations) 1 and 2 are connected via a broadband ISDN 3, and voice information 4, numerical value / document information 5, control information 6, and moving image information 7 are transmitted from one to the other. Both workstations can display documents, videos, etc. having the same content.
Also at one workstation. When the user inputs an operation command for the shared resource, the content of the command is reflected on the other workstation, both application programs execute the same data processing operation, and the same display screen and other output devices are displayed. The output result can be obtained.

The workstation 1 has, for example, a screen 8
Is displayed, and the content displayed on the screen 9 is displayed on the workstation 2. The document 10 is displayed in one window of the screen 8, and the same content as this is displayed.
It is displayed as a document 11 on the screen 9. If the data of the above document is pre-distributed to the other station by e-mail or the like from the original station, the conference operation is started while both workstations have the same document data. Then, in the workstation 1, the data of the document 10 is read from the numerical value / document database 12 and displayed on the screen 8. Similarly, at the workstation 2, the data of the document 11 is
Data is read from the numerical value / document database 13 and displayed on the screen.

When a reading operation or a page turning operation is performed on the above-mentioned document data at any of the workstations,
This operation command is output as control information 6 on the broadband ISDN 3 and delivered to the partner station. As a result, the same operation command is executed on the two workstations, and the documents having the same contents are displayed on both workstation screens. That is, when the same information exists in both sides, the information itself is read locally at each station, but the operation information for reading is transmitted to the partner device via the network and both applications are linked. The information sharing process is realized by.

A moving picture 14 is displayed in another window of the screen 8, and the same content as that is displayed as a moving picture 15 on the screen 9. Here, it is assumed that the moving image information exists in the moving image database 16 of the workstation 1 and the workstation 2 does not have the corresponding moving image information. In this case, the workstation 1 reads the moving image information from the moving image database 16 and outputs it to the screen in response to the moving image reading command from the user.

When the above-mentioned command is given, the workstation 1 confirms whether the workstation 2 has the above-mentioned moving picture information, and if it is found that the other device does not have the same information, the work station 1 judges from the moving picture database 16. In parallel with the output operation of the read moving image information to the own device screen, the broadband IS
Send to partner workstation 2 via DN3. The workstation 2 outputs the moving image information received from the broadband ISDN 3 on the screen.

When the user on the workstation 2 side gives an operation command for reading, fast-forwarding, stopping, rewinding, etc. to the moving image, the control information 6 including the operation command is given.
Are transmitted to the workstation 1 via the broadband ISDN 3. In response to the command, the workstation 1 reads the information from the moving image database 16 and displays it, as if the user of the own station gave an operation instruction, and displays the same moving image information 7 in the broadband ISDN 3 Send to. That is, when the information exists only in one station, the station that owns the information performs the file access operation in response to the command, and outputs it to the output device of its own station.
This is sent to the other station via the communication network, and the station that does not have the information outputs the information received from the communication network to the output device so that both output devices can output the information linked with each other. I have to.

FIG. 2 shows the construction of the workstation. Reference numeral 21 is a processing device for controlling the entire workstation, and 22 is a broadband ISDN communication control unit for transmitting and receiving various kinds of information to and from the broadband ISDN. The processing device 21 controls selection, distribution, receipt, etc. of shared information based on a shared information management table described in detail later. That is, when the user issues an operation command for the shared information used in the conference (consultation), the processing device 21 checks the ownership state of the shared information that is the operation target by using the shared information management table. If the station owns the necessary information, the selection control unit 23 is operated to read the information from the information storage device 24, and the read information is sent to the display / output control unit 25 for display / output. Device 2
6 to display. Further, as a result of checking the shared information management table, when it is determined that the partner station does not own the above information, the read information is transferred to the data communication control unit 27 and the moving image communication control unit according to the type of information. 28 or via the voice communication control unit 29 to the broadband ISDN communication control unit 22 and transmit to the partner station.

As a result of checking the shared information management table, when it is determined that the necessary shared information is not owned by the own station, the operation command is transmitted to the partner station via the broadband ISDN communication control section 22 and the selection is made. Control unit 2
3, and the data communication control unit 2 according to the type of information
7. The video communication control unit 28 or the voice communication control unit 29 is selectively activated so that the information received via the broadband ISDN communication control 22 can be processed and displayed / outputted.

FIG. 3 shows the functional configuration of the workstation and the data flow. The workstation includes an application execution unit 31, a sharing control unit 32, a display / output unit 33, an input unit 34, an information storage unit 35, and a communication control unit 36.

When a user operates a key or a mouse, an operation instruction (operation command) a37 is issued from the input unit 34 to the shared control unit 32, and the shared control unit 32 sends it to the application execution unit 31 and the communication control unit 36. Tell. The operation instruction is given to the communication control unit 36 in order to convey the operation instruction to the partner station.

In response to the operation instruction, the application execution section 31 instructs the sharing control section 32 to read out a39.
Give out. The sharing control unit 32 checks whether or not the information to be read exists in the information storage unit 35 in the sharing information management table,
If it is owned, the read instruction b40 is given to the information storage unit 35. The information read from the information storage unit 35 is transferred from the sharing control unit 32 to the display / output unit 32 and displayed / outputted, as shown in the information flow a42. When the sharing control unit 32 determines that the partner station does not own the above information, the sharing control unit 32 gives the read information to the communication control unit 36 and causes the partner device to transmit the information, as shown in the information flow b43.

The shared control unit 32 gives a read instruction c41 to the communication control unit 36 when the shared information management table is checked and it is found that the necessary information is not in the information storage unit 35. In this case, as shown in the information flow c44, the information received from another station via the communication control unit 36 is sent to the display / output unit 32 via the sharing control unit 32, and displayed / outputted.

When the operation input made at the partner workstation is input through the communication network, the operation instruction b38 received from the communication control unit 36 is handled as if the shared control unit 32 inputs it from its own station. This is passed to the application execution unit 31. Upon receiving this, the application execution unit sends the read instruction a39 to the sharing control unit 32 as described above, and the operation according to ownership / non-owning of information is performed. When the information is read from the information storage unit 35, if the partner station does not own the information, it is sent from the communication control unit 36 to the partner device as described above.

FIG. 4 shows the structure of the shared information management table. The shared information management table is a shared information name field 45 that is the name of a resource used at a meeting (consultation, dialogue).
And a column 46 showing the state of ownership of the shared information in the own station,
It comprises a column 47 showing the ownership of other stations and a supply right column 48 showing whether or not the shared information should be supplied from the own station to the other station.

The own-owned-state column 46 of the own station comprises a field indicating the presence or absence of the shared information, a field indicating the version of the shared information when owned, and a field indicating the storage location of the shared information. . The ownership status column 47 of the other station is a field indicating whether or not the shared information exists in the other station, a field indicating the version of the shared information when the station is owned, and a field indicating the address of the owning station. Consists of.

By referring to the shared information management table, it is possible to determine from which station the shared information should be obtained during the conference. This table is
Records can be added and deleted at any time during a meeting. For example, when trying to use certain information as shared information in a meeting, the shared information management table is checked, and if it is not registered as shared information, Register the name, check if the information exists in the own station, and set the presence / absence, version, and storage location. Next, it is checked whether the other station has the shared information, and the presence / absence of the other station, the version, and the address of the owning station are set. If there is more than one other station as the partner, register the data related to the station having the new version information.

The supply right is given to the own station when the shared information is in the own station but not in the other station. If the station owns it or another station, the station that owns the new version of the information is given the supply right.

FIG. 5 shows the flow of processing performed by the sharing controller 32. Here, for simplification of explanation, it is assumed that any station (workstation) in the conference owns the shared information. When the shared information is processed in conjunction with another station, it is checked by referring to the shared information management table whether or not the own station has the shared information to be processed (step 51). If you own it, read the information from the information storage unit (step 5
2) The shared information management table is checked to see if the partner station has the same information (step 53). If not owned by the communication station, the communication control unit is instructed to transmit the shared information (step 54). Next, a command for displaying the read information is issued to the display device (step 56).

When it is determined in step 53 that the partner station owns the same information, the display instruction is issued in step 56 without issuing the shared information transmission command.

When it is determined in step 51 that the shared information is not owned by the own station, the communication control unit transmits an operation command to the other station and receives (reads) information from the other station. An instruction is given (step 55), and the received information is displayed on the display (step 56).

Next, an embodiment in which a conference is held among three or more stations will be described.

In a conference held by three or more remote parties, if the shared information to be operated is owned by only one of the stations, the information read out by the owning station is shared by other stations. It may be transmitted to all stations, but when the same information is owned by two or more stations, it is necessary to specify the information supply source to the non-owning station in one place.

Therefore, in the example shown in FIG. 6, step 58 is added to the control procedure shown in FIG.
In addition, this is solved by replacing step 54 with 58.

That is, when it is determined in step 57 that another station owns the same shared information, step 58
So, we will compete for information supply rights with other owned stations. In this case, the superiority or inferiority of the supply right is determined by the version of the information or the like as described in the shared information management table of FIG. When the own station obtains the information supply right, step 59
Then, using the multicast function of broadband ISDN,
Participating in the conference, the shared information processed by the operation command (for example, the shared information read from the database) is transmitted to all the other stations. When the information supply right has not been obtained, the reception of information from the communication control unit is awaited without processing the shared information of the own station, and the received information is output to the display.

Next, control for synchronizing shared information with another station will be described. When holding a conference while viewing continuous information having a time axis such as audio information and moving image information as shared information, it is necessary to synchronize the output operation of shared information at all stations. For example, when two stations that make up a conference own the same information, if each station independently reads the information from its own information source and outputs the information, a display scene or station There is a gap in the output time of the sound, and the synchronization is lost.

For such a problem, for example, a system is known in which control information for synchronization is exchanged between stations, and output control is performed in synchronization with the control information. In this system, audio information or moving image information is used. Is not effective for real-time continuous information such as In particular, in the case of a conference of three or more parties, synchronization becomes complicated.

Therefore, in the present invention, when an operation command is given to continuous information having a time axis, the station that processes shared information is limited to one location according to the processing procedure shown in FIG. A method of supplying processed information to all other stations is adopted. In a two-party conference, even if both stations have the same information, only one station accesses the information source and sends the processed information to the other station, and both stations have the same information. The continuous information is synchronized by outputting the information read from the information source.

Next, the display of the pointer on the output screen of the shared information and the writing of the marking will be described. The display of the shared information is performed under the control of the sharing control unit 32 shown in FIG. 3, regardless of whether the shared information is owned or not owned by the own station. When the user performs a pointer display operation or a marking writing operation, an operation instruction a37 is issued from the input unit 34 to the sharing control unit 32, and this is sent to the application execution unit 31 via the sharing control unit 32, and on the screen, The pointer display or marking writing output in response to the user operation is performed. In this case, the operation instruction a37 is
Since it is transmitted to the partner workstation via the communication control unit 36, the operation instruction b38 is also sent to the application execution unit via the communication control unit in the partner station as well, and the same pointer display or marking writing on the screen is performed. Output is done.

Next, a medical consultation system to which the present invention is applied will be described.

In the system configuration of FIG. 1, it is assumed that the workstations 1 and 2 are used by doctors who are remote from each other. Each workstation has a voice,
By utilizing the fact that data and images can be handled, a doctor who is a user can use, for example, voice, document data, numerical data such as heartbeat recording sounds, medical records, biochemical examination data, radiographs, MRI images, and endoscopic images. Assists in diagnosis or consults while mutually referencing multimedia medical information consisting of still images and videos.

Here, it is assumed that one of the two doctors to be discussed is a general physician and the other is a specialist. Conventionally, when a general physician makes a medical consultation with a specialist, the details of the medical data cannot be examined only by telephone, so that one has to go to the other to have a direct interview. Therefore, when a general doctor in a remote place seeks the opinion of a specialist, time loss becomes a big problem.

According to the consultation system of the present invention, the general physician transmits the medical record and biochemical test data of his / her patient to the workstation of the specialist at a remote place in advance, and the general consultation by the consultation system is performed. After the conversation between the doctor and the specialist is started, it is possible to explain the medical condition of the patient to the specialist by linking the medical chart and biochemical test data sent in advance on both workstation screens. This corresponds to the communication in the case where the both have the same information.

When a specialist requests, for example, the display of an X-ray photograph (still image) or an endoscopic image (moving image) taken by a gastric camera in order to better understand the medical condition, the necessary image information is generally Real-time transmission from a physician's workstation to a specialist's workstation via a broadband ISDN. At this time, the general physician can explain the image being output on the screen to the specialist through the telephone. This corresponds to the case where only one of the stations has the shared information and the other station does not have the shared information.

In the case of a moving image, the general practitioner can add a description while stopping or returning the image being displayed / transmitted as necessary. When these operations are performed, the images on the display screens of the workstations of the general doctor and the specialist change in conjunction with each other. In these dialogues, by appropriately using a pointer and marking writing, a specialist can point to the general physician, or vice versa, to explain the treatment while pointing out the affected area.

[0050]

As is apparent from the above description, according to the present invention, multimedia data including video information can be transmitted between terminal devices located in remote areas regardless of whether information is owned or not owned by the own terminal. You can consult or discuss while referring to as a shared material.

According to the present invention, when each terminal device owns the same shared material, it is possible to obtain information output that changes in conjunction with each other with a small communication amount for communicating only operation information. If you want to refer to the material owned by only a specific terminal device as shared information, by transmitting this to other terminal devices, even continuous information with a time axis such as audio and video will be synchronized on all terminals. It is possible to obtain information output that changes with

[Brief description of drawings]

FIG. 1 is a diagram showing an overall configuration of a consultation system according to the present invention.

FIG. 2 is a diagram showing a configuration of a workstation.

FIG. 3 is a diagram showing a functional configuration of a workstation and a data flow.

FIG. 4 is a diagram showing a configuration of a shared information management table.

FIG. 5 is a flowchart showing a processing procedure executed by a sharing control unit.

FIG. 6 is a flowchart showing a processing sequence of a shared control unit when including three or more stations.

[Explanation of symbols]

31 ... Application execution unit, 32 ... Sharing control unit, 3
3 ... Display / output unit, 34 ... Input unit, 35 ... Information storage unit,
36 ... Communication control unit, 37 ... Operation instruction a, 38 ... Operation instruction b, 39 ... Read instruction a, 40 ... Read instruction b, 4
1 ... Read instruction c, 42 ... Information flow a, 43 ... Information flow b, 44 ... Information flow c.

Claims (3)

[Claims] 1. A plurality of terminal devices (hereinafter referred to as stations) connected via a broadband ISDN communication network, each station comprising:
It has an information storage means, an information output means, and a table means for indicating whether or not a plurality of types of information (hereinafter referred to as shared information) whose output is to be shared with another station is owned by the own station or another station. When there is an operation command from the user of the own station for any of the shared information, the station which executes the operation command and the transmission destination of the processing result are determined by referring to the table means,
The operation command to be executed by another station or the processing result executed by the own station is output to the communication network, and the processing result executed by the own station or the processing result received from another station is output to the output means. Remote consultation system characterized by. 2. When one of the shared information includes moving image information and an operation command for the moving image information is received at any station, the station which owns or generates the moving image information transmits another information via the communication network. The moving picture information is transmitted to the stations, and the moving picture information which is being transmitted or received by each station is outputted to the output means to obtain the moving picture outputs synchronized with each other. Remote consultation system. 3. Each station is provided with a control means for determining whether or not the shared information which is the target of the operation command is owned by the own station by referring to the table means, and the user of the own station. When an operation command for the shared information is given from the control means, the control means stores the storage means if the own control station owns the corresponding shared information, and if not, the communication connected to the communication network. 3. The remote consultation system according to claim 1, wherein the control means is selected as an information source to execute the application program.