JP2005222431A - Cooperative work system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system for recording/filing, in a form allowing easy reuse, a character string (chat) by user input, and the other media such as a voice, video in synchronous communications such as a conference. <P>SOLUTION: Monitoring means 24, 25 monitor the input of the other media data. An association processing means 21 associates by inserting character string notations M1, M2... specifying the other media data, in a location in character string data by the user input corresponding to the inputted other media data at input time timing. An association filing means 22 records associated information on the character string data by the user input and the other media data. Furthermore, a display means 14 outputs the character string by the user input and the character string notations of the other media data in accordance with the associated information by using a recording file and by the control of a conference reproduction control means 16. The other media data corresponding to the character string notations are reproduced and outputted on condition that the picture of the character string notations is displayed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique for recording data composed of a plurality of discontinuous character strings in association with other media data input in the same work environment in which the character string data is input. For example, using network technology The present invention relates to a technique suitable for being applied to a system in which a plurality of users work together using terminals, such as a conference system.

With the development of computer technology and network communication technology, real-time communication among a plurality of users has been attempted using a system in which user terminals are connected to a network.
For such communication, in addition to voice communication using a microphone and a speaker connected to the user terminal, a chat that displays a character string input from the user terminal by a keyboard operation on the screen of each user terminal is also used. Yes.

In such a chat, when a character of each user participating in communication is displayed on the screen of each user terminal and a certain user inputs a specific character string such as “hahaha”, A technique for generating an event that causes the user's character to laugh is known (see Patent Document 1).
Also, a technique is known in which chat is performed using a three-dimensional browser of a user terminal, and when a specific command exists in the character string, the character string is three-dimensionalized (see Patent Document 2).

In addition to chatting with character strings using a network system that realizes similar remote synchronous communication, various media such as voice, images (moving images, still images), graphics, etc. can be used for meetings, lectures, A system for realizing a lecture or the like is also known, for example, a web conference system.
JP 2002-325965 A JP 2002-288101 A

  As described above, in synchronous communication, various media such as voice, video (video), graphics drawn on a whiteboard shared in a collaborative work environment, etc. are used in addition to a character string (chat) input by a user. However, when this collaborative work is recorded and filed as it is, there is a problem that it is complicated to use the recorded file later.

  Specifically, when a recorded file is read out and played back by a computer, the utterance recording by a character string input by the user is such that each character string is intermittent in time, and the contents of each character string are concise. Therefore, if the plurality of recorded character strings are scroll-displayed on the screen, the user can easily find the target character string. On the other hand, when you want to play a certain video displayed at a meeting, or when you want to play an image of a certain page of a document displayed at a meeting, If you want to replay the information that you want to replay, or if you want to replay a graphic drawn on the whiteboard during a meeting, you can find and replay the target part if you do not replay all the recorded files. Therefore, it becomes a complicated work that requires as much time as the recording of the conference.

The present invention has been made in view of the above-described conventional circumstances, and utilizes the property of character string data for a work environment using character string data input by a user and other media data such as voice and video. The purpose of this is to create a recording file that can be easily reused by associating these data.
In addition, the present invention also includes character string data input by the user (a plurality of intermittently continuous data) and other media data such as voice and video (for example, continuous data that is continuous in time and a document of a plurality of pages). Uses the nature of intermittent data for work environments that use data that is different from user-entered character string data, such as data that is displayed intermittently on a page-by-page basis. The purpose is to create a recording file that can be easily reused by associating continuous data.
Another object of the present invention is to facilitate the reuse of collaborative work records by creating such a record file.
Further objects of the present invention will be apparent from the following description.

The filing device according to the present invention includes a plurality of character string data input by a user (for example, text data such as a message, a comment, and a memo), and a character string input in the same work environment where the character string data is input. A device that records other media data of different types (for example, continuous data such as voice, image, application data, etc.) in association with each other, the monitoring means monitors the input of other media data, and the association means is input In addition, a character string notation that identifies the other media data is inserted at the position in the character string data of the user input corresponding to the input time timing and related to the other media data, and the filing means associates the character string data of the user input with the other Record related information with other media data.
Therefore, the other media data is displayed in the same character string as the chat at the position where the other media data is input in a plurality of character strings accumulated in intermittent time series every time the user input in the chat format. Is associated with the form.

Also, the playback device according to the present invention uses the recording file created as described above, and the display means displays a character string table of user input character strings and other media data according to the related information stored in the filing means. The reproduction means reproduces and outputs other media data corresponding to the character string display on the condition that the character string is displayed on the screen.
Therefore, for example, a character string table similar to the corresponding position in the chat depending on whether the character string notation is displayed on the screen automatically or when the user selects the character string display displayed on the screen. Is reproduced and displayed, and the user can discriminate at a glance what kind of media data is used at the position by the character string notation, and the media data can be reproduced and output.

  The present invention can be realized as a collaborative work system that provides an environment in which a plurality of users participate and perform some kind of communication such as conferences, lectures, lectures, etc., and monitoring means for monitoring the input of other media data, Associating means for associating the inputted other media data by inserting a character string notation that identifies the other media data at a position in the character string data of the user input corresponding to the input time timing, a character string of the user input A function called filing means for recording related information between data and other media data may be realized in a management device (management server) connected to each user terminal via a network. Also, the management device and the user terminal It is also possible to implement this in a user terminal without providing such a management device. It may be.

  Therefore, the present invention provides a work field including text data composed of character strings input by a user in time series and other media data of a different type from the text data input at the time of inputting the character strings to a plurality of users. It is also a filing method configured by connecting terminals to the network, and the management server connected on the network identifies other media data input at the corresponding position in the input time in the character string displayed on the screen of the user terminal This is a filing method that inserts a character string notation and records related information between user input character string data and other media data.

  The present invention also relates to a filing method for associating and recording a plurality of character string data input by a user and a character string input in the same work environment in which the character string data is input and other types of media data. The computer comprises a monitoring means, an associating means, and a filing means, the monitoring means monitors the input of other media data, and the other media data input by the associating means is detected at an input time timing. A character string notation that identifies the other media data is inserted and associated at a position in the corresponding character string data of the user input, and the filing unit records related information between the character string data of the user input and the other media data. It is a filing method.

  The present invention also comprises a filing means, a display means, and a reproducing means configured by a computer, wherein the filing means intermittently inputs a plurality of character string data and the same work environment in which the character string data is input. Related information that is associated with continuous media data that is input to the user by inserting a character string notation that identifies the continuous media data at a position in the user-input character string data corresponding to the continuous media data at the input time timing. The display means displays and outputs a user input character string and a character string notation that identifies other media data in accordance with the related information stored in the filing means, and the playback means is required to display the character string notation on the screen. In addition, this is a reproduction method for reproducing and outputting other media data corresponding to the character string display.

  According to the present invention, a character string displayed by a user input generated in collaborative work and other media data (typically continuous data) such as voice and video, and a character string display for specifying the other media data are displayed as user input characters. Since it is related by inserting it at the corresponding position in the row, when reusing the recording file of the joint work, other target media data can be easily found and reproduced and output. .

The present invention will be specifically described based on examples.
FIG. 1 shows the overall configuration of a conference system as a collaborative work system according to an embodiment of the present invention.
The conference system of this example has a system configuration in which a plurality of user terminals 1 and a management server 2 are connected by a network N. The user terminal 1 and the management server 2 are both computers that have a network communication function. It is comprised by hold | maintaining and executing the program for implementing the various functions which concern.

  With such a configuration, in the conference system, a collaborative work environment in which a plurality of users communicate with each other using the user terminal 1 is formed. That is, when a user inputs a text string in the text format from the user terminal 1 and speaks, the input character string is displayed on the screen of the user terminal 1 and also displayed on the other user terminals 1 in synchronization with the screen. In addition, other types of media data such as voice and video can be shared and input or output between the user terminals 1.

  The user terminal 1 includes an input operation unit 10 including a keyboard for a user to input a remark character string and an operation instruction, a camera 11 that captures a video image of a user who is participating using the user terminal 1, a user A microphone 12 that picks up the speech of a user who is participating using the terminal 1, a speaker 13 that outputs the voice of another user who has spoken in a conference (joint work environment), a speech string input by the user, A display unit 14 having a display for displaying and outputting other media data such as audio and images input in the conference environment is provided.

Further, the user terminal 1 manages character strings and other media data input from the user terminal 1 in order to realize a collaborative work environment in cooperation with other user terminals 1 under control management by the management server 2. Conference control means for transmitting to the server 2 and receiving a character string or other media data input from another user terminal 1 from the management server 2 and realizing communication input / output synchronized with the other user terminal 1 15, and constitutes a user interface to the conference feel as a collaborative work environment.
The user terminal 1 also functions as an apparatus for reproducing and outputting a recording file of the conference held in the management server 2. The user terminal 1 receives data of the recording file from the management server 2, and outputs sound from the speaker 13 and display means 14. Is provided with a meeting reproduction control means 16 that realizes a meeting reproduction output by a screen display output by the above, and constitutes a user interface for reproducing a meeting recording file held in the management server 2.

  The management server 2 performs a data communication with each user terminal 1 and controls a synchronized output among the terminals, thereby controlling a conference field shared by a plurality of participating user terminals 1, An association processing means 21 for associating and processing user input text character string data input to the conference field with other media data, and a recording file including at least association information between the associated text character string data and other media data The association file means 22 to be held, and the reproduction output means 23 for providing the data held in the association file means to the user terminal 1 in response to the reproduction request from the user terminal 1 are provided. A management apparatus for creating and holding a conference recording file is configured.

Here, in particular, the association processing means 21 is different from the text input monitoring means 24 for monitoring the text string input to the conference field by the user input from the user terminal 1 and the text such as voice or video image to the conference field. And other media monitoring means 25 for monitoring other types of media data input, and as shown in FIGS. 4 to 6, the user input characters corresponding to the input other media data at the input time timing A process of inserting and relating a character string notation specifying the other media data at a position in the column data is performed.
That is, the association processing means 21 determines and specifies the type of the media data based on the attribute information added to each media data, generates a character string notation based on the type, and handles the correspondence in the text character string. Insert it at the position you want.

  For example, in the case of a video image, a character string “Video” indicating the type and a character string “10 mim” indicating the time length of the video are used as other text input by the user as the character string notation specifying other media data. In the case of speech, a character string “Sound” representing a type and a character string “5 mim” representing a time length of the speech are used. Is a character string notation <Sound, 5 mim> surrounded by a distinguishing symbol <>, and in the case of a graphic drawn on a whiteboard, a character string “WhiteBoard” indicating a type and a character string indicating stroke data for drawing the graphic A document notation such as PDF format is set as a character string notation <WhiteBoard, source = AA> in which “source = AA” is surrounded by a distinction symbol <>. In the case of a file image, a character string notation <Image, file = XX, in which a character string “Image” representing a type and a character string “file = XX, YYpage” representing the document material or its page are surrounded by a distinguishing symbol <>. YYpage>.

Since this character string notation is the same text data as the chat character string, it is displayed on the screen together with the chat character string by being inserted during the chat. In addition, the character string notation is also a link notation to the corresponding media data (actual data). Upon reproduction, the character string notation is displayed on the screen or the user selects and instructs the character string notation displayed on the screen. As a result, the necessary media data application is activated and the associated media data is reproduced and output on the user terminal 1.
These character string representations may be generated each time, but the association processing means 21 refers to a conversion table prepared in advance and extracts it from the table and inserts it into the text character string chat. It may be.

As shown in FIG. 3, for example, associating files that are associated and stored in the associating file means 22 include text data (chat message character strings) and other media data (voice data, image data, graphics). Data, etc.) is a recording file for each conference, and the association is structured in detail later.
The recording file for each meeting may be associated with text data or other media data actual data, or may be associated with only an identifier for identifying these actual data. In this case, actual data may be acquired and output based on the identifier during reproduction.

FIG. 2 shows a screen display example of the display means 14 constituting the user interface of each user terminal 1. In the screen 14, a message input display unit 14a, a message display unit 14b, a participant image unit 14c, and a shared material unit 14d are provided as display areas on the same screen.
When the user inputs a character string (message) as an utterance from the keyboard 10 of the input operation means, the character string corresponding to the input operation is displayed on the message input display unit 14a and displayed on the message input display unit 14a. When the user performs a determination key operation, the character string displayed on the message input display unit 14a is added to the user name and displayed on the message display unit 14b.

  That is, for example, when a character string message “Today's agenda is” is entered in the message input display section 14a which is an input work area, the input character string is determined by the conference control unit 15 by the conference control unit 20 of the management server 2. All the user terminals participating in the conference are sent to each user terminal 1 as a message of the form “A) Today's agenda” added with the user name A by the conference control means 20 1 is displayed on the message display section 14b. Therefore, the character string message input from each user terminal 1 is sequentially displayed on the message display units 14b of all the user terminals 1 in a format with the respective user names, and is displayed on the message display units 14b of all the user terminals 1. A chat with a text message between participant users is displayed. These character string messages are time-series data arranged in the order of user input, but the temporal arrangement is not continuous but intermittent.

In addition, the user image taken from the camera 11 is transmitted to the conference control unit 20 of the management server 2 by the conference control unit 15 and distributed to the other user terminals 1 by the conference control unit 20 to participate in the conference. It is displayed on the participant image part 14c of all the other user terminals 1. For example, when three users A, B, and C are participating in the conference, images of other users B and C are displayed in the participant image portion 14c of the user A's terminal.
The shared material part 14d is an area for displaying an image of media data of a different type from the character string message shared in the conference, such as a video image, a drawing figure, and a document material image input in the conference field. Shared material data or shared material data acquired by the user terminal 1 from a file server (not shown) on the network is transmitted to the conference control unit 20 of the management server 2 by the conference control unit 15, and the conference control unit 20 It is distributed to other user terminals 1 and the shared material is displayed as an image in the shared material part 14d of all user terminals 1. Note that media data different from text messages such as video images, drawing figures, and voices can be said to be continuous data that is temporally continuous.

The voice input from the microphone 12 of the user terminal 1 is transmitted to the conference control unit 20 of the management server 2 by the conference control unit 15 and distributed to the other user terminals 1 by the conference control unit 20 to participate in the conference. Are output from the speakers 13 of all other user terminals 1.
Therefore, under the control of the management server 2, the character string, image, and sound input in the conference field are output synchronously at all user terminals 1 participating in the conference.

FIG. 4 shows an example of an association mode of the shared material or voice and the character string message by the association processing means 21, and FIG. 5 shows another example of the association mode.
As shown in FIG. 4, a plurality of character string messages a to l... Displayed on the message display unit 14b are recorded in an intermittent time series in the order of user input. When the voice is input from the microphone 12 to the conference field at the time timing between the character string message b and the character string message b, the character string notation M1 specifying the voice data is inserted between the character string message a and the character string message b. Then, the voice data is linked to the character string notation M1. Further, for example, when a video image is input to the shared material part 14d at a time timing between the character string message g and the character string message h, a character string notation M2 for specifying the video moving image data is displayed in the character string message g. And the character string message h, and the video moving image data is linked to the character string notation M2.

In the example shown in FIG. 4, continuous data such as video and audio without interruption in time is started from the position (that is, the insertion position of character notation) in the corresponding character string message at the time timing when they are input. However, as shown in FIG. 5, such continuous data may be divided into several pieces and associated with the character string message.
For example, when video moving image data having a certain length is input, the association processing unit 21 divides the video moving image data into a plurality of video data portions a-1 and a-2 at regular time intervals. Character string notations M3 and M4 are respectively inserted at positions in the character string message corresponding to the heads of the video data parts a-1 and a-2, and the respective video data parts a-1 and a-2 are respectively inserted. Are linked to the character string notations M3 and M4.
In this way, if continuous data is divided and association is performed for each part, even if continuous data extends for a long time, the necessary part in the continuous data is determined based on the character string notation inserted in the character string message. Can be easily found.

FIG. 6 shows a screen display example of the character string message (chat) associated as described above. The chat display in such a mode is performed when the user terminal 1 controls conference playback during playback of a recorded file. The recording file is obtained from the association file unit 22 of the management server 2 by the unit 16 and displayed on the display unit 14 for output.
In the illustrated example, a character string table that identifies other media data input in the conference field as described above during recording of each person speaking in a character string message in a conference by three users A, B, and C. Is displayed surrounded by special characters <>. These character string representations include descriptions for the user to imagine the media data, such as the type, length, page, etc. of the corresponding media data. Therefore, based on the character string message and the character string notation, the user is asked about where the media data was input (used) during the meeting, what the media data was. The above-mentioned information can be easily obtained, and necessary media data (and necessary portions thereof) can be quickly found and reproduced and output in the reproduction operation.

FIG. 7 shows an outline of the association recording process of the character string message and other media data mainly performed by the association processing unit 21 of the management server 2.
When data is input from the user terminal 1 to the conference field, it is determined whether or not the input data is a character string message (step S1). It transmits to the user terminal 1 and displays it on the message display part 14b (step S2). On the other hand, if the input data is not a character string message, it is determined whether or not it is other media data as described above (step S3), and a text character string notation that specifies the other input media data is generated. (Step S4), an association process for inserting and linking the character string notation at a position corresponding to the input timing during the chat is performed (Step S5).
When the conference field is active, the input data is constantly monitored, and the associated data (at least related information for specifying the related relationship) is stored in the related file means 22 as a recording file of the conference.

FIG. 8 shows an outline of a reproduction process based on a conference recording file mainly performed by the conference reproduction control means 16 of the user terminal.
Based on an instruction from the input operation unit 10 by the user, the conference reproduction control unit 16 obtains a recording file of the target conference from the association file unit 22 of the management server 2 and displays on the display screen of the display unit 14 as shown in FIG. A character string message chat is displayed (step S11). In general, most chat recordings such as conferences take a long time, and the entire chat cannot be displayed on the screen at one time. Therefore, the user can instruct the input operation unit 10 to scroll the screen display and view the chat. Will be. When a character string notation is present in the portion displayed on the screen by such a scroll operation (step S12), the conference reproduction control means 16 starts an application necessary for reproducing the media data specified by the character string notation. (Step S13), the media data is reproduced and output (Step S14). For example, in the case of audio data, it is reproduced and output from the speaker 13, and in the case of moving image or still image data or drawing graphic data, a display window is formed on the screen 14 to reproduce and display the image.

Here, in the above reproduction process, when the character string notation is displayed on the screen, other corresponding media data is automatically reproduced and output. In the present invention, however, the user selects the character string notation displayed on the screen. Other media data corresponding to the instruction may be reproduced and output.
In other words, when a selection instruction is input from the input operation unit 10 by the user to the character string notation in the chat displayed on the screen, the conference reproduction control means 16 reproduces the media data specified by the character string notation. An application necessary for processing is activated, and the media data is reproduced and output.

It is a figure which shows the structure of the conference system which concerns on one Example of this invention. It is a figure which shows the example of a screen display of the user terminal which concerns on one Example of this invention. It is a figure explaining the association file which concerns on one Example of this invention. It is a figure explaining the association structure which concerns on one Example of this invention. It is a figure explaining the association structure which concerns on one Example of this invention. It is a figure which shows the example of a screen display of the character string message containing the character string description concerning one Example of this invention. It is a figure explaining the association recording process which concerns on one Example of this invention. It is a figure explaining the reproduction | regeneration processing based on one Example of this invention.

Explanation of symbols

1: user terminal, 2: management server,
10: input operation means, 11: camera,
12: Microphone, 14: Display means,
14a: message input display section, 14b: message display section,
14c: Participant image part, 14d: Shared document part,
15: Conference control means, 16: Conference playback control means,
20: Conference control means, 21: Association processing means,
22: Association file means 24: Text input monitoring means,
25: Other media input monitoring means, M1, M2, M3, M4: character string notation,

Claims (9)

A filing device that records a plurality of character string data input by a user and a character string input to the same work environment where the character string data is input and other types of media data in association with each other.
A monitoring means for monitoring the input of the other media data, and a character string notation that identifies the other media data at a position in the character string data of the user input corresponding to the input other media data at an input time timing A filing apparatus comprising: association means for inserting and associating; and filing means for recording related information between user-input character string data and other media data. The filing device according to claim 1, wherein
The plurality of character string data input by the user is a plurality of data sets in which the character string data is intermittent in time,
The filing apparatus according to claim 1, wherein the other media data is continuous data that is continuous in time. A plurality of character string data intermittently input by a user and continuous media data input in the same work environment in which the character string data was input are converted to user input corresponding to continuous media data at input time timing. Filing means for storing related information by inserting a character string notation specifying the continuous media data at a position in the character string data; and
Display means for displaying and outputting a user input character string and other character string notation specifying other media data according to the related information stored in the filing means;
A playback device comprising playback means for playing back and outputting other media data corresponding to the character string display when the character string notation is displayed on the screen. A plurality of character string data intermittently input by a user and continuous media data input in the same work environment in which the character string data was input are converted to user input corresponding to continuous media data at input time timing. Filing means for storing related information by inserting a character string notation specifying the continuous media data at a position in the character string data; and
Display means for displaying and outputting a user input character string and other character string notation specifying other media data according to the related information stored in the filing means;
A playback device comprising: playback means for playing back and outputting other media data corresponding to the character string display in response to the user selecting the character string display displayed on the screen. Character string data generated in the joint work by connecting multiple user terminals over the network to form a collaborative work environment, sharing the character strings input from the user terminals among the user terminals and displaying them in time series A collaborative work system for sharing and outputting other media data of different types between user terminals,
A monitoring means for monitoring the input of the other media data, and a character string notation that identifies the other media data at a position in the character string data of the user input corresponding to the input other media data at an input time timing A collaborative work system comprising: association means for associating by inserting a file; and filing means for recording related information between character string data input by a user and other media data. The collaborative work system according to claim 5,
Display means for displaying and outputting a user input character string and other character string notation specifying other media data according to the related information stored in the filing means;
Other media data corresponding to the character string display is reproduced and output according to at least one of the conditions that the character string notation is displayed on the screen or the user selects the character string display displayed on the screen. And a reproducing means. A work field including text data composed of character strings input by a user in time series and other media data of a different type from the text data input at the time of inputting the character strings is connected to a plurality of user terminals over a network. Configure
The management server connected to the network inserts a character string notation that identifies other media data input at the corresponding position in the character string displayed on the screen of the user terminal, and the user input character string A filing method characterized by recording information related to data and other media data. A filing method for recording a plurality of character string data input by a user and a character string input in the same work environment in which the character string data is input and other types of media data in association with each other.
The computer constitutes monitoring means, associating means, and filing means,
The monitoring means monitors the input of the other media data,
The associating means inserts and associates the inputted other media data by inserting a character string notation that identifies the other media data at a position in the character string data of the user input corresponding to the input time timing,
A filing method characterized in that filing means records related information between user-input character string data and other media data. The computer constitutes filing means, display means, and playback means,
The filing means supports a plurality of character string data input intermittently by the user and continuous media data input to the same work environment in which the character string data is input, with continuous media data at the input time timing. Storing the related information associated by inserting a character string notation that identifies the continuous media data at a position in the character string data of the user input
The display means displays and outputs a user input character string and a character string notation that identifies other media data in accordance with the related information stored in the filing means,
A reproducing method, wherein the reproducing means reproduces and outputs other media data corresponding to the character string display on the condition that the character string notation is displayed on the screen.

Images