WO2022224584A1 - 情報処理装置、情報処理方法、端末装置及び表示方法 - Google Patents
情報処理装置、情報処理方法、端末装置及び表示方法 Download PDFInfo
- Publication number
- WO2022224584A1 WO2022224584A1 PCT/JP2022/007572 JP2022007572W WO2022224584A1 WO 2022224584 A1 WO2022224584 A1 WO 2022224584A1 JP 2022007572 W JP2022007572 W JP 2022007572W WO 2022224584 A1 WO2022224584 A1 WO 2022224584A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- group
- groups
- unit
- terminal device
- Prior art date
Links
- 230000010365 information processing Effects 0.000 title claims abstract description 99
- 238000000034 method Methods 0.000 title claims description 62
- 238000003672 processing method Methods 0.000 title claims description 6
- 238000012545 processing Methods 0.000 claims description 146
- 230000008569 process Effects 0.000 claims description 32
- 239000000284 extract Substances 0.000 claims description 31
- 238000000605 extraction Methods 0.000 claims description 23
- 238000003860 storage Methods 0.000 description 68
- 230000006870 function Effects 0.000 description 38
- 238000010586 diagram Methods 0.000 description 37
- 230000005540 biological transmission Effects 0.000 description 31
- 238000004891 communication Methods 0.000 description 24
- 230000008859 change Effects 0.000 description 11
- 238000009826 distribution Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 8
- 229960005486 vaccine Drugs 0.000 description 7
- 230000008901 benefit Effects 0.000 description 6
- 239000000463 material Substances 0.000 description 6
- 238000004458 analytical method Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 208000015181 infectious disease Diseases 0.000 description 5
- 239000013598 vector Substances 0.000 description 5
- 101000679365 Homo sapiens Putative tyrosine-protein phosphatase TPTE Proteins 0.000 description 4
- 102100022578 Putative tyrosine-protein phosphatase TPTE Human genes 0.000 description 4
- 230000033001 locomotion Effects 0.000 description 4
- 230000015654 memory Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 101000821981 Homo sapiens Sarcoma antigen 1 Proteins 0.000 description 3
- 102100021466 Sarcoma antigen 1 Human genes 0.000 description 3
- 230000009471 action Effects 0.000 description 3
- 238000007796 conventional method Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 239000004065 semiconductor Substances 0.000 description 3
- 241000711573 Coronaviridae Species 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005401 electroluminescence Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 241000605059 Bacteroidetes Species 0.000 description 1
- 101000614439 Homo sapiens Keratin, type I cytoskeletal 15 Proteins 0.000 description 1
- 102100040443 Keratin, type I cytoskeletal 15 Human genes 0.000 description 1
- JAWMENYCRQKKJY-UHFFFAOYSA-N [3-(2,4,6,7-tetrahydrotriazolo[4,5-c]pyridin-5-ylmethyl)-1-oxa-2,8-diazaspiro[4.5]dec-2-en-8-yl]-[2-[[3-(trifluoromethoxy)phenyl]methylamino]pyrimidin-5-yl]methanone Chemical compound N1N=NC=2CN(CCC=21)CC1=NOC2(C1)CCN(CC2)C(=O)C=1C=NC(=NC=1)NCC1=CC(=CC=C1)OC(F)(F)F JAWMENYCRQKKJY-UHFFFAOYSA-N 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000005674 electromagnetic induction Effects 0.000 description 1
- 230000008921 facial expression Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 238000003892 spreading Methods 0.000 description 1
- 238000010897 surface acoustic wave method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
- H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
- G06F40/35—Discourse or dialogue representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/16—Arrangements for providing special services to substations
- H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
- H04L12/1813—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for computer conferences, e.g. chat rooms
- H04L12/1831—Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/21—Monitoring or handling of messages
- H04L51/216—Handling conversation history, e.g. grouping of messages in sessions or threads
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
- H04N7/155—Conference systems involving storage of or access to video conference sessions
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/16—Arrangements for providing special services to substations
- H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
- H04L12/1813—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for computer conferences, e.g. chat rooms
- H04L12/1822—Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
Definitions
- the present disclosure relates to an information processing device, an information processing method, a terminal device, and a display method.
- Non-Patent Document 1 a technique of dividing users participating in such a remote meeting into a plurality of groups.
- the present disclosure proposes an information processing device, an information processing method, a terminal device, and a display method that can efficiently confirm information about a plurality of groups.
- an information processing apparatus divides a plurality of users participating in a remote meeting, and among a plurality of groups in which each group can have a conversation, at least one or more an acquisition unit that acquires group content information related to the content of a conversation in the group, and an output unit that outputs, together with the plurality of groups, a plurality of group list information that displays the group content information in association with the corresponding group.
- FIG. 3 is a diagram showing an example of information processing according to an embodiment of the present disclosure
- FIG. FIG. 4 is a diagram showing an example of a display of the entire remote lecture system
- FIG. 10 is a diagram showing an example of display for an administrator
- 1 is a diagram showing a configuration example of a remote lecture system according to an embodiment of the present disclosure
- FIG. 2 is a diagram illustrating a configuration example of a remote meeting server according to an embodiment of the present disclosure
- FIG. 4 is a diagram illustrating an example of a group information storage unit according to an embodiment of the present disclosure
- FIG. It is a figure showing an example of a history information storage part concerning an embodiment of this indication.
- FIG. 1 is a diagram illustrating a configuration example of a terminal device according to an embodiment of the present disclosure
- FIG. 4 is a flow chart showing a processing procedure of the information processing device according to the embodiment of the present disclosure
- 4 is a flow chart showing a processing procedure for group work
- It is a figure which shows an example of a display by a distribution graph. It is a figure which shows an example of a display by frequency time series. It is a figure which shows an example of a display by a radar chart. It is a figure which shows an example of a display by a line graph. It is a figure which shows an example of a display by a pie chart. It is a figure which shows an example of a display by a bar graph.
- FIG. 4 is a flow chart showing a processing procedure of the information processing device according to the embodiment of the present disclosure
- 4 is a flow chart showing a processing procedure for group work
- It is a figure which shows an example of a display by a distribution graph. It is a
- FIG. 5 is a diagram showing an example of multiple group list information and assist information; It is a figure which shows an example of provision of the comment using an assist. It is a figure which shows another example of the content of assistance. It is a figure which shows an example of the content of assistance. It is a figure which shows an example of a keyword and summary content.
- FIG. 10 is a diagram showing an example of the flow of settings for group work; FIG. 10 is a diagram showing an example of input of an operation based on a line of sight; FIG. 10 is a diagram showing an example of input of an operation based on a line of sight; 1 is a diagram showing an example of components of a remote lecture system; FIG. 1 is a hardware configuration diagram showing an example of a computer that implements functions of an information processing apparatus; FIG.
- Embodiment 1-1 Outline of information processing according to embodiment of present disclosure 1-1-1. Display example of the entire remote lecture system 1-1-2. Display example of terminal device 1-1-3. Background and Effects 1-2. Configuration of remote lecture system according to embodiment 1-3. Configuration of Information Processing Apparatus According to Embodiment 1-4. Configuration of terminal device according to embodiment 1-5. Information processing procedure according to embodiment 1-5-1. Procedure of processing related to information processing apparatus 1-5-2. Procedures for Processing Related to Groupwork 1-6. Display example 1-6-1. Distribution graph 1-6-2. Frequency time series 1-6-3. Radar chart 1-6-4. Line graph 1-6-5. Pie chart 1-6-6. Bar graph 1-7. Processing example 1-7-1.
- Example of Assist and Comment 1-7-2 Data example 1-7-3. Group work setting example 1-8. Operation example (line of sight) 1-9. System example 2. Other Embodiments 2-1. Other configuration examples 2-2. Others 3. Effects of the present disclosure 4 . Hardware configuration
- FIG. 1 is a diagram illustrating an example of information processing according to an embodiment of the present disclosure.
- Information processing according to the embodiment of the present disclosure is realized by the remote lecture system 1 including the remote meeting server 100 , the terminal device 10 and the plurality of member terminals 20 .
- each member terminal 20 may be described as a member terminal 20a, a member terminal 20b, and a member terminal 20c in order to distinguish and describe each member terminal 20.
- FIG. Note that the member terminal 20a, the member terminal 20b, the member terminal 20c, and the like are referred to as the "member terminal 20" when they are not distinguished from each other.
- three member terminals 20 are illustrated in FIG. 1, there may be four or more member terminals 20 .
- the remote lecture system 1 has a function of dividing users (members) participating in a remote meeting (hereinafter also referred to as "meeting") into a plurality of groups and performing group work. As described above, in the remote lecture system 1, a plurality of users are assigned to each group, and group work such as discussion (conversation) and production can be performed among the members belonging to each group.
- FIG. 1 shows a case where a plurality of users participating in a meeting are divided into a plurality of groups such as groups A to D and are doing group work.
- a meeting host such as a lecturer or a facilitator divides a plurality of users participating in the meeting into a plurality of groups and sets group work.
- the lecturer FC will be described as an example of the administrator.
- the lecturer FC who is the lecturer of the meeting, uses the terminal device 10 to divide a plurality of members A-1 to D-4 into a plurality of groups A to D and set group work. A setting example related to group work will be described later.
- step numbers such as S11-1 shown in FIG. is executed as needed.
- FIG. 1 three members A-1, A-2, and A-3 are illustrated for explanation. users) includes a large number of users (for example, 100 or 1000).
- the member terminal 20 used by member A-1 is member terminal 20a
- the member terminal 20 used by member A-2 is member terminal 20b
- the member terminal 20 used by member A-3 is member terminal 20c. described as.
- An application for participating in a remote meeting provided by the remote lecture system 1 (hereinafter also referred to as a "remote meeting application") is installed in each member terminal 20, and each user participates in the remote meeting using the remote meeting application. participate.
- a plurality of members A-1 to D-4, etc. are divided into a plurality of groups A to D, and each group performs work related to group work such as discussion (conversation) within the group.
- group A members A-1, A-2, A-3, and A-4 have conversations based on topics related to the Olympics, for example.
- members B-1, B-2, B-3, and B-4 have conversations based on topics related to the Olympics, for example.
- Groups C and D also have conversations among the members belonging to the groups.
- Each member terminal 20 transmits/receives information about the remote meeting to/from the remote meeting server 100 at any time.
- Each member terminal 20 transmits to the remote meeting server 100 information such as utterances and images of the user using the member terminal 20, and receives information to be output about the remote meeting from the remote meeting server 100.
- step S11-1 communication (transmission/reception) of information relating to the remote meeting input/output at the member terminal 20a is performed (step S11-1).
- the member terminal 20a transmits information such as member A-1's speech and images to the remote meeting server 100, and receives information to be output about the remote meeting from the remote meeting server 100.
- FIG. The member terminal 20a displays the intra-group screen CT11 (see FIG. 2) corresponding to the group A to which the member A-1 belongs, and outputs the utterances of the other members in the group A by voice. In this way, each member terminal 20 displays information about the group to which the member belongs and outputs audio.
- step S11-22 communication (transmission/reception) of information relating to the remote meeting input/output at the member terminal 20b is performed (step S11-2).
- the member terminal 20b transmits information such as member A-2's speech and images to the remote meeting server 100, and receives information to be output about the remote meeting from the remote meeting server 100.
- FIG. The member terminal 20b displays the intra-group screen corresponding to the group A to which the member A-2 belongs, and outputs the utterances of the other members in the group A by voice.
- step S11-3 communication (transmission and reception) of information relating to the remote meeting input/output at the member terminal 20c is performed (step S11-3).
- the member terminal 20c transmits information such as member A-3's speech and images to the remote meeting server 100, and receives information to be output about the remote meeting from the remote meeting server 100.
- FIG. The member terminal 20b displays the intra-group screen corresponding to the group A to which the member A-3 belongs, and outputs the utterances of the other members in the group A by voice.
- the member terminals 20 used by members A-4, B-1 to B-4, C-1 to C-4, and D-1 to D-4 also execute similar processing. Illustration is omitted in FIG.
- the member terminal 20 used by the member B-1 displays the intra-group screen CT21 (see FIG. 2) corresponding to the group B to which the member B-1 belongs, and outputs the utterances of the other members in the group B by voice.
- the member terminal 20 used by the member D-1 displays the intra-group screen CT41 (see FIG. 2) corresponding to the group B to which the member D-1 belongs, and voices the utterances of the other members in the group D. Output.
- Step S11 is a step of communicating (transmitting/receiving) information about a remote meeting such as group work between the remote meeting server 100 and each member terminal 20.
- FIG. Steps S11-1 to S11-3 are not limited to being executed once, but are executed as needed according to the progress of a remote meeting such as group work.
- the remote meeting server 100 collects information received from each member terminal 20, and executes processing for generating multiple group list information that displays group content information about the content of conversation in each group in association with the corresponding group. (Step S12).
- remote meeting server 100 generates group content information SM1 of group A based on the speech histories of members A-1, A-2, A-3, and A-4.
- the group content information SM1 may include information such as the number of appearances of each keyword in conversations in group A, summary contents of conversations in group A, and the like, the details of which will be described later.
- the remote meeting server 100 generates group content information SM2 of group B based on the speech histories of members B-1, B-2, B-3, and B-4.
- Remote meeting server 100 generates group content information SM3 of group C based on the speech histories of members C-1, C-2, C-3, and C-4.
- Remote meeting server 100 generates group content information SM4 of group D based on the speech histories of members D-1, D-2, D-3, and D-4.
- the remote meeting server 100 generates multiple group list information CT1 in which each of the group content information SM1 to SM4 is associated with the corresponding groups AD.
- remote meeting server 100 associates member lists ML1 to ML4 of groups A to D with group content information SM1 to SM4, respectively, and generates multiple group list information CT1 arranged in areas AR1 to AR4. .
- the remote meeting server 100 transmits the multiple group list information CT1 to the terminal device 10 (step S13).
- the terminal device 10 that has received the multiple group list information CT1 from the remote meeting server 100 displays the multiple group list information CT1 (step S14).
- a member list ML1 indicating the members of group A and group content information SM1 are arranged in association with an area AR1.
- the member list ML1 is arranged in the left area (also referred to as "first area") in the area AR1 of the multiple group list information CT1, and the right area in the area AR1 of the multiple group list information CT1 (" Group content information SM1 is arranged in the second area. In this way, group content information SM1 of group A is displayed in association with member list ML1 of group A in multiple group list information CT1.
- the member list ML21 is arranged in the first area on the left side in the area AR2 of the multiple group list information CT1, and the group content information SM2 is arranged in the second area on the right side in the area AR2 of the multiple group list information CT1.
- group content information SM2 of group B is displayed in association with member list ML2 of group B in multiple group list information CT1.
- group content information SM3 of group C is displayed in area AR3 in association with member list ML3 of group C
- group content information SM4 of group D is displayed in member list ML4 of group D. They are displayed in the area AR4 in association with each other.
- the terminal device 10 displays the multiple group list information CT1 showing group content information in association with the corresponding group.
- the remote lecture system 1 makes it possible for lecturers, facilitators, etc. who set group work such as lecturer's FC to efficiently check information about multiple groups.
- the terminal device 10 requests transmission of comments to the members of Group A in response to the operation of transmission of comments to the members of Group A by the instructor FC (step S15).
- the lecturer FC requests the remote meeting server 100 to send comments to the members of group A by operating an operation UI (User Interface) arranged in the area AR5 of the multiple group list information CT1.
- the terminal device 10 in response to an operation by the instructor FC, sends comment transmission information including designation information indicating that the destination of the comment is Group A and character information indicating the comment to be sent to the members of Group A. to the remote meeting server 100 .
- the terminal device 10 requests the remote meeting server 100 to send comments to the members of Group A.
- the remote meeting server 100 Upon receiving the comment transmission information from the terminal device 10, the remote meeting server 100 transmits the comment indicated by the comment transmission information to the member terminals 20 of the members of Group A. For example, the remote meeting server 100 transmits the comment indicated by the comment transmission information to the member terminal 20a of the member A-1 (step S16-1).
- the remote meeting server 100 transmits the comment indicated by the comment transmission information to the member terminal 20b of member A-2 (step S16-2).
- the remote meeting server 100 transmits the comment indicated by the comment transmission information to the member terminal 20c of member A-3 (step S16-3).
- the remote meeting server 100 transmits the comment indicated by the comment transmission information to the member terminal 20 of member A-4.
- step S16 When describing steps S16-1 to S16-3 without distinction, they are collectively referred to as step S16. Steps S16-1 to S16-3 are executed at any time when the conditions are satisfied according to the progress of the remote meeting such as group work. Moreover, step S16 may be performed together with step S11. Also, comments may be sent not only to members of group A, but also to members of groups B, C, and D, or to all members of groups A to D collectively.
- a lecturer, facilitator, etc. who sets up group work such as lecturer FC can comment at a desired timing for a group where the conversation is not active. Efficient operation becomes possible.
- FIG. 2 shows an example of information displayed on each of the terminal device 10 used by the instructor FC such as an instructor and facilitator and the member terminal 20 used by each member.
- FIG. 2 is a diagram showing an example of a display of the entire remote lecture system. Note that description of the same points as in FIG. 1 will be omitted as appropriate.
- the multiple group list information CT1 shown in FIG. 2 is a screen for grasping the whole, and is similar to the multiple group list information CT1 in FIG. 1, and shows group content information in association with corresponding groups.
- the multiple group list information CT1 is displayed on the terminal device 10 used by the lecturer FC such as the lecturer and the facilitator.
- An area (first area) on the left side of each of the areas AR1 to AR4 is a member information display area in which a list of members belonging to the corresponding group is displayed.
- the terminal device 10 highlights the member who is speaking.
- member A-2 is speaking in group A
- member B-3 is speaking in group B
- member C-1 is speaking in group C
- member D is speaking in group D.
- -42 indicates the state of speaking.
- the area (second area) on the right side of each of the areas AR1 to AR4 is the log information area of the corresponding group, in which keywords and summary information of the content of the discussion are displayed. For example, group content information of the corresponding group is displayed in the second area of each of the areas AR1 to AR4.
- the area AR5 is an operation UI area, in which interfaces for performing various operations such as buttons used when sending comments to all groups are arranged.
- Group GPA shown in FIG. 2 indicates group A to which members A-1 to A-4 belong.
- the intra-group screen CT11 shows an intra-group screen corresponding to group A displayed on the member terminal 20 used by member A-1.
- Various types of information of the corresponding group such as member information and logs of the corresponding group are displayed on the intra-group screen.
- an area AR11 of the intra-group screen CT11 is a statement display area, in which the contents of the statements made by the members of the group A are displayed.
- an area other than the area AR11 of the intra-group screen CT11 is a material area, in which screens shared within the group A are displayed.
- the intra-group screen CT14 shows the intra-group screen corresponding to group A displayed on the member terminal 20 used by member A-4, and shows the same content as the intra-group screen CT11.
- an area AR14 of the intra-group screen CT14 is a message display area, and displays the same information as the area AR11 of the intra-group screen CT11.
- the area of the intra-group screen CT14 other than the area AR14 is a material area, and displays the same information as the intra-group screen CT11.
- Group GPB shown in FIG. 2 indicates group B to which members B-1 to B-4 belong.
- the intra-group screen CT21 shows an intra-group screen corresponding to the group B displayed on the member terminal 20 used by the member B-1.
- the area AR21 of the intra-group screen CT21 is a statement display area, and the contents of the statements made by the members of the group B are displayed.
- the area other than the area AR21 of the intra-group screen CT21 is a material area, in which screens shared within the group B are displayed.
- the intra-group screen CT24 shows the intra-group screen corresponding to group B displayed on the member terminal 20 used by member B-4, and shows the same content as the intra-group screen CT21.
- the area AR24 of the intra-group screen CT24 is a message display area, and displays the same information as the area AR21 of the intra-group screen CT21.
- an area other than the area AR24 of the in-group screen CT24 is a material area, and displays the same information as the in-group screen CT21.
- Group GPD shown in FIG. 2 indicates group D to which members D-1 to D-4 belong.
- the intra-group screen CT41 shows an intra-group screen corresponding to the group D displayed on the member terminal 20 used by the member D-1.
- an area AR41 of the intra-group screen CT41 is a statement display area, in which the contents of the statements made by the members of the group D are displayed.
- an area other than the area AR41 of the intra-group screen CT41 is a material area, in which screens shared within the group D are displayed.
- the intra-group screen CT44 shows the intra-group screen corresponding to group D displayed on the member terminal 20 used by member D-4, and shows the same content as the intra-group screen CT41.
- an area AR44 of the intra-group screen CT44 is a comment display area, and displays the same information as the area AR41 of the intra-group screen CT41.
- an area other than the area AR44 of the in-group screen CT44 is a material area, and displays the same information as the in-group screen CT41.
- the display example in FIG. 2 is merely an example, and the multiple group list information CT1 can adopt any display mode as long as the group work as a whole can be grasped.
- the remote lecture system 1 provides multiple group list information CT1, which is an example of a screen for grasping the whole that lecturers and facilitators see, and intra-group screens CT11 to CT44, etc., which are examples of intra-group screens that members of each group see.
- CT1 group list information
- CT11 to CT44 intra-group screens
- the overview screen displays keywords extracted from the utterance information being discussed in each group and information summarizing the contents thereof.
- information indicating which member of the group is giving an opinion is also displayed on the screen for grasping the whole.
- FIG. 3 is a diagram showing an example of a display for an administrator.
- FIG. 3 shows, as an example, a display on the terminal device 10 used by the instructor FC for group work discussing the "2021 Olympics".
- the same reference numerals are assigned to the same points as those in FIGS. 1 and 2, and the description thereof is omitted as appropriate.
- the group content information SM1 indicates a case where it is displayed in a keyword/summary display mode that displays information such as the number of appearances of each keyword in the conversation in Group A and the content of the summary of the conversation in Group A.
- the group content information SM1 indicates that the number of occurrences of the keywords "corona” and “increase” in the conversation within group A is 8, which is the maximum number of occurrences.
- the group content information SM1 indicates that the number of appearances of the keyword “postpone” is 7 times, and the number of appearances of the keywords "held” and "infected person” is 4 times in conversations within Group A.
- the group content information SM1 indicates that the number of appearances of the keyword “economic effect” is three times, and the number of appearances of the keywords “athlete” and "2022” is two times in conversations within Group A.
- the group content information SM1 is that the summary of the conversation in Group A is "It is dangerous to hold the meeting in the midst of the corona crisis. The number of infected people may increase. Shouldn't we consider postponing it?" indicate.
- the remote meeting server 100 analyzes the log information of the conversation in Group A to generate information indicating the above-described keywords and their appearance counts. Also, the remote meeting server 100 generates the above-mentioned summary by analyzing the log information of the conversation in group A.
- keyword extraction and summary generation may be performed using various conventional techniques such as sentence analysis and natural language processing as appropriate, and detailed description thereof will be omitted.
- the group content information SM2 indicates the case where it is displayed in a keyword/summary display mode that displays information such as the number of appearances of each keyword in the conversation in Group B and the content of the summary of the conversation in Group B.
- the group content information SM2 indicates that the number of appearances of the keyword "countermeasures” is 10 times in conversations in group B, which is the maximum number of appearances.
- the group content information SM2 indicates that the number of appearances of the keyword "corona” is nine times and the number of appearances of the keyword “vaccine” is seven times in the conversations in group B.
- the group content information SM2 indicates that the number of appearances of the keyword "medicine” is six times, and the number of appearances of the keywords "holding” and "IOC” is two times in conversations in group B.
- group content information SM2 indicates that the summary of the conversation in Group B is "Vaccine development is important. Measures should be taken to prevent the spread of corona. The problem is that border control measures have not been implemented."
- the remote meeting server 100 analyzes the log information of the conversation in group B to generate information indicating the above-described keywords and their appearance counts. Also, the remote meeting server 100 generates the above-described summary by analyzing the log information of the conversation in group B.
- Figure 3 shows an example of the content of discussions between Groups A and B.
- Group A appears to be debating whether the Olympics should be held or postponed.
- Group B seems to be talking about vaccine development and problems with the spread of infection.
- the remote lecture system 1 can inform the administrator of what is being discussed in each group through the display mode of the keyword/summary display as shown in the group content information SM1 and SM2 in FIG. can be confirmed.
- various modes can be adopted when sending comments from the instructor FC to the members of each group. For example, if the lecturer prepared this group work as a place to summarize "what do you think about holding the Olympics", send comments to group B so that they can discuss the pros and cons of holding You should do it like this.
- the method of sending a comment is to select (click, etc.) the area AR2 of group B on the multiple group list information CT1, which is the overall grasp screen, in the remote lecture system 1, and indicate the intention to send the comment to group B. After displaying, the comment is sent only to group B by speaking.
- the provision of the above comments is merely an example, and the remote lecture system 1 may provide comments to the member terminals 20 in various ways, which will be described later.
- the remote lecture system 1 solves the situation in which it is difficult for lecturers and facilitators to grasp the content and progress of each group's discussions in group work conducted in a remote environment.
- the remote lecture system 1 in each group, it is possible to get an overview of who is speaking, what is being said, and whether discussions are actively taking place.
- the remote lecture system 1 can provide an environment for lecturers and facilitators to smoothly and efficiently proceed with group work in lectures and brainstorming in a remote environment.
- the remote lecture system 1 can provide an environment in which students and brainstorming participants can have a fulfilling discussion.
- the remote lecture system 1 solves the situation in which it is difficult for lecturers and facilitators to provide information such as comments to each group, but this point will be described later.
- remote lecture system 1 includes remote meeting server 100 , terminal device 10 , and multiple member terminals 20 .
- the remote meeting server 100, the terminal device 10, and each of the plurality of member terminals 20 are communicatively connected via a predetermined communication network (network N) by wire or wirelessly.
- FIG. 4 is a diagram showing a configuration example of the remote lecture system according to the embodiment. Although only three member terminals 20 are shown in FIG. 4, the remote lecture system 1 includes member terminals 20 in a number equal to or greater than the number of users participating in the remote meeting. Further, the remote lecture system 1 shown in FIG. 4 may include multiple remote meeting servers 100 and multiple terminal devices 10 .
- the remote meeting server 100 is a computer used to provide remote meeting services to users.
- the remote meeting server 100 is an information processing device that provides a meeting service for group work and the like.
- the remote meeting server 100 sends to the terminal device 10 multiple group list information that displays, together with the multiple groups, group content information about the content of conversations in at least one group among the multiple groups in association with the corresponding group. Send.
- the remote meeting server 100 transmits the member's comment received from the terminal device 10 to the corresponding member terminal 20 .
- the remote meeting server 100 also has a voice recognition function.
- the remote meeting server 100 has functions of natural language understanding (NLU) and automatic speech recognition (ASR).
- the remote meeting server 100 may have software modules for speech signal processing, speech recognition, speech semantic analysis, dialog control, and the like.
- the remote meeting server 100 may convert the user's utterance into text, and use the textualized utterance (that is, character information of the utterance) to estimate the content of the user's utterance.
- the remote meeting server 100 communicates with a speech recognition server having functions of natural language understanding and automatic speech recognition, and acquires from the speech recognition server utterances converted into text by the speech recognition server and information indicating the content of the estimated utterances. may
- the terminal device 10 is a computer that displays a plurality of groups as well as multiple group list information that displays group content information about the content of conversations in at least one group among the plurality of groups in association with the corresponding group. .
- the terminal device 10 is a device used by a user (administrator) such as a lecturer or facilitator who manages (facilitates) a remote meeting. For example, the terminal device 10 invites other users (members) to a meeting, or divides other users (members) into groups during group work, according to the operations of a user who acts as an administrator (host). do.
- the terminal device 10 outputs information about the remote meeting.
- the terminal device 10 displays an image (video) of the remote meeting and outputs audio of the remote meeting.
- the terminal device 10 transmits user's speech and images (video) to the remote meeting server 100 and receives voice and images (video) of the remote meeting from the remote meeting server 100 .
- the terminal device 10 transmits comments to members.
- the terminal device 10 accepts input from the user.
- the terminal device 10 receives voice input by user's utterance and input by user's operation.
- the terminal device 10 may be any device as long as it can implement the processing in the embodiments.
- the terminal device 10 may be any device as long as it has a function of displaying information about a remote meeting and outputting audio.
- the terminal device 10 may be a notebook PC (Personal Computer), a tablet terminal, a desktop PC, a smart phone, a smart speaker, a television, a mobile phone, a PDA (Personal Digital Assistant), or other device.
- the terminal device 10 may have a speech recognition function such as natural language understanding and automatic speech recognition.
- the terminal device 10 may convert the user's utterance into text, and use the textualized utterance (that is, character information of the utterance) to estimate the content of the user's utterance.
- Member terminals 20 are devices used by users (members) who participate in remote meetings.
- the member terminal 20 outputs information regarding the remote meeting.
- the member terminal 20 displays an image (video) of the remote meeting and outputs audio of the remote meeting.
- the member terminal 20 transmits user's speech and images (video) to the remote meeting server 100 and receives voice and images (video) of the remote meeting from the remote meeting server 100 .
- the member terminal 20 displays an intra-group screen.
- the member terminal 20 receives comments from administrators such as instructors and facilitators.
- the member terminal 20 displays the comment received from the administrator.
- the member terminal 20 accepts input from the user.
- the member terminal 20 accepts voice input by user's utterance and input by user's operation.
- the member terminal 20 may be any device as long as it can implement the processing in the embodiment.
- the member terminal 20 may be any device as long as it has a function of displaying information about a remote meeting and outputting audio.
- the member terminal 20 may be a device such as a notebook PC, tablet terminal, desktop PC, smart phone, smart speaker, television, mobile phone, PDA, or the like.
- the member terminal 20 may have a speech recognition function such as natural language understanding or automatic speech recognition.
- the member terminal 20 may convert the user's utterance into text, and use the textualized utterance (that is, character information of the utterance) to estimate the content of the user's utterance.
- the member terminal 20 may be any device as long as it can participate in a meeting as a member other than an administrator (host) such as a lecturer and perform processing.
- the member terminal 20 is similar to the terminal device 10 in terms of input/output relating to the remote meeting, except for input/output specific to administrators such as instructors. Note that the member terminal 20 has the same functional configuration as the terminal device 10 shown in FIG. 8, so a detailed description of the functional configuration will be omitted.
- FIG. 5 is a diagram illustrating a configuration example of a remote meeting server according to an embodiment of the present disclosure
- the remote meeting server 100 has a communication section 110, a storage section 120, and a control section .
- the remote meeting server 100 includes an input unit (for example, a keyboard, a mouse, etc.) for receiving various operations from the administrator of the remote meeting server 100, and a display unit (for example, a liquid crystal display, etc.) for displaying various information. may have.
- the communication unit 110 is implemented by, for example, a NIC (Network Interface Card) or the like.
- the communication unit 110 is connected to the network N (see FIG. 4) by wire or wirelessly, and transmits and receives information to and from other information processing devices such as the terminal device 10 and the member terminal 20 . Also, the communication unit 110 may transmit and receive information to and from a user terminal (not shown) used by the user.
- the storage unit 120 is implemented by, for example, a semiconductor memory device such as RAM (Random Access Memory) or flash memory, or a storage device such as a hard disk or optical disk.
- the storage unit 120 according to the embodiment has a user information storage unit 121, a group information storage unit 122, and a history information storage unit 123, as shown in FIG.
- the user information storage unit 121 stores various information about users.
- the user information storage unit 121 stores information of users participating in the remote meeting.
- the user information storage unit 121 stores attribute information and the like of each user.
- the user information storage unit 121 stores user information corresponding to information identifying each user (user ID, etc.) in association with each other.
- the group information storage unit 122 stores various information about groups.
- the group information storage unit 122 stores various types of information about multiple groups corresponding to each of the multiple groups in the remote meeting.
- FIG. 6 is a diagram illustrating an example of a group information storage unit according to an embodiment of the present disclosure; In the example shown in FIG. 6, the group information storage unit 122 includes items such as "group ID”, "group content information", and "members".
- Group ID indicates identification information for identifying a group.
- Group content information indicates group content information in the corresponding group.
- group content information indicates group content information based on interactions in the group.
- FIG. 6 shows an example in which conceptual information such as 'SM1' is stored in 'group content information', but in reality, specific information such as the number of appearances of each keyword and summary content is stored.
- Member indicates a member of the group. For example, “member” stores information (user ID, etc.) identifying a member (user) belonging to the group.
- Group A includes, as members, a user identified by user ID "A-1” (member A-1), a user identified by user ID "A-2” (member A-2), user ID It indicates that the user (member A-3) identified by "A-3” and the user (member A-4) identified by the user ID "A-4" belong.
- group information storage unit 122 may store various information not limited to the above, depending on the purpose.
- the history information storage unit 123 stores various types of information regarding the history of past group work.
- the history information storage unit 123 stores the content of discussions in past lectures, brainstorming sessions, and the like.
- FIG. 7 is a diagram illustrating an example of a history information storage unit according to an embodiment of the present disclosure;
- the history information storage unit 123 includes items such as "keyword”, "summary content", and "comment case”.
- the history information storage unit 123 may store various types of information, not limited to the above, depending on the purpose.
- the history information storage unit 123 may store a log ID that identifies each log.
- Key indicates the keyword corresponding to the summary content and comment case.
- keyword is a keyword specified by an administrator such as a lecturer or a keyword extracted from dialogue.
- Summary content indicates a summary of the content of the keyword in the group dialogue.
- Comment example indicates an example of a comment made by an administrator such as a lecturer on the corresponding keyword and summary content.
- the summary content for the keywords "held” and “postponed” is "It is dangerous to hold the event in the midst of the corona crisis. Shouldn't we consider postponing it?". In other words, it indicates that the summary of the content of the group dialogue on the keywords "held” and “postponed” is "It is dangerous to hold the meeting in the midst of the corona crisis. Shouldn't we consider postponing it?"
- history information storage unit 123 may store various types of information, not limited to the above, depending on the purpose.
- the storage unit 120 may store various information other than the above.
- the storage unit 120 stores various information regarding remote meetings.
- the storage unit 120 stores various data for providing output data to the terminal device 10 and the member terminal 20 .
- the storage unit 120 stores various information used to generate information displayed on the terminal device 10 and the member terminal 20 .
- the storage unit 120 stores information about content displayed by applications (remote meeting applications, etc.) installed in the terminal device 10 and the member terminal 20 .
- the storage unit 120 stores information about content displayed by the remote meeting app. Note that the above is merely an example, and the storage unit 120 may store various types of information used to provide remote meeting services to users.
- the storage unit 120 stores information of a speech recognition application (program) that implements the speech recognition function.
- the terminal device 10 can perform speech recognition by activating a speech recognition application (simply referred to as “speech recognition”).
- the storage unit 120 stores various information used for speech recognition.
- the storage unit 120 stores information of a dictionary (speech recognition dictionary) used for the speech recognition dictionary.
- Storage unit 120 stores information of a plurality of speech recognition dictionaries.
- the control unit 130 uses a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc. to store programs (for example, an information processing program according to the present disclosure) stored inside the remote meeting server 100 in RAM (Random Access Memory) or the like as a work area. Also, the control unit 130 is implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).
- ASIC Application Specific Integrated Circuit
- FPGA Field Programmable Gate Array
- the control unit 130 includes an acquisition unit 131, a processing unit 132, an extraction unit 133, a generation unit 134, and a transmission unit 135, and has functions and actions of information processing described below. realize or perform Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 5, and may be another configuration as long as it performs information processing described later. Further, the connection relationship between the processing units of the control unit 130 is not limited to the connection relationship shown in FIG. 5, and may be another connection relationship.
- the acquisition unit 131 acquires various types of information. Acquisition unit 131 acquires various types of information from an external information processing device. The acquisition unit 131 acquires various types of information from the terminal device 10 and the member terminal 20 . The acquisition unit 131 acquires information detected by the terminal device 10 from the terminal device 10 . The acquisition unit 131 acquires information detected by the member terminal 20 from the member terminal 20 . Acquisition unit 131 acquires various types of information from storage unit 120 .
- the acquisition unit 131 acquires information about remote meetings.
- the acquisition unit 131 acquires information such as user's utterances and images.
- the acquisition unit 131 receives information such as utterances and images of the user using the terminal device 10 from the terminal device 10 .
- the acquisition unit 131 receives information such as utterances and images of users using the member terminals 20 from the member terminals 20 .
- the acquisition unit 131 acquires group content information regarding the content of conversations in at least one or more groups among a plurality of groups in which a plurality of users participating in a remote meeting are divided and each group can have a conversation.
- the acquisition unit 131 acquires group content information from the group information storage unit 122 .
- the processing unit 132 executes various processes.
- the processing unit 132 executes image processing.
- the processing unit 132 executes processing related to speech recognition.
- the processing unit 132 uses the information stored in the storage unit 120 to perform speech recognition processing.
- the processing unit 132 converts the user's utterance into text information, thereby converting the voice of the user's utterance into text.
- the processing unit 132 can be realized by using existing speech semantic analysis technology.
- the processing unit 132 analyzes the content of the user's utterance.
- the processing unit 132 estimates the content of the user's utterance by analyzing the user's utterance using various conventional techniques as appropriate. For example, the processing unit 132 analyzes the content of the user's utterances using natural language understanding (NLU) and automatic speech recognition (ASR) functions.
- NLU natural language understanding
- ASR automatic speech recognition
- the processing unit 132 estimates (specifies) the content of the user's utterance by semantic analysis using the character information corresponding to the user's utterance. For example, the processing unit 132 estimates the content of the user's utterance corresponding to the character information by analyzing the character information using various conventional techniques such as syntactic analysis as appropriate.
- the processing unit 132 executes processing related to data retention.
- the processing unit 132 accumulates information such as image/audio and voice recognition results transmitted from each terminal device 10 and the member terminal 20 .
- the processing unit 132 stores information such as images and voices and voice recognition results transmitted from each terminal device 10 and the member terminal 20 in the storage unit 120 .
- the processing unit 132 executes keyword extraction processing.
- the processing unit 132 extracts keywords based on the conversation of the group.
- the processing unit 132 extracts an utterance corresponding to the specified keyword or the extracted keyword from among the utterances of the members in the group.
- the processing unit 132 searches the utterance histories of the members in the group using the specified keyword or the extracted keyword as an extraction keyword, and extracts an utterance containing a character string that matches the extraction keyword.
- the processing unit 132 also executes a summary generation process.
- the processing unit 132 selects an utterance corresponding to the keyword from among the utterances of the members in the group, and generates based on the selected utterance.
- the processing unit 132 selects an utterance corresponding to the keyword from among the utterances of the members in the group, and uses the sentence of the selected utterance as the summary sentence. For example, the processing unit 132 may select utterances containing a predetermined number or more of keywords from among the utterances of members in the group, and use sentences of the selected utterances as summary sentences. In addition, the processing unit 132 may select utterances containing a predetermined number or more of keywords from among the utterances of members in the group, and generate a summary using sentences of the selected utterances. In this case, the processing unit 132 generates a summary using the selected utterance sentence using a technique related to summary generation.
- the extraction unit 133 searches for various types of information and extracts the information.
- the extraction unit 133 extracts the target comment from the history of comments made in the past.
- the extraction unit 133 extracts the target comment corresponding to the target group based on the information about the target group to be assisted.
- the extraction unit 133 extracts the target comment by searching the history using the designated keyword.
- the extraction unit 133 extracts the target comment based on the degree of similarity between each comment in the history and the keyword.
- the extraction unit 133 extracts a target comment based on a comparison between the degree of similarity between each comment in the history and the keyword and a designated threshold.
- the generation unit 134 generates various types of information.
- the generation unit 134 generates various types of information based on information from an external information processing device and information stored in the storage unit 120 .
- the generation unit 134 generates various types of information based on information from other information processing devices such as the terminal device 10 and the member terminal 20 .
- the generation unit 134 generates various types of information based on the information stored in the user information storage unit 121, the group information storage unit 122, and the history information storage unit 123.
- the generation unit 134 generates various types of information to be displayed on the terminal device 10 and the member terminal 20 based on the information determined by the processing unit 132 .
- the generating unit 134 executes various processes related to images to be provided to the terminal device 10 and the member terminal 20.
- the generation unit 134 prepares information in a necessary output form for the voice and image of the group in which the user of each member terminal 20 participates as a member.
- the generator 134 generates output data to be provided to the member terminal 20 using the adjusted parameters.
- the generating unit 134 generates output data that is used for outputting remote meeting information in the member terminal 20 .
- the generation unit 134 generates output data including parameters indicating the sound volume of each group and the arrangement position and size of the image of each group.
- the generation unit 134 may generate a display screen (content) to be displayed on the terminal device 10 or the member terminal 20 as output data.
- the generating unit 134 may generate a screen (content) to be provided to the terminal device 10 or the member terminal 20 by appropriately using various techniques such as Java (registered trademark).
- the generation unit 134 may generate a screen (content) to be provided to the terminal device 10 or the member terminal 20 based on CSS, JavaScript (registered trademark), or HTML format.
- the generating unit 134 may generate screens (contents) in various formats such as JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), and PNG (Portable Network Graphics).
- the transmission unit 135 functions as an output unit that executes output processing.
- the transmitter 135 transmits information to the terminal device 10 .
- the transmitter 135 transmits the information generated by the generator 134 to the terminal device 10 .
- the transmitter 135 transmits the output data generated by the generator 134 to the terminal device 10 .
- the transmission unit 135 transmits information to the member terminal 20.
- the transmission unit 135 transmits the information generated by the generation unit 134 to the member terminal 20 .
- the transmission unit 135 transmits the output data generated by the generation unit 134 to the member terminal 20 .
- the transmission unit 135 outputs multiple group list information that displays group content information in association with the corresponding groups together with multiple groups.
- the transmission unit 135 transmits the multiple group list information to the terminal device 10 used by the manager (lecturer, etc.) who manages the meeting.
- the transmitting unit 135 transmits assist information for commenting on at least one or more groups among a plurality of groups to the terminal device 10 .
- the transmission unit 135 transmits information about comments made in the past to the terminal device 10 as assist information.
- FIG. 8 is a diagram illustrating a configuration example of a terminal device according to an embodiment of the present disclosure.
- the terminal device 10 includes a communication unit 11, an audio input unit 12, an audio output unit 13, a camera 14, a display unit 15, an operation unit 16, a storage unit 17, a control unit 18.
- the communication unit 11 is implemented by, for example, a NIC, a communication circuit, or the like.
- the communication unit 11 is connected to a predetermined communication network (network) by wire or wirelessly, and transmits and receives information to and from an external information processing device.
- the communication unit 11 is connected to a predetermined communication network by wire or wirelessly, and transmits and receives information to and from the remote meeting server 100 .
- the voice input unit 12 functions as an input unit that receives operations by voice (utterance) of the user.
- the voice input unit 12 is, for example, a microphone or the like, and detects voice.
- the voice input unit 12 detects user's speech.
- the voice input unit 12 accepts voice input indicating operations related to groups displayed in the multiple group list information. Note that the voice input unit 12 may have any configuration as long as it can detect the user's speech information necessary for processing.
- the audio output unit 13 is realized by a speaker that outputs audio, and is an output device for outputting various types of information as audio.
- the audio output unit 13 audio-outputs content provided from the remote meeting server 100 .
- the audio output unit 13 outputs audio corresponding to information displayed on the display unit 15 .
- the terminal device 10 performs voice input/output using a voice input section 12 and a voice output section 13 .
- the camera 14 has an image sensor (image sensor) that detects images.
- a camera 14 photographs the users participating in the remote meeting.
- the terminal device 10 is a notebook computer
- the camera 14 may be built in the terminal device 10 and arranged above the display section 15 .
- the camera 14 may be an in-camera built into the terminal device 10 .
- the display unit 15 is a display screen of a tablet terminal realized by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device for displaying various information.
- a liquid crystal display or an organic EL (Electro-Luminescence) display is a display device for displaying various information.
- the display unit 15 displays various information related to the remote meeting.
- the display unit 15 displays content.
- the display unit 15 displays various information received from the remote meeting server 100 .
- the display unit 15 outputs the remote meeting information received from the remote meeting server 100 .
- the display unit 15 divides a plurality of users participating in a remote meeting into a plurality of groups in which each group can converse, and displays group content information regarding the content of conversations in at least one or more groups among the plurality of groups. Display multiple group list information displayed in association with the corresponding group.
- the display unit 15 changes the display mode according to the user's operation received by the voice input unit 12 or the operation unit 16.
- the display unit 15 displays the assist information when the voice input unit 12 or the operation unit 16 receives an operation to display the assist information to the user.
- the display unit 15 displays information about past comments as assist information.
- the display unit 15 changes the display mode of the group content information when the voice input unit 12 or the operation unit 16 accepts an operation to change the display mode of the group content information.
- the display unit 15 displays multiple group list information in which group content information is displayed in association with the corresponding groups together with the multiple groups.
- the display unit 15 displays assist information for commenting on at least one or more groups among a plurality of groups.
- the display unit 15 displays information about comments made in the past as assist information.
- the operation unit 16 functions as an input unit that receives various user operations.
- the operation unit 16 receives an operation related to the groups displayed in the multiple group list information from the user using the terminal device 10 .
- the operation unit 16 is a keyboard, mouse, or the like.
- the operation unit 16 may have a touch panel capable of realizing functions equivalent to those of a keyboard and a mouse.
- the operation unit 16 receives various operations from the user via the display screen using the functions of a touch panel realized by various sensors.
- the operation unit 16 receives various operations from the user via the display unit 15 .
- the operation unit 16 receives operations such as user designation operations via the display unit 15 of the terminal device 10 .
- the tablet terminal mainly adopts the capacitance method, but there are other detection methods such as the resistive film method, the surface acoustic wave method, the infrared method, and the electromagnetic induction method. Any method may be adopted as long as the user's operation can be detected and the function of the touch panel can be realized.
- the terminal device 10 may have a configuration that accepts (detects) various types of information as input, not limited to the above.
- the terminal device 10 may have a line-of-sight sensor that detects the user's line of sight.
- the line-of-sight sensor detects the user's line-of-sight direction using eye-tracking technology based on the detection results of the camera 14 mounted on the terminal device 10, an optical sensor, and a motion sensor (all not shown).
- the line-of-sight sensor determines a region of the screen that the user is gazing at based on the detected line-of-sight direction.
- the line-of-sight sensor may transmit line-of-sight information including the determined gaze area to the remote meeting server 100 .
- the terminal device 10 may have a motion sensor that detects a user's gesture or the like.
- the terminal device 10 may receive an operation by a user's gesture using a motion sensor.
- the storage unit 17 is implemented by, for example, a semiconductor memory device such as RAM (Random Access Memory) or flash memory, or a storage device such as a hard disk or optical disk.
- the storage unit 17 stores various information received from the remote meeting server 100, for example.
- the storage unit 17 stores, for example, information about applications (for example, remote meeting applications, etc.) installed in the terminal device 10, such as programs.
- the storage unit 17 stores user information.
- the storage unit 17 stores the user's utterance history (speech recognition result history) and action history.
- the control unit 18 is a controller.
- various programs stored in a storage device such as the storage unit 17 inside the terminal device 10 are executed by the CPU, MPU, or the like using the RAM as a work area. Realized.
- these various programs include programs of applications (for example, remote meeting applications) that perform information processing.
- the control unit 18 is a controller, and is realized by an integrated circuit such as ASIC or FPGA, for example.
- control unit 18 includes an acquisition unit 181, a transmission unit 182, a reception unit 183, and a processing unit 184, and implements or executes the information processing functions and actions described below.
- the internal configuration of the control unit 18 is not limited to the configuration shown in FIG. 8, and may be another configuration as long as it performs the information processing described later.
- the connection relationship between the processing units of the control unit 18 is not limited to the connection relationship shown in FIG. 8, and may be another connection relationship.
- the acquisition unit 181 acquires various types of information. For example, the acquisition unit 181 acquires various information from an external information processing device. For example, the acquisition unit 181 stores the acquired various information in the storage unit 17 . The acquisition unit 181 acquires user operation information accepted by the operation unit 16 .
- the acquisition unit 181 acquires the user's utterance information.
- the acquisition unit 181 acquires user utterance information detected by the voice input unit 12 .
- the transmission unit 182 transmits information to the remote meeting server 100 via the communication unit 11.
- the transmission unit 182 transmits information regarding the remote meeting to the remote meeting server 100 .
- the receiving unit 183 transmits information input by user's speech or operation.
- the receiving unit 183 transmits comments by administrators such as instructors and facilitators.
- the receiving section 183 receives information from the remote meeting server 100 via the communication section 11 .
- the receiving unit 183 receives information provided by the remote meeting server 100 .
- the receiving unit 183 receives content from the remote meeting server 100 .
- the processing unit 184 executes various types of processing.
- the processing unit 184 executes processing according to the user's operation accepted by the voice input unit 12 or the operation unit 16 .
- the processing unit 184 transmits the comment to the group corresponding to the operation. Execute the process.
- the processing unit 184 executes processing of transmitting comments to all of the plurality of groups.
- the processing unit 184 displays various information via the display unit 15.
- the processing unit 184 functions as a display control unit that controls display on the display unit 15 .
- the processing unit 184 outputs various kinds of information as voice through the voice output unit 13 .
- the processing unit 184 functions as an audio output control unit that controls audio output of the audio output unit 13 .
- the processing unit 184 divides a plurality of users participating in a remote meeting into a plurality of groups in which each group can have a conversation, and group content information about the content of conversations in at least one or more groups among the plurality of groups.
- the display unit 15 is caused to display the plural group list information displayed in association with the corresponding group.
- the processing unit 184 changes the display mode according to the user's operation received by the voice input unit 12 or the operation unit 16.
- the processing unit 184 causes the display unit 15 to display the assist information when the voice input unit 12 or the operation unit 16 accepts an operation to display the assist information to the user.
- the processing unit 184 causes the display unit 15 to display information about past comments as assist information.
- the processing unit 184 causes the display unit 15 to change the display mode of the group content information.
- the processing unit 184 causes the display unit 15 to display multiple group list information in which group content information is displayed in association with the corresponding groups together with the multiple groups.
- the processing unit 184 causes the display unit 15 to display assist information for commenting on at least one or more groups among the plurality of groups.
- the processing unit 184 causes the display unit 15 to display information about comments made in the past as assist information.
- the processing unit 184 outputs the information received by the acquisition unit 181.
- the processing unit 184 outputs content provided by the remote meeting server 100 .
- Processing unit 184 outputs the content received by acquisition unit 181 via audio output unit 13 or display unit 15 .
- the processing unit 184 displays content via the display unit 15 .
- the processing unit 184 outputs the contents as audio through the audio output unit 13 .
- the processing unit 184 transmits various information to an external information processing device via the communication unit 11 .
- the processing unit 184 transmits various information to the remote meeting server 100 .
- the processing unit 184 transmits various information stored in the storage unit 17 to an external information processing device.
- the processing unit 184 transmits various information acquired by the acquisition unit 181 to the remote meeting server 100 .
- the processing unit 184 transmits the sensor information acquired by the acquisition unit 181 to the remote meeting server 100 .
- the processing unit 184 transmits the user operation information received by the operation unit 16 to the remote meeting server 100 .
- the processing unit 184 transmits information such as speech and images of the user using the terminal device 10 to the remote meeting server 100 .
- each process performed by the control unit 18 described above may be implemented by, for example, JavaScript (registered trademark).
- each unit of the control unit 18 may be realized by the predetermined application, for example.
- processing such as information processing by the control unit 18 may be realized by control information received from an external information processing device.
- the control unit 18 may have an application control unit that controls the predetermined application or dedicated application, for example.
- FIG. 9 is a flow chart showing the processing procedure of the information processing device according to the embodiment of the present disclosure. Specifically, FIG. 9 is a flowchart showing the procedure of information processing by the remote meeting server 100, which is an example of an information processing apparatus.
- the remote meeting server 100 divides a plurality of participants participating in a remote meeting into a plurality of groups in which each group can have a conversation. Acquire group content information about (step S101).
- the remote meeting server 100 transmits multiple group list information in which group content information is displayed in association with the corresponding groups together with the multiple groups (step S102).
- the remote meeting server 100 transmits to the terminal device 10, together with a plurality of groups, group content information displayed in association with the corresponding group.
- the terminal device 10 displays multiple group list information in which group content information is displayed in association with the corresponding groups together with the multiple groups.
- FIG. 10 is a flow chart showing a processing procedure for group work.
- the remote lecture system 1 performs processing will be described as an example, but the processing shown in FIG. may be performed by any device.
- the remote lecture system 1 branches the process depending on whether group work is to be started (step S201). If the remote lecture system 1 does not start group work (step S201: No), it ends the processing related to group work.
- step S201 if group work is to be started (step S201: Yes), the remote lecture system 1 sets group work (step S202). For example, when starting group work, the remote lecture system 1 sets the group work as shown in FIG.
- step S203: Yes the remote lecture system 1 reflects the set keyword and threshold in the extraction process (step S204), and performs the processes from step S205.
- step S203: No the remote lecture system 1 performs the processes from step S205 onward without performing the process of step S204.
- the remote lecture system 1 branches the processing depending on whether the lecturer confirms the assistance (step S205). If the lecturer confirms the assistance (step S205: Yes), the remote lecture system 1 feeds back the corresponding content from the stored data (step S206), and performs the processing from step S207. For example, in step S206, the remote meeting server 100 transmits assist information to the terminal device 10 as feedback of relevant content from the held data.
- step S205 if the lecturer does not confirm the assistance (step S205: No), the remote lecture system 1 performs the processes from step S207 onward without performing the process of step S206.
- the remote lecture system 1 branches the processing depending on whether or not the lecturer sends a comment (step S207). If the lecturer sends a comment (step S207: Yes), the remote lecture system 1 feeds back the contents to the student side (step S208), and performs the processing from step S209. For example, in step S208, the remote meeting server 100 transmits the lecturer's comment to the member terminal 20 as feedback of the contents to the student side.
- step S207 if the lecturer does not send a comment (step S207: No), the remote lecture system 1 performs the processing from step S209 onwards without performing the processing of step S208.
- the remote lecture system 1 branches the processing depending on whether or not an utterance is made within the group (step S209). If an utterance is made in the group (step S209: Yes), the remote lecture system 1 analyzes the contents of the utterance (step S210) and registers the processing result as holding data (step S211). After that, the remote lecture system 1 performs the processing from step S212. For example, in steps S210 and S211, the remote meeting server 100 analyzes the contents of the speech based on the speech information in the group, and registers the processing result as held data.
- step S209 No
- the remote lecture system 1 performs the processes from S212 onward without performing the processes of steps S210 and S211.
- the remote lecture system 1 branches the process depending on whether the group work is to end (step S212). For example, the remote lecture system 1 determines whether or not to end the group work using arbitrary end conditions such as set time and conditions related to convergence of the group work. When the remote lecture system 1 ends the group work (step S212: Yes), it ends the processing related to the group work.
- step S212 if the group work is not finished (step S212: No), the remote lecture system 1 returns to step S205 and repeats the process.
- the display mode shown in FIG. 3 is merely an example, and the group content information may be displayed in various display modes. For example, it is possible to change the group content information, which is the information on the overall comprehension screen, so that the log information can be further visually comprehended. This point will be described below. It should be noted that descriptions of the same points as those described above will be omitted as appropriate.
- FIG. 11 is a diagram showing an example of display by a distribution graph.
- Group content information SM5 shows an example of display of group content information in a distribution graph display mode.
- Each circle in the group content information SM5 indicates the number of appearances of the keyword displayed within the circle. That is, the larger the size of the circle, the more frequently the keyword appears. Also, the distance between each circle indicates the similarity of keywords. For example, the distance between circles of keywords with high similarity becomes short.
- the terminal device 10 displays group content information SM5.
- the terminal device 10 may display the group content information not only in characters but also in the form of a graph. By changing the display, the terminal device 10 makes it easy (intuitive) for the manager to see whether the discussion is divergent or converged within the group, whether there is a difference in the degree of progress with other groups, and the like. can be grasped.
- the number of occurrences of the extracted keywords is represented by the size of the circle, and by projecting the word vector of each keyword, the circles of potentially closely related words are closer to each other. expressed in the form This allows the manager to graphically grasp the contents of the discussion. By confirming this information, the manager will be able to facilitate the process smoothly.
- FIG. 12 is a diagram showing an example of display in frequency time series.
- the group content information SM6a indicates the number of appearances (frequency) of each keyword at the time (10 o'clock) corresponding to the point PT displayed on the time bar.
- the waveform of the group content information SM6a indicates the frequency of each keyword. That is, it indicates the frequency of keywords arranged on the wave. The higher the position of the waveform, the more frequently the keyword appears. Also, each keyword is arranged based on the similarity relationship. For example, keywords with high similarity are arranged at close positions.
- the terminal device 10 displays group content information SM6a.
- the terminal device 10 can display the group content information SM6a by time forwarding according to the operation of the user such as the administrator.
- the administrator who uses the terminal device 10 may perform time forwarding (playback) by selecting the play button PB or the like, or may perform time forwarding by moving the point PT.
- FIG. 12 shows a display example after the group content information SM6b is time-advanced from the group content information SM6a.
- the group content information SM6b indicates the number of appearances (frequency) of each keyword at the time (10:10) corresponding to the point PT moved by the administrator using the terminal device 10 . Other points are the same as the group content information SM6a.
- FIG. 12 shows that the group content information SM6b has a higher frequency of corona and postponement than the group content information SM6a.
- the terminal device 10 may display group content information so that changes over time can be confirmed. As a result, the terminal device 10 allows the administrator to easily (intuitively) grasp the situation change within the group.
- the remote lecture system 1 provides a diagram that allows easy understanding of the types of extracted keywords and the appearance frequency of the keywords.
- the frequency time-series display in FIG. 12 has a function that allows you to check transitions due to time changes. You can check whether the By confirming this information, the manager will be able to facilitate the process smoothly.
- the administrator can grasp whether the discussion content of the entire group is converging or diverging by referring to each display. For example, when various keywords are widely distributed, it can be seen that the discussion is spreading. In addition, when only one keyword or a keyword around it has a high frequency intensively, it can be understood that the discussion is focused on a certain theme.
- FIG. 13 is a diagram showing an example of a radar chart display.
- Group content information SM7 shows an example of display of group content information in a radar chart display mode.
- Group content information SM7 shows a radar chart for five keywords: corona, vaccine, holding, postponement, and infection. A larger value corresponding to each keyword indicates that the keyword appears more frequently.
- the terminal device 10 displays group content information SM7.
- the terminal device 10 displays in the form of a radar chart to indicate whether the discussion is divergent or converged within the group, whether there is a difference in the degree of progress with other groups, and the like. can be easily (intuitively) grasped by the administrator.
- FIG. 14 is a diagram showing an example of display by a line graph.
- Group content information SM8 shows an example of display of group content information in a line graph display mode.
- the group content information SM8 shows a line graph for five keywords: corona, vaccine, holding, postponement, and infection. A larger value corresponding to each keyword indicates that the keyword appears more frequently.
- the terminal device 10 displays group content information SM8.
- the terminal device 10 can display group content information in the form of a line graph so that changes over time can be confirmed. As a result, the terminal device 10 allows the administrator to easily (intuitively) grasp the situation change within the group.
- FIG. 15 is a diagram showing an example of display using a pie chart.
- Group content information SM9 shows an example of display of group content information in a pie chart display mode.
- the group content information SM9 shows a pie chart for five keywords: corona, vaccine, holding, postponement, and infection. The larger the proportion of the region corresponding to each keyword (the larger the angle), the more frequently that keyword appears.
- the terminal device 10 displays group content information SM9.
- the terminal device 10 displays in the form of a pie chart to indicate whether the discussion is divergent or converged within the group, whether there is a difference in the degree of progress with other groups, and the like. can be easily (intuitively) grasped by the administrator.
- FIG. 16 is a diagram showing an example of a bar graph display.
- Group content information SM10 shows an example of display of group content information in a bar graph display mode.
- Group content information SM10 shows a bar graph for five keywords: corona, vaccine, holding, postponement, and infection. The larger the ratio of the area corresponding to each keyword, the more frequently that keyword appears.
- the terminal device 10 displays group content information SM10.
- the terminal device 10 displays a plurality of bar graphs side by side according to time, thereby displaying group content information so that changes over time can be confirmed.
- the terminal device 10 allows the administrator to easily (intuitively) grasp the situation change within the group.
- FIG. 17 is a diagram showing an example of multiple group list information and assist information.
- the area AR5 which is the operation UI area of the multiple group list information CT1 includes a plurality of operation buttons AR51 to AR54.
- the terminal device 10 accepts the instructor's operations on the plurality of operation buttons AR51 to AR54.
- An operation button AR51 labeled "line-of-sight correspondence on/off" indicates whether or not to accept input by line of sight. Note that the case of accepting an input based on the line of sight will be described later.
- the operation button AR52 labeled "general comment” is a button for sending comments to all members of all groups. When sending a comment to the entire group, the lecturer can collectively send the comment to all the groups by selecting the operation button AR52.
- the terminal device 10 transmits the instructor's comment, designation information indicating that the destination of the comment is all groups (that is, all members), and transmission to all group members. and character information indicating the comment to be made is transmitted to the remote meeting server 100 .
- the remote meeting server 100 Upon receiving the comment transmission information from the terminal device 10, the remote meeting server 100 transmits the comment indicated by the comment transmission information to the member terminals 20 of the members of all the groups.
- the remote lecture system 1 can quickly provide common content and time management information to all students.
- the operation button AR53 labeled "Switch display” is a button for switching the display.
- the lecturer can switch the display contents of the multiple group list information CT1 by selecting the operation button AR53.
- the lecturer designates a target group (target group) from groups A to D and selects the operation button AR53 to switch the display mode of the group content information of the target group.
- the terminal device 10 accepts the selection as an operation for designating the target group.
- the terminal device 10 switches the display mode of the group content information SM1 of Group A.
- the terminal device 10 changes the display mode of the group content information SM1 into a frequency time series, a distribution graph, or the like in response to selection of the operation button AR53. switch.
- the terminal device 10 may sequentially switch the display mode in a predetermined order in accordance with the selection of the operation button AR53, or may receive a designation of which display mode to switch to from the instructor.
- FIG. 18 is a diagram illustrating an example of provision of comments using assist.
- FIG. 19 is a diagram showing another example of content of assistance.
- the remote lecture system 1 can store, for example, in the history information storage unit 123 or the like, the content that has been discussed in lectures, brainstorming sessions, and the like.
- the remote lecture system 1 can use the held data to assist the administrator such as the lecturer. Assisting is performed by searching for past content similar to what is currently being discussed and offering the content that was being discussed there as present help.
- the lecturer can control the assisting of members such as students.
- the operation button AR54 labeled "Refer to assist search" in the area AR5 is a button for the lecturer to receive assistance with comments.
- the lecturer can select the operation button AR54 to search for comment records for discussions from past log information and receive assistance based on the search results.
- the remote meeting server 100 generates information for assisting an administrator such as a lecturer (also referred to as “assist information”), and transmits the generated assist information to the terminal device 10 . This point will be described below.
- the search can be performed using keywords extracted within the group, or the instructor can specify the keyword itself, or both can be used. can do.
- the remote lecture system 1 performs a search according to the selection of the lecturer. For example, the instructor designates a group (target group) to assist. As a result, the remote lecture system 1 performs a search based on the information of the target group to be assisted.
- the terminal device 10 displays information for specifying search targets such as keyword specification, similarity specification, and group specification. For example, the terminal device 10 displays information other than the comment case, which is the search result (assist information), among the information shown in the search information AS1. After inputting the designated information, the lecturer requests the remote meeting server 100 to perform a search by selecting a search button BT1 in the search information AS1.
- the terminal device 10 requests the remote meeting server 100 to search based on the keyword, similarity, group, etc. specified by the lecturer.
- the terminal device 10 transmits to the remote meeting server 100 the specified information indicating the keyword specified by the lecturer, the degree of similarity, the group, etc. in accordance with the selection of the search button BT1 in the search information AS1, thereby executing the search to the remote meeting server. Request 100.
- the remote meeting server 100 may extract (select) a keyword based on conversations in all groups and perform a search based on that keyword. For example, if no keyword is specified, the remote meeting server 100 may determine a keyword based on the topic of group work and perform a search based on that keyword. For example, the remote meeting server 100 may use the information in the keyword list associated with the agenda to determine a keyword corresponding to the agenda of group work, and perform a search based on the keyword.
- the remote meeting server 100 extracts (selects) a keyword based on conversations in the specified group, and performs a search based on the keyword. you can go Note that the above is just an example, and the remote meeting server 100 may determine keywords using various types of information.
- the remote meeting server 100 performs a search using a predetermined set value (default value) as a threshold. If the degree of similarity is specified, the remote meeting server 100 performs a search using the specified degree of similarity as a threshold. For example, if no group is specified, the remote meeting server 100 performs a search based on the specified keyword and conversations of all groups using a default value (for example, 0.6) as a similarity threshold. In FIG. 12, the terminal device 10 requests to perform a search based on the conversations of Group A by transmitting information designating Group A. In FIG.
- the remote lecture system 1 calculates the similarity of keyword information and summary content, searches for example comments that can be helpful from past data, and provides feedback to the lecturer.
- the remote meeting server 100 searches for past comments (past comments) in the history information storage unit 123 .
- the remote meeting server 100 performs a search using a designated keyword or a keyword extracted from a conversation of a designated group as a search query (target keyword).
- target keyword a search query
- the remote meeting server 100 extracts keywords such as “postpone”, “held”, “medical care”, and “economy” based on the conversation of group A.
- the remote meeting server 100 may use designated keywords "postpone", “hold”, “medical care”, “economy”, and the like.
- the remote meeting server 100 calculates the degree of similarity between the keyword associated with each past comment in the history information storage unit 123 and the search query (target keyword), and calculates past comments whose degree of similarity is equal to or greater than the threshold.
- the remote meeting server 100 vectorizes each keyword and calculates the cosine similarity between vectors of each keyword as the similarity between keywords.
- the remote meeting server 100 converts keywords into vectors using arbitrary models (vector conversion models) such as Word2Vec and BoW (Bag of words).
- the remote meeting server 100 extracts past comments associated with keywords whose degree of similarity to the search query (target keyword) is equal to or greater than a threshold. Note that the above is just an example, and the remote meeting server 100 may use various information as appropriate to calculate the degree of similarity.
- the remote meeting server 100 asks, "Let's list the merits and demerits of holding the event," "What will be necessary for support?" Extract past comments such as For example, the remote meeting server 100 extracts the past comment “Let's list the merits and demerits of holding the event” associated with the keywords “postpone” and "hold”. In addition, the remote meeting server 100 extracts the past comment "What will be necessary for support” associated with the keywords “medical care” and "infected person”. In addition, the remote meeting server 100 extracts the past comment "What are the elements for economic promotion?" associated with the keyword “economic effect”.
- the remote meeting server 100 transmits the extracted past comments to the terminal device 10 as assist information.
- Remote meeting server 100 transmits search information AS1 including the extracted past comments as assist information to terminal device 10 .
- the terminal device 10 that has received the search information AS1 displays the search information AS1.
- the lecturer can confirm the past comments provided as assist information.
- the instructor can obtain information from the retrieved results (assist information), generate comments for the group, and provide the comments to the group.
- the lecturer can use the assist information as it is to send comments to the group.
- the information sent to the group can be controlled by the lecturer using check boxes or the like.
- the comment candidate specified in the check box is actually sent to the group as a comment.
- FIG. 18 shows the case where the check boxes CK1 and CK2 are selected among the check boxes CK1 to CK3.
- the terminal device 10 selects the past comment "Let's list the merits and demerits of holding the event” and the past comment "What will be necessary for the support?" as comments to be transmitted. .
- the lecturer After specifying the comment to be sent, the lecturer requests the remote meeting server 100 to send the comment to be sent by selecting the send button BT2 in the search information AS1.
- the terminal device 10 requests the remote meeting server 100 to transmit the comment to be transmitted specified by the lecturer.
- the terminal device 10 transmits to the remote meeting server 100, according to the selection of the send button BT2 in the search information AS1, the specified information indicating the comments and groups to be sent specified by the lecturer to the remote meeting server 100.
- Request to submit comments In FIG. 18, the terminal device 10 displays the past comment "Let's list the merits and demerits of holding the event" corresponding to the check box CK1, and the past comment "What do you need to support?" corresponding to the check box CK2. It requests the remote meeting server 100 to send two comments to group A, "Come on".
- the remote meeting server 100 transmits the comment to the member terminals 20 used by the members of the group (step S21).
- the remote meeting server 100 receives two comments from the lecturer, namely, the past comment "Let's list the advantages and disadvantages of holding the event" and the past comment "What will be necessary for support?" It is transmitted to the member terminals 20 of group A.
- the remote meeting server 100 groups comment information CM1 indicating two lecturer comments: "Let's list the advantages and disadvantages of holding a meeting" and "What will be necessary for support?" It is transmitted to the member terminal 20 of A.
- the member terminal 20 that has received the comment information CM1 displays the comment information CM1.
- the remote lecture system 1 can facilitate commenting by administrators such as lecturers to members. Therefore, in the remote lecture system 1, even if the lecturer is new, it is possible to provide group work similar to that of a veteran lecturer.
- the lecturer himself/herself can learn by using the past data and referring to the comments made by the lecturers in the past. For example, in the remote lecture system 1, when a lecturer is studying, the contents of the registered data can be referred to by specifying the keyword "refer to assist search" without performing group work.
- the terminal device 10 designates the similarity threshold as “0.4” and transmits information designating Group A, thereby setting the threshold as “0.4” based on the conversation of Group A. , and requests the remote meeting server 100 to perform a search.
- the remote meeting server 100 transmits to the terminal device 10 search information AS2 including the extracted past comments as assist information.
- the terminal device 10 that has received the search information AS2 displays the search information AS2.
- the remote meeting server 100 asks, "Let's list the merits and demerits of holding the event", "What will be necessary for support?", "What are the factors for economic promotion?" Extract past comments such as For example, the remote meeting server 100 displays past comments associated with the keywords “postpone” and “holding”, “Let's list the advantages and disadvantages of holding the event", "What is important if you are in a position to be involved in the operation", and Extract “What was necessary to prevent postponement?" In addition, the remote meeting server 100 extracts the past comment "What will be necessary for support” associated with the keywords “medical care” and “infected person". In addition, the remote meeting server 100 extracts the past comment "What are the elements for economic promotion?" associated with the keyword “economic effect”.
- the instructor selects the comments to be sent to the members by selecting the check boxes corresponding to the past comments to be sent as comments from among the check boxes CK11 to CK15.
- the comments to be transmitted are provided to the members as the lecturer's comments according to the selection of the lecturer.
- FIG. 20 shows an example of information provided as an assist.
- FIG. 20 is a diagram showing an example of assistance contents.
- the contents of the assistance shown in FIG. 20 are generated by the remote meeting server 100 based on information such as the lecturer's data holding list.
- comment list CL1 in FIG. 20 is generated from past comments in history information storage unit 123 .
- Remote meeting server 100 extracts past comments from history information storage unit 123 and generates comment list CL1 using the extracted past comments.
- the past comments shown in the comment list CL1 are the same as the past comments shown in the search information AS2 of FIG. 19, and thus the description thereof is omitted.
- FIG. 21 is a diagram showing an example of keywords and summary contents.
- FIG. 21 shows, as an example, a discussion on the 2021 Olympics in which four keywords "postponement", “held”, “coronavirus”, and “medical care” specified by the lecturer are targeted.
- the dialogue log DA1 in FIG. 21 shows the flow of dialogue within the group between 10:00 and 10:10 regarding the discussion on the 2021 Olympics.
- the remote meeting server 100 collects information as shown in the interaction log DA1.
- the remote meeting server 100 Based on the dialogue log DA1, the remote meeting server 100 generates list information of keywords and summary contents associated with time, as shown in the dialogue summary data DT1 (step S31).
- the dialogue summary data DT1 shows an example of a held list of student-side data.
- the remote meeting server 100 generates dialogue summary data DT1 by analyzing the dialogue log DA1 for the four keywords "postpone", "held”, "coronavirus", and "medical care" specified by the lecturer.
- the remote meeting server 100 stores the dialogue log DA1 and the dialogue summary data DT1 in the storage unit 120.
- FIG. 22 is a diagram showing an example of the flow of settings for group work.
- FIG. 22 shows, as an example, a case where the lecturer starts group work.
- the first screen FS1 in FIG. 22 shows the initial screen.
- the instructor sets the number of groups and keywords by operating the terminal device 10 on which the first screen FS1 is displayed.
- a second screen FS2 in FIG. 22 shows a screen for designating the number of groups (4 in the case of FIG. 22) and performing member division.
- the lecturer operates the terminal device 10 on which the second screen FS2 is displayed to allocate members to each group.
- a third screen FS3 in FIG. 22 shows a screen for setting keywords.
- the instructor sets a keyword by operating the terminal device 10 on which the third screen FS3 is displayed.
- a fourth screen FS4 in FIG. 22 shows a screen for setting a threshold of keyword similarity.
- the lecturer operates the terminal device 10 on which the fourth screen FS4 is displayed to set the degree of similarity of the keywords.
- the instructor when starting group work, the instructor can specify the number of groups, members to be included in the group, and keywords to be extracted (picked up) in the group work.
- the keyword is not an essential specification item. If not specified, the remote lecture system 1 automatically handles proper nouns and common nouns that are treated as keywords.
- the remote lecture system 1 selects and extracts words similar to the specified keyword (similar words obtained by using similarity using word vectors, etc.).
- the remote lecture system 1 can also set a threshold value for determining similar words.
- FIG. 22 is only an example, and any setting mode can be adopted for group work settings.
- multiple members participating in group work may be automatically divided into multiple groups.
- members may be able to change the group to which they belong.
- Operations on the terminal device 10 are not limited to the above-described keyboard, mouse, touch panel, voice input, etc., and may be accepted by various input modes.
- the terminal device 10 may accept an operation based on the operator's (user's) line of sight.
- the operation button, etc. it is also possible to specify using the line of sight. This point will be described with reference to FIGS. 23 and 24.
- FIG. 23 and 24 are diagrams showing examples of display modes. Note that description of the same points as those described above will be omitted as appropriate.
- line-of-sight information of the lecturer FC is acquired from the camera 14, and a pointer SP1 indicating the fixation point of the line of sight of the lecturer FC is displayed on the screen of the terminal device 10 based on the line-of-sight information.
- the terminal device 10 displays the pointer SP1 superimposed on the overall comprehension screen.
- the remote lecture system 1 various processes are executed according to the position of the pointer SP1, which is the line-of-sight pointer of the lecturer FC.
- the pointer SP1 when the pointer SP1 is included in the area of any group, the voice of the lecturer FC is transmitted to the member terminals 20 of the group where the pointer SP1 is positioned.
- comments may be sent from the lecturer FC to the group based on the line of sight information of the lecturer FC.
- the instructor FC moves the position of the pointer SP2, which is the line-of-sight pointer, into the area AR1 of the group A, and in the state in which the area AR1 of the group A is selected (the highlight of the frame of the area AR1 is fed back).
- a comment is sent to Group A.
- the terminal device 10 remotely transmits the speech information and the designation information indicating that the designated group is the group A.
- Send to meeting server 100 The remote meeting server 100 that has received the speech information and the designation information transmits the speech information to the member terminals 20 of group A designated by the designation information. After receiving the speech information, the member terminal 20 outputs the speech information.
- the lecturer FC when the lecturer FC sends a comment to the entire class, it may be done based on the lecturer FC's line of sight information. For example, the instructor FC moves the position of the pointer SP2, which is the line-of-sight pointer, into the area 52 labeled "general comment", and selects the "general comment” (feedback such as highlighting the frame of the area 52). ), by uttering a comment, the content will be sent to the entire group.
- the pointer SP2 which is the line-of-sight pointer
- the pointer SP2 when the pointer SP2 is positioned in the "whole" frame, for example, when positioned in an area other than the areas AR1 to AR5, the voice of the lecturer FC is transmitted to the member terminals 20 of all groups. be done.
- the above operation may be an operation as a touch panel.
- FIG. 25 is used to conceptually show components based on functions in the remote lecture system 1 .
- FIG. 25 is a diagram showing an example of components of a remote lecture system. 23 may be implemented by any of various devices such as the remote meeting server 100, the terminal device 10, the member terminal 20, etc. included in the remote lecture system 1.
- FIG. 25 is a diagram showing an example of components of a remote lecture system. 23 may be implemented by any of various devices such as the remote meeting server 100, the terminal device 10, the member terminal 20, etc. included in the remote lecture system 1.
- the remote lecture system 1 includes three functional elements: a first element EL1, a second element EL2, and a third element EL3.
- the first element EL1 is an element corresponding to a lecturer (administrator), and its function is realized by the terminal device 10.
- the second element EL2 is an element corresponding to a student (member), and its function is realized by the member terminal 20.
- the third element EL3 is an element corresponding to an entity that provides various services to lecturers and students, and its function is realized by the remote meeting server 100.
- the third element EL3 has three functions: an input/output control section FC1, an information processing section FC2, and a data holding section FC3.
- the information processing section FC2 includes an image processing section FC21 and an audio processing section FC22.
- the input/output control unit FC1 corresponds to the communication unit 110 of the remote meeting server 100
- the information processing unit FC2 corresponds to the control unit 130 of the remote meeting server 100
- the data storage unit FC3 corresponds to the remote meeting server 100.
- the image processing unit FC21 and the audio processing unit FC22 correspond to the processing unit 132 of the control unit 130.
- the input/output control unit FC1 controls the transmission of images and sounds input to the second element EL2 (system), other information from the touch panel, buttons, etc. to the information processing unit FC2, and controls the information processing unit FC2. It controls the presentation of the information processed in the above to the system user.
- the information processing unit FC2 uses information from the input/output control unit FC1 to create data to be held and information to be output.
- the image processing unit FC21 has a function of analyzing information such as the movement, orientation, and facial expression of the teacher's and/or the student's eyes and face.
- the speech processing unit FC22 has a function of removing noise from input speech and converting waveform information into text information.
- information such as image information, touch panels, and buttons is used to switch display methods for system users and to control input information.
- the information processing unit FC2 creates keywords and a summary of the content from the utterance information converted into text.
- the information processing unit FC2 counts the number of times the keywords have appeared, sorts the abstracts according to the degree of similarity, edits them in a form that is easy for the system user to understand, and generates information to be output.
- the information processing section FC2 refers to the information of the extracted keywords and summary content, and the information held in the data holding section FC3, and creates information to be fed back to the system user.
- the information of extracted keywords and summary contents is divided between the lecturer side and the student side of the system user, and each data is sent to the data holding section FC3. For example, the keywords and abstract information extracted from the student side are sent to the data holding unit FC3 together with the time information of the current time.
- the data holding unit FC3 holds information processed by the information processing unit FC2.
- Data on the instructor side and extracted information on the student side are retained as time-series data.
- the stored data can be corrected by the lecturer if there is an error in the content of the summary through dialogue with the students and hearing of the summary.
- the member terminals 20 and terminal devices 10 transmit information such as user's utterances and images to the remote meeting server 100, and output information received from the remote meeting server 100.
- the remote lecture system 1 has been described as an example of the configuration of a so-called centralized system, such as a client-server system in which main processing is performed on the server side. It should be noted that the above is merely an example, and any form of division of functions in the remote lecture system 1 can be adopted.
- the system configuration in which the remote meeting server 100 performs image processing and voice recognition has been described as an example, but the image processing and voice recognition are performed by the terminal device 10 and the member terminal 20 respectively.
- the terminal device 10 may perform past comment search processing, assist information generation processing, and the like.
- the terminal device 10 functions as an information processing device that outputs, together with a plurality of groups, multiple group list information in which group content information is displayed in association with the corresponding groups.
- the terminal device 10 holds information stored in the storage unit 120 and has functions of a processing unit 132 , an extraction unit 133 and a generation unit 134 .
- the terminal device 10 functions as a rich client that performs processing related to speech recognition, information retrieval (extraction), generation, and output (display).
- the remote meeting server 100 collects information from the terminal device 10 and each member terminal 20 and provides necessary information to the terminal device 10 and each member terminal 20 .
- the terminal device 10 and each member terminal 20 communicate information for providing a service related to a remote meeting between the terminal device 10 and each member terminal 20 in a so-called P2P (Peer-to-Peer) manner.
- the remote lecture system 1 may not include a remote meeting server.
- the remote lecture system 1 executes, for example, the main processing on the user's terminal (client) side, and the server only manages information related to remote meetings, or a system configuration that does not include a server, a so-called autonomous It may be a distributed system configuration.
- the remote lecture system 1 may have any configuration, such as a centralized type or an autonomous decentralized type.
- the remote lecture system 1 may have any functional division mode and any device configuration as long as it can provide the above-described remote meeting-related service. good too.
- each component of each device illustrated is functionally conceptual and does not necessarily need to be physically configured as illustrated.
- the specific form of distribution and integration of each device is not limited to the one shown in the figure, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured.
- the information processing apparatus includes an acquisition unit (the acquisition unit 131 or the acquisition unit 181 in the embodiment), an output unit (the , a transmission unit 135 or a display unit 15) is provided.
- the acquisition unit acquires group content information about the content of conversation in at least one or more groups among a plurality of groups in which a plurality of users participating in a remote meeting are divided and each group can have a conversation.
- the output unit outputs multiple group list information in which group content information is displayed in association with the corresponding groups together with the multiple groups.
- the information processing apparatus outputs, together with a plurality of groups, group list information in which group content information is displayed in association with the corresponding group, thereby efficiently displaying information about a plurality of groups. can be made identifiable to
- the output unit outputs assist information for commenting on at least one or more groups among the plurality of groups.
- the information processing device can appropriately assist the dialogue of the group by outputting assist information for commenting to the group.
- the output unit outputs information about comments made in the past as assist information.
- the information processing device can appropriately assist the group's dialogue by outputting information about comments made in the past.
- the information processing device also includes an extraction unit (extraction unit 133 in the embodiment).
- the extraction unit extracts a target comment from a history of past comments. In this way, the information processing apparatus can provide assistance based on appropriate past comments by extracting target comments from the history of past comments.
- the extraction unit extracts the target comment corresponding to the target group based on the information about the target group to be assisted.
- the information processing apparatus can extract the target comment corresponding to the target group to be assisted, thereby enabling assist based on appropriate past comments corresponding to the group.
- the extraction unit extracts the target comment by searching the history with the specified keyword.
- the information processing apparatus can perform an assist based on an appropriate past comment corresponding to the keyword by searching the history using the specified keyword and extracting the target comment.
- the extraction unit extracts the target comment based on the degree of similarity between each comment in the history and the keyword.
- the information processing device extracts the target comment based on the degree of similarity between each comment in the history and the keyword, thereby making it possible to provide assistance based on appropriate past comments corresponding to the keyword. can.
- the output unit transmits the multiple group list information to the terminal device.
- the information processing device can efficiently confirm information about a plurality of groups by transmitting the multiple group list information to the terminal device.
- the output unit transmits the multiple group list information to the terminal device used by the manager who manages the meeting.
- the information processing device can efficiently confirm information about a plurality of groups by transmitting the list information of the plurality of groups to the terminal device used by the administrator who manages the meeting.
- the terminal device (the terminal device 10 in the embodiment) according to the present disclosure includes the display unit (the display unit 15 in the embodiment).
- the display part corresponds to a plurality of groups in which a plurality of users participating in a remote meeting are divided and each group can have a conversation, as well as group content information related to the content of conversations in at least one or more groups among the plurality of groups. Displays the multiple group list information displayed in association with the group to be displayed.
- the terminal device efficiently displays information about multiple groups by displaying multiple group list information that displays group content information in association with the corresponding group together with multiple groups. can be verified.
- the terminal device also includes an input unit (the voice input unit 12 or the operation unit 16 in the embodiment).
- the input unit receives an operation related to the groups displayed in the multiple group list information from the user using the terminal device.
- the terminal device can efficiently confirm information about a plurality of groups by accepting an operation related to the groups displayed in the multiple group list information, and the group displayed in the multiple group list information. It is possible to improve the convenience for the user of operations related to.
- the display unit changes the display mode according to the user's operation received by the input unit.
- the terminal device can display appropriate information according to the user's operation by changing the display mode according to the user's operation.
- the display unit displays the assist information when the input unit accepts an operation to display the assist information to the user.
- the terminal device can assist at an appropriate timing by displaying the assist information in accordance with the user's operation.
- the display unit displays information about past comments as assist information.
- the terminal device can appropriately assist the group's dialogue by displaying information about comments made in the past.
- the display unit changes the display mode of the group content information when the input unit receives an operation to change the display mode of the group content information.
- the terminal device can display appropriate information according to the user's operation by changing the display mode of the group content information according to the user's operation.
- the terminal device also includes a processing unit (processing unit 184 in the embodiment).
- the processing unit executes processing according to the user's operation received by the input unit. In this way, the terminal device can efficiently confirm information about a plurality of groups and appropriately execute processing according to the user's operation by executing the process according to the user's operation. can.
- the processing unit executes a process of transmitting the comment to the group corresponding to the operation.
- the terminal device sends the comment to the group at an appropriate timing by executing the process of sending the comment to the group corresponding to the operation.
- the processing unit executes processing of sending comments to all of the plurality of groups.
- the terminal device can send comments collectively to all of a plurality of groups by executing a process of sending comments to all of a plurality of groups.
- FIG. 26 is a hardware configuration diagram showing an example of a computer 1000 that implements the functions of the information processing device.
- the remote meeting server 100 according to the embodiment will be described below as an example.
- the computer 1000 has a CPU 1100 , a RAM 1200 , a ROM (Read Only Memory) 1300 , a HDD (Hard Disk Drive) 1400 , a communication interface 1500 and an input/output interface 1600 .
- Each part of computer 1000 is connected by bus 1050 .
- the CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400 and controls each section. For example, the CPU 1100 loads programs stored in the ROM 1300 or HDD 1400 into the RAM 1200 and executes processes corresponding to various programs.
- the ROM 1300 stores a boot program such as BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started, and programs dependent on the hardware of the computer 1000.
- BIOS Basic Input Output System
- the HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by the CPU 1100 and data used by such programs.
- HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450 .
- a communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (for example, the Internet).
- CPU 1100 receives data from another device via communication interface 1500, and transmits data generated by CPU 1100 to another device.
- the input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000 .
- the CPU 1100 receives data from input devices such as a keyboard and mouse via the input/output interface 1600 .
- the CPU 1100 also transmits data to an output device such as a display, speaker, or printer via the input/output interface 1600 .
- the input/output interface 1600 may function as a media interface for reading a program or the like recorded on a predetermined recording medium.
- Media include, for example, optical recording media such as DVD (Digital Versatile Disc) and PD (Phase change rewritable disk), magneto-optical recording media such as MO (Magneto-Optical disk), tape media, magnetic recording media, semiconductor memories, etc. is.
- the CPU 1100 of the computer 1000 implements the functions of the control unit 130 and the like by executing the information processing program loaded on the RAM 1200.
- the HDD 1400 also stores an information processing program according to the present disclosure and data in the storage unit 120 .
- CPU 1100 reads and executes program data 1450 from HDD 1400 , as another example, these programs may be obtained from another device via external network 1550 .
- the present technology can also take the following configuration.
- an acquisition unit that acquires group content information related to the content of conversations in at least one or more groups among a plurality of groups in which a plurality of users participating in a remote meeting are divided and can converse within each group; an output unit that outputs multiple group list information that displays the group content information in association with the corresponding groups together with the multiple groups; Information processing device.
- the output unit The information processing apparatus according to (1), wherein assist information for commenting on at least one group among the plurality of groups is output.
- the output unit The information processing apparatus according to (2), wherein information relating to comments made in the past is output as the assist information.
- an extraction unit that extracts a target comment from the history of comments made in the past; The information processing device according to (3).
- the extractor is The information processing apparatus according to (4), wherein the target comment corresponding to the target group is extracted based on information about the target group to be assisted.
- the extractor is The information processing apparatus according to (4) or (5), wherein the target comment is extracted by searching the history using a designated keyword.
- the extractor is The information processing apparatus according to (6), wherein the target comment is extracted based on the degree of similarity between each comment in the history and the keyword.
- the output unit transmitting the multiple group list information to the terminal device; The information processing device according to any one of (1) to (7). (9) The output unit transmitting the multiple group list information to the terminal device used by an administrator who manages the meeting; The information processing device according to (8).
- a plurality of users participating in a remote meeting are separated, and among a plurality of groups in which each group can have a conversation, group content information about the content of the conversation in at least one or more groups is acquired, Along with the plurality of groups, outputting multiple group list information that displays the group content information in association with the corresponding group; An information processing method that performs processing.
- a plurality of users participating in a remote meeting are divided into a plurality of groups in which each group can converse, and group content information about the content of conversations in at least one or more groups among the plurality of groups is sent to the corresponding group.
- the terminal device comprising: (13) The display unit (12) The terminal device according to (12), wherein a display mode is changed according to the user's operation received by the input unit. (14) The display unit (13) The terminal device according to (13), wherein the assist information is displayed when an operation for displaying the assist information to the user is accepted by the input unit. (15) The display unit The terminal device according to (14), wherein information about past comments is displayed as the assist information.
- the display unit The terminal device according to any one of (12) to (15), wherein the display mode of the group content information is changed when an operation to change the display mode of the group content information is accepted by the input unit. .
- a processing unit that executes processing according to the user's operation received by the input unit; (12) The terminal device according to any one of (16).
- the processing unit is (17) executing a process of sending a comment to the group corresponding to the operation when an operation of commenting on at least one group among the plurality of groups is accepted by the input unit; Terminal equipment as described.
- the processing unit is (17) or (18), the terminal according to (17) or (18), wherein, when an operation to comment on all of the plurality of groups is received by the input unit, a process of transmitting the comment to all of the plurality of groups is performed.
- Device (20) A plurality of users participating in a remote meeting are divided into a plurality of groups in which each group can converse, and group content information about the content of conversations in at least one or more groups among the plurality of groups is sent to the corresponding group. Acquire multiple group list information to be displayed in correspondence, displaying the multiple group list information; The display method that performs the processing.
- remote lecture system 100 remote meeting server (information processing device) 110 communication unit 120 storage unit 121 user information storage unit 122 group information storage unit 130 control unit 131 acquisition unit 132 processing unit 133 extraction unit 134 generation unit 135 transmission unit 10 terminal device (information processing device) 11 communication unit 12 voice input unit (input unit) 13 audio output unit 14 camera 15 display unit 16 operation unit (input unit) 17 storage unit 18 control unit 181 acquisition unit 182 transmission unit 183 reception unit 184 processing unit
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Business, Economics & Management (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Entrepreneurship & Innovation (AREA)
- Computer Networks & Wireless Communication (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Data Mining & Analysis (AREA)
- Economics (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
1.実施形態
1-1.本開示の実施形態に係る情報処理の概要
1-1-1.リモート講義システム全体の表示例
1-1-2.端末装置の表示例
1-1-3.背景及び効果等
1-2.実施形態に係るリモート講義システムの構成
1-3.実施形態に係る情報処理装置の構成
1-4.実施形態に係る端末装置の構成
1-5.実施形態に係る情報処理の手順
1-5-1.情報処理装置に係る処理の手順
1-5-2.グループワークに関する処理の手順
1-6.表示例
1-6-1.分布グラフ
1-6-2.頻度時系列
1-6-3.レーダーチャート
1-6-4.折れ線グラフ
1-6-5.円グラフ
1-6-6.棒グラフ
1-7.処理例
1-7-1.アシスト及びコメント例
1-7-2.データ例
1-7-3.グループワーク設定例
1-8.操作例(視線)
1-9.システム例
2.その他の実施形態
2-1.その他の構成例
2-2.その他
3.本開示に係る効果
4.ハードウェア構成
[1-1.本開示の実施形態に係る情報処理の概要]
図1は、本開示の実施形態に係る情報処理の一例を示す図である。本開示の実施形態に係る情報処理は、リモート会合サーバ100や端末装置10や複数のメンバー端末20を含むリモート講義システム1によって実現される。図1では各メンバー端末20を区別して説明するために、各メンバー端末20を、メンバー端末20a、メンバー端末20b及びメンバー端末20cとして説明する場合がある。なお、メンバー端末20a、メンバー端末20b及びメンバー端末20c等について、特に区別せずに説明する場合は、「メンバー端末20」と記載する。また、図1では、3つのメンバー端末20を図示するが、メンバー端末20は4つ以上あってもよい。
ここから、図2及び図3を用いて、リモート講義システム1における各種の表示例について説明する。まず、図2を用いて、講師やファシリテータ等の講師FCが利用する端末装置10及び各メンバーが利用するメンバー端末20の各々に表示される情報の一例を示す。図2は、リモート講義システム全体の表示の一例を示す図である。なお、図1と同様の点については適宜説明を省略する。
ここで、図3を用いて、講師やファシリテータ等の管理者が利用する端末装置10での表示例について説明する。図3は、管理者への表示の一例を示す図である。図3では、「2021年オリンピック」について議論するグループワークについて、講師FCが利用する端末装置10における表示を一例として示す。なお、図3において、図1や図2と同様の点については同様の符号を付すなどにより適宜説明を省略する。
従来のリモート環境におけるグループワークでは、グループごとに分かれたメンバーの中だけで環境が閉じられる。このため、講師やファシリテータは各グループの進捗や議論内容を知るために、各グループの環境に入らなければならない。別のグループ環境の状況を把握するためには、入っているグループ環境から一度抜け、さらに別のグループ環境へと移動しなければならず、手間が発生する。
図4に示すリモート講義システム1について説明する。図4に示すように、リモート講義システム1は、リモート会合サーバ100と、端末装置10と、複数のメンバー端末20とが含まれる。リモート会合サーバ100と、端末装置10と、複数のメンバー端末20の各々とは所定の通信網(ネットワークN)を介して、有線または無線により通信可能に接続される。図4は、実施形態に係るリモート講義システムの構成例を示す図である。なお、図4では3個のメンバー端末20のみを図示するが、リモート講義システム1には、リモート会合に参加するユーザの数以上の数のメンバー端末20が含まれる。また、図4に示したリモート講義システム1には、複数のリモート会合サーバ100や複数の端末装置10が含まれてもよい。
次に、実施形態に係る情報処理を実行する情報処理装置の一例であるリモート会合サーバ100の構成について説明する。図5は、本開示の実施形態に係るリモート会合サーバの構成例を示す図である。
次に、実施形態に係る情報処理を実行する情報処理装置の一例である端末装置10の構成について説明する。図8は、本開示の実施形態に係る端末装置の構成例を示す図である。
次に、図9及び図10を用いて、実施形態に係る各種情報処理の手順について説明する。
まず、図9を用いて、情報処理装置に係る処理の流れについて説明する。図9は、本開示の実施形態に係る情報処理装置の処理手順を示すフローチャートである。具体的には、図9は、情報処理装置の一例であるリモート会合サーバ100による情報処理の手順を示すフローチャートである。
次に、図10を用いて、リモート講義システム1におけるグループワークに関する処理について説明する。図10は、グループワークに関する処理手順を示すフローチャートである。なお、以下では、リモート講義システム1が処理を行う場合を一例として説明するが、図10に示す処理は、リモート講義システム1に含まれる装置構成に応じて、リモート会合サーバ100及び端末装置10等のいずれの装置が行ってもよい。
図3に示した表示態様は一例に過ぎず、様々な表示態様でグループ内容情報を表示してもよい。例えば、全体把握用画面の情報であるグループ内容情報は、ログ情報をさらに視覚的に把握できるように変更することも可能である。この点について、以下説明する。なお、上述した内容と同様の点については適宜説明を省略する。
まず、図11を用いて、分布グラフでの表示について説明する。図11は、分布グラフによる表示の一例を示す図である。
次に、図12を用いて、頻度時系列での表示について説明する。図12は、頻度時系列による表示の一例を示す図である。
次に、図13を用いて、レーダーチャートでの表示について説明する。図13は、レーダーチャートによる表示の一例を示す図である。
次に、図14を用いて、折れ線グラフでの表示について説明する。図14は、折れ線グラフによる表示の一例を示す図である。
次に、図15を用いて、円グラフでの表示について説明する。図15は、円グラフによる表示の一例を示す図である。
次に、図16を用いて、棒グラフでの表示について説明する。図16は、棒グラフによる表示の一例を示す図である。
ここから、図2等を参照しつつ、管理者の一例である講師による操作に応じた処理例等を示す。リモート講義システム1は、講師による操作等に応じて対応する処理を行う。
ここから、アシストに関連する各種処理等について、図17~図19を用いて説明する。図18は、アシストを用いたコメントの提供の一例を示す図である。図19は、アシストの内容の他の一例を示す図である。
ここで、データ例について説明する。まず、図20を用いて、アシストとして提供される情報の一例を示す。図20は、アシスト内容の一例を示す図である。図20に示すようなアシスト内容は、講師側データ保持一覧表等の情報を基にリモート会合サーバ100が生成する。例えば、図20中のコメント一覧CL1は、履歴情報記憶部123中の過去のコメントから生成される。リモート会合サーバ100は、履歴情報記憶部123から過去のコメントを抽出し、抽出した過去のコメントを用いてコメント一覧CL1を生成する。コメント一覧CL1に示す過去コメントは、図19の検索情報AS2に示す過去コメントと同様であるため、説明を省略する。
ここで、グループワーク設定の一例について図22を用いて説明する。図22は、グループワークの設定の流れの一例を示す図である。図22では、講師がグループワークを開始する場合を一例として示す。
端末装置10に対する操作は、上述したキーボード、マウス、タッチパネル、音声入力等に限らず、様々な入力態様により受け付けられてもよい。例えば、端末装置10は、操作者(ユーザ)の視線による操作を受け付けてもよい。上記操作ボタン等の指定に関しては、視線を利用した指定も可能である。この点について、図23及び図24を用いて説明する。図23及び図24は、表示態様の一例を示す図である。なお、上述した点と同様の点は適宜説明を省略する。
ここで、図25を用いて、リモート講義システム1における機能に基づく構成要素を概念的に示す。図25は、リモート講義システムの構成要素の一例を示す図である。なお、図23に示す各要素の機能は、リモート講義システム1に含まれるリモート会合サーバ100、端末装置10、メンバー端末20等の各種装置のいずれが実現してもよい。
上述した各実施形態に係る処理は、上記各実施形態や変形例以外にも種々の異なる形態(変形例)にて実施されてよい。
上記のリモート講義システム1では、メンバー端末20や端末装置10がユーザの発話や画像等の情報をリモート会合サーバ100へ送信し、リモート会合サーバ100から受信した情報を出力するデバイス、いわゆるシンクライアントとして機能する。このように、上述した例では、リモート講義システム1は、例えばサーバ側で主な処理を実行するクライアントサーバ方式のシステムのような、いわゆる中央集権型システムの構成を一例として説明した。なお、上記は一例に過ぎず、リモート講義システム1における機能の分割は任意の態様が採用可能である。
また、上記各実施形態において説明した各処理のうち、自動的に行われるものとして説明した処理の全部または一部を手動的に行うこともでき、あるいは、手動的に行われるものとして説明した処理の全部または一部を公知の方法で自動的に行うこともできる。この他、上記文書中や図面中で示した処理手順、具体的名称、各種のデータやパラメータを含む情報については、特記する場合を除いて任意に変更することができる。例えば、各図に示した各種情報は、図示した情報に限られない。
上述のように、本開示に係る情報処理装置(例えば、実施形態ではリモート会合サーバ100または端末装置10)は、取得部(実施形態では取得部131または取得部181)と、出力部(実施形態では送信部135または表示部15)とを備える。取得部は、リモートでの会合に参加する複数のユーザが分かれ、各グループ内で会話できる複数のグループのうち、少なくとも1つ以上のグループでの会話の内容に関するグループ内容情報を取得する。出力部は、複数のグループとともに、グループ内容情報を対応するグループに対応付けて表示する複数グループ一覧情報を出力する。
上述してきた各実施形態に係るリモート会合サーバ100や端末装置10等の情報処理装置(情報機器)は、例えば図26に示すような構成のコンピュータ1000によって実現される。図26は、情報処理装置の機能を実現するコンピュータ1000の一例を示すハードウェア構成図である。以下、実施形態に係るリモート会合サーバ100を例に挙げて説明する。コンピュータ1000は、CPU1100、RAM1200、ROM(Read Only Memory)1300、HDD(Hard Disk Drive)1400、通信インターフェイス1500、及び入出力インターフェイス1600を有する。コンピュータ1000の各部は、バス1050によって接続される。
(1)
リモートでの会合に参加する複数のユーザが分かれ、各グループ内で会話できる複数のグループのうち、少なくとも1つ以上のグループでの会話の内容に関するグループ内容情報を取得する取得部と、
前記複数のグループとともに、前記グループ内容情報を対応するグループに対応付けて表示する複数グループ一覧情報を出力する出力部と、
を備える情報処理装置。
(2)
前記出力部は、
前記複数のグループのうち、少なくとも1つ以上のグループに対してコメントを行うためのアシスト情報を出力する
(1)に記載の情報処理装置。
(3)
前記出力部は、
前記アシスト情報として、過去に行われたコメントに関する情報を出力する
(2)に記載の情報処理装置。
(4)
過去に行われたコメントの履歴から対象コメントを抽出する抽出部、
を備える(3)に記載の情報処理装置。
(5)
前記抽出部は、
アシストの対象となる対象グループに関する情報に基づいて、前記対象グループに対応する前記対象コメントを抽出する
(4)に記載の情報処理装置。
(6)
前記抽出部は、
指定されたキーワードにより前記履歴を検索することにより、前記対象コメントを抽出する
(4)または(5)に記載の情報処理装置。
(7)
前記抽出部は、
前記履歴中の各コメントと前記キーワードとの類似度に基づいて、前記対象コメントを抽出する
(6)に記載の情報処理装置。
(8)
前記出力部は、
前記複数グループ一覧情報を端末装置に送信する、
(1)~(7)のいずれか1つに記載の情報処理装置。
(9)
前記出力部は、
前記複数グループ一覧情報を、前記会合を管理する管理者が利用する前記端末装置に送信する、
(8)に記載の情報処理装置。
(10)
リモートでの会合に参加する複数のユーザが分かれ、各グループ内で会話できる複数のグループのうち、少なくとも1つ以上のグループでの会話の内容に関するグループ内容情報を取得し、
前記複数のグループとともに、前記グループ内容情報を対応するグループに対応付けて表示する複数グループ一覧情報を出力する、
処理を実行する情報処理方法。
(11)
リモートでの会合に参加する複数のユーザが分かれ、各グループ内で会話できる複数のグループとともに、前記複数のグループのうち少なくとも1つ以上のグループでの会話の内容に関するグループ内容情報を対応するグループに対応付けて表示する複数グループ一覧情報を表示する表示部、
を備える端末装置。
(12)
前記複数グループ一覧情報に表示されたグループに関連する操作を、前記端末装置を利用するユーザから受け付ける入力部、
を備える(11)に記載の端末装置。
(13)
前記表示部は、
前記入力部により受け付けられた前記ユーザの操作に応じて、表示態様を変更する
(12)に記載の端末装置。
(14)
前記表示部は、
前記入力部により、前記ユーザへのアシスト情報の表示を行う操作が受け付けられた場合、前記アシスト情報を表示する
(13)に記載の端末装置。
(15)
前記表示部は、
前記アシスト情報として、過去のコメントに関する情報を表示する
(14)に記載の端末装置。
(16)
前記表示部は、
前記入力部により、前記グループ内容情報の表示態様の変更を行う操作が受け付けられた場合、前記グループ内容情報の表示態様を変更する
(12)~(15)のいずれか1つに記載の端末装置。
(17)
前記入力部により受け付けられた前記ユーザの操作に応じた処理を実行する処理部、
(12)~(16)のいずれか1つに記載の端末装置。
(18)
前記処理部は、
前記入力部により、前記複数のグループのうち、少なくとも1つ以上のグループに対してコメントを行う操作が受け付けられた場合、前記操作に対応するグループへコメントを送信する処理を実行する
(17)に記載の端末装置。
(19)
前記処理部は、
前記入力部により、前記複数のグループの全てに対してコメントを行う操作が受け付けられた場合、前記複数のグループの全てにコメントを送信する処理を実行する
(17)または(18)に記載の端末装置。
(20)
リモートでの会合に参加する複数のユーザが分かれ、各グループ内で会話できる複数のグループとともに、前記複数のグループのうち少なくとも1つ以上のグループでの会話の内容に関するグループ内容情報を対応するグループに対応付けて表示する複数グループ一覧情報を取得し、
前記複数グループ一覧情報を表示する、
処理を実行する表示方法。
100 リモート会合サーバ(情報処理装置)
110 通信部
120 記憶部
121 ユーザ情報記憶部
122 グループ情報記憶部
130 制御部
131 取得部
132 処理部
133 抽出部
134 生成部
135 送信部
10 端末装置(情報処理装置)
11 通信部
12 音声入力部(入力部)
13 音声出力部
14 カメラ
15 表示部
16 操作部(入力部)
17 記憶部
18 制御部
181 取得部
182 送信部
183 受信部
184 処理部
Claims (20)
- リモートでの会合に参加する複数のユーザが分かれ、各グループ内で会話できる複数のグループのうち、少なくとも1つ以上のグループでの会話の内容に関するグループ内容情報を取得する取得部と、
前記複数のグループとともに、前記グループ内容情報を対応するグループに対応付けて表示する複数グループ一覧情報を出力する出力部と、
を備える情報処理装置。 - 前記出力部は、
前記複数のグループのうち、少なくとも1つ以上のグループに対してコメントを行うためのアシスト情報を出力する
請求項1に記載の情報処理装置。 - 前記出力部は、
前記アシスト情報として、過去に行われたコメントに関する情報を出力する
請求項2に記載の情報処理装置。 - 過去に行われたコメントの履歴から対象コメントを抽出する抽出部、
を備える請求項3に記載の情報処理装置。 - 前記抽出部は、
アシストの対象となる対象グループに関する情報に基づいて、前記対象グループに対応する前記対象コメントを抽出する
請求項4に記載の情報処理装置。 - 前記抽出部は、
指定されたキーワードにより前記履歴を検索することにより、前記対象コメントを抽出する
請求項4に記載の情報処理装置。 - 前記抽出部は、
前記履歴中の各コメントと前記キーワードとの類似度に基づいて、前記対象コメントを抽出する
請求項6に記載の情報処理装置。 - 前記出力部は、
前記複数グループ一覧情報を端末装置に送信する、
請求項1に記載の情報処理装置。 - 前記出力部は、
前記複数グループ一覧情報を、前記会合を管理する管理者が利用する前記端末装置に送信する、
請求項8に記載の情報処理装置。 - リモートでの会合に参加する複数のユーザが分かれ、各グループ内で会話できる複数のグループのうち、少なくとも1つ以上のグループでの会話の内容に関するグループ内容情報を取得し、
前記複数のグループとともに、前記グループ内容情報を対応するグループに対応付けて表示する複数グループ一覧情報を出力する、
処理を実行する情報処理方法。 - リモートでの会合に参加する複数のユーザが分かれ、各グループ内で会話できる複数のグループとともに、前記複数のグループのうち少なくとも1つ以上のグループでの会話の内容に関するグループ内容情報を対応するグループに対応付けて表示する複数グループ一覧情報を表示する表示部、
を備える端末装置。 - 前記複数グループ一覧情報に表示されたグループに関連する操作を、前記端末装置を利用するユーザから受け付ける入力部、
を備える請求項11に記載の端末装置。 - 前記表示部は、
前記入力部により受け付けられた前記ユーザの操作に応じて、表示態様を変更する
請求項12に記載の端末装置。 - 前記表示部は、
前記入力部により、前記ユーザへのアシスト情報の表示を行う操作が受け付けられた場合、前記アシスト情報を表示する
請求項13に記載の端末装置。 - 前記表示部は、
前記アシスト情報として、過去のコメントに関する情報を表示する
請求項14に記載の端末装置。 - 前記表示部は、
前記入力部により、前記グループ内容情報の表示態様の変更を行う操作が受け付けられた場合、前記グループ内容情報の表示態様を変更する
請求項12に記載の端末装置。 - 前記入力部により受け付けられた前記ユーザの操作に応じた処理を実行する処理部、
請求項12に記載の端末装置。 - 前記処理部は、
前記入力部により、前記複数のグループのうち、少なくとも1つ以上のグループに対してコメントを行う操作が受け付けられた場合、前記操作に対応するグループへコメントを送信する処理を実行する
請求項17に記載の端末装置。 - 前記処理部は、
前記入力部により、前記複数のグループの全てに対してコメントを行う操作が受け付けられた場合、前記複数のグループの全てにコメントを送信する処理を実行する
請求項17に記載の端末装置。 - リモートでの会合に参加する複数のユーザが分かれ、各グループ内で会話できる複数のグループとともに、前記複数のグループのうち少なくとも1つ以上のグループでの会話の内容に関するグループ内容情報を対応するグループに対応付けて表示する複数グループ一覧情報を取得し、
前記複数グループ一覧情報を表示する、
処理を実行する表示方法。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP22791366.2A EP4329292A1 (en) | 2021-04-22 | 2022-02-24 | Information processing device, information processing method, terminal device, and display method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2021-072658 | 2021-04-22 | ||
JP2021072658 | 2021-04-22 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2022224584A1 true WO2022224584A1 (ja) | 2022-10-27 |
Family
ID=83722782
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2022/007572 WO2022224584A1 (ja) | 2021-04-22 | 2022-02-24 | 情報処理装置、情報処理方法、端末装置及び表示方法 |
Country Status (2)
Country | Link |
---|---|
EP (1) | EP4329292A1 (ja) |
WO (1) | WO2022224584A1 (ja) |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013205971A (ja) * | 2012-03-27 | 2013-10-07 | Fujitsu Ltd | グループ作業支援方法、グループ作業支援プログラム、グループ作業支援サーバ及びグループ作業支援システム |
JP2014191355A (ja) * | 2013-03-26 | 2014-10-06 | Oki Data Corp | 文字入力装置及び文字入力方法 |
JP2019168551A (ja) * | 2018-03-22 | 2019-10-03 | Kddi株式会社 | 研修におけるユーザ同士の議論内容を分析する装置、プログラム及び方法 |
-
2022
- 2022-02-24 WO PCT/JP2022/007572 patent/WO2022224584A1/ja active Application Filing
- 2022-02-24 EP EP22791366.2A patent/EP4329292A1/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013205971A (ja) * | 2012-03-27 | 2013-10-07 | Fujitsu Ltd | グループ作業支援方法、グループ作業支援プログラム、グループ作業支援サーバ及びグループ作業支援システム |
JP2014191355A (ja) * | 2013-03-26 | 2014-10-06 | Oki Data Corp | 文字入力装置及び文字入力方法 |
JP2019168551A (ja) * | 2018-03-22 | 2019-10-03 | Kddi株式会社 | 研修におけるユーザ同士の議論内容を分析する装置、プログラム及び方法 |
Non-Patent Citations (1)
Title |
---|
"Use breakout rooms in Teams meetings", 15 April 2021, MICROSOFT CORPORATION |
Also Published As
Publication number | Publication date |
---|---|
EP4329292A1 (en) | 2024-02-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20220036013A1 (en) | In-Call Experience Enhancement for Assistant Systems | |
US10891436B2 (en) | Device and method for voice-driven ideation session management | |
WO2023240943A1 (zh) | 生成数字人的方法、模型的训练方法、装置、设备和介质 | |
US11457061B2 (en) | Creating a cinematic storytelling experience using network-addressable devices | |
JP5123591B2 (ja) | 発想支援装置、発想支援システム、発想支援プログラム及び発想支援方法 | |
US11494434B2 (en) | Systems and methods for managing voice queries using pronunciation information | |
JP2018506113A (ja) | ユーザーエクスペリエンスを向上させるためにあいまいな表現を弁別すること | |
US20200403816A1 (en) | Utilizing volume-based speaker attribution to associate meeting attendees with digital meeting content | |
US10762906B2 (en) | Automatically identifying speakers in real-time through media processing with dialog understanding supported by AI techniques | |
WO2016173326A1 (zh) | 基于主题的交互系统及方法 | |
WO2018094952A1 (zh) | 一种内容推荐方法与装置 | |
US20210034662A1 (en) | Systems and methods for managing voice queries using pronunciation information | |
TW202230199A (zh) | 一同管理與語音檔有關的文本轉換記錄和備忘錄的方法、系統及電腦可讀記錄介質 | |
KR102135077B1 (ko) | 인공지능 스피커를 이용한 실시간 이야깃거리 제공 시스템 | |
CN113111658B (zh) | 校验信息的方法、装置、设备和存储介质 | |
TW202301080A (zh) | 輔助系統的多裝置調解 | |
US11410656B2 (en) | Systems and methods for managing voice queries using pronunciation information | |
US20240171418A1 (en) | Information processing device and information processing method | |
WO2022224584A1 (ja) | 情報処理装置、情報処理方法、端末装置及び表示方法 | |
JP7196393B2 (ja) | 情報提示装置、情報提示システム、情報提示方法およびプログラム | |
JP6756211B2 (ja) | 通信端末、音声変換方法、及びプログラム | |
KR20190094080A (ko) | 사용자간 대화 세션에 대한 모니터링에 기초하여 능동적으로 주문 또는 예약 서비스를 제공하는 대화형 ai 에이전트 시스템, 방법 및 컴퓨터 판독가능 기록 매체 | |
JPWO2019017027A1 (ja) | 情報処理装置および情報処理方法 | |
JP6962849B2 (ja) | 会議支援装置、会議支援制御方法およびプログラム | |
US11741298B1 (en) | Real-time meeting notes within a communication platform |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22791366 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 18554383 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2022791366 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
ENP | Entry into the national phase |
Ref document number: 2022791366 Country of ref document: EP Effective date: 20231122 |
|
NENP | Non-entry into the national phase |
Ref country code: JP |