JP2023047132A - Information processing device and information processing program - Google Patents

Abstract

To provide an information processing device and a program capable of determining a priority of a speaker of overlapped utterance when utterance overlaps during an online meeting.SOLUTION: An information processing device executes the steps of: determining whether or not there is overlapped utterance spoken by another speaker during utterance of one speaker in an online meeting; and determining a priority of a speaker who has performed overlapped utterance according to a predetermined rule when there is overlapped utterance.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing device and an information processing program.

In Patent Document 1, in a teleconferencing system in which a management server and participant terminals are connected via a network and in which simultaneous conversation is possible, when a plurality of conference participants speak at the same time, the management server has a function of equalizing opportunities to speak. A speech management system is disclosed comprising:

Patent Document 2 discloses a conference support system that supports the operation of a remote conference in which participants participate at each base, and at each base, voice and image information acquisition for acquiring voice and image information of utterances made by participants. a storage unit that stores voice and image information acquired for each site; a speaker determination unit that determines whether or not a plurality of participants have spoken at the same time or if there is an overlap in the speeches; A meeting support system is disclosed that includes a statement presenting unit that presents a plurality of statements of a type.

Patent Literature 3 discloses a speech opportunity equalization method for conducting a conference by transmitting and receiving voices of conference participants via a network, comprising a step of capturing the voices of the conference participants by voice capture means; a step of capturing an image of the conference participant by an input means; a step of detecting an action of the conference participant from the image by an action detecting means; calculating a next utterance recommendation value indicating the priority of the utterance according to the order in the speaker change history describing the speaker and the action detected in the detecting step; adjusting the sound pressure level of the voice of the conference participant based on the loudness; transmitting and receiving the sound pressure level-adjusted voice of the conference participant by means of transmission and reception; and selecting a conference participant who is currently speaking as a speaker and recording it in the speaker change history.

Japanese Patent Application Laid-Open No. 2003-304337 Japanese Patent Application Laid-Open No. 2006-229903 Japanese Patent No. 5432805

Speakers' remarks during an online meeting may overlap.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an information processing apparatus and program capable of determining the priority of speakers of duplicate statements when statements are duplicated during an online meeting.

An information processing apparatus according to a first aspect includes a processor, the processor determines whether or not there is a duplicate statement made by another speaker during a statement made by one speaker during an online meeting, and If there is, according to a predetermined rule, the priority of the speaker who made the duplicate remark is determined.

In the information processing apparatus according to the second aspect, the processor acquires voice data of an utterance made by a speaker during the online meeting, generates removed voice data by removing noise from the voice data, and removes noise from the voice data. to determine the priority.

In the information processing apparatus according to the third aspect, the processor determines when the speech of the one speaker starts and when it ends, and when the speech of another speaker occurs between the start time and the end time. If it is started, it is determined that the duplicate utterance has occurred.

In the information processing device according to the fourth aspect, the processor notifies the priority.

In the information processing apparatus according to the fifth aspect, the notification of the priority order is directed to predetermined participants among the participants of the online meeting.

In the information processing apparatus according to the sixth aspect, the notification of the priority order is displayed superimposed on the front side of the material displayed on the display unit in the online meeting.

In the information processing device according to the seventh aspect, the predetermined rule is a predetermined attribute of the speaker.

An information processing apparatus according to an eighth aspect, wherein the predetermined rule is a feature of the speech of the speaker.

In the information processing apparatus according to the ninth aspect, the predetermined rule is determined by weighting predetermined attributes of the speaker and features of the speaker's utterance.

In the information processing apparatus according to the tenth aspect, the attribute is position information of the speaker.

In the information processing apparatus according to the eleventh mode, the attribute is whether the speaker is a mandatory participant or an optional participant in the online meeting.

In the information processing device according to the twelfth aspect, the feature is speed in the duplicate utterances.

An information processing program according to a thirteenth aspect causes a computer to determine whether or not there is a duplicate statement made by another speaker during a statement made by one speaker during an online meeting, and if there is the duplicate statement, , to execute a process of determining the priority of the speaker who made the duplicate utterance according to a predetermined rule.

According to the first aspect, it is possible to determine the priority order of the speakers of the duplicated statements when the statements are duplicated during the online meeting.

According to the second aspect, there is an effect that priority can be determined without being affected by noise, compared to the case where noise is not removed.

According to the third aspect, it is possible to determine whether or not there is a duplicate utterance by determining whether or not another utterance has started between the start and end of the utterance.

According to the fourth aspect, it is possible to know the order in which speakers who have made duplicate statements can speak.

According to the fifth aspect, there is an effect that priority can be notified only to necessary participants.

According to the sixth aspect, there is an effect that it is possible to make it easier for the person who has been notified of the priority order to notice the notification, compared to the case of superimposing and displaying the information on the back side of the material.

According to the seventh aspect, there is an effect that priority can be determined according to the attribute of the speaker.

According to the eighth aspect, there is an effect that priority can be determined according to the features of the speaker.

According to the ninth aspect, there is an effect that priority can be determined according to the speaker's attributes and characteristics.

According to the tenth aspect, it is possible to determine the order of priority based on the position information.

According to the eleventh aspect, there is an effect that priority can be determined depending on whether the participant is a mandatory participant or an optional participant.

According to the twelfth aspect, there is an effect that priority can be given to the speech of the person who spoke earlier.

According to the thirteenth aspect, it is possible to provide an information processing program capable of determining the priority order of the speakers of the duplicated statements when the statements are duplicated during an online meeting.

1 is a schematic configuration diagram of an information processing system according to an embodiment of the present invention; FIG. 1 is a schematic block diagram of a PC according to an embodiment of the invention; FIG. 1 is a schematic block diagram of a server and an online meeting server according to embodiments of the invention; FIG. 4 is a flow chart showing an example of the flow of processing for determining and displaying the priority of speakers of duplicate utterances according to the embodiment of the present invention. It is an example of prior information registration of priority determination according to the embodiment of the present invention. It is an example of a case where the priority according to the embodiment of the present invention is determined by combining the prior information and the characteristics of duplicate utterances. It is an example of the setting of the notification of the priority according to the embodiment of the present invention. It is an example of the display of the display unit of the PC when notifying the priority according to the embodiment of the present invention. It is another example of the display of the display unit of the PC when notifying the priority according to the embodiment of the present invention. FIG. 4 is an explanatory diagram for explaining removal of noise from audio data according to the embodiment of the present invention;

(embodiment)
An example of an embodiment of the present disclosure will be described below with reference to the drawings. In each drawing, the same or equivalent components and portions are given the same reference numerals. Also, the dimensional ratios in the drawings are exaggerated for convenience of explanation, and may differ from the actual ratios.

An example of an information processing system 10 according to this embodiment will be described with reference to FIG.

FIG. 1 is a diagram showing an example of a schematic configuration of an information processing system 10 according to this embodiment.
As shown in FIG. 1, an information processing system 10 according to this embodiment includes a plurality of PCs (Personal Computers) 20, a server 50, and an online meeting server 60. FIG. PC 20 , server 50 and online meeting server 60 are connected via network N. For this network N, for example, a LAN (=Local Area Network), a WAN (=Wide Area Network), the Internet, etc. are applied. Here, in this embodiment, the server 50 is an example of an information processing device. Note that the online meeting server 60 may function as the server 50 .

The PC 20 is a terminal device used by participants participating in the online meeting in the information processing system 10, and includes a tablet PC and a smartphone. In this embodiment, one PC is used for each participant. In addition, each PC 20 stores information identifying participants to be used, such as a user ID. It is sent to server 60 . Therefore, it is possible to determine who is speaking during the online meeting.

The online meeting server 60 is a server computer that provides online meeting services in the information processing system 10 .

Further, in the information processing system 10 according to the present embodiment, when a participant is using the PC 20 to connect to the online meeting server 60 and is holding an online meeting, the server 50 receives the speech of the participant from the PC 20 or the online meeting server 60 . , and determines the order of priority of the speakers of the duplicated utterances when the utterances of multiple participants overlap. In other words, in an online meeting, there are times when multiple participants' remarks overlap. This prevents the possibility of online meetings becoming inefficient, as they may hesitate to speak to avoid repetitive remarks.

FIG. 2 is a block diagram showing the hardware configuration of the PC 20 according to this embodiment.

As shown in FIG. 2, the PC 20 includes a CPU (Central Processing Unit) 21, which is an example of a processor, a ROM (Read Only Memory) 22, a RAM (Random Access Memory) 23, a storage section 24, an input section 25, and a display section 26. , a camera 27 , a microphone 28 , a speaker 29 and a communication interface 30 . Each component is communicably connected to each other via a bus 31 .

The CPU 21 is a central processing unit that executes various programs and controls each section. That is, the CPU 21 reads a program from the ROM 22 or the storage unit 24 and executes the program using the RAM 23 as a work area. The CPU 21 performs control of the above components and various arithmetic processing according to programs recorded in the ROM 22 or the storage unit 24 . In this embodiment, a program is stored in the ROM 22 or the storage unit 24 .

The ROM 22 stores various programs and various data. The RAM 23 temporarily stores programs or data as a work area. The storage unit 24 is configured by a HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various programs including an operating system and various data.

The input unit 25 includes a pointing device such as a mouse and a keyboard, and is used for various inputs. The input unit 25 is used for inputting advance information for determining the priority order of duplicate utterances, as will be described later.

The display unit 26 is, for example, a liquid crystal display. The display unit 26 displays various information under the control of the CPU 21 .

A camera 27 photographs the participants participating in the online meeting.

A microphone 28 picks up speeches of participants participating in the online meeting.

The speaker 29 outputs speeches of other participants participating in the online meeting using other PCs 20 connected via the network N. FIG.

The communication interface 30 is connected to the network N, and communicates the PC 20 with other PCs 20 and other devices such as the server 50 .

FIG. 3 is a block diagram showing the hardware configuration of the server 50 and the online meeting server 60. As shown in FIG. Since the server 50 and the online meeting server 60 are basically general computer configurations, the server 50 will be described as a representative.

As shown in FIG. 3 , the server 50 has a CPU 51 , ROM 52 , RAM 53 , storage section 54 and communication interface 55 . Each component is communicatively connected to each other via a bus 56 . Note that an input unit such as a keyboard and a display unit such as a display may be provided.

The CPU 51 is a central processing unit that executes various programs and controls each section. That is, the CPU 51 reads a program from the ROM 52 or the storage unit 54 and executes the program using the RAM 53 as a work area. The CPU 51 performs control of the above components and various arithmetic processing according to programs recorded in the ROM 52 or the storage unit 54 . In this embodiment, programs and various data are stored in the ROM 52 or the storage unit 54 .

The ROM 52 stores various programs and various data. The RAM 53 temporarily stores programs or data as a work area. The storage unit 54 is configured by an HDD or SSD, and stores various programs including an operating system and various data. Various data transmitted from the PC 20 and the online meeting server 60 are stored in the storage unit 54 of the server 50 .

The communication interface 55 is for communicating with other devices such as the PC 20 and the online meeting server 60. For example, standards such as the Internet, intranet, Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark) are used. Used.

Next, operation of the information processing system 10 will be described.

FIG. 4 is a flow chart showing an example of the flow of processing in which the server 50 determines the priority of speakers who make duplicate utterances and notifies them. Such processing may be performed by the online meeting server 60 instead of the server 50. FIG.

As a premise of this flow chart, the online meeting manager, moderator, participants, etc. perform the predetermined settings shown in FIGS. 5 to 7 before the online meeting starts. It should be noted that the setting is not limited to the case of making such settings for each online meeting, and settings of past online meetings may be used as they are or partially.

First, the server 50 and the online meeting server 60 are connected. Through such a connection, the server 50 acquires speech data of the speech made by the speaker from the online meeting server 60 . The audio data is not limited to being obtained from the online meeting server 60, and may be obtained from the PC 20. FIG.

Also, as shown in FIG. 5, the online meeting manager or moderator accesses the server 50 from the PC 20 and registers prior information to the server 50 for determining the order of priority of duplicate utterances. Prior information is an attribute of a participant that is an example of a predetermined rule for determining priority of duplicate statements. As shown in FIG. 5, the attributes of participants include, for example, "post information", "evaluation performance", "mandatory participant", "optional participant", and "priority speaker". "Position information" is information related to a position such as a manager or a group leader, and priority is given to remarks made by superiors among duplicate remarks. This has been adopted as a participant attribute to determine priorities, since superiors are more likely to make important remarks. In addition, even with the same position, employees with high evaluation results are more likely to make more useful comments than employees with low evaluation results. Adopted. In addition, since there is a possibility that the importance of remarks of "mandatory participants" is higher than that of "optional participants," used as an attribute. In addition to the attributes described above, an input field for "preferred speaker" is provided so that it is possible to register in advance an employee who particularly wishes to speak.

Note that the attributes of the participants are not limited to those shown in FIG. 5, and other attributes may be used. Also, the priority of utterances may be determined by using only one of the attributes shown in FIG. 5, or may be determined by combining a plurality of attributes. A mode of determining the priority of utterances by combining a plurality of such attributes will be described later with reference to FIG. Also, if there is an existing database in which internal organization information such as mailers and groupware is registered, such attributes of the participants may be obtained from the information in the database. If attributes of participants are acquired from an existing database in this way, it is possible to reduce the burden of inputting advance information for each online meeting.

Also, the priority of duplicate utterances may be determined based on the characteristics of duplicate utterances other than the prior information. For example, they are "speech speed", "loudness of voice", and "magnitude of gesture/hand gesture". "Speech speed" gives priority to an utterance whose utterance timing is early among duplicate utterances. "Volume of voice" gives priority to utterances with a loud voice among duplicate utterances. For the "size of gesture/hand gesture", priority is given to remarks in which the speaker's gesture or hand gesture captured by the camera 27 is large among duplicate utterances. Since it is presumed that the speaker is confident in the content of the statement or that he or she wants to make a strong statement, it is adopted as a characteristic of repeated statements that determine priority. Note that the characteristics at the time of repeated utterances are not limited to the characteristics described above, and other characteristics may be used. Also, the priority of utterances may be determined by using only one of the features described above, or the priority of utterances may be determined by combining a plurality of features. Also, the priority of utterances may be determined in combination with the prior information described above. A mode of determining the priority of utterances by combining such prior information and the characteristics of duplicate utterances will be described later with reference to FIG.

With reference to FIG. 6, a manner of determining the priority of utterances by combining the prior information and the characteristics of duplicate utterances will be described. In such a case, as shown in FIG. 6, the manager or moderator of the online meeting accesses the server 50 from the PC 20 and sends the server 50 prior information priority (weighting) for determining the priority of duplicate utterances. ). In the example shown in FIG. 6, the priority "1" is "position information", the priority "2" is "priority speaker", the priority "3" is "mandatory participant", and the priority "4" is "speech Priority "5" is "Voice volume" Priority "6" is "Evaluation performance" Priority "7" It is a “voluntary participant”. For example, first, the “position information” of the duplicate utterances with the priority “1” is referred to, and the utterances of the top utterances of the position information are prioritized. If the position information is the same, the "priority speaker" with priority "2" is referred to, and the speech of the speaker with the higher rank is prioritized. If the ranks are the same, refer to the priority "3" "mandatory participant", and if only one of the speakers of the duplicate remarks is a mandatory participant, the remarks of the mandatory participant Prioritize. If there are multiple mandatory participants, reference is made to the priority "4" and "speech speed". In this way, the order of priority of statements of duplicate statements is determined according to the priority. Such priority determination is performed in step S140, which will be described later.

Also, as shown in FIGS. 7 to 9, the online meeting manager or moderator accesses the server 50 from the PC 20 and makes settings for notification of the decided speaker's priority order. For example, as shown in FIG. 7, the settings for notification of priority include "setting of notification destination" and "setting of notification method".

"Setting of notification destination" is a setting of who to notify among the participants of the online meeting. In the example shown in FIG. 7, there are "everyone", "moderator and duplicate speakers only", and "designated participants". "Everyone" is a setting for notifying all participants of the online meeting. "Only the moderator and duplicate speakers" is a setting that notifies only the moderator of the online meeting and the person who made duplicate statements. "Specified participants" is a setting for notifying specified participants among the participants.

"Notification method setting" is a setting for how to notify during an online meeting. In the example shown in FIG. 7, there are "list display" and "mark on face photo". As shown in FIG. 8, the "list display" displays the name and priority of "priority 1", "priority 2" on the display unit 26 of the PC 20 of the participant set as the notification destination in the "notification destination setting". ” and so on. Here, it is desirable that the names and priorities are displayed superimposed on the front side of the online meeting materials S displayed on the display unit 26 . Participants in the online meeting often pay attention to the material S, and by displaying the material S in front of the attention-grabbing material S, it is possible to prevent overlooking of the priority compared to displaying in other places. making it possible. As shown in FIG. 9, the "mark on face photo" indicates the priority of the participant's face photo G prepared in advance and the face image of the participant taken by the camera 27 during the online meeting. ” is a setting to display numerical values. Here, it is desirable that the face photograph G and the priority order are displayed superimposed on the front side of the online meeting material S displayed on the display unit 26 . By superimposing it on the front side of the material S, it is possible to prevent overlooking of the priority.

After performing the predetermined settings shown in FIGS. 5 to 7, the process of the flowchart shown in FIG. 4 is started. Note that the predetermined settings shown in FIGS. 5 to 7 are not limited to being performed before the start of the online meeting, and may be performed during the online meeting or changed during the online meeting. good.

In step S100 shown in FIG. 4, the CPU 51 of the server 50 acquires voice data spoken by any participant participating in the online meeting. Such voice data is obtained from the online meeting server 60 one by one. Also, the voice data includes information for identifying the participant who has spoken. Then, it is determined whether or not there is a duplicate utterance. Here, whether or not there is a duplicate utterance is determined by judging the start and end times of the utterance of one speaker. Determine that there is a statement. The start time and end time are calculated based on the time managed by the online meeting server 60 . That is, the start time and the end time of a statement are obtained, and if the start time of another statement is included between these times, the statements are determined to be duplicate statements. It should be noted that the timing at which the server 50 starts acquiring voice data from the online meeting server 60 is not limited to the case where it is determined that there is a duplicate utterance from the start time and end time of the utterance, but the start time of the utterance is determined. Alternatively, the timing at which acquisition of voice data ends may be determined as the end of an utterance, and when acquisition of other voice data is started during that timing, the utterances may be determined as duplicate utterances. . Then, the process proceeds to the next step S110.

If it is not determined in step S110 that there was a duplicate utterance in step S100, the process proceeds to next step S120.

In step S120, the CPU 51 of the server 50 determines whether or not the online meeting has ended. If it is determined that the online meeting has ended, the processing ends. If it is not determined that the online meeting has ended, the process returns to step S100.

When the CPU 51 of the server 50 determines in the above-described step S110 that there is a duplicate utterance, the CPU 51 of the server 50 proceeds to the next step S130 to generate removed voice data by removing noise from the voice data. Here, an example of noise removal will be described with reference to FIG. For noise removal, as shown in FIG. 10, the audio data is first frequency-converted. Next, from the frequency-converted audio data, for example, pop noise that frequently occurs in online meetings is removed. The noise to be removed is not limited to pop noise, and may be other noise. Thus, noise-removed audio data is generated. Then, the process proceeds to the next step S140. Here, noise removal is performed for priority determination in step S140, which will be described later, but is not limited to this. It may be performed for the purpose of determination.

In step S140, the CPU 51 of the server 50 identifies the speaker from the information for identifying the participant who made the speech contained in the voice data. Then, according to the predetermined rules for determining the priority order of duplicate utterances shown in FIGS. 5 and 6, the priority order of speakers who made duplicate utterances is determined. Then, the process proceeds to the next step S150.

In step S150, the CPU 51 of the server 50 notifies the speaker's priority determined in step S140 as shown in FIG. 8 or 9 according to the setting shown in FIG. Then, the process proceeds to the next step S120 described above.

The present invention is not limited to the embodiments described above, and various modifications and applications are possible without departing from the gist of the present invention.

In addition, in the above-described embodiment, a mode in which the program is pre-stored (installed) in the ROM or the storage unit has been described, but the present invention is not limited to this. The program may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), and a USB (Universal Serial Bus) memory. Also, the program may be downloaded from an external device via the network N. FIG.

Further, in the above-described embodiments, the processor refers to a processor in a broad sense, and includes a general-purpose processor (eg, CPU: Central Processing Unit, etc.) and a dedicated processor (eg, GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

In addition, the operations of the processors in the above-described embodiments may be performed not only by one processor but also by cooperation of a plurality of physically separated processors. Also, the order of each operation of the processor is not limited to the order described in each of the above-described embodiments, and may be changed as appropriate.

10 information processing system 20 PC
50 Server 60 Online Meeting Server

Claims (13)

with a processor
The processor
Determining whether or not there was a duplicate statement made by another speaker during the statement of one speaker during the online meeting,
An information processing device that, when there is the duplicate statement, determines the priority of the speaker who made the duplicate statement according to a predetermined rule. The processor
Acquiring audio data of a speech made by a speaker during the online meeting,
generating removed audio data by removing noise from the audio data;
2. The information processing apparatus according to claim 1, wherein said priority is determined from said removed audio data. The processor
Determining the start and end times of the speech of the one speaker,
3. The information processing apparatus according to claim 1, wherein it is determined that said duplicate utterance has occurred when another utterance starts between said start time and said end time. The processor
4. The information processing apparatus according to any one of claims 1 to 3, wherein the priority order is notified. 5. The information processing apparatus according to claim 4, wherein notification of said priority order is directed to predetermined participants among participants of said online meeting. 6. The information processing apparatus according to claim 4, wherein the notification of the priority order is superimposed on the front side of the material displayed on the display unit in the online meeting. 7. The information processing apparatus according to any one of claims 1 to 6, wherein said predetermined rule is a predetermined attribute of said speaker. 7. The information processing apparatus according to any one of claims 1 to 6, wherein said predetermined rule is a feature of said speaker's utterance. 7. The predetermined rule according to any one of claims 1 to 6, wherein said predetermined attribute of said speaker and characteristics of said speaker's utterance are determined by predetermined weighting. Information processing equipment. 10. The information processing apparatus according to claim 7, wherein said attribute is position information of said speaker. 10. The information processing apparatus according to claim 7, wherein the attribute is whether the speaker is a mandatory participant or an optional participant in the online meeting. 10. The information processing apparatus according to claim 8, wherein said feature is speed in said duplicate utterances. to the computer,
Determining whether or not there was a duplicate statement made by another speaker during the statement of one speaker during the online meeting,
An information processing program for executing a process of determining the priority of a speaker who made the duplicate utterance according to a predetermined rule when the duplicate utterance occurs.

Images