WO2023032461A1 - 会議支援システム、会議支援方法、および会議支援プログラム - Google Patents

会議支援システム、会議支援方法、および会議支援プログラム Download PDF

Info

Publication number
WO2023032461A1
WO2023032461A1 PCT/JP2022/026624 JP2022026624W WO2023032461A1 WO 2023032461 A1 WO2023032461 A1 WO 2023032461A1 JP 2022026624 W JP2022026624 W JP 2022026624W WO 2023032461 A1 WO2023032461 A1 WO 2023032461A1
Authority
WO
WIPO (PCT)
Prior art keywords
conference
content
user
time
online
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/026624
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
昭彦 戀塚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dwango Co Ltd
Original Assignee
Dwango Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Dwango Co Ltd filed Critical Dwango Co Ltd
Priority to US18/292,257 priority Critical patent/US20240388462A1/en
Priority to CN202280045941.XA priority patent/CN117581528A/zh
Publication of WO2023032461A1 publication Critical patent/WO2023032461A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00—Data switching networks
    • H04L12/02—Details
    • H04L12/16—Arrangements for providing special services to substations
    • H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
    • H04L12/1813—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for computer conferences, e.g. chat rooms
    • H04L12/1831—Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L17/00—Speaker identification or verification techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M3/00—Automatic or semi-automatic exchanges
    • H04M3/42—Systems providing special services or facilities to subscribers
    • H04M3/56—Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/15—Conference systems

Definitions

  • One aspect of the present disclosure relates to a conference support system, a conference support method, and a conference support program.
  • Patent Document 1 There is a well-known mechanism that supports the understanding of meeting content in online meetings over a network.
  • conference information is recorded while the conference is in progress, and when a midway participant in the conference is detected, a summary of the conference information up to that point is created, and the created summary is sent to the midway participant.
  • a network conferencing system is described that provides individual access to.
  • Patent Literature 2 describes a teleconferencing system that rewinds and reproduces the video or audio of an electronic conference participant's speech when the participant misses the speech.
  • a meeting support system includes at least one processor. At least one processor records meeting data including audio of the online meeting, acquires a point in time prior to the online meeting from the user's terminal, generates content corresponding to the meeting data in the time span after the point in time, and performs the online meeting. In progress, content is played back on the user's terminal at a faster playback speed than the original playback speed of the conference data.
  • content is generated that corresponds to meeting data after the online meeting has been traced back.
  • the content is then rapidly played back on the user's terminal to catch up with the ongoing online conference.
  • FIG. 10 is a diagram showing an example of a playback screen;
  • the example (a) of FIG. 5 is an example of a playback screen, which is one frame forming the first half of the content.
  • the example (b) of FIG. 5 is an example of a playback screen, which is one frame constituting the second half of the content.
  • 4 is a sequence diagram showing operations of the conference support system according to the embodiment;
  • FIG. FIG. 10 is a diagram showing an example of a playback screen;
  • the example (a) of FIG. 5 is an example of a playback screen, which is one frame forming the first half of the content.
  • the example (b) of FIG. 5 is an example of a playback screen, which is one frame constituting the second half of the content.
  • 4 is a sequence diagram showing operations of the conference support system according to the embodiment;
  • FIG. 11 is a diagram showing an example of a functional configuration related to a conference support system according to another embodiment;
  • FIG. It is a figure which shows another example of a conference screen.
  • Example (a) of FIG. 8 is an example of a conference screen displaying status and progress.
  • Example (b) of FIG. 8 is another example of a conference screen displaying status and progress.
  • FIG. 10 is a sequence diagram showing operations of a conference support system according to another embodiment;
  • a conference support system is a computer system that supports a user of an online conference.
  • An online conference is a conference held via a plurality of user terminals connected to a network, and is also called a web conference or network conference.
  • a user is a person who uses the conference support system.
  • a user terminal is a computer used by one or more users. "Supporting the user" is performed by providing the user with the progress of the online conference prior to the current point in time as content.
  • Content refers to data from which a person can perceive some information at least aurally.
  • the content may be a moving image (video) including audio, or may be audio only. Provisioning refers to the process of transmitting information to a user terminal via a network.
  • the conference support system obtains from the user terminal a request specifying the point in time to go back to the online conference.
  • the point in time preceding the online conference is the point in time at which content reproduction is started (hereinafter referred to as "content start point").
  • the conference support system generates content data, which is electronic data representing the content, based on the content start time and electronic data recording the online conference, and transmits the content data to the user terminal.
  • the user terminal receives and processes the content data and performs high-speed chasing playback of the content. Chasing playback refers to the function of playing back voice being recorded or moving images being recorded with a delay.
  • ⁇ Contents (progress) of the meeting (online conference) before the current time'' is the first range from the content start time to the time when the content start time is specified (in other words, the time when chasing playback is instructed). including the conduct of meetings in The real-time conference continues while the chasing playback of the content corresponding to the first range is performed.
  • Conference (online conference) content (progress) before the current point in time” further indicates the progress of the conference in the second range from the point at which the content start point is specified (point at which follow-up playback is instructed) to the current point. can contain.
  • the progress of the conference in the second range is the content of the conference that continues during chasing playback.
  • FIG. 1 is a diagram showing an example of application of the conference support system 1.
  • the conference support system 1 has a server 10 .
  • the server 10 is a computer (meeting support server) that transmits content to at least one user terminal 20 .
  • the server 10 connects with a plurality of user terminals 20 via a communication network N.
  • FIG. Although five user terminals 20 are shown in FIG. 1, the number of user terminals 20 is not limited.
  • the configuration of the communication network N is not limited.
  • the communication network N may include the Internet, or may include an intranet.
  • the type of user terminal 20 is not limited.
  • the user terminal 20 may be a mobile terminal such as a high-performance mobile phone (smartphone), tablet terminal, wearable terminal (eg, head-mounted display (HMD), smart glasses, etc.), laptop personal computer, or mobile phone.
  • the user terminal 20 may be a stationary terminal such as a desktop personal computer.
  • the content in this disclosure is a moving image in which a live-action image and sound are combined.
  • a photographed image is an image of the real world, and is obtained by an imaging device such as a camera.
  • the conference support system 1 may be used for various purposes.
  • the conference support system 1 may be used for television conferences (video conferences), online seminars, or the like. That is, the conference support system 1 may be used as a means of communication for sharing moving images among a plurality of users.
  • the conference support system 1 may be used for teleconferencing or the like in which only audio is shared.
  • FIG. 2 is a diagram showing an example of a hardware configuration related to the conference support system 1.
  • the server 10 includes a processor 101, a main storage section 102, an auxiliary storage section 103, and a communication section 104 as hardware components.
  • the processor 101 is a computing device that executes an operating system and application programs. Examples of processors include CPUs (Central Processing Units) and GPUs (Graphics Processing Units), but the type of processor 101 is not limited to these.
  • CPUs Central Processing Units
  • GPUs Graphics Processing Units
  • the main storage unit 102 is a device that stores programs for causing the server 10 to function, calculation results output from the processor 101, and the like.
  • the main storage unit 102 is composed of, for example, at least one of ROM (Read Only Memory) and RAM (Random Access Memory).
  • the auxiliary storage unit 103 is generally a device capable of storing a larger amount of data than the main storage unit 102.
  • the auxiliary storage unit 103 is configured by a non-volatile storage medium such as a hard disk or flash memory.
  • the auxiliary storage unit 103 stores a server program P1 for causing at least one computer to function as the server 10 and various data.
  • the conference support program is implemented as a server program P1.
  • the communication unit 104 is a device that performs data communication with other computers via the communication network N.
  • the communication unit 104 is configured by, for example, a network card or a wireless communication module.
  • Each functional element of the server 10 is realized by loading the server program P1 onto the processor 101 or the main storage unit 102 and executing the program.
  • the server program P1 includes codes for realizing each functional element of the server 10.
  • the processor 101 operates the communication unit 104 according to the server program P1 to read and write data in the main storage unit 102 or the auxiliary storage unit 103 .
  • Each functional element of the server 10 is realized by such processing.
  • the server 10 can be composed of one or more computers. When a plurality of computers are used, these computers are connected to each other via a communication network N to logically configure one server 10 .
  • the user terminal 20 includes a processor 201, a main storage unit 202, an auxiliary storage unit 203, a communication unit 204, an input interface 205, an output interface 206, and an imaging unit 207 as hardware components.
  • the processor 201 is a computing device that executes an operating system and application programs.
  • Processor 201 can be, for example, a CPU or GPU, but the type of processor 201 is not limited to these.
  • the main storage unit 202 is a device that stores programs for making the user terminal 20 function, calculation results output from the processor 201, and the like.
  • the main storage unit 202 is composed of, for example, at least one of ROM and RAM.
  • the auxiliary storage unit 203 is generally a device capable of storing a larger amount of data than the main storage unit 202.
  • the auxiliary storage unit 203 is configured by a non-volatile storage medium such as a hard disk or flash memory.
  • the auxiliary storage unit 203 stores a client program P2 and various data for causing the computer to function as the user terminal 20 .
  • the communication unit 204 is a device that performs data communication with other computers via the communication network N.
  • the communication unit 204 is configured by, for example, a network card or a wireless communication module.
  • the input interface 205 is a device that accepts data based on user's operations or actions.
  • the input interface 205 is composed of at least one of a keyboard, operation buttons, pointing device, microphone, sensor, and camera.
  • a keyboard and operation buttons may be displayed on the touch panel. Since the type of input interface 205 is not limited, data to be input is not limited.
  • input interface 205 may accept data entered or selected by a keyboard, operating buttons, or pointing device.
  • input interface 205 may accept voice data input via a microphone.
  • the input interface 205 may accept, as motion data, data representing non-verbal actions of the user (eg, eye gaze, gestures, facial expressions, etc.) detected by a motion capture function using a sensor or camera.
  • the output interface 206 is a device that outputs data processed by the user terminal 20 .
  • the output interface 206 is composed of at least one of a monitor, touch panel, HMD and speaker.
  • Display devices such as monitors, touch panels, and HMDs display the processed data on their screens.
  • a speaker outputs the sound indicated by the processed audio data.
  • the imaging unit 207 is a device that captures an image of the real world, and is specifically a camera.
  • the imaging unit 207 may shoot a moving image (video) or may shoot a still image (photograph).
  • the imaging unit 207 processes the video signal based on a given frame rate to obtain a series of frame images arranged in time series as a moving image.
  • the imaging unit 207 can also function as the input interface 205 .
  • Each functional element of the user terminal 20 is realized by loading the client program P2 onto the processor 201 or the main storage unit 202 and executing the program.
  • the client program P2 includes codes for realizing each functional element of the user terminal 20.
  • FIG. The processor 201 operates the communication unit 204, the input interface 205, the output interface 206, or the imaging unit 207 according to the client program P2, and reads and writes data in the main storage unit 202 or the auxiliary storage unit 203.
  • FIG. Each functional element of the user terminal 20 is implemented by this process.
  • At least one of the server program P1 and the client program P2 may be provided after being fixedly recorded on a tangible recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory.
  • a tangible recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory.
  • at least one of these programs may be provided over the communication network N as a data signal superimposed on a carrier wave. These programs may be provided separately or together.
  • FIG. 3 is a diagram showing an example of a functional configuration related to the conference support system 1.
  • the server 10 includes a conference control unit 11, a recording unit 12, a request reception unit 13, a content generation unit 14, and an output unit 15 as functional elements.
  • the conference control unit 11 is a functional element that controls display of an online conference on the user terminal 20 .
  • the recording unit 12 is a functional element that records conference data including audio of the online conference.
  • the request receiving unit 13 is a functional element that receives from the user terminal 20 a content generation request including a content start point.
  • the content generation unit 14 is a functional element that generates content data based on the content start point and conference data.
  • the content data has a time span from when the content starts until it catches up with the real-time conference.
  • Content data is, for example, one or more data in streaming format.
  • the output unit 15 is a functional element that transmits content data to the user terminal 20 .
  • the user terminal 20 includes a conference display unit 21, a request transmission unit 22, and a content reproduction unit 23 as functional elements.
  • the conference display unit 21 is a functional element that displays an online conference in cooperation with the conference control unit 11 of the server 10 .
  • the request transmission unit 22 is a functional element that transmits a content generation request to the server 10 .
  • the content reproduction unit 23 is a functional element that reproduces content data received from the server 10 .
  • the conference database 30 is a non-temporary storage medium or storage device that stores conference data, which is electronic data of online conferences.
  • Conference data in the present disclosure is a moving image including audio of an online conference.
  • the conference data may further include user identification information that identifies the user who is the speaker of the voice.
  • FIG. 4 is a diagram showing an example of the conference screen 300.
  • the conference screen 300 is a screen that displays an ongoing online conference in real time.
  • the conference screen 300 is displayed on the user terminal 20 of each user participating in the online conference.
  • the conference screen 300 is displayed on each user terminal 20 of four users (User A, User B, User C and User D).
  • the conference screen 300 includes, for example, display areas 301 to 304, name display fields 301A to 304A, a time point input field 305, and a chasing playback button 306.
  • Display areas 301 to 304 are screen areas for displaying moving images of the user.
  • a moving image of the user is a moving image of the user captured by the user terminal 20 .
  • the number of display areas 301-304 corresponds to the number of users. For example, four display areas 301-304 each display moving images of four users. As the number of users increases or decreases, the display area also increases or decreases.
  • the display areas 301 to 304 may display one frame image forming a moving image, or may display one still image.
  • the display areas 301 to 304 may be highlighted when the displayed user is speaking.
  • Name display columns 301A to 304A are screen areas that display the names of users participating in the online conference.
  • a user's name may be set by accepting user input when joining an online meeting.
  • the user's name may be recorded in the conference database 30 as user identification information.
  • the name display fields 301A-304A correspond to the display areas 301-304 one-to-one, respectively. For example, a moving image of user A is displayed in display area 301, and the name of user A is displayed in name display column 301A.
  • the time input field 305 is a screen element for accepting user input regarding the content start time.
  • the time point input field 305 accepts an input operation or a selection operation of the content start time, such as five minutes before.
  • a follow-up playback button 306 is a screen element for performing follow-up playback from the content start point entered in the point input field 305 .
  • the mode of the point-in-time input field 305 and the follow-up playback button 306 is not limited to this.
  • the follow-up playback button 306 may be displayed alone with the content start time as a fixed value.
  • the display of the conference screen 300 is controlled by cooperation between the conference control unit 11 of the server 10 and the conference display unit 21 of the user terminal 20.
  • the conference display unit 21 captures a moving image of the user and transmits the moving image and user identification information to the server 10 .
  • the conference control unit 11 generates a conference screen 300 based on the moving images and user identification information received from a plurality of user terminals 20, and transmits the conference screen 300 to each user's user terminal 20.
  • the conference display unit 21 processes the received conference screen 300 and displays it on the display device.
  • FIG. 5 is a diagram showing an example of the playback screen 400.
  • the playback screen 400 is a screen that displays the progress of an online conference in the past. More specifically, the playback screen 400 is a screen that displays the progress of the past online conference recorded from the start of the content until it catches up with the real-time progress.
  • the playback screen 400 is displayed on the user terminal 20 triggered by, for example, pressing the chasing playback button 306 on the conference screen 300 .
  • the user may miss the content of the conference or simply want to hear the content of the conference again due to various reasons such as being away from the desk or communication failure of the communication network N, for example. In such a case, the user checks the content of the conference by chasing and reproducing the content.
  • the playback screen 400 is displayed on the user terminal 20 of the user D who has temporarily left the conference.
  • Example (a) of FIG. 5 shows, as an example of the playback screen 400, a playback screen 400A that is one frame constituting the first half of the content.
  • the playback screen 400A is a screen for grasping the content of the meeting that has passed.
  • the playback screen 400A includes display areas 401-404, name display columns 401A-404A, a playback speed column 405, an operation interface 406, a playback time column 407, and a progress bar 408.
  • the display areas 401-404 and the name display columns 401A-404A correspond to the display areas 301-304 and the name display columns 301A-304A of the conference screen 300, respectively.
  • the display area 401 is highlighted by a double frame, and the user D is not displayed in the display area 404 . That is, playback screen 400A indicates that user A is speaking and user D is away.
  • the playback speed column 405 is a screen element that displays the content playback speed.
  • the playback speed of the content is a playback speed faster than the original playback speed of the conference data.
  • the original playback speed means that the playback speed of the conference data has not been changed.
  • the playback speed of the content is, for example, n times the original playback speed (n>1.0). In one example, the playback speed of the content is 2.0x.
  • the playback speed field 405 may accept user input regarding changing the playback speed of the content.
  • the operation interface 406 is a user interface for performing various operations related to content reproduction.
  • the operation interface 406 receives operations from the user regarding, for example, playback/pause switching, cueing, and the like.
  • a playback time column 407 is a screen element that displays the elapsed time from the start of content playback.
  • a progress bar 408 is a screen element that displays the progress rate of content over time. That is, the playback time column 407 and progress bar 408 indicate the playback position of the content.
  • the example (b) of FIG. 5 shows, as an example of the playback screen 400, a playback screen 400B that is one frame constituting the second half of the content.
  • the playback screen 400B is a screen for grasping the content of the conference that continues to progress during chasing playback.
  • Playback screen 400B has a playback position indicated by playback time column 407 and progress bar 408 after playback screen 400A. In other words, playback screen 400B indicates that time has passed since playback screen 400A.
  • a moving image of user D is displayed in a display area 404 . This indicates that User D has returned to his seat.
  • a playback screen 400B shows an online conference while user D is playing back content. In other words, the playback screen 400B shows that users A, B, and C are having an online conference, while user D, who is playing content, is not participating in the conference.
  • FIG. 6 is a sequence diagram showing the operation of the conference support system 1 as a processing flow S1.
  • four users User A, User B, User C and User D
  • the conference control unit 11 of the server 10 and the conference display unit 21 of the user terminal 20 cooperate to display the conference screen 300 (see FIG. 4) on each of the user terminals 20 of the four users.
  • step S11 the recording unit 12 of the server 10 records the video including the voice of the online conference in the conference database 30 as conference data.
  • the recording unit 12 continuously records the conference data as the online conference progresses.
  • Conference data may further include user identification information.
  • the server 10 receives moving images shot at the same time from the user terminals 20 of each user. Therefore, the recording unit 12 can specify the correspondence relationship between the voice and the user identification information at a certain time. The recording unit 12 associates this correspondence with the conference data in chronological order and records them in the conference database 30 .
  • step S12 the conference display unit 21 of the user terminal 20 receives user input regarding the content start time.
  • the conference display unit 21 receives user input regarding the content start point through the point input field 305 of the conference screen 300 .
  • the meeting display unit 21 accepts a user input that sets the content start point to five minutes before.
  • step S13 the request transmission unit 22 of the user terminal 20 transmits to the server 10 a content generation request including the content start time point (the time point before the online conference).
  • the request transmission unit 22 acquires the content start time point entered in the time point input field 305 by using the follow-up playback button 306 as a trigger.
  • the request transmission unit 22 generates a content generation request including the content start point and transmits the content generation request to the server 10 .
  • the request receiving unit 13 of the server 10 acquires the content start time by receiving the content generation request.
  • step S14 the content generation unit 14 of the server 10 reads the conference data in the time span after the content start point from the conference database 30, and generates content data corresponding to the conference data.
  • the content generation unit 14 generates content data corresponding to meeting data from 5 minutes ago.
  • the content data generation method and data structure are not limited.
  • the content generation unit 14 may generate content data by associating the speaker of the voice with the user identification information.
  • the content generation unit 14 continues to generate content data until the content reproduction on the user terminal 20 catches up with the real-time online conference. Therefore, the end point of the time width may vary depending on, for example, the playback speed of the content or the length of the playback time of the content.
  • step S15 the output unit 15 of the server 10 transmits the content data to the user terminal 20.
  • the content reproducing section 23 receives the content data.
  • step S16 the content reproduction unit 23 reproduces the content at a higher reproduction speed than the original reproduction speed of the conference data while the online conference is in progress.
  • the content reproduction unit 23 processes the content data received from the server 10 and displays the content on the display device. When the content is not rendered on the server 10 side, the content reproduction unit 23 performs rendering based on the content data to display the content. If the content data indicates the content itself, the content reproducing unit 23 displays the content as it is.
  • the user terminal 20 outputs audio from the speaker in accordance with the display of the content. In this manner, the content reproduction unit 23 displays the reproduction screen 400 (see example (a) and example (b) in FIG. 5) on the user terminal 20 .
  • the playback speed of the content should be faster than the original playback speed of the meeting data.
  • the playback speed of the content may be 2.0x speed.
  • the content reproduction unit 23 reproduces content at high speed while the online conference is in progress.
  • the content playback speed may be determined by the content generator 14 or the content playback unit 23 .
  • the conference display unit 21 displays the conference screen 300 on the user terminal 20 again. In this manner, the display on the user terminal 20 is switched from the playback screen 400 to the conference screen 300 .
  • the end point of the time width may be determined in relation to step S14.
  • the end point of the duration may be the acquisition time at which the content start time is acquired.
  • the acquisition time may be the time at which the server 10 receives the content generation request, or the time at which the follow-up playback button 306 is pressed by the user, or the like.
  • content data from the content start point to the time indicating the end point of the time width is generated and transmitted to the user terminal 20 .
  • the content reproduction unit 23 may reproduce the content and the conference display unit 21 may display the online conference.
  • content playback and real-time online conference display may be performed in parallel.
  • the conference display unit 21 continues displaying the online conference.
  • FIG. 7 is a diagram showing an example of a functional configuration related to the conference support system 1A.
  • the conference support system 1A differs from the conference support system 1 in that the server 10 further includes a situation determination unit 16 as a functional element, and the user terminal 20 further includes a sharing unit 24 as a functional element.
  • the status determination unit 16 is a functional element that determines the status of the user and the progress of content reproduction.
  • a status refers to a user's participation state in a conference.
  • the progress state refers to the state of progress regarding the reproduction of the content.
  • the sharing unit 24 is a functional element that cooperates with the situation determination unit 16 of the server 10 to share the user's status and the content reproduction progress.
  • FIG. 8 is a diagram showing another example of the conference screen 300.
  • FIG. Example (a) of FIG. 8 shows a conference screen 300A displaying status and progress.
  • 300 A of meeting screens are provided with the time display column 304B and the status message 307.
  • FIG. The time display field 304B is a screen element that displays the time until content reproduction ends.
  • the time display column 304B may be displayed within a display area where a moving image of the user who is reproducing the content is displayed.
  • the time display field 304B may be displayed within the display area 304 where the user D's moving image is displayed.
  • the time display column 304B displays the remaining time such as "remaining: 0 minutes and 30 seconds".
  • a status message 307 is a screen element that displays as a status that the user is currently playing content.
  • the status message 307 displays information indicating which user is currently playing the content, such as "user D is playing chasing.”
  • the modes of the time display field 304B and the status message 307 are not limited to this, and for example, the time display field 304B and the status message 307 may be displayed together.
  • Example (b) of FIG. 8 shows a conference screen 300B that displays status and progress.
  • Conference screen 300B comprises indicator 304C and status message 308.
  • the indicator 304C is a screen element that displays the progress rate of content playback.
  • the indicator 304C may be displayed within the display area where the moving image of the user playing the content is displayed.
  • the indicator 304C may be displayed within the display area 304 where the moving image of user D is displayed.
  • the indicator 304C displays the progress rate in the time span of the content.
  • indicator 304C displays the rate of progress, such as by a progress bar or percentage.
  • a status message 308 is a screen element that displays the speaker of the audio along with the status along with the progress of content playback.
  • Status message 308 has an embedded portion 309 that displays user identification information.
  • the status message 308 displays information such as, for example, "User D is playing back what 'speaker' said.”
  • the “speaker” corresponds to the embedding section 309 .
  • User identification information can be displayed in the embedded portion 309 along with the progress of content reproduction.
  • the user identification information of user A is displayed in the embedded portion 309, such as "User D is reproducing the statement of 'user A'.”
  • the aspects of the indicator 304C and the status message 308 are not limited to this, and for example, the indicator 304C and the status message 308 may be displayed together.
  • time display field 304B, indicator 304C, and status messages 307 and 308 described above may not be displayed, may be displayed individually, or may be displayed in any combination.
  • FIG. 9 is a sequence diagram showing the operation of the conference support system 1A as a processing flow S2.
  • four users User A, User B, User C and User D
  • the conference control unit 11 of the server 10 and the conference display unit 21 of the user terminal 20 cooperate to display the conference screen 300 (see FIG. 4) on each of the user terminals 20 of the four users.
  • the user terminal 20 of the user who reproduces the content is described as a first user terminal, and the user terminal 20 of a user different from the user is described as a second user terminal.
  • Steps S21 to S26 are the same as steps S11 to S16 of the processing flow S1, respectively, so description thereof will be omitted.
  • step S27 the sharing unit 24 of the first user terminal notifies the server 10 of the content playback speed.
  • the sharing unit 24 acquires the reproduction speed of the content displayed in the reproduction speed column 405 with the reproduction of the content as a trigger.
  • the sharing unit 24 notifies the server 10 of the reproduction speed.
  • the situation determination unit 16 may determine that the user of the first user terminal is currently reproducing the content by receiving the notification of the reproduction speed from the first user terminal.
  • the process of step S27 may be further executed using, for example, a change in the playback speed of the content, cueing of the content, or the like as a trigger.
  • step S28 the status determination unit 16 of the server 10 calculates the progress status based on the content playback speed and elapsed time.
  • the situation determination unit 16 multiplies the playback speed of the content by the elapsed time to calculate the playback position in the length of the playback time of the content as the progress.
  • the elapsed time may be acquired, for example, from the first user terminal, or may be measured as the time when the playback speed notification is received in the process of step S27 as the start time.
  • the situation determination unit 16 may calculate the time until the content reproduction ends as the progress.
  • the status determination unit 16 may calculate the progress rate of content reproduction as the progress status.
  • step S29 the conference control unit 11 performs conference display control on the second user terminal.
  • the conference control unit 11 transmits the status and progress to the second user terminal.
  • the conference display unit 21 acquires the progress and status.
  • the conference display unit 21 displays the progress and status.
  • the conference display units 21 of the user terminals 20 of the users A, B, and C display the conference screen 300A (see example (a) in FIG. 8) on the display device.
  • Users A, B, and C share the status of user D and the progress of content reproduction by displaying time display field 304B and status message 307 on conference screen 300A.
  • step S27 if the content playback speed is determined on the server 10 side, the sharing unit 24 does not need to notify the playback speed.
  • the conference support system includes at least one processor.
  • At least one processor records meeting data including audio of the online meeting, acquires a point in time prior to the online meeting from the user's terminal, generates content corresponding to the meeting data in the time span after the point in time, and performs the online meeting. In progress, content is played back on the user's terminal at a faster playback speed than the original playback speed of the conference data.
  • a conference support method is executed by a conference support system including at least one processor.
  • the meeting support method includes the steps of recording meeting data including the voice of the online meeting, acquiring from the user's terminal a point in time prior to the online meeting, and generating content corresponding to the meeting data in the time span after the point in time. and causing the user's terminal to reproduce the content at a faster reproduction speed than the original reproduction speed of the conference data while the online conference is in progress.
  • a meeting support program includes steps of recording meeting data including audio of an online meeting, acquiring from a user's terminal a point in time prior to the online meeting, and recording meeting data in a time span after the point in time
  • a computer is caused to generate corresponding content and cause the user's terminal to reproduce the content at a faster reproduction speed than the original reproduction speed of the conference data while the online conference is in progress.
  • content is generated that corresponds to meeting data after the online meeting has been traced back.
  • the content is then rapidly played back on the user's terminal to catch up with the ongoing online conference.
  • Patent Document 1 conference information is recorded while the conference is in progress, and when a midway participant in the conference is detected, a summary of the conference information up to that point is created, and the created summary is sent to the midway participant.
  • a network conferencing system is described that provides individual access to.
  • the technique of Patent Document 1 is not a technique for chasing and playing back the content of a conference that you missed while participating in the conference. Also, with the technique of Patent Document 1, the content of the meeting may be lost due to the creation of the abstract.
  • Patent Document 2 above describes a teleconferencing system that rewinds and reproduces the video or audio of an electronic conference participant's utterance when he or she misses hearing it.
  • the technique described in Patent Document 2 is not a technique for reproducing audio or video at high speed during rewinding. Therefore, the participants cannot quickly grasp the connection before and after the content of the conference.
  • the content corresponding to the conference data in the time span after the online conference is reproduced at high speed. Therefore, it is possible to catch up and play back the content of the conference that was missed while participating in the conference. In addition, since the contents are played back at high speed, the user can quickly grasp the connection before and after the contents of the conference.
  • At least one processor may cause a terminal of a user other than the user to display a status indicating that the user is playing content.
  • the status of the user who is playing the content is shared among the users participating in the online conference. Another user can grasp the status, so the online meeting can proceed smoothly.
  • At least one processor may calculate the progress based on the playback speed and elapsed time of the content, and display the progress on another user's terminal.
  • the users who are participating in the online conference share the progress of the users who are reproducing the content. Since another user can grasp the progress, the online meeting can proceed smoothly.
  • At least one processor acquires user identification information that identifies a user who is a speaker of speech, associates the speech with the user identification information to generate content
  • the user's terminal may display the user identification information along with the status along with the progress.
  • the users who are participating in the online conference share information about whose speech the user who is reproducing the content is listening to. This allows another user to grasp the details of the progress.
  • At least one processor may calculate the time until content reproduction ends as progress. In this case, another user shares the time until the playback of the content ends. This allows another user to accurately grasp the progress.
  • At least one processor may calculate a content reproduction progress rate as the progress.
  • the content reproduction progress rate is shared with other users. This allows another user to intuitively grasp the progress.
  • the end point of the duration may be the time when the time point was acquired.
  • At least one processor may cause the content to be played and the online meeting to be displayed at the user's terminal.
  • the content from the previous time point to the time when the time point was obtained is reproduced at high speed, and the online conference after the time when the time point was obtained is displayed in real time. This makes it possible to reduce the time required to reproduce the content.
  • the content generation unit 14 may generate content data in text data format by executing speech recognition on the conference data. For example, the content generation unit 14 may generate content data in which at least the user's speech is converted into text. The content generation unit 14 may generate content data configured only in the text data format, or may generate content data configured in a combination of the text data format and audio or moving images. The content generation unit 14 may associate the user identification information with the text data to generate content data specifying the speaker for each voice. The content reproduction unit 23 may display the content data in the text data format on the display device. In this case, it is possible to provide an environment for quickly grasping the content of the conference.
  • the content reproduction unit 23 may perform skip reproduction in which a part of the content is skipped.
  • Skip playback may be triggered by, for example, a change in the playback position of the progress bar 408, a cue operation of the operation interface 406, or the like. Skip playback can reduce the time required for content playback.
  • Content may be labeled with one or more labels.
  • the content generator 14 may detect the volume of the conference data or the number of speakers in chronological order, and determine whether the detected value is equal to or greater than a predetermined threshold.
  • the content generation unit 14 may generate content data in which a label such as "active meeting" is attached to the time equal to or greater than the threshold.
  • Other examples of labeling include “meeting is quiet", “a particular user is speaking", “speaker switching", and the like.
  • the content reproduction unit 23 may perform skip reproduction using the label as the cue position. Cueing may be performed with a user operation as a trigger, or may be performed automatically without receiving a user operation. In one example, the content reproduction unit 23 may perform skip reproduction by automatic cueing so as to reproduce only the content of the time width indicated by the label. The user's convenience is improved by making it possible to reproduce the parts where the conference was lively.
  • the online conference has been described as a conference format in which moving images are shared, but it may be a conference format in which only audio is used. Further, in the above-described embodiment, the situation determination unit 16 is explained to calculate the progress, but the progress may be shared from the first user terminal to the second user terminal. As another example, the second user terminal may transmit an interruption request to interrupt the reproduction of the content to the first user terminal.
  • the conference display unit 21 of the first user terminal that has received the interruption request may display the conference screen 300 .
  • the conference support system may be applied to an online conference between the user terminals 20 without using the server 10.
  • each functional element of the server 10 may be installed in one of the user terminals, or may be installed separately in a plurality of user terminals.
  • the conference support program may be implemented as a client program.
  • the conference support system may be configured using a server, or may be configured without using a server. That is, the conference support system may be of a client-server type, or may be of a client-client type of P2P (Peer to Peer) or E2E (End to End) encryption.
  • P2P Peer to Peer
  • E2E End to End
  • the client-client method improves the confidentiality of online meetings.
  • the conference support system can prevent the audio of the online conference from being leaked to a third party by E2E-encrypting the online conference between the user terminals 20 .
  • the expression “at least one processor executes the first process, the second process, . . .” is a concept including the case where the executing subject (that is, the processor) of n processes from process 1 to process n changes in the middle. That is, this expression is a concept that includes both the case where all of the n processes are executed by the same processor and the case where the processors are changed according to an arbitrary policy in the n processes.
  • the processing procedure of the method executed by at least one processor is not limited to the examples in the above embodiments. For example, some of the steps (processes) described above may be omitted, or the steps may be performed in a different order. Also, any two or more of the steps described above may be combined, and some of the steps may be modified or deleted. Alternatively, other steps may be performed in addition to the above steps.
  • any part or all of each functional unit described in this specification may be implemented by a program.
  • the program referred to in this specification may be recorded non-temporarily on a computer-readable recording medium and distributed, or may be distributed via a communication line (including wireless communication) such as the Internet. , may be distributed as installed on any terminal.
  • a configuration described as one device (or member; hereinafter the same) in this specification may be realized by a plurality of devices. good.
  • configurations described herein as multiple devices may be implemented by a single device.
  • some or all of the means or functions included in one device eg a server
  • may be included in another device eg a user terminal.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Telephonic Communication Services (AREA)
  • Information Transfer Between Computers (AREA)
PCT/JP2022/026624 2021-08-31 2022-07-04 会議支援システム、会議支援方法、および会議支援プログラム Ceased WO2023032461A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US18/292,257 US20240388462A1 (en) 2021-08-31 2022-07-04 Meeting assistance system, meeting assistance method, and meeting assistance program
CN202280045941.XA CN117581528A (zh) 2021-08-31 2022-07-04 会议辅助系统、会议辅助方法以及会议辅助程序

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-140963 2021-08-31
JP2021140963A JP7030233B1 (ja) 2021-08-31 2021-08-31 会議支援システム、会議支援方法、および会議支援プログラム

Publications (1)

Publication Number Publication Date
WO2023032461A1 true WO2023032461A1 (ja) 2023-03-09

Family

ID=81215051

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/026624 Ceased WO2023032461A1 (ja) 2021-08-31 2022-07-04 会議支援システム、会議支援方法、および会議支援プログラム

Country Status (4)

Country Link
US (1) US20240388462A1 (https=)
JP (2) JP7030233B1 (https=)
CN (1) CN117581528A (https=)
WO (1) WO2023032461A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2024100075A (ja) * 2023-01-13 2024-07-26 コニカミノルタ株式会社 情報処理システム

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11177962A (ja) * 1997-12-09 1999-07-02 Toshiba Corp 情報再生サーバ装置、情報再生装置および情報再生方法
JP2005244522A (ja) * 2004-02-25 2005-09-08 Pioneer Electronic Corp ネットワーク会議システム、会議サーバ、記録サーバおよび会議端末
US20150312518A1 (en) * 2013-07-02 2015-10-29 Family Systems, Ltd. Systems and methods for improving audio conferencing services

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003339033A (ja) * 2002-05-17 2003-11-28 Pioneer Electronic Corp ネットワーク会議システム、ネットワーク会議方法およびネットワーク会議プログラム
JP4845581B2 (ja) 2006-05-01 2011-12-28 三菱電機株式会社 画像及び音声通信機能付テレビジョン放送受像機
WO2007132690A1 (ja) * 2006-05-17 2007-11-22 Nec Corporation 音声データ要約再生装置、音声データ要約再生方法および音声データ要約再生用プログラム
US20090327425A1 (en) * 2008-06-25 2009-12-31 Microsoft Corporation Switching between and dual existence in live and recorded versions of a meeting
JP2010219866A (ja) * 2009-03-17 2010-09-30 Konica Minolta Business Technologies Inc コミュニケーション支援装置及びコミュニケーション支援システム
US8797380B2 (en) * 2010-04-30 2014-08-05 Microsoft Corporation Accelerated instant replay for co-present and distributed meetings
US20130339431A1 (en) * 2012-06-13 2013-12-19 Cisco Technology, Inc. Replay of Content in Web Conferencing Environments
US10748529B1 (en) * 2013-03-15 2020-08-18 Apple Inc. Voice activated device for use with a voice-based digital assistant
JP2014236288A (ja) * 2013-05-31 2014-12-15 株式会社東芝 再生装置、再生方法、及び再生プログラム
US20160285929A1 (en) * 2015-03-27 2016-09-29 Intel Corporation Facilitating dynamic and seamless transitioning into online meetings
JP2018032912A (ja) * 2016-08-22 2018-03-01 株式会社リコー 情報処理装置、情報処理方法、情報処理プログラムおよび情報処理システム
US10693824B2 (en) * 2016-09-14 2020-06-23 International Business Machines Corporation Electronic meeting management

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11177962A (ja) * 1997-12-09 1999-07-02 Toshiba Corp 情報再生サーバ装置、情報再生装置および情報再生方法
JP2005244522A (ja) * 2004-02-25 2005-09-08 Pioneer Electronic Corp ネットワーク会議システム、会議サーバ、記録サーバおよび会議端末
US20150312518A1 (en) * 2013-07-02 2015-10-29 Family Systems, Ltd. Systems and methods for improving audio conferencing services

Also Published As

Publication number Publication date
JP7030233B1 (ja) 2022-03-04
JP2023035787A (ja) 2023-03-13
US20240388462A1 (en) 2024-11-21
JP7777465B2 (ja) 2025-11-28
CN117581528A (zh) 2024-02-20
JP2023034633A (ja) 2023-03-13

Similar Documents

Publication Publication Date Title
JP7379907B2 (ja) 情報処理装置、情報処理プログラム、情報処理システム、情報処理方法
US10567448B2 (en) Participation queue system and method for online video conferencing
US10163077B2 (en) Proxy for asynchronous meeting participation
US9392037B2 (en) Method and apparatus for reconstructing a communication session
JP6801317B2 (ja) 問い合わせ回答を要求する方法、プログラム及びサーバ装置
CN108702483A (zh) 通信事件
CN116437147B (zh) 直播任务交互方法、装置、电子设备及存储介质
CN117897930A (zh) 用于混合在线会议的流式数据处理
US20140194152A1 (en) Mixed media communication
JP6752349B1 (ja) コンテンツ配信システム、コンテンツ配信方法、およびコンテンツ配信プログラム
JP7030233B1 (ja) 会議支援システム、会議支援方法、および会議支援プログラム
WO2024067597A1 (zh) 线上会议方法、装置、电子设备及可读存储介质
JP2010093583A (ja) 会議支援装置
JP2016063477A (ja) 会議システム、情報処理方法、及びプログラム
JP2023111906A (ja) 記録情報作成システム、記録情報作成方法、プログラム
JP2004165946A (ja) Ｗｅｂ会議システム
US12615341B2 (en) Digital overlay
JP2023123119A (ja) 通信端末、及び通信システム
WO2024203284A1 (ja) 会議システム、制御装置、制御方法、プログラム、および記録媒体
JP2024008266A (ja) コミュニケーション制御装置及びコンピュータープログラム
HK40071405A (en) Participation queue system and method for online video conferencing
CN120050471A (zh) 视频播放方法、装置、电子设备及可读存储介质
JP2022130117A (ja) オンライン会議システム
JP2022127676A (ja) サーバシステム、プログラム、及び通信システム
WO2018074263A1 (ja) 情報処理装置、情報処理方法、プログラム、およびコミュニケーションシステム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22864049

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280045941.X

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22864049

Country of ref document: EP

Kind code of ref document: A1