JP6304941B2 - CONFERENCE INFORMATION RECORDING SYSTEM, INFORMATION PROCESSING DEVICE, CONTROL METHOD, AND COMPUTER PROGRAM - Google Patents

Description

  The present invention relates to a conference information recording system, an information processing apparatus, a control method, and a computer program.

  Along with the advancement of digital technology, a conference information recording system that records the state of a conference as video (moving image and audio) has been proposed. In the conference information recording system, in addition to the video, the conference material displayed on the projector or the display can be recorded. This conference material can be recorded as an electronic file, and it may be a still image obtained by recording a display image. The conference record also includes agenda text (meeting minutes and memos) entered by a conference participant operating a computer such as a PC. The conference record also includes a tag to be recorded based on an operation instruction from a conference participant, for example, time information indicating that an important conversation has been made before and after the operation, and a conference scene. The conference scene is a state of the conference such as “opening”, “presentation”, “question / answer”, “discussion”, “contemplation / silence”, “summary”. As the state of the conference recording, for example, data collected by various sensors such as volume, temperature, and brightness can be recorded.

  In addition, data generated by analyzing and processing data collected for the conference can also be recorded. For example, the recorded moving image is analyzed, and the face image information obtained by extracting the face part of the conference participant is recorded. This is useful for referring to the facial expressions of conference participants. Keywords that are important phrases obtained by statistically analyzing speaker information obtained by analyzing speech, text obtained by speech recognition processing, text obtained by OCR processing of conference materials, and proceedings text Etc. can also be recorded.

  For example, Patent Document 1 discloses a conference information recording system that improves the usability of conference information by collecting conference information, adding attribute information to each conference information by separating the collected conference information. Yes.

JP 2007-199866 A

  By using the conference information recording system, it is possible to look back on the state of past conferences in more detail and accurately. However, since the conference information is recorded as a large amount of data, it takes time and effort to find and browse a desired portion of the user. In the conference information recording system disclosed in Patent Document 1, specific information can be searched from conference information using a “search key” such as attribute information, but it is still necessary to refer to a large amount of data. . Further, when searching for specific information using a search key, it is necessary for the user to remember at least information that can be used as a search key in past meetings. However, the user does not always remember such information, and in that case, it is necessary to refer to all data of the conference. The data to be referred to is composed of a plurality of types, and these are presented in time series. Therefore, the user does not know what type of data should be focused in each time zone, and cannot efficiently look back on past meetings.

  An object of the present invention is to provide a conference information recording system that presents to a user what type of data should be focused in each time period by determining items to be highlighted in accordance with a conference scene. To do.

A conference information recording system according to an embodiment of the present invention includes an acquisition unit that acquires different types of data related to a conference, and displays a screen generated based on the different types of data acquired by the acquisition unit. An information processing system, wherein the display unit displays the screen in which the data acquired by the acquisition unit is arranged in order of time acquired by the acquisition unit for each type, and a conference scene at each time during the conference And setting means for setting one of a plurality of conference scenes , and the display means changes the type of data to be highlighted for each conference scene set by the setting means .

  According to the conference information recording system of the present invention, it is possible to present to the user what type of data to focus on in each time zone by determining the items to be highlighted in accordance with the conference scene. . Thereby, the user can look back on the past meetings more efficiently.

It is a figure which shows the structural example of the meeting information recording system of a present Example. It is a block diagram which shows the structural example of a meeting information collection apparatus. It is a block diagram which shows the structural example of a meeting information recording server and a computer. It is an example of a screen which the display device of a meeting information collection device displays. It is a figure which shows the data structural example of meeting information. It is a figure which shows the data structural example of meeting information. It is a figure which shows the example of a data structure of the meeting information produced | generated by analysis and processing. The example of a screen which the display device of a computer displays is shown. The example of a screen which the display device of a computer displays is shown. It is the flowchart which showed the procedure which produces | generates a display image. It is the flowchart which showed the procedure which produces | generates a display image. It is the flowchart which showed the procedure which produces | generates a display image. It is the flowchart which showed the procedure which produces | generates a display image. It is the flowchart which showed the procedure which produces | generates a display image. The example of a screen which the display device of a computer of Example 2 displays is shown. It is the flowchart which showed the procedure which produces | generates a display image. It is the flowchart which showed the procedure which produces | generates a display image. It is the flowchart which showed the procedure which produces | generates a display image. The example of a data structure which is the control information of a scroll process is shown. It is a flowchart which shows the procedure which produces | generates scroll control information. It is a flowchart which shows the procedure which produces | generates scroll control information. It is a flowchart which shows the procedure which produces | generates scroll control information. It is a flowchart which shows the procedure which produces | generates scroll control information. It is a flowchart which shows the procedure which produces | generates scroll control information. It is a flowchart which shows the procedure which produces | generates scroll control information. It is a flowchart which shows the procedure of the process which scrolls a display image.

  FIG. 1 is a configuration diagram illustrating a configuration of a conference information recording system according to the present embodiment. The conference information recording system 100 includes a conference information collection device 101, a conference information recording server 102, a computer 103, a computer 104, and a computer 105. These apparatuses / devices are connected via the network 106. Any network 106 may be used as long as it can communicate with each other regardless of wired / wireless. The conference information recording system 100 shown in FIG. 1 includes a plurality of computers, but the number of computers may be one, or four or more computers.

  The conference information collection device 101 is arranged at a conference venue such as a conference room, and collects a plurality of types of data such as moving images, audio, and conference scenes. Hereinafter, data of a plurality of types obtained by collecting and generating a conference will be referred to as “conference information”. In the conference information recording system 100, the various data included in the conference information can be recorded together with the time information, thereby making it possible to associate the data. The collected data and time information are used as meeting information, and are periodically transmitted to the meeting information recording server 102. In addition, the conference information collection apparatus 101 has a projector device, and can project an output image from the computer 103 described later onto a screen, a wall surface, or the like. The conference information collection device 101 records conferences held in, for example, offices or predetermined venues, but the conference to which the present invention is applied is a group that involves the visual / speaking behavior of a plurality of persons. Often, it is not limited to meetings held in offices or predetermined venues. For example, interviews and surveys are also applicable to the present invention.

  The conference information recording server 102 is a general PC, receives a plurality of items of data as conference information from the conference information collection device 101, records and manages the conference information generated by analyzing and processing the conference information. To do. In this embodiment, the conference information collection device 101 and the conference information recording server 102 are described as separate devices, but may be configured as a single device having the functions of both devices. In this case, the conference information collection device 101 and the conference information recording server 102 function as a conference information creation device that collects a plurality of items of data related to the conference and creates conference information based on the collected data.

  The computer 103, the computer 104, and the computer 105 are general PCs, but may be terminal devices such as tablets and smartphones. In this embodiment, the computer 103 is arranged at a meeting holding place, and generates an output image from meeting material (electronic document) for presentation use. The computer 103 can transmit the output image to the conference information collecting apparatus 101 and project and display the contents by a projector device included in the conference information collecting apparatus 101. The computer 104 is also arranged at the meeting place and is used for inputting the agenda and memo via the input device. The computer 105 is an information processing apparatus for browsing conference information recorded by the conference information recording server 102 after the conference. Note that the user may browse the conference information not only using the computer 105 but also using the computer 103 or the computer 104. Moreover, the control method of this embodiment is a control method of the conference information recording system shown in FIG. The computer program of this embodiment causes a computer to execute this control method.

  FIG. 2 is a block diagram illustrating a hardware configuration example of the conference information collection apparatus 101 in FIG. The conference information collection apparatus 101 includes a CPU 201, a ROM 202, a RAM 204, a storage 205, an input device 206, a display device 207, and an external interface 208. The conference information collecting apparatus 101 includes a camera device 209, a microphone device 210, a projector device 211, a sensor input device 212, a temperature sensor 213, a volume sensor 214, and a brightness sensor 215. Each unit can transmit and receive data to and from each other via the data bus 203. CPU is an abbreviation for Central Processing Unit. RAM is an abbreviation for Random Access Memory. ROM is an abbreviation for Read Only Memory.

  The CPU 201 is a controller for controlling the entire conference information collection device. The CPU 201 starts an OS (operating system) by a boot program stored in a ROM 202 which is a nonvolatile memory. The CPU 201 executes the controller program stored in the storage 205 on this OS. The controller program is a program for controlling the entire conference information collecting apparatus. The CPU 201 controls each unit via a bus such as the data bus 203. The RAM 204 operates as a temporary storage area such as the main memory or work area of the CPU 201. The storage 205 is a non-volatile memory that can be read and written, and stores the aforementioned controller program. The conference information collection apparatus 101 collects various types of conference information and periodically transmits the conference information to the conference information recording server 102. The conference information is temporarily stored in the storage 205 until the conference information is collected.

  The input device 206 is an input device including a touch panel, hard keys, a mouse, and the like. The display device 207 is a display device such as an LCD or CRT. When the input device 206 receives a user operation instruction, the input device 206 transmits the instruction to the CPU 201. The display device 207 displays the display data generated by the CPU 201 on the screen. The CPU 201 determines an operation based on the instruction information received from the input device 206 and the display data displayed on the display device 207. The CPU 201 controls the conference information collection apparatus 101 according to the determination result, generates new display data according to the operation content, and causes the display device 207 to display the display data. The external interface 208 transmits or receives various data to / from a separate external device via a network such as a LAN, a telephone line, or a near field radio such as infrared rays.

  The camera device 209 is a so-called digital camera and can shoot a moving image. The camera device 209 can shoot the entire conference venue by recording a 360-degree direction so that the entire conference venue can be recorded, or by including a plurality of lenses and image elements. It is preferable to constitute so. The camera device 209 acquires a captured image acquired by the lens and the image element as moving image data such as MPEG by acquiring a predetermined frame (for example, 30 frames, frame rate 30 fps) per second.

  The microphone device 210 converts the input sound into a digital signal and acquires it as sound data such as WAVE, for example. The projector device 211 can project and display an output image on a screen, a wall surface, or the like using a light source and a projection lens. The sensor input device 212 is connected to various sensors and acquires the input signal by converting it into a digital signal. In this embodiment, the sensor input device 212 is connected to the temperature sensor 213, the volume sensor 214, and the brightness sensor 215.

  FIG. 3 is a block diagram showing the configuration of the conference information recording server 102 and the computer 105 of FIG. First, the conference information recording server 102 will be described. The conference information recording server 102 includes a CPU 301, ROM 302, RAM 304, storage 305, input device 306, display device 307, and external interface 308. Each unit can transmit and receive data to and from each other via the data bus 303. The CPU 301 is a controller for controlling the entire conference information recording server. The CPU 301 activates the OS by a boot program stored in a ROM 302 that is a nonvolatile memory. On this OS, the conference information recording server program stored in the storage 305 is executed. Each process of the conference information recording server 102 is realized by the CPU 301 executing this conference information recording server program. The CPU 301 controls each unit via a bus such as the data bus 303.

  The RAM 304 operates as a temporary storage area such as the main memory or work area of the CPU 301. The storage 305 is a non-volatile memory that can be read and written, and stores the server program described above. The conference information recording server 102 records and manages various conference information collected by the conference information collection apparatus 101 and conference information generated by analyzing and processing the conference information in the storage 305.

  The input device 306, the display device 307, and the external interface 308 have the same configuration as each unit described with reference to FIG. The CPU 301 determines an operation from the instruction information received from the input device 306 and the display data displayed on the display device 307. The CPU 301 controls the conference information recording server 102 according to the determination result, and generates new display data according to the operation content and causes the display device 307 to display it. The input device 306 and the display device 307 are mainly used for the purpose of maintenance of the conference information recording server 102 by the server administrator.

  Next, the computer 105 will be described. The computer 105 includes a CPU 311, ROM 312, RAM 314, storage 315, input device 316, display device 317, and external interface 318. Each unit can transmit / receive data to / from each other via the data bus 313. Each unit has the same configuration as each unit of the conference information recording server 102 described with reference to FIG. The CPU 311 implements each process of the computer 105 by executing the conference information display program stored in the storage 305. Note that the configurations of the computer 103 and the computer 104 are the same as the configuration of the computer 105, and a description thereof will be omitted.

  Next, display contents on the display device 207 of the conference information collection apparatus 101 will be described. FIG. 4 is a screen diagram illustrating a display example of the display device 207 of the conference information collection apparatus 101. A screen 400 in FIG. 4A is a screen displayed before the start of the conference.

  A “conference start” button 401 is used by the user to notify the conference information collection apparatus 101 of the start of the conference. When the CPU 201 receives an instruction to the “conference start” button 401 via the input device 206, the CPU 201 displays a screen 410 shown in FIG. 4B on the display device 207. A screen 410 is a screen displayed during the meeting. The contents for receiving an operation instruction by the user for collecting the meeting information are displayed.

  FIG. 4B shows an example of a meeting scene selection screen. The display device 207 functions as a presentation unit that presents the conference scene selection screen. In the screen example shown in FIG. 4B, the display device 207 displays a group of buttons such as an opening 411, a presentation 412, a question / response 413, a discussion 414, a contemplation / silence 415, and a summary 416. Each button 411 to 416 is a group of buttons for causing the conference information collecting apparatus 101 to record the current conference scene. When the user designates one of these buttons, the conference information collecting apparatus 101 records that the time period from the designated time until the next button is designated is the conference scene indicated by the designated button. Can be made. That is, each conference scene is recorded in association with time information.

  Here, the “conference scene” will be described. In the present embodiment, a conference scene is defined as one that can take one of the following values as a conference state. “Opening” is a situation where the background, purpose, output, etc. of the conference are explained at the beginning of the conference. “Presentation” is a so-called presentation that is given at a conference, and is a situation in which a presenter explains the content of a proposal, a plan, etc. to other conference participants and gains understanding. In general, this is often performed while projecting a conference material on a projector or display.

  The “question / answer” is a so-called question-and-answer session in which a question and answer are exchanged with respect to a certain topic, for example, the content of a presentation. “Discussion” is a so-called discussion, and is a situation in which conference participants fight opinions in order to reach an agreement on some topic. “Contemplation / silence” is a situation in which a conference participant summarizes his / her opinions and thoughts and presents ideas at the conference. Or, the discussion was stuck and stagnated, and the place was silent. “Summary” is a situation in which items and action items determined in the conference are reviewed at the end of the conference.

  When the user presses the buttons 411 to 416 displayed on the screen 410 using the input device 206 or the like, the user assigns a conference scene to the current conference and records it in the storage 205 of the conference information collection apparatus 101. be able to.

  The “important” button 417 is an icon for recording an important point of the conference. The effect is that the time at which the “important” button 417 is instructed is recorded. For example, when an important statement is made or a characteristic event occurs, the user records the importance with this button. Can be made. Alternatively, the user may instruct the button when he / she says to make an important statement. When the “important” button 417 is selected by the user, the CPU 201 records the important tag information in association with the time when the “important” button 417 is selected. As a result, the display device 317 can draw a mark indicating the importance level when performing the drawing process on the section of the unit time corresponding to the time when the important tag information is recorded. In other words, by using the “important” button 417, it is possible to attach a “mark” when looking back on conference information that is a large amount of data later. The “conference end” button 418 is a button for notifying the conference information collection apparatus 101 of the end of the conference. When the CPU 201 receives an instruction to the “end conference” button 417, the CPU 201 displays a screen similar to the screen 400 on the display device 207 and ends the recording of the conference information.

  Here, a “minimum time interval” is defined. According to the conference information recording system in the present embodiment, the state of the conference is recorded as conference information composed of a plurality of types of data items. In this embodiment, when the computer 105 or the like displays the conference information on the screen, the conference information is presented side by side on a timeline (time series). The conference information presentation interval on the timeline is configured to be changed by a user operation, and the minimum value of this time interval is defined as “minimum time interval”. In this embodiment, the minimum time interval is defined as 10 seconds.

  Next, meeting information collected by the meeting information collecting apparatus 101 will be described with reference to FIGS. 5 and 6 show examples of the data structure of conference information collected by the conference information collection device 101 and temporarily recorded in the storage 205. FIG. The conference information received by the conference information recording server 102 from the conference information collection device 101 and recorded in the storage 305 has the same configuration as that shown in FIGS. When the CPU 201 receives an instruction to the “conference start” button 401, the CPU 201 starts a meeting information collection process described below.

  FIG. 5A shows a moving image information table 500. The CPU 201 captures the state of the meeting with the camera device 209, collects moving image data, and records the collected data in the moving image information table 500. The CPU 201 records the shooting start time (conference start time) in the shooting start time column 501 and records the shooting end time in the shooting end time column 502. Further, the CPU 201 records moving image data as a file in the storage 205 and records the file name in the moving image data string 503.

  FIG. 5B shows the audio information table 510. The CPU 201 records the conversation during the conference by the microphone device 210, collects voice data, and records the collected data in the voice information table 510. The CPU 201 records the recording start time (conference start time) in the recording start time column 511 and records the recording end time in the recording end time column 512. In addition, the CPU 201 records audio data as a file in the storage 205 and records the file name in the audio data string 513.

  FIG. 5C shows an agenda text information table 520. The computer 104 transmits the text data input from the input device to the conference information collecting apparatus 101 every time a line feed code is input. At this time, the time information when the line feed code is input is also transmitted. The CPU 201 records the received text data and time information in the agenda text information table 520. The CPU 201 records the input time of the line feed code in the input time column 521 and the text data in the proceeding text data column 522, respectively.

  FIG. 5D shows a conference material information table 530. The CPU 201 receives an output image generated by the computer 103 based on the conference material via the external interface 208 and projects it by the projector device 211. At this time, the CPU 201 stores the projected output image in the conference material information table 530 together with the current time (output image acquisition time) at every minimum time interval. If no output image is input to the projector device 211, the conference material information is not collected. The CPU 201 records the output image acquisition time in the output image acquisition time column 531. In addition, the CPU 201 records the output image as a file in the storage 205, and records the output image in the output image sequence 532 in association with a file name such as slide 1 at the output image acquisition time.

  FIG. 5E shows a temperature information table 540. The CPU 201 acquires temperature data from the sensor input device 212 and the temperature sensor 213 as the environment information of the conference venue at intervals of the minimum time. And CPU201 memorize | stores in the temperature information table 540 with time information (temperature data acquisition time). The CPU 201 records the temperature data acquisition time in the temperature data acquisition time column 541 and the temperature data in the temperature data column 542, respectively.

  FIG. 6A shows a volume information table 600. The CPU 201 acquires volume data from the sensor input device 212 and the volume sensor 214 as information indicating the degree of activity of the conference at every minimum time interval. Then, the CPU 201 records it in the volume information table 600 together with time information (volume data acquisition time). The CPU 201 records the volume data acquisition time in the volume data acquisition time column 601 and records the volume data in the volume data column 602.

  FIG. 6B shows a brightness information table 610. The CPU 201 acquires lightness data from the sensor input device 212 and the lightness sensor 215 every minimum time interval as environment information of the conference venue. Then, the CPU 201 stores it in the lightness information table 610 together with time information (lightness data acquisition time). The CPU 201 records the lightness data acquisition time in the lightness data acquisition time column 611 and the lightness data in the lightness data column 612, respectively.

  FIG. 6C shows an important tag information table 620. When receiving an instruction to the “important” button 417, the CPU 201 records time information (important tag assignment time) in the important tag assignment time column 621.

  FIG. 6D shows a conference scene information table 630. When the CPU 201 receives an instruction to any one of the buttons 411 to 416, the CPU 201 records a value indicating the conference scene corresponding to the button in the conference scene information table 630 together with time information (conference scene start time). The conference scene to be recorded is as described above. The conference scene start time is recorded in the conference scene start time column 631, and the conference scene is recorded in the conference scene column 632, respectively. When the CPU 201 accepts an instruction to the “end conference” button 418, the conference information collection process ends. Then, the conference information recorded in the storage 205 is transmitted to the conference information recording server 102 via the external interface 208.

  Next, conference information generated by analysis and processing by the conference information recording server 102 will be described with reference to FIG. FIG. 7 shows a data structure of conference information generated by the conference information recording server 102 and recorded in the storage 305. When the conference information collected by the conference information collection apparatus 101 is received via the external interface 308, the CPU 301 records the conference information in the storage 305. Then, the recorded conference information is analyzed and processed as follows to newly generate conference information and recorded in the storage 305 together with the previously recorded conference information.

  FIG. 7A shows the analyzed and processed face image information table. CPU301 produces | generates the meeting information which shows the mode of the transition of the facial expression of the meeting participants in a meeting as face image information. The CPU 301 extracts frame images for each minimum time interval from the moving image data recorded in the file indicated by the moving image data string 503. The position of the face portion in the frame image is specified using a discriminator or the like based on the Haar-Like feature and cut out as a face image (a plurality of images corresponding to the number of participants in the conference are obtained). Further, the time information (face image acquisition time) of the extracted frame image can be obtained from the shooting start time, the minimum time interval, and the frame rate of the moving image data. The CPU 301 records the generated face image information in the face image information table 700 together with the face image acquisition time. The CPU 301 records the face image acquisition time in the face image acquisition time column 701. In addition, the CPU 301 records each face image as a file in the storage 305, concatenates the file names separated by commas, and records them in the face image sequence 702.

  FIG. 7B shows a speaker information table that has been analyzed and processed. CPU301 produces | generates the meeting information which shows the mode of switching of a speaker as speaker information about the conversation contained in audio | voice information. The CPU 301 extracts a sound section from the sound data recorded in the file indicated by the sound data string 513. The CPU 301 divides each extracted voiced section into several sections, and performs voice signal processing on the voice data included in each section to obtain a voiceprint. If the matching process is performed on the voiceprint group thus obtained, it is possible to specify the speech of the same person. The CPU assigns speaker names such as speakers A and B to the speaker specified in this way.

  Also, the time information of the speech (from the speech start time to the speech end time) can be obtained from the recording start time of the speech information and the sampling frequency of the speech data. As described above, for each conference participant, the information indicating the speaker and the start time and end time of the speech can be obtained for each speech in the conference.

  Furthermore, a plurality of frame images before and after the speech start time are obtained from the frame images included in the moving image information, and a face image can be obtained by the same method as the generation of the face image information described above. Then, if the movement of the mouth part is detected and analyzed for the face images of the conference participants over a plurality of frames, the face image of the speaker can be estimated.

  The CPU 301 records the generated speaker information in the speaker information table 710. The CPU 301 stores the speech time (the speech start time and the speech end time are connected by “-”) in the speech time column 711, and the speaker in the speaker sequence. 712, respectively. Further, the CPU 301 records the face image estimated as described above, extracted from the frame image nearest to the utterance time, as a file in the storage 305, and records the file name in the speaker face image sequence 713. .

  FIG. 7C shows the keyword information table 720. CPU301 acquires a text from meeting information and extracts a keyword from the said text. The target conference information is text obtained by performing voice recognition processing on the voice data indicated by the voice data string 513 and text recorded in the agenda text data string 522. The target conference information is text obtained by performing OCR processing on the output image to the projector device 211 recorded in the output image sequence 532.

  As an example of keyword extraction, the following method can be mentioned, for example. The CPU 301 analyzes these texts and analyzes them into morphemes. Then, for each decomposed morpheme, an index of importance is calculated by TF-IDF (Term Frequency-Inverse Document Frequency). Those whose values are equal to or greater than a certain value are extracted as keywords.

  Furthermore, the appearance time column 721 and the extraction source column 723 are calculated for each of the extracted keywords. If the extracted keyword is included in the text obtained by the speech recognition process, speech information can be cited as an extraction source. The appearance time column 721 can be calculated from the processing section and the sampling rate of the speech data that has been recognized. If the extracted keyword is included in the agenda text information, the agenda text information can be cited as an extraction source. The appearance time can be specified by the value of the input time column 521 corresponding to the agenda text data column 522 including the keyword. If the extracted keyword is included in the text obtained by the OCR processing of the output image, the conference material information can be cited as the extraction source. The appearance time can be specified by the value of the output image acquisition time column 531 of the output image including the keyword. The CPU 301 records the appearance time, keyword, and extraction source thus obtained in the keyword information table 720. The CPU 301 records the appearance time in the appearance time column 721, the keyword in the keyword column 722, and the extraction source in the extraction source column 723, respectively. Through the above processing, the conference information collection server 102 can collect, generate and record conference information related to the conference.

  Next, conference information displayed on the screen of the computer 105 (or the computer 103 or the computer 104) will be described. Since the conference information is information composed of a large amount of data, it is displayed on the screen so that the type of data to be noted is suggested so that the user can browse efficiently. For this reason, the display method of each item of conference information is determined as follows according to the type of conference scene information.

  When the meeting scene information is “opening”, it is considered that important information such as the purpose and background of the meeting is recorded as the agenda text. Therefore, in this case, the agenda text information is highlighted and displayed. When the conference scene information is “presentation”, it is considered that the presenter made a presentation using the conference material projected by the projector device 211. Therefore, what should be noted is the meeting material. Also, during presentations using projectors, the lighting is often turned off so that the projection surface can be seen well. When the presentation is interrupted for some reason, an operation such as raising the lighting is once performed. In this way, the level of presentation and lighting are highly relevant. Therefore, when the conference scene information is “presentation”, the conference material information and the brightness information are highlighted and displayed.

  When the conference scene information is “Q & A”, it is considered important who asked what kind of content and who answered such content. Further, the facial expressions of the speakers (questioners and respondents) at that time are also important information for reviewing the conference. For example, it is possible to grasp whether it was a constructive question or a negative question. Therefore, when the meeting scene information is “Q & A / Response”, the agenda text information and the speaker information are highlighted.

  When the conference scene information is “discussion”, it is important for reviewing the conference to be able to easily grasp the flow and excitement of the discussion. Also, in order to infer the quality of the discussion, the facial expressions of conference participants other than the speakers are also useful information. Therefore, when the conference scene information is “discussion”, the speaker information, the volume information, and the face image information are highlighted and displayed. In addition, the content of the remarks under discussion is in the middle of the process, and it may change over and over, and the conclusions and decisions to be summarized later are important. Therefore, the agenda text information that will be recorded during the discussion is not important.

  When the conference scene information is “contemplation / silence”, the progress of the conference is stagnant, so there is little information useful for reflection. Therefore, when the conference scene information is “contemplation / silence”, no conference information is highlighted.

  When the meeting scene information is “summary”, the conclusion in the meeting is made, and the decision items and action items are recorded in the agenda text. Therefore, when the meeting scene information is “Summary”, the agenda text information is highlighted and displayed in text.

  Based on the above, the display of conference information will be described. 8 and 9 show examples of screens displayed on the display device 317 of the computer 105 according to the present embodiment. The CPU 311 receives the conference information from the conference information recording server 102 via the external interface 318, generates a display image 800 (conference information screen) based on the conference information, and displays it on the display device 317.

  The timeline display part 801 displays a timeline. The timeline is displayed with scales at regular intervals. This fixed interval (distance between scales) is called “unit time” of the timeline. In this embodiment, it is assumed that the display image 800 shows an example in which the unit time of the timeline is 5 minutes. In the display image 800, data of each item (type) of conference information is displayed side by side in a direction perpendicular to the timeline (horizontal direction in the example of FIG. 8). When the user instructs the unit time reduction button 802 shown in FIG. 9 via the input device 316, the unit time of the timeline can be changed to a value smaller than 5 minutes. When the user designates the unit time enlargement button 803 shown in FIG. 9, the unit time on the timeline can be changed to a value larger than 5 minutes. The minimum value of the unit time is the minimum time interval, and the maximum value may be 10 minutes, for example, and an arbitrary unit time may be set.

  The meeting scene information display part 804 is a display part showing meeting scene information. The CPU 311 generates a display image of the conference scene information display part 804 according to the contents of the conference scene information table 630. The CPU 311 assigns patterns or colors to the conference scenes “opening”, “presentation”, “question / answer”, “discussion”, “contemplation / silence”, and “summary”. Then, according to the values of the conference scene start time column 631 and the conference scene column 632, a band-like rectangle along the timeline is drawn with the above-described pattern (color). The change of the belt-like rectangle indicates a change in the conference scene. In this embodiment, the switching of the conference scene is displayed depending on the color. For example, when the user uses the input device 316 to move a cursor (not shown) to a belt-like rectangle, the conference scene name pops up. May be.

  The agenda text information display part 805 is a display part showing the agenda text information. The CPU 311 generates a display image of the agenda text information display part 805 in accordance with the contents of the agenda text information table 520. For a section of a unit time on the timeline, a record in which the time recorded in the input time column 521 is included in the time zone of the section is searched. Then, the values of the agenda text data string 522 of the corresponding record are drawn side by side in the timeline direction. At this time, the CPU 301 emphasizes the agenda text information display area 805 when the meeting scene information of the section of the timeline is “opening”, “question / answer”, or “summary”, as compared to the case where it is not. And draw. For example, the font size is increased, the typeface is bold, and the agenda text information display area 805 is surrounded by a red frame. When a plurality of conference scenes are included in a certain section, the one having the largest ratio is determined as the conference scene in that section. The same applies to the description of other display parts described below.

  The meeting material information display part 806 is a display part showing meeting material information. The CPU 311 generates a display image of the meeting material information display part 806 according to the contents of the meeting material information table 530. In the conference material information table 530, there may be a case where a plurality of records corresponding to a unit time section with a timeline are recorded. In this case, the record whose output image acquisition time column 531 is closest to the start time of the section is selected from the plurality of records. Then, the output image indicated by the value of the output image sequence of the record is output to the display image 800.

  At this time, when the conference scene information in the section of the timeline is “presentation”, the CPU 311 draws the conference material information display region 806 with emphasis as compared with the case where the conference scene information is not. For example, the output image is drawn with a larger size (for example, enlarged by 1.5 times), and the meeting material information display region 806 is drawn with a red frame. Even if the conference scene information is “presentation”, the conference scene information of the previous section is “presentation”, and the output image drawn in the previous section and the output image drawn in the current section If there is no difference between them, the highlighting may not be performed. That is, even when the conference scene information is “presentation”, the highlight is displayed only when the content of the conference material changes. In addition, each rectangle shown in the part of the meeting material information display part 806 in the drawing indicates an output image.

  The speaker information display part 807 is a display part showing speaker information. The CPU 311 generates a display image of the speaker information display part 807 according to the contents of the speaker information table 710. The CPU 311 assigns a line type (straight line, broken line, dotted line, etc.) or color to each speaker recorded in the speaker column 712. Then, according to the time in the speech time column 711, a line segment along the time line is drawn with the above-described line type or color for the time the speaker is speaking. The change of the line segment indicates the change of the speaker.

  In addition, the face image at the start of the speech indicated by the value of the speaker face image sequence 713 is acquired and drawn near the drawing start position of the line segment. At this time, the CPU 311 emphasizes and draws the speaker information display part 807 when the conference scene information of the relevant section of the timeline is “question / answer” or “discussion”, as compared with the case where it is not. To do. For example, the speaker face image is enlarged (for example, enlarged by 1.5 times) and output, and the speaker face image is drawn surrounded by a broken line frame or a red frame. Alternatively, when the conference scene is “question / answer” and “discussion”, not only the speaker face image at the start of the speech, but also a plurality of speaker face images during the speech may be drawn. In addition, each rectangle shown in the portion of the speaker information display portion 807 in the figure indicates a speaker face image.

  The keyword information display part 808 is a display part indicating keyword information. The CPU 311 generates a display image of the keyword information display part 808 according to the content of the keyword information table 720. For a section of a unit time on the timeline, a record in which the time recorded in the appearance time column 721 is included in the time zone of the section is searched. Then, the values in the keyword column 722 of the retrieved records are drawn side by side in the timeline direction. Therefore, if the unit time is large, more keyword information is included in the section, and conversely if the unit time is small, the keyword information is included in the section. At this time, the character color or font for displaying the keyword may be changed according to the value of the extraction source column 723. Thereby, it can be shown in which meeting information the keyword characterizing the meeting is included.

  The important tag information display part 809 is a display part showing important tag information. The CPU 311 generates a display image of the important tag information display part 809 according to the contents of the important tag information table 620. For a section of a unit time on the timeline, a record search is performed in which the time recorded in the important tag assignment time column 621 is included in the time zone of the section. Then, in accordance with the number of retrieved records, marks (for example, check marks) indicating important tags are drawn side by side. Therefore, if the unit time is large, the important tag information included in a certain unit time increases. Conversely, if the unit time is small, the important tag information included in a certain unit time decreases.

  The temperature information display part 810 is a display part showing temperature information. The CPU 311 generates a display image of the temperature information display part 810 according to the contents of the temperature information table 540. From the time in the temperature data acquisition time column 541 and the temperature data in the temperature data column 542, a line graph along the time line is drawn. The change in the mountain of the line graph indicates the environment of the conference venue (the cold day and the conference room were cold, etc.).

  The volume information display part 811 is a display part showing volume information. The CPU 311 generates a display image of the volume information display part 811 according to the contents of the volume information table 600. A line graph along the timeline is drawn from the time in the volume data acquisition time column 601 and the volume data in the volume data column 602. The change in the mountain of the line graph shows the excitement of the meeting and contributes to recalling the contents of the meeting. At this time, when the conference scene information of the unit time in the timeline is “discussion”, the CPU 311 emphasizes and draws the volume information display portion 811 as compared to the case where the conference scene information is not. For example, the line graph line is thickened and the volume information display region 811 is drawn surrounded by a broken line frame or a red frame.

  The lightness information display part 812 is a display part showing lightness information. The CPU 311 generates a display image of the lightness information display part 812 according to the content of the lightness information table 610. A line graph along the timeline is drawn from the time in the brightness data acquisition time column 611 and the brightness data in the brightness data column 612. The change in the mountain of the line graph indicates the environment of the conference hall (the lighting was dark and the lighting was dark). At this time, when the conference scene information of the unit time in the timeline is “presentation”, the CPU 311 emphasizes and draws the lightness information display part 812 as compared to the case where it is not. For example, the line graph line is thickened and the brightness information display region 812 is drawn with a red frame.

  The moving image information display part 813 is a display part indicating moving image information. The CPU 311 obtains and draws a frame image corresponding to the start time for a certain unit time interval on the timeline from the moving image data indicated by the moving image data string 503. When an operation instruction is given to the moving image information display portion 813, the moving image data can be reproduced and displayed from the start time of the section. In addition, each rectangle shown in the moving image information display part 813 in the figure indicates a frame image.

  The face image information display part 814 is a part for displaying face image information. The CPU 311 generates a display image of the face image information display part 814 according to the contents of the face image information table 700. For a section of a unit time on the timeline, a record in the face image acquisition time column 701 in which the nearest time is recorded after the start time is searched. Then, the face image indicated by the value recorded in the face image sequence 702 of the record is acquired and drawn. At this time, when the conference scene information of the unit time on the timeline is “discussion”, the CPU 311 emphasizes and displays the face image information display part 814 as compared to the case where the conference scene information is not. For example, the face image is drawn with a larger size (for example, enlarged by 1.5 times), and the face image information display portion 814 is drawn with a red frame. In addition, each rectangle shown in the part of the face image information display part 814 in the figure indicates a face image.

  Next, a method in which the CPU 311 generates the display image 800 shown in FIGS. 8 and 9 will be described with reference to FIGS. 10 to 14. 10 to 14 are flowcharts showing the procedure of the display image generation process. The CPU 311 of the computer 105 reads the conference information display program recorded in the storage 315, expands it in the RAM 314, and executes it. Accordingly, the computer 105 can execute display image generation processing and display the screens shown in FIGS. 8 and 9 on the display device 317. The CPU 311 of the computer 105 executes display image generation processing when an instruction to display conference information is made by the user or when an instruction to operate the unit time reduction button 802 or the unit time enlargement button 803 is made by the user.

  First, in step 901, the CPU 311 receives conference information to be displayed from the conference information recording server 102 via the external interface 318, and temporarily records it in the storage 315. In step 902, the CPU 311 determines a unit time of the timeline. For example, it is fixedly determined to be 5 minutes by default, but when a unit time reduction button 802 or a unit time expansion button 803 is instructed to operate, the value is set to a value smaller / larger than 5 minutes depending on the operation content. decide.

  In step 903, the CPU 311 calculates the start time and end time of the timeline. For this, first, the shooting start time column 501 and the shooting end time column 502 of the moving image information table 500 are acquired as the conference start time and the conference end time, respectively. Then, the time immediately before the conference start time and divisible by the unit time is obtained as the timeline start time. For example, if the conference start time is 13:00:15 (13:00:15) and the unit time is 5 minutes, 13:00:00 is obtained as the timeline start time. As another example, if the conference start time is 13:08:00 and the unit time is also 5 minutes, 13:05:00 is obtained as the timeline start time. Also, the time immediately after the conference end time and divisible by the unit time is obtained as the timeline end time. For example, if the conference end time is 14:28:55, 14:30: 00 is obtained as the timeline end time.

  In step 904, the CPU 311 calculates the number of sections per unit time from the timeline start time to the timeline end time. This is obtained by dividing the difference between the timeline end time and the timeline start time by the unit time. For example, if the timeline start time is 13:00 and the timeline end time is 14:30, the difference is 90 minutes. Dividing by the unit time of 5 minutes gives 18 sections. The CPU 311 performs a drawing process for each calculated interval.

  In step 905, the CPU 311 determines whether or not the conference information of those sections has been drawn on the display image 800 by the number of sections obtained in step 904. In other words, it is determined whether or not the processing from step 906 to step 918 has been executed for the number of sections. When it determines with YES, a process transfers to step 919, and when it determines with NO, a process transfers to step 906.

  In step 906, the CPU 311 draws the timeline of the processing target section on the display image 800. A straight line of the timeline and a scale of the start time of the section are drawn. In step 907, the CPU 311 draws the conference scene information display part 804 in the processing target section on the display image 800. The CPU 311 searches the conference scene information table 630 for records in which the time in the conference scene start time column 631 is included in the section. Then, according to the contents of the acquired record, as described above, a band-like rectangle is drawn on the display image 800 with a pattern or color assigned to each conference scene.

  In step 908, the CPU 311 aggregates the records in the conference scene information table 630 acquired in step 907 and calculates the ratio of the conference scene in the section to be processed. When a plurality of conference scenes are included in the section, the one having a large ratio is determined as the conference scene of the section. And it uses for the determination of the display method of the other meeting information mentioned later.

  In step 909, the CPU 311 draws the agenda text information display part 805 of the section to be processed on the display image 800. In step 910, the CPU 311 draws the conference material information display region 806 in the processing target section on the display image 800. In step 911, the CPU 311 draws the speaker information display part 807 in the processing target section on the display image 800. In step 912, the CPU 311 draws the keyword information display part 808 in the processing target section on the display image 800. Details of the processing in steps 909 to 912 will be described later with reference to FIG.

  In step 913, the CPU 311 draws the important tag information display part 809 in the processing target section on the display image 800. In step 914, the CPU 311 draws the temperature information display part 810 of the section to be processed on the display image 800. In step 915, the CPU 311 draws the volume information display part 811 in the processing target section on the display image 800. Details of the processing in steps 913 to 915 will be described later with reference to FIG. In step 916, the CPU 311 draws the brightness information display part 812 of the section to be processed on the display image 800. Details of the processing in step 916 will be described later with reference to FIG. In step 917, the CPU 311 draws the moving image information display part 813 in the processing target section on the display image 800. In step 918, the CPU 311 draws the face image information display part 814 in the processing target section on the display image 800. Details of the processing of step 917 and step 918 will be described later with reference to FIG.

  The CPU 311 draws the conference information of the sections as many as the number of sections on the display image 800, and then advances the processing from step 905 to step 919. The unit time reduction button 802 and the unit time enlargement button 803 are displayed on the display image 800. Draw for 800. Then, the process ends.

  Next, with reference to FIG. 11, the details of the processing of step 909 to step 912 described in FIG. 10 will be described. A process 1000 shown in FIG. 11A is a flowchart showing details of step 909 (drawing of the agenda text information display part 805). In step 1001, the CPU 311 acquires agenda text information to be drawn. That is, the CPU 311 searches the agenda text information table 520 for a record whose time in the input time column 521 is included in the section to be processed.

  In step 1002, the CPU 311 determines whether or not the agenda text information to be drawn has been acquired in step 1001, that is, whether or not the number of acquired records is one or more. When it determines with YES, a process transfers to step 1003, and when it determines with NO, a process is complete | finished.

  In step 1003, the CPU 311 determines whether the conference scene in the section determined in step 908 is “opening”, “question / answer”, or “summary”. If YES is determined, the process transitions to step 1004, and if NO is determined, the process transitions to step 1005.

  In step 1004, the CPU 311 arranges the values of the agenda text data string 522 of the record acquired in step 1001 in the timeline direction and draws it with emphasis on the display image 800. As described above, “emphasis” means, for example, increasing the font size, making the typeface bold, and enclosing the agenda text information display portion 805 with a broken line frame or a red frame. The above-described enhancement processing is not limited to the above-described example, and arbitrary enhancement processing may be performed. For example, different emphasis processing may be performed on each of “opening”, “question / answer”, and “summary”. On the other hand, in step 1005, the CPU 311 draws the values of the agenda text data string 522 of the record acquired in step 1001 in the timeline direction (without emphasis).

  A process 1010 shown in FIG. 11B is a flowchart showing details of step 910 (drawing of the conference material information display part 806). In step 1011, the CPU 311 acquires conference material information to be drawn. In other words, the CPU 311 searches the conference material information table 530 for a record in which the time in the output image acquisition time column 531 is included in the processing target section. Further, among the acquired records, the record closest to the start time of the section is specified. And the output image which the output image sequence 532 of the said record shows is acquired.

  In step 1012, the CPU 311 determines whether the drawing target meeting material information has been acquired in step 1011, that is, whether the number of acquired records is one or more. When it determines with YES, a process transfers to step 1013, and when it determines with NO, a process is complete | finished.

  In step 1013, the CPU 311 determines whether or not the conference scene in the section determined in step 908 is “presentation”. If YES is determined, the process transitions to step 1014, and if NO is determined, the process transitions to step 1015.

  In step 1014, the CPU 311 draws the output image acquired in step 1011 with emphasis on the display image 800. As described above, the “enhancement” processing in step 1014 is, for example, by enlarging the output image (for example, enlarging by 1.5 times) and enclosing the meeting material information display portion 806 with a broken line frame or a red frame. is there. On the other hand, in step 1015, the CPU 311 draws the output image acquired in step 1011 (without emphasis).

  A process 1020 shown in FIG. 11C is a flowchart showing details of step 911 (drawing of the speaker information display portion 807). In step 1021, the CPU 311 acquires speaker information to be drawn. That is, the CPU 311 searches the speaker information table 710 for a record in which the time in the speech time string 711 is included in the section to be processed.

  In step 1022, the CPU 311 determines whether or not the speaker information to be drawn has been acquired in step 1011, that is, whether or not the number of acquired records is one or more. When it determines with YES, a process transfers to step 1023, and when it determines with NO, a process is complete | finished. In step 1023, the CPU 311 determines whether or not the conference scene in the section determined in step 908 is “Q & A” or “Discussion”. When it determines with YES, a process transfers to step 1024, and when it determines with NO, it changes to step 1025.

  In step 1024, as described above, the CPU 311 displays the line segment corresponding to the speaker in the speaker column 712 of the record acquired in step 1021 and the speaker face image indicated by the value of the speaker face image column 713 as the display image. Drawing with emphasis on 800. The “enhancement” process in step 1024 is to output the speaker face image larger (for example, 1.5 times larger) as described above, and to surround the speaker face image with a broken line frame or a red frame. It is.

  On the other hand, in step 1025, the CPU 311 displays a line segment corresponding to the speaker in the speaker column 712 of the record acquired in step 1021 and the face image indicated by the value of the speaker face image column 713 with respect to the display image 800. Draw (without emphasis).

  A process 1030 shown in FIG. 11D is a flowchart showing details of step 912 (drawing of the keyword information display portion 808). In step 1031, the CPU 311 acquires keyword information to be drawn. That is, the CPU 311 searches the keyword information table 720 for a record in which the time in the appearance time column 721 is included in the section to be processed.

  In step 1032, the CPU 311 determines whether or not the keyword information to be drawn has been acquired in step 1031, that is, whether or not the number of acquired records is one or more. When it determines with YES, a process transfers to step 1033, and when it determines with NO, a process is complete | finished. In step 1033, the CPU 311 arranges the values in the keyword string 722 of the record acquired in step 1031 in the timeline direction and draws the display image 800.

  Next, with reference to FIG. 12, the details of the processing in steps 913 to 915 described in FIG. 10 will be described. A process 1100 shown in FIG. 12A is a flowchart showing details of step 913 (drawing of the important tag information display portion 809). In step 1101, the CPU 311 acquires important tag information to be drawn. That is, the CPU 311 searches the important tag information table 620 for records in which the time of the important tag assignment time column 621 is included in the processing target section.

  In step 1102, the CPU 311 determines whether or not the important tag information to be drawn has been acquired in step 1101, that is, whether or not the number of acquired records is one or more. When it determines with YES, a process transfers to step 1103, and when it determines with NO, a process is complete | finished. In step 1103, the CPU 311 arranges marks (for example, check marks) indicating important tags in accordance with the number of records acquired in step 1101, and draws them on the display image 800.

  The process 1110 shown in FIG. 12B is a flowchart showing details of step 914 (drawing of the temperature information display part 810). In step 1111, the CPU 311 acquires temperature information to be drawn. That is, the CPU 311 searches the temperature information table 540 for a record in which the time in the temperature data acquisition time column 541 is included in the section to be processed. In step 1112, the CPU 311 draws a line graph on the display image 800 in accordance with the value of the temperature data string 542 of the record acquired in step 1111.

  A process 1120 shown in FIG. 12C is a flowchart showing details of step 915 (drawing of the volume information display portion 811). In step 1121, the CPU 311 acquires volume information to be drawn. That is, the CPU 311 searches the volume information table 600 for a record in which the time in the volume data acquisition time column 601 is included in the section to be processed. In this embodiment, the acquired records are sorted in ascending order based on the volume data acquisition time column 601. In step 1122, the CPU 311 determines whether or not the processing in steps 1123 to 1129 has been repeated for the number of records acquired in step 1121. If yes, the process ends. If no, the process proceeds to step 1123.

  In step 1123, the CPU 311 determines whether or not the conference scene at the time in the volume data acquisition time column 601 of the record to be processed is “discussion”. For this determination, the CPU 311 searches the conference scene information table 630, and the record in which the time in the conference scene start time column 631 is immediately before and immediately after the volume data acquisition time 601 of the record to be processed is immediately after. Get the most recent record. Then, if any of the acquired conference scenes of both records is “discussion”, the CPU 311 determines YES, and the process transitions to step 1124. If NO, the process transitions to step 1128.

  In step 1124, the CPU 311 determines whether or not the conference scene is in the “discussion” state. For this, it is determined whether or not the volume discussion flag data is stored in the RAM 314. The volume discussion flag is information indicating that, when step 1123 has been executed previously, YES is determined (the conference scene is determined as “discussion”), and the state continues. In other words, it indicates that the determination in step 1123 for the previous section was a “discussion” determination. If YES is determined, the process transitions to step 1126, and if NO is determined, the process transitions to step 1125.

  The determination of NO in step 1124 indicates that the conference scene has changed from a state other than “discussion” to a “discussion” state. Therefore, the CPU 311 draws a lower horizontal line of a frame for highlighting (a broken line frame or a red frame) in step 1125. Further, at this time, the volume discussion flag data is stored in the RAM 314.

  In step 1126, the CPU 311 draws a line graph with emphasis on the display image 800 based on the value of the volume data string 602 of the previous processing record and the current processing target record. The “emphasis” processing in step 1126 is, for example, to thicken the line graph line and surround the volume information display portion 811 with a broken line frame or a red frame as described above. Since the lower horizontal line and the upper horizontal line of the broken line frame or the red frame are drawn in step 1125 and step 1129, respectively, only the left vertical line and right vertical line are drawn in step 1126.

  On the other hand, in step 1127, the CPU 311 draws a line graph on the display image 800 (without emphasis) based on the value of the volume data string 602 of the previous processing target record and the current processing target record. . If it is determined in step 1123 that the conference scene is not “discussion” (NO), it is determined in step 1128 whether or not the conference scene has been “discussion”. For this determination, the CPU 311 determines whether or not the volume discussion flag data is stored in the RAM 314. If the CPU 311 determines YES, the process transitions to step 1129, and if NO, the process transitions to step 1127.

  A determination of YES in step 1128 indicates that the conference scene has changed from a “discussion” state to a state other than “discussion”. Therefore, in step 1129, the CPU 311 draws the upper horizontal line of a frame for highlighting (a broken line frame or a red frame). By the processing in step 1129, a frame for highlighting is completed. At this time, the volume-under-consideration flag data is deleted from the RAM 314. Then, the process proceeds to step 1127.

  A process 1200 shown in FIG. 13 is a flowchart showing details of step 916 (drawing of the brightness information display part 812) described with reference to FIG. In step 1201, the CPU 311 acquires lightness information to be drawn. That is, the CPU 311 searches the brightness information table 610 for a record in which the time in the brightness data acquisition time column 611 is included in the processing target section. In this embodiment, the acquired records are sorted in ascending order based on the brightness data acquisition time column 611.

  In step 1202, the CPU 311 determines whether or not the processing in steps 1203 to 1209 has been repeated for the number of records acquired in step 1201. If yes, the process ends. If no, the process proceeds to step 1203. In step 1203, the CPU 311 determines whether or not the conference scene at the time in the brightness data acquisition time column 611 of the record to be processed is “presentation”. For this determination, the CPU 311 searches the conference scene information table 630, and records that the time in the conference scene start time column 631 is the most recent record with respect to the time in the brightness data acquisition time column 611 of the record to be processed. , Immediately after, get the record that is recent. If any of the acquired conference scenes of both records is “presentation”, the determination is YES, and the process proceeds to step 1204. Otherwise (if NO), the process transitions to step 1208.

  In step 1204, the CPU 311 determines whether or not the conference scene is in the “presentation” state. For this determination, the CPU 311 determines whether or not the brightness presenting flag data is stored in the RAM 314. The lightness presenting flag is information indicating that, when Step 1203 has been executed previously, it is determined as YES (the conference scene is determined as “Presentation”) and the state continues. When it determines with YES, a process transfers to step 1206, and when it determines with NO, a process transfers to step 1205.

  The determination of NO in step 1204 indicates that the conference scene has changed from a state other than “presentation” to a “presentation” state. Therefore, in step 1205, the CPU 311 draws a lower horizontal line of a frame for highlighting (a broken line frame or a red frame). At this time, the CPU 311 stores lightness presenting flag data in the RAM 314.

  In step 1206, the CPU 311 draws a line graph with emphasis on the display image 800 based on the value of the lightness data string 612 of the previous processing record and the current processing target record. The “enhancement” processing in step 1206 is, for example, making the line graph line thick and surrounding the brightness information display region 812 with a broken line frame or a red frame as described above. Since the lower horizontal line and the upper horizontal line of the broken line frame or the red frame are drawn in step 1205 and step 1209, respectively, only the left vertical line and right vertical line are drawn in step 1206).

  On the other hand, in step 1207, the CPU 311 draws a line graph on the display image 800 (without emphasis) based on the value of the brightness data string 612 of the previous processing record and the current processing target record. .

  If it is determined in step 1203 that the conference scene is not “presentation” (NO), the CPU 311 determines in step 1208 whether the conference scene has been “presentation”. For this determination, the CPU 311 determines whether or not the brightness presenting flag data is stored in the RAM 314. When it determines with YES, a process transfers to step 1209, and when it determines with NO, a process transfers to step 1207.

  The determination of NO in step 1208 indicates that the conference scene has changed from the “presentation” state to a state other than “presentation”. Therefore, in step 1209, the CPU 311 draws the upper horizontal line of a frame for highlighting (a broken line frame or a red frame). At this time, the lightness presenting flag data is erased from the RAM 314. Then, the process proceeds to step 1207.

  Next, the details of the processing of step 917 and step 918 described in FIG. 10 will be described with reference to FIG. A process 1210 shown in FIG. 14A is a flowchart showing details of step 917 (drawing of the moving image information display portion 813). In step 1211, the CPU 311 acquires moving image information to be drawn. That is, the CPU 311 acquires the frame image immediately after the start time of the section to be processed from the frame image in the moving image data indicated by the moving image data string 503. In step 1212, the CPU 311 draws the frame image acquired in step 1211 on the display image 800.

  A process 1220 shown in FIG. 14B is a flowchart showing details of step 918 (drawing of the face image information display part 814). In step 1221, the CPU 311 acquires face image information to be drawn. That is, the CPU 311 searches the face image information table 700 for a record in which the time in the face image acquisition time column 701 is included in the processing target section. Further, among the acquired records, the record closest to the start time of the section is specified. Then, the face image indicated by the face image sequence 702 of the record is acquired.

  In step 1222, the CPU 311 determines whether or not the conference scene in the section determined in step 908 is “discussion”. When it determines with YES, a process transfers to step 1223, and when it determines with NO, a process transfers to step 1224. In step 1223, the CPU 311 draws the face image acquired in step 1221 with emphasis on the display image 800. As described above, “enhancement” means, for example, enlarging the face image (for example, enlarging it by 1.5 times) and surrounding the face image information display region 814 with a broken line frame or a red frame. On the other hand, in step 1224, the CPU 311 draws the face image acquired in step 1221 (without emphasis).

  As described above, according to the present embodiment, when conference information including a plurality of items is displayed, the item to be noted is highlighted according to the conference scene in order to suggest the type of data to be noted. . Thereby, even if a user is a lot of meeting information, he can browse efficiently and suitably.

  When an operation instruction is given to the moving image information display portion 813, the display device 317 reproduces and displays moving image data from the start time of the unit time section corresponding to the instruction target portion. At this time, other conference information may be displayed together with the moving image data. At this time, it is preferable to update the display contents of the conference information as the timeline progresses. Furthermore, the item of the conference information displayed together with the moving image data may determine the item of the conference information to be displayed according to the conference scene information in the same manner as highlighted in the generation of the display image 800.

  In the present embodiment, as a method for collecting a conference scene, a method in which a user selects and inputs a conference scene by operating an input device of the conference information collection device is shown, but another method may be used. For example, as disclosed in Japanese Patent Application Laid-Open No. 2006-302045, a conference scene may be estimated from the number of speakers and the degree of overlap of utterances.

  In the present embodiment, an example in which the conference scene is “opening”, “presentation”, “question / answer”, “discussion”, and “summary” is shown, but this is not restrictive. Other meeting scenes may be defined, the type of data to be noticed may be determined accordingly, and the display method may be determined accordingly. Also, the types of data to be noted in each conference scene are not fixed to those shown in the present embodiment, and other combinations may be used. Furthermore, the conference scene and the type of data to be noted in the conference scene may be settable.

  In this embodiment, the conference information recording system 100 includes a plurality of devices. However, the system configuration is not limited to this embodiment. For example, a device in which the conference information collection device 101 and the conference information recording server are integrated may be used. If a display device is mounted on this apparatus, the same processing as that of the system described in this embodiment can be executed by one apparatus.

(Example 2)
In the first embodiment, an example in which items of conference information are highlighted and displayed according to the conference scene is shown. In the present embodiment, an example will be described in which notable items are displayed, but notable items are extracted and displayed.

  FIG. 15 shows an example of a screen displayed by the display device 317 of the computer 105 in the second embodiment. In the first embodiment, the CPU 311 uses all the meeting information received from the meeting information collection server 102 and generates the display image 800 by emphasizing the items of the meeting information according to the meeting scene. In the present embodiment, items of conference information are extracted according to the conference scene, and the display image 1300 is generated using only the extracted items. The items of important conference information for each conference scene are the same as those in the first embodiment, but in this embodiment, only the items of important conference information are extracted and displayed in the display image 1300 as shown in the figure. draw.

  The time zone 1301 is a time zone in which the conference scene information is “opening”. In this time zone, the CPU 311 extracts the agenda text information and draws it on the display image 1300. The time zone 1302 is a time zone in which the conference scene information is “presentation”. In this time zone, the CPU 311 extracts the conference material information and the brightness information and draws them on the display image 1300. The time zone 1303 is a time zone in which the conference scene information is “Q & A”. In this time zone, the CPU 311 extracts the agenda text information and the speaker information and draws them on the display image 1300. The time zone 1304 and the time zone 1306 are time zones in which the conference scene is “discussion”. In this time zone, the CPU 311 extracts speaker information, volume information, and face image information and draws them on the display image 1300. The time zone 1305 is a time zone in which the conference scene is “contemplation / silence”. The CPU 311 determines that there are no items to be extracted during this time period, and draws nothing on the display image 1300 other than the conference scene information. The time zone 1307 is a time zone in which the conference scene is “summary”. In this time zone, the CPU 311 extracts the agenda text information and draws it on the display image 1300.

  The whole display button 1308 is a button for instructing display of items other than the items extracted as described above. When the whole display button 1308 is instructed, the CPU 311 generates a display image 800 and causes the display device 317 to display it.

  Next, a method in which the CPU 311 generates the display image 1300 illustrated in FIG. 13 will be described with reference to FIGS. 16 to 18 are flowcharts showing the procedure of the display image generation process. In the computer 105, the CPU 311 reads the conference information display program recorded in the storage 315, expands it in the RAM 314, and executes it. As a result, the computer 105 can execute the display image generation process and display the screen shown in FIG. 15 on the display device 317. The CPU 311 of the computer 105 executes display image generation processing when an instruction to display conference information is made by the user or when an instruction to operate the unit time reduction button 802 or the unit time enlargement button 803 is made by the user.

  Steps 901 to 908 and 919 shown in FIG. 16 are the same as the steps described in FIG. When the CPU 311 determines the conference scene of the section to be processed in S908, the process proceeds to step 1401. In step 1401, the CPU 311 draws agenda text information for the section to be processed. In step 1402, the CPU 311 draws conference material information for the section to be processed. In step 1403, the CPU 311 draws speaker information for the section to be processed. Details of the processing in steps 1401 to 1403 will be described later with reference to FIG.

  In step 1404, the CPU 311 draws volume information for the section to be processed. In step 1405, the CPU 311 draws lightness information for the section to be processed. In step 1406, the CPU 311 draws face image information for the section to be processed. The CPU 311 advances the processing to step 1407 after drawing the conference information of the sections on the display image 1300 by the number of sections. Details of the processing in steps 1404 to 1406 will be described later with reference to FIG. In step 1407, the entire display button 1308 is drawn on the display image 1300. Then, the process ends.

  Next, details of the processing of Steps 1401 to 1403 described with reference to FIG. 16 will be described with reference to FIG. A process 1500 shown in FIG. 17A is a flowchart showing details of step 1401 (drawing of agenda text information). In step 1501, the CPU 311 determines whether the conference scene in the section determined in step 908 is “opening”, “question / answer”, or “summary”. If YES is determined, the process proceeds to step 1502. If NO is determined, the process ends without drawing anything on the display image 1300. In step 1502, the CPU 311 acquires agenda text information to be drawn. That is, the CPU 311 searches the agenda text information table 520 for a record whose time in the input time column 521 is included in the section to be processed. In step 1503, the CPU 311 arranges the values of the agenda text data string 522 of the record acquired in step 1502 in the timeline direction and draws it on the display image 1300.

  A process 1510 shown in FIG. 17B is a flowchart showing details of step 1402 (drawing conference material information). In step 1511, the CPU 311 determines whether or not the conference scene in the section determined in step 908 is “presentation”. If YES is determined, the process proceeds to step 1512. If NO is determined, the process ends without drawing anything on the display image 1300. In step 1512, the CPU 311 acquires conference material information to be drawn. That is, the CPU 311 searches the conference material information table 530 for a record in which the time of the output image acquisition time 531 is included in the section to be processed and acquires it. Further, among the acquired records, the record closest to the start time of the section is specified. And the output image which the output image sequence 532 of the said record shows is acquired. In step 1513, the CPU 311 draws the output image acquired in step 1512 on the display image 1300.

  A process 1520 shown in FIG. 17C is a flowchart showing details of step 1403 (drawing speaker information). In step 1521, the CPU 311 determines whether or not the conference scene in the section determined in step 908 is “Q & A” or “Discussion”. If YES is determined, the process proceeds to step 1522. If NO is determined, the process ends without displaying anything on the display image 1300. In step 1522, the CPU 311 acquires speaker information to be drawn. That is, the CPU 311 searches the speaker information table 710 for a record in which the time in the speech time string 711 is included in the section to be processed. In step 1523, as described above, the CPU 311 displays the line segment corresponding to the speaker in the speaker column 712 of the record acquired in step 1522 and the speaker face image indicated by the value of the speaker face image column 713 as the display image. Draw for 1300.

  Next, details of the processing of Steps 1404 to 1406 described with reference to FIG. 16 will be described with reference to FIG. A process 1530 shown in FIG. 18A is a flowchart showing details of step 1404 (drawing of volume information). In step 1531, the CPU 311 acquires volume information to be drawn. That is, the CPU 311 searches the volume information table 600 for a record in which the time in the volume data acquisition time column 601 is included in the section to be processed. In this embodiment, the acquired records are sorted in ascending order based on the volume data acquisition time column 601.

  In step 1532, the CPU 311 determines whether or not the processing in steps 1533 to 1535 has been repeated for the number of records acquired in step 1531. If yes, the process ends. If no, the process proceeds to step 1533.

  In step 1533, the CPU 311 determines whether or not the conference scene at the time in the volume data acquisition time column 601 of the record to be processed is “discussion”. For this, the conference scene information table 630 is searched, and the time in the conference scene start time column 631 is the record immediately before and the latest immediately after the volume data acquisition time 601 of the record to be processed. Get a record. If any of the acquired conference scenes of both records is “discussion”, the determination is YES, and the process proceeds to step 1534. Otherwise (if NO), go to step 1532. At this time, the volume discussion flag data (described later) is deleted from the RAM 314.

  In step 1534, the CPU 311 determines whether or not the conference scene is in the “discussion” state. For this, it is determined whether or not the volume discussion flag data is stored in the RAM 314. The volume-in-progress flag is information indicating that when step 1533 has been executed previously, it is determined as YES (the conference scene is determined as “discussion”) and the state continues. If the volume discussion flag data is stored and it is determined YES, the process transitions to step 1535, and if it is determined NO, the process transitions to step 1532. In step 1535, the CPU 311 draws a line graph on the display image 1300 based on the value of the volume data string 602 of the previous processing record and the current processing target record.

  A process 1540 is a flowchart showing details of step 1405 (lightness information drawing). In step 1541, the CPU 311 acquires lightness information to be drawn. That is, the CPU 311 searches the brightness information table 610 for a record in which the time in the brightness data acquisition time column 611 is included in the processing target section. In this embodiment, the acquired records are sorted in ascending order based on the brightness data acquisition time column 611.

  In step 1542, the CPU 311 determines whether or not the processing in steps 1543 to 1545 has been repeated for the number of records acquired in step 1541. If yes, the process ends. If no, the process proceeds to step 1543.

  In step 1543, the CPU 311 determines whether or not the conference scene at the time in the brightness data acquisition time column 611 of the record to be processed is “presentation”. For this, the conference scene information table 630 is searched, and the record in the conference scene start time column 631 is the record immediately before and the record immediately after the record in the brightness data acquisition time column 611 of the record to be processed. Get a record. If any of the acquired conference scenes of both records is “presentation”, the determination is YES, and the process proceeds to step 1544. Otherwise (if NO), transition to step 1542. At this time, lightness presenting flag data (described later) is deleted from the RAM 314.

  In step 1544, the CPU 311 determines whether or not the conference scene is in the “presentation” state. For this, it is determined whether or not the brightness presenting flag data is stored in the RAM 314. The lightness discussion flag is information indicating that, when step 1543 has been executed previously, it is determined as YES (determined that the conference scene is “presentation”) and the state continues. If the lightness presenting flag data is stored and the determination is YES, the process proceeds to step 1545. If the determination is NO, the process proceeds to step 1542.

  In step 1545, the CPU 311 draws a line graph on the display image 1300 based on the value of the lightness data string 612 of the previous processing record and the current processing target record.

  A process 1550 shown in FIG. 18C is a flowchart showing details of step 1406 (drawing face image information). In step 1551, the CPU 311 determines whether or not the conference scene in the section determined in step 908 is “discussion”. If YES is determined, the process proceeds to step 1552. If NO is determined, the process ends without drawing anything on the display image 1300. In step 1552, the CPU 311 acquires face image information to be drawn. That is, the CPU 311 searches the face image information table 700 for a record in which the time in the face image acquisition time column 701 is included in the processing target section. Further, among the acquired records, the record closest to the start time of the section is specified. Then, the face image indicated by the face image sequence 702 of the record is acquired. In step 1553, the CPU 311 draws the output image acquired in step 1552 on the display image 1300.

  As described above, according to the present embodiment, when conference information including a plurality of items is displayed, items to be noted are extracted and displayed according to the conference scene in order to browse the items to be noted. Thereby, for example, when referring to conference information on a computer or terminal having a screen with a small display area, the user can efficiently and suitably view the conference information.

(Example 3)
In the third embodiment, a method of scrolling the conference information displayed on the display device 317 of the computer 105 in the direction according to the timeline will be described. If the meeting time is long, not all meeting information is displayed on the screen of the display device. Further, when the user uses a unit time reduction button as shown in FIG. 9, more unit time sections increase. In such a case, the user can issue a scroll instruction in a direction according to the timeline via the input device 316. In the present embodiment, a scroll display method is suitably determined according to the conference scene.

  First, the CPU 311 generates control information for scroll processing for each section. FIG. 19 is a data configuration diagram showing a configuration of data which is control information of the scroll processing. The scroll control information table 1620 is generated when the CPU 311 generates the display image 800 and is stored in the RAM 314. As will be described later, the CPU 311 controls the speed of scrolling the display image 800 in the timeline direction according to the contents of the scroll control information table 1620.

  The section position column 1621 stores the position of the section whose speed should be changed during scrolling. This position is the number of pixels in the height direction from the lower end of the display image 800, and is recorded by connecting the start position pixel number and the end position pixel number of the section whose speed is to be changed with a comma. The CPU 311 determines a part on the display image to be referred to according to the conference scene, and records the position of the part. The CPU 311 controls and changes the scroll speed according to this value when scrolling. The CPU 311 stores a deceleration (acceleration) of the scroll speed in the deceleration value column 1622. This is a deceleration value of the scroll speed determined for the section shown in the section position column 1621. In this embodiment, the CPU 311 records “A2” or “A3” as the deceleration value. The values are A2 <A1 <0 <A3, A1, which will be described in detail later.

  Next, a procedure for generating the scroll control information table 1620 will be described with reference to FIGS. FIG. 20 to FIG. 25 are flowcharts showing the procedure of the generation process of the display image 800 in this embodiment. Note that steps 901 to 919 shown in FIG. 20 are the same as the processes with the same reference numerals described in FIG.

  After determining the conference scene in the section to be processed in step 908, the CPU 311 determines in step 1701 whether the conference scene in the section is “contemplation / silence”. When it determines with YES, a process transfers to step 1702, and when it determines with NO, it changes to step 909.

  In step 1702, the CPU 311 adds the position of the section to be processed to the section position column 1621 and stores it. A3 is recorded in the deceleration value column 1622. As a result, the section in which the conference scene is “contemplation / silence” can be recorded in the scroll control information table 1620. If there is a record that is adjacent to the section to be added and has the same value in the deceleration value column 1622 for a record that has already been registered, this record is not added to the record. The value in the section position column 1621 is updated. For example, a record with the section position column 1621 having the value “480,560” (the deceleration value column 1622 has the value “A2”) already exists, and the section position column 1621 has the value “560,640” (the deceleration value). When it is desired to add a record having a value “A2”) in the column 1622, the following is performed. The CPU 311 updates the value of the section position column 1621 of the registered record to “480, 640” without adding a new record. The method for adding this record is the same in the storage processing in the scroll control information table 1620 in the subsequent steps.

  Next, details of the processing of steps 909, 910, 911, 915, and 918 of FIG. 20 will be described with reference to FIGS. A process 1800 in FIG. 21 is a drawing process of the agenda text information. Steps 1001 to 1005 are the same as the process shown in FIG. In step 1004, the CPU 311 emphasizes and draws the agenda text information of the section to be processed, and then adds and stores the position of the section to be processed in the section position column 1621 in step 1801. The deceleration value column 1622 records the deceleration value A2. As a result, a section in which the conference scene is “opening”, “question / answer”, or “summary” and the agenda text information exists can be recorded in the scroll control information table 1620.

  FIG. 22 is a drawing process for conference information. In process 1810, steps 1011 to 1015 are the same as the processes shown in FIG. After the drawing process of step 1014, the CPU 311 has the presentation material in step 1811 as “presentation”, the conference material information (output image) drawn in that zone, and the conference material information of the current processing target segment. It is determined whether or not there is a difference. When it determines with YES, a process transfers to step 1812, and when it determines with NO, a process is complete | finished.

  In step 1812, the CPU 311 adds the position of the section to be processed to the section position column 1621 and stores it. The deceleration value column 1622 records the deceleration value A2. As a result, a section in which the conference scene information is “presentation” and the conference material information has changed can be recorded in the scroll control information table 1620.

  FIG. 23 shows speaker information drawing processing. In process 1820, steps 1021 to 1025 are the same as the process shown in FIG. After the drawing process in step 1024, the CPU 311 determines in step 1821 whether or not the drawing target includes line segments of a plurality of speakers, that is, there is a speaker change. When it determines with YES, a process transfers to step 1822, and a process is complete | finished when it determines with NO.

  In step 1822, the CPU 311 adds the position of the section to be processed to the section position column 1621 and stores it. Further, the CPU 311 records the deceleration value A2 in the deceleration value column 1622. As a result, a section in which the conference scene is “question / answer” or “discussion” and the speaker is changed can be recorded in the scroll control information table 1620.

  FIG. 24 shows sound information drawing processing. In the processing 1900, steps 1121 to 1127 are the same as the processing shown in FIG. In step 1126, the CPU 311 emphasizes and draws the volume information of the section to be processed. In step 1901, the CPU 311 determines whether or not the volume data value is equal to or greater than a predetermined value. As this predetermined value, for example, an average value of all sound volume data may be calculated, and the calculated value may be set as the predetermined value. If YES is determined, the process transitions to step 1902, and if NO is determined, the process transitions to step 1122. In step 1902, the CPU 311 adds the position of the section to the section position column 1621 and stores it. The deceleration value column 1622 records the deceleration value A2. As a result, it is possible to record in the scroll control information table 1620 a section where the conference scene is “discussion” and which is greater than or equal to a predetermined value, that is, a section estimated to show excitement.

  FIG. 25 is a drawing process of face image information. In the process 2000, steps 1221 to 1224 are the same as the processes shown in FIG. After the drawing process in step 1223, the CPU 311 performs smile determination on each face image in step 2001, and determines whether any face image is a smile. The smile determination may be performed by analyzing changes such as an increase in the corner of the mouth and a decrease in the corner of the eye using a statistical identification method. When it determines with YES, a process transfers to step 2002, and when it determines with NO, a process is complete | finished.

  In step 2002, the CPU 311 adds the position of the section to the section position column 1621 and stores it. The deceleration value column 1622 records the deceleration value A2. The above is interpreted as the concentration of discussion and incandescence when there is no smile. As a result, it is possible to record in the scroll control information table 1620 the section in which the conference scene is “discussion” and it is estimated that the conference scene is concentrated and incandescent.

  Next, processing when the CPU 311 updates the display of the display device 317 by scrolling the display image 800 in the timeline direction will be described. FIG. 26 is a flowchart showing the procedure of the processing. This process is executed when the user gives a scroll instruction via the input device 316. First, in step 2101, the CPU 311 determines a scrolling (initial) speed v and a deceleration a. Here, the speed v is a speed value V1, and the deceleration a is a deceleration value A1 (A1 <0).

  In step 2102, the CPU 311 determines whether to stop scrolling. This is determined as YES if the speed v is less than or equal to a predetermined threshold, and NO if not. The predetermined threshold may be determined in advance as a numerical value close to the value 0. When it determines with NO, a process transfers to step 2103, and when it determines with YES, a process is complete | finished.

  In step 2103, the CPU 311 determines whether or not the end position is stored in the RAM 314. The end position will be described later. When it determines with YES, a process transfers to step 2104, and when it determines with NO, it changes to step 2106.

  In step 2104, the CPU 311 determines whether the position on the display image 800 indicated by the end position stored in the RAM 314 is included in the display area of the display device 317. If YES is determined, the process transitions to step 2105, and if NO is determined, the process transitions to step 2106. In step 2105, the CPU 311 updates the deceleration a to the deceleration value A1. At this time, the end position is deleted from the RAM 314.

  In step 2106, the CPU 311 calculates the movement amount d. This can be obtained by the following equation. d = v * t + 0.5 * a * t * t. In step 2107, when the display image 800 is moved by the movement amount d in step 2107, the CPU 311 performs the following operation for an area that is outside the current display area with respect to the display area of the display device 317 and is within the display area after the movement. Determine. The CPU 311 determines whether or not the start position of the value of the section position column 1621 is included in the area in the records registered in the scroll control information table 1620. When it determines with YES, a process transfers to step 2108, and when it determines with NO, a process transfers to step 2110.

  In step 2108, the CPU 311 acquires the value of the deceleration value column 1622 of the record of the scroll control information table 1620 used for the determination in step 2107, and updates the deceleration a with the value. If the value in the deceleration value column 1622 is A2, the scroll movement is controlled to be further decelerated from the next calculation of the movement amount d. If the deceleration value 1622 is A3, the scroll movement is controlled to accelerate.

  In step 2109, the CPU 311 acquires the end position of the value in the section position column 1621 of the scroll control information table 1620 used for the determination in step 2107 and stores it in the RAM 314. By determining the end position stored in the RAM 314 in the processing from step 2103 to step 2105, the value of the deceleration a updated in step 2108 can be returned to the original value A1. In step 2110, the CPU 311 moves the position of the display image 800 relative to the display area of the display device 317 in the timeline direction by the movement amount d. Then, the display device 317 updates the display.

  As described above, scrolling of the display image 800 can be suitably determined according to the conference scene. More specifically, in a portion on the display image 800 where the conference scene corresponds to “presentation”, when the conference material information changes, the scrolling speed is reduced to suggest viewing to the user. Similarly, at the location where the meeting scene corresponds to “Q & A”, the user can reduce the scrolling speed based on the fact that the proceeding text information is recorded or the speaker is switched for the speaker information. Suggest browsing.

  In addition, in places where the meeting scene corresponds to “discussion”, consideration is given to the switching of the speaker information, the excitement of the discussion estimated from the volume information, and the incandescence of the discussion estimated from the face image information. To do. Based on this, the scrolling speed is reduced to suggest browsing to the user. Where the conference scene corresponds to “contemplation / silence”, the progress of the conference is stagnant, and it is considered that there is little information useful for reviewing the conference information. Therefore, in this case, the scroll speed is accelerated, and the part is quickly scrolled. Alternatively, it may be configured such that scrolling is skipped at a location where the conference scene corresponds to “contemplation / silence”.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program. In this case, the program and the storage medium storing the program constitute the present invention.

101 meeting information collecting apparatus 102 meeting information recording server 105 computer 311 CPU
317 Display device

Claims (15)

An information processing system that includes an acquisition unit that acquires different types of data related to a conference, and that displays a screen generated based on the different types of data acquired by the acquisition unit,
Display means for displaying the screen in which the data acquired by the acquisition means are arranged in order of time acquired by the acquisition means for each type;
A setting means for setting one meeting scene at each time during the meeting from a plurality of meeting scenes,
The information processing system characterized in that the display means changes the type of the data to be highlighted for each conference scene set by the setting means. The display means displays a selection screen for accepting selection of the conference scene,
The setting means accepts the selection via the selection screen from when the selection of the conference scene is accepted via the selection screen displayed on the display means until the next selection of the conference scene is accepted. The information processing system according to claim 1, wherein the information processing system is set as the conference scene. The acquisition means acquires a start time and an end time of the conference,
The said display means displays the said screen which divided | segmented the meeting time based on the start time and the end time of the said meeting acquired by the said acquisition means for every predetermined time. The Claim 1 or 2 characterized by the above-mentioned. Information processing system. The information processing system according to claim 2 , further comprising storage means for storing the time when the conference scene is selected via the selection screen and the selected conference scene in association with each other. The information processing system according to claim 4, wherein the storage unit further stores the data acquired by the acquisition unit in association with the acquisition time of the data. The information processing system according to any one of claims 1 to 5, wherein the conference scene includes at least a presentation, a question and answer session, a discussion, and a summary. The acquisition means acquires at least audio data, text data, image data output as meeting material,
The apparatus further comprises keyword extraction means for extracting a conference keyword based on any of the audio data, the text data, and the image data output as conference material acquired by the acquisition means. Item 7. The information processing system according to Item 6. The acquisition means acquires at least audio data,
7. The apparatus according to claim 6, further comprising speaker information extraction means for extracting speaker data and a start time and an end time of the speaker's speech based on the voice data acquired by the acquisition means. The information processing system described. The information processing system further includes an imaging unit,
The acquisition means acquires image data captured by the imaging unit at least during a meeting,
The information processing system according to claim 6, further comprising face image extraction means for extracting face images of participants in the conference from the image data acquired by the acquisition means. 8. The information processing according to claim 7, wherein, when the conference scene is the presentation, the display unit enlarges and displays the image data output as the conference material among the different types of data. system. 9. The information according to claim 8, wherein, when the conference scene is the question and answer session, the display unit enlarges and displays the data of the speaker performing the question and answer session among the different types of data. Processing system. The information processing method according to claim 9, wherein when the conference scene is the discussion, the display unit enlarges and displays the face image of the speaker who performs the discussion among the different types of data. system. The information processing system according to claim 7, wherein, when the conference scene is the summary, the display unit enlarges and displays the text data among the different types of data. A control method in an information processing system that has an acquisition unit that acquires different types of data related to a conference and displays a screen generated based on the different types of data acquired by the acquisition unit,
A display step of displaying the screen in which the data acquired by the acquisition unit is arranged in order of time acquired by the acquisition unit for each type,
A setting step for setting one meeting scene from a plurality of meeting scenes at each time during the meeting;
Have
In the display step, the type of data to be highlighted and changed for each conference scene set in the setting step is changed. A computer program for executing the control method of the information processing system according to claim 14 by a computer.

Images