JP2006085440A - Information processing system, information processing method and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To produce a video index reflecting a psychological state of a person participating in a conference to efficiently look back on the conference. <P>SOLUTION: At least one of material used in the conference and a conference participant is recorded as video information of the conference, and view line information of the conference participant, or biological information such as brain waves or a pulse rate is acquired as information about the conference participant. The psychological state of the conference participant is estimated on the basis of the information about the conference participant. A part of voice video information of the conference is selected on the basis of estimated psychological state information of the conference participant, and is recorded as index information. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an information processing system and information processing method and a computer program for recording and reproducing information (conference information) such as video information, audio information, document information (presentation material) in a conference, and in particular, video in a conference. The present invention relates to an information processing system, an information processing method, and a computer program that support a conference information access operation for searching for a desired scene from conference information in which information, audio information, document information, and the like are recorded.

  More specifically, the present invention relates to an information processing system, an information processing method, and a computer for recording audio / video information in a conference, creating an index for the recorded information, and efficiently searching for desired information. The present invention relates to a program, and more particularly, to an information processing system and information processing method that efficiently creates a video index that reflects the psychological state of a person who has participated in a conference, and efficiently looks back on the conference, and a computer program.

  With the recent development of computing technology, computers have penetrated deeply into offices and general households. Coupled with this, various applications can now receive computer support. For example, conferences are frequently held in industrial activities and various other daily lives in a company. By incorporating a computer into a conference room, the operation of the conference can be supported.

  As an example of the conference support system, a remote conference system can be taken up. According to the remote conference system, in a plurality of remotely located conference rooms, image / audio input devices such as cameras and microphones, and image / audio output devices such as monitors and speakers are installed, and communication lines are used. Events such as remarks and questions and answers held in each conference room can be shared, and proceedings can be synchronized between remote locations.

  In a conference, presentation materials are projected on a screen, and many conversations are exchanged between a plurality of participants. Therefore, various media data related to the conference such as whiteboards used in the conference, slides and other presentation materials, as well as the video of the conference participants and the audio of the participants' remarks. It can be recorded and managed on a computer in synchronism with video and audio, that is, the progress of the conference. By storing the data related to the conference in this way, these can be reused in the next conference.

  Meetings are held almost every day at multiple meetings. For this reason, the amount of conference data captured and accumulated for each conference is enormous, and the number thereof increases day by day. Here, when it is desired to reuse the conference data, such as looking back on the matters to be determined in the conference, it becomes troublesome or difficult or impossible to find the desired location (reproduction position) in the desired conference. The problem emerges.

  Decisions made at meetings, etc. can be reviewed by reviewing the minutes issued later. However, since the details of the decision are not recorded, it cannot be reviewed. Meeting minutes are usually made into text from notes based on audio information spoken by the participants during the meeting. Therefore, there are few meeting information to be converted into text, and there are problems that important utterances during the meeting are not recorded and that the contents are unclear and cannot be accurately reflected. In addition, there are cases where you want to remember later things that do not appear in the minutes, such as the content of utterances and materials that seem to be important to the person, but that are off the subject.

  There is a method of using a moving image as a technology for supporting the reflection of the conference. It is a method of supporting recall by taking a video of a meeting and playing back a scene that you want to remember later. For this purpose, technology for quickly searching for a desired scene is indispensable.

  For example, in a conference system, proposals have been made for an agenda content storage device that stores not only conference participants 'comments but also images of the participants' facial expressions and graphs used as the agenda content (for example, see Patent Document 1). See From each terminal, an audio / video signal in which the content of a speech is superimposed on a video signal in which the facial expressions of the conference participants are copied is transmitted to the communication conference node via the communication network. The agenda content storage device in the communication conference node obtains the highest audio level from the audio / video signals transmitted from each terminal by the audio selection unit, and accumulates them in the audio / video storage unit. The audio-video information recorded in this way can clearly convey the expression of the speaker and the nuances of words. In addition, when receiving a request for agenda content from the terminal, the communication conference node connects the playback processing unit and the terminal via the communication network, and sends the audio / video signal stored in the audio / video storage unit to the terminal. send.

  In addition, many proposals have been made on methods for efficiently managing and retrieving video information without imposing a burden on the user. For example, a video index creation apparatus that creates a video index by appropriately selecting a representative image captured when a scene change is detected has been proposed (see, for example, Patent Document 2). In this case, the input video is first stored in the video storage unit, and a scene change is detected by the scene change detection unit. At the time of detection, a representative image is captured from the input video and stored in the representative image storage unit. Next, the control unit instructs the video reproduction unit by operation to reproduce the video before and after the specified scene change from the video storage unit. Next, an image having the highest value is selected from the reproduced video by the operation, the still image is captured by the replacement image capturing unit, and replaced with the representative image in the representative image storage unit. A video index generation unit generates a video index from the group of representative images that have been replaced, and an index printing unit prints a video index such as a video list.

  In addition, an information processing apparatus has been proposed that can create an index for image information search with efficient processing and appropriately search for a target image without burdening the user (for example, , See Patent Document 3). When accessing a storage medium for recording image information, the position of the line of sight with respect to the user's image information is detected by a position detector and stored in association with the image information. At the time of analyzing the correspondence between the image and the line of sight, the point of interest determination unit obtains the coordinates of the point of interest based on the detected line of sight, and the edge recognition unit and the image partial extractor extract the image of the focused part. Based on this, an index for image search is generated. A list for selecting a target image is generated by presenting the index image at the time of image information reproduction.

  In order to look back on the conference efficiently, it is desirable to be able to reproduce the content of the conference as much as possible.

  A system for recording audio-video information as conference information is becoming common, but the recorded audio-video information is enormous. In an agenda content storage device (see Patent Document 1) that records video information such as facial expressions of conference participants and graphs used as well as recording audio information of conferences, the content of the conference is looked back from the audio video information. Since it is necessary to reproduce the recorded video, the same time as the time spent for the meeting is required, which is very inefficient. In addition, the information required for the conference participants is often only a part of the conference record, not the entire conference record.

  In addition, a video index creation device that generates a video index from video information and effectively detects a required scene from the video information (see Patent Document 2) detects a scene change in which the video changes rapidly. However, the scene detected when this scene change is detected depends on the image characteristics of the video, and it is not always possible to detect a high-value scene that the user needs. There is no problem.

  In addition, an information processing apparatus that creates an image search index using the user's line-of-sight information (see Patent Document 3) detects the line-of-sight position of an image presented on the screen and generates an index. It is supposed to be. However, since the sight line detection is performed on the image, there is a problem that the application range is small. In this case, it cannot be applied to gaze detection for an object not presented in the image, for example, gaze detection in a meeting that is in direct contact with a person.

JP 7-46571 A JP-A-9-93527 Japanese Patent Laid-Open No. 7-200362

  An object of the present invention is to provide an excellent information processing system capable of suitably supporting a conference information access operation for retrieving a desired scene from conference information in which video information, audio information, document information, etc. in a conference are recorded. It is to provide an information processing method and a computer program.

  A further object of the present invention is to provide an excellent information processing system and information processing method capable of recording audio / video information in a conference and creating an index for the recorded information to efficiently search for desired information. And providing a computer program.

  The present invention has been made in consideration of the above problems, and a first aspect of the present invention is an information processing system for managing conference information in which the content of a conference is recorded, and conference information input means for inputting conference information And conference participant information acquisition means for acquiring biometric information expressed from the conference participant during the conference as conference participant information, estimating the state of the conference participant based on the conference participant information, and based on the state A data processing means for processing the conference information. The state of the conference participant mentioned here is, for example, a psychological state.

  Here, the conference information includes at least one of conference audio, video, slides used in the conference, and other conference materials. The conference information includes at least one of materials used in the conference and conference participants as video information of the conference.

  Further, the conference participant information acquisition means acquires the conference participant's line-of-sight information, brain waves, pulse rate, or other biological information as conference participant information. In this case, for example, based on the line-of-sight information, it is possible to determine an object that the conference participant is interested in in the conference information.

  The data processing means can select a part of the audio / video information recorded as the conference information as the index information based on the psychological state information of the conference participant estimated from the conference participant information.

  The data processing means estimates the psychological state of the conference participant based on the conference participant information and processes the conference information based on the psychological state, but estimates the psychological state for a plurality of conference participants Can do. For example, the data processing means estimates a psychological state in which a meeting participant's interest or attention is not directed to a meeting point.

  And the said data processing means can calculate the target object in which the meeting participant is highly interested in meeting information, or the interested object based on the psychological state estimated from meeting participant information.

  Further, the data processing means selects a part of the audio / video information recorded by the meeting status recording means based on the psychological state information of the meeting participant estimated from the meeting participant information, and uses them as index information. Can be recorded as

  As described above, by generating the index information of the conference information according to the present invention and performing the recording / management processing, it is possible to look back on the state of the conference with the audio-video information. Further, since an index reflecting the psychological state of the person is added to the audio / video information, it is easy to search for desired audio / video information from the audio / video information of the conference, and the conference can be looked back on efficiently. In addition, since the user is freed from taking notes during the meeting, it is possible to look back on the state of the meeting after the meeting without being distracted from the agenda of the meeting.

  According to a second aspect of the present invention, there is provided a computer program written in a computer-readable format so as to execute processing for managing conference information in which the content of a conference is recorded on a computer system. A conference information input recording step for inputting information, a conference participant information acquisition step for acquiring biometric information expressed from the conference participant during the conference as conference participant information, and a conference participant based on the conference participant information A computer program comprising: a data processing step for estimating a state and processing conference information based on the state.

  The computer program according to the second aspect of the present invention defines a computer program described in a computer-readable format so as to realize predetermined processing on a computer system. In other words, by installing the computer program according to the second aspect of the present invention in the computer system, a cooperative action is exhibited on the computer system, and the information processing according to the first aspect of the present invention is performed. The same effect as the system can be obtained.

  According to the present invention, an excellent information processing system and information processing method capable of recording audio-video information in a conference, creating an index for the recorded information, and efficiently searching for desired information, In addition, a computer program can be provided.

  In addition, according to the present invention, an excellent information processing system and information processing method capable of creating a video index reflecting the psychological state of a person who participated in a conference and efficiently performing a review of the conference, and a computer A program can be provided.

  By performing the process of recording / managing conference information according to the present invention, it is possible for a person who has participated in the conference to look back on the state of the conference using audio-video information. Further, since an index reflecting the psychological state of the person is added to the audio / video information, it is easy to search for desired audio / video information from the audio / video information of the conference, and the conference can be looked back on efficiently. In addition, since the user is freed from taking notes during the meeting, it is possible to look back on the state of the meeting after the meeting without being distracted from the agenda of the meeting.

  Other objects, features, and advantages of the present invention will become apparent from more detailed description based on embodiments of the present invention described later and the accompanying drawings.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  The present invention relates to a system for recording / managing conference information. Specifically, the present invention records audio / video information in a conference and a video index reflecting the psychological state of a person who participated in the conference. It is a system that creates and efficiently looks back on meetings.

  FIG. 1 schematically shows a configuration of a conference recording apparatus according to an embodiment of the present invention. The illustrated conference recording apparatus includes a conference status recording unit 20, a participant information recording unit 30, and a data processing unit 40.

  The meeting status recording unit 20 records the state of the meeting 10 as audio video information. The participant information recording unit 30 executes processing and recording of information on conference participants. The data processing unit 40 executes processing for creating an index of the conference 10 based on the data recorded by the conference status recording unit 20 and the data recorded by the participant information recording unit 30.

  FIG. 2 shows in more detail the internal configuration of the participant information recording unit 30 and the data processing unit 40 in the conference recording apparatus.

  The participant information recording unit 30 acquires and records the conference participant's line-of-sight information, brain waves, pulse rate, or other biological information, which is used to determine the interest level of the conference participant, as conference participant information. In the illustrated embodiment, it is configured to handle line-of-sight information among biological information, and the participant information recording unit 30 includes a line-of-sight detection unit 31, a visual target determination unit 32, and a line-of-sight analysis unit 33. .

  The line-of-sight detection unit 31 acquires eye images for all or part of the participants in the conference 10. For example, a commercially available video camera can be used to capture an eye image.

  In addition, the line-of-sight analysis unit 33 extracts eye feature amounts (for example, the center of the pupil and the eyes) from the face images of the conference participants captured by the camera, and extracts feature amounts for eye-gaze direction estimation. The eye image analysis is described in, for example, Sakaguchi-gai, Shingaku Giho (PRU95-28, 1995).

  In addition, the visual target determination unit 32 attaches an LED marker that transmits an individual ID to a key point in a conference room or a conference participant, reads the LED marker from a camera image attached to the participant's head, Can be automatically determined. For the visual target determination, refer to, for example, Kakugai, Human Interface Symposium 2002 (general presentation, 2002).

  In addition, when an electroencephalogram is used instead of the line-of-sight information, the electroencephalogram can usually be known by measuring a minute potential change through an electrode attached on the scalp. As a simple electroencephalogram measurement apparatus, for example, there is a system (Human Sensing Co., Ltd., HSK central rhythm monitor system) that measures brain action potentials from two channels of the left and right frontal regions on the scalp.

  In the data processing unit 40, the psychological state of the conference participant is estimated based on the information of the conference participant recorded by the participant information recording unit 30, and the conference participation is included in the conference information based on the estimated psychological state. Indexes are generated using information on the corresponding places in the audio-video information recorded by the conference information recording unit 20 by determining the time point at which the person is highly interested or the object of interest. For example, the psychological state is estimated for a plurality of conference participants. In addition, the psychological state in which the attention or attention of the conference participant is not directed to the issue of the conference is estimated.

  In the embodiment shown in FIG. 2, the data processing unit 40 includes an interest level determination unit 41, an index information generation unit 42, and an index information recording unit 43, and determines the psychological state of the conference participants in the form of the interest level. To do. Based on the determination of the degree of interest, a part of the audio / video information recorded by the conference status recording unit 20 is selected as index information.

  FIG. 3 illustrates an embodiment that is to be recorded by the meeting status recording unit 20 in the meeting 10.

  In the example shown in the figure, the conference 10 is operated by four conference participants A11, B12, C13, and D14, and a document 16 is presented using a PC (Personal Computer) 15. The conference status recording unit 20 records video information including audio information of the conference, and the line-of-sight information recording unit 30 records the line-of-sight information of the conference participants.

  First, an outline of processing executed by each processing unit will be described with reference to FIGS. 2 and 3, and then specific processing operations of the system will be described in detail.

  For example, a video camera can be used as a specific device of the conference status recording unit 20 in FIG. The conference status recording unit 20 records video including at least one of the conference participants 11 to 14 or the material 16 presented at the conference, and the material presented at the utterance or conference of the conference participants 11 to 14 16 and recorded as video information to which time information is added.

  The line-of-sight detection unit 31 in the line-of-sight information recording unit 30 includes, for example, an eye camera and detects the direction of the line of sight of the conference participants 11 to 14 in the conference 10 and the time information for detecting the direction of the line of sight. The line-of-sight determination unit 32 determines an object that is directed to the line of sight within the conference 10 from the line-of-sight direction information of the conference participants 11 to 14 detected by the line-of-sight detection unit 31, and at the time when the line of sight is directed. The visual target information is acquired in association with each other.

  The line-of-sight analysis unit 33 analyzes the movement of the line of sight from the line-of-sight information of the conference participants 11 to 14 detected by the line-of-sight detection unit 31. Normally, one eye camera detects the line of sight of one person, so when detecting the line of sight of all the participants in the conference, eye cameras for the number of conference participants are used.

  The degree-of-interest determination unit 41 in the data processing unit 40 estimates and acquires the psychological state of the conference participant from the line-of-sight information that is the output of the line-of-sight information recording unit 30. The interest level determination unit 41 can acquire the psychological state of the conference participant in the conference together with the time information.

  The index information generation unit 42 in the data processing unit 40 associates the psychological state data that is the output of the interest level determination unit 41 with the audio / video information recorded by the conference status recording unit 20, and the audio / video of the conference status recording unit 20. A video index is generated from the information.

  The index information recording unit 43 records the video index created by the index information generating unit 42.

  The minutes presentation unit 43 can display the video index recorded by the index information recording unit 43. By arbitrarily selecting the video index, the conference participant can efficiently look back on the conference 10 from the video information in the conference status recording unit 20.

  Next, a specific processing example will be described. An example is a conference in which a plurality of conference participants proceed while viewing a document presented using a computer.

  For example, a video camera is available as an apparatus for recording the status during the conference by the conference status recording unit 20. It is assumed that the target of the video recorded by the video camera includes at least one of conference participants or presented materials. The materials presented at the conference 10 include, for example, electronic materials projected on a wall using a computer, electronic materials projected on a computer display, and printed materials distributed to conference participants. It is done. At this time, not only the video but also the audio is recorded simultaneously with the video. Examples of the voice to be recorded include the speech of a conference participant, the voice uttered from a material presented using a computer, and the like. Data recorded by the video camera is converted into data that can be digitally processed and transmitted to the data processing unit.

  The line-of-sight information acquisition unit 30 acquires line-of-sight information of conference participants. As a method for acquiring line-of-sight information, for example, there is a line-of-sight detection method based on image analysis of an eye camera. FIG. 4 shows a processing procedure for obtaining line-of-sight information, and performing eye-gaze determination and line-of-sight analysis.

  First, using an eye camera, acquisition of the eye image of a conference participant and the time when the eye image was acquired are acquired (step S301).

  Next, software processing is performed on the eye image acquired in step S301, and the eye image is analyzed (step S302). The line-of-sight analysis unit 33 extracts eye feature amounts (for example, the center of the pupil and the eyes) from the face images of the conference participants imaged by the camera, and extracts feature amounts for eye-gaze direction estimation. The eye image analysis is described in, for example, Sakaguchi-gai, Shingaku Giho (PRU95-28, 1995).

  Next, from the result of the eye image analysis analyzed in step S302, the direction of the line of sight of the conference participant is calculated by software processing, and the direction of the line of sight is recorded together with time information for detecting the direction of the line of sight (step S303).

  The line-of-sight detection unit 31 executes processing corresponding to steps S301 to S303.

  Next, the target to which the line of sight is directed is determined from the direction of the line of sight of the conference participant calculated in step S303 (step S304). The visual target determination unit 32, for example, attaches an LED marker that transmits an individual ID to a key point in a conference room or a conference participant, reads the LED marker from a video image of a camera attached to the participant's head, and automatically sets it as a visual target. Can be determined automatically. For the visual target determination, refer to, for example, Kakugai, Human Interface Symposium 2002 (general presentation, 2002).

  Next, the visual target information determined in step S304 is acquired (step S305). Since the visual target is acquired, for example, the presence / absence information of the switching of the target to which the line of sight is directed can also be recorded.

  The visual target acquisition unit 32 executes processing corresponding to steps S304 to S305.

  Next, the movement of the line of sight is analyzed from the line-of-sight direction calculated in step S303 (step S306). The movement of the line of sight mentioned here is, for example, a stopping time, a moving distance, and the like. The line-of-sight analysis unit 33 executes a process corresponding to step S306.

  Then, the visual target information acquired in step S305 and the visual line movement information acquired in step S306 are stored (step S307).

  FIG. 5 shows an example of data obtained by the line-of-sight information acquisition unit. The time when the line of sight is detected is recorded at intervals of 1 second, and the visual target acquired by the visual target determination unit 32, the presence / absence of switching of the visual target, the gaze stop time and the movement distance acquired by the visual line analysis unit 31, and the time Correspondingly recorded. In addition, although the time interval of the acquisition data in this embodiment is 1 second, the summary of this invention is not limited to this.

  The line-of-sight information obtained through steps S301 to S307 illustrated in FIG. 4 is transmitted to the data processing unit 40.

  The interest level determination unit 41 of the data processing unit 40 estimates the psychological state of the conference participant from the transmitted line-of-sight information, and records the degree of interest associated with the time information.

  The degree of interest represents the psychological state held by the conference participants. FIG. 6 shows a processing procedure for estimating a person's psychological state in the interest level determination unit 41 in the form of a flowchart.

  When the degree-of-interest determination unit 41 acquires the line-of-sight information obtained by the line-of-sight information detection unit 31 (step S401), the interest level determination unit 41 collates this with the line-of-sight database and estimates the human psychological state (step S402).

  FIG. 7 shows an example of data stored in the line-of-sight database. Here, the line-of-sight database is a database prepared in advance, and stores data in which the movement of the line of sight is associated with the visual target and the psychological state of the person. The line-of-sight movement is composed of, for example, a stop time, a movement distance, etc., and the line-of-sight database includes a line-of-sight record, a line-of-sight movement (line-of-sight direction and stop time, movement distance, presence / absence of switching of a visual target), The user's state (psychological state) is stored. A person's psychological state is, for example, whether they are interested, have doubts, or are bored.

  In addition, it is not necessary to estimate a person's psychological state at all times recorded by the line-of-sight information acquisition unit.

  The index information generation unit 42 selects a part of the audio / video information recorded by the conference status recording unit 20 as index information from the degree of interest corresponding to the time information, which is the output of the interest level determination unit 41.

  In FIG. 8, from the time t1 (701) to t5 (705) when the interest level determination unit 41 determines that the person's psychological state is “interested” and the audio video information corresponding to the time, the audio video information is obtained. Is selected (r1 to r5), and an image index (u1 to u5) is generated.

  As described above, the degree of interest can be determined by line-of-sight analysis or visual target determination. Of course, as shown in the figure, the brain waves and pulses of the conference participants recorded during the meeting, and other biological information (not shown) The degree of interest can also be determined based on.

  The selected audio / video information is audio / video information corresponding to the start time of the time determined to be “interesting”, and the video index is generated from the selected audio / video information. Still image.

  The psychological state determined by the interest level determination unit 41 is not limited to “interested”. For example, as another psychological state, the start time of the time determined as “feeling doubt” by the interest level determination unit 41 so that it is possible to generate a video index when “feeling doubt” Can be used to generate a still image as an index from audio-video information. In addition, the psychological state determined by the interest level determination unit 41 is combined with a plurality of psychological states such as “I am interested”, “I feel doubt”, “I am bored”, etc., and a still image is indexed from the audio-video information. Can also be generated.

  A part of the audio-video information selected by the index information generation unit 42 is not limited to the start time of the time when the specific psychological state is determined. It is also possible to substitute other time information such as the center value of the time at which a specific psychological state is determined.

  Further, when the index information generating unit 42 selects a part of the audio / video information recorded by the meeting status recording unit 20 and generates a video index, the index information generating unit 42 takes the output of the line-of-sight information into the output of the interest level determination unit 41. It is also possible to select a part of the audio / video information recorded by the conference status recording unit 20. For example, since the visual line information output includes visual target information associated with time, it is possible to generate a video index on the visual target when selecting a part of the audio-video information recorded by the meeting status recording unit 20 And

  The index information recorded in the index information recording unit 43 is the still image generated by the index information generating unit 42 and the time information when the still image is acquired.

  The proceeding presentation unit 50 presents to the user the proceedings made up of time information and still images obtained from the still images recorded in the index information recording unit 43. As shown in FIG. 9, time information 801 and a still image 802 obtained by acquiring a still image recorded in the index information recording unit 43 are presented. For example, audio / video information from the time corresponding to the still image can be reproduced on the display 803 by selecting and executing the still image displayed on the display with a mouse or the like.

  The above is one embodiment of the present invention. Note that the number of video cameras installed in the conference 10 shown in FIG. 3 is not limited to one, and a plurality of video cameras may be installed, and the target to be recorded as video by each video camera may be limited. For example, it is possible to install three video cameras, one for conference participant A, one for conference participant B, and the other for video recording of the presented electronic materials. In this case, the audio / video information selected from the output of the interest level determination unit 41 by the index information generation unit 42 can be arbitrarily selected from the conference participant A, the conference participant B, or electronic information. And

  Since the line-of-sight information recording unit 30 can provide the number of conference participants, the direction of the line-of-sight is obtained for each of the conference participants 11 to 14 shown in FIG. It is also possible to do.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiment without departing from the gist of the present invention.

  In the present specification, the present invention has been described in detail by taking an embodiment in which conference information is indexed as an example, but the gist of the present invention is not limited thereto. In the system so as to handle audio-video information other than conference recording, or recorded content other than audio-video, by applying the present invention, the psychological state is estimated based on the user's line-of-sight information and other biological information, Based on the estimation result, it is possible to suitably perform indexing of the recorded content and other processing for classifying and organizing the content.

  In short, the present invention has been disclosed in the form of exemplification, and the description of the present specification should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims should be taken into consideration.

FIG. 1 is a diagram schematically showing a configuration of a conference recording apparatus according to an embodiment of the present invention. FIG. 2 is a diagram showing in more detail the internal configuration of the participant information recording unit 30 and the data processing unit 40 in the conference recording apparatus. FIG. 3 is a diagram for explaining an embodiment to be recorded by the conference status recording unit 20 in the conference 10. FIG. 4 is a diagram illustrating a processing procedure for acquiring line-of-sight information, performing line-of-sight determination and line-of-sight analysis. FIG. 5 is a diagram illustrating an example of data obtained by the line-of-sight information acquisition unit. FIG. 6 is a flowchart showing a processing procedure for estimating a person's psychological state in the interest level determination unit 41. FIG. 7 is a diagram illustrating an example of data stored in the line-of-sight database. FIG. 8 shows the audio video information from the time t1 (701) to t5 (705) when the interest level determination unit 41 determines that the person's psychological state is “interested” and the audio video information corresponding to the time. It is the figure which showed the example which selected a part (r1-r5) and produced | generated the video index (u1-u5). FIG. 9 is a diagram showing a screen configuration example in which the user is presented with the time information obtained from the still image recorded in the index information recording unit 43 and the proceedings made up of the still image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Meeting 20 ... Meeting status recording part 30 ... Participating information recording part 31 ... Gaze detection part 32 ... Gaze target determination part 33 ... Gaze analysis part 40 ... Data processing part 41 ... Interest level determination part 42 ... Index information generation part 43 ... Index information recording part 50 ... Agenda presentation part

Claims (21)

An information processing system for managing conference information recording the content of a conference,
Meeting information input means for inputting meeting information;
Conference participant information acquisition means for acquiring biometric information expressed from the conference participants during the conference as conference participant information;
Data processing means for estimating the state of the conference participant based on the conference participant information and processing the conference information based on the state;
An information processing system comprising: The state of the conference participants is a psychological state,
The information processing system according to claim 1. The meeting information consists of at least one of meeting audio, video, slides used in the meeting, and other meeting materials.
The information processing system according to claim 1. The conference information includes at least one of materials used in the conference and conference participants as video information of the conference.
The information processing system according to claim 2. The conference participant information acquisition means acquires at least one of gaze information, brain waves, pulse rate, or other biological information of the conference participant as conference participant information.
The information processing system according to claim 1. The data processing means selects a part of the audio-video information recorded as the conference information as index information based on the status information of the conference participant estimated from the conference participant information.
The information processing system according to claim 1. The data processing means estimates a psychological state for a plurality of conference participants.
The information processing system according to claim 1. The data processing means estimates a psychological state in which a meeting participant's interest or attention is not directed to a meeting point;
The information processing system according to claim 1. Based on the psychological state estimated from the conference participant information, the data processing means was made to correspond to the conference participant's psychological state in the conference information (not only high interest, but also felt doubtful, bored, To identify objects that correspond to the point in time or psychological state (including not only high interest but also the state of feeling doubtful, tired, etc.)
The information processing system according to claim 1. Based on the line-of-sight information acquired as the conference participant information, the data processing means is based on the psychological state of the conference participant during the conference information (not only high interest, but also felt doubtful, bored, etc. The target)
The information processing system according to claim 5. An information processing method for managing conference information in which the content of a conference is recorded,
Meeting information input recording step for inputting meeting information;
Meeting participant information acquisition step for acquiring biological information expressed from the meeting participant during the meeting as meeting participant information;
A data processing step for estimating a state of the conference participant based on the conference participant information and processing the conference information based on the state;
An information processing method comprising: The state of the conference participants is a psychological state,
The information processing method according to claim 11. The meeting information consists of at least one of meeting audio, video, slides used in the meeting, and other meeting materials.
The information processing method according to claim 11. The conference information includes at least one of materials used in the conference and conference participants as video information of the conference.
The information processing method according to claim 11. In the conference participant information acquisition step, at least one of the conference participant's line-of-sight information, brain wave, pulse rate, or other biological information is acquired as conference participant information.
The information processing method according to claim 11. In the data processing step, based on the state information of the conference participant estimated from the conference participant information, a part of the audio / video information recorded as the conference information is selected as index information.
The information processing method according to claim 11. In the data processing step, a psychological state is estimated for a plurality of conference participants.
The information processing method according to claim 11. In the data processing step, based on the psychological state estimated from the conference participant information, it was made to correspond to the psychological state of the conference participant in the conference information (not only high interest, but also felt doubtful, bored, To identify objects that correspond to the point in time or psychological state (including not only high interest but also the state of feeling doubtful, tired, etc.)
The information processing method according to claim 11. In the data processing step, based on the psychological state estimated from the conference participant information, it was made to correspond to the psychological state of the conference participant in the conference information (not only high interest, but also felt doubtful, bored, To identify objects that correspond to the point in time or psychological state (including not only high interest but also the state of feeling doubtful, tired, etc.)
The information processing method according to claim 11. In the data processing step, based on the line-of-sight information acquired as the conference participant information, an object in which the conference participant is interested in the conference information is determined.
The information processing method according to claim 13. A computer program written in a computer-readable format so as to execute processing for managing conference information in which the content of a conference is recorded on a computer system,
Meeting information input recording step for inputting meeting information;
Meeting participant information acquisition step for acquiring biological information expressed from the meeting participant during the meeting as meeting participant information;
A data processing step for estimating a state of the conference participant based on the conference participant information and processing the conference information based on the state;
A computer program comprising:

Images