JP2008124702A - Electronic conference system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve an electronic conference system facilitating retrieval work when reproducing electronic minutes with audio/video information of a conference. <P>SOLUTION: The electronic conference system for generating the electronic minutes with the audio/video information in a conference comprises: a motion detection section 120 for detecting the motion of a conference participant included in video information photographed in the conference; an index information generation section 140 for generating index information corresponding to the video information based on the motion of the conference participant detected by a motion detection means; and a storage section 160 for storing the audio/video information and the index information. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an electronic conference system capable of creating minutes having an index from video during a conference.

  It is possible to save audio / video information in a meeting as electronic minutes by electronic data. In this case, the electronic data that can be used on the computer is saved on a server or the like, so that participants and non-participants can check the contents by playing the electronic minutes on their computers. .

In the electronic minutes in such an electronic conference system, an index for cueing is required. And as a technique for giving this index, proposals have been made in the following patent documents.
JP-A-9-93527 (first page, FIG. 2) Japanese Patent Laying-Open No. 2005-341005 (first page, FIG. 2) Japanese Patent Laid-Open No. 2002-109099 (first page, FIG. 2)

Patent Document 1 describes a video index creation device that creates a video index by reselecting a representative image captured when a scene change is detected.
In this case, the scene detected when the scene change is detected is based on the image characteristics of the video. For this reason, there is a problem that the scene that the user wants to cue cannot be detected.

In the above-mentioned Patent Document 2, a system is proposed in which an important statement can be cued as a keyword in the same manner as in the character-based minutes.
In this case, the moment when an important keyword is spoken is an index. For this reason, when the index is used later as a cue, there is a problem in that it is not possible to know the situation immediately before the important keyword is spoken, and it is necessary to play back retroactively. Even in that case, there is a problem that it is difficult to know where to go back.

  In the above-mentioned Patent Document 3, a method for synchronizing and reproducing a material and video / audio by associating the material used in the presentation with the video / audio explaining the material is proposed. In this case, there is a problem that it can be used only as an explanation of presentation materials.

  The present invention has been made to solve the above-described problems, and it is an object of the present invention to realize an electronic conference system that can easily perform a search operation when reproducing electronic minutes based on audio and video information of a conference. Objective.

The present invention for solving the above problems is as described below.
(1) The invention according to claim 1 is an electronic conference system for generating electronic minutes based on audio-video information in a conference, and a motion for detecting a motion of a conference participant included in video information photographed in the conference An external unit for storing the audio / video information, comprising: detecting means; and index information generating means for generating index information corresponding to the video information based on the motion of the conference participant detected by the motion detecting means. The electronic conference system is characterized in that the index information is supplied to the storage means.

  (2) The invention according to claim 2 is an electronic conference system for generating electronic minutes based on audio-video information in a conference, and a motion for detecting a motion of a conference participant included in video information photographed in the conference A detecting means, an index information generating means for generating index information corresponding to the video information based on the movement of the conference participant detected by the motion detecting means, the audio / video information and the index information are stored. And an electronic conferencing system.

  (3) The invention according to claim 3 is an electronic conference system for generating electronic minutes based on audio-video information in a conference, and a motion for detecting a motion of a conference participant included in video information photographed in the conference Detection means, determination means for determining presence / absence of movement with reference to a predetermined threshold for the movement of the conference participant detected by the movement detection means, and the determination means determining that there is movement Based on the movement of the conference participant, the index information generating means for generating index information corresponding to the video information, the storage means for storing the audio / video information and the index information, and the determination means are used as the determination criteria. And the audio / video information reproduced from the storage means is input again to the motion detection means, and the index information And feedback means for re-generate the index information by forming means is an electronic conference system, comprising a.

  (4) The invention described in claim 4 is provided with a target specifying means for specifying a conference participant who is a target of motion detection by the motion detection means, and the target information is included in the index information. It is an electronic conference system as described in any one of Claims 1 thru | or 3.

  (5) In each of the above claims, as a motion detection target, in addition to a conference participant, a target such as a whiteboard, a screen, a podium, or a spotlight can be selected, or only the target or the target can be moved. Instead, the index information may be generated by a change in brightness.

According to the electronic conference system of the present invention, the following effects can be obtained.
(1) In the first aspect of the invention, when the electronic minutes are generated by the audio / video information in the conference, the movement of the conference participant included in the video information photographed in the conference is detected, and the detected conference participation is detected. Index information corresponding to the video information is generated based on the movement of the person, and the index information is supplied to an external storage means for storing the audio / video information.

  In this case, since the index information is generated by detecting the movement of the conference participant, for example, the timing when the presenter moves to the explanation location, the timing when the explanation location returns to his / her seat, and a large number of participants leave. Index information can be generated at an appropriate timing such as a break timing. As a result, the electronic minutes based on the audio / video information is stored in the external storage means and can be easily retrieved when being reproduced.

  (2) In the invention according to claim 2, when generating the electronic minutes by the audio / video information in the meeting, the movement of the meeting participant included in the video information photographed in the meeting is detected, and the detected meeting participation is detected. Based on the movement of the person, index information corresponding to the video information is generated, and the audio / video information and the index information are stored.

  In this case, since the index information is generated by detecting the movement of the conference participant, for example, the timing when the presenter moves to the explanation location, the timing when the explanation location returns to his / her seat, and a large number of participants leave. Index information can be generated at an appropriate timing such as a break timing. As a result, the electronic minutes based on the audio / video information is stored in the storage means and can be easily retrieved when it is played back.

  (3) In the invention according to claim 3, when the electronic minutes are generated by the audio / video information in the conference, the movement of the conference participant included in the video information photographed in the conference is detected, and the detected conference participation is detected. About a person's movement, the presence or absence of a movement is determined on the basis of a predetermined threshold, and index information corresponding to the video information is generated based on the movement of the conference participant determined to have movement, and the audio / video information And index information.

Then, the motion threshold value used as a criterion is changed, and audio / video information stored and reproduced is input again, and index information is generated again.
In this case, since the index information is generated by detecting the movement of the conference participant, for example, the timing when the presenter moves to the explanation location, the timing when the explanation location returns to his / her seat, and a large number of participants leave. Index information can be generated at an appropriate timing such as a break timing. Further, it is possible to regenerate the index information by changing the threshold value for motion determination.

As a result, when the electronic minutes based on the audio / video information are stored in the storage means and reproduced, the search operation can be easily performed with appropriate index information.
(4) The invention according to claim 4, further comprising target specifying means for specifying a conference participant as a target of motion detection by the motion detection means, wherein the index information includes target specifying information. It is an electronic conference system as described in any one of Claims 1 thru | or 3.

  In this case, since index information is generated by detecting the movement while identifying the conference participants, for example, the timing when the presenter moves to the explanation location, the timing when the explanation location returns to his / her seat, a large number of participants Index information can be generated at an appropriate timing, such as the timing of a break when the person leaves the room.

  As a result, electronic minutes based on audio-visual information are stored in an external storage means, and a desired index can be found based on the movements of the conference participants specified at the time of playback. Can be easily performed.

  (5) In each of the above claims, as a motion detection target, in addition to a conference participant, a target such as a whiteboard, a screen, a podium, or a spotlight can be selected, or only the target or the target can be moved. Instead, the index information can be generated at a more appropriate timing by generating the index information based on changes in brightness.

The best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described below in detail with reference to the drawings.
(1) Configuration of electronic conference system:
FIG. 2 is a block diagram illustrating a detailed configuration in the electronic conference system 100 according to the first embodiment of this invention. In FIG. 2, the periphery of the portion necessary for the description of the operation of the present embodiment is mainly described, and various other portions such as a power switch and a power circuit known as the other electronic conference system 100 are omitted. It is.

  The electronic conference system 100 according to the present embodiment includes a control unit 101 that controls each unit, a time code generation unit 102 that generates time information, an operation unit 104 that serves as an operation unit for inputting various operations by an operator, A signal processing unit 110 as a signal processing unit that converts an input video signal and audio signal (audio / video information) into recording format data, and a motion detection unit as a motion detection unit that detects a predetermined motion in a conference video 120, a determination unit 130 serving as a determination unit that determines a detected motion based on a certain threshold, and an index information generation unit configured to generate index information based on the detected / determined motion in the conference video. Index generating unit 140 and target specifying unit 1 as a target specifying unit for specifying a conference participant or the like that is a target of motion detection / determination in a conference video 0, a storage unit 160 as a storage unit for storing audio / video information and index information, and a playback instruction unit for generating an instruction to play back the audio / video information stored in the storage unit 160 based on the index information A reproduction instruction unit 170, a signal processing unit 180 serving as a signal processing unit that converts audio / video information reproduced from the storage unit 160 into a signal format that can be reproduced and displayed, and audio / video information from the signal processing unit 180 are displayed as images. A display unit 190 as a display unit for outputting sound and an I / F 195 as a connection unit for transmitting audio / video information from the signal processing unit 180 to an external device are configured.

  The storage unit 160 includes an audio / video information storage unit 161 that stores audio / video information and an index information storage unit 162 that stores index information. Note that both the video information and the index information can be synchronized by holding the time code generated by the time code generation unit 162.

  Although not shown, audio / video information is input from a video camera and a microphone installed in the conference hall. In this case, the audio / video information may be not only a general video signal and audio signal but also digital data handled by a computer. The video camera may be a stereo camera described in Japanese Patent Application Laid-Open No. 2004-279111. In addition, it is possible to detect the presence of a conference participant using a known face extraction unit and detect the conference participants who have moved by the two-dimensional motion detection unit.

  Further, the motion detection unit 120 not only detects the motion included in the video signal, but also attaches an ID such as an IC tag to the participant in advance, and the SAW described in Japanese Patent Laid-Open No. 2003-263612 and the like. -ID technology can be used. The motion detection unit 120 can also use indoor GPS technology using a GPS mobile phone device that is being studied in the Advanced Positioning Social Infrastructure Research Forum. Further, as described in Japanese Patent Application Laid-Open No. 8-205114, the motion detection unit 120 is provided with a seating sensor on the chair to detect the separation / seating, and Japanese Patent Application Laid-Open No. 10-74175 or Japanese Patent Application Laid-Open No. 10-74175. A technique for associating a seat and a seated person as described in Japanese Patent Laid-Open No. 2005-267335 can also be applied. FIG. 1 shows an example in which motion detection is performed from a video signal.

(2) Operation of electronic conference system (1):
Hereinafter, with reference to the flowchart of FIG. 2 and the explanatory diagram of FIG.

  First, when a conference is started, audio video information including an audio signal from a microphone installed in a conference hall and a video signal from a camera is input to an input terminal of the electronic conference system 100 according to the present embodiment ( Step S201 in FIG. In this case, it may be input in a general audio signal or video signal format, or may be input in a data format that can be used by a computer via a network. Moreover, the conference hall does not need to be a single one, and may be a format in which audio and video information from a plurality of conference halls is mixed.

  Then, in response to the input of the audio / video information, the signal processing unit 110 converts the audio / video information into a recordable signal format. The audio / video information converted into a recordable signal format is stored in the audio / video information storage unit 161 in the storage unit 160 (step S202 in FIG. 2). In this case, it is desirable to store the audio / video information from the start to the end of the conference as one file. Note that the time code generated by the time code generation unit 102 is stored in a state of being added to the audio / video information for correspondence with index information described later.

Note that the audio / video information and the index information may be copied or moved to a large-capacity storage device (not shown) until they are reproduced after being stored.
Here, with respect to the input audio-video information, the motion detection unit 120 monitors whether or not a motion occurs in the video (step S203 in FIG. 2). Then, the determination unit 130 determines whether the motion detected by the motion detection unit 120 corresponds to a predetermined motion.

Here, the predetermined movement that should be determined as having movement is:
・ A movement where the presenter moves away from his / her seat to the explanation location in the center of the conference hall,
・ Return to the seat from the explanation center in the center of the conference hall,
・ The explainer gets up at his or her desk and starts explaining,
・ A large number of participants move away (start break),
・ A lot of participants are seated (restarting the meeting)
Etc. In other words, it is desirable that a certain amount of movement worth using as index information can be detected as “movement”.

In other words, it is desirable to be able to exclude the operation of a conference participant who has not announced, the operation of drinking water, the operation of taking notes, and the operation of facing sideways and releasing to the side.
For this purpose, the determination unit 130 sets a weight in advance for the movement direction and the movement amount of the movement in the video, and compares the value obtained by multiplying the movement in the video by the weight with a threshold value. It is desirable to make a determination.

  When the presence of the predetermined motion is confirmed by the above detection and determination (YES in step S203 in FIG. 2), the index generation unit 140 sets an index number for newly generating index information to the index information storage unit 162. (Step S204 in FIG. 2).

  When the presence of the predetermined movement is confirmed by the above detection and determination (YES in step S203 in FIG. 2), the target specifying unit 150 specifies the target having moved (step in FIG. 2). S205). Then, information (target specifying information) of the specified target (such as a conference participant) is transmitted to the index information storage unit 162.

  The target specifying unit 150 may generate target specifying information by specifying a moving target by associating a participant's seating position with a participant's name in the video at the start of the conference in advance. It becomes possible. In this case, the video at the start of the conference is displayed on the display unit 190, the seating position of the participant is designated from the operation unit 104, and the participant's name is input. And the object specific | specification part 150 hold | maintains the relationship between a participant's seating position and a participant's name as a table.

  Here, the index information storage unit 162 that has received the index number and the target identification information acquires the time code from the time code generation unit 102 (step S206 in FIG. 2). Here, the time code is one of or both of time (date and time) information, elapsed time information from the start of the conference.

  Then, the index information storage unit 162 associates the index number from the index generation unit 140, the target identification information from the target identification unit 150, and the time code from the time code generation unit 102 as shown in FIG. Index information is generated (step S207 in FIG. 2), and this index information is stored (step S208 in FIG. 2).

In FIG. 4, the index information includes the index number, time code (date / time information / elapse information), and data of the subject / object.
While the input of audio / video information continues (YES in step S209 in FIG. 2), the above audio / video information storage (step S202 in FIG. 2) to index information storage (step in FIG. 2). S208) is repeated. If the input of the audio / video information is stopped (NO in step S209 in FIG. 2), the audio / video information and index information files are closed and stored in association with each other (step S210 in FIG. 2). The process is terminated.

  In the above description, the object of motion detection is a conference participant, but it may be a motion of a specific object such as raising or lowering a projector screen or moving a whiteboard. Further, it may be a change in brightness or movement, such as illumination of a spotlight to a specific place or subject.

  In the above case, since the index information is generated by detecting the movement of the conference participant, for example, the timing when the explainer of the conference participants moves to the explanation location, the timing when returning from the explanation location to his / her seat, many Index information can be generated at an appropriate timing such as a break timing at which the participants of the party leave.

  In the above description, the movement of the target person when the index information is generated may be added to the index information. For example, all members / seated, Tanaka / seating, Tanaka / seating, all / seating can be performed.

  In the above description, the movement of the object when the index information is generated may be added to the index information. For example, whiteboard / light, screen / descent, screen / light, podium, light, conference room / dark, spotlight / moving, etc. are possible.

(3) Operation of electronic conference system (2):
Hereinafter, with reference to the flowchart of FIG. 3 and the explanatory diagram of FIG. Although reproduction (browsing of electronic minutes) can be performed from an external computer, a specific example is a case where a reproduction operation is performed via the operation unit 104.

  First, a user who wants to view the electronic minutes of a meeting selects a file to be played from the operation unit 104 (step S301 in FIG. 3). When a file to be reproduced is selected, the reproduction instruction unit 170 reads the index information (see FIG. 5) of the corresponding file from the index information storage unit 162 and displays it on the display unit 190 (step S302 in FIG. 3).

  In FIG. 5, an index list display unit 1910 displaying a predetermined number of index information in a state in which the index number, time code (date / time information / elapsed information), and data of the target person / target object are included, and playback / stop thereof. A state is shown in which an operation screen 1920 for performing operations such as feeding and returning is shown.

  In this operation screen, when the user inputs an index number to be played and gives a playback instruction (YES in step S303 in FIG. 3), the playback instruction unit 170 displays the corresponding index number in the index information. The time code corresponding to is extracted. Then, the playback instruction unit 170 instructs playback of the audio / video information from the timing corresponding to the time code (step S304 in FIG. 3), and starts playback of the audio / video information (step S305 in FIG. 3).

During the reproduction of the audio / video information, it is desirable that the operation screen 1920 is continuously displayed at the corner of the screen without the index list display unit 1910 being displayed.
If an instruction to reproduce the next index is given from the operation screen 1920 during browsing of the electronic minutes, that is, reproduction of the audio / video information (YES in step S306 in FIG. 3), the reproduction instruction unit 170 displays the index number. 1 (step S307 in FIG. 3), the time code corresponding to the corresponding index number in the index information is extracted, and the reproduction of the audio / video information is instructed from the timing corresponding to the time code (in FIG. 3). Step S304), the reproduction of the audio / video information is started (Step S305 in FIG. 3).

  Also, when browsing the electronic minutes, that is, during the reproduction of the audio / video information, if there is an instruction to reproduce the previous index from the operation screen 1920 (YES in step S308 in FIG. 3), the reproduction instruction unit 170 displays the index number. -1 (step S309 in FIG. 3), the time code corresponding to the corresponding index number in the index information is extracted, and the reproduction of the audio / video information is instructed from the timing corresponding to the time code (FIG. 3). Middle step S304), the reproduction of the audio-video information is started (step S305 in FIG. 3).

  Further, during browsing of electronic minutes, that is, during reproduction of audio-video information, if there is a display request (operation 1920a in FIG. 5) from the operation screen 1920 (YES in step S310 in FIG. 3). The reproduction instruction unit 170 displays the index list display unit 1910 on the display screen 1900 (step S302 in FIG. 3).

  As described above, when the index number of the index list display unit 1910 is selected, the audio / video information at the timing of the time code corresponding to the index number is reproduced. If there is an instruction to stop reproduction (YES in step S311 in FIG. 3), the processing is terminated, and browsing of the electronic minutes, that is, reproduction of audio-video information is stopped.

  As a result of the above, for electronic minutes based on audio-visual information, it is possible to find the desired index based on the movements of the specified conference participants, and to facilitate the search operation during playback. Become.

  In addition, in the above description, when index information is generated, the movement of the target person / object (all / seated, Tanaka / seated, Tanaka / seated, all / seated, etc.) is added to the index information In this case, the search operation at the time of reproduction is further facilitated.

(4) Operation of electronic conference system (3):
It should be noted that the index information can be regenerated when there is too much generated index information and the search is complicated, or when there is too little generated index information to perform a sufficient search.

  That is, when an instruction to regenerate index information is input from the operation unit 104, the audio / video information is reproduced from the audio / video information storage unit 161, and this audio / video information is supplied again to the input terminal via the feedback unit 106. The

  At this time, the audio / video information storage unit 161 is not stored, but the motion detection unit 120, the determination unit 130, the index generation unit 140, and the target identification unit 150 operate, and the determination criterion of the determination unit 130 The index generation unit 140 generates new index information by changing the threshold value. The new index information is stored in the index information storage unit 162. In this case, the time code generation unit 102 refers to the time code included in the reproduced audio / video information and generates a necessary time code.

  In this case, less index information is generated by tightening (increasing) a threshold value that is a determination criterion in the determination unit 130. On the other hand, a lot of index information is generated by loosening (lowering) the threshold value that is the determination criterion in the determination unit 130.

  Also, by specifying the direction of movement or changing the threshold of movement in a specific direction, index information can be added only at the start of presentation or index information can be added only at the end of presentation. Is possible.

Further, by assigning a large weight to the movement of the specific person, it is possible to give index information linked to the movement of the specific attention person of the conference participant.
(5) Other:
In the above description, an instruction at the time of reproduction is input from the operation unit 104, but the same process can be executed when browsing the electronic minutes on an external computer connected to the network. Is possible. In that case, it is desirable to set that the authority of the person in charge is necessary for changing the index information.

  In the above description, the storage unit 160 is built in the electronic conference system 100. However, an external storage device, a dedicated storage server, or the like can be used.

It is a block diagram which shows the structure of embodiment of this invention. It is a flowchart which shows operation | movement of embodiment of this invention. It is a flowchart which shows operation | movement of embodiment of this invention. It is explanatory drawing which shows the index information in embodiment of this invention. It is explanatory drawing which shows the index information in embodiment of this invention.

Explanation of symbols

100 Electronic Conferencing System 101 Control Unit 110 Signal Processing Unit 120 Motion Detection Unit 130 Determination Unit 140 Index Generation Unit 150 Target Identification Unit 160 Storage Unit 170 Playback Instruction Unit 180 Signal Processing Unit 190 Display Unit

Claims (4)

An electronic conference system for generating electronic minutes by audio-video information in a conference,
A motion detection means for detecting a motion of a conference participant included in the video information captured in the conference;
Index information generating means for generating index information corresponding to the video information based on the movement of the conference participant detected by the movement detecting means;
And providing the index information to an external storage means for storing the audio / video information. An electronic conference system for generating electronic minutes by audio-video information in a conference,
A motion detection means for detecting a motion of a conference participant included in the video information captured in the conference;
Index information generating means for generating index information corresponding to the video information based on the movement of the conference participant detected by the movement detecting means;
Storage means for storing the audio-video information and the index information;
An electronic conferencing system comprising: An electronic conference system for generating electronic minutes by audio-video information in a conference,
A motion detection means for detecting a motion of a conference participant included in the video information captured in the conference;
A determination unit that determines the presence or absence of movement with reference to a predetermined threshold for the movement of the conference participant detected by the movement detection unit;
Index information generating means for generating index information corresponding to the video information based on the movement of the conference participant determined to be moving by the determining means;
Storage means for storing the audio-video information and the index information;
The determination unit changes the threshold value of the motion used as a determination criterion, and audio / video information reproduced from the storage unit is input to the motion detection unit again, and the index information generation unit re-registers the index information. Feedback means to generate,
An electronic conferencing system comprising: Comprising target specifying means for specifying a conference participant who is a target of motion detection by the motion detecting means, and includes target specifying information in the index information;
The electronic conference system according to any one of claims 1 to 3, wherein the electronic conference system is characterized in that:

Images