JP2001268562A - Method and device for automatically recording live image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize an automatic live image recording capable of sharply reducing manual operation for the record of a lecture. SOLUTION: The device for recording a lecture in space 10 where a lecturer gives the lecture is provided with a camera group 105 for recording images and sounds, an observing camera 102 for observing the space 10 where a lecture gives the lecture, a means 103 for extracting information concerned with the positions, moving directions and operation of the lecturer and other objects from the images of the camera 102, and a means 104 for controlling the camera work of the camera group 105 and the reproduction of electronic teaching materials. The device is provided also with a means 106 for switching and selecting a recording camera image and the reproduced image of the electronic teaching materials and a means 108 for recording the selected image and sound accompanying the image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a system for recording video and audio using a video camera or the like in a space where a lecturer teaches, and more particularly to processing video information obtained from an observation camera. The present invention relates to a technology for detecting and recognizing an object in a video and realizing efficient automatic video recording for creating a remote lecture or a lecture archive.

[0002]

2. Description of the Related Art With the advancement of multimedia-related technology,
2. Description of the Related Art Attempts to digitize video and audio and develop them into various applications have been actively made. Particularly in the education field, as an effective means for solving the barrier of time and space, the use of video and audio-related equipment has led to diversification of attending forms. For example, in a remote lecture, it is possible to take a lecture at a remote place via a network, and to create a lecture archive, it is possible to take a self-learning course at a free time.

[0003] In these cases, the trainee views the lecture through the field of view of the camera, and the adjustment of the camera field of view, which has a significant effect on the visibility of the video, the audibility of the sound, and the understanding of the contents of the lecture, is difficult. It is a very important factor. Conventionally, the adjustment of the field of view of the camera is performed by specialized staff for almost one camera, so that a large amount of human operation is required. In addition, to the person in charge of camera work,
Subtle adjustments to the camera's field of view taking into account the viewer's concentration,
Professional skills were needed.

[0004]

As described above, video recording of conventional live-type materials requires not only a person having specialized skills but also a lot of human operation. Therefore, there is a problem that recording opportunities are restricted and productivity is remarkably reduced.

[0005] An object of the present invention is to solve the above-mentioned problems in the prior art and to automate the recording of live material images, thereby having a functional level equivalent to that of a professional staff.
Another object of the present invention is to provide an automatic recording system that requires less human operation.

[0006]

SUMMARY OF THE INVENTION According to the present invention, in recording a video and an audio using one or more recording cameras in a space where a lecturer teaches, a lecturer or the like is obtained from an observation camera video. Information on the position, moving direction, and motion is extracted, and based on the extracted information, camera work control such as panning, tilting, and zooming of the recording camera and reproduction control of the digitized teaching material are performed. It is characterized in that reproduced videos of the educational material are selected, and the selected video and accompanying audio are recorded.

[0007] For control of the recording camera, a manual camera operation history is input in advance, the manual camera operation history is associated with information extracted from the observation camera image, and the results are accumulated. During actual recording, the camera operation information associated with the information extracted from the observation camera video is read to control the recording camera.

[0008] This makes it possible to automatically record live material in consideration of the ease of use of the viewer without arranging a specialized photographer, and to manually perform remote lectures and lecture archive creation. Can be greatly reduced.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a system according to an embodiment of the present invention. In FIG. 1, 101 is a subject to be recorded, 102 is a group of observation cameras for acquiring various types of information for automatic recording, 103 is an observation video analysis unit that analyzes observation camera images, and 104 is an automatic video recording unit. 105 is a camera control unit for operation history learning and recording for generating camera control information and video switching control information for recording, and a video recording camera group 105 for creating a lecture archive.
Reference numeral 6 denotes a video switching unit for switching between a recording camera video and a teaching material video; 107, a manual operation unit for manually performing video switching and camera control; 108, a video recording unit for recording a recording camera video and a teaching material video; Is a teaching material playback control unit for playing back digitized teaching materials, and 110 is a teaching material projection unit. These substantially constitute a computer control system as a whole.
The operation history learning and recording camera control unit 104 is an image of a central processing unit, and the rest is an image of a peripheral device.

Hereinafter, the overall operation of the system configuration shown in FIG. 1 will be described. The operation is roughly divided into manual input of a camera operation history and actual automatic recording in advance.

In the manual operation history input, the recording camera group 105 is manually operated by the manual operation unit 107 based on an actual lecture in advance, and the operation history is input. Specific operation contents include camera selection, pan, tilde, zoom, and the like. On the other hand, the observation image analysis unit 103 determines the position and movement direction of the instructor based on the image given through the line 111 from the observation camera group 102.
Extract information about actions. These pieces of information will be used later for tracking of an instructor or the like, selection of an appropriate video, setting of a method of photographing an object, and the like. The operation history learning and recording camera control unit 104 inputs the respective control values (control information) of the manual camera operation history by the manual operation unit 107 through the line 120, and the camera operation history by the manual operation and the observation image analysis unit. 1
03 is associated with the information extracted, and the result is recorded.

Next, at the time of actual automatic recording, first, based on an image provided from the observation camera group 102 through the line 111, the observation image analysis unit 103 similarly operates the position, the moving direction, and the operation with respect to the instructor. Extract information about Next, the operation history learning and recording camera control unit 104
Reads the information on the instructor position and the like provided from the observation video analysis unit 103 through the line 112 and the associated camera operation information, controls the recording camera to obtain an appropriate video, controls the teaching material reproduction according to the situation, Any one of the plurality of camera images for recording, or various information for switching control of the educational material playback image, etc.
115 and the like. The recording camera group 105 has a line 11
The camera selection, pan, tilde, zoom, and the like are automatically performed based on the camera work drive control instruction information of No. 3. The teaching material playback control unit 109 controls the output of electronic teaching materials used during the lecture, for example, the output of drawing software running on a personal computer, the output of moving image playback software, etc., based on the teaching material playback control information of the line 115. The reproduced video is output to lines 117 and 119. The video switching unit 106 selects, based on the video selection information on the line 114, a recording camera video provided from the recording camera group 105 via the line 116 or a teaching material playback video provided from the teaching material playback control unit 109 via the line 117. , Line 118.

The video recording unit 108 includes the video switching unit 10
The video selected by 6 is written to a recording medium such as a tape together with the audio accompanying the video. This recorded medium can be used later as a lecture archive. The teaching material projecting unit 110 projects the output of the electronic teaching material image transmitted by the line 119 on a screen in the lecture room.

Next, the processing of the observation video analysis unit 103 and the operation of the operation history learning and recording camera control unit 104 will be described in detail.

FIG. 2 shows an example of the configuration of the observation video analysis unit 103. The observation video analysis unit 103 includes an image difference detection unit 210,
It comprises an object detection unit 220, an object movement detection unit 230, and an instruction operation detection unit 240.

First, a line 111 is obtained from the observation camera group 102.
Is transmitted to the image difference detection unit 210 and the object detection unit 220. Among them, the video difference detection unit 210 sequentially holds images before a certain time, and determines whether there is a change between the observation video at a certain time t1 and the observation video at t2 after a certain time tΔ (second) has elapsed. Detect whether or not. Here, tΔ is a numerical value that can be set arbitrarily, and is usually set to a value of about 1 to 5 (seconds). Various algorithms have been proposed for detecting the image difference, but any of the algorithms can be used in the present invention. For example, in the case of a color image, R
For each of the three primary colors (red), G (green), and B (blue), a method of determining that an image change has occurred when a predetermined numerical value is exceeded can be used. This image difference detection unit 2
The output information from 10 includes the presence or absence of an image difference and the image of the difference portion, and is transmitted to the object movement detection unit 230 and the instruction operation detection unit 240.

On the other hand, the object detecting section 220 operates at the time interval t
For each Δ, it is determined whether or not an image corresponding to the target object specified in advance exists in the input video. The output of the object detection unit 220 includes the presence / absence of the designated object and an image of the detected object, and is transmitted to the object movement detection unit 230 and the instruction operation detection unit 240. Various algorithms have been proposed for this object detection, but any of the algorithms can be used in the present invention. For example, the features of each object to be detected are described and held in advance using parameters such as shape, color, and the like,
At the time of detection, the area described by these parameters is
A method such as searching in the input video can be used.

The object movement detecting section 230 determines from the output of the image difference detecting section 210 that an image difference has been detected,
Further, when it is determined from the output of the object detection unit 220 that the designated object has been detected, a change in the position of the object in the video is calculated based on the difference image, and the object is maintained for a time tΔ. It is determined whether or not it has moved to. The output of the object movement detection unit 230 is a result of the movement detection of the object and the type of the movement object.

On the other hand, the pointing operation detecting section 240 determines that an image difference has been detected from the output of the image difference detecting section 210, and further detects a designated object corresponding to the pointing operation from the output of the object detecting section 220. If it is determined that
The change in the position of the object in the video is calculated based on the difference image, and the presence or absence of the instruction operation pattern is determined over a plurality of time tΔ sections. This instruction operation detecting unit 2
The output of 40 is the detection result of the instruction operation and the type of the instruction operation.

As described above, the observation image analysis unit 103
Then, based on the video of the observation camera group 102, at the time of manual operation or actual recording, the movement detection result of the object, the type of the movement object, the detection result of the pointing operation, and the type of the pointing operation are momentarily Is transmitted to the operation history learning and recording camera control unit 104 through the line 112.

FIG. 3 shows an example of the configuration of the operation history learning and recording camera control unit 104. The operation history learning and recording camera control unit 104 includes an operation history recording unit 310, an association unit 320 between the operation and the object situation, a related appearance frequency measurement unit 330, an operation pattern recording unit 340, a camera drive control unit 350, a video It comprises a switching control unit 360 and the like.

Operation history learning and recording camera control unit 1
The processing of step 04 can be broadly divided into camera operation history learning processing and recording camera control processing. Hereinafter, these processes will be described in order.

First, the camera operation history learning process will be described. First, the manual operation signal transmitted by the line 120 from the manual operation unit 107 is transmitted to the recording camera group 102 through the line 113 as it is and stored in the operation history recording unit 310. The associating unit 320 with the operation target object state receives the manual operation signal from the operation history recording unit 310 and inputs the manual operation signal.
The information given from 3 through the line 112, that is, the detection result of the movement of the object, the type of the movement object, the detection result of the instruction operation, and the type of the instruction operation are input. The associating unit 320 for associating the operation with the object situation associates the information from the observation video analyzing unit 103 input at the same time as each manual operation, and associates the result with the related appearance frequency measuring unit 33.
Tell 0. The related appearance frequency measuring unit 330 measures the appearance frequency of the inputted association, detects an association exceeding a predetermined appearance frequency, and transmits the detected result to the operation pattern recording unit 340. Operation pattern recording unit 34
In the case of 0, the camera manual operation corresponding to the detected association, the movement detection result of the object output from the observation video analysis unit 103, the type of the movement object, the detection result of the instruction operation, and the type of the instruction operation Record the set internally.

With the above processing, the recording of the camerawork control history learning corresponding to the result of the analysis of the observed video based on the manual operation can be realized.

Next, processing such as control of a recording camera will be described. At the time of actual recording, from the observation video analysis unit 103,
From time to time, the movement detection result of the object via the line 112,
The type of the moving object, the detection result of the instruction operation, and the type of the instruction operation are input, and these are directly input to the operation object state and the association unit 320 and the related appearance frequency measurement unit 33.
0 to the operation pattern recording unit 340. The operation pattern recording unit 340 reads out the operation pattern associated with the input information from the information recorded in advance, and transmits the operation pattern to the camera drive control unit 350 and the video switching control unit 360. The camera drive control unit 350 outputs a camera drive control instruction corresponding to the transmitted operation pattern to the line 113. Further, the video switching control unit 360 outputs the corresponding video selection information to the line 114. Further, the operation pattern transmitted to the video switching control unit 360 is also output to the line 115 as teaching material reproduction control information.

With the above processing, the recording camera group 105
In contrast to this, it is possible to automatically drive appropriate camera work corresponding to the observed video, and realize control of switching between the image of the recording camera group 105 and the image of the electronic teaching material used during the lecture. it can.

[0027]

As described above, according to the present invention, it is possible to realize the mechanization of the camera work according to the dedicated photographer, and thereby it is possible to greatly reduce the operation during the lecture recording. Automatic recording of live video can be realized.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration example of an observation video analysis unit in FIG. 1;

FIG. 3 is a diagram showing a configuration example of a camera control unit for operation history learning and recording in FIG. 1;

[Description of Signs] 101 Recording target object 102 Observation camera group 103 Observation video analysis unit 104 Operation history learning and recording camera control unit 105 Recording camera group 106 Video switching unit 107 Manual operation unit 108 Video recording unit 109 Teaching material playback control Unit 110 Teaching material projection unit 210 Image difference detection unit 220 Object detection unit 230 Object movement detection unit 240 Pointing operation detection unit 310 Operation history recording unit 320 Associating unit between operation and object situation 330 Related occurrence frequency measurement unit 340 Operation pattern Recording unit 350 Camera drive control unit 360 Image switching control unit

Continuation of front page (72) Inventor Susumu Fujiwara 2-3-1 Otemachi, Chiyoda-ku, Tokyo Nippon Telegraph and Telephone Corporation (72) Inventor Masahiko Tsujimoto 2-3-1 Otemachi, Chiyoda-ku, Tokyo Sun F Term (Reference) 5C022 AA01 AB36 AB63 AB65 AB66 AC69 CA02 5C053 GB19 LA01 LA06 5C054 AA01 CG06 CG08 FE24 FF03 GA01 GB02 GD04 HA31

Claims (4)

[Claims] 1. A method of recording video and audio using one or more recording cameras in a space where the instructor teaches, comprising: an observing camera for observing the space where the instructor teaches; Prepare, extract information on the position, moving direction, and movement of the lecturer and other objects from the observation camera video, and control camera work for the recording camera and playback control of digitized teaching materials based on the extracted information. A live video recording method, wherein a recording camera video and a playback video of an electronic teaching material are selected, and the selected video and accompanying audio are recorded. 2. A method for automatically recording live video according to claim 1, wherein a history of manual camera operation is input in advance, and the manual camera operation history is associated with information extracted from the observation camera video. A method for automatically recording live video, comprising storing results, reading out camera operation information associated with information extracted from observational camera video during actual recording, and controlling the recording camera. 3. An apparatus for recording video and audio using one or more recording cameras in a space where the instructor teaches, wherein an observation camera for observing the space where the instructor teaches is provided. Means for extracting information on the position, moving direction, and movement of the lecturer and other objects from the observation camera video, and camera work control for the recording camera and reproduction of the electronic teaching materials based on the extracted information. Means for controlling, a means for selecting a playback camera video and a playback video of the digitized teaching material, and a means for recording the selected video and accompanying audio. Automatic recording device. 4. A live video automatic recording apparatus according to claim 3, wherein means for inputting a manual camera operation history in advance is associated with the manual camera operation history and information extracted from the observation camera video. Means, means for accumulating the associated results, and means for reading camera operation information associated with information extracted from the video of the observation camera during actual recording and controlling the recording camera. An automatic recording device for live video.