JP4742196B2 - Presentation system and content creation system - Google Patents

Description

  The present invention relates to a presentation system that displays an image generated using a computer, and a content creation system that creates content including an image generated by the presentation system.

  A presentation system having a large display screen is used for lectures, lectures, presentations, and the like. As a presentation system, there is a computer-based system that displays an image generated using a computer on a liquid crystal projector, a large flat panel display, or the like. In addition, it has also been proposed to generate and display image information in which handwritten information is added to a digitized document created in advance when giving a presentation or lecture (see Patent Document 1 and Patent Document 2).

  In order to switch the display of the presentation image or to display the handwritten image, an input unit operated by a presenter, a lecturer, or the like is required, and a keyboard, a mouse, a tablet, or the like is generally used. However, since these input means are fixedly installed near the computer, it is difficult to make presentations and lectures while moving.

  Japanese Patent Application Laid-Open No. 2004-228561 describes a technique that enables a computer to be operated at a position away from the installation position. In the pointing method described in Patent Document 3, a camera that captures an image including a color tag held by an operator is installed, and a cursor is placed on the display screen based on the position of the color tag in the captured image. To display. Therefore, since the operator can move the cursor by moving the color tag, the operator can operate the computer even from a position away from the vicinity of the installation position of the computer.

  In the pointing method described in Patent Document 3, it is necessary to set a conversion parameter for associating the position of the color tag with the cursor position in the captured image. This conversion parameter is for conversion by rotation / enlargement / reduction / movement processing in consideration of the camera installation position, display installation position, and camera angle of view, and the display is viewed from the average standing position of the operator. Preliminary calibration is set so that the appropriate range can be designated. Therefore, the range in which the operator can operate the computer is limited to a predetermined angle of view of the camera.

JP 2002-44585 A JP 2001-16384 A JP 2004-272598 A

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a presentation system capable of extending a moving range in which a presenter can operate a display image. It is another object of the present invention to provide a content creation system that creates content including an image created using such a presentation system.

The presentation system of the present invention is a presentation system that displays an image generated by using a computer, and is an image capturing unit that captures an image of a presenter, and composite image data that partially includes captured image data from the image capturing unit. Image display means for generating an image, display means for displaying an image based on the composite image data, and pointing processing means for recognizing a pointing operation by the presenter based on the captured image data included in the composite image data. The pointing processing means identifies a specific object moved by the presenter included in the captured image data, recognizes a relative position coordinate of the specific object in an image display region based on the captured image data, and Point on display screen based on relative position coordinates It recognizes the Ingu position, the image synthesizing means, in which by combining the mirror image data of the photographed image data to generate the synthetic image data.

  According to the present invention, the area to be synthesized can be easily changed in the image data taken by the imaging means, so that the area of the image for recognizing the pointing position by the presenter, that is, the specific object moved by the presenter can be displayed. You can easily change the area. Therefore, by changing the region to be synthesized even if the presenter moves, pointing becomes possible, and the range in which the presenter can operate the display image can be expanded. Also, if the presenter's captured image contains a disturbing object that hinders identification of a specific object, the pointing position recognition accuracy is improved by using an image of the area that does not include the disturbing object as a composite area. Can be made. In addition, since the captured image data included in the composite image data for display can be only the captured image data of the portion necessary for the pointing process, the recognition accuracy of the pointing position can be improved, and the recognition processing burden is increased. Can be reduced. In addition, it is easy for the presenter to point over the entire area of the display screen.

  The presentation system of the present invention includes one in which the image synthesizing unit recognizes a region to be synthesized of the photographed image data based on an operation on the photographed image displayed on the display unit. According to the present invention, a captured image area for pointing position recognition can be easily changed.

  The presentation system of the present invention includes one in which the pointing processing means recognizes coordinates obtained by scaling the relative position coordinates to coordinates of the entire display area of the display means as the pointing position.

  The presentation system of the present invention includes one in which the specific object is an object that emits or reflects light of a specific color. Specifically, a light emitter, a specific color tape, or the like can be used.

  The content creation system according to the present invention includes a video signal generation unit that generates a display composite image signal based on the composite image data generated by the image synthesis unit in the presentation system, and a digital signal based on the display composite image signal. And a moving image file generating means for generating a moving image file including moving image data.

  The lecture video creation system of the present invention outputs the moving image file generated using the content creation system described above as a lecture video.

  As is clear from the above description, according to the present invention, it is possible to provide a presentation system that can expand the moving range in which the presenter can operate the display image. In addition, it is possible to provide a content creation system that creates content including an image created using such a presentation system.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, a lecture video creation system that uses a presentation system for lectures and creates lecture videos at the same time is used as an application example.

  FIG. 1 is a diagram showing a schematic configuration of a lecture video creation system which is an example of a content creation system. The lecture video creation system shown in FIG. 1 creates a lecture video simultaneously with a lecture in a classroom or the like. The lecturer computer 1, camera 2, tablet 3, projector 4, scan converter 5, recording computer 6, microphone 7 The video server 8 is included.

  The instructor computer 1 is a computer used by a lecturer who is a presenter for a lecture, and is a notebook PC, for example. The lecturer computer 1 is prepared with lecture materials created in advance by presentation software such as Power Point. In addition, when using the contents of a website for lectures, a web browser is installed and connected to the Internet.

  A camera 2 and a tablet 3 are connected to the instructor computer 1 by, for example, USB connection. The camera 2 is an instructor photographing camera that photographs an instructor during a lecture, and inputs a moving image to the instructor computer 1. It is for input. The instructor computer 1 includes a keyboard, a mouse, and a liquid crystal display unit (all not shown) as operation means. The instructor computer 1 is operated by such an operation means to perform switching of a display image, and further, a pointing operation based on a photographed image of the camera 2 is also possible. The pointing operation will be described later.

  The instructor computer 1 is preinstalled with software for displaying video from the camera 2 on the desktop and software for drawing handwritten information from the tablet 3 on the desktop. These software can be easily created by a known technique. The already created software can be downloaded by, for example, “COE e-Learning Tools”, <URL: http://coe-el.sfc.keio.ac.jp/>. The software downloaded from this site generates composite image data obtained by combining a captured moving image from the camera 2 and a handwritten image from the tablet 3 with one or more lecture material images.

  The composite image data generated here may be a superposition of a plurality of images, a partial overwrite of a part of the images, or a configuration in which each image is arranged in a predetermined size area. Further, the size of each application image (including a camera image and a handwritten image) to be combined is arbitrary and can be changed by the lecturer. In addition, the instructor can also change the synthesis area in the camera image. However, the camera image is combined with a part of the display area so as not to disturb the display of the lecture material. Further, for a pointing operation described later, it is preferable that the camera image is synthesized after being converted into mirror image data.

  The operation for changing the size of the camera image and the operation for changing the composition area are performed by operating the mouse. FIG. 2 shows an example of an operation for changing the size of the camera image. Assume that the camera image is displayed in the size of the area 210a in the lower right portion of the display screen 200. In this state, the size of the camera image is changed by dragging the point 220a at the upper left corner of the area 21a. For example, when the point 220a is dragged to the point 220b, the camera image is displayed with the size of the area 210b, and when the point 220c is dragged, the camera image is displayed with the size of the area 210c.

  FIG. 3 shows an example of the operation for changing the composite area of the camera image. It is assumed that the entire camera image is displayed in the lower right portion of the display screen 300 with the size of the area 310a. In this state, a camera image is selected and moved. This processing can be performed by selecting and moving a known display window. Now, when the selected camera image is moved in the lower right direction, the camera image displayed on the display screen 300 is cropped in a region 310b (a region with hatching in the lower left direction). Displayed with status. When the image is further moved in the lower right direction, the camera image is displayed in an area 310c (an area with diagonally lower right hatching). In other words, the parts of the areas 320b and 320c virtually shown in FIG. 3 excluding the captured image data are combined.

  In the above description, the camera image is displayed at the lower right of the display screen. However, the position is not limited to the lower right as long as it does not interfere with the display of the lecture material. In the system of FIG. 1, the number of cameras 2 is one, but a plurality of cameras may be provided and displayed in separate windows. Further, the camera image is trimmed so as to leave the upper left part of the camera image by moving it, but it is also possible to leave an arbitrary part by operating the mouse.

  The composite image data for displaying such a composite image is written in a frame buffer (not shown) of the instructor computer 1 and used for displaying the desktop screen of the instructor computer 1. Also, the image data in the frame buffer is used for pointing processing described later.

  A projector 4 for displaying a desktop screen as an image on a large-scale screen (not shown) is connected to an external monitor output terminal (not shown) of the instructor computer 1. The scan converter 5 is connected to an external monitor output terminal (not shown) of the instructor computer 1 and converts a digital signal output from the output terminal into an analog video signal which is one of display image signals. is there.

  The recording computer 6 inputs the analog video signal acquired by the scan converter 5 through a video capture board and encodes it in real time into a moving image file such as Windows Media format (.WMV) using existing video capture software. Video files in Windows Media format are very lightweight. For example, if the recording resolution is set to 640 pixels × 480 pixels and the distribution bit rate is set to 250 bps, the file capacity per hour is about 100 MB. The recording resolution is 640 pixels × 480 pixels, and the material on the screen of the instructor computer 1 and the board drawing by the tablet drawing can be read without any problem. Also, the frame rate is about 10 fps, and it is possible to browse the instructor's facial expression and the movement of the board without any discomfort. The performance of the recording computer 6 is, for example, Pentium IV. A 4 GHz processor, a memory of 1 GB, and a hard disk capacity of 180 GB.

  The microphone 7 is used to acquire a lecturer's audio signal, and is connected to the recording computer 6. The recording computer 6 adds audio data when the moving image file is generated. Although the microphone 7 is connected to the recording computer 6 in FIG. 1, the audio data acquired by the instructor computer 1 may be sent to the recording computer 6 by connecting to the instructor computer 1.

  The video server 8 uploads the moving image file created by the recording computer 6 and distributes it in a streaming manner. The video server 8 is, for example, a computer in which Windows 2000 Server is installed, and its performance is a Pentium III, 750 MHz processor, memory 512 MB, and hard disk capacity 240 GB.

  The operation of the lecture video creation system having such a configuration will be described. In the lecture room, devices other than the lecturer computer 1 are prepared in advance. The lecturer connects the camera 2 and the tablet 3 to the USB terminal of his computer 1 storing the lecture material, and connects the projector 4 and the scan converter 5 to the video output terminal. Then, all the devices are operated, and a lecture material display application prepared in the instructor computer 1 is started.

  The lecturer starts the lecture in such a system state, and advances the lecture while displaying necessary lecture materials on the instructor computer 1. Since the image display signal of the instructor computer 1 is sent to the projector 4, it is also displayed on a large-scale screen (not shown). On the instructor computer 1, a photographed image from the camera 2 is displayed on a part of the lecture material. FIG. 4 shows an example of the display image.

  FIG. 4 schematically shows a state in which the display image 410 by the presentation software is displayed in almost the most area of the display screen 400 and the instructor's photographed image 420 is displayed in the lower right part of the display screen 400. Is. The lecturer 430 in the photographed image 420 of the lecturer holds an object 430a for recognizing the pointing position. A cursor 440 is displayed at a position on the display screen 400 corresponding to the relative position coordinates of the object 430a in the area occupied by the captured image 420. The object 430a for recognizing the pointing position is, for example, an object that emits or reflects light of a specific color.

  The image display signal of the instructor computer 1 is sent to the scan converter 5 at the same time, and the scan converter 5 generates an analog video signal based on the image display signal. The generated analog video signal is sent to the recording computer 6 and converted into a digital moving image file. That is, the analog video signal is input via a video capture board (not shown) of the recording computer 6 and encoded in real time into digital image data in Windows Media format (WMV) using existing video capture software. . At that time, the audio signal input by the microphone 7 is also digitized and output together.

  A Windows Media format (WMV) video file created by the recording computer 6 is uploaded to the video server 8. Then, it is provided for distribution of lecture videos via a network (not shown). Since the uploaded lecture video is a moving image file, it is possible to distribute the stream. Therefore, live distribution can be performed almost simultaneously with the actual lecture, and a remote lecture can be realized.

  A Windows Media format (WMV) video file created by the recording computer 6 is uploaded to the video server 8. Then, it is provided for distribution of lecture videos via a network (not shown). Since the uploaded lecture video is a moving image file, it is possible to distribute the stream. Therefore, live distribution can be performed almost simultaneously with the actual lecture, and a remote lecture can be realized.

  Next, the pointing process performed by the instructor computer 1 will be described. The pointing process is a process for recognizing a pointing operation performed by a lecturer based on camera image data included in the composite image data. Specifically, a specific object (for example, an object that emits or reflects light of a specific color) held by the instructor included in the camera image data is identified, and the relative relationship of the specific object in the image display area based on the camera image data The position coordinates are recognized, and the pointing position on the display screen is recognized based on the recognized relative position coordinates. The pointing position on the display screen is obtained by scaling the recognized relative position coordinates to the coordinates of the entire display area.

FIG. 5 shows a schematic flow of an example of the pointing process. In step S101, the composite image data stored in the frame buffer is acquired, and in step S102, data specifying the camera image display window in the display area is acquired. Now, it is assumed that the display screen is made up of (X ₀ +1) × (Y ₀ +1) pixels, and pixel data is stored in the frame buffer in correspondence with the coordinates as shown in FIG. An area 610 in FIG. 6B is a window area of the camera image. When such composition is performed, the coordinates of the upper left point of the area 610 are acquired as data for specifying the camera image window.

  Subsequently, in step S103, the specific object in the camera image window area is searched. For example, a technique disclosed in Patent Document 3 can be used for image processing for searching for a specific object. If one specific object can be recognized in the search result camera image window area, it is determined that the search is successful, and if the specific object cannot be recognized or if two or more specific objects can be recognized, the search is determined to be unsuccessful. (Step S104).

If the search is successful, the relative coordinates of the center of gravity position of the recognized specific object are calculated (step S105). The relative coordinates are coordinates in the window area 610 and can be obtained as follows. As shown in FIG. 6A, in the display screen coordinate system (X, Y), the coordinates of the upper left point of the window area 610 are (X _W , Y _W ), and the coordinates of the centroid position P ₁ of the specific object are It is assumed that (X ₁ , Y ₁ ). Here, considering a coordinate system (x, y) with A as the origin as shown in FIG. 6B, the relative coordinates (x ₁ , y ₁ ) of the point P ₁ are (X ₁ −X _W , Y ₁ ).

Next, the coordinate system of the point _{P 1} in the window area 610 (x, y) coordinates relative to, the scale transformation to the coordinates of the entire display screen (step S106). Since the number of pixels of the entire display screen is (X ₀ +1) × (Y ₀ +1) and the number of pixels in the window area 610 is (X ₀ −X _W +1) × (Y _W +1), the point P _The coordinates obtained by converting the scale of ₁ are ((X ₀ +1) (X ₁ −X _W ) / (X ₀ −X _W +1), (Y ₀ +1) (Y ₁ ) / (Y _W +1)). In the example of FIG. 6A, the position of the point Pc is the scaled coordinate. In step S107, the scale-converted coordinates are output as pointing position information. This pointing position information is used, for example, for cursor display and creation of handwritten trajectory data.

  By the pointing process, the same pointing as that of the mouse can be performed in a state away from the computer. Note that the input corresponding to the mouse click is performed by operating another remote control terminal, for example. Since the remote control terminal in this case outputs one or two on / off signals, a small and inexpensive one can be used. It is also possible to recognize a click operation by using a specific object held by the presenter that can emit or reflect light of a plurality of colors and changing the color of the specific object.

  If it is determined in step S104 that the search is unsuccessful, the search unsuccessful notification is sent to the OS of the instructor computer 1 in step S108, and the process ends.

  As described above, in the present invention, since the specific object identification process is performed based on the composite image data, the procedure of the pointing process itself is not changed, and the processing load does not increase. In addition, since an image captured by the camera is displayed on a part of the display unit, the presenter can arbitrarily set the pointing position by moving the specific object to be included in the image.

  Furthermore, by appropriately changing the area of the camera image data to be synthesized, it is possible to synthesize only the portion suitable for the pointing process, so that the recognition accuracy of the pointing position can be improved and the recognition process burden is reduced. Can be reduced. It is also possible for the presenter to synthesize an area that can be easily pointed over the entire area of the display screen. The change operation of the composite area may be performed by the presenter using a mouse or the like, but may be performed by a person who assists the presentation using another mouse or the like.

The figure which shows schematic structure of the lecture video creation system of embodiment of this invention The figure which shows an example of the operation which changes the magnitude | size of the camera image in the lecture video creation system of embodiment of this invention The figure which shows an example of the change operation of the synthetic | combination area | region of a camera image in the lecture video creation system of embodiment of this invention The figure which shows an example of the display image in the lecture video creation system of embodiment of this invention The figure which shows the schematic flow of an example of the pointing process in the lecture video production system of embodiment of this invention The figure explaining an example of the pointing process in the lecture video creation system of embodiment of this invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Instructor computer 2 ... Camera 3 ... Tablet 4 ... Projector 5 ... Scan converter 6 ... Recording computer 7 ... Microphone 8 ... Video server

Claims (6)

A presentation system for displaying an image generated using a computer,
Imaging means for taking an image of the presenter;
Image composition means for generating composite image data including a part of the captured image data from the imaging means;
Display means for displaying an image based on the composite image data;
Pointing processing means for recognizing a pointing operation by the presenter based on the captured image data included in the composite image data,
The pointing processing means identifies a specific object to be moved by the presenter included in the captured image data, recognizes a relative position coordinate of the specific object in an image display area based on the captured image data, and Recognize the pointing position on the display screen based on the coordinates ,
The image synthesizing unit is a presentation system that synthesizes mirror image data of the captured image data to generate the synthesized image data . The presentation system according to claim 1,
The presentation system that recognizes an area to be synthesized of the photographed image data based on an operation on the photographed image displayed on the display means . The presentation system according to claim 1 or 2,
The presentation system in which the pointing processing means recognizes coordinates obtained by scaling the relative position coordinates to coordinates of the entire display area of the display means as the pointing position . The presentation system according to any one of claims 1 to 3,
The presentation system , wherein the specific object is an object that emits or reflects light of a specific color . Video signal generation means for generating a composite image signal for display based on the composite image data generated by the image composition means in the presentation system according to any one of claims 1 to 4 ,
A content creation system comprising: a moving image file generating unit that generates a moving image file including digital moving image data based on the composite image signal for display. A lecture video creation system for outputting the video file generated by using the content creation system according to claim 5 as a lecture video.

Images