JP2010272078A - System, and control unit of electronic information board, and cursor control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an electronic information board system which makes a coordinate position of a cursor on a computer terminal correspond to a coordinate position of infrared light on a screen region. <P>SOLUTION: A projector projects a display image output from an electronic information board controller, and a camera generates a picked-up image obtained by imaging infrared light emitted from a light-emitting pen in a screen region on which the display image is projected, and the electronic information board controller reads a display image displayed on a display part of the computer terminal to output the display image to the projector and calculates an absolute coordinate position of infrared light included in the picked-up image generated by the camera and transmits the calculated absolute coordinate position to the computer terminal, and the computer terminal receives the absolute coordinate position and inputs the received absolute coordinate position to a cursor control part of the computer terminal itself and displays the cursor in a coordinate position indicated in the display image on the display part by the input absolute coordinate position. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an electronic information board system, an electronic information board control device, and a cursor control method for moving a cursor on a computer terminal according to infrared light on a screen area.

  In recent years, an image displayed on a display unit of a computer terminal is projected onto a screen area such as a whiteboard by a projector, a pen moving on the screen area in accordance with a user operation is detected by a camera, and the locus of the detected pen There is provided an electronic information board system that draws an image on a computer terminal according to the operation. As the pen for moving the screen area, for example, a light emitting pen that emits infrared light by contacting a whiteboard is used. Then, the cursor on the computer terminal is moved according to the operation of the light-emitting pen captured by the camera that captures infrared light, and drawing is performed on the image on the computer terminal in accordance with the movement of the cursor. According to such an electronic information board system, a plurality of people can simultaneously view the image projected on the screen area by the projector, and the user can write on the projected image by moving the pen on the screen area. Can provide an electronic blackboard. Patent Document 1 proposes a remote lecture system using such an electronic information board system.

  In such an electronic information board system, the amount of movement such as the moving angle and direction of infrared light in the screen area imaged by the camera is represented by relative coordinates and input to a mouse emulator provided in the computer terminal. . The mouse emulator is software that moves the mouse on the computer terminal in accordance with the input relative coordinates. The relative coordinates are information indicating the coordinate position of the infrared light after moving within a certain time with respect to the coordinate position of the infrared light at a certain time. For example, when the infrared light existing at the position of the symbol a in FIG. 9 moves to the position of the symbol a ′ in FIG. 10 after a certain time, the amount of movement of the infrared light is the coordinates of the symbols a and a ′. It is represented by a difference in position (100 (pixels) in the X-axis direction).

JP 2005-234368 A

  However, generally, the moving speed of the cursor indicating the moving amount of the cursor relative to the moving amount of the physical mouse in the computer terminal can be changed by setting. For example, when the mouse is physically moved by 1 inch by a user operation, the correspondence relationship of how many pixels the cursor on the screen of the computer terminal is moved may be different for each computer terminal. That is, when the relative coordinates based on the physical movement amount of infrared light are transmitted to the computer terminal, the computer terminal determines the movement amount of the cursor according to the movement speed set by itself. For this reason, the coordinate position of the infrared light that moves on the screen area may differ from the coordinate position of the cursor that moves on the display screen of the computer terminal. In this case, drawing cannot be performed at a position on the computer terminal corresponding to the position of the light emitting pen moved by the user on the screen area.

  The present invention has been made in view of such a situation, and an electronic information board system, an electronic information board control device, and an electronic information board control device that correspond the coordinate position of a cursor on a computer terminal to the coordinate position of infrared light on a screen area. A cursor control method is provided.

  In order to solve the above-described problems, the present invention provides a projector that projects an input display image on a screen area, a camera that captures a screen area on which the display image is projected by the projector, and generates a captured image. Used to illuminate a light emitting pen that emits infrared light between the screen area and the camera, an electronic information board control device that receives a captured image generated by the camera, and a display unit that displays the display image An electronic information board system including a computer terminal, wherein the projector projects a display image output from the electronic information boat control device, and the camera emits red light emitted from a light-emitting pen in a screen area on which the display image is projected. The electronic information board control device generates a picked-up image obtained by picking up external light. A display image input unit that reads a display image displayed on the display unit, a display image output unit that outputs a display image read by the display image input unit to a projector, and infrared light included in a captured image generated by the camera. An absolute coordinate calculation unit that calculates an absolute coordinate position and transmits the absolute coordinate position to the computer terminal. The computer terminal receives the absolute coordinate position calculated by the electronic information board control device, and controls the received absolute coordinate position with its own cursor. An absolute coordinate position input unit that is input to the unit, and a cursor control unit that displays a cursor at the coordinate position indicated by the absolute coordinate position input by the absolute coordinate position input unit in the display image.

  The present invention also provides that the electronic information board control apparatus and the computer terminal are connected via a USB device, and the absolute coordinate position input unit of the computer terminal is connected to the USB port of its own USB port. It is installed by a standard HID driver.

  The present invention also includes a projector that projects an input display image on a screen area, a camera that generates a captured image obtained by capturing infrared light emitted by a user's light-emitting pen in the screen area on which the display image is projected, An electronic information board control device that receives a captured image generated by the camera, and a display image that is displayed on the display unit provided in the electronic information board control device, and the absolute coordinate position received from the electronic information board control device is at a coordinate position indicated on the display unit An electronic information board control device in an electronic information board system including a computer terminal for displaying a display image input unit for reading a display image displayed on a display unit of the computer terminal, and a display read by the display image input unit A display image output unit that outputs images to the projector and a camera Calculates an absolute coordinate position of the infrared light contained in the image picture, characterized in that it comprises an absolute coordinate calculation unit that transmits to the computer terminal.

  The present invention also provides a projector that projects an input display image on a screen area, a camera that captures a screen area on which the display image is projected by the projector and generates a captured image, and a screen area that is used by a user. An electronic equipped with a light emitting pen that emits infrared light with a camera, an electronic information board control device that receives a captured image generated by the camera, and a computer terminal that displays a display image on its own display unit A cursor control method for an information board system, wherein the projector projects a display image output from the electronic information boat control device, and the camera emits infrared light emitted from a light-emitting pen in a screen area on which the display image is projected. A step of generating a captured image obtained by imaging light, and display of the electronic information board control device; An image input unit reads a display image displayed on a display unit of a computer terminal, a display image output unit outputs a display image read by the display image input unit to a projector, and an absolute coordinate calculation unit The absolute coordinate position of the infrared light included in the captured image generated by the camera is calculated and transmitted to the computer terminal, and the absolute coordinate position input unit of the computer terminal is calculated by the electronic information board controller. The step of receiving the coordinate position and inputting the received absolute coordinate position to its own cursor control unit, and the cursor control unit moves the cursor to the coordinate position indicated by the absolute coordinate position input by the absolute coordinate position input unit in the display image And a step of displaying.

  As described above, according to the present invention, the projector projects the display image output from the electronic information boat control device, and the camera emits infrared light emitted from the light-emitting pen in the screen area on which the display image is projected. The electronic information board control device reads out the display image displayed on the display unit of the computer terminal and outputs it to the projector, and the infrared light contained in the captured image generated by the camera is generated. The coordinate position is calculated and transmitted to the computer terminal, the computer terminal receives the absolute coordinate position, the received absolute coordinate position is input to its own cursor control unit, and the input absolute coordinate position is the display image on the display unit. Since the cursor is displayed at the coordinate position shown in, the cursor movement speed setting on the computer terminal is set. Regardless, the coordinate position of the cursor is displayed in a computer terminal, it becomes possible to coordinate located at the same coordinate position of the infrared light on the screen area.

It is a figure which shows the concept of the electronic information board system by one Embodiment of this invention. It is a block diagram which shows the structural example of the electronic information board system by one Embodiment of this invention. It is a figure which shows the example of the display image by one Embodiment of this invention. It is a figure which shows the example of the captured image by one Embodiment of this invention. It is a figure which shows the example of the drawing toolbox by one Embodiment of this invention. It is a figure which shows the example of the absolute coordinate position by one Embodiment of this invention. It is a figure which shows the example of the absolute coordinate position by one Embodiment of this invention. It is a flowchart which shows the operation example of the electronic information board system by one Embodiment of this invention. It is a figure which shows the example of the relative coordinate by a prior art. It is a figure which shows the example of the relative coordinate by a prior art.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing the concept of the electronic information board system 1 according to the present embodiment. Here, an example in which a user makes a presentation using the electronic information board system 1 is shown. For example, a whiteboard that is a screen area 200 for projecting a screen is installed in a room where a presentation is performed. The projector 100 is installed so as to project an image toward the screen area 200. The user stands in front of the screen area 200 and moves the light-emitting pen 210 on the screen projected on the screen area 200 so as to write on the blackboard with chalk. A light emitting unit that emits infrared light is provided at the tip of the light emitting pen 210 and emits infrared light when it is detected that it is in contact with the screen region 200. The camera 300 is installed so as to capture the entire screen area 200. The electronic information board control device 500 is connected to a computer terminal 400, a digital camera 480, a document camera 490, etc., in which display images are stored, via a USB (Universal Serial Bus) cable or the like.

FIG. 2 is a block diagram showing the configuration of the electronic information board system 1 according to the present embodiment. The projector 100 is a display device that receives a display image output from the electronic information board control device 500 and projects the received display image onto the screen area 200.
The screen area 200 is a projection surface of a display image projected by the projector 100. In the present embodiment, the screen area 200 is described as a whiteboard. However, the screen area 200 may be anything as long as an image is projected from the projector 100, and may be a flat part of an indoor wall, cloth, vinyl sheet, or the like. A screen may be used.

  The camera 300 is an imaging device that images the screen area 200. The camera 300 has a function of imaging infrared light. For example, only infrared light is captured by imaging through a visible light cut filter (infrared light transmission filter) installed in front of a lens included in the imaging camera 300. Can be imaged.

The electronic information board control device 500 includes a display image input unit 510, a display image output unit 520, a captured image reception unit 530, an infrared light detection unit 531, a projective conversion unit, a drawing processing unit 550, and an overlay process. The computer device includes a unit 560 and an absolute coordinate calculation unit 570.
The display image input unit 510 receives an input of an image to be projected by the projector 100. For example, the display image input unit 510 is connected to the digital camera 480 by a USB cable or the like, reads a photograph stored in the connected digital camera 480, and receives an input using the read photograph as a display image. Alternatively, the display image input unit 510 is connected to the document camera 490 and receives an input as an image captured by the connected document camera 490 as a display image. Alternatively, the display image input unit 510 is connected to the user's computer terminal 400 and accepts an input as an image stored in a storage area of the connected computer terminal 400 as a display image. In the present embodiment, the display image input unit 510 will be described assuming that a presentation image read from the computer terminal 400 is input. The display image input unit 510 may store the input image in its own storage area and output it to the display image output unit 520, or display an image that changes according to the user's operation on the computer terminal 400 or the document. It may be received at any time from the camera 490 or the like and output to the display image output unit 520.

Display image output unit 520 transmits the display image input to display image input unit 510 to projector 100.
The captured image receiving unit 530 receives a captured image of the screen area 200 captured by the camera 300. Here, the captured image received by the image receiving unit 440 from the camera 300 is an image distorted as compared with the image output by the display image output unit 520 due to distortion of the screen region 200 itself or aberration distortion of the camera 300. For example, when the display image output from the display image output unit 520 is a rectangle along the X axis and the Y axis as shown in FIG. 3, the captured image captured by the camera is as shown in FIG. The image will swell in an ellipse.
The infrared light detection unit 531 detects infrared light included in the captured image received by the captured image reception unit 530, and detects the coordinate position of the infrared light in the captured image.

  The projective conversion unit 540 performs projective conversion on the coordinate position of the infrared light detected by the infrared light detection unit 531 and converts the coordinate position of the infrared light in the coordinate system of the captured image into the coordinate position in the coordinate system of the display image. Process. For example, when the infrared light included in the captured image received by the captured image receiving unit 530 moves, the coordinate position of the infrared light after the movement is based on the projective transformation parameter stored in the projective transformation parameter storage unit 470. By performing projective transformation, the coordinate position in the display image corresponding to the coordinate position of the infrared light is calculated. For example, the projective transformation unit 480 calculates the point (X, Y) in the projection image shown in FIG. 3 corresponding to the point (x, y) in the captured image shown in FIG. In this way, not all the captured images received by the captured image receiving unit 530 are projectively converted, but only the infrared light coordinate position included in the captured image is projectively converted, so that the light emitting pen on the screen region 200 is displayed. The position 210 can be efficiently converted into a coordinate position in the coordinate system of the display image.

  The drawing processing unit 550 performs a drawing process according to the coordinate position calculated by the projective conversion unit 540 based on a predetermined setting. For example, the drawing processing unit 550 performs the same drawing process as that operated by the mouse at the coordinate position in the display image calculated by the projective conversion unit 540. Here, the drawing processing unit 550 projects a drawing tool box as shown in FIG. 5 on the projector 100, receives input of setting information related to drawing processing, and stores the drawing setting information in its own storage area. Anyway. For example, when “pen” is selected in the drawing toolbox, the drawing processing unit 550 draws a drawing image in which a line having a thickness corresponding to the setting is drawn at the coordinate position of the infrared light that has undergone the projective transformation. Generate.

  The overlay processing unit 560 performs overlay processing in which the drawing image generated by the drawing processing unit 550 is superimposed on the image output to the projector 100 by the display image output unit 520 and combined. Further, the image synthesized by the overlay processing unit 560 may be stored in the storage area of the computer terminal 400.

  The absolute coordinate calculation unit 570 calculates the absolute coordinate position of the infrared light subjected to the projective conversion by the projective conversion unit 540 and transmits the calculated absolute coordinate position to the computer terminal 400. The absolute coordinate position is information represented by a value indicating a distance from a reference point with a predetermined point in the display image as a reference point. For example, the symbol a in FIG. 6 indicates the absolute coordinate position of the cursor in the display image at a certain time. The cursor indicated by symbol a in the display image is at a coordinate position of 100 in the X-axis direction and 150 in the Y-axis direction with the upper left vertex of the display image as a reference point (0, 0), and the absolute coordinate position of the cursor is (100, 150). The unit of the numerical value of the coordinate position is not particularly limited as long as it indicates a distance, but is a pixel, for example. Reference symbol a ′ in FIG. 7 indicates the absolute coordinate position of the cursor in the display image of the words that have passed for a certain period of time from the time in FIG. Here, the absolute coordinate position of the cursor is 200 (150, 150) in the X-axis direction and 150 in the Y-axis direction from the reference point, and is represented as (200, 150).

The computer terminal 400 includes a display unit 410, a display control unit 420, a mouse 430, an absolute coordinate input unit 440, an absolute coordinate input unit 440, and a cursor control unit 450, and an OS (Operating System) or the like. Each unit is operated by an operation command from the control unit.
The display unit 410 is a display that outputs an image, and is, for example, an LCD (Liquid Crystal Display). Here, it is assumed that the resolution of the image output from the display unit 410 matches the resolution of the image output from the display image output unit 510 of the electronic information board control apparatus 500. For example, XGA (eXtended Graphics Array) which is a standard resolution can be applied as the resolution.
The display control unit 420 causes the display unit 410 to output image information input from the OS or the like. For example, the display control unit 420 causes the display unit 410 to output a presentation image stored in advance in the storage area of the computer terminal 400 as a display image.

The mouse 430 is a pointing device that outputs a two-dimensional movement distance corresponding to a user operation to the cursor control unit 450.
The absolute coordinate input unit 440 receives the absolute coordinate position of the infrared light calculated by the absolute coordinate calculation unit 570 and inputs it to the cursor control unit 450 as the absolute coordinate position of the cursor. Here, when the electronic information board controller 500 is connected to the USB port of the computer terminal 400, the absolute coordinate input unit 440 is installed by a driver based on the OS standard HID (Human Interface Devices) specification.

  The cursor control unit 450 moves the cursor displayed on the display control unit 420 according to the movement amount output from the mouse 430. The cursor control unit 450 has a function of receiving a cursor control instruction based on absolute coordinates. When the absolute coordinate position of the cursor is input from the absolute coordinate input unit 440, the cursor control unit 450 causes the display control unit 420 to display the cursor at a coordinate position corresponding to the input absolute coordinate position.

Next, an operation example of cursor control by the electronic information board system 1 will be described with reference to the drawings. FIG. 8 is a flowchart showing an example of cursor control processing performed by the electronic information board system 1.
On the screen area 200, the presentation image received from the computer terminal 400 of the user by the projector 100 is projected. The user moves the light emitting pen 210 while making contact with the screen area 200 (step S1). The camera 300 images the screen area 200 including the infrared light emitted from the light-emitting pen 210 (step S2). The captured image receiving unit 530 of the electronic information board control device 500 receives the captured image captured by the camera 300 (step S3). The infrared light detection unit 531 detects the coordinate position of the infrared light included in the captured image received by the captured image reception unit 530. The projective transformation unit 540 performs projective transformation on the coordinate position of the infrared light detected by the infrared light detection unit 531 and calculates the coordinate position of the display image corresponding to the coordinate position of the infrared light included in the captured image (step). S4).

  The drawing processing unit 550 performs a predetermined drawing process according to the coordinate position calculated by the projective conversion unit 540, and generates a drawing image (step S5). The overlay processing unit 560 performs overlay processing to superimpose the drawing image generated by the drawing processing unit 550 on the image output by the display image output unit 520. The display image output unit 520 outputs the image on which the drawing image is overlaid to the projector 100 (step S6). The projector 100 projects the image transmitted from the display image output unit 520 of the electronic information board control device 500 onto the screen area 200 (step S7).

  On the other hand, the absolute coordinate calculation unit 570 calculates the absolute coordinate position of infrared light based on the coordinate position calculated by the projective conversion unit 540 in step S4 (step S8). The absolute coordinate calculation unit 570 transmits the calculated absolute coordinates to the absolute coordinate input unit 440 (step S9). The cursor control unit 450 displays the cursor at the absolute coordinate position input from the electronic information board control device 500 in the image that the display control unit 420 outputs to the display unit 410 (step S10). The display control unit 420 causes the display unit 410 to display a display image including a cursor at the absolute coordinate position (step S11).

As described above, according to the present embodiment, the coordinate position at which the cursor is displayed in the display image displayed on the display unit 410 of the computer terminal 400 is displayed in the screen area regardless of the setting of the moving speed of the cursor in the computer terminal 400. The coordinate position of the infrared light in the upper projected image can be set to the same coordinate position.
Further, the functions required for the computer terminal 400 are only functions provided in a general computer terminal. That is, since it is not necessary to install a dedicated application on the computer terminal 400 in order to use the electronic information board system 1, the user simply connects his / her computer terminal 400 to the electronic information board control device 500. A presentation using the information board system 1 can be performed.

  It should be noted that a program for realizing the function of the processing unit in the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed to perform cursor control. May be. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible disk, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” is a volatile memory (RAM) inside a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 1 Electronic information board system 100 Projector 200 Screen area 210 Light emitting pen 300 Camera 400 Computer terminal 410 Display part 420 Display control part 430 Mouse 440 Absolute coordinate input part 450 Cursor control part 480 Digital camera 490 Document camera 510 Display image input part 520 Display image Output unit 530 Captured image reception unit 531 Infrared light detection unit 540 Projection conversion unit 550 Drawing processing unit 560 Overlay processing unit 570 Absolute coordinate calculation unit 500 Electronic information board control device

Claims (4)

A projector that projects an input display image onto a screen region, a camera that captures the screen region on which the display image is projected by the projector and generates a captured image, and the screen region and the camera that are used by a user A light emitting pen that emits infrared light, an electronic information board control device that receives the captured image generated by the camera, and a computer terminal that displays the display image on a display unit included in the electronic information board control device. Electronic information board system,
The projector is
Projecting the display image output from the electronic information boat control device;
The camera
Generating the captured image obtained by imaging the infrared light emitted by the light-emitting pen in the screen area on which the display image is projected;
The electronic information board controller is
A display image input unit for reading the display image displayed on the display unit of the computer terminal;
A display image output unit that outputs the display image read by the display image input unit to the projector;
An absolute coordinate calculation unit that calculates an absolute coordinate position of the infrared light included in the captured image generated by the camera and transmits the absolute coordinate position to the computer terminal, and
The computer terminal is
An absolute coordinate position input unit that receives the absolute coordinate position calculated by the electronic information board control device and inputs the received absolute coordinate position to its own cursor control unit;
The cursor control unit for displaying a cursor at the coordinate position indicated by the absolute coordinate position input by the absolute coordinate position input unit in the display image;
An electronic information board system comprising: The electronic information board control device and the computer terminal are connected via a USB device,
The electronic information board system according to claim 1, wherein the absolute coordinate position input unit of the computer terminal is installed by an OS standard HID driver when the USB device is connected to its own USB port. A projector that projects an input display image onto a screen area, a camera that generates a captured image obtained by imaging infrared light emitted by a user's light-emitting pen in the screen area on which the display image is projected, and the camera The electronic information board control device that receives the captured image, and the display image is displayed on a display unit included in the electronic information board control device, and the absolute coordinate position received from the electronic information board control device is the coordinate position indicated in the display image. The electronic information board control device in an electronic information board system comprising a computer terminal for displaying a cursor,
A display image input unit for reading the display image displayed on the display unit of the computer terminal;
A display image output unit that outputs the display image read by the display image input unit to the projector;
An absolute coordinate calculation unit that calculates an absolute coordinate position of the infrared light included in the captured image generated by the camera, and transmits the absolute coordinate position to the computer terminal;
An electronic information board control device comprising: A projector that projects an input display image onto a screen region, a camera that captures the screen region on which the display image is projected by the projector and generates a captured image, and a screen that is used by a user, the screen region and the camera A light emitting pen that emits infrared light, an electronic information board control device that receives the captured image generated by the camera, and a computer terminal that displays the display image on a display unit included in the electronic information board control device. A cursor control method for an electronic information board system,
The projector is
Projecting the display image output from the electronic information boat control device;
The camera is
Generating the captured image obtained by imaging the infrared light emitted by the light-emitting pen in the screen area on which the display image is projected;
Of the electronic information board control device,
A display image input unit reading the display image displayed on the display unit of the computer terminal;
A display image output unit outputting the display image read by the display image input unit to the projector;
An absolute coordinate calculation unit calculating an absolute coordinate position of the infrared light included in the captured image generated by the camera, and transmitting the absolute coordinate position to the computer terminal;
Of the computer terminal,
An absolute coordinate position input unit receives the absolute coordinate position calculated by the electronic information board control device, and inputs the received absolute coordinate position to its own cursor control unit;
The cursor control unit displaying a cursor at a coordinate position indicated by the absolute coordinate position input by the absolute coordinate position input unit in the display image;
A cursor control method comprising:

Images