JP2012014361A - Information processor and control method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To identify a corresponding relationship between a position on a screen to which respective beams are irradiated and a pointing device irradiating beams to the position.SOLUTION: An information processor includes: first detection means 106 which detects a position 203 on a screen to which beams are irradiated; second detection means 107 which detects a position and an angle of a pointing device 202 existing in a predetermined space; and identifying means which identifies a pointing device irradiating beams to the position detected by the first detection means.

Description

  The present invention relates to a technique for specifying a point pointed by a pointing device such as a laser pointer.

  2. Description of the Related Art Conventionally, there is known a technique for detecting a position irradiated with a light beam in a system that irradiates a light beam on a predetermined screen with a laser pointer. Such an apparatus is generally irradiated with a light beam by one operator, and therefore it is not necessary to specify which laser pointer emits which light beam.

  Normally, when the distance from the screen to the laser pointer is far, the point irradiated with the light beam moves greatly by moving the laser pointer a little. On the other hand, when the distance from the screen to the laser pointer is short, the point irradiated with the light beam hardly moves even if the laser pointer is moved a little.

Japanese Patent Laid-Open No. 9-80372

  For example, in a scene where a plurality of operators simultaneously irradiate laser pointers, a light beam will be irradiated by a plurality of laser pointers on the screen. In such a case, it is difficult to determine which operator has irradiated each light beam. If an additional image suitable for each operator is to be displayed for each irradiation position, the device that performs the display itself appropriately sets the relationship between the irradiation position and the operator. Need to know.

  The present invention has been made in view of the above points, and has as its main object to specify the correspondence between the position where each light beam is irradiated on the screen and the pointing device that irradiated the light beam at that position. .

  In order to achieve the above object, according to the information processing apparatus of the present invention, the first detection means for detecting the position on the screen irradiated with the light beam, and the position and angle of the pointing device existing in the predetermined space are detected. And a second detection unit, and a specifying unit that specifies a pointing device that irradiates the position detected by the first detection unit with a light beam based on the position and the angle detected by the second detection unit. And

  ADVANTAGE OF THE INVENTION According to this invention, the correspondence of the position where each light ray was irradiated on the screen and the pointing device which irradiated the light ray to the position can be specified.

The figure which shows the hardware constitutions of information processing apparatus External view when the information processing device is installed indoors The figure which shows the function structure of information processing apparatus Flow chart showing operation procedure of information processing apparatus The figure which shows the appearance of the additional image which is displayed on the screen The figure which shows a mode that the light ray was irradiated from the some pointing device Diagram explaining additional image (guide display)

[Embodiment 1]
FIG. 1 is a diagram illustrating a hardware configuration of an information processing apparatus applied to the present embodiment. A CPU (central processing unit) 101 executes a program corresponding to each function shown in FIG. 3 and an operation procedure shown in FIG. The ROM 102 stores a program code corresponding to the above program. The RAM 103 is used as a work area when executing the program. The storage device 104 is a device that stores various data used in the present embodiment, and corresponds to, for example, a hard disk. The display unit 105 displays an image on the screen. For example, a liquid crystal display that displays an image represented by input image data on a screen, a projector that projects the image on an external object surface, or bitmap data represented by the image is transmitted to an external display device. This corresponds to a display control device to be displayed. The screen photographing camera 106 is a camera that photographs the screen on which the image is displayed by the display unit 105. The camera 106 detects the position on the screen irradiated with light by a pointing device described later in cooperation with the CPU. The pointing device photographing camera 107 is a camera capable of photographing various images existing in a predetermined space that can be photographed by the camera. The camera 107 detects the position and angle of the pointing device existing in the predetermined space in cooperation with the CPU. Note that the information processing apparatus 100 described above may be used as the hardware configuration described above incorporated in one casing, but is used as a system that cooperates by separating into a plurality of casings. May be.

  FIG. 2 is an external view when the information processing apparatus 100 applied to the present embodiment is installed indoors. In this figure, the same components as those described in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted. In this example, it is assumed that the information processing apparatus 100 is installed above the operator's head. The display unit 105 functions as a projector that projects an image. An image projected from the display unit 105 is displayed on the screen 201. The screen photographing camera 106 photographs the room so that the screen 201 is at least included in the photographing range. The pointing device photographing camera 107 photographs a space in which a pointing device 202 described later may exist. The operator 200 holds a pointing device 202, which is represented by a laser pointer or the like and points on the screen by irradiating a predetermined light beam. A predetermined light beam emitted from the pointing device 202 is irradiated on the screen 201. Then, a point 203 (pointer bright spot) is formed on the screen 201 that reflects with a much higher luminance than other areas. The screen photographing camera 106 and the CPU 101 cooperate to detect that the pointer bright spot 203 is on the screen 201. The pointing device photographing camera 107 and the CPU 101 cooperate to detect the position and angle of the pointing device 202.

  FIG. 3 is a diagram illustrating a functional configuration of the information processing apparatus. As described above, each function shown in FIG. 3 is realized by the cooperative processing of each unit centering on the CPU 101 in FIG. However, the present invention is not limited to this, and the present invention may be applied to an apparatus or system that includes a processing unit dedicated to each function described below as a hardware configuration and executes each function independently. The image output unit 301 is the same as the display unit 105 of FIG. 1 described above, and is a processing unit that displays an image on the screen 201 shown in FIG. The image input unit 302a corresponds to the camera 106 described above, and the image input unit 302b corresponds to the camera 107 described above. The pointer bright spot position detection unit 303 is a processing unit that detects the position of the pointer bright spot 203 shown in FIG. 2, that is, the coordinates on the screen 201, based on the image input from the image input unit 302a. This detection function may be provided as a built-in function of the image input unit 302a (camera 106), or the image input unit 302a may be executed in cooperation with the CPU 101. The pointing device position detection unit 304 is a processing unit that detects the position of the pointing device based on the image input from the image input unit 302b. The pointing device angle detection unit 305 is a processing unit that detects the angle of the pointing device based on the image input from the image input unit 302b. This angle detection method is not particularly limited as long as it can determine the correspondence between a plurality of pointer bright spots and the pointing device in the subsequent processing. For example, it is detected how many times it is tilted to the left or right with respect to the angle at which the operator irradiates the pointing device (laser pointer) toward the screen facing the screen 201 shown in FIG. It ’s fine. Note that the two functions of the pointing device position detection unit 304 and the pointing device angle detection unit 305 may be regarded as one function as an analysis of the state of the pointing device based on an image input from the same input unit. It will be possible. The pointer identifying unit 306 irradiates each pointer luminescent spot and a light beam based on the position (coordinates) of the pointer luminescent spot detected by the detecting unit 303 and the position and angle of the pointing device detected by the detecting units 304 and 305. The correspondence relationship with the pointing device is specified. The size setting unit 307 obtains the distance between each pointer bright spot and the corresponding pointing device based on the identified correspondence. This distance can be obtained from the position of the pointer bright spot detected by the pointer bright spot position detection unit 303 and the position of the pointing device detected by the pointing device position detection unit 304. Then, the size setting unit 307 generates an additional image having a size based on the distance. The size is set so as to increase as the distance increases. Further, the size setting unit 307 controls the display of the image by the image output unit 301 so that the additional image whose size has been adjusted is displayed superimposed on the position of the corresponding pointer bright spot 203 or in the vicinity of the bright spot. The instruction determination unit 308 determines whether or not there is an instruction using the pointing device 202 by the operator 200 for the additional image generated by the size setting unit 307 and displayed via the image output unit 301. This determination is performed based on the result of the pointer bright spot position detection unit 303 detecting the relative movement of the pointer bright spot 203 with respect to the additional image displayed on the screen 201. That is, the above instruction is given when the operator 200 moves the pointing device 202 and the pointer bright spot 203 moves accordingly. Therefore, the instruction determination unit 308 periodically receives information indicating the position of the pointer bright spot 203 from the pointer bright spot position detection unit 303 and recognizes the change in the position. The instruction determination unit 308 also periodically receives information indicating what additional image is displayed from the size setting unit 307 or the image output unit 301. In addition, the instruction determination unit 308 detects the correlation between the position of the additional image being displayed and the position of the pointer bright spot 203 in real time. If these correlations satisfy a predetermined condition, it is determined that the operator 200 has input a predetermined instruction command. The instruction execution unit 309 executes an operation corresponding to the instruction command in response to the input of the instruction command.

  FIG. 4 is a flowchart showing an operation procedure of the information processing apparatus 100. First, in step S401, images captured by the cameras 106 and 107 are input by the image input units 302a and 302b. In step S402, the pointer bright spot position detection unit 303 detects whether or not there is a pointer bright spot based on the captured image input by the image input unit 302a. As a result, if a pointer bright spot is detected, the process proceeds to step S403. If a pointer bright spot is not detected, the process returns to step S401. Note that the pointer bright spot detection by the pointer bright spot position detection unit 303 is performed by recognizing that a specific brightness, color, and shape are present in the captured image. For example, when the pointing device 202 is assumed to be a laser pointer that irradiates a small round area using a green light beam, the pointer bright spot position detection unit 303 reads “lightness above a predetermined threshold, green, round It is only necessary to detect the image portion of “shape”. Here, it is assumed that two or more pointer luminescent spots are detected, and when there are a plurality of pointer luminescent spot candidates, the subsequent processes are performed using each as a pointer luminescent spot. In step S403, the position (coordinates) of the pointer bright spot detected in the above step is detected. If there are a plurality of pointer bright spots, each position is detected and information indicating the position is held. In step S <b> 404, the pointing device position detection unit 304 detects the position of the pointing device that exists in a space where the camera 107 can shoot. Subsequently, in step S405, the pointing device angle detection unit 305 detects the angle at which the pointing device is present. Since this angle is necessary to know in which direction the pointing device is irradiating the light beam, it is detected which direction the light beam of the pointing device is irradiated. Here, it is assumed that two or more pointing devices are detected, and information indicating the position and angle of each pointing device is held as a set. In step S406, the pointer identification unit 306 identifies which pointer bright spot is formed by the light beam emitted from which pointing device, and the corresponding relationship. This specification is performed based on the pointer bright spot detected in step S402 and the position and angle of the pointing device detected in steps S404 and S405. For example, if one pointer bright spot is detected and one set of pointing device position and angle is detected, they will be associated. On the other hand, if two pointer luminescent spots are detected and two pairs of pointing device positions and angles are detected, a pointing device suitable for each pointer luminescent spot is obtained by analyzing the position and angle pairs. It is associated. In step S407, the size setting unit 307 calculates the distance from each pointer bright spot to the corresponding pointing device based on the identified correspondence. A further specific example of this distance corresponds to the distance from the center of gravity of the pointer bright spot to the tip of the pointing device that emits the light beam. Further, the size setting unit 307 sets the size of the pointer bright spot corresponding to the distance or the size of the additional image to be displayed around the bright spot according to the distance. An example of the additional image will be described later. In step S <b> 408, based on the command from the size setting unit 307, the additional image with the size set is displayed from the image output unit 301. This step S408 may be applied to an example in which some kind of additional image is simply displayed. On the other hand, as shown in FIG. 5 to be described later, after further determining whether or not the additional image should be displayed, the additional image may be displayed as necessary.

  FIG. 5 shows a case where the pointing device (laser pointer) emits a light beam and displays an additional image while an object such as an image or a document is being displayed on the screen 201 in the stage of step S408. It is a figure which shows an example. In this case, in step S402, it is necessary to determine not only “whether there is a pointer bright spot” but also “whether the pointer bright spot 203 is pointing to an object such as an image or a document”. Let's go. Then, in step S402, if “a pointer bright spot exists and the pointer bright spot 203 is in a state pointing to an object such as an image or a document”, the process may proceed to step S403. In FIG. 5, when an object such as an image or a document is displayed, the additional image is displayed on the condition that the object is pointed by the pointing device. Therefore, even if the screen 201 on which no object is displayed is pointed by the pointing device, steps S401 and S402 are repeated. FIGS. 5A and 5C show examples in the case where the distance from the pointer bright spot to the corresponding pointing device is short. FIGS. 5B and 5D show an example in which the distance from the pointer bright spot to the corresponding pointing device is long. FIGS. 5A and 5B show the situation when the operator 200, the screen 201, and the like are photographed from the upper part of the room shown in FIG. 5C and 5D show the state of the screen 201 that can be seen by the operator when the operator is facing the screen 201. In these drawings, reference numeral 501 indicates the tip of the pointing device that emits the light beam, and reference numeral 502 indicates the position of the pointer bright spot. In the screen 201 in FIGS. 5C and 5D, an object 503 such as an image or a document is displayed in advance. Furthermore, in the present embodiment, an additional image 504 is displayed as a guide display so that the operator 200 can perform various operations using the pointing device. The additional image 504 (guide display) applied in the present embodiment is configured with an image as shown in FIG. This additional image 504 is provided with sides radially from its center in eight directions, and is provided with a round image portion at each tip portion of these eight sides. Actually, as shown in FIGS. 5C and 5D, the additional image 504 is displayed so as to overlap the object 503. Here, as can be seen by comparing FIG. 5C and FIG. 5D, “from the pointer bright spot to the corresponding pointing device rather than“ when the distance from the pointer bright spot to the corresponding pointing device is short ”. In the case where the distance of “is far”, the length of each of the eight sides is longer. That is, the distance from the central part of the additional image 504 to the round image part which is a front-end | tip part is long. One reason for this is to make it easy to see the additional image 504 even when viewed from a remote operator 200 as shown in FIG. Another reason is to avoid an error in which an instruction contrary to the intention of the operator 200 is given because the remote pointing device moves from the center to the end of the additional image 504 with a slight movement. It is to do. Next, an example of an instruction from the operator by moving the pointer bright spot 203 will be briefly described with reference to FIGS. First, it is assumed that the pointing device is operated by the operator 200 so that the object 503 is irradiated with the pointer bright spot 203. Accordingly, on the screen 201, an additional image 504 as shown in the figure centered on the irradiated object 503 is displayed as an operation guide. In the present embodiment, it is assumed that the operator 200 gives an instruction by moving the pointer bright spot 203 in any of eight directions from this state. Specifically, when the pointer bright spot 203 enters any one of the areas A to H shown in FIG. 7B and subsequently returns to the object 503, an instruction to perform a predetermined process is generated. Judge. For example, when the pointer bright spot 203 once enters the region A and then returns to the object 503, it is determined that an instruction “perform processing A on the object 503” has occurred. Further, when the pointer bright spot 203 once enters the region B and then returns to the object 503, it is determined that an instruction “perform processing B on the object 503” has occurred. The other areas C to H are also determined to have been instructed for processes C to H corresponding to the area in which the pointer bright spot 203 is entered. The process A is, for example, “temporary storage (copy) of an object”, and the process B is “pasting an object temporarily stored”. Further, the process C is “deletion of object”, and the process D is “cut object (deletion and temporary storage of displayed object)”.

  In step S409, the instruction determination unit 308 determines whether an instruction for the object 503 described with reference to FIG. This is realized by detecting that the pointer bright spot 203 returns to the object 503 after moving to any of the areas A to H. If there is an instruction here, the process proceeds to step S410. In step S410, the instruction determination unit 308 determines whether the instruction is processing A to H based on which of the areas A to H illustrated in FIG. . In step S <b> 411, the instruction execution unit 309 executes the process in which the determination is made as a process for the object 503.

  FIG. 6 is a display example of the additional image 504 described above “when two pointer bright spots are detected and two sets of the pointing device position and angle are detected”. As described above, each of the pointer bright spots 603 and 604 corresponds to the light beam emitted from each pointing device by analyzing the pair of position and angle of each pointing device. I know if Therefore, as shown in FIG. 6, even if the light beams emitted from a plurality of operators cross before reaching the screen 201, the information processing apparatus 100 can identify the bright spot corresponding to each operator. . Therefore, according to the operation procedure of each step of FIG. 4 described above, additional images of appropriate sizes (here, 605 and 606, respectively) can be displayed on the pointer bright spots 603 and 604, respectively. In the description of FIG. 4 described above, one pointer bright spot has been described for the processing in steps S409 to S411 for the sake of simplicity. However, when N pointer bright spots (additional images) are to be displayed, steps S409 to S411 are executed in parallel in N steps.

  According to the present embodiment, it is possible to accurately identify the correspondence between the position where each light beam is irradiated on the screen and the pointing device that irradiated the light beam at that position. Thereby, for example, as shown in FIG. 6, even when the light beams are irradiated from a plurality of operators so as to cross each other on the way, the respective pointer bright spots can be associated without misjudgment. Therefore, the size of the additional image shown as a guide display to each operator is also appropriate for each.

[Modification]
In the above embodiment, the pointing device position detection unit 304 detects a pointing device from an image captured by the camera 107, but the present invention is not limited to this. For example, a sensor that irradiates infrared rays and detects the reflected light may be used, and the shape, position, angle, and the like of the pointing device may be acquired based on distance information acquired by the sensor.

  In addition, the cameras 106 and 107 are not separate bodies, and a single camera having both the screen 201 and the pointing device 202 as an imaging range may be applied.

  The additional image described in the above embodiment is an example, and the present invention can be applied to another additional image. For example, instead of an additional image consisting of 8 sides, an additional image of 4 sides or 16 sides based on the same concept may be used as the guide display. Furthermore, it is applicable as long as it is an additional image to be displayed around the pointer bright spot instead of the guide display for the instruction by the pointer bright spot.

  Each embodiment described above is also realized by executing the following processing. That is, a process of supplying software (computer program) that realizes each process and function of the above-described embodiment to the system via a network or a storage medium, and the computer (or CPU or the like) of the system reads and executes the program. It is. The computer program and a computer-readable storage medium storing the computer program are also included in the scope of the present invention.

Claims (7)

First detection means for detecting a position on the screen irradiated with the light beam;
Second detection means for detecting the position and angle of the pointing device existing in the predetermined space;
An information processing apparatus comprising: a specifying unit that specifies a pointing device that irradiates a light beam at a position detected by the first detection unit based on a position and an angle detected by the second detection unit. When the first detection means detects a plurality of positions as positions on the screen irradiated with light, and the second detection means detects the positions and angles of a plurality of pointing devices existing in a predetermined space,
The specifying unit specifies a position detected by the first detecting unit corresponding to a position and an angle represented by each set based on a set of a plurality of positions and angles detected by the second detecting unit. The information processing apparatus according to claim 1, wherein a pointing device that irradiates light to each of a plurality of positions detected by the first detection unit is specified.   The information processing apparatus according to claim 1, further comprising display control means for displaying an additional image at a position on the screen detected by the first detection means. And a size setting means for setting the size of the additional image according to a distance from a position detected by the first detection means to a position of a pointing device that irradiates the position with light.
The information processing apparatus according to claim 3, wherein the display control unit displays an additional image whose size is set by the size setting unit on the screen.   The computer program which makes the said computer function as a display control apparatus of any one of Claim 1 thru | or 4 by being read and executed by a computer.   A computer-readable storage medium storing the computer program according to claim 5. A method for controlling an information processing apparatus including first detection means, second detection means, and identification means,
A first detection step of detecting a position on the screen irradiated with light by the first detection means;
A second detection step of detecting the position and angle of the pointing device existing in the predetermined space by the second detection means;
A specifying step of specifying a pointing device that irradiates light at the position detected in the first detection step based on the position and angle detected in the second detection step by the specifying means. Control method to do.

Images