JP2010152717A - Image processor, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve such a problem that a special mechanism is required on a laser pointer side for distinguishing laser beams from a plurality of pointer operators and the configuration of the laser pointer becomes complex in a position indicating system using the laser pointer and a projector. <P>SOLUTION: By tracing the trajectory of a screen side light bright point 30 formed on a screen 700 by a light beam radiated from a laser pointer 100, the association between the screen side bright point 30 and pointer operator identification information is maintained, and an operator identification image created by the pointer operator information is displayed on the screen 700. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus or the like that displays an image on a screen by an image display device and points a position on the screen using a pointer device that irradiates light to the screen.

  In recent years, presentations and conferences are often performed while projecting an image for presentation onto a large screen using an electronic projector. In such a case, the laser pointer is often used to point to a part of the image, and a plurality of instructors often use the laser pointer to point to different parts at the same time.

  In such a case, it is often performed to change the light emission color of the laser pointer for each instructor so that the locations indicated by the plurality of instructors can be distinguished. However, the spot diameter of the screen-side bright spot formed by reflecting the light beam irradiated from the laser pointer on the screen is generally small, and it is difficult to discriminate only by the color. Laser pointers that emit laser light other than red are relatively expensive, and there are few types of colors that can be emitted.

  In addition to changing the emission color, it is often possible to distinguish the locations indicated by a plurality of instructors by changing the irradiation pattern figure of the laser pointer for each instructor such as a straight line or a circle. However, the laser pointer is relatively expensive because it requires a mechanism for changing the irradiation pattern figure by vibrating the laser emission part and moving the screen side bright spot at high speed. Further, it is difficult to point a fine portion in the image with a laser pointer.

  In order to solve such a problem, a technique for displaying an instruction image such as an arrow superimposed on a presentation image in the vicinity of a screen-side bright spot and displaying different instruction images when there are a plurality of instructors is disclosed in Patent Literature 1 is disclosed.

Japanese Patent Application Laid-Open No. 2002-244813

  In Patent Document 1, in order to distinguish laser beams from a plurality of instructors, it is necessary for each pointer device possessed by each instructor to irradiate laser beams at different modulation frequencies. For this reason, the pointer device requires a mechanism for changing the modulation frequency of the laser beam, and there is a problem that the configuration of the pointer device is complicated. The present invention solves such problems and provides an image processing apparatus and the like that can easily or easily distinguish laser beams from a plurality of instructors.

The image processing apparatus of the present invention is an image processing apparatus for displaying an image on a screen and pointing a position using a pointer means for irradiating a light beam to the screen. Screen-side bright spot measuring means for measuring the position and light emission time of the screen-side bright spot formed on the screen by the light from the pointer means, and means for distinguishing the screen-side bright spot corresponding to the pointer means. It is characterized by having.
The image processing method of the present invention is an image processing method for displaying an image on a screen, and displaying an image or an image on a screen in an image processing method for displaying an image on a screen and pointing a position using a pointer unit that irradiates a light beam to the screen, A screen-side bright spot measuring step for measuring a position of a screen-side bright spot formed on the screen by a light beam from the pointer means and a light emission time, and a step of distinguishing the screen-side bright spot corresponding to the pointer means. It is characterized by having.
The program of the present invention is a program that causes each step of the above-described method to be executed by a computer.

  According to the present invention, the pointer device only needs to have a simple laser light emitting unit and a button that can be turned on / off, and even with such a simple pointer device, a point indicated by a plurality of instructors can be distinguished. An image processing apparatus can be provided.

Hereinafter, embodiments of the present invention will be described.
(First embodiment)
FIG. 1 shows a system configuration according to the first embodiment.

  As shown in FIG. 1, the present embodiment includes an image display device 400 with a screen-side bright spot measuring device 500 and a control device 200 connected to the image display device 400 by a communication line 300. 600 and a photographing apparatus 900 connected to this by a communication line 800 are used. Note that the screen 700 displays images and images projected from the image display device 400. The pointer operator 10 may operate the pointer device 100 to irradiate the screen 700 with a laser beam and indicate a part of the presentation image by the screen-side bright spots 30 formed by reflection on the screen 700. Is possible. Further, by incorporating the function of the control device 200 into the image display device 400, an integrated image processing device may be used. In this case, since the function of the communication line 300 can be regarded as being replaced by wiring inside the image processing apparatus, detailed description is omitted.

  FIG. 3 shows a detailed configuration diagram of each apparatus according to the present embodiment.

  The pointer device 100 includes a battery unit 101, a UI unit 102, a laser light emitting unit 103, and a light beam irradiation unit 104, and the laser light emitting unit 103 is turned on / off by using a button or the like of the UI unit 102. Controls the irradiation of laser light. Further, a pointer-side bright spot is formed in the light beam irradiation unit 104 by the light scattered when the laser beam is irradiated from the light beam irradiation unit 104.

  The imaging apparatus 900 includes an imaging unit 901 and an external IF 902, and the external IF 902 is connected to the communication line 800. Video data photographed by the photographing unit 901 is encoded and transmitted to the control device 200 through the communication line 800. Inside the photographing apparatus 900 is a program storage area 905. Specifically, these storage areas can be realized using a hard disk or a flash memory, but it goes without saying that the present invention does not depend on a specific storage medium. The photographing apparatus 900 includes a CPU 903 and a RAM 904, and the CPU 903 interprets and executes a program read from the program storage area 905 to the RAM 904, thereby performing various controls and calculations in the apparatus, video data encoding, and transmission processing. Is possible.

  The control device 200 is connected to a UI device 201 that is a user interface. Specifically, a light emitting diode and a liquid crystal panel for displaying the state of the apparatus and processing contents, a remote controller for giving an operation instruction to the apparatus, and various buttons are connected. Moreover, it has the display part 203 and displays various video data and character data. Further, the external IF 208 is connected to the communication line 800. An imaging device 900 is connected to the communication line 800. The video data sent from the communication line 800 is decoded and displayed on the display unit 203 and simultaneously sent from the external IF 206 to the image display device 400 through the communication line 300. Examples of implementation of the communication line 300 include DVI (Digital Visual Interface) and HDMI (High-Definition Multimedia Interface), but it goes without saying that the present invention does not depend on a specific connection method. Similarly, examples of realizing the communication line 800 include a wired LAN (Local Area Network) and a wireless LAN, but it goes without saying that the present invention does not depend on a specific connection method.

  Further, a program storage area 207 exists in the control device 200. Specifically, these storage areas can be realized using a hard disk or a flash memory, but it goes without saying that the present invention does not depend on a specific storage medium. The control device 200 includes a CPU 204 and a RAM 205, and the CPU 204 interprets and executes a program read from the program storage area 207 to the RAM 205, thereby performing various controls and calculations in the device, decoding of video data, and transmission / reception processing. Is possible.

  The image display apparatus 400 includes a projection unit 401 and an external IF 402, and the external IF 402 is connected to the communication line 300. Video data input from the communication line 300 through the external IF 402 is sent to the projection unit 401 and projected onto the screen 700. Further, it has an image calibration photographing unit 406 that is used for auto-focusing, automatic wall color correction, and the like of the projected image, and can be controlled by the CPU 403. In this embodiment, the image calibration photographing unit 406 is used as a screen-side bright spot measuring device. Specifically, the position of the screen-side bright spot and the light emission time are measured by analyzing the image of the image captured by the image calibration imaging unit 406. Inside the image display device 400, a program storage area 405 exists. Specifically, these storage areas can be realized using a hard disk or a flash memory, but it goes without saying that the present invention does not depend on a specific storage medium.

  Further, the image display device 400 includes a CPU 403 and a RAM 404. By the CPU 403 interpreting and executing a program read from the program storage area 405 to the RAM 404, various control and calculation in the device and reception processing of video data are possible. It is.

  FIG. 5 shows a view of the system configuration according to the present embodiment as seen from above. Although FIG. 5 shows a case where there are two pointer operators, it goes without saying that the present invention does not depend on a specific number of people.

  In the present embodiment, as shown in FIG. 12A, the screen side bright spot is associated with the pointer operator information by comparing the light emission timings of the pointer side bright spot 40 and the screen side bright spot 30. Then, as shown in FIG. 12B, an operator identification image created based on the pointer operator information is displayed in the vicinity of the screen-side bright spot, so that the operator can be distinguished.

  FIG. 7 shows a display sequence of the operator identification image according to the present embodiment. When the image capturing apparatus 900 measures the position of the pointer-side bright spot and the light emission time in Sp100, it notifies the control apparatus 200 of the position of the bright spot and the light emission time. Similarly, when the screen-side bright spot measuring device 500 measures the position of the screen-side bright spot and the light emission time in Ss100, the screen side bright spot measuring apparatus 500 notifies the controller 200 of the position of the bright spot and the light emission time. Here, the position of the bright spot and the measurement of the light emission time are performed by the photographing apparatus 900 or the screen side bright spot measuring apparatus 500. However, the photographed image is transmitted as it is to the control apparatus 200, and the control apparatus 200 transmits the position of the bright spot. Measurement of the light emission time may be performed.

  Receiving the position of the bright spot and the emission time, the control device 200 associates the screen-side bright spot with the pointer operator information in Sc100. Specifically, the light emission time of the pointer-side bright spot is compared with the light emission time of the screen-side bright spot, and if they match, the pointer operator information is estimated from the position of the pointer-side bright spot. At this time, the pointer device outline including the bright spot position and the pointer operator outline holding the pointer device are extracted, the pointer operator face image is extracted by tracing the pointer operator outline, The pointer operator may be estimated based on the position and shape.

  Further, in Sc200, an operator identification image is generated from the obtained pointer operator information. Specifically, the database is searched using the pointer operator information as a key, and an image representing the name of the pointer operator is generated. Finally, the image display device 400 is notified of the operator identification image generated and the position of the screen-side bright spot. Upon receiving the notification, the image display apparatus 400 displays the operator identification image superimposed on the presentation image in the vicinity of the screen-side bright spot in Sv100.

  FIG. 8 shows a flowchart describing in detail the processing of Sp100. The photographed image I is an image in which the pointer side bright spot 40 is reflected as shown in FIG.

  All pixels of the captured image I are scanned by the loop processing of Sp101 and Sp102. In Sp103, the luminance value in the pixel at the position (x, y) of the captured image I is compared with a certain threshold value. When the luminance value is larger than the threshold value, it is determined that there is a bright spot at the position (x, y), so (x, y) is added to the set Cp of bright spot positions in the current frame in Sp104. Further, in Sp105, the bright spot position set Bp in the immediately preceding frame is searched to determine whether or not a bright spot exists in the vicinity of (x, y) in the immediately preceding frame. When the bright spot does not exist, it is determined that a new bright spot has appeared at (x, y). Therefore, (x, y) is substituted into the position Pp of the bright spot newly emitted in Sp106, and the emission time The photographing time Tc is substituted for Tp. Finally, in Sp107, Cp, Pp, and Tp are notified to the control device 200.

  FIG. 9 shows a flowchart describing in detail the processing of Ss100. The captured image I is an image in which the screen side bright spot 30 is reflected.

  All pixels of the captured image I are scanned by the loop processing of Ss101 and Ss102. In Ss103, the luminance value in the pixel at the position (x, y) of the captured image I is compared with a certain threshold value. When the luminance value is larger than the threshold value, it is determined that there is a bright spot at the position (x, y). Next, in Ss104, it is determined whether or not there is a bright spot position belonging to the vicinity of the position (x, y) with respect to all the bright spot positions in the immediately preceding frame. When the bright spot position Bs (i) belonging to the vicinity is found, the bright spot position is updated in Ss105. When a bright spot position belonging to the vicinity is not found, it is determined that a new bright spot has appeared at (x, y), and therefore, at the position Ps of the bright spot newly emitted in Ss106, (x, y). Is substituted, and the photographing time Tc is substituted for the light emission time Ts. In Ss107, (x, y) is added to the set Cs of bright spot positions in the current frame. Finally, in Ss108, Cs, Ps, and Ts are notified to the control device 200.

  FIG. 10 shows a flowchart describing in detail the processing of Sc100. In Sc101, the light emission time Tp of the pointer side bright spot is compared with the light emission time Ts of the screen side bright spot. As a result of the comparison, if it is determined that the time is substantially the same, the pointer operator information is estimated from the position of the bright spot on the pointer side in Sc110, and is distinguished from other operators. In this embodiment, pointer operator information is estimated by determining to which operator region 50 the bright spot detection point 45 belongs, as shown in FIG. Specifically, the conditional expression of Sc112 is executed for all individual operator areas by the loop process of Sc111 of FIG. The definition of the operator area may be defined in advance by dividing the shooting area at equal intervals, or may be defined in accordance with the arrangement of conference participants. When it is determined that the pointer-side bright spot belongs to the i-th operator area, the pointer operator information J is a numerical value i that is an index of the operator area as shown in Sc113.

  In addition to using the bright spot detection point 45 as it is, the pointer operator information can be estimated by determining to which operator region 50 the face detection point 15 belongs as shown in FIG. Good. The face detection portion 15 extracts the pointer device outline including the bright spot position and the pointer operator outline holding the pointer device by image analysis, and extracts the pointer operator's face image by tracing the pointer operator outline. Can be detected. Furthermore, pointer operator information may be estimated based on the shape of the extracted face image.

  The association between the screen side bright spot and the pointer operator information is maintained by tracking the trajectory of the screen side bright spot. As a result, even if the screen side bright spot moves, it is possible to continue displaying the operator identification image in the vicinity of the screen side bright spot. In the present embodiment, it is possible to track the locus of the screen-side bright spot by monitoring the set Cs of bright spot positions notified from the screen-side bright spot measuring device. Specifically, if the pointer operator information J is associated with the bright spot position Cs (i) by Sc100, Cs (i) is also obtained when the bright spot position is notified from the screen side bright spot measuring device next time. It represents the updated bright spot position. Therefore, Cs (i) can be used as the position of the screen-side bright spot associated with the pointer operator information J as it is, and the association can be maintained.

  In this embodiment, a visible light laser is used as a light beam emitted from the pointer device. However, an invisible light laser such as an infrared ray may be used. When an invisible laser is used, the screen-side luminescent spot cannot be seen with the naked eye. Therefore, the indication point image 35 shown in FIG. 14 is displayed superimposed on the presentation image at the position of the screen-side luminescent spot. To do.

(Second Embodiment)
In the second embodiment, a configuration obtained by removing the imaging apparatus 900 from the system configuration used in the first embodiment is used. In the present embodiment, as shown in FIG. 13A, a pallet area 25 is provided on the screen 700, and the operator identification image 20 is arranged in the area. The palette area 25 and the operator identification image 20 are associated with pointer operator information. The pointer operator 10 can move the screen-side bright spot 30 by moving the pointer device 100, and the locus can be detected using the screen-side bright spot measuring device 500. In the present embodiment, it is detected that the screen-side bright spot 30 has performed a specific operation on the palette area 25 so as to draw a circle, and the screen-side bright spot is associated with the pointer operator information. In addition to this, a specific operation such as blinking the screen-side bright spot 30 on the palette area 25 may be performed. In FIG. 13A, since the pointer operator information associated with the screen-side bright spot 30 (1) corresponds to the operator identification image 20 (1), the operator identification image 20 from the palette area 25 (1) is displayed. It is possible to drag (1).

  Similarly, the operator identification image 20 (2) can be dragged by moving the screen side bright spot 30 (2). As shown in FIG. 13B, it is possible to instruct a part of the presentation image while the operator identification image 20 is displayed in the vicinity of the screen-side bright spot 30.

(Third embodiment)
In the present embodiment, a configuration is used in which a plurality of image processing apparatuses used in the first embodiment are used and are connected to each other via a network. FIG. 11 shows a configuration in which two image processing apparatuses 600 are connected via a network 1000.

  The image processing apparatus 600 (1) on the left side of FIG. 11 displays the pointing point image 35 (3) at the position on the local screen corresponding to the position of the remote (right) screen side bright spot 30 (3). Superimposed on the display. Further, the image processing apparatus 600 (1) displays the operator identification image 20 (3) corresponding to the remote pointer operator 10 (3) superimposed on the presentation image in the vicinity of the indication point image 35 (3). To do. Similarly, the right image processing apparatus 600 (2) also displays the indication point images 35 (1) and (2) and the operator identification images 20 (1) and (2) superimposed on the presentation image.

(Fourth embodiment)
FIG. 2 shows the system configuration of the fourth embodiment. In the present embodiment, an image display device 450 having a screen-side bright spot measuring device 550 and a control device 200 connected to the image display device 450 by a communication line 300, and an image processing device 650 that communicates with the image processing device 650 are provided. An imaging device 900 connected by a line 800 is used. Note that the screen 700 displays images and images projected from the image display device 450. The pointer operator 10 operates the pointer device 100 to irradiate the screen 700 with a laser beam, and can indicate a part of the presentation image by the screen-side bright spot 30 formed by reflection on the screen 700. It is. Further, by incorporating the function of the control device 200 into the image display device 450, an integrated image processing device may be used. In this case, since the function of the communication line 300 can be regarded as being replaced by wiring inside the image processing apparatus, detailed description is omitted.

  FIG. 4 shows a detailed configuration diagram of each device according to the present embodiment. Since the pointer device 100, the photographing device 900, and the control device 200 have the same configuration as that of the first embodiment, detailed description thereof is omitted.

  The image display device 450 includes a projection unit 451 and an external IF 452, and the external IF 452 is connected to the communication line 300. Video data input from the communication line 300 through the external IF 452 is sent to the projection unit 451 and projected onto the screen 700. In addition, a light detection unit 456 having a plurality of light detectors arranged uniformly on the screen is controllable by the CPU 453. In the present embodiment, the light detection unit 456 is used as a screen side bright spot measuring device. Specifically, the position of the screen-side bright spot and the light emission time are measured by analyzing the light detection signal from the light detection unit 456. Inside the image display device 450 is a program storage area 455. Specifically, these storage areas can be realized using a hard disk or a flash memory, but it goes without saying that the present invention does not depend on a specific storage medium.

  The image display device 450 has a CPU 453 and a RAM 454. By reading and executing the program read from the program storage area 455 into the RAM 454, the CPU 453 can perform various controls and calculations in the device, and receive video data. It is. In the present embodiment, since the image calibration photographing unit 406 in FIG. 3 can be regarded as being replaced with the light detection unit 456 in FIG. 4, detailed description is omitted.

(Fifth embodiment)
In the fifth embodiment, the system configuration used in the first embodiment and the fourth embodiment is used. In the present embodiment, in Sc200 of FIG. 7, as the operator identification image generated from the obtained pointer operator information, an image representing any number, character, or symbol corresponding to the position of the pointer operator is generated. . Specifically, when the pointer operator information is an index of the operator area, as shown in FIG. 14A, 1, 2, 3,. . . An image representing the number is generated. Similarly, as shown in FIG. 14B, A, B, C,. . . Or an image representing a symbol such as a right arrow or a left arrow as shown in FIG.

(Sixth embodiment)
In the sixth embodiment, the system configuration used in the first embodiment and the fourth embodiment is used. In the present embodiment, in Sc200 of FIG. 7, a face image of the pointer operator is generated as the operator identification image generated from the obtained pointer operator information. Specifically, the database is searched using the pointer operator information as a key, and an image in which the face image of the pointer operator is inserted into the balloon frame as shown in FIG. 15B is generated. The face image may be an image deformed like an illustration. Alternatively, an image in which the face image obtained from the face detection portion 15 in FIG. 6B is directly inserted into the blowing frame may be generated.

  Of course, the present invention is also realized by a program installed in a computer.

It is a system configuration figure concerning a 1st embodiment of the present invention. It is a system configuration figure concerning a 4th embodiment of the present invention. It is a detailed block diagram of each apparatus which concerns on the 1st Embodiment of this invention. It is a detailed block diagram of each apparatus which concerns on the 4th Embodiment of this invention. It is the figure which looked down at the system configuration concerning the 1st embodiment of the present invention from the upper part. It is the figure which showed an example of the image image | photographed with the imaging device which concerns on the 1st Embodiment of this invention. It is a sequence diagram of a process of displaying an operator identification image in the first embodiment of the present invention. It is a flowchart of the process which measures the position of a point side luminescent spot, and the light emission time based on the 1st Embodiment of this invention. It is a flowchart of the process which measures the position and light emission time of a screen side luminescent spot based on the 1st Embodiment of this invention. It is a flowchart of the process which matches the screen side luminescent point and point operator identification information based on the 1st Embodiment of this invention. It is a system configuration figure concerning a 3rd embodiment of the present invention. It is a figure for demonstrating an example for comparing the timing of light emission with the point side luminescent point and the screen side luminescent point based on the 1st Embodiment of this invention. It is a figure for demonstrating the 2nd Embodiment of this invention, and is the figure which showed the example dragged above operator identification from a pallet area | region. It is a figure for demonstrating the 5th Embodiment of this invention, and is the figure which showed the example in case operator identification information is a number or a character. It is a figure for demonstrating the 5th, 6th embodiment of this invention, and is the figure which showed the example in case operator identification information is a symbol and a face image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Pointer apparatus 200 Control apparatus 300 Communication line 400,450 Image display apparatus 500,550 Screen side bright spot measuring apparatus 600,650 Image processing apparatus 700 Screen 800 Communication line 900 Imaging apparatus

Claims (6)

In an image processing apparatus for displaying an image on a screen and pointing a position using a pointer means for irradiating a light beam to the screen,
Image display means for displaying video and images on the screen;
Screen-side bright spot measuring means for measuring the position of the screen-side bright spot formed on the screen by the light from the pointer means and the light emission time;
An image processing apparatus comprising: means for distinguishing a screen-side bright spot corresponding to the pointer means.   The means for distinguishing the screen-side bright spot corresponding to the pointer means is a means for distinguishing by estimating pointer operator information, which is information for distinguishing the pointer operator, and associating it with the screen-side bright spot. The image processing apparatus according to claim 1.   The image processing apparatus according to claim 1, wherein an association between the screen side bright spot and the pointer operator information is maintained by tracking a trajectory of the screen side bright spot.   The image processing apparatus according to claim 1, wherein a display capable of distinguishing a screen-side bright spot corresponding to the pointer unit is performed on the screen. In an image processing method for displaying an image on a screen and pointing a position using a pointer means for irradiating a light beam to the screen,
An image display process for displaying video and images on the screen;
A screen side bright spot measuring step for measuring the position and light emission time of the screen side bright spot formed on the screen by the light from the pointer means;
And a step of distinguishing screen-side bright spots corresponding to the pointer means.   A program that causes a computer to execute each step of the method according to claim 5.

Images