JP2011028629A - Presentation system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a presentation system for superimposing a drawing image drawn by a pointing stick on a basic image, and to provide an imaging apparatus included in the presentation system. <P>SOLUTION: The presentation system 10 including a projection type image display device 40 for projecting image light to a projection surface, and an imaging apparatus 60 for imaging the projection surface also includes: a pointer image acquisition part 130 for acquiring a pointer image composed of infrared light; a specification part 140 for specifying a position pointed by the infrared light on the basis of the pointer image; and an image control part 160 for controlling the image light on the basis of the position pointed. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a presentation system for specifying an indicated position indicated by a pointer light in an image displayed on a projection surface.

  2. Description of the Related Art Conventionally, a projection display apparatus that projects image light emitted from a projection lens unit onto a projection surface is known. A case where a projection display apparatus is used in a presentation or a meeting is conceivable. In a presentation or a meeting, a user generally performs an explanation while instructing an image displayed on a projection surface with an instruction bar or the like.

  Here, it has been proposed to improve the convenience of the user by detecting an indication position indicated by an indication stick or the like in an image displayed on the projection surface. For example, applications that use the designated position include (a) an application that displays a cursor image at the designated position, and (b) an application that draws a line or a curve along the transition of the designated position.

As a technique for detecting the pointing position, there has been proposed a pointing position detection system that uses the shadow of the pointing bar projected on the projection surface when the image light emitted from the projection lens unit is blocked by the pointing bar. (For example, patent document 1).
JP 2001-222375 A

  However, in a system that uses the shadow of the indicator bar projected on the projection surface, the position indicated by the indicator bar and the position of the indicator bar are not provided unless the projection display device is placed facing the screen. Deviation occurs in the positional relationship with the shadow. That is, when using the application as described in (b) above, the drawn video is shifted from the position designated by the user, and the convenience for the user cannot be sufficiently improved.

  Accordingly, the present invention has been made to solve the above-described problems, and an object of the present invention is to provide a presentation system and an imaging apparatus that can superimpose a drawn video drawn with a pointing stick on a basic video. And

  In order to solve the above-described problems, the present invention has the following features. First, in one aspect of the present invention, a presentation system (presentation system 10) includes a projection display apparatus (projection display apparatus 40) that projects image light on a projection plane (screen 30), and the projection plane. And an imaging device (imaging device 60) for imaging. This presentation system includes a pointer image acquisition unit (pointer image acquisition unit 130) that acquires a pointer image composed of infrared light based on a captured image captured by the imaging device, and based on the pointer image. A specifying unit (instructed position specifying unit 140) for specifying the indicated position instructed by the infrared light; and a video control unit (video control unit 160) for controlling the video light based on the designated position. Is the gist.

  According to such a feature, the specifying unit provides an instruction in a video composed of video light based on the visible light image acquired by the visible light image acquisition unit and the pointer image acquired by the pointer image acquisition unit. The indicated position indicated by a stick or the like is specified. Then, a drawing image is generated based on the indication position indicated by the indication bar. Therefore, it is possible to superimpose a drawing image drawn by the pointer on the basic image. Thereby, the user's convenience can be improved.

  In one aspect of the present invention, the imaging device continuously captures the captured image, and the specifying unit includes the first captured image (N-1th frame) of the consecutive captured images. The designated position designated by the infrared light is specified on the basis of the captured image).

  With this configuration, it is possible to reduce the amount of calculation in the specific unit, and in turn, it is possible to improve the response speed for generating a drawn image.

  In one aspect of the present invention, the presentation system further includes a storage unit (captured image storage unit 110) that stores a captured image captured by the imaging device. The storage unit corrects distortion due to the position of the imaging device and stores a captured image.

  With this configuration, it is possible to store an image of the projection surface from a directly facing position regardless of the position of the imaging device. Therefore, it is possible to accurately specify the designated position, and it is also possible to save a clear image when it is desired to leave the image light projected on the projection surface in the record.

  ADVANTAGE OF THE INVENTION According to this invention, the presentation system which makes it possible to superimpose the drawing image drawn with the indicator on the basic image can be provided.

It is a figure which shows the structure of the presentation system which concerns on 1st Embodiment. It is a figure which shows the imaging lens vicinity of the imaging device which concerns on 1st Embodiment. It is a block diagram which shows the control unit which concerns on 1st Embodiment. It is a figure which shows an example of the captured image (1st captured image) imaged with the imaging device which concerns on 1st Embodiment. It is a figure which shows an example of the captured image (2nd captured image) imaged with the imaging device which concerns on 1st Embodiment. It is a figure which shows the other example of the captured image imaged with the imaging device which concerns on 1st Embodiment. It is a figure which shows the example which correct | amends the distortion of the captured image imaged with the imaging device which concerns on 1st Embodiment. It is a figure which shows an example which tracks a specific subject from the captured image imaged with the imaging device which concerns on 1st Embodiment. It is a figure which shows an example of the basic image, drawing image, and superimposed image which concern on 1st Embodiment.

  Hereinafter, a presentation system according to an embodiment of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

  However, it should be noted that the drawings are schematic and ratios of dimensions and the like are different from actual ones. Therefore, specific dimensions and the like should be determined in consideration of the following description. Moreover, it is a matter of course that portions having different dimensional relationships and ratios are included between the drawings.

[First Embodiment]
(Presentation system configuration)
The configuration of the presentation system according to the first embodiment will be described below with reference to the drawings. FIG. 1 is a diagram illustrating the configuration of the presentation system according to the first embodiment, and FIG. 2 is a diagram illustrating the vicinity of the imaging lens of the imaging device according to the first embodiment.

  As shown in FIG. 1, the presentation system 10 includes a personal computer (hereinafter referred to as a PC 20), a projection display device 40 that enlarges and projects the image light I onto a screen 30 that is a projection surface, and light in a specific wavelength band. Used for instructing a part of the image light I, an image pickup device 60 for picking up an image of the selective transmission element 50 that transmits the light, the screen 30 and the mounting table 32 provided below the screen 30, and an infrared LED 71 at the tip. Is provided with an indicator rod 70 arranged.

  The PC 20 includes a display unit 21 that displays an image and the like, and a projection video display device 40 and an operation unit 22 that operates the PC 20. In addition, although the number of PCs 20 is one in FIG. 1, it is not always necessary to be one, and may be a plurality of units. In the presentation system 10, another information terminal device may be used instead of the PC 20.

  The screen 30 reflects the image light I from the projection display apparatus 40. The area of the image light I projected from the projection display apparatus 40 onto the screen 30 is defined as an image 31 constituted by the image light I. The screen 30 includes a mounting table 32 at the lower part and a projection display device 40 at the upper part. The sample placed on the placing table 32 is picked up by the image pickup device 60 and projected onto the screen 30 by the projection display device 40.

  The projection display apparatus 40 is provided on the upper part of the screen 30. The projection display apparatus 40 enables projection with a very short focal length from the upper part of the screen 30 by a projection lens group (not shown) and an aspherical concave mirror.

  Although not shown in the drawing, the projection display 40 is a light source that emits white light, a UV / IR cut filter that transmits visible light and blocks non-visible light (external light), and uniform light. Fly eye lens unit, PBS array that aligns the polarization state of light, and liquid crystal panel that modulates visible light (red component light, green component light, and blue component light) (eg, liquid crystal panel for red component light, green component) A liquid crystal panel for light and a liquid crystal panel for blue component light) and a cross dichroic prism for synthesizing light emitted from the liquid crystal panel.

  As illustrated in FIG. 2, the selective transmission element 50 is provided on the light incident side of the imaging device 60. Specifically, the selective transmission element 50 transmits light in a first state where visible light has a first wavelength band and light in an infrared wavelength band (second wavelength band) emitted from the infrared LED 71. The second state can be switched.

  The imaging device 60 includes an imaging lens 61 that collects the light irradiated on the screen 30 (that is, the light reflected from the screen 30) and the light from the mounting table 32 provided below the screen 30, and the imaging lens. And an image sensor 62 that images the light emitted from 61.

  In this case, the imaging lens 61 is suitably a wide-angle lens that can collect light from a wide range including the screen 30 and the mounting table 32. When the imaging lens 61 is a normal condensing lens, the switching unit (not shown) collects light from the region including the screen 30 (instructed position specifying mode) and condenses from the region including the surface on the mounting table 32. The direction to be performed (document camera mode) may be switched.

  The imaging element 62 converts the light emitted from the imaging lens 61 into an analog signal. Then, the image sensor 62 converts the analog signal into a digital signal (that is, a captured image). The image sensor 62 outputs the captured image to the PC 20. The imaging element 62 captures the first captured image (see FIG. 4) when the selective transmission element 50 is in the first state. On the other hand, the imaging element 62 captures a second captured image (see FIG. 5) when the selective transmission element 50 is in the second state. Details of the first captured image and the second captured image will be described later.

  The instruction bar 70 is a tool for instructing an image displayed on the screen 30 by a user (not shown). The indicator bar 70 includes an infrared LED 71 that emits infrared light at the tip. The indicator bar 70 has a light emission button 72 for emitting infrared light from the infrared LED 71. That is, when the user presses the light emitting button 72, infrared light is emitted from the pointing rod 70.

(Configuration of control unit)
Next, the configuration of the control unit according to the first embodiment will be described with reference to the drawings. FIG. 3 is a block diagram illustrating the control unit according to the first embodiment, and FIGS. 4 to 6 are diagrams illustrating an example of a captured image captured by the imaging apparatus according to the first embodiment.

  Here, FIG. 4A is a diagram illustrating a wavelength band of light (visible light) transmitted through the selective transmission element 50 when the selective transmission element 50 is in the first state, and FIG. FIG. 11 is a diagram showing a first captured image captured by the image sensor 62 when the selective transmission element 50 is in the first state. Note that the first captured image includes a basic image constituted by the image light I projected on the screen 30 by the projection display apparatus 40, an image of a user's hand, and an image of the pointing bar 70.

  FIG. 5A is a diagram illustrating a wavelength band of light (infrared component light) transmitted through the selective transmission element 50 when the selective transmission element 50 is in the second state, and FIG. It is a figure which shows the 2nd picked-up image imaged with the image pick-up element 62, when the selective transmission element 50 is a 2nd state. At this time, it is assumed that the user is pressing the light emitting button 72 and infrared light is emitted from the infrared LED 71.

  FIG. 6 is a diagram illustrating a second captured image when the user does not press the light emitting button 72 and infrared light is not emitted from the infrared LED 71.

  As shown in FIG. 3, the control unit 100 is provided in the PC 20. The control unit 100 specifies the indicated position indicated by the indication rod 70. Specifically, the control unit 100 includes a captured image storage unit 110, a visible light image acquisition unit 120, a pointer image acquisition unit 130, an indicated position specifying unit 140, a basic video generation unit 150, and a drawing video generation unit. 155 and a video control unit 160.

  The captured image storage unit 110 stores a captured image captured by the image sensor 62. For example, the captured image storage unit 110 stores a first captured image (see FIG. 4), a second captured image (see FIG. 5), and the like.

  Here, the imaging device 60 includes a wide-angle imaging lens 61 that images the entire screen 30 from a distance of 50 cm to 100 cm from the screen 30. Accordingly, light incident on the image sensor 62 from the screen 30 is inclined at a predetermined angle with respect to the surface of the screen 30. Therefore, the captured image storage unit 110 corrects the distortion of the image due to the position of the imaging device 60 using the following method and stores the captured image captured by the imaging element 62 as the first captured image and the second captured image. To do.

  In order to generate a corrected image, a conversion parameter for converting a captured image into a corrected image is required. The captured image storage unit 110 calibrates the conversion parameter so that an appropriate corrected image is obtained, that is, on the corrected image, so that the shape of the calibration pattern is square (or substantially square). Is used to convert the captured image into a corrected image.

  Specifically, the homography matrix H is uniquely determined if a correspondence relationship between four coordinate values is known between the captured image and the corrected image. A known technique may be used as a technique for obtaining a homography matrix (projection transformation matrix) based on the four-point coordinate value correspondence between the captured image and the corrected image.

  That is, the coordinate values (x1, y1), (x2, y2), (x3, y3) and (x4, y4) of the captured image as shown in FIG. 7 are respectively the coordinate values (X1, Y1) on the corrected image. ), (X2, Y2), (X3, Y3) and (X4, Y4), the elements of the homography matrix H are obtained. If the homography matrix H is obtained, an arbitrary point on the captured image can be converted to a point on the corrected image.

  Actually, according to the latest homography matrix H, table data indicating the correspondence between each coordinate (x, y) on the captured image and each coordinate (X, Y) on the corrected image is created, and this is captured. Stored in the image storage unit 110. By using this table data, the captured image can be converted into a corrected image, that is, a first captured image as shown in FIG. 4 or a second captured image as shown in FIG.

  The visible light image acquisition unit 120 acquires a visible light image composed of visible light irradiated on the screen 30 based on the captured image captured by the imaging element 62. Specifically, the visible light image acquisition unit 120 acquires the first captured image (see FIG. 4) as a visible light image.

  The pointer image acquisition unit 130 is emitted from the position of the distal end portion of the pointing rod 70, specifically, the infrared LED 71 provided at the distal end of the pointing rod 70 based on the captured image captured by the imaging element 62. A pointer image composed of infrared component light is acquired. That is, the pointer image acquisition unit 130 acquires a pointer image based on the second captured image (see FIG. 5).

  The pointing position specifying unit 140 uses the visible light image (see FIG. 4) acquired by the visible light image acquisition unit 120 and the pointer light (see FIG. 5) acquired by the pointer image acquisition unit 130 based on the video light. In the image 31 constituted by I, the designated position designated by the pointing rod 70 is specified.

  Specifically, the designated position specifying unit 140 has a tracking function for tracking a specific subject in the acquired captured image (visible light image and pointer image). Such a function is realized by the tracking processing unit 141 included in the designated position specifying unit 140. For example, the tracking processing unit 141 tracks the tracking target by tracking color information of the tracking target (for example, a color signal including R, G, and B signals) acquired by imaging.

  For example, a color signal based on infrared component light emitted from the infrared LED 71 is registered in the instructed position specifying unit 140 in advance in the internal memory as color information to be tracked. Here, the infrared component light is not visually recognized as a color by humans but is referred to as a color signal for convenience. The tracking processing unit 141 sets a tracking target color signal registered at the start of tracking as a tracking color. The tracking processing unit 141 detects the position and size of a tracking target from sequentially input captured images (frame images).

  As a tracking processing method based on color information, the methods described in JP-A-5-284411, JP-A-2000-48211, JP-A-2001-169169, and the like can be used. As described above, the color information that the tracking target has is expressed by the set tracking color. For this reason, the tracking processing unit 141 extracts a region having a color with high similarity to the tracking color from the captured image based on the color information of the captured image.

  The extracted area is regarded as a pointer image area constituted by infrared component light emitted from the infrared LED 71. More specifically, as shown in FIG. 8, a tracking frame T having the same size as the size of the tracking target pointer image area is set in the captured image P. Similarity evaluation between the color of the image and the tracking color within the tracking frame T is executed while sequentially changing the position of the tracking frame T within the search range S. It is determined that the tracking target pointer image area exists at the position of the tracking frame where the maximum similarity is obtained.

  The search range S in the Nth frame is set based on the position of the tracking target in the captured image of the (N-1) th frame. Usually, the search range S is a rectangular region centered on the position of the tracking target in the previous captured image, and the size of the search range (image size) is set smaller than the size of the entire region of the captured image.

  The tracking processing unit 141 detects the position of the tracking target in each tracking target frame image by executing the tracking process based on the color information described above for the captured images that are input one after another. For example, the position of the tracking target is expressed by the center coordinate value of the tracking target pointer image area. Further, the size of the tracking target on the captured image P changes due to a change in the distance in the real space between the tracking target and the imaging device 60.

  For this reason, it is necessary to appropriately change the size of the tracking frame T according to the size of the tracking target on the captured image P. This change is realized by using a subject size detection method used in a known tracking algorithm.

  The image feature includes luminance information and color information. The contour of the tracking target is estimated by this classification. Then, the size of the tracking target is estimated from the contour, and the size of the tracking frame is set according to the estimated size. Setting the size of the tracking frame is synonymous with detecting the size of the pointer image area to be tracked in the captured image. Further, since the size of the tracking target is proportional to the size of the pointer image area, the size of the tracking target is specified simultaneously with the detection of the size of the pointer image area. For this reason, the tracking processing unit 141 detects the position and size of the tracking target in each captured image.

  Thus, by specifying the position of the pointing rod 70, particularly the infrared LED 71, using the tracking function, the amount of calculation in the pointing position specifying unit 140 can be reduced.

  The basic video generation unit 150 generates a basic video (see FIG. 9A) based on the video input signal. Specifically, the basic video generation unit 150 generates a basic video composed of video light I projected on the screen 30 by the projection video display device 40.

  The drawn video generation unit 155 generates a drawn video (see FIG. 9B) based on the designated position specified by the infrared LED 71 specified by the specified position specifying unit 140. Here, the drawing image generation unit 155 can generate a drawing image without specifying the movement of the fine pointer 70 (infrared LED 71) in order to remove the influence of the user's hand shake.

  The video control unit 160 superimposes the drawn video generated by the drawn video generation unit 155 on the basic video generated by the basic video generation unit 150, and includes a superimposed video composed of visible light (see FIG. 9C). Is generated.

  Note that the video controller 160 may delete the drawn video from the superimposed video and regenerate the basic video after a certain period of time has elapsed. Moreover, although the drawing image generation unit 155 has been described as generating the drawing image at the designated position designated by the infrared LED 71 in the image 31, the present invention is not limited to this.

  With the above configuration, in the image 31 constituted by the image light I projected on the screen 30 from the projection display apparatus 40, the indication position indicated by the indication rod 70 is specified. Then, the drawn video generated by the drawn video generation unit 155 is superimposed on the basic video generated by the basic video generation unit 150, and a superimposed video (see FIG. 9C) configured by visible light is generated. . Therefore, a superimposed image (that is, a basic image and a drawn image) is projected from the projection display apparatus 40 onto the screen.

(Action / Effect)
In the first embodiment, the basic video generation unit 150 generates a basic video based on video data. The drawing video generation unit 155 generates a drawing video based on the drawing data. The video controller 160 superimposes the drawn video generated by the drawn video generator 155 on the basic video generated by the basic video generator 150 to generate a superimposed video composed of the video light I. Then, a superimposed image (that is, a basic image and a drawn image) is projected on the screen 30 from the projection display apparatus 40. Therefore, a drawing image can be projected from the projection display device 40 on the screen 30 in accordance with a user instruction, and the convenience for the user can be further improved.

[Other Embodiments]
Although the present invention has been described with reference to the above-described embodiments, it should not be understood that the descriptions and drawings constituting a part of this disclosure limit the present invention. From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art.

  Specifically, the selective transmission element 50 has been described as being configured to be switchable between the first state and the second state, but is not limited to this, for example, only specific color component light A plurality of elements such as a filter that transmits light may be provided.

  The selective transmission element 50 has been described as being provided on the light incident side of the imaging device 60, but is not limited thereto, and is configured to be switchable between the first state and the second state. For example, it may be provided between the imaging lens 61 and the imaging element 62.

  The infrared LED 71 provided at the tip of the indicator rod 70 has been described on the assumption that the light emitting button 72 is turned on when the light emitting button 72 is pressed. However, the present invention is not limited to this, and a plurality of lighting patterns are selected. Or a plurality of light emission buttons may be provided so that the frequency can be adjusted.

  In addition, by arranging a plurality of infrared LEDs at a predetermined distance, the position and inclination of the entire pointing bar 70 may be specified, and various drawing images may be drawn based on the information. . It may be set so that the movement of the indicator bar 70 can be detected and the lighting pattern of the infrared LED 71 can be changed by incorporating an acceleration sensor in the indicator bar 70. With these configurations, the buttons arranged on the pointing bar 70 can be deleted, and a simple pointing bar 70 can be provided.

  Furthermore, the drawing image may not be erased after a certain time has elapsed, but a light stop button may be arranged so that the drawing image can be erased.

  In the first embodiment, the instruction bar 70 has been described as means for adding a drawn image to the basic image, but the role of the instruction bar 70 is not limited to this. It can be used as a means to replace the mouse connected to the PC 20 by combining the lighting pattern and the icon as described above.

  In addition, when the distance between the projection display device 40 (imaging device 60) and the projection surface 30 is very short, a pen-type device may be used instead of the pointing rod 70. In this case, since the imaging device 60 captures an image from an extremely oblique direction, the light from the infrared LED 71 is not blocked by a person holding the pointer 70. Therefore, the convenience for the user is further improved by making the shape easy to handle (for example, pen type).

  Although not particularly mentioned in the embodiment, the PC 20 may be connected to the projection display apparatus 40 and the imaging apparatus 60 by wire or may be connected wirelessly. Similarly, the projection display apparatus 40 may be connected to the imaging apparatus 60 by wire or wireless.

  From this disclosure, various alternative embodiments, examples and operational techniques will be apparent to those skilled in the art. Therefore, the technical scope of the present invention is defined only by the invention specifying matters according to the scope of claims reasonable from the above description.

  DESCRIPTION OF SYMBOLS 10 ... Presentation system, 20 ... PC, 21 ... Display part, 22 ... Operation part, 30 ... Screen, 31 ... In image, 32 ... Mounting stand, 40 ... Projection type video display apparatus, 50 ... Selective transmission element, 60 ... Imaging Device: 61 ... Imaging lens, 62 ... Imaging element, 70 ... Indicator bar, 71 ... Infrared LED, 72 ... Light exit button, 100 ... Control unit, 110 ... Captured image storage unit, 120 ... Visible light image acquisition unit, 130 ... Pointer image acquisition unit, 140 ... Instructed position specifying unit, 141 ... Tracking processing unit, 150 ... Basic video generation unit, 155 ... Drawing video generation unit, 160 ... Video control unit

Claims (3)

A presentation system comprising a projection display device that projects image light onto a projection surface, and an imaging device that images the projection surface,
A pointer image acquisition unit that acquires a pointer image composed of infrared light based on a captured image captured by the imaging device;
Based on the pointer image, a specifying unit that specifies the indicated position indicated by the infrared light,
A presentation system comprising: a video control unit that controls the video light based on the indicated position. The presentation system according to claim 1, wherein
The imaging device continuously captures the captured image,
The said specification part specifies the instruction | indication position instruct | indicated with the said infrared light based on the previous captured image among the said continuous captured images. The presentation system according to claim 1, wherein
A storage unit that stores a captured image captured by the imaging device;
The said storage part correct | amends the distortion by the position of the said imaging device, and memorize | stores a captured image, The recording system characterized by the above-mentioned.