JP2009295031A - Image projection device and its control method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve an image projection technology with which the visibility or operability of an operation menu can be improved. <P>SOLUTION: An image projection device includes: a projection means for projecting an image and a user interface image on a projection face; an object detection means for detecting information related with a prescribed operation object existing on the projection face; an operation detection means for detecting the operation on the basis of the information related with the operation object detected by the object detection means on the projection face on which the user interface image is projected; a size decision means for deciding the size of an image projected by the projection means; a determination means for determining at least either the display position or size of the user interface image to be projected on the projection face according to the projection image size decided by the size decision means; and a control means for controlling the projection means on the basis of at least either the display position or size determined by the decision means, and for projecting the user interface image with the image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image projection technique for projecting a user interface image together with an image onto a projection plane.

  Conventionally, a system for operating a computer using an image projection apparatus such as a projector and an imaging apparatus such as a camera has been proposed (see, for example, Patent Document 1). In this system, an operation on a user's desk is photographed by a camera, and the position of the user's hand / fingertip is detected by performing image processing on the photographed image. Using the detected position of the user's hand / fingertip, a barcode instruction placed on the desk by the user is recognized, and an image related to the barcode is projected on the desk.

In addition, an apparatus that can perform a setting menu operation of an image projection apparatus on a projected image has been proposed (see, for example, Patent Document 2).
JP 2001-282456 A JP-A-2005-141151

  However, in the technique described in Patent Document 2, the display position and direction of menus and operation icons are fixed regardless of the size of the projection image (projection screen). For this reason, there has been a problem that operation by the user may be difficult depending on the size of the projected image. For example, when a menu is displayed on a projection screen having the same position and size as that of a projected image of 20 inches, the menu is displayed at a position higher than the user. For this reason, in order for the user to select a menu with his / her hand, it is necessary to forcibly extend his / her hand upward, or it is necessary to move to the vicinity of the menu.

  The present invention has been made in view of the above problems, and can determine the visibility and operability of the operation menu by determining the display position and size of the operation menu according to the size of the projection image and the size of the user's hand. Realize projection technology.

  In order to solve the above-described problems, an image projection apparatus according to the present invention includes a projection unit that projects a user interface image together with an image onto a projection plane, and an object detection that detects information about a predetermined operation object existing on the projection plane. Means, operation detection means for detecting an operation based on information on the operation object detected by the object detection means on the projection plane on which the user interface image is projected, and a size of an image projected by the projection means Size determining means for determining, at least one of a display position and a size of the user interface image to be projected on the projection plane according to the projection image size determined by the size determining means, and the determination The projection means based on at least one of the display position and size determined by the means Controlled, and a control means for projecting the user interface image together with the image.

  The image projection apparatus of the present invention includes a projection unit that projects a user interface image together with an image onto a projection plane, an object detection unit that detects information about a predetermined operation object existing on the projection plane, and the user interface image. An operation detection unit that detects an operation based on information about the operation object detected by the object detection unit on the projection surface on which the object is projected, a size determination unit that determines a size of the operation object, and the size determination unit A determination unit that determines at least one of a display position and a size of the user interface image to be projected onto the projection plane according to the size of the operation object determined by the step, and at least the display position and the size determined by the determination unit Controlling the projection means on the basis of the user interface image And a control means for projecting with serial images.

  The image projection apparatus control method according to the present invention includes a projection step of projecting a user interface image together with an image on a projection plane by a projection unit, and an object detection step of detecting information related to a predetermined operation object existing on the projection plane. And an operation detection step for detecting an operation based on information on the operation object detected by the object detection step on the projection plane on which the user interface image is projected, and a size of an image projected by the projection step. A size determining step for determining; a determining step for determining at least one of a display position and a size of the user interface image to be projected on the projection plane in accordance with the projection image size determined in the size determining step; and the determining step The projection hand based on at least one of the display position and size determined by Controls, and a control step of projecting the user interface image together with the image.

  The image projection apparatus control method according to the present invention includes a projection step of projecting a user interface image together with an image on a projection plane by a projection unit, and an object detection step of detecting information related to a predetermined operation object existing on the projection plane. An operation detecting step for detecting an operation based on information on the operating object detected by the object detecting step on the projection surface on which the user interface image is projected, and a size determining step for determining the size of the operating object And a determination step for determining at least one of a display position and a size of the user interface image to be projected onto the projection plane according to the size of the operation object determined by the size determination step, and the determination step determined by the determination step The projection means is controlled based on at least one of the display position and size, and the user The The interface image and a control step of projected together with the image.

  According to the present invention, the visibility and operability of the operation menu are improved by determining the display position and size of the operation menu according to the size of the projection image and the size of the user's hand.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings.

  The embodiment described below is an example as means for realizing the present invention, and should be appropriately modified or changed according to the configuration and various conditions of the apparatus to which the present invention is applied. It is not limited to the embodiment.

[First Embodiment]
First, as a first embodiment, an apparatus that determines at least one of the display position and size of an operation menu according to the size of a projection image will be described.

  FIG. 1 is a block diagram showing a schematic configuration of an image projection apparatus according to a first embodiment of the present invention.

  In the image projection apparatus 10 illustrated in FIG. 1, the projection unit 100 includes a light source, a lens, a display device, and the like, and projects an image supplied from the projection image generation unit 101. The projection image generation unit 101 generates a projection image from image data stored in the storage medium 102 such as a memory card and supplies the projection image to the projection unit 100. The system control unit 103 is, for example, a microprocessor including a CPU, a ROM, and a RAM. The CPU reads out a program stored in the ROM and executes the program, thereby controlling the operation of the image projection apparatus 10.

  The imaging unit 104 is, for example, a camera that can capture a moving image, and includes an imaging element such as a CCD image sensor or a CMOS image sensor, and a lens for forming an optical subject image on the imaging element. In the present embodiment, the imaging unit 104 is an infrared camera that can capture only the infrared light region. In the present embodiment, the imaging range of the imaging unit 104 is a range including the projection range of the projection unit 100. The imaging unit 104 supplies the captured image to the hand region detection unit 105.

  The hand region detection unit 105 is an image region (hand region) that is identified as a human hand, specifically a human hand, as an example of a predetermined specific subject from the captured image supplied from the imaging unit 104. Is detected. Information regarding the hand area (hand size, etc.) detected by the hand area detection unit 105 is given to the user operation detection unit 106 and the menu display control unit 107.

  The user operation detection unit 106 detects a user operation based on information on the hand area. The projection image size determination unit 110 determines the image size projected by the projection unit 100 and outputs the determination result to the menu display control unit 107. The menu display control unit 107 controls menu display as a user interface image based on information on the size of the projection image and the hand area.

  FIG. 2 is a diagram schematically showing how an image is projected and captured by the image projection apparatus shown in FIG.

  In the example shown in FIG. 2, the projection plane 210 is a horizontal plane formed by a desk or table, and shows a state in which the projection image 211 is projected on the projection plane 210. The projection surface 210 may be a wall or the like according to the user's usage environment.

  The imaging unit 104 captures an area including the projected image 211 projected. The size of the projection image changes depending on the distance from the projection plane 210 and the zoom state of the projection unit 100. Therefore, the imaging unit 104 changes the angle of view so that the ratio of the area of the projected image 211 in the captured image is always the same. Specifically, when the size of the projected image is small, the image is taken on the telephoto side, and when the size of the projected image is large, the image is taken on the wide-angle side to keep the relationship between the captured image and the projected image constant. Further, although not shown, the user can operate the apparatus 10 by inserting a hand into the region of the projection image 211.

(Image projection operation)
With reference to FIG. 1 again, the image projecting operation of the image projecting device of the present embodiment will be described.

  A plurality of image data is recorded in the storage medium 102. The system control unit 103 designates any of a plurality of image data recorded in the storage medium 102 and issues a read request to the projection image generation unit 101. The projection image generation unit 101 reads the designated image data from the storage medium 102, and performs image data (display image data for display) on the projection unit 100, for example, performs decoding processing if the image data is encoded. ) Is generated. The display image data is supplied to the projection unit 100, and the projection unit 100 projects an image based on the display image data.

  Note that the image to be projected may be designated by the user from among a plurality of image data stored in the storage medium 102, for example. This designation can be performed using any known method such as using a file browser provided by the basic software.

(User operation detection operation)
Next, a detection operation of a user operation in the image projection apparatus of this embodiment will be described.

  As described above, the imaging unit 104 is an infrared camera that can capture only an infrared light region, and images a projection range, that is, a region on a projection plane including a projection image. The imaging unit 104 captures an image of n frames per second, and outputs image data obtained by the imaging to the hand region detection unit 105 in units of frames, for example. Here, n may be an integer of 1 or more, but is preferably about 10 to 30.

  The hand region detection unit 105 detects a region corresponding to a predetermined subject, in this example, a region (hand region) corresponding to the user's hand, from the image data supplied from the imaging unit 104. FIG. 3A shows an example of a state in which the user holds (inserts) the hand 302 over the projection image 301 on the projection plane 300.

  The hand region detection unit 105 can extract the user's hand region from the infrared image, for example, by the same method as in Patent Document 1. Specifically, since the imaging unit 104 is an infrared camera, it measures the radiated light from the user's hand region, and the subject has a temperature in a predetermined range (approximately 30 ° C. to 34 ° C., which is close to human body temperature). Just extract. Therefore, only the user's hand area appears in the captured image. The hand region detection unit 105 performs noise removal processing on the captured image data supplied from the imaging unit 104, and then detects the hand region and the fingertip position of the hand by pattern matching. FIG. 3B shows the hand region 311 and fingertip positions 320 to 324 detected from the image 310 captured by the imaging unit 104 in the state of FIG.

  The hand region detection unit 105 outputs, for example, information indicating the range of the hand region and information indicating the fingertip position to the user operation detection unit 106 and the menu display control unit 107 as information regarding the detected hand region.

(Projected image size judgment operation)
With reference to FIG. 1 again, the projection image size determination operation in the image projection apparatus of this embodiment will be described.

  The projection image size determination unit 110 acquires zoom and focus information of the optical system from the projection unit 100. Then, a projection image size corresponding to the zoom and focus state is calculated. It is assumed that the relationship between the zoom and focus state and the projection size is held in advance as table information. Here, it is assumed that three types of projection image sizes, large, medium, and small, are determined.

  The projection image size information determined by the projection image size determination unit 110 is output to the menu display control unit 107. In addition, the hand region detection result by the hand region detection unit 105 is also output to the menu display control unit 107.

(Menu display control)
Next, the menu display control operation of the present embodiment will be described with reference to the flow of FIG.

  FIG. 4 is a flowchart showing the operation of the menu display control unit 107 of this embodiment. The menu controlled by the menu display control unit 107 indicates an operation menu that accepts an operation by the user.

  In FIG. 4, in S401, based on the detection result of the hand region by the hand region detection unit 105, it is determined whether or not the hand has been inserted into the projection image for a certain time or more. If the hand has been inserted for a predetermined time or longer, the process proceeds to S402 to display the menu. If the hand has not been inserted or the fixed time has not elapsed since the hand was inserted, the menu is not displayed. Therefore, this process ends.

  In S402, the display position and size of the menu are determined according to the size of the projection image. FIG. 5 illustrates the relationship between the projected image size and the menu display position and size. 5A, when the projection image size is small (20 inches or less), FIG. 5B is when the projection image size is medium (20 to 80 inches), and FIG. 5C is the projection image size is large (100). The operation menus 501 to 503 are shown as examples.

  Here, the reason for determining the display position and size of the menu according to the projection image size will be described. First, regarding the menu size, the smaller the projected image size, the larger the user's hand size. Therefore, the smaller the projected image size is, the larger the menu display size is controlled so that the user can easily select it by hand.

  As for the display position of the menu, a desk or a wall can be considered as the projection plane when the projection image size is small. Further, since the projected image size is small, the user can operate the menu by extending his hand anywhere in the projected image. Therefore, as shown in FIG. 5A, the menu is displayed using the entire screen including the center of the projected image.

  On the other hand, when the projection image size becomes large, such as 100 inches, it is considered that the projection onto the wall is mainly performed. If the menu is displayed above the projected image as shown in FIG. 5A when projected onto the wall, a situation in which it is difficult for the user to reach, such as being difficult for the user to reach, occurs. Therefore, when the projection image size is large, as shown in FIG. 5C, a menu is displayed on the side (right end) and below the projection image considered to be close to the position of the user (in FIG. 5C). (Displayed in the lower right). FIG. 5B is an intermediate state between FIGS. 5A and 5C, and the menu is not displayed at the center of the projection screen but is displayed on the side (right end) of the screen.

  Returning to FIG. 4, in step S <b> 403, a request for displaying the menu image superimposed on the projection image is sent to the system control unit 103. In addition, information regarding the display position and size of the menu determined in S <b> 402 is also sent to the system control unit 103.

  Upon receiving a menu display request from the menu display control unit 107, the system control unit 103 sends information regarding the menu display position and size to the projection image generation unit 101. The projection image generating unit 101 generates display image data by superimposing and synthesizing the menu image at a position corresponding to the menu display position on the image from the storage medium 102 based on the information regarding the menu display position and size. Output to the projection unit 100. As a result, an image on which the menu is superimposed is projected from the projection unit 100.

  An example of the menu display result is shown in FIG. FIG. 6A shows a state in which the user's hand is not inserted on the projected image 601 and the projected image is composed only of images read from the storage medium 102. FIG. 6B shows a state in which the user's hand is inserted on the projection image 601 for a certain period of time, and a menu 602 is superimposed on the projection image 601 at a display position corresponding to the size of the projection image.

  The user operation detection unit 106 detects a user instruction operation based on the hand region information from the hand region detection unit 105, and outputs the detected instruction operation information to the system control unit 103. The user instruction operation is, for example, a pointing operation by the user's hand as shown in FIG. The pointing operation can be detected based on, for example, that the number of fingertip position information is one or the characteristics of the shape of the hand region. When the user operation detection unit 106 detects the pointing operation of the user, the user operation detection unit 106 outputs fingertip position information to the system control unit 103.

  The system control unit 103 determines whether the user's instruction operation has continued for a predetermined time or more with respect to the displayed menu item. That is, as shown in FIG. 6C, it is determined whether the fingertip position continues to exist in the button area included in the menu 602 for a predetermined time or more. When it is determined that the fingertip position continues to exist in the menu item area for a predetermined time or longer, the system control unit 103 determines that the user is operating the menu item, and performs an operation corresponding to the menu item. I do.

  The example of FIG. 6C shows a state where the fingertip position 604 exists in the area of the button 603 included in the menu 602 for a predetermined time or more. In this case, the system control unit 103 determines that the button 603 has been operated, and performs an operation according to the button 603.

  For example, if the button 603 is a button for switching the projection image to the next image stored in the storage medium 102, the system control unit 103 controls the projection image generation unit 101 to read the next image, Display image data on which the menu 602 is superimposed is generated. As a result, the projected image is switched as shown in FIG. 6D, for example.

  The user may be notified by voice or the like that it is determined that the menu item has been operated. Thereby, the user does not have to keep pointing to the menu item indefinitely, which is easy to use.

  In addition, after the menu is displayed, if the hand image is not continuously detected in the photographed image for a certain period of time, the menu display is stopped.

  As described above, according to the present embodiment, the visibility and operability of the menu superimposed on the projected image are determined by determining at least one of the display position and size of the operation menu according to the size of the projected image. Can be improved.

  In this example, the size of the projection image is determined from the zoom and focus information of the projection unit 100, but the present invention is not limited to this. For example, distance detection means such as a distance measuring sensor capable of measuring the distance to the projection plane may be provided, and the projection image size may be determined from the distance to the projection plane and zoom information.

  In the above example, the example in which the projection image size is divided into three stages of large, medium, and small has been described. However, the present invention is not limited to this, and it may be configured to determine two stages or four or more stages. Good.

[Second Embodiment]
Next, as a second embodiment, an apparatus for determining at least one of the display position and size of the operation menu according to the size of the user's hand area will be described.

  FIG. 7 is a block diagram illustrating a schematic configuration of the image projection apparatus according to the second embodiment. The same components as those in FIG. 1 are denoted by the same reference numerals.

  In FIG. 7, the projection image size determination unit 110 is deleted from the configuration of FIG. 1, and a hand size determination unit 710 is added. As will be described later, the menu display control unit 107 determines the display position and size of the menu based on the determination result by the hand size determination unit 710. 7 are the same as those in the first embodiment, description thereof is omitted here.

  In FIG. 7, the hand size determination unit 710 obtains information regarding the hand region detected by the hand region detection unit 105 and determines the size of the hand region. Specifically, the center of gravity position of the hand is calculated, and the distance between the center position of the hand and the fingertip position is calculated. FIG. 8 shows processing in the hand size determination unit.

  In FIG. 8, reference numeral 800 denotes a hand region, reference numeral 801 denotes a center of gravity position of the hand, and reference numeral 802 denotes a fingertip position that is farthest from the center of gravity position 801 of the hand. The hand size determination unit 710 calculates the distance between the center of gravity position 801 of the hand and the fingertip position 802. Then, the calculated distance is compared with a predetermined threshold value, and the size of the hand region is determined as one of large, medium, and small. Specifically, it is determined that the larger the calculated distance is, the larger the hand region size is, and the shorter the distance is, the smaller the hand region size is determined.

  The hand size determination unit 710 outputs the size information of the hand area to the menu display control unit 107.

(Menu display control)
Next, the menu display control operation of this embodiment will be described with reference to the flow of FIG.

  FIG. 9 is a flowchart showing the operation of the menu display control unit of the present embodiment.

  In FIG. 9, in S901, based on the detection result of the hand region by the hand region detection unit 105, it is determined whether or not the hand has been inserted into the projection image for a certain time or more. The processing here is the same as S401 in FIG. If the hand has been inserted for a certain time or longer, the process proceeds to S902 to display the menu. If the hand has not been inserted or a certain time has not passed since the hand was inserted, the menu is not displayed. Therefore, this process ends.

  In step S902, the display position and size of the menu are determined according to the size of the hand area determined by the hand size determination unit 710. FIG. 10 illustrates the relationship between the size of the hand area and the display position and size of the menu. 10, (a) shows the operation menu 1001 when the size of the hand region 1011 is large, (b) shows the size of the hand region 1012, and (c) shows each operation menu 1001 when the size of the hand region 1013 is small. -1003.

  As described above, when the size of the projection image changes, the imaging unit 104 changes the angle of view in order to capture an area including the entire projection image. Therefore, the larger the size of the projected image, the smaller the size of the hand region to be shot, and the smaller the size of the projected image, the relatively large size of the hand region to be shot.

  Here, the reason for determining the display position and size of the operation menu according to the size of the hand area will be described. First, regarding the size of the hand area, if the size of the hand area is large, the selection operation by hand becomes difficult if the menu is displayed small. Therefore, as the size of the hand area increases, the display size of the menu is increased so that the user can easily select by hand.

  Regarding the display position of the menu, when the size of the hand area is large, the projection image is small, and a desk or a wall can be considered as the projection plane. Further, since the projected image size is small, the user can operate the menu by extending his hand anywhere in the projected image. Therefore, as shown in FIG. 10A, the menu is displayed using the whole including the central portion of the projection image.

  On the other hand, the case where the size of the hand region is reduced is considered to correspond to the case where the projection image size in the first embodiment is increased. Therefore, as in the case where the projection image size of the first embodiment is large, a menu is displayed below the left and right sides of the projection image considered to be close to the user's position as shown in FIG. FIG.10 (b) is an intermediate state of Fig.10 (a) and (c).

  Returning to FIG. 9, in S <b> 903, a request for displaying the menu image superimposed on the projection image is sent to the system control unit 103. In addition, information regarding the display position and size of the menu determined in S <b> 902 is also sent to the system control unit 103.

  Upon receiving a menu display request from the menu display control unit 107, the system control unit 103 sends information regarding the menu display position and size to the projection image generation unit 101. The projection image generating unit 101 generates display image data by superimposing and synthesizing the menu image at a position corresponding to the menu display position on the image from the storage medium 102 based on the information regarding the menu display position and size. Output to the projection unit 100. As a result, an image on which the menu is superimposed is projected from the projection unit 100.

  In addition, it monitors user operations and accepts menu selection operations and the like. These processes are the same as those in the first embodiment.

  As described above, according to the present embodiment, the visibility and operability of the menu superimposed on the projection image are determined by determining at least one of the display position and the size according to the size of the user's hand region. Can be improved.

  In this example, the size of the hand region is determined from the distance between the center of gravity position of the hand and the fingertip position, but the present invention is not limited to this. For example, the size of the hand region may be determined from the area of the hand region.

  Further, in the above example, the example in which the size of the hand region is divided into three stages of large, medium, and small has been described, but the present invention is not limited to this, and it is configured to determine two stages or four or more stages. Also good.

[Other Embodiments]
In any of the above-described embodiments, the hand is set as a predetermined subject as an operation object used for the menu operation, and the hand region detected by the hand region detection unit 105 as the object detection unit in the captured image. An example in which menu display is performed based on information has been described. However, the present invention can be applied even when the user does not directly operate the menu with his / her hand. For example, when the user operates the menu using a pen, an indicator, or the like, the pen or the indicator may be detected as a specific subject instead of the hand. As described above, the present invention essentially can be any method for detecting the shape and position of the operation object as long as the operation object used for operating the menu and its position can be detected.

  Therefore, it is not essential to use a captured image obtained by capturing a range including a projected image for detection of an operation object used for operating a menu, and it can be detected by any other method. For example, a position sensor may be attached to the user's hand, and determination may be made based on the position detected by the position sensor and its change. In this case, a position sensor is used instead of the imaging unit 104, and a position detection unit that detects a position from the output of the position sensor is provided instead of the hand region detection unit 105.

  Furthermore, in each of the embodiments described above, the imaging unit 104 is an infrared camera, but a camera that images a visible light region can also be used. In this case, the hand area can be detected by extracting a skin color area or the like in the captured image.

  It goes without saying that the object of the present invention can also be achieved by supplying a camera with a storage medium (or recording medium) storing software program codes for realizing the functions of the above-described embodiments. In this case, this is achieved by the camera computer (or CPU or MPU) reading the program code from the storage medium and executing it.

  When the present invention is applied to the above storage medium, the storage medium stores the menu display control program and various tables described above. These program codes can be provided as firmware that can be updated, for example.

1 is a block diagram illustrating a schematic configuration of an image projection apparatus according to a first embodiment of the present invention. It is a figure which shows typically the mode of the projection and imaging of an image by the image projector shown in FIG. It is a figure explaining operation | movement of the hand area | region detection part of the image projector of 1st Embodiment. It is a flowchart which shows the display control process of the menu of 1st Embodiment. It is a figure which illustrates the relationship between a projection image size and the display position and size of a menu. It is a figure which illustrates the menu display result in a 1st embodiment. It is a block diagram which shows schematic structure of the image projector of 2nd Embodiment which concerns on this invention. It is a figure explaining operation | movement of the size determination part of the hand of the image projector of 2nd Embodiment. It is a flowchart which shows the display control process of the menu of 2nd Embodiment. It is a figure which illustrates the menu display result in 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image projection apparatus 100 Projection part 101 Projection image generation part 102 Storage medium 103 System control part 104 Imaging part 105 Hand area detection part 106 User operation detection part 107 Menu display control part 110 Projection image size determination part 710 Hand size determination part

Claims (19)

Projection means for projecting a user interface image together with the image onto a projection plane;
Object detection means for detecting information on a predetermined operation object existing on the projection plane;
Operation detecting means for detecting an operation based on information on the operating object detected by the object detecting means on the projection surface on which the user interface image is projected;
Size determination means for determining the size of the image projected by the projection means;
A determining unit that determines at least one of a display position and a size of the user interface image to be projected onto the projection plane according to the projection image size determined by the size determining unit;
An image projection apparatus comprising: control means for controlling the projection means based on at least one of the display position and size decided by the decision means and projecting the user interface image together with the image.   The image projection apparatus according to claim 1, wherein the determination unit determines the display position of the interface image to the side and below the projection plane as the projection image size increases.   The image projection apparatus according to claim 1, wherein the determination unit reduces the size of the user interface image as the projection image size increases.   4. The size determination unit according to claim 1, wherein the size determination unit determines the projection image size based on at least one of zoom information and focus information of an optical system of the projection unit. Image projection device. A distance detecting means for detecting a distance from the projection means to the projection surface;
The size determination means determines the projection image size based on at least one of zoom information of an optical system of the projection means and a distance from the projection means to the projection plane. The image projector of any one of these.   The object detection unit detects the operation object using a captured image obtained by imaging the projection plane including an image projected by the projection unit, and detects information about the operation object. 6. The image projecting device according to any one of 5 above.   The said determination means determines the said display position and size, when it determines with the said operating object existing on the image projected on the said projection surface more than predetermined time. The image projection apparatus described in 1.   The image projecting apparatus according to claim 1, wherein the operation object is a human hand. Projection means for projecting a user interface image together with the image onto a projection plane;
Object detection means for detecting information on a predetermined operation object existing on the projection plane;
Operation detecting means for detecting an operation based on information on the operating object detected by the object detecting means on the projection surface on which the user interface image is projected;
Size determination means for determining the size of the operation object;
A determining unit that determines at least one of a display position and a size of the user interface image to be projected on the projection plane according to the size of the operation object determined by the size determining unit;
An image projection apparatus comprising: control means for controlling the projection means based on at least one of the display position and size decided by the decision means and projecting the user interface image together with the image.   The image projection apparatus according to claim 9, wherein the determination unit determines the display position of the interface image to the side and below the projection plane as the size of the operation object is smaller.   11. The image projection apparatus according to claim 9, wherein the determination unit reduces the size of the user interface image as the size of the operation object is smaller.   11. The image projection apparatus according to claim 9, wherein the determination unit determines the size of the user interface image so that a ratio with the size of the operation object is constant.   10. The object detection unit detects the operation object using a captured image obtained by imaging the projection plane including an image projected by the projection unit, and detects information about the operation object. The image projector according to any one of 12.   The determination unit is configured to determine the display position and size when it is determined that the operation object is present for a predetermined time or longer on the image projected on the projection plane. The image projection apparatus described in 1.   The image projection apparatus according to claim 9, wherein the operation object is a human hand.   The image projection apparatus according to claim 15, wherein the object detection unit determines the size of the operation object based on a distance between a gravity center position of the human hand and a fingertip position.   The image projection apparatus according to claim 15, wherein the object detection unit determines a size of the operation object based on an area of the human hand. A projecting step of projecting the user interface image together with the image onto the projection surface by the projecting means;
An object detection step of detecting information on a predetermined operation object existing on the projection plane;
An operation detection step of detecting an operation based on information on the operation object detected by the object detection step on the projection surface on which the user interface image is projected;
A size determination step of determining the size of the image projected by the projection step;
A determination step of determining at least one of a display position and a size of the user interface image to be projected on the projection plane according to the projection image size determined by the size determination step;
And a control step of controlling the projection unit based on at least one of the display position and size determined by the determination step and projecting the user interface image together with the image. Control method. A projecting step of projecting the user interface image together with the image onto the projection surface by the projecting means;
An object detection step of detecting information on a predetermined operation object existing on the projection plane;
An operation detection step of detecting an operation based on information on the operation object detected by the object detection step on the projection surface on which the user interface image is projected;
A size determination step of determining a size of the operation object;
A determination step of determining at least one of a display position and a size of the user interface image to be projected onto the projection plane according to the size of the operation object determined by the size determination step;
And a control step of controlling the projection unit based on at least one of the display position and size determined by the determination step and projecting the user interface image together with the image. Control method.

Images