JP6210466B1 - Information input device - Google Patents

Abstract

When operating an electronic device, not only a special device such as a remote controller but also a dedicated projection target such as a screen for projecting an operation image is unnecessary, and the other electronic device can be operated by a user with only one hand. It is possible to operate. An information input image including a plurality of option images (buttons B) corresponding to a plurality of processes for operating an electronic device 20 on a palm P and a palm P capable of detecting a human palm P. An information input device 10 that includes a projector 5 to project, a detection unit 3 and a control unit 1 that controls the projector 5, wherein the control unit 1 is positioned at a palm P detected by the detection unit 3 at a predetermined timing. The corresponding absolute position is stored, the projector 5 projects an information input image with the absolute position as a reference, and the detection unit 3 projects a predetermined option image on the center of the palm P from the predetermined position by the palm P. Is detected, the control unit 1 determines that the predetermined option image (button B) is selected, and performs processing corresponding to the selected option image (button B). To send. [Selection] Figure 9

Description

  The present disclosure relates to an information input device that operates an electronic device.

  Conventionally, when a user operates various electronic devices, an information input device such as a remote control is generally used to input information for the operation. The location of the remote control is often unknown, and the user often feels inconvenient.

  Therefore, various devices capable of inputting information without using a device such as a remote controller have been proposed. For example, Patent Literatures 1 to 4 disclose information input devices including a detection unit that detects a projection surface onto which irradiation light from a projection device is projected. By this irradiation light, an information input image for operation input for operating the electronic device is projected on the projection surface. The user can operate the electronic device by touching the information input image.

  In particular, in Patent Documents 1 to 4, it is assumed that the palm is used as the projection surface, and the user can operate the electronic device without preparing a dedicated projection target such as a screen. .

International Publication No. 2012/173001 International Publication No. 2014/034527 Japanese Patent Laying-Open No. 2015-94828 JP, 2015-111172, A

  The information input devices of Patent Documents 1 to 4 can project an information input image on the palm without a dedicated projection target. However, when the projection surface is in the palm of the hand, the area for projecting the information input image is limited. Also, it is assumed that the palm is simply used as the projection plane, and the operation with the other hand is essential for inputting information. Therefore, the conventional information input device still has inconveniences.

  The present disclosure relates to an information input apparatus that does not prepare a dedicated projection target and enables simple operation input with only one hand of a user.

An information input device according to the present disclosure includes a detection unit capable of detecting a palm of a person, and a projection device that projects an information input image including a plurality of option images corresponding to a plurality of processes for operating an electronic device on the palm. And a control unit that controls the detection unit and the projection device, wherein the control unit stores an absolute position corresponding to a palm position detected by the detection unit at a predetermined timing. Then, the projection device projects the information input image based on the absolute position, and the detection unit detects a predetermined operation by the palm from a state in which the predetermined option image is projected at the center of the palm. when the control unit transmits determines that the predetermined option images is selected, a process that corresponds to a selection image selected to the electronic device, the control unit, or the projection device Wherein generating a plurality of information input image that is stacked along a direction leading to an absolute position, the projection device projects a plurality of information input image that is laminated along the direction.

  According to the information input device of the present disclosure, when operating an electronic device, not only a special device such as a remote controller but also a dedicated projection target such as a screen for projecting an operation image is unnecessary, and a user can Other electronic devices can be operated with only one hand, and simpler operation input is possible.

FIG. 1 is a block diagram according to an embodiment of an information input device of the present disclosure. FIG. 2 is a conceptual diagram showing a state in which the information input device projects an information input image including a button that is an option image on the palm of a person. FIG. 3 is a conceptual diagram showing an input processing operation by the information input device. (A) shows a series of operations for detecting a palm in space, and (b) projects an information input image including buttons. A series of operations is shown. FIG. 4 is a conceptual diagram showing an input processing operation by the information input device, where (a) shows an example of a series of operations for selecting a specific button, and (b) is a specific button instead of (a). Another example of a series of operations for selecting is shown. FIG. 5 is a conceptual diagram illustrating an initial state in which the information input device projects an information input image including a button that is an option image on the palm of a person. FIG. 5A illustrates an example displayed at the initial time. , (B) shows a state in which the palm is moved after being displayed at the initial stage. FIG. 6 is a conceptual diagram showing an input processing operation by the information input device. (A) shows a series of operations for determining execution of the selected button, and (b) and (c) are inputs by the information input device. An example of the operation | movement which orders the completion | finish of a processing operation is shown. FIG. 7 is a flowchart of an example of an input processing operation by the information input device. FIG. 8 is a flowchart of another example of the input processing operation by the information input device. FIG. 9 is a conceptual diagram showing an initial state in which the information input device projects an information input image including a button which is an option image on the palm of a person, and (a) shows an example displayed at the initial time. , (B) shows a state in which the palm is moved after being displayed at the initial stage. FIG. 10 is a conceptual diagram for explaining one application example of the information input device. FIG. 10A is a diagram showing images of other buttons in the vertical direction in addition to the button that is an information input image displayed first. (B) is an example of displaying a character (message) informing that there is another button in the vertical direction on the button that is the information input image displayed first, (C) is an example in which, in addition to the button that is the information input image that is displayed first, the images of other buttons are superimposed and displayed in the vertical direction. FIG. 11 is a conceptual diagram illustrating an example in which the user can easily grasp the projection range by projecting a display image that informs the projection range when the information input image has a wide projection range. ) Shows an example in which the projector projects a horizontal double arrow indicating the projection range, and (b) shows an example in which the projector projects a left arrow indicating the projection range. FIG. 12 is a conceptual diagram showing an example in which the size of an information input image is determined in accordance with the vertical position of the palm. FIG. 12A is a basic example, and is based on the principle of optics. This is an example in which the size of the button that is an information input image changes following the movement, and (b) is based on the distance to the palm and optimizes the button size independently of the vertical movement of the palm. This is an example of what to do. FIG. 13 is a conceptual diagram showing an example of calculating the area of the palm. (A1) and (a2) are examples calculated based on the base of the finger, and (b1) and (b2) An example of recognizing the root and calculating the area of the circumscribed circle in contact with the root is shown. FIG. 14 is a conceptual diagram showing an example of determining the shape of the information input image in accordance with the inclination angle of the palm. FIG. 14A is a basic example, and the palm in the lower position is determined based on the principle of optics. This is an example in which the shape of the button, which is an information input image, is changing according to the inclination angle, and (b) shows the shape of the button based on the inclination angle of the palm and the distance to the palm. This is an example of optimizing. FIG. 15 is a conceptual diagram showing an example in which the focus of the projector is determined in accordance with the position of the palm in the height direction. FIG. 15A is a basic example, and the palm is moved up and down by optical theory. This is an example in which the subject is out of focus and (b) is an example in which the optimum focus is always achieved independently of the vertical movement of the palm based on the distance to the palm. FIG. 16 is a conceptual diagram showing an example of recognizing another human body part in addition to the palm and projecting an information input image. FIG. 16A shows an information input image of a circular button, and FIG. Shows an example of recognizing the palm, head and shoulder, (c) shows an example of recognizing the palm and shoulder, (d) shows an example of recognizing the palm and head, (e ) Shows an example in which the palm and both shoulders are recognized, assuming that the linear axis connecting both shoulders and the point that bisects them are the head position, and (f) shows the palm and both shoulders. An example of recognizing whether the palm is inside or outside both shoulders in the recognized situation is shown. FIG. 17 is a conceptual diagram showing a concept for defining a range in which an information input image is projected in the vertical direction. FIG. 18 is a conceptual diagram showing the concept of processing when the projection angle for projecting the information input image to the palm is not 90 °, and (a) shows a state in which the detection unit detects the tilt angle of the palm. , (B) shows a situation in which it is determined that the movement of the palm in the plane direction is a button selection process in the plane, and the movement of the palm in the depth direction is determined to change the button projection position itself. It is a conceptual diagram.

  Hereinafter, an embodiment (hereinafter referred to as “the present embodiment”) that specifically discloses an information input apparatus according to the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

  Hereinafter, preferred embodiments for carrying out the present disclosure will be described in detail with reference to the drawings.

(Basic configuration)
FIG. 1 is a block diagram of an embodiment of an information input device of the present disclosure. The information input device 10 includes a control unit 1, a detection unit 3, and a projector (projection device) 5. The information input device 10 is a device for a user to input information for operating the electronic device 20 without requiring a device such as a remote controller. The type of the electronic device 20 is not particularly limited, and includes various devices such as an indoor lighting system, an air conditioning system, a content device that reproduces and records various content such as still images, moving images, and audio. By connecting the information input device 10 and the electronic device 20 via a communication line such as the Internet, the electronic device 20 can be operated from a remote location.

  The control unit 1 is a processor or computer device that controls the overall operation of the information input device 10, and may be provided as a separate device from the detection unit 3 and the projector 5, or provided in the detection unit 3 or the projector 5. However, the detection unit 3 and the projector 5 can be controlled. The control unit 1 has a storage device (not shown) such as a memory, and can store various programs for operating the information input device 10, coordinates of the object detected by the detection unit 3, and other information. It is.

  The detection unit 3 is an infrared type depth sensor including, for example, an infrared light emitting device and an infrared camera. The infrared light emitting device irradiates the space with infrared rays, and the infrared camera receives infrared reflected light from an object existing in the space and captures it, and position coordinate data (x, y) for each pixel of the infrared image and The depth data (d) indicating the distance to the imaged object corresponding to each pixel is output to the control unit 1. However, the type of the detection unit 3 is not particularly limited as long as the device can grasp the position and distance of the object itself.

  As shown in FIG. 2, an object that is a detection target in the present disclosure includes at least a palm P (so-called “par” state) formed by a person (user) spreading his / her hand. The control unit 1 stores a palm image pattern in consideration of individual differences in a predetermined storage unit, and compares the infrared image captured by the detection unit 3 with the image pattern and satisfies a predetermined condition. When the difference between the infrared image and the image pattern is equal to or less than a predetermined value, it is determined that the palm P is detected. Then, the control unit 1 outputs the position coordinate data of the palm P and the depth data to the projector 5.

  The projector 5 is a projection device that projects an image by irradiating the palm, which is the object in the present disclosure, with visible light based on the position coordinate data of the palm P received from the control unit 1 and the depth data. The type is not particularly limited. As shown in FIG. 2, the projector 5 projects an information input image for inputting operation information for operating the electronic device 20 by projecting visible light onto the palm P detected by the detection unit 3. Form on top. In the present embodiment, an information input image including a plurality of buttons B that are a plurality of circular images is formed. As will be described later, each button B corresponds to a plurality of option images corresponding to a plurality of processes (switch-on, various mode switching, etc.) for operating the electronic device 20.

(Overview of operation)
3 to 6 are conceptual diagrams illustrating an input processing operation by the information input device 10 of the present disclosure. Fig.3 (a) shows a series of operation | movement which detects the palm P in space. The detection unit 3 starts irradiating infrared rays with the information input device 10 being powered on or the like as a trigger (FIGS. 3A and 3A). After the detection unit 3 detects the palm P (FIGS. 3A and 3B), the control unit 1 determines that the palm P is an object to be tracked and irradiates the projector 5 with visible light. Instructed to irradiate the palm P with visible light so as to form the tracking pointer C (FIGS. 3A and 3).
In addition, when the user recognizes the specific space for detecting the palm P in advance, the tracking pointer C may not be irradiated.

  FIG. 3B shows a series of operations for forming the button B that is an option image. After irradiation of the tracking pointer C, the control unit 1 instructs the detection unit 3 to supplement the object, and the detection unit 3 continues to supplement the position coordinate data and depth data of the palm P that is the object (FIG. 3). (B) (1)). Thereafter, the user grasps the hand along the arrow A1 indicating the time series, the shape is changed to a fist F (so-called “goo” state), the hand is further expanded to form the palm P, and the detection unit 3 This series of operations is detected based on the position coordinate data and depth data that are continuously supplemented (FIGS. 3B and 2). Based on the detected operation, the control unit 1 determines that the user is ready to operate the electronic device 20, and forms a plurality of buttons B constituting the information input image on the palm P (FIG. 3B). (3)). At this time, the control unit 1 stores the position in the palm space detected by the detection unit 3 at the operation timing of FIGS. 3B and 3B as an absolute position in a storage device (not shown), and then presses the button B. Project. The “absolute position” corresponds to the position of the palm P detected by the detection unit 3 at a predetermined timing as shown in FIGS. 3B and 3B, and is an absolute position in the space that can be detected by the detection unit 3. Corresponds to coordinates. Therefore, the absolute position does not change even if the palm P moves. Note that the absolute position may slightly follow the movement of the palm P within a limited range.

  FIG. 4A shows an example of a series of operations for selecting a specific button. With the plurality of buttons B formed on the palm P (FIGS. 4A and 1), the user moves the palm P in the horizontal direction (lateral direction) (FIGS. 4A and 2). When the specific button BF is located at the center of the palm P, when the palm P is stopped, the control unit 1 determines that the user desires the selection of the button BF, and the selection that the button BF is selected. The button (option image) is determined (FIGS. 4A and 3).

FIG. 4B shows another example of a series of operations for selecting a specific button, which is different from FIG. With the plurality of buttons B formed on the palm P (FIG. 4B (1)), the user slides the palm P in the vertical direction (height direction) A2 (FIGS. 4B and 2). )). At this time, the palm P moves, for example, to an upper height indicated by a palm P1 or a lower height indicated by a palm P2. The buttons displayed at the respective heights change as indicated by buttons B1 and B2. That is, the control unit 1 generates a plurality of information input images stacked along the vertical direction, and the projector 5 projects the plurality of information input images stacked along the vertical direction.
As shown in FIG. 5, when a plurality of buttons B are first formed on the palm P (initial time), it is desirable not to arrange the selectable button B near the center of the palm P. (See FIG. 5 (a)). The buttons near the center of the palm P are often selected unintentionally by the user at the initial stage. Therefore, in the initial formation, the selectable button B is not arranged near the center of the palm P, and an empty space is provided. Or an area for displaying a message or the like. Other than the initial formation, a selectable button B may be arranged near the center of the palm P.

  Thereafter, the control unit 1 can determine the button BF as the selection button by performing the operations shown in FIGS. 4A, 2B, and 4A, 3 at the respective heights ( (Refer FIG.5 (b)). That is, the user slides the palm in the horizontal direction at the respective heights indicated by the palms P, P1, and P2 (FIGS. 4A and 2). When the specific button BF is positioned at the center of the palm and the palm is stopped, the control unit 1 determines that the user desires to select the button BF, and determines that the button BF is a selection button (FIG. 4). (A) (3)).

  FIG. 6A shows a series of operations for determining execution of a process corresponding to the selected selection button BF. After determining the button BF as a selection button (FIGS. 6A and 1), the control unit 1 determines the final selection of the button BF after the user grasps the palm and forms the fist F. When the palm is opened from the state of the fist F, an instruction to execute the processing operation indicated by the button BF is transmitted to the electronic device 20 (FIGS. 6 (a) (2), 6 (a) (3)). In order to visually notify the user that the final selection has been made, the display mode of the button BF may be changed.

  FIG. 6B shows an example of an operation for instructing the end of the input processing operation by the information input device 10. When the user makes the palm P downward and vertical, the control unit 1 determines the end of the input processing operation. FIG. 6 (c) shows another example of the operation to mitigate the end. After the user moves the palm P up and down several times (FIG. 6 (c) (1)), this operation is stopped. When a predetermined operation of the palm P is detected (FIGS. 6C and 2), the control unit 1 determines the end of the input processing operation. The projector 5 stops projecting the information input image including the plurality of buttons B. Of course, the end determination operation is not particularly issued. Also, when the palm P is out of the detection area that the detection unit 3 can detect, the control unit 1 may determine the end of the input processing operation. However, for example, even when the projection of the button ends, the reproduction of the content by the electronic device 20 may be continued. When ending the reproduced (operated) content, for example, another ending button may be selected.

(Input processing operation)
FIG. 7 is a flowchart of an example of an input processing operation by the information input device 10. This flowchart shows processing steps including the operation of selecting a specific button shown in FIG. 4 (a), and the button BF is determined as a selection button by moving only the horizontal direction without moving the palm P vertically. (FIG. 4 (a) (3)), so-called two-dimensional selection processing.

  First, the detection unit 3 continues the detection until the palm is detected (step S1; No). When the palm is detected (step S1; Yes), the projector 5 detects the palm according to the instruction from the control unit 1 that has received the detection. The tracking pointer is projected to the detection unit 3, and the detection unit 3 tracks the tracking pointer that moves as the palm moves (step S2). The steps so far correspond to the operations in FIGS. 3A, 3B, and 1).

  Next, the detection unit 3 detects a palm motion (gesture) as shown in FIGS. 3B and 3B, and if no motion is detected, returns to step S1 and starts palm detection again. (Step S3; No). On the other hand, when the detection unit 3 detects the operation of the palm as shown in FIGS. 3B and 3B (step S3; Yes), a plurality of buttons B, which are option images, are projected onto the palm (step S4). ). Here, the information input image including the button B is projected with reference to a specific absolute position (FIGS. 3B and 3). In addition, the operation | movement which determines to project the button B is not limited to the operation | movement which grasps and spreads the hand shown to FIG.3 (b) (2), Other predetermined operation | movement may be sufficient.

  And the detection part 3 detects a palm again, and when it does not detect here, the control part 1 determines with the detection of a palm failing, returns to step S1, and starts the detection of a palm again (step). S5; No). On the other hand, when the detection unit 3 detects the palm again and detects the palm (step S5; Yes), the control unit 1 next determines whether there is any operation of the palm (step S6). Here, when the detection unit 3 detects the palm movement as shown in FIGS. 3B and 3B (step S6; Yes), the control unit 1 again determines the absolute value of the information input image including the button B. It is determined that the position should be determined, and the process returns to step S4. On the other hand, when the detection unit 3 does not detect any movement of the palm (step S6; No), the control unit 1 determines whether or not the position of the center of the palm matches the position of the projected specific button B for a predetermined time or more. If it does not match, the process returns to step S5 to detect the palm again (step S7; No). On the other hand, when the control unit 1 determines that the position of the center of the palm matches the position of the projected specific button B for a predetermined time or more (step S7; Yes), as shown in FIGS. Further, the display of the specific button is changed, and the selection button BF is displayed (step S8).

  Furthermore, the detection part 3 detects some operation | movement (gesture) of a palm, and when some operation | movement of a palm is not detected, it returns to step S5 and starts detection of a palm again (step S9; No). On the other hand, when the detection unit 3 detects any operation of the palm as shown in FIGS. 6A and 6A (step S9; Yes), the selection button BF as shown in FIGS. Is changed (step S10). And the control part 1 transmits the execution command of the process allocated to the said selection button BF to the electronic device 20 (step S11), and a process is complete | finished.

  FIG. 8 is a flowchart of another example of the input processing operation by the information input device 10. In this flowchart, steps S71 to S82 are performed instead of steps S7 and S8 in FIG. That is, FIG. 4 (b), FIG. 4 (a) (2), and FIG. 4 (a) (3) show processing steps including the operation of selecting a specific button. Moving in the direction as well, the button BF is determined as a selection button (FIGS. 4A and 3), so to speak, a three-dimensional selection process is shown.

  That is, when the detection unit 3 does not detect any movement (gesture) of the palm (step S6; No), the control unit 1 matches the position of the center of the palm and the projected button within a plane at a specific height. If it does not match, the process returns to step S5, and palm detection is performed again (step S71; No). On the other hand, when the control unit 1 determines that the position of the center of the palm matches the projected button in the plane (step S71; Yes), the button display is changed (step S8). Further, the control unit 1 determines whether or not the positions of the palm and the projected button match in the vertical direction. If they do not match, the control unit 1 returns to step S5 to detect the palm again (step S81; No). . On the other hand, when the control unit 1 determines that the position of the palm and the projected button matches in the vertical direction (step S71; No), the display of the button is changed (step S82). In the input processing operation flow of FIG. 8, the order of step 71 and step 81 may be interchanged.

  FIG. 9 is a conceptual diagram illustrating main feature points of the information input device 10 according to the present disclosure. As shown in FIG. 9A, the information input device 10 can display a plurality of buttons B, which are a plurality of option images, over a wide range that covers the palm P, and gives an operation option to the user. be able to. Further, as is clear from the comparison between FIG. 9A and FIG. 9B, even if the palm P moves, the information input device 10 does not follow the movement of the palm P, such as the button B. An information input image including an option image can be projected with reference to the absolute position in space. As a result, the selection button BF to be selected and the other buttons B are projected side by side on the palm P, and the user can easily and stably select the button B.

  Further, another button B adjacent to the button BF located at the center of the palm P is also projected onto the palm P. Therefore, the user can select the button BF while comparing the button BF with another button B, and the user can select a button corresponding to a desired process with an easy operation. Furthermore, the selected button can be finally determined by some action (gesture) such as grasping the palm P, and the selection process can be completed with only one hand, so that convenience can be improved. Although the "center" of the palm P is not strictly defined, it is a position having a predetermined range that the user can recognize as the center of the palm, and the position of the predetermined range by moving the palm by the user It suffices if the option image (button B) can be positioned on the screen.

  Although not shown in FIG. 9, the content of the information input image can be changed also according to the vertical position (height position) of the palm P viewed from the information input device 10. For this reason, it becomes possible to further expand the variation of operation of the electronic device 20.

(Application examples)
FIG. 10 is a conceptual diagram illustrating one application example of the information input device 10. In the example shown in FIG. 4B, the user can select different types of information input images (and option images included therein) according to the vertical direction (height direction) of the palm. If there is no mark, it may be difficult to determine whether there are other information input images above and below. Therefore, it is desirable that there is some means for notifying the user whether or not another information input image exists in the height direction.

  FIG. 10 (a) shows that when there is an information input image including other buttons in the vertical direction in addition to the button B, which is the image displayed first, the image is displayed in an overlapping manner so that the user can This is an example informing that there are a plurality of buttons in the vertical direction. In this example, the button B1 is displayed with the button B1 in the upward direction and the buttons B2 and B3 are stacked in the downward direction, and the control unit 1 has a plurality of options stacked in the vertical direction. A laminated state display image Ba that displays the laminated state of the buttons B, B1, B2, and B3, which are images, is generated, and the projector 5 projects the laminated state display image Ba. By intuitively moving the palm P in the vertical direction, the user can grasp that other buttons can be selected, and convenience is improved.

  Note that the buttons B1, B2, and B3 in the stacked state display image Ba play a role of notifying the existence of other buttons, and are a part of the stacked state display image Ba. The buttons that can be selected at the palm position are as follows. Only button B is available, and buttons B1, B2, and B3 cannot be selected. The user can select the buttons B1, B2, and B3 by looking at the stacked state display image Ba and moving the palm up and down as shown in FIG. 4B.

  FIG. 10B shows a message (characters or the like) informing the user that there is another button in the vertical direction on the button B, which is the image displayed first, thereby displaying a plurality of messages in the height direction. This is an example of informing that a button exists. In other words, the control unit 1 generates a message Bb that is stacking order information that displays the stacking order of each of the buttons that are a plurality of option images stacked in the vertical direction, and the projector 5 displays a message in each button B. Bb is displayed. In this example, the message Bb of “1/4 Movie 1” is displayed. By this message, the user has a total of four buttons in the vertical direction of the button B for selecting a movie. You can see that it is the first button from the bottom. The user can grasp that other buttons can be selected by intuitively moving the palm P in the height direction, and convenience is improved.

  In FIG. 10C, in addition to the button B expressing the thickness that is the first information input image displayed, when other buttons exist in the vertical direction, images of different shapes are displayed in an overlapping manner. In this example, the user is notified that there are a plurality of buttons in the vertical direction. A string-shaped button Bx is displayed in the upper direction of the button B, and a string-shaped button By is displayed in a superimposed state in the lower direction. The user moves the palm P upward from the position of the button B, for example, so that the currently selected circular button B is deformed into a string shape and overlaps the lower string-shaped button By, and the upper string The shape button Bx is located at the center in the height direction and is selected by being transformed into a thick circle. In this way, by transforming the option image depending on the presence or absence of the selection, the user can intuitively understand the selection of the specific option image, and the convenience is improved.

  In addition, the aspect which shows the lamination | stacking state of the information input image and the choice image (button) is not specifically limited to the thing of Fig.10 (a)-(c). 4B and 10A to 10C, a plurality of information input images are stacked in the vertical direction (height direction), and the user moves the palm in the vertical direction. Thus, an example of selecting each information input image and option image is shown. However, the information input device 10 may be installed on a side wall or the like, and the direction in which a plurality of information input images are stacked is not limited to the vertical direction (height direction), but is horizontal or oblique. It may be. Generally speaking, the control unit 1 may generate a plurality of information input images stacked along the direction from the projector 5 to the absolute position detected by the detection unit 3, and the projector 5 moves along this direction. A plurality of stacked information input images may be projected.

  FIG. 11 is a conceptual diagram illustrating one application example of the information input device 10. As shown in FIG. 9, when the projection range of the information input image such as the button B is limited only to the periphery of the palm P, the user can intuitively grasp all the information input images projected. However, when the projection range is limited only to the periphery of the palm P, the number and types of information input images to be projected on one surface (plane) are limited. On the other hand, if the projection range is widened, it becomes difficult to intuitively grasp information input images and option images that are not projected around the palm P except in the vicinity of the palm P.

  FIG. 11 shows a position where the projection range of the information input image in one surface is wide so as to cover an area exceeding the size of the palm, and the projection range by the projector 5 and the position of the palm are notified when a large number of option images are formed. It is a conceptual diagram which shows the example which a user can grasp | ascertain a projection range easily by projecting a display image. In the example of FIG. 11A, an example is shown in which the projector 5 projects a position display image that is a horizontal double-headed arrow A3 indicating the projection range of the information input image that is the option image group. By projecting a horizontal double-headed arrow A3 indicating the position of the palm in the information input image onto the palm P, the user can expand the projection range of the information input image on both the left and right sides of the palm P. It can be intuitively understood that there are other selectable option images (button B) in the left-right direction. In the example of FIG. 11B, an example in which the projector 5 projects a left arrow A4 indicating the projection range onto the palm P is shown. Since the projection range of the image group extends to the left side of the palm P, the user can intuitively understand that there is another selectable button in the left direction by indicating the left arrow A4. . The distance from the end of the image group may be expressed by the length of the arrow.

  The position display image is not limited to an arrow, and may be other characters, figures, symbols, etc., and is not particularly limited. The projection range may extend before and after the palm P or in an oblique range, and a position display image corresponding to the spread range is generated.

  FIG. 12 is a conceptual diagram illustrating an example in which the size of the information input image and the option image is determined in accordance with the position of the palm P in the height direction. FIG. 12A is a basic example, and the size of the button B, which is an option image, changes following the vertical movement of the palm P (arrow A5) according to the principle of optics. On the other hand, as shown in FIG. 12B, the control unit 1 is independent of the vertical movement of the palm P (arrow A6) based on the distance from the projector 5 to the palm P detected by the detection unit 3. The size of the button B can be made constant regardless of the optimum size, for example, the height. The control unit 1 calculates the optimum information input image and option image size based on the distance to the palm P, and instructs the projector 5 to output the information input image and option image having such a size. Output. By performing such control, it is possible to maintain the optimum size of the information input image and the option image regardless of the position of the palm P in the height direction.

  The method for determining the optimum information input image and option image size is not particularly limited. As described above, the control unit 1 can also calculate the optimum information input image and option image size based on the distance from the detection unit 3 to the palm P. Further, the control unit 1 can also calculate the optimum information input image and option image size based on the plane size (area) of the palm P acquired by the detection unit 3. Specifically, the control unit 1 changes the size of the information input image and the option image from the distance to the palm P and the plane size (area) of the palm P. For example, the control unit 1 calculates the size of the calculated palm P ( Information input images and option images having a size (area) of 80% of (area) may be projected. Specifically, as shown in FIG. 13, the size (area) of the palm P may be calculated as an area other than the finger. For example, the base Q of the finger is recognized and the position of the finger base Q is used as a reference. The size of the palm P can be calculated (see FIGS. 13A1 and 13A2). Alternatively, the base Q of the finger may be recognized, and the size of the palm P may be calculated based on the area of the circumscribed circle in contact with the base (see FIGS. 13B1 and 13B2). Also, the size of the information input image and option image corresponding to the distance from the detection unit 3 to the palm P is calculated in advance, and the size of the information input image and option image is changed depending on only this distance. Projection may be performed, and the processing load on the control unit 1 can be reduced by such processing.

  FIG. 14 is a conceptual diagram illustrating an example in which the shape of the input image is determined in accordance with the inclination angle of the palm P. FIG. 14A is a basic example, and the palm P in the lower position is inclined with respect to the projector 5 according to the principle of optics, and an option image included in the information input image according to the inclination angle. The shape of a certain button B has changed. Specifically, the button B extends in the lateral direction and protrudes from the palm P. The case where the palm P is not inclined with respect to the projector 5 is a case where the palm P is arranged so as to be orthogonal to the irradiation axis of the irradiation term of the projector 5.

  On the other hand, as shown in FIG. 14B, the information input device 10 optimizes the shape of the button B based on the tilt angle of the palm P with respect to the projector 5 detected by the detection unit 3 and the distance to the palm P. It can be constant regardless of the tilt angle, for example. The control unit 1 calculates an optimal information input image shape based on the tilt angle of the palm P, and outputs an instruction to the projector 5 so as to output the information input image having such a shape. By performing such control, the optimum shape of the information input image can be maintained regardless of the inclination angle of the palm P.

  FIG. 15 is a conceptual diagram illustrating an example of determining the focus of the information input image and the option image in accordance with the position of the palm P in the height direction. FIG. 15A is a basic example, and the focus is shifted following the vertical movement (arrow A7) of the palm P according to the principle of optics. In this example, since the optimum focus is given to the palm P at the lower position, the palm P at the upper position is out of focus.

  On the other hand, as shown in FIG. 15B, the information input device 10 is independent of the vertical movement of the palm P (arrow A8) based on the distance from the projector 5 to the palm P detected by the detection unit 3. Thus, the optimum focus of the projector 5 can always be determined. The control unit 1 calculates an optimum focus based on the distance to the palm P, and outputs an instruction to the projector 5 so as to output such an information input image and option image of the focus. By performing such control, it is possible to maintain an optimal focus information input image and option image regardless of the position of the palm P in the height direction.

The focus setting method is not particularly limited. As described above, the control unit 1 can also calculate the optimum focus based on the distance from the projector 5 to the palm P (active method). Moreover, the control part 1 can also calculate an optimal focus based on the light which passed the lens, when the detection part 3 is equipped with an RGB camera (passive system).
Note that the process of determining the focus of the information input image and the option image in accordance with the position of the palm P in the height direction is the same as the information input image and the position of the palm P described in FIG. The process of determining the size of the option image and the process of determining the shape of the input image in accordance with the inclination angle of the palm P described in FIG. 14 may be combined.

  FIG. 16 is a conceptual diagram illustrating one application example of the information input device 10. Depending on the position of the user's palm, the appearance of the information input image and the option image changes. Therefore, it is desirable to change the display direction of the information input image and the option image in a direction that is easy for the user to see. For example, as shown in FIG. 16A, when the option image is a circular button B, it is conceivable that the display direction is determined by rotating in the circumferential direction indicated by the arrow A9.

  For this reason, in addition to the palm, the detection unit 3 detects other human body parts such as the user's elbow, shoulder, and head from the shape, and calculates a linear axis that connects the other human body part and the palm. The controller 1 determines an appropriate display direction of the information input image with reference to the linear axis, and the projector 5 projects the information input image having this display direction. For example, when the linear axis connects the palm and the head, the information input image is preferably arranged in a display direction that is substantially parallel to or substantially perpendicular to the linear axis. For example, when the linear axis connects the palm and the shoulder, the information input image is displayed by recognizing whether the palm is the left or right hand and rotated within a range of about 0 to 60 ° from the linear axis. It should be arranged in the direction. Of course, the display direction is not uniquely defined, and is appropriately determined depending on various factors such as the type of apparatus, installation location, and user characteristics.

  FIG. 16B shows a situation in which the controller 1 recognizes the palm P, the head H, and the shoulder S, the linear axis AX1 that connects the palm P and the head H, and the palm P and the shoulder S. The conceptual diagram which determines the two linear axes of the connecting linear axis AX2, and determines the display direction of the button B which is an information input image from these two linear axes is shown. FIG. 16C shows a situation in which the detection unit 3 recognizes the palm P and the shoulder S, and the control unit 1 determines a linear axis AX2 connecting the palm P and the shoulder S, and an information input image is obtained from the linear axis. The conceptual diagram which determines the display direction of the button B which is is shown. FIG. 16D shows a situation in which the detection unit 3 recognizes the palm P and the head H, and the control unit 1 determines a linear axis AX1 that connects the palm P and the head H, and an information input image is obtained from the linear axis. The conceptual diagram which determines the display direction of the button B which is is shown.

  FIG. 16E shows that in the situation where the detection unit 3 recognizes the palm P and shoulder S, particularly both shoulders, the control unit 1 determines the linear axis AX3 connecting the shoulders, and bisects the linear axis AX3. Assuming that the point is the position of the head H, a linear axis AX1 that connects the palm P and the head H is determined, and a conceptual diagram for determining the display direction of the button B that is an information input image from the linear axis AX1 is shown. . This method is useful when it is difficult to recognize the head H with a hat, hairstyle, or the like.

  FIG. 16 (f) shows the case where the palm P is inside or outside both shoulders in the situation where the detection unit 3 recognizes the palm P, shoulder S, especially both shoulders. The display direction is changed depending on the case. In this example, when the palm P is inside the shoulders, the display direction is set so as to face the body. On the other hand, when the palm P is outside the shoulders, the same as in FIG. By this method, the linear axis AX3 and the linear axis AX1 are determined, and the display direction of the button B which is the information input image is determined from the linear axis AX1.

  In this process, as described above, the detection unit 3 detects various other human body parts in addition to the palm, and the control unit 1 detects the information input image based on the relative positional relationship between the palm and the other human body parts. Can be determined. Even if the detection unit 3 is not located immediately above the user, the detection unit 3 can be converted into a situation where it is spatially located directly above the user. The shoulder recognition method may be skeleton estimation or upper body mass estimation. Moreover, you may determine a display direction using an elbow part. Also, if the orientation of the human body is limited by the usage environment, such as a chair or a display, or if the orientation can be estimated by other than body recognition by detecting the rotation angle of the chair, etc., use that means. The display angle may be changed.

  FIG. 17 is a conceptual diagram showing a concept for defining a range in which an information input image is projected in the height direction. The control unit 1 calculates the relative distance (relative height) from the head H or shoulder S detected by the detection unit 3 to the palm P, and the range in which the projector 5 projects the information input image based on the relative distance. Is specified. For example, when the relative distance from the shoulder S to the palm P is used (shoulder reference), the information input image can be determined to be projected when the palm P is below the height of the shoulder S (distance D1). Further, it can be determined that the information input image is projected within a range where the palm P is 600 mm below the height of the shoulder S (distance D2). When the relative distance from the head H to the palm P is used (head reference), calculation is performed by adding 150 mm to the shoulder reference (distances D3 and D4).

  In this process, the angle of the direction for determining the height reference is not limited to the gravitational direction (vertical direction) as shown in FIG. 17, and for example, when the user sits on a reclining chair, the upper body tilts. It is also possible to calculate based on the angle. That is, the detection unit 3 detects a human body part other than the palm, and the control unit 1 determines a range for projecting the information input image based on the relative positional relationship between the palm and the other human body part. it can. In the present process, even if the detection unit 3 is not located directly above the user (such as a general monitoring camera), it can be spatially converted into the present process. When sitting on a clinching chair, the information input image is projected by recognizing or presetting the floor, chair seat height, etc. instead of the head H or shoulder S. The range may be defined, and the height standard is not limited.

  FIG. 18 is a conceptual diagram showing a processing concept when the irradiation light irradiated from the projector 5 and the palm P do not intersect perpendicularly, that is, when the projection angle for projecting the information input image to the palm P is not 90 °. is there. As shown in FIG. 18A, the detection unit 3 detects the tilt angle c of the palm P, and corrects the shape of the information input image by the method shown in FIG. 14B, for example, according to the tilt angle. To do.

Further, when the movement of the palm P is detected, the control unit 1 performs a decomposition operation in the movement direction using the inclination angle c. That is, the projection position is not changed by the movement of the palm P in the plane direction a, and the control unit 1 determines that the button B selection process is in the plane. On the other hand, when the palm P moves in the depth direction b, the projection position of the button B itself is changed (see FIG. 18B).
Even if the irradiation light emitted from the projector 5 and the palm P do not intersect perpendicularly by these correction treatments, the user can slide the palm P vertically with respect to the palm P as shown in FIG. The buttons displayed at the height change as indicated by the buttons B1 and B2.

  If the tilt angle c of the palm P is enlarged, the projected image becomes unclear and the hand recognition becomes difficult. Therefore, the tilt angle is set to 60 ° as the upper limit, and when the tilt angle is larger than that, the control unit 1 sets the tilt angle 60 °. Process using as the maximum angle. Further, although drawing / calculation processing is strictly performed with respect to the inclination angle c of the palm P, when flickering of calculation power / display occurs, calculation is performed in steps of, for example, 5 °, 10 °, and 15 °.

  Various actions (gestures) by the palm are conceivable and are not limited to specific ones including the above-described embodiment. For example, a gesture that spills an information input image such as a button projected on the palm of a predetermined object such as a table can be considered. The projected part of the button may be moved to the object to be spilled and displayed by changing the number and size of the information input images to be displayed.

  In addition, after the execution of processing by the electronic device 20 is started, a beam or the like may be separately projected so as to pay attention to the electronic device (display or the like). Specifically, the button B projected on the palm P may be moved in the direction of the electronic device, or the button B projected on the palm P may be displayed with an arrow indicating the direction of the electronic device. It may be.

  Further, the processing may be determined by an operation that tears a specific information input image, or the information input image may be changed to flow by tilting the palm.

  In the above-described embodiment, the example in which the operation is completed with only one hand has been described. However, it is also possible to make a decision by touching the information input image with the other hand. However, there may be a disadvantage that the palm of one hand is hidden by the finger of the other hand, and the difference between the height of the palm and the height of the finger cannot be measured accurately. In order to eliminate such inconvenience, for example, the unevenness of the palm surface is recorded before displaying the information input image, and three or more areas outside the area where the information input image is displayed and which do not touch the finger are displayed. Then, the observation point is detected, and the inclination of the palm is detected by measuring the observation point. The height of the surface of the palm can be sequentially corrected based on the recorded surface irregularities and the inclination of the palm. Further, the red blood of the palm may be observed with a VGA (Video Graphics Array) camera.

  Various relationships are conceivable regarding the configuration, position, and the like of the detection unit 3 and the projector 5. For example, it is conceivable to arrange the detection unit 3 at a position that does not cover the projection field angle of the projector 5 or to arrange the projector 5 at a position that does not cover the detection area of the detection unit 3. You may arrange | position the projection surface which projects the information input image of the projector 5, and the surface which the detection part 3 detects on the same surface.

  Further, the projector 5 and the detection unit 3 may be fixed with the same parts, and the optical axis of the projector 5 and the axis of the detection unit may be arranged in the same direction. Further, the aspect ratio of the detection unit 3 and the aspect ratio of the projector 5 may be matched.

  Further, as a calibration method for calibrating the projector 5 and the detection unit 3, for example, by displaying a diagonal point by some kind of calibration announcement, and by having the user hold the hand over the center of the two diagonal points, the detected position of the hand It is conceivable to collate and calibrate the projection position. Further, a deviation between the center of the pointer and the center of the palm may be recognized and adjusted by image recognition. Moreover, you may adjust with respect to a characteristic structure (chair etc.).

  According to the information input device of the present disclosure, when operating the electronic device, not only a special device such as a remote controller but also a dedicated projection target such as a screen for projecting an operation image is unnecessary, and the user Other electronic devices can be operated with only one hand, and simpler operation input is possible.

  Further, according to the information input device of the present disclosure, the information input image is basically projected on the palm, but the information input image can be projected without being limited by the size of the palm, and various electronic devices can be operated. Processing is possible.

  Further, according to the information input device of the present disclosure, the absolute position at which the information input image is projected can be defined by a specific timing, in particular, a specific palm operation (gesture) after the palm is detected. Therefore, an arbitrary position (space) can be set as the projection position, and a dedicated projection target is not required. Further, even if the user moves the palm, the projection position does not follow, so the user can move the palm to the option image of the information input image to be selected.

  Further, according to the information input device of the present disclosure, after moving a hand so that a specific option image is positioned at the center of the palm, selection of the option image is determined by a predetermined operation. Therefore, setting of the projection position and selection and determination of option images can be performed with the same hand, and the electronic apparatus can be easily operated with only one hand of the user. Further, unlike the conventional case, it is not necessary to perform a complicated detection process as in the case where the selection image is determined with the other hand, so that the determination process can be performed more reliably and easily. it can.

  Although the embodiment of the information input device according to the present disclosure has been described above with reference to the drawings, the present disclosure is not limited to the example. It is obvious for those skilled in the art that various modifications, modifications, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims. Of course, it is understood that it belongs to the technical scope of the present disclosure.

  The information input device of the present disclosure is useful for applications in which an electronic apparatus is easily and accurately operated without preparing a dedicated projection target.

DESCRIPTION OF SYMBOLS 1 Control part 3 Detection part 5 Projector (projection apparatus)
10 Information Input Device 20 Electronic Device B Button (Option Image)
F Fist H Head P Palm Q Finger base S Shoulder

Claims (12)

A detection unit capable of detecting the palm of a person,
A projection device that projects an information input image including a plurality of option images corresponding to a plurality of processes for operating an electronic device on a palm;
A control unit for controlling the detection unit and the projection device;
An information input device comprising:
The control unit stores an absolute position corresponding to the palm position detected by the detection unit at a predetermined timing;
The projection device projects the information input image based on the absolute position,
When the detection unit detects a predetermined action by the palm from a state where the predetermined option image is projected at the center of the palm, the control unit determines that the predetermined option image is selected, and the selected option Send processing corresponding to the image to the electronic device ,
The control unit generates a plurality of information input images stacked along a direction from the projection device to the absolute position, and the projection device stores a plurality of information input images stacked along the direction. An information input device that projects . The information input device according to claim 1,
The control unit generates a stacked state display image that displays a stacked state of a plurality of option images stacked along the direction,
The projection device is an information input device that projects the stacked state display image. The information input device according to claim 1 ,
The control unit generates stacking order information that displays the stacking order of each of the plurality of option images stacked along the direction,
The projection device is an information input device that displays the stacking order information in each of the option images. The information input device according to claim 1,
The information input device projects the information input image onto an area exceeding the size of a palm in one plane.   The information input device according to claim 1, wherein the projection device projects a position display image indicating where the palm is in the information input image onto the palm. The information input device according to claim 1,
The information input device, wherein the projection device stops projecting the information input image when the detection unit detects that a predetermined operation by a palm is performed. The information input device according to claim 1,
When the projection device projects the image onto the palm at an initial stage, at least the option image is projected outside the vicinity of the center of the palm. The information input device according to claim 1,
The detection unit detects a distance from the projection device to a palm;
The control unit is an information input device that determines a size of the information input image based on the distance. The information input device according to claim 1,
The detection unit detects a tilt angle of a palm relative to the projection device;
The control unit is an information input device that determines a shape of the information input image based on the tilt angle. The information input device according to claim 1,
The detection unit detects a distance from the projection device to a palm;
The control unit is an information input device that determines a focus of the projection device based on the distance. The information input device according to claim 1,
The detection unit detects a human body part other than the palm,
The said control part is an information input device which determines the direction of the said image for information input based on the relative positional relationship of a palm and another human body part. The information input device according to claim 1,
The detection unit detects a human body part other than the palm,
The said control part is an information input device which determines the range which projects the said image for information input based on the relative positional relationship of a palm and another human body part.

Images