JP2013152711A - Projector and display device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a convenient projector.SOLUTION: A projector includes a projection unit capable of projecting a projection image in an area corresponding to a position of a user, and a determination unit that determines if the projection image can be projected in the area.

Description

  The present invention relates to a projection device and a display device.

  Conventionally, in order to operate a device such as a lighting fixture, a virtual remote controller is projected at a position where the user's limbs can reach, and illumination is performed according to a user operation (pointing or the like) on the virtual key of the projected virtual remote controller. It has been proposed to control the instrument (for example, Patent Document 1).

JP 2011-9066 A

  However, the conventional proposals are limited to projecting the virtual remote controller at a position where the user's limbs can reach, and are not always easy to use.

  SUMMARY An advantage of some aspects of the invention is that it provides an easy-to-use projection apparatus and display apparatus.

  The projection apparatus according to the present invention includes a projection unit (10) capable of projecting a projection image onto an area corresponding to a user's position, a determination unit (70) that determines whether the projection image can be projected onto the area, It has.

  In this case, the determination unit may determine whether the projection is possible based on a detection result of a thermal sensor (31) that detects a physical quantity related to heat in the region. The determination unit may determine whether the projection is possible based on a detection result of a distance sensor (32) that detects a step of the region. In addition, a dangerous object detection unit (22) that detects the presence or absence of a dangerous substance based on an imaging result of an imaging unit that images the region, the determination unit based on a detection result of the dangerous substance detection unit, It may be determined whether or not projection is possible.

  The projection apparatus of the present invention may include an adjustment unit (15) that adjusts the projection unit based on an imaging result of an imaging unit that images the region. In this case, the adjustment unit can adjust at least one of the size, position, and luminance of the projected image. In addition, the adjustment unit can adjust the size of the projection image according to the number of people imaged by the imaging unit.

  In the projection device of the present invention, the projection unit can project operation menus of a plurality of electronic devices onto the area. In this case, a control unit (16) for projecting an operation menu reflecting the arrangement of the plurality of electronic devices onto the area may be provided. In this case, it has a communication part (60) which can communicate with the plurality of electronic devices, and the control unit detects the arrangement of the plurality of electronic devices according to the communication state with the plurality of electronic devices, and An operation menu reflecting the arrangement may be projected on the projection unit.

  The projection device of the present invention includes an imaging unit (21) capable of imaging a user, a projection unit (10) capable of projecting a projection image onto an area corresponding to the position of the user, and the projection unit And a determination unit (70) for determining whether the projected image can be captured by the imaging unit.

  In this case, an object detection sensor (32, 20) may be provided that detects whether or not there is an object that blocks the image capturing by the image capturing unit in a space including the image capturing unit and the region. In this case, the object detection sensor may have a distance sensor. The object detection sensor may have an image sensor. Further, an adjustment unit (15) may be provided that adjusts a position at which the projection image is projected when at least a part of the projection image projected by the projection unit cannot be captured by the imaging unit.

  Further, an actuator may be provided that drives the imaging unit when at least a part of the projected image projected by the projection unit cannot be captured by the imaging unit. Moreover, the said imaging part can image the said user and a structure, and the said determination part is good also as determining the said user's direction based on the imaging result of the said structure. Further, the imaging unit may be arranged to image the user from above the user. In this case, the determination unit may determine the posture of the user based on the imaging result of the imaging unit.

  The projection device of the present invention projects a detection unit (70) that detects the posture of the user from an imaging result of the imaging unit (21) that captures an image of the user from above, and projects near the user according to the posture of the user. A projection unit (10) for projecting an image.

  In this case, the detection unit can detect the user's posture by detecting the size of the user from the imaging result of the imaging unit.

  The projection device of the present invention includes a detection unit (70) that detects the position of the user, and a projection unit (10) that projects a menu related to operations of a plurality of electronic devices according to the detection result of the detection unit.

  In this case, a control unit (70) that projects the menu according to the arrangement of the plurality of electronic devices may be provided. The menu may be a hierarchical menu corresponding to each of the plurality of electronic devices, and the control unit may be shared with at least a part of the display of the hierarchical menu.

  The display device of the present invention includes a display unit (10) that performs display in an area corresponding to the position of the user, and a determination unit (70) that determines whether display in the area is possible.

  The display device of the present invention includes a detection unit (21, 70) that detects information about the height of the user, and a display unit that displays in the vicinity of the user according to the height of the user detected by the detection unit. (10).

  In addition, in order to explain the present invention in an easy-to-understand manner, the above description has been made in association with the reference numerals of the drawings representing the embodiments. However, the present invention is not limited to this, and the configuration of the embodiments described later is provided. May be modified as appropriate, or at least a part thereof may be replaced with another component. Further, the configuration requirements that are not particularly limited with respect to the arrangement are not limited to the arrangement disclosed in the embodiment, and can be arranged at a position where the function can be achieved.

  The present invention has an effect of providing a projection device and a display device that are easy to use.

It is a figure which shows typically the state of the living room where the projection and apparatus control system which concerns on 1st Embodiment is installed. 1 is a block diagram of a projection / device control system and an electronic device according to a first embodiment. FIGS. 3A to 3D are diagrams illustrating an example of an operation menu image according to the first embodiment. FIG. 4A is a diagram illustrating an example of a reference image, and FIG. 4B is a diagram illustrating an example of an image captured by the imaging unit. It is a flowchart which shows control of CPU of the projection and apparatus control system which concerns on 1st Embodiment. It is a figure for demonstrating the determination method of the projection area | region when an operation menu image cannot be imaged by an imaging part. It is a figure which shows typically the state of the place where the projection and apparatus control system which concerns on 2nd Embodiment is installed. It is a block diagram of a projection / device control system, a telephone set, and an electronic device according to a second embodiment. FIG. 9A and FIG. 9B are diagrams illustrating an example of an operation menu image according to the second embodiment. It is a flowchart which shows control of CPU of the projection and apparatus control system which concerns on 2nd Embodiment.

<< First Embodiment >>
Hereinafter, the first embodiment will be described in detail with reference to FIGS. FIG. 1 is a diagram showing a schematic configuration of a projection / device control system 100 according to the first embodiment.

  As shown in FIG. 1, the projection / device control system 100 is provided on the ceiling of a living room, and operates an electronic device 80 such as a television 210, an air conditioner 220, and a lighting device 230 on the upper surface or floor of a table 200. Project an operation menu image. In addition, when there is a user operation (gesture) on the operation menu image, the projection / device control system 100 executes control of the electronic device 80 according to the operation.

  FIG. 2 is a block diagram of the projection / device control system 100. As shown in FIG. 2, the projection / device control system 100 includes a projection unit 10, an image system unit 20, a sensor unit 30, a gesture recognition unit 40, a flash memory 50, a communication unit 60, a CPU 70, Is provided. Hereinafter, each part will be described in detail.

(Projector 10)
The projection unit 10 includes a light source 11, an illumination optical system 12, a light modulation element 13, a projection optical system 14, a projection image adjustment unit 15, and a menu display unit 16. The number of projection units 10 can be determined according to the size and layout of the living room.

  The light source 11 is, for example, a lamp that emits light. The illumination optical system 12 illuminates the light modulation element 13 with the light beam emitted from the light source 11. The light modulation element 13 is a liquid crystal panel, for example, and generates an image (an image (described later) based on image data input from the menu display unit 16) to be projected onto the upper surface of the table 200 or the like. The projection optical system 14 is an optical system that projects the light beam from the light modulation element 13 toward the upper surface or floor of the table 200, and includes a zoom lens that adjusts the size of an image to be projected, and a focus lens that adjusts the focus position. Have Note that the position of the lens included in the zoom lens or the focus lens is adjusted by an actuator (not shown).

  The projection image adjustment unit 15 determines an area (size) and a distance of an image to be projected toward the table 200 and the floor based on the imaging result of the image system unit 20 and the detection result of the sensor unit 30. . The projection image adjusting unit 15 controls an actuator (not shown) according to the determined image area (size) and distance, and adjusts the position of the lens included in the zoom lens and the focus lens of the projection optical system 14. Further, the projection image adjustment unit 15 adjusts the contrast processing, the adjustment (change) of the color of the menu to be projected, and the luminance of the light source 11 according to the color and brightness of the upper surface of the table 200 and the floor. The projection image adjustment unit 15 also performs settings related to the contents of an image to be displayed (icon arrangement, etc.).

  The menu display unit 16 projects the operation menu image for operating the electronic device 80 toward the upper surface of the table 200 or the floor on the projection optical system 14 based on the imaging result of the image system unit 20 or the detection result of the sensor unit 30. Display through.

  FIGS. 3A to 3D show an example of the operation menu image 17 that the menu display unit 16 displays via the projection optical system 14. The operation menu image 17 displayed by the menu display unit 16 has a hierarchical structure (a structure in which the images of FIGS. 3A to 3D are displayed according to the operation).

  FIG. 3A shows an initial menu image (an operation menu image to be projected first). The initial menu image in FIG. 3A includes an icon 171 for selecting the electronic device 80 to be operated, an icon 172 for enlarging / reducing the image, and the end of projection (meaning “return”). An icon 173 is included. In FIG. 3A, as icons 171 for selecting the electronic device 80, icons of lighting devices, air conditioners, and televisions corresponding to the devices arranged in the living room of FIG. 1 are displayed. In FIG. 3A, icons are displayed according to the room layout (vertical relationship) so that the user can select the electronic device 80 intuitively. Specifically, the lighting device 230 is displayed at the top, then the air conditioner 220, and the icon of the television 210 is displayed at the bottom. In this case, when the position of the electronic device 80 can be identified from a general layout (a layout in which lighting is often at the top, an air conditioner is below the lighting, a television is near the floor, etc.) And sometimes cannot be identified. If the position of the electronic device 80 cannot be specified, the position of each electronic device 80 may be registered in the flash memory 50 in advance. Further, as described later, the position may be specified from the result of communication with the electronic device 80. Note that the CPU 70 sets the icon arrangement and the like.

  FIG. 3B is an operation menu image displayed when the icon of the lighting device is selected in the initial menu image of FIG. The operation menu image of FIG. 3B includes a display of lighting equipment, an enlargement / reduction icon 181, an ON / OFF switch icon 182, an up / down switch icon 183, and a back icon 184. ing. The up / down switch icon 183 is an icon for adjusting the illuminance of the lighting device 230.

  FIG. 3C is an operation menu image displayed when an air conditioner is selected. The operation menu image in FIG. 3C includes an air conditioner display, an enlargement / reduction icon 191, an ON / OFF switch icon 192, an up / down switch icon 193, a return icon 194, a mode change (cooling) , Heating, dehumidifying, etc.) icons 195 are included. The up / down switch icon 193 is an icon for performing adjustment (change of set temperature, set humidity, etc.) according to the set mode.

  FIG. 3D is an operation menu image displayed when the television is selected. The operation menu image shown in FIG. 3D includes a television display, an enlargement / reduction icon 161, an ON / OFF switch icon 162, an up / down switch icon 163, a return icon 164, a mode change ( An icon 165 indicating the channel, volume, etc.) is included. The up / down switch icon 163 is an icon for performing adjustment (channel change, volume setting, etc.) according to the set mode.

  As shown in FIGS. 3B to 3D, the operation menu image of each electronic device 80 displayed by the menu display unit 16 is a part of the operation menu image (for example, an enlargement / reduction icon, an ON / OFF switch, etc.). Icon, up / down switch icon, and icon indicating return). Thus, the user can operate without a sense of discomfort even when the type of the selected electronic device 80 is different.

(Image system unit 20)
Returning to FIG. 2, the image system unit 20 recognizes whether or not the user in the living room requests the projection of the operation menu image, or images the inside of the living room before the projection unit 10 projects the operation menu image 17. The location where the operation menu image 17 should be projected is determined. As shown in FIG. 2, the image system unit 20 includes an imaging unit 21 and an image comparison unit 22.

  The imaging unit 21 includes a photographic lens and an imaging device, and images structures and users in the living room from the ceiling of the living room. A wide-angle lens corresponding to the size of the living room (for example, a wide-angle lens capable of imaging the entire living room) may be used as the photographing lens. Note that the number of the imaging units 21 is not limited to one, and may be a number (plural) according to the size of the living room. Moreover, you may image the inside of a living room by driving the imaging part 21 two-dimensionally with an actuator not shown. As the imaging device, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) can be employed. The image sensor has a color filter in which RGB three primary colors are arranged in a Bayer array, and can obtain color signals corresponding to the respective colors. The imaging by the image system unit 20 may be a still image, a moving image, or a combination thereof.

  The imaging unit 21 may be provided integrally with the projection unit 10 or may be provided independently. In any case, it is only necessary to know the mechanical positional relationship between the imaging unit 21 and the projection unit 10, and the positional relationship may be stored as a system constant in the flash memory 50 or the like.

  The image comparison unit 22 compares the living reference image (see FIG. 4A) with the image captured by the imaging unit 21 (see FIG. 4B). The living reference image is an image stored in the flash memory 50, for example, an image of the entire living room. It should be noted that the reference image may be changed in accordance with the layout change or seasonal pattern change. In this case, the reference image may be changed under the user's instruction. For example, when a change in season is detected based on a change in air conditioner settings (switching between heating and cooling), the reference image is automatically changed. You may make it update. The flash memory 50 also stores various images of dangerous objects (cutters, knives, pots, kettles, etc.).

  The image comparison unit 22 compares the image captured by the imaging unit 21 with the reference image, and determines whether the image captured by the imaging unit 21 is the same as the reference image. In addition, the image comparison unit 22 compares the image captured by the imaging unit 21 with various images of dangerous materials, and whether or not the captured image includes a dangerous material image and where is the dangerous material. Determine.

  When the projection unit 10 projects the operation menu image 17 based on the position of the user imaged by the imaging unit 21, whether the projection location should be a floor or a table depending on whether the user is standing or sitting. Is different. In order to cope with such a case, as described in JP 2011-141730 A, the CPU 70 has the size of the head in the captured image (changes according to the distance between the imaging unit 21 and the user's head). Based on the above, it may be determined whether the user is standing or sitting. Specifically, when the user is standing, the distance between the imaging unit 21 and the user's head is closer, so the user's head is imaged larger. On the other hand, when the user is sitting, the distance between the imaging unit 21 and the user's head is greater than when the user is standing. Therefore, the CPU 70 determines the posture of the user according to the size of the head imaged by the imaging unit 21, and the projection unit 10 projects the operation menu image 17 on either the floor or the table based on the determination result. You just have to do it. Note that information such as the user's height and head size may be stored in the flash memory 50, and the CPU 70 may determine whether the user is standing or sitting using the information. Note that the CPU 70 may determine whether the user is standing or sitting based on the width of the shoulder width of the user in the captured image instead of the size of the head.

(Sensor part 30)
As shown in FIG. 2, the sensor unit 30 includes a temperature (heat amount) detection unit 31 and a distance sensor 32.

  The temperature detection unit 31 includes, for example, an infrared sensor, an actuator that drives the infrared sensor in a two-dimensional manner to detect a temperature (amount of heat) in a predetermined area of the living room, and a temperature that is greater than or equal to a predetermined value among the temperatures detected by the infrared sensor And a set temperature detection unit for detecting temperature (for example, 40 degrees or more). In the first embodiment, the temperature detection unit 31 detects a temperature higher than a predetermined temperature in the living room, for example, when a high-temperature pan is placed on a table, This is to prevent the operation menu image 17 from being projected in the vicinity.

  Note that the temperature detection unit 31 may detect a human surface temperature to detect a place where the user is present. In this case, the projection / apparatus control system 100 may be activated from the time when it is detected that the user is in the living room. However, when such activation control is performed, it is necessary to turn on the infrared sensor according to the opening / closing operation of the living room door or to always drive the infrared sensor.

  The distance sensor 32 includes, for example, an ultrasonic sensor, and detects whether or not an object such as a small portable device or a toy is placed on the floor or table. The reason why the step detection is performed in the first embodiment is to prevent the projection unit 10 from projecting the operation menu image on a portion having a step.

  Note that the above-described image comparison unit 22 may detect the level difference by comparing the reference image (an image without a level difference) with the image captured by the imaging unit 21. However, when the table cloth on the table 200 is replaced, even a portion having almost no step may be detected as a step. For this reason, in the first embodiment, a portion where there is a difference between the image captured by the imaging unit 21 and the reference image is detected, and the distance sensor 32 detects whether there is a step in the difference portion. And In addition, various heights, such as 5 mm or more or 10 mm or more, can be employ | adopted as the height of the level | step difference to detect.

(Gesture recognition unit 40)
The gesture recognition unit 40 recognizes a user's gesture in cooperation with the imaging unit 21. The gesture recognizing unit 40 recognizes whether or not the user requests projection of the operation menu image, and recognizes the operation state of the user on the operation menu image 17.

  For example, when the gesture recognition unit 40 acquires an image captured by the imaging unit 21 and detects the user's face in the image, the gesture recognition unit 40 determines that the operation menu image is requested to be projected. Such a determination can be made because the imaging unit 21 is provided on the ceiling as described above, and thus the imaging unit 21 can usually capture the face of the user while imaging the face of the user. It is because it is not possible. The gesture recognizing unit 40 may recognize that the user is requesting the projection of the operation menu when the user performs an operation of pointing the image capturing unit 21. The gesture recognizing unit 40 may recognize that the user requests the projection of the operation menu by a combination of the face detection and the pointing operation.

  On the other hand, the gesture recognizing unit 40 acquires an image captured by the image capturing unit 21, and a user's hand or finger is placed on the operation menu image for a predetermined time (for example, from 1 second) by color recognition (skin color recognition or the like) in the image. 5 seconds), the user's operation on the operation menu image 17 is recognized. In addition, a moving image is captured after the user's hand or finger is positioned on the operation menu image or a still image is continuously captured, and the user's gesture is recognized from the change (trajectory) of the position of the hand or finger. Good. Note that the user's hand or finger may directly touch the operation menu image, but if there is a part of the user's body in the projection area of the operation menu image, it is recognized that the user is operating. It may not be in contact.

  An image of the user's hand may be stored in the flash memory 50. The gesture recognition unit 40 may recognize the position of the user's hand using the image of the user's hand.

  The gesture recognition unit 40 determines the amount of operation of the up / down switch icons 183, 193, and 163 shown in FIGS. 3B to 3D according to the number of fingers of the user on the icons. It is good. For example, assume that the mode is set to volume in the television operation shown in FIG. In this case, if three fingers are on the up switch icon, the gesture recognizing unit 40 may recognize that the operation is to increase the volume by three levels.

(Flash memory 50)
The flash memory 50 is a non-volatile memory, such as a program executed by the CPU 70, information on the layout of the living room, system constants such as a mechanical positional relationship between the imaging unit 21 and the projection unit 10, reference images, dangerous objects, and the like. Various image data, user information, information on electronic devices 80 (such as lighting devices, televisions, and air conditioners) that communicate with the communication unit 60 are stored.

(Communication unit 60)
The communication unit 60 includes a wireless communication unit for communicating with the electronic device 80 (the television 210, the air conditioner 220, the lighting device 230, and the like), and the electronic device 80 receives the gesture recognized by the gesture recognition unit 40. Various control commands are sent to it. The communication unit has a wireless module and an antenna. It is assumed that the antenna can be rotated and tilted by an actuator (not shown). Since the antenna can be rotated and tilted, the approximate position of each electronic device 80 can be recognized based on the radio wave intensity from each electronic device 80 received by the rotated and tilted antenna. ing. For example, when a new electronic device 80 is installed in a living room, the position of the new electronic device 80 can be recognized. Instead of rotating the antennas, a plurality of antennas may be provided, and the approximate position of each electronic device 80 may be recognized based on the radio field intensity of the plurality of antennas.

(CPU 70)
The CPU 70 comprehensively controls the electronic apparatus 80 in addition to the projection / apparatus control system 100. In the first embodiment, the operation menu image is projected and the user recognizes the gesture on the operation menu image. The electronic device 80 corresponding to the gesture is controlled.

  As shown in FIG. 1, the electronic device 80 is a television 210, an air conditioner 220, a lighting device 230, and the like. The configuration of each electronic device 80 is various, but as shown in FIG. 60, a communication unit 81 for communicating with 60, a CPU 82 that controls the electronic device 80 according to a communication result of the communication unit 81, and a controlled unit 83 that is controlled by the CPU 82. Note that the communication unit 81, the CPU 82, and the controlled unit 83 are appropriately configured according to each electronic device 80 (the TV 210, the air conditioner 220, and the lighting device 230).

  Hereinafter, control by the CPU 70 of the projection / device control system 100 configured as described above will be described in detail with reference to the flowchart of FIG. FIG. 5 shows the operation of each part of the projection / apparatus control system 100 under the control of the CPU 70.

  In step S <b> 10 of FIG. 5, the CPU 70 determines whether there is a user in the living room based on the output of the temperature detection unit 31. If the determination here is affirmed, the process proceeds to step S12.

  If transfering it to step S12, CPU70 will control the imaging part 21 and will start a user's imaging. In addition, when the image pick-up by the image pick-up part 21 is image pick-up with a still image, as long as the user continues to exist in the living room, the image may be picked up almost continuously, or picked up at predetermined intervals (2-3 seconds). May be performed. On the other hand, when the operation menu image is projected by the projection unit 10, it is desirable to capture images almost continuously or at a predetermined interval. Similarly, in the case of imaging by moving images, it is desirable to increase the frame rate when the operation menu is projected.

  Next, in step S14, the CPU 70 determines whether or not there has been a projection request for an operation menu image by the user. The CPU 70 instructs the gesture recognizing unit 40 to recognize whether the user has performed a specific operation from the image captured by the image capturing unit 21. Then, the CPU 70 determines that there is a projection request when the gesture recognition unit 40 recognizes a specific operation. As described above, the specific operation includes an operation of looking at the ceiling, an operation of pointing, or a combination of these operations.

  At the same time as the determination in step S14, the CPU 70 can confirm the position and orientation of the user based on the image captured by the imaging unit 21. The CPU 70 can confirm the user's posture from the size of the head in the image as described above. In the first embodiment, it is assumed that the user is sitting on a chair and is present near the table.

  If the determination in step S14 is negative, that is, if there is no request from the user, step S14 is repeated, but if there is a request from the user and the determination in step S14 is positive, Control goes to step S16.

  If transfering it to step S16, CPU70 will acquire the information of the vicinity where a user is located from the sensor part 30. FIG. In this case, the CPU 70 acquires the detection result of the temperature detection unit 31 and detects whether there is a region where the temperature exceeds 40 ° C. near the table 200. In addition, when there is a difference between the image captured by the imaging unit 21 and the reference image stored in the flash memory 50, the CPU 70 determines whether there is a step in the difference, that is, a table. Whether there is a dish or the like on 200 is determined from the detection result of the distance sensor 32. Furthermore, the CPU 70 compares the image captured by the image capturing unit 21 with various images of dangerous objects stored in the flash memory 50, and creates a risk such as a fork, knife, cutter knife, scissors, knife, etc. on the table 200. Determine if there is something.

  Next, in step S <b> 18, the CPU 70 determines the projection condition of the operation menu image by the projection unit 10. In this case, the CPU 70 extracts a candidate area for projecting the operation menu from the information detected in step S16. Here, description will be made assuming that it is determined that there is no dangerous object or step near the table and that the operation menu can be displayed anywhere on the table.

  Here, when there are a plurality of areas where the operation menu can be displayed, the CPU 70 gives priority to the area on the front or right side of the user to display the operation menu image. This is because many users are right-handed, and thus the operation menu is displayed on the front or right side of the user, thereby facilitating the operation. If the gesture recognition unit 40 recognizes that the user frequently performs left-handed operations (left-handed), the CPU 70 may project the operation menu image on the left side of the user. When it is recognized that the user is left-handed, the size of the head, the shoulder width, and information indicating that the user is left-handed may be stored in the flash memory 50 in association with each other. In this case, the CPU 70 determines whether or not the user in the image is left-handed from the size and width of the head detected from the image captured by the imaging unit 21 and the information stored in the flash memory 50. May be.

  Here, the orientation of the user may be determined from an image captured by the imaging unit 21, and when sitting on a chair, the direction opposite to the backrest of the chair may be determined as the front of the user. . When the user is in the vicinity of the television, the user is normally facing the television, so the direction in which the television is present as viewed from the user may be determined as the user's orientation. If the user is moving, the moving direction can be determined as the user's direction.

  The projection image adjustment unit 15 adjusts the zoom lens of the projection optical system 14 so that the operation menu image 17 is projected on the table 200 with a size of about 150 mm × 150 mm or 100 mm × 200 mm by default. In this case, the distance from the projection optical system 14 to the projection surface may be detected by the distance sensor 32.

  In addition, the projection image adjustment unit 15 adjusts the color of the operation menu image according to the color signal from the image sensor of the imaging unit 21. In this case, the visibility of the operation menu image may be improved by making the outer edge of the operation menu image darker. Instead of this, when there are a plurality of candidate areas for projecting the operation menu image, the CPU 70 selects a white system (for example, a white table cloth) or an achromatic system (for example, an achromatic system) according to the color signal from the image sensor. The area of the colored carpet) may be determined as the projection area. This also improves the visibility of the operation menu.

  The projected image adjustment unit 15 may improve the visibility of the operation menu by adjusting the luminance of the light source 11 according to the time or the illuminance of the lighting device. Alternatively, an illuminometer may be provided in the living room, and the luminance of the light source 11 may be adjusted according to the detection result of the illuminometer, or the illuminance of the lighting device 230 may be adjusted. On the contrary, when the operation menu image 17 is projected by the projection unit 10, the visibility of the operation menu may be improved by dimming the illumination of the portion where the operation menu image is projected.

  When there is a dangerous object or a step in the region where the operation menu image is projected, the projection image adjustment unit 15 adjusts the size of the operation menu image based on the position or step of the dangerous object, and Do not accept expansion. For example, in the case of FIG. 4B, the operation menu image is adjusted so as to avoid the pan 132 which is a dangerous material and the box 136 which forms a step, and the table cloth 134 (not a dangerous material and no step is formed). Is not considered in the adjustment of the operation menu image.

  After the process of step S18 of FIG. 5 is performed as described above, the process proceeds to step S20, and the CPU 70 instructs the projection unit 10 based on the projection condition determined in step S18. Then, the projection unit 10 projects an operation menu image on the table 200 according to the instruction. Here, it is assumed that the operation menu image (initial menu image) of FIG. 3A is projected on the table 200 as the initial operation menu image. The CPU 70 may generate a sound such as “projected operation menu screen” from a speaker (not shown) in order to notify the user that the operation menu image has been projected.

  Next, in step S <b> 22, the CPU 70 starts measuring time to measure the time during which the operation menu image is projected by the projection unit 10.

  Next, in step S24, the CPU 70 determines whether or not there has been a gesture by the user (operation by the operation menu image). If the determination here is affirmed, the process proceeds to step S30. If the determination is negative, the process proceeds to step S26.

  When the process proceeds to step S26, it is determined whether or not a predetermined time (gesture reception waiting time (for example, 10 seconds)) has elapsed since the operation menu image was projected (after the start of timing). When judgment here is denied, it returns to step S24. On the other hand, if the determination in step S26 is affirmative, that is, if there is no user gesture for 10 seconds or more, the process proceeds to step S28, and the projection of the operation menu image is terminated because the user does not intend to perform the menu operation. Then, the entire process of FIG. The waiting time for accepting the gesture in step S26 is not limited to 10 seconds, and can be set as appropriate.

  On the other hand, when a gesture by the user (operation by the operation menu image) is performed and the determination in step S24 is affirmed, in step S30, the CPU 70 issues an instruction to the gesture recognition unit 40, and the gesture recognition unit 40 is instructed by the user. Recognize gestures made by. Here, as described above, since the operation menu image of FIG. 3A is projected by the projection unit 10, the gesture recognition unit 40 determines which icon of the operation menu image of FIG. Or part of the body). For example, when the imaging unit 21 captures an image of a state in which the user's hand is placed on the television icon in FIG. 3A, the gesture recognition unit 40 recognizes that the user has selected an operation on the television. The gesture recognizing unit 40 recognizes that the user has made a gesture when the imaging unit 21 has captured a finger on the same icon for a predetermined number (2 to 5) or a predetermined time (1 to 3 seconds). Good.

  Next, in step S32, the CPU 70 ends the timing started in step S22. Next, in step S34, the CPU 70 determines whether it is necessary to display the next operation menu image. Here, since the television is selected from the initial menu image, the CPU 70 determines that the next operation menu image needs to be displayed, and returns to step S20. Thereafter, the CPU 70 repeats steps S20 to S34 until the operation on the television is determined, and proceeds to step S36 when the operation on the television is determined (the determination in step S34 is negative).

  In step S 36, the CPU 70 communicates the operation command to the communication unit 81 of the television 210 through the communication unit 60 based on the television operation (for example, changing the channel) selected from the operation menu image. The CPU 82 of the television 210 controls the television 210 based on the operation command received by the communication unit 81.

  After that, the CPU 70 returns to step S24, and when there is a user gesture, the CPU 70 executes a process according to the gesture, and when the process by the gesture is finished (when the determination at step S26 is affirmed). After step S28, all the processes in FIG.

In step S18, the projection position of the operation menu image may be determined from the viewpoint of whether the imaging unit 21 can capture the projected operation menu image. FIG. 6 is a diagram showing a state in which an object (bottle) 110 is placed on the table 200. In this case, since there is no object (bottle) 110 in the reference image (see FIG. 4A), the distance to the object is detected by the distance sensor 32.
Here, in the case of FIG. 6, it is assumed that there are regions where an operation menu image can be projected on the left and right of the object (bottle) 110. However, depending on the positional relationship between the imaging unit 21 and the object (bottle) 110 included in the projection / device control system 100 and the height of the object (bottle) 110, the operation menu image is projected on the right side of the object (bottle) 110. In some cases, the imaging unit 21 cannot capture an operation menu image.

  In such a case, the CPU 70 projects an operation menu image on an area on the left side of the object (bottle) 110 based on the positional relationship between the imaging unit 21 and the object (bottle) 110. In addition, when the height of the object is low and the operation menu image can be captured by the imaging unit 21, the operation menu may be projected on the right side of the object. In this case, the CPU 70 may determine whether the operation menu image can be captured by the imaging unit 21 from the height of the object detected by the distance sensor 32 from the table surface, the mounting position and the angle of view of the imaging unit 21. It is also possible to actually project the operation menu image on the right side of the object by the projection unit 10 and test whether the operation menu image can be captured by the imaging unit 21. When the operation menu image cannot be captured by the image capturing unit 21, an actuator that drives the image capturing unit 21 is provided in the image system unit 20 in advance, and the position and orientation of the image capturing unit 21 are adjusted by the actuator to perform the operation. You may test whether you can capture a menu image.

  The CPU 70 may confirm whether or not the projected operation menu image can be captured by the imaging unit 21 in the next step S20. When the operation menu can be imaged by the imaging unit 21, the CPU 70 proceeds to the next step S22, and when a part (about 10% to 25%) cannot be imaged, the projection position by the projection unit 10 is adjusted, for example, the operation If more than half of the menu cannot be imaged, the process may return to step S18.

  Although not described in the flowchart of FIG. 5, when the user operates the enlargement / reduction icon of the operation menu image 17, the projection image adjustment unit 15 determines in step S <b> 30 according to the user's operation. It is assumed that the size of the operation menu image 17 is changed.

  As described above in detail, according to the first embodiment, the projection unit 10 projects the operation menu image 17 in an area corresponding to the position of the user, and the CPU 70 responds to the position of the user. Since it is determined whether or not the operation menu image 17 can be projected onto the area, the operation menu image can be projected at a position suitable for projection according to the position of the user. Thereby, the usability of the operation menu image is improved.

  In the first embodiment, the CPU 70 determines whether or not the operation menu image 17 can be projected based on the detection result of the temperature detection unit 31 for detecting the temperature (heat amount) of a predetermined area in the living room. Therefore, it is possible to prevent the operation menu image from being projected on an object having a high temperature. Thereby, an operation menu image can be projected on a safe position.

  In the first embodiment, the CPU 70 determines whether or not the operation menu image 17 can be projected based on the detection result of the distance sensor 32 that detects the step of the region. It is possible to prevent the image 17 from being projected. As a result, the operation menu image 17 can be projected at a convenient position.

  In the first embodiment, the image comparison unit 22 detects the presence / absence of a dangerous substance based on the imaging result of the imaging unit 21, and the CPU 70 projects the operation menu image 17 based on the detection result. Since it is determined whether or not it can be performed, it is possible to prevent the operation menu image from being projected onto the dangerous object. As a result, the operation menu image can be projected to a safe position.

  In the first embodiment, since the projection image adjustment unit 15 adjusts the projection optical system 14 based on the imaging result of the imaging unit 21, the operation menu image is projected at an appropriate position, size, and luminance. can do.

  In the first embodiment, since the projection unit 10 projects the operation menu images of the plurality of electronic devices 80, it is possible to operate the plurality of electronic devices 80 using the operation menu images. .

  In the first embodiment, the communication unit 60 that can communicate with the plurality of electronic devices 80 is provided, and the menu display unit 16 is arranged according to the communication state with the plurality of electronic devices 80. And the operation menu image reflecting the arrangement is projected on the projection unit. Thereby, it is possible to automatically detect the arrangement of the plurality of electronic devices 80 according to the communication state, and it is possible to improve the usability of the user by projecting the operation menu image reflecting the arrangement. It becomes.

  In the first embodiment, the CPU 70 determines whether the operation menu image projected by the projection unit 10 can be captured by the imaging unit 21, so that the operation menu image is displayed at an appropriate position (user operation (gesture)). Can be projected to a position where the image can be recognized.

  In the first embodiment, the CPU 70 uses the distance sensor 32 and the image system unit 20 (image comparison unit 22) to detect whether there is an object that blocks the imaging of the operation menu image by the imaging unit 21. To do. Thus, before the operation menu image is projected, an area where the operation menu image can be captured by the imaging unit 21 can be determined, and the operation menu image can be projected onto the area.

  In the first embodiment, the imaging unit 21 images the user and a structure such as the table 200 and the television 210, and the CPU 70 determines the orientation of the user based on the imaging results of the table 200 and the television 210. Is going to judge. Thereby, it is possible to determine the orientation of the user without confirming the shape of the user's body.

  In the first embodiment, since the imaging unit 21 captures the user from above the user, various controls using the imaging result of the imaging unit 21 can be performed while protecting the user's privacy. is there.

  In the first embodiment, the CPU 70 detects the user's posture from the imaging result of the imaging unit 21 that images the user from above, and the projection unit 10 operates in the vicinity of the user according to the user's posture. Since the menu image 17 is projected, the operation menu image 17 can be projected at a position convenient for the user. Further, in the first embodiment, the projection unit 10 projects the operation menu images 17 of the plurality of electronic devices 80 according to the user's position, so that the plurality of electronic devices 80 are located at positions that are convenient for the user. The operation menu image 17 can be projected.

  In the first embodiment, a case has been described in which the CPU 70 determines based on the movement of the face or hand that the user has requested the operation menu image projection in step S14. However, the present invention is not limited to this, and the CPU 70 may determine that there is a projection request when a specific sound is input from the user to the microphone.

  In the first embodiment, when there are a plurality of people in the living room, the operation menu image may be displayed in a large size so that the plurality of people can operate the operation menu image. In this case, the operation menu image may be displayed on the front or right side of the person who made the projection request. It should be noted that the operation menu image may be operated by all or only a specific person (for example, the person who requested the projection). When only the person who has made the projection request can perform the operation, the CPU 70 may determine whether or not it is the hand of the person who has requested the projection from the direction of the hand or the direction in which the finger is pointed. In the first embodiment, the child may be restricted from performing an operation using the operation menu image. In this case, the CPU 70 may determine whether or not it is a child's hand from the size and shape of the hand.

  Further, for example, a liquid crystal display device may be used for displaying the operation menu image. Specifically, the surface of the table 200 may be a liquid crystal display screen, and the CPU 70 may display a plurality of operation menus on the liquid crystal display screen when the user makes the above gesture or voice. Also in this case, the sensor unit 30 may detect whether or not there is anything blocking the display on the liquid crystal display screen, and the display area of the liquid crystal display screen may be selected based on the detection result. Note that a weight sensor may be provided on the table 200 to detect whether something is placed on the table 200. Alternatively, a touch panel may be mounted on the liquid crystal display screen and a menu may be selected by a touch operation. This liquid crystal display screen may be provided on a wall or a window glass, or may be used in combination with the projection unit 10 as appropriate. When using the liquid crystal display screen provided on the wall or the window glass, the CPU 70 reaches the user's hand by analogizing the posture and height of the user according to the size of the user's head imaged by the imaging unit 21. Display on the liquid crystal display screen may be performed at an easy position (if the liquid crystal screen includes a plurality of divided areas, the divided areas that are easily reachable by the user may be selected and displayed). In addition, it is good also as using the projection part 10 for the display on a wall or a window glass (curtain).

  Further, the projection unit 10 may be provided below the table 200 and an operation menu image may be displayed on the upper surface of the table 200.

  In the first embodiment, the case where the electronic device 80 is a device in the living room (such as the television 210, the air conditioner 220, and the lighting device 230) has been described. However, the present invention is not limited to this, and communication with the communication unit 60 is performed. If possible, the electronic device 80 may be arranged in a room different from the living room. In this way, it is possible to perform operations such as turning off the lighting device in the adjacent room while in the living room.

  In the above-described embodiment, the case where the projection / device control system 100 is applied in the living room has been described. However, the present invention is not limited to this. For example, the projection / apparatus control system 100 can be applied in a commercial facility. For example, when the projection / equipment control system 100 is provided in a restaurant, the menu may be projected onto the table when a customer is seated or when a request is made by a gesture. Further, the product selected by the customer may be registered in a POS system or the like.

<< Second Embodiment >>
Next, a projection / apparatus control system 100 ′ according to the second embodiment will be described with reference to FIGS. The projection / device control system 100 ′ of the second embodiment projects a menu image for supporting the user to the user who uses the telephone, and electronically operates based on an operation (gesture) on the menu image by the user. It controls equipment.

  FIG. 7 is a diagram showing a schematic configuration of a projection / apparatus control system 100 ′ according to the second embodiment. FIG. 7 shows a telephone 90 provided with a face imaging unit 94 in addition to the projection / device control system 100 ′.

  FIG. 8 is a block diagram of the projection / device control system 100 ′ and the telephone 90. In the following, description will be made centering on differences from the projection / apparatus control system 100 (FIG. 2) according to the first embodiment. In addition, the same code | symbol is attached | subjected to the same or equivalent structure as the block diagram of FIG. 2, and the description shall be abbreviate | omitted.

  As shown in FIG. 8, in the second embodiment, the telephone 90 can communicate with the projection / apparatus control system 100 ′, and the projection / apparatus control system 100 ′ A facial expression detection unit 72. In addition, the electronic device 80 that is a target to be controlled by the projection / device control system 100 ′ is a door phone 85, a gas stove 86, and a recorder 87.

(Telephone 90)
The telephone 90 includes a telephone function unit 91, an answering machine function unit 92, an interface unit 93, a face imaging unit 94, and a communication unit 95.

  The telephone function unit 91 realizes functions that a general telephone should have, such as an outgoing / incoming function, a microphone, and a speaker function. In the second embodiment, the user's voice input to the microphone is transmitted to the projection / apparatus control system 100 ′ via the communication unit 95.

  The answering machine function unit 92 controls starting and stopping of the answering machine function and has a message function. Further, the answering machine function unit 92 shifts to the answering machine function even during a call by the user selecting an icon of the answering machine function projected by the projection / device control system 100 ′ as will be described later. Or send a message.

  The interface unit 93 is an interface for performing voice input / output, LED lighting, and various displays in cooperation with various switches and buttons of the telephone 90.

  The face imaging unit 94 is an imaging unit for imaging a user's facial expression. In FIG. 1, the face imaging unit 94 is mounted on the telephone 90 so that the user's facial expression can be easily imaged. However, the face imaging unit 94 is not limited to this and may be arranged at a location other than the telephone if it can capture the user's facial expression. Good. A plurality of face imaging units 94 may be arranged.

  The communication unit 95 can employ various communication methods such as wired and wireless, and performs two-way communication with the communication unit 60 of the projection / device control system 100 ′. The communication unit 95 transmits the user's voice input to the microphone of the telephone function unit 91 and the user's facial expression captured by the face imaging unit 94 to the projection / device control system 100 ′.

(Speech analysis unit 71)
The voice analysis unit 71 analyzes the user's voice input to the microphone of the telephone function unit 91. The speech analysis unit 71 has a speech recognition dictionary, and the input speech is honorific, polite, humility, everyday language (plain language), or other broken words. Classify. In the present embodiment, the voice analysis unit 71 is provided on the projection / device control system 100 ′ side, but may be provided on the telephone 90 side.

(Facial expression detection unit 72)
The facial expression detection unit 72 detects the facial expression of the user imaged by the facial imaging unit 94, and detects a facial expression with a eyelid between the eyebrows or a facial expression with a narrowed eye rather than a smile, and is the user confused? Determine if.

  Here, the facial expression detection unit 72 stores an image with a eyelid between the eyebrows in the flash memory 50 as a reference image, and detects the eyelid between the eyebrows by performing pattern matching between the reference image and the captured image. Can do. Further, the facial expression detection unit 72 may detect the eyebrows between the eyebrows from the shadow distribution of the portion between the left eye and the right eye. Note that detection of eyelids between eyebrows is also disclosed in US Patent Application Publication No. 2008-292148.

  The facial expression detection unit 72 stores an average eye size captured by the face imaging unit 94 in the flash memory 50, and the face imaging unit 94 captures an image of the stored eye size. By comparing the sizes of the eyes, it is possible to detect facial expressions that narrow the eyes.

  Note that, as described in Japanese Patent Laid-Open No. 2002-258682, whether or not the user is confused by detecting the amount of facial muscle contraction from the feature point coordinate change of the facial component in the user's moving image. It may be determined.

(Electronic device 80)
The door phone 85 can communicate with the projection / apparatus control system 100 ′ by a communication unit (not shown). The door phone 85 has a function of generating a pseudo chime sound in response to an instruction from the CPU 70 of the projection / device control system 100 ′.

  The gas stove 86 can communicate with the projection / device control system 100 ′ by a communication unit (not shown). The gas stove 86 has a function of turning off a switch (not shown) or weakening the thermal power in accordance with an instruction from the CPU 70 of the projection / device control system 100 ′.

  The recorder 87 can communicate with the projection / device control system 100 ′ by a communication unit (not shown). The recorder 87 has a function of recording a program shown on the television screen in accordance with an instruction from the CPU 70 of the projection / device control system 100 ′.

  FIGS. 9A and 9B show operation menu images 17a and 17b displayed by the menu display unit 16 in the second embodiment. The operation menu image 17a in FIG. 9A displays doorphone, gas stove, and recorder icons. In the operation menu image 17b in FIG. 9B, in addition to the doorphone, gas stove, and recorder icons, An icon is displayed. In the second embodiment, the menu display unit 16 projects one of the operation menu images shown in FIGS. 9A and 9B through the projection optical system 14 according to the user's wording. To do.

  Hereinafter, the operation of each unit in FIG. 2 under the control of the CPU 70 will be described in detail with reference to the flowchart in FIG. 10. The flowchart of FIG. 10 starts from the stage where the user receives a call.

  In FIG. 10, in step S <b> 50, the CPU 70 starts measuring time for measuring the call time. Next, in step S <b> 52, the CPU 70 starts imaging by the imaging unit 21 and the face imaging unit 94. The imaging unit 21 captures an area including the upper half of the user who is using the telephone 90. The face imaging unit 94 images the user's facial expression as described above. The image captured by the face imaging unit 94 is sent to the facial expression detection unit 72. Then, the facial expression detection unit 72 detects the user's facial expression based on the image and determines whether or not the user is confused.

  Next, in step S54, the CPU 70 determines whether or not there is a gesture input from the user based on the imaging result of step S52. Here, when the gesture recognition unit 40 recognizes that the user imaged by the imaging unit 21 is consciously performing a predetermined gesture, or when the facial expression detection unit 72 determines that the user is confused Then, it is determined that there is a gesture input from the user. Note that the gesture recognition unit 40 recognizes the user's gesture based on a reference image of a gesture performed by the user when the received call is a nuisance call (assumed to be stored in advance in the flash memory 50). . Note that gestures performed by the user when the call is a nuisance call include gestures such as lowering the thumb, pointing the palm toward the imaging unit 21, and creating a batten mark with both index fingers.

  If the determination in step S54 is negative, the process proceeds to step S56, and the CPU 70 determines whether a predetermined time (for example, 10 minutes or more) has elapsed from the time started in step S50, and the predetermined time has elapsed. If so, the process proceeds to step S58, and if the predetermined time has not elapsed, the process returns to step S52. Note that the reason for shifting to step S58 when the predetermined time has elapsed is that some users may be embarrassed with the nuisance call but may not appear in their facial expressions. Note that when the facial expression detection unit 72 detects the smile of the user, the process may return to step S52 without proceeding to step S58. Smile detection is disclosed in, for example, Japanese Patent No. 4,453,721.

  On the other hand, when the determination in step S54 or step S56 is affirmed (when a gesture is input or when a predetermined time elapses) and the process proceeds to step S58, the CPU 70 selects a projection menu according to the user's way of speaking.

  For example, it is assumed that the voice analysis unit 71 analyzes that the user is speaking in a polite tone such as “is” or “masu”, or using an honorific such as “speak” or “have you”? . In this case, the CPU 70 determines that the call is a long call (nuisance call) from a superior person, and selects the operation menu image 17a of FIG. 9A as a menu to be projected. On the other hand, it is assumed that the voice analysis unit 71 analyzes that the user is not speaking in polite language (speaking in everyday language (plain language)). In this case, the CPU 70 determines that the call is a nuisance call such as sales, and selects the operation menu image 17b of FIG. 9B as a menu to be projected. When the user is speaking in broken words, the CPU 70 may select the operation menu image 17a in FIG. 9A, or may determine that the operation is a long phone call with a friend. It is good also as complete | finishing all the processes.

  Next, in step S60, the CPU 70 projects the operation menu image 17a or 17b in the vicinity of the telephone 90 based on the selection result in step S58. In this case, the CPU 70 may determine the projection region based on the detection result of the sensor unit 30 and the imaging result of the imaging unit 21 as in the first embodiment.

  Next, in step S62, the CPU 70 starts measuring time for measuring the time during which the operation menu image 17a or 17b is projected.

  Next, in step S64, the CPU 70 determines whether or not a user's gesture input has been made on the operation menu image projected in step S60. If the determination here is affirmed, the process proceeds to step S66, and if the determination is negative, the process proceeds to step S70.

  When the process proceeds to step S70, the CPU 70 determines whether or not a predetermined time (for example, 10 seconds) has elapsed. When the determination is affirmed, the process proceeds to step S72, and when the determination is denied. Return to step S64. When the process proceeds to step S72, the CPU 70 ends the projection of the operation menu image and ends all the processes in FIG.

  On the other hand, when the determination in step S64 is affirmed and the process proceeds to step S66, the CPU 70 controls the door phone 85, the gas stove 86, the recorder 87, or the answering function unit 92 based on the gesture input of the user.

  For example, suppose that the user touches the doorphone icon in the operation menu image 17a or 17b when the user does not come up with an excuse to hang up the phone. In this case, the CPU 70 instructs the door phone 85 to cause the door phone 85 to sound a pseudo chime. As a result, the user can hang up on the pretext that the door phone 85 must be handled.

  Further, it is assumed that the user touches the icon of the answering machine when there is a case such as sales. In this case, the CPU 70 forcibly turns on the answering machine function to end the call or reproduce the message that is refused. As a result, the user can forcibly hang up the phone for sales or the like.

  Further, it is assumed that the user touches the icon of the gas stove or the recorder when the user wants to continue the call but the gas stove 86 is lit or there is a television to watch. In this case, the CPU 70 stops the gas stove 86 or starts recording the program shown on the television by the recorder 87. As a result, the user can ensure the safety of the gas stove 86 or record the program being viewed. When the user touches the icon of the gas stove 86 or the recorder 87, a detailed operation of the gas stove 86 or the recorder 87 is realized by displaying a hierarchical menu as in the first embodiment. May be.

  As described above, when the process of step S66 ends, the process proceeds to step S72. In step S72, the CPU 70 ends the projection of the operation menu image. After that, all the processes in FIG. 10 are finished.

  As described above in detail, according to the second embodiment, the face imaging unit 94 capable of imaging the user who is using the telephone 90, and the telephone 90 based on the imaging result of the face imaging unit 94. Includes a projection unit 10 that projects operation menu images 17a and 17b of different electronic devices 80. Thereby, the operation menu image of the electronic device 80 can be projected at an appropriate timing based on the facial expression of the user who is using the telephone 90, and user convenience can be improved. Further, by performing the projection based on the facial expression, the operation menu image can be projected at an appropriate timing without the user performing a predetermined gesture. That is, since the user does not have to remember the gesture, the usability of the user can be improved from this point.

  According to the second embodiment, the voice analysis unit 71 detects information related to the voice of the user who is using the telephone 90, and the projection unit 10 operates based on the analysis result of the voice analysis unit 71. Since the menu image is projected, an appropriate operation menu image can be projected based on the voice of the user using the telephone 90.

  In addition, according to the second embodiment, the voice analysis unit 71 analyzes the wording of the user's voice, and therefore can project an appropriate operation menu according to the wording of the user on the telephone.

  In the second embodiment, the imaging unit 21 captures the state of the user's hand with respect to the operation menu projected by the projection unit 10, and the CPU 70 controls the electronic device 80 based on the imaging result. The user who is on the phone can operate the electronic device 80 by moving his / her hand on the operation menu image.

  Further, in the second embodiment, after projecting the operation menu image, the projecting unit 10 does not communicate with the electronic device 80 for a predetermined time (the CPU 70 controls the electronic device 80). When the operation menu image is not projected (step S70: No), the operation menu image can continue to be projected for a long time even though the user does not use the operation menu image. Can be suppressed.

  In the second embodiment, the CPU 70 measures the time during which the user is using the telephone 90, and the projection unit 10 projects an operation menu image based on the measurement time (S56). Therefore, in the case of a long phone call, it is possible to project an operation menu image at an appropriate timing.

  In the second embodiment, since the operation menu image also includes an icon related to the operation of the telephone 90 (FIG. 9B), the user can select an electronic device other than the telephone 90 and the telephone in the operation menu image. The device 80 can be operated.

  In the second embodiment, the projection unit 10 projects information related to the telephone 90 and the electronic device 80 according to the state of the user who is using the telephone 90. Etc.) can be provided.

  In the second embodiment, a recording / reproducing apparatus that records a baby voice or a whistling kettle sound may be employed as the electronic device 80. Also in this case, when making a nuisance call, the user can make an excuse to hang up the telephone by operating the recording / playback device via the operation menu image.

  In the second embodiment, the case where the icons included in the operation menu image are predetermined as shown in FIGS. 9A and 9B has been described. However, the present invention is not limited to this. is not. For example, only devices that have been used before the user answers the phone may be displayed. In this case, only the electronic device 80 that can communicate with the communication unit 60 may be displayed when the telephone 90 is being used. For example, if the gas stove is not used when the telephone 90 is used privately, the gas stove icon may be removed from the menus of FIGS. 9A and 9B.

  In the second embodiment, when the content of the conversation can be acquired as the analysis result of the voice analysis unit 71, the CPU 70 may project information based on the content of the conversation via the projection unit 10. Good. For example, the CPU 70 may summarize and project the content of the conversation, or may project a calendar image when content related to the date appears in the conversation.

  In the second embodiment, the operation menu image to be projected is selected based on the wording. However, the operation menu image to be projected may be selected based on the incoming call number together with the wording. Good.

  In the second embodiment, the operation menu images in FIGS. 9A and 9B are selected and projected according to the situation. However, the present invention is not limited to this. For example, the operation menu image shown in FIG. 9A or 9B may be always projected.

  In the second embodiment, the case where the device used by the user is the telephone 90 has been described as an example. However, the present invention is not limited to this. For example, the device used by the user may be a door phone. In this case, the operation menu image is projected when the doorphone 85 is sold. Thereby, for example, it is possible to make an excuse to refuse to sell by generating a pseudo telephone call by an operation using an operation menu image.

  In the second embodiment, a liquid crystal display device may be used instead of the projection unit 10 or in combination with the projection unit 10. In this case as well, a liquid crystal display device may be provided on a table holding the telephone 90, a wall near the telephone 90, or a window glass.

  In each of the above embodiments, the CPU 70 controls the electronic device 80 based on the hand state with respect to the operation menu images 17, 17a, and 17b. However, the present invention is not limited to this. That is, the CPU 70 may control the electronic device 80 based on the state of a part of the body (including clothes worn by the feet, etc.) with respect to the operation menu images 17, 17a, 17b.

  Each embodiment mentioned above is an example of suitable implementation of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Projection part 15 Projection image adjustment part 16 Menu display part 20 Image system part 21 Imaging part 22 Image comparison part 31 Temperature detection part 32 Distance sensor 60 Communication part 70 CPU
71 Voice analysis unit 80 Electronic device 90 Telephone 94 Face imaging unit 100, 100 'Projection / device control system

Claims (28)

A projection unit capable of projecting a projection image onto an area according to the position of the user;
And a determination unit that determines whether or not the projection image can be projected onto the region.   The projection apparatus according to claim 1, wherein the determination unit determines whether the projection is possible based on a detection result of a thermal sensor that detects a physical quantity related to heat in the region.   The projection apparatus according to claim 1, wherein the determination unit determines whether the projection is possible based on a detection result of a distance sensor that detects a step of the region. A dangerous substance detection unit that detects the presence or absence of a dangerous substance based on an imaging result of an imaging unit that images the region;
The projection device according to claim 1, wherein the determination unit determines whether the projection is possible based on a detection result of the dangerous substance detection unit.   The projection apparatus according to claim 1, further comprising an adjustment unit that adjusts the projection unit based on an imaging result of an imaging unit that images the region.   The projection apparatus according to claim 5, wherein the adjustment unit adjusts at least one of a size, a position, and luminance of the projection image.   The projection apparatus according to claim 6, wherein the adjustment unit adjusts the size of the projection image according to the number of people imaged by the imaging unit.   The projection device according to claim 1, wherein the projection unit projects an operation menu of a plurality of electronic devices onto the region.   The projection apparatus according to claim 8, further comprising a control unit that projects an operation menu reflecting the arrangement of the plurality of electronic devices onto the area. A communication unit capable of communicating with the plurality of electronic devices;
The control unit detects an arrangement of the plurality of electronic devices according to a communication state with the plurality of electronic devices, and causes the projection unit to project an operation menu reflecting the arrangement. 10. The projection device according to 9. An imaging unit capable of imaging a user;
A projection unit capable of projecting a projection image on an area corresponding to the position of the user;
A projection apparatus comprising: a determination unit that determines whether or not the projection image projected by the projection unit can be captured by the imaging unit.   The projection apparatus according to claim 11, further comprising: an object detection sensor configured to detect whether or not there is an object that blocks imaging by the imaging unit in a space including the imaging unit and the region.   The projection apparatus according to claim 12, wherein the object detection sensor includes a distance sensor.   The projection apparatus according to claim 12, wherein the object detection sensor includes an image sensor.   15. The adjustment unit according to claim 11, further comprising an adjustment unit that adjusts a position where the projection image is projected when at least a part of the projection image projected by the projection unit cannot be captured by the imaging unit. The projection device according to any one of the above.   16. The actuator according to claim 11, further comprising an actuator that drives the imaging unit when at least a part of the projection image projected by the projection unit cannot be captured by the imaging unit. Projection device. The imaging unit is capable of imaging the user and the structure,
The projection device according to claim 11, wherein the determination unit determines the orientation of the user based on an imaging result of the structure.   The projection device according to any one of claims 11 to 17, wherein the imaging unit is arranged to capture the user from above the user.   The projection device according to claim 18, wherein the determination unit determines the posture of the user based on an imaging result of the imaging unit. A detection unit that detects the posture of the user from an imaging result of an imaging unit that images the user from above;
A projection apparatus comprising: a projection unit that projects a projection image in the vicinity of the user according to the posture of the user.   The projection device according to claim 20, wherein the detection unit detects the user's posture by detecting the size of the user from an imaging result of the imaging unit. A detection unit for detecting the position of the user;
A projection apparatus comprising: a projection unit that projects a menu related to operation of a plurality of electronic devices according to a detection result of the detection unit.   23. The projection apparatus according to claim 22, further comprising a control unit that projects the menu according to the arrangement of the plurality of electronic devices. The menu is a hierarchical menu corresponding to each of the plurality of electronic devices,
24. The projection apparatus according to claim 23, wherein the control unit is shared with at least a part of the hierarchical menu display. A display unit for displaying in an area according to the position of the user;
And a determination unit that determines whether display in the area is possible. A detection unit for detecting information about the height of the user;
And a display unit that displays in the vicinity of the user according to the height of the user detected by the detection unit.   27. The display device according to claim 25, wherein the display unit displays an operation menu of the plurality of electronic devices based on an arrangement of the plurality of electronic devices.   28. The display device according to claim 27, wherein the detection unit includes an imaging unit that images the user from above.

Images