JP2020053009A - Control device, control method, program, and storage medium - Google Patents

Abstract

To provide a control device outputting predetermined light capable of reducing the power consumption.SOLUTION: The control device (detection unit 1500) includes: determination means (area determination unit 1605) that determines an output area of the predetermined light; output means (light source control part 1606) that outputs the predetermined light to the output area; and detection means (CPU 1601) that detects the predetermined light to thereby detecting an object intrusion into the output area. The determination means is configured to change the output area from a first area to a second area wider than the first area according to the detection result by the detection means.SELECTED DRAWING: Figure 16

Description

  The present invention relates to a control device, a control method, a program, and a storage medium.

  There is a projection device that can detect a user operation performed near an image projected by a projection device (projector).

  In Patent Literature 1, a laser scanning type equipped with a laser light source that emits visible light such as red (R), green (G), and blue (B) and a laser light source that emits invisible light such as infrared light is mounted. A projector is disclosed. In such a projector, a user's operation or the like is detected by detecting the reflected light of the projected invisible light.

JP 2012-58581 A

  However, in the related art disclosed in Patent Document 1, since infrared light is always projected on the entire screen in order to detect an operation, invisible light is projected regardless of the presence or absence of a user operation. There is a problem that is increased. In particular, in recent years, a projector is driven by a battery of an electronic device connected to the projector, and a demand for reduction in power consumption is increasing.

  Therefore, an object of the present invention is to provide a technique capable of reducing power consumption of a control device that outputs predetermined light.

One embodiment of the present invention provides
Determining means for determining a predetermined light output area,
Output means for outputting the predetermined light to the output area,
Detecting means for detecting intrusion of an object into the output area by detecting the predetermined light;
With
The determining unit changes the output region from a first region to a second region wider than the first region, according to a detection result of the detecting unit.
A control device characterized in that:

One embodiment of the present invention provides
A determining step of determining a predetermined light output area,
An output step of outputting the predetermined light to the output area,
A detecting step of detecting an intrusion of an object into the output area by detecting the predetermined light;
Has,
In the determining step, the output area is changed from a first area to a second area wider than the first area according to a detection result of the detecting step.
A control method characterized by the following.

  One embodiment of the present invention is a program for causing a computer to function as each unit of the projection device.

  One embodiment of the present invention is a computer-readable storage medium that stores the program.

  ADVANTAGE OF THE INVENTION According to this invention, the power consumption of the control apparatus which outputs predetermined light can be reduced.

1 is a block diagram illustrating an example of a projection device according to a first embodiment. 5 is a flowchart illustrating an example of an image projection process according to the first embodiment. 5 is a flowchart illustrating an example of an image projection process according to the first embodiment. FIG. 4 is a conceptual diagram illustrating a method for calculating an invisible light projection area according to the first embodiment. A conceptual diagram showing an invisible light projection area according to the first embodiment. FIG. 5 is a diagram illustrating a distance calculation method using invisible light according to the first embodiment. FIG. 3 is a conceptual diagram illustrating a reference distance image according to the first embodiment. FIG. 3 is a conceptual diagram illustrating a distance image according to the first embodiment. FIG. 4 is a conceptual diagram illustrating a difference image according to the first embodiment. FIG. 4 is a conceptual diagram illustrating a UI operation image according to the first embodiment. FIG. 2 is a block diagram illustrating an example of a projection device and an external device according to a second embodiment. 11 is a flowchart illustrating an example of an image projection process according to the second embodiment. FIG. 6 is a block diagram illustrating an example of a projection device and an external device according to a third embodiment. Block diagram showing an example of a projection device according to a fourth embodiment. 14 is a flowchart illustrating an example of an image projection process according to the fourth embodiment. FIG. 14 is a diagram illustrating an example of an invisible light projection area according to the fourth embodiment. FIG. 14 is a diagram illustrating an example of an invisible light projection region according to the fifth embodiment. Schematic diagram of a detection device according to Embodiment 6. Functional block diagram of a detection device according to Embodiment 6. Flowchart of detection processing according to the sixth embodiment The figure which shows the irradiation area | region of the invisible light which concerns on Embodiment 6.

(Embodiment 1)
<Overview>
Hereinafter, the projection device 100 according to the present embodiment will be described. The projection device 100 according to the present embodiment can project an image (projection image) on a projection area on a projection surface using red (R), green (G), and blue (B) visible light. Electronic equipment. Further, the projection device 100 irradiates (projects) infrared laser light (invisible light) to a region included in the projection image where a UI (User Interface; User Interface) is displayed. In addition, the projection apparatus 100 determines whether or not there is a user, a user operation, Detect gestures and the like. Projection device 100 performs a predetermined process based on the detected result.

  The projection device 100 according to the present embodiment uses the information indicating the position of the UI of the projection image such that the region where the invisible light is projected is smaller than the region where the projection image is projected using visible light (projection region). Controls the region where the invisible light is projected based on Hereinafter, the overall configuration and processing contents of the projection device 100 according to the present embodiment will be sequentially described.

<Overall configuration>
FIG. 1 shows a block diagram of a projection device 100 according to the present embodiment. The projection device 100 is an information processing device (computer) including an arithmetic device (processor), a memory, a storage device, an input / output device, and the like. When the projection device 100 executes the program stored in the storage device, functions of the projection device 100 described below are provided. Some or all of these functions may be implemented by dedicated logic circuits such as ASICs and FPGAs. Further, the projection device 100 can be regarded as, for example, a projector (electronic device) having a built-in function such as a personal computer or a smartphone.

  The CPU 101 controls the operation of each function of the projection device 100 by executing a program stored in a memory 102 described later. Further, the CPU 101 executes an application stored in the memory 102.

  The memory 102 is a storage medium that stores a program executed by the CPU 101, image data, and the like.

  The image generation unit 103 is a functional unit that generates projection image data in response to a request from the CPU 101. In the present embodiment, an example will be described in which the projection image includes a user interface image (hereinafter, referred to as a UI image) for a user to perform an operation.

  The UI position acquisition unit 104 is a functional unit that acquires information (UI position information) for specifying the position of the UI image in the projection image data. Further, the UI position acquisition unit 104 extracts the shape information of the UI image.

  The region determining unit 105 is a functional unit that determines a region on which an infrared laser beam is projected (an infrared projection region). Further, the area determination unit 105 outputs information on the determined infrared projection area (infrared projection area information) to the light source control unit 106, which will be described later. The area determination unit 105 acquires information (UI position information) for specifying the position of the UI image in the projection image data from the UI position acquisition unit 104. The UI position information can be acquired from the image generation unit 103. The details of the method of the infrared projection area information will be described later. The projection device 100 according to the present embodiment projects invisible light when a UI image exists in the projection image (position information and the like can be acquired).

  The light source control unit 106 generates a light source control signal for controlling the laser light sources 108R, 108G, 108B that output visible light and the infrared laser light source IR based on the projection image data and the infrared projection area information. Note that the projection image data may be projection image data read from the memory 102. The light source control unit 106 generates a light source control signal for each of three visible light sources of red (R), green (G), and blue (B) and an invisible light source of infrared (IR). The light source control unit 106 generates a synchronization signal (vertical synchronization signal, horizontal synchronization signal) for controlling a scanning mirror (MEMS) to scan the laser light output from each laser light source on a projection surface, and outputs the synchronization signal. Output to the control unit 112.

  The laser driver 107 outputs a drive current for each laser light source based on the light source control signal output from the light source control unit 106. Specifically, the laser driver 107 modulates the drive current of the laser light source corresponding to each light control signal, and outputs a drive current to each of the laser light sources 108R, 108G, 108B and the infrared laser light source 110.

The laser light sources 108R, 108G, and 108B are visible light sources that emit laser light based on a drive current supplied from the laser driver 107. The laser light source 108R emits red laser light (R), the laser light source 108G emits green laser light (G), and the laser light source 108B emits blue laser light (B).

  The dichroic mirror 109 has a property of reflecting light of a specific wavelength and transmitting light of other wavelengths. The dichroic mirror 109 according to the present embodiment combines the laser light of each color emitted from the three laser light sources 108R, 108G, and 108B using the above-described properties, and generates red light, green light, and blue light. Outputs laser light containing light components.

  The infrared laser light source 110 is an invisible light source that emits infrared laser light based on a drive current supplied from the laser driver 107.

  The dichroic mirror 111 combines the laser light synthesized by the dichroic mirror 109 and the infrared laser light emitted from the infrared laser light source 110 to form a laser including red light, green light, blue light, and infrared light components. Outputs light.

  The mirror control unit 112 generates a drive signal for driving the scanning mirror in a predetermined angle range in the horizontal and vertical directions based on the horizontal and vertical synchronization signals output from the light source control unit 106, and performs scanning. It is a functional unit that supplies to the mirror 113. The angle range can be a fixed value and can be stored in the memory 102 in advance.

  The scanning mirror 113 is a reflecting member that reflects the laser light emitted from the dichroic mirror 111 and changes the optical path in the direction of the projection plane. By changing the angle of the reflection surface of the scanning mirror 113 by the mirror control unit 112, the laser light reflected by the scanning mirror 113 can be scanned on the projection surface (two-dimensional scan scanning). Although the projection device 100 according to the present embodiment performs image projection by a laser scanning method, the method of projecting an image is not particularly limited. For example, the image projection is performed by using a laser projection method or a liquid crystal panel. You may. The projection device uses a laser light source, a laser driver, a dichroic mirror, a mirror control unit, a scanning mirror, and the like. Instead of these, a light source unit, a light source control unit, a liquid crystal control unit, a liquid crystal element, and the like are used to perform projection. May go.

  The setting unit 120 is a functional unit for changing the settings of the projection device 100. In the present embodiment, the setting unit 120 changes the setting of the projection device 100 by inputting an operation signal based on a user operation.

  The light receiving unit 130 is a functional unit that detects infrared laser light emitted from the infrared laser light source 110 and reflected on a projection surface (projection surface or an object). As the light receiving unit 130, for example, a photodiode, a two-dimensional sensor, or the like can be used.

  The distance detecting unit 131 calculates a distance to the projection surface based on the infrared laser light emitted by the infrared laser light source 110 and the infrared laser light (reflected light) detected by the light receiving unit 130. Department. For example, the distance detection unit 131 detects a distance by a TOF (Time of Flight) method. Specifically, the distance is calculated by measuring a time difference between when the infrared laser light source 110 emits the infrared laser light and when the light receiving unit 130 detects the infrared laser light. Note that the distance detection unit 131 only needs to be able to calculate the distance between the projection device 100 and the projection plane, and the distance measurement method is not limited to the above. Details of the distance calculation will be described later.

  Note that the image generation unit 103, the UI position acquisition unit 104, the area determination unit 105, and the like may be functional units implemented by the CPU 101.

<Processing content>
FIG. 2 is a flowchart showing the processing content of the projection device 100 according to the present embodiment. The process illustrated in FIG. 2 is started when the projection device 100 is activated and the initialization is completed.

  In S201, the CPU 101 determines whether or not an operation detection mode, which is a mode for detecting a user operation, is valid. If the operation detection mode is valid, the process proceeds to S202; otherwise, the process proceeds to S211. Information indicating ON (valid) / OFF (invalid) of the operation detection mode can be read from the memory 102. The setting of the operation detection mode can be set by the user via an operation unit (not shown).

  In S202, the image generation unit 103 acquires (generates) projection image data for projecting a visible light image on a projection plane. In the present embodiment, the projection image data is image data read from the memory 102, but may be an image input from an external device. Here, when the projection device 100 projects a UI image (operation image) that is an operation image in a superimposed manner, a process of combining the projection image data with the UI image data is performed.

  In S203, the UI position obtaining unit 104 obtains UI position information for specifying the position of the UI image in the projection image data. Here, the area corresponding to the UI image indicates an area of the projection image where the UI image is displayed. The UI position information is, for example, coordinate information indicating the center position of the UI image in the projection image data and pattern information indicating the shape of the UI image. Further, the UI position information may be image data (invisible light image data) in which the area corresponding to the UI image is set to a predetermined gradation value and the gradation values of other areas are set to 0.

  In S204, the light source control unit 106, the laser driver 107, and the mirror control unit 112 project a projection image based on the projection image data onto a projection surface using visible light. The light source control unit 106 determines a light source control signal indicating the intensity of the laser light corresponding to each of the laser light sources 108R, 108G, and 108B based on the R, G, and B tone values of each pixel of the projection image data. . The light source control unit 106 generates a synchronization signal based on the projection image data and outputs it to the mirror control unit 112. The laser driver 107 generates a drive signal for each laser light source 108 based on the light source control signal, and outputs the drive signal to each laser light source 108. Each laser light source 108 outputs a laser beam based on the input drive signal.

  The mirror control unit 112 adjusts the angle (reflection angle) of the scanning mirror 113 so that the position of the projection plane corresponding to the position of each pixel is irradiated with laser light emitted with an output corresponding to the pixel value of the pixel. Control. The scanning mirror 113 is controlled so that the reflection angle of the image in the horizontal direction periodically oscillates at the resonance frequency, and the reflection angle of the image in the vertical direction periodically oscillates at the frequency corresponding to the frame rate. The vertical vibration of the scanning mirror 113 is controlled so as to synchronize with the synchronization signal acquired from the light source control unit 106. In addition, the light source control unit 106 and the laser driver 107 synchronize with the resonance frequency at which the scanning mirror 113 vibrates in the horizontal direction, and complete the output of the laser light so that the output corresponding to one horizontal line of pixels is completed. Control output timing and intensity. As described above, by controlling the output intensity, output timing, and scanning direction of each laser beam based on the projection image data, a projection image of visible light is projected on the projection area on the projection surface.

In S205, the light source control unit 106, the laser driver 107, and the mirror control unit 112 project the invisible light on the projection surface based on the infrared projection area information. The light source control unit 106 controls the output intensity and output of the infrared laser light source 110 based on the infrared projection area information so that the invisible light is projected on the area including the UI image in the projection image projected on the projection area. Controls timing and scanning direction. The light source control unit 106 controls the output timing of the invisible laser beam so that only the area corresponding to the UI image that is smaller (for example, the area) is smaller than the projection area in the projection area on the projection surface. And intensity. As described above, when the invisible light image data is generated, the output intensity of the infrared laser light may be controlled based on the invisible light image data. By irradiating the invisible light only to an area smaller than the projection area where the projection image is projected, power consumption is reduced as compared to irradiating the invisible light to the entire projection image (projection area). The region to be irradiated with the invisible light is not limited to the region where the UI image exists, but may be a region including the UI image and its vicinity.

  In S206, the CPU 101 detects (detects) a user operation. The projection device 100 according to the present embodiment detects a distance between the projection device 100 and a projection surface or an object by a TOF (Time of Flight) method, and detects a user operation based on a change in the distance. A method for detecting a user operation will be described later.

  In S207, the CPU 101 determines whether a user operation has been performed. If a user operation is detected in S206, the process proceeds to S208; otherwise, the process proceeds to S209.

  In S208, the CPU 101 outputs an operation signal for executing a process corresponding to the user operation in response to the detection of the user operation. It is assumed that the operation signal is predetermined in association with the UI image to be projected. For example, when a user operation is performed, the CPU 101 outputs a signal for controlling the image generation unit 103 so as to execute a process of switching a projection image (switching projection image data). Further, CPU 101 outputs an operation signal for changing the setting of projection device 100 to the setting unit. That is, when a user's predetermined operation is performed on a UI image that is an operation image associated with a predetermined process executable by the projection device 100, the CPU 101 performs a predetermined process previously associated with the UI image. Execute

  Further, the operation signal may be output to an external device. For example, when an image is projected based on image data input from an external device, it is also possible to output an operation signal for instructing the external device to page forward / backward.

  In the above description, the example in which the visible light image projection processing, the invisible light irradiation processing, and the user operation detection processing are executed in this order has been described, but the execution order is not particularly limited. For example, the visible light laser light and the invisible light laser light may be output in synchronization with scanning by a common scanning mirror. In the former case, the scanning in the vertical direction is performed at a frequency four times the frame rate. In the latter case, the scanning of the mirror can be controlled at a frequency corresponding to the frame rate.

  In step S209, the CPU 101 determines whether the detection mode has been switched from valid to invalid by a user operation during image projection. If the detection mode has been switched, the process proceeds to S201; otherwise, the process proceeds to S210.

  In S210, the CPU 101 determines whether or not an operation for ending image projection has been performed by a user operation. If the end operation has been performed, the present process ends, and if not, the process proceeds to S202. The CPU 101 determines that the ending operation has been performed, for example, when an instruction to end the image projection or an instruction to turn off the power is received from the user.

  In steps S211 and S212, the same processes as those in steps S202 and S204 are performed, and a description thereof is omitted.

In S213, the CPU 101 determines whether the detection mode has been switched from invalid to valid by a user operation during image projection. If the detection mode has been switched, the process proceeds to S201; otherwise, the process proceeds to S214.

  In S214, the CPU 101 determines whether or not an operation for ending image projection has been performed by a user operation. If the end operation has been performed, the present process ends, and if not, the process proceeds to S211. The CPU 101 determines that the ending operation has been performed, for example, when an instruction to end the image projection or an instruction to turn off the power is received from the user.

抽出 User operation extraction processing≫
Next, a process for extracting a user operation will be described with reference to the flowchart shown in FIG.

  In step S301, the distance detection unit 131 performs distance measurement on each pixel of the projection image in a state where there is no obstacle (object) between the projection device 100 and the projection plane. Specifically, the distance is detected by detecting the reflected light of the projected infrared laser light. Then, the distance detection unit 131 generates a reference distance image which is an image indicating distance information corresponding to each pixel of the projection image. Each pixel value of the reference distance image is a distance detected by the distance detection unit 131 (FIG. 7A). The generated reference distance image is stored in the memory 102. Note that the generation of the reference distance image may be performed only once when the projection device 100 is activated. Further, the distance detection unit 131 may measure the distance to some of the pixels of the projection image.

  In S302, the CPU 101 determines whether or not the projection image includes a UI image. If it is determined that the projection image includes the UI image, the process proceeds to S303; otherwise, the process proceeds to S307.

  In S303, the distance detection unit 131 detects the distance from the projection device 100 to the projection plane or the obstacle, similarly to the processing in S301 described above.

  In S304, the CPU 101 generates a distance image based on the distance obtained by the distance detection unit 131. The generation of the distance image will be described later.

  In step S305, the CPU 101 compares the distance image with each pixel value of the reference distance image to generate a difference image (difference data).

  In step S306, the CPU 101 detects a user operation in an area corresponding to the UI image based on the difference image and the projection image. For example, the CPU 101 detects a touch operation or the like on a button displayed on the projection image as a user operation. For example, the CPU 101 determines that the touch operation has been performed when the distance in the area corresponding to the UI image in the difference image increases after the distance has decreased and further decreases.

  In step S <b> 307, the CPU 101 determines whether to end the user operation (UI operation) extraction processing. If it is determined that the present process is to be continued, the process returns to S302; otherwise, the present process is terminated. Note that the extraction process of the user operation may be performed simultaneously as a process different from the image projection process.

算出 Infrared projection area calculation method≫
4A and 4B are conceptual diagrams illustrating a method for calculating an infrared projection area according to the present embodiment.

FIG. 4A is a schematic diagram showing the projection image data. The projection image data is 3 in the horizontal direction.
This is image data composed of a plurality of pixels of 2 pixels and 18 pixels in the vertical direction. Projection image 400 includes UI 401 and UI 402. The UI 401 and the UI 402 are generated by the image generation unit 103 in advance. In the projection image 400, the coordinates of the upper left pixel are (0, 0), and the coordinates of the lower right pixel are (17, 31).

  FIG. 4B shows infrared projection area information 450 indicating an area of the projection image 400 to which the infrared laser light is irradiated. In the present embodiment, the infrared projection area information 450 is image data indicating an area to be irradiated with invisible light. The area determination unit 105 receives the coordinates indicating the center position of the UI 401 and the UI 402 from the UI position acquisition unit 104 and the shape information of the UI image. For example, the center position of the UI 401 is the coordinates (14, 5), and the shape information is a 5 × 5 rectangle. The center position of the UI 401 is the coordinates (14, 26), and the shape information is a 5 × 5 rectangle. Then, the area determination unit 105 sets the value of the pixel of the area corresponding to the position of the UI 401 and the UI 402 (the shaded area in FIG. 4B) to 1 and sets the values of the other pixels to 0. The infrared projection area information 450 is generated. That is, a portion having a pixel value of 1 in the infrared projection area information 450 is an infrared projection area.

  FIG. 5 is a conceptual diagram illustrating a projection region of the infrared laser light according to the present embodiment. The scanning mirror 113 scans and projects a laser beam obtained by combining the laser light sources 108R, 108G, 108B and the infrared laser light source 110 onto a two-dimensional scanning line 501. Note that the two-dimensional scanning lines 501 in FIG. 5 have a large spacing between the scanning lines for the sake of explanation, but are actually small enough to project the resolution of an image.

  The light source control unit 106 modulates the laser light sources 108R, 108G, and 108B based on the projection image 500 to turn on the laser light on the two-dimensional scanning line 501 corresponding to the image. Further, the light source control unit 106 uses the infrared projection area information 450 to emit infrared light to a point 503 where the two-dimensional scanning line 501 and the UI area 502 indicating the area where the UI image is displayed overlap. The infrared laser light source 110 is controlled to irradiate. In this way, the projection device 100 projects the infrared laser light only on the UI region 502.

≪Distance calculation method≫
FIGS. 6A and 6B show a distance calculation method using infrared laser light according to the present embodiment.

  FIG. 6A is a diagram showing detection of reflected light corresponding to the projected infrared laser light. The infrared laser light output from the infrared laser light source 110 is projected onto the projection surface 600 via the scanning mirror 113. Then, the infrared laser light reflected on the projection surface 600 or the obstacle 601 (for example, a user's hand) is detected by the light receiving unit 130.

  FIG. 6B illustrates the infrared laser light output from the infrared laser light source 110 and the reflected light received by the light receiving unit 130. The waveform 620 is a waveform in which the horizontal axis represents time, the vertical axis represents the intensity of infrared light, and the lighting timing of the infrared laser light output from the infrared laser light source 110. The waveform 621 is a waveform similar to the waveform 620, in which the horizontal axis indicates time and the vertical axis indicates intensity, and indicates the light receiving timing of the infrared laser light received by the light receiving unit 130. Δt indicates a time until the lit laser light of the infrared laser light source 110 is reflected on the projection surface or the obstacle and returns.

The distance detection unit 131 calculates the projection device 100 and the projection surface 600 or the obstacle 601 from the rising time difference (phase difference between the emitted light and the reflected light) of the waveforms 620 and 621 using the following equation (1). Is calculated.
(Equation 1)
d = c × Δt / 2 (1)
Here, d indicates the distance and c indicates the speed of light. Note that the above time difference can be regarded as a time difference from the irradiation of the invisible light to the detection of the reflected light.

  After repeating distance measurement for the number of pixels of the projection image, the distance detection unit 131 generates a distance image having the same number of pixels as the infrared projection area information.

<< Generation of UI operation information >>
7A to 7D are conceptual diagrams illustrating generation of UI operation information according to the present embodiment.

  FIG. 7A shows a reference distance image 700 indicating a distance between the projection apparatus 100 and the projection plane 600 (hereinafter, also referred to as a reference distance). The reference distance image 700 is a distance image acquired by the distance detection unit 131 without any obstacle between the projection device 100 and the projection plane 600. Each pixel value in the reference distance image 700 is a value indicating the distance between the projection device 100 and the projection plane 600. In the present embodiment, since the distance detection is performed only on the area where the UI image is displayed (FIG. 4B), a value indicating the distance (“10” in the example of FIG. 7A) is obtained only for the corresponding pixel. Have been. Note that there is no particular limitation on the pixel for which the reference distance is to be obtained, and the distance measurement may be performed on all the pixels.

  FIG. 7B shows a distance image 710 indicating the distance between the projection device 100 and the projection plane 600 or the obstacle 601. The distance image 710 is a distance image acquired by the distance detection unit 131 in a state where there is an obstacle between the projection device 100 and the projection plane 600. Further, in the present embodiment, the distance detection unit 131 performs distance measurement on pixels included in the area where the UI image is displayed. For pixels that are not distance measurement targets, a distance image 710 is generated using the value of the reference distance image 700. It should be noted that the distance measurement target pixel is not particularly limited, and distance measurement may be performed for all pixels.

  FIG. 7C shows a difference image 720 obtained by comparing each pixel value of the reference distance image 700 and the distance image 710.

  FIG. 7D shows a UI operation image 730 obtained by calculating the product of each pixel of the difference image 720 and the infrared projection area information 450 described above. The CPU 101 analyzes the UI area 731 and the UI area 732 to detect a user operation on the UI 401 and the UI 402. For example, the CPU 101 can detect a user operation on the UI by comparing the average of each pixel value of the UI area 731 or the UI area 732 with a predetermined value. Here, the predetermined value is a value for determining whether or not the difference between the distance acquired by the distance detection unit 131 and the reference distance is based on a user operation, and is set to, for example, 2. Can be. In this case, the average value of each pixel value in the UI area 732 shown in FIG. 7D is 5.64, and the average value of the UI area 732 is larger than a predetermined value. Therefore, the CPU 101 determines that the user operation on the UI 402 has been performed. I do. The CPU 101 operates the application using the result of the UI operation determination. Since the average value of the UI area 731 is 0, the CPU 101 determines that no user operation has been performed on the UI 401. The predetermined value is not particularly limited. Further, the CPU 101 may determine that the user operation has been performed if the UI operation image 730 is other than 0 without using the predetermined value.

<Advantageous effects of this embodiment>
As described above, in the detection of the user operation on the UI image included in the projection image, by irradiating only the UI image with the infrared laser light, unnecessary projection of the infrared laser light is suppressed, and the power consumption of the projection device 100 is reduced. Can be reduced.

(Embodiment 2)
<Overview>
In the first embodiment, an example in which a projection image is generated inside the projection device has been described. The projection device according to the present embodiment detects a user operation on a UI image included in a screen displayed by an application executed on an external device. In the configuration of the projection device 200 according to the present embodiment, the same functional units as those in the above-described first embodiment are denoted by the same reference numerals, and description thereof will be omitted.

<Overall configuration>
FIG. 8 is a block diagram illustrating a configuration of the projection device 200 and the external device 250 according to the present embodiment.

<< Projector 200 >>
The projection device 200 projects a projection image output from the external device 250 using a liquid crystal panel. The projection device 200 projects the infrared laser light by performing two-dimensional scanning using the scanning mirror 113.

  The UI image analysis unit 204 is a functional unit that analyzes whether a projection image input from the external device 250 includes a UI. When the projection image includes a UI, the UI image analysis unit 204 acquires information for specifying the position of the UI image in the projection image data.

  The light source control unit 206 is a functional unit that generates a light source control signal for controlling the infrared laser light source IR based on the infrared projection area information. In the present embodiment, invisible light is projected by a laser scanning method. However, the image projection method is not particularly limited, and for example, image projection using a liquid crystal panel may be performed.

  The laser driver 207 is a functional unit that modulates a drive current of the infrared laser light source 110 based on the light source control signal output by the light source control unit 206 and outputs the drive current to the infrared laser light source 110.

  The communication unit 214 is a functional unit that performs communication between the projection device 200 and the external device 250. In the present embodiment, the communication unit 214 receives (inputs) projection image data from the external device 250. The communication unit 214 transmits (outputs) the UI operation information detected by the projection device 200 to the external device 250.

  The light source control unit 215 is a functional unit that performs on / off control and brightness control of a light source unit 216 described later. The light source control unit 215 is, for example, a control microprocessor. The light source control unit 215 does not need to be a dedicated microprocessor. For example, the CPU 101 may execute the same processing as the light source control unit 215.

  The light source unit 216 is a functional unit that outputs light for projecting an image, and includes, for example, a laser, an LED halogen lamp, a xenon lamp, and a high-pressure mercury lamp.

  The liquid crystal control unit 217 controls the voltage applied to the liquid crystal of each pixel (R pixel, G pixel, and B pixel) of the liquid crystal element unit 218 described later based on the input projection image data, and reflects the liquid crystal element. This is a functional unit for adjusting the transmittance or transmittance.

  The liquid crystal element unit 218 applies a voltage corresponding to the pixel value of a projected image to a liquid crystal element provided with a red (R), green (G), and blue (B) filter to apply a red (R), green (G) ) And blue (B) light.

  The projection unit 219 is a functional unit that combines red (R), green (G), and blue (B) light transmitted through the liquid crystal element unit 218. For example, the projection unit 219 is a dichroic mirror, a prism, or the like. The light obtained by combining the red (R), green (G), and blue (B) components is sent to a projection optical system (not shown). At this time, the liquid crystal element unit 218 is controlled by the liquid crystal control unit 217 so as to have a light transmittance corresponding to the input projection image data. Therefore, when the light combined by the projection unit 219 is projected by the projection optical system, an image corresponding to the input projection image is displayed on the projection plane. In the present embodiment, the image projection is performed using the liquid crystal panel. However, the image projection method is not particularly limited. For example, the image projection may be performed by the laser scanning method as in the first embodiment.

≪External equipment 250≫
The external device 250 is an information processing device (computer) including an arithmetic device (processor), a memory, a storage device, an input / output device, and the like. When the external device 250 executes the program stored in the storage device, a function of the external device 250 described later is provided. Some or all of these functions may be implemented by dedicated logic circuits such as ASICs and FPGAs. The external device 250 is specifically a personal computer, a smartphone, a digital camera, or the like. The external device 250 transmits (outputs) image data to be projected to the projection device 200. Note that the projection device 200 may be connected to the database via a network (not shown) using a database on a network instead of the external device 250, and the projection device 200 may acquire an image from the database on the network.

  The CPU 251 controls the operation of each function of the external device 250 by executing a program stored in a memory 252 described later. The CPU 251 executes an application in the external device 250 stored in the memory 252.

  The memory 252 stores a program code executed by the CPU 251, image data, and the like.

  The UI combining unit 253 is a functional unit that outputs an image obtained by combining the UI image with the image generated by the CPU 251 to the communication unit 254.

  The communication unit 254 outputs the projection image data to the projection device 200. In addition, the CPU 251 receives UI operation information from the projection device 200 via the communication unit 254 and performs an operation of the application.

<Processing content>
FIG. 9 is a flowchart illustrating processing of the projection device 200 and the external device 250 according to the present embodiment. First, the operation of the external device 250 will be described.

≪External equipment 250≫
In S951, the CPU 251 makes a request for connecting the external device 250 and the projection device 200, and establishes the connection. The external device 250 proceeds to S952 after obtaining permission to project an image on the projection device 200.

  In S952, CPU 251 outputs projection image data to projection device 200 via communication unit 254.

In S953, the CPU 251 receives the operation information of the UI image superimposed on the projection image. For example, the CPU 251 periodically checks the interrupt of the communication unit 254 or the state of the communication unit 254 and receives the operation information of the UI image.

  In S954, the CPU 251 determines whether to continue projecting the image. If the projection of the image is to be continued, the process returns to S952; otherwise, the process ends. Next, the operation of the projection device 200 will be described.

<< Projector 200 >>
In step S901, the CPU 101 waits for a connection request (request) from the external device 250. Upon receiving the connection request, the CPU 101 transmits a connection permission to the external device 250 and proceeds to step 902. The processing in S902 is the same as that in S201 described above, and a description thereof will not be repeated.

  In step S903, the CPU 101 acquires projection image data from the external device 250.

  In step S <b> 904, the CPU 101 analyzes the projection image data and acquires a position of the projection image at which the UI image is displayed. In the present embodiment, in the CPU 101, the UI image analysis unit 204 analyzes the projection image data and detects the position, size, and the like of the UI image in the projection image data. For example, the UI image analysis unit 204 can detect a rectangular portion having a small gradation change and detect a portion including a character as a UI image. Further, the UI image analysis unit 204 may acquire the position of the UI image using an artificial intelligence algorithm such as deep learning. Specifically, the UI image analysis unit 204 may acquire (detect) a portion corresponding to the UI image in the projection image using a classifier obtained by learning the UI image in advance. Further, the CPU 101 outputs the position, size, and the like of the UI image obtained by the UI image analysis unit 204 to the region determination unit 105. The processing of S905 to S908 is the same as that of S204 to S207, respectively, and a description thereof will be omitted.

  In step S909, the CPU 101 outputs an operation signal for executing a corresponding process to the external device 250 in response to receiving the detection signal of the user operation. For example, it is also possible to output an operation signal for a UI image such as a button for instructing the external device 250 to page forward / return. The processing of S910 to S911 is the same as that of S209 to S210, respectively, and thus the description is omitted.

  In step S912, the CPU 101 acquires projection image data from the external device 250. The processing of S913 to S915 is the same as that of S212 to S214, respectively, and thus the description is omitted.

  As described above, regarding the UI operation of the application executed on the external device by connecting the external device and the projection device, the projection of the infrared laser light unnecessary for the UI operation is suppressed without sacrificing the operability of the UI. The power consumption of the device can be reduced.

(Embodiment 3)
<Overview>
In the above-described second embodiment, an example has been described in which the projection device acquires projection image data from an external device and acquires the position and the like of a UI image in the projection device. The projection device according to the present embodiment acquires projection image data, position information of a UI image, and the like from an external device. Regarding the configuration of the projection device 300 according to the present embodiment, the same functional units as those in the above-described first or second embodiment are denoted by the same reference numerals, and description thereof is omitted.

<Overall configuration>
FIG. 10 is a block diagram illustrating a configuration of the projection device 300 and the external device 350 according to the present embodiment.

<< Projector 300 >>
The image input unit 320 is a functional unit that inputs projection image data from the external device 350.

  The communication unit 314 is a functional unit that inputs UI image data to be combined with projection image data and position information of the UI image from the external device 350.

  The UI combining unit 321 is a functional unit that combines UI image data with projection image data. The image data after the combination can be stored in the memory 102.

≪External equipment 350≫
The image output unit 355 is a functional unit that outputs projection image data to the projection device 300.

  The communication unit 354 is a functional unit that outputs the UI image data and the position information of the UI image to the projection device 300. In addition, the communication unit 354 acquires UI operation information from the projection device 300. The CPU 251 operates an application to be executed using the acquired UI operation information.

  As described above, regarding the UI operation of the application executed on the external device by connecting the external device and the projection device, projection of the infrared laser light unnecessary for the UI operation is suppressed without sacrificing the operability of the UI. Power consumption can be reduced.

(Embodiment 4)
<Overview>
In the first embodiment described above, an example in which the infrared laser light is projected on the UI image of the projection image to reduce the power consumption of the projection device has been described. In the present embodiment, an example will be described in which there are two modes in projection of infrared laser light. In the present embodiment, when a user is near the projected image, an infrared laser beam is projected on the UI image in the same manner as in the above-described embodiment (first projection mode). An infrared laser beam is projected on the outer frame of the image (second projection mode). Further, an example will be described in which invisible light is projected as upper, lower, left, and right end regions in the projected image as an outer frame of the projected image. Regarding the configuration of the projection device 400 according to the present embodiment, the same functional units as those in the above-described first embodiment are denoted by the same reference numerals, and description thereof is omitted.

<Overall configuration>
FIG. 11 is a block diagram illustrating a configuration of a projection device 400 according to the present embodiment.

  The infrared projection mode setting unit 422 is a functional unit that determines whether a user is present near the projection plane and sets the projection mode of the infrared laser light according to the determination result. The determination as to whether or not a user exists will be described later. When a user is present near the projection plane, the infrared projection mode setting unit 422 sets a first projection mode that is a mode for projecting an infrared laser beam onto the UI portion. When there is no user near the projection plane, the infrared projection mode setting unit 422 sets the second projection mode, which is a mode for projecting the infrared laser light onto the outer frame of the projection image.

  The region determining unit 405 is a functional unit that determines a region on which an infrared laser beam is projected (an infrared projection region). Here, the region determination unit 405 acquires the projection mode setting information from the infrared projection mode setting unit 422. Then, in the case of the first projection mode, the region determining unit 405 determines the infrared projection region so as to project the infrared laser light onto a portion corresponding to the UI region. Further, in the case of the second projection mode, the region determining unit 405 determines the infrared projection region so as to project the infrared laser light on the outer frame of the projection surface. The area determination unit 405 acquires information (UI position information) for specifying the position of the UI image in the projection image data from the UI position acquisition unit 104.

<Processing content>
FIG. 12 is a flowchart illustrating the projection processing according to the present embodiment. The projection device 100 according to the present embodiment is invisible along the outer frame of the projection image when the operation detection mode is valid and there is no user operation (including insertion of a hand, an indicator, or the like). Irradiate light (second projection mode). The invisible light in the second projection mode can be regarded as an image for detecting that the user has approached the projection plane. Further, when the user's hand or pointer is inserted into the invisible light projected along the outer frame of the projection image, the projection device 100 according to the present embodiment, as in the first embodiment, A process of irradiating invisible light to a region corresponding to the UI image is performed (first projection mode).

  The processing in steps S1201 to S1202 is the same as the processing in steps S201 to S202, and a description thereof will not be repeated.

  In S1203, the CPU 101 determines whether the projection mode is the first projection mode. In the present embodiment, the case where the projection mode is not the first projection mode is the case where the projection mode is the second projection mode. If the projection mode is the first projection mode, the process proceeds to S1204; otherwise, the process proceeds to SS1212. In the present embodiment, an example in which two projection modes can be set will be described. However, three or more projection modes may be provided. Note that an operation flag that is a flag (0 or 1 or the like) indicating whether or not a user operation is expected to occur may be used. The case where the operation flag is 0 is, for example, a case where the user or the pointer has not been inserted into the projection area and the state has been continued for a predetermined period (threshold) (see the above-described first example). 2 mode). The case where the operation flag is 1 is a case where the user or the pointer is inserted into the projection area. In addition, the case where the elapsed time after the insertion of the user or the indicator is stopped is equal to or shorter than a predetermined period. The processing in steps S1204 to S1209 is the same as the processing in steps S203 to S208, and a description thereof will not be repeated.

  In S1210, the CPU 101 determines whether the time during which no user operation is performed is longer than a predetermined time (threshold). Specifically, when the CPU 101 does not detect a user operation for a predetermined time or more, the CPU 101 determines that there is no user, and proceeds to S1217. If it is determined that a user exists, the process advances to step S1211.

  In S1211, the CPU 101 sets the projection mode to the second projection mode. Note that the CPU 101 may change the projection mode to the second mode when there is no user between the projection device 400 and the projection surface or when there is no pointer. In S1207 to S1211, in the present embodiment, the projection mode is changed when the user operation is detected and the user operation is not performed for a predetermined period. The projection mode may be changed when does not exist. In this case, similarly, the projection mode may be changed when no user or the like exists for a predetermined period. The processing in S1212 is the same as the processing in S204, and a description thereof will be omitted.

  In step S1213, the CPU 101 projects invisible light in the second projection mode. In the present embodiment, the CPU 101 projects the invisible light along the outer periphery of the projection area. It should be noted that the invisible light may be projected on areas other than the outer periphery of the projection area, for example, on the left and right ends of the projection area. The invisible light in the second projection mode may have any shape as long as it can detect that a user or the like enters the projection area. The invisible light need not always be projected, and may be projected at predetermined time intervals.

In S1214, the CPU 101 detects the presence or absence of a user in the projection area. Specifically, the presence or absence of a user is detected by receiving the reflected light of the invisible light projected in S1213. Note that the CPU 101 may detect a user operation as in S1207.

  In S1215, the CPU 101 determines whether a user exists in the projection area based on the detection result in S1214. If it is determined that a user exists, the process proceeds to S1216; otherwise, the process proceeds to S1217.

  In S1216, the CPU 101 sets the projection mode of the infrared laser light to the first projection mode. The processing in steps S1217 to S1222 is the same as the processing in steps S209 to S214, and a description thereof will be omitted.

  In the present embodiment, by executing the above processing, when no user is present in the projection area, the invisible light is projected to the outer frame to reduce the output of the invisible light, thereby reducing power consumption.

  FIGS. 13A and 13B show a projection shape of infrared laser light according to the present embodiment. FIG. 13A is a conceptual diagram when an infrared laser beam is projected in the first projection mode. The projection image 1300 is an image projected by the projection device 400. In the example of FIG. 13A, a part of the projection image 1300 includes a UI 1301 that is a UI image for the user to operate the projection image. When the projection mode is the first projection mode, the projection device 400 detects the user's UI operation by projecting an infrared laser beam onto the UI 1301 portion.

  FIG. 13B is a conceptual diagram in the case where infrared laser light is projected in the second projection mode. An area 1302 indicates a projection shape of the infrared laser light projected by the projection device 400. The projection device 400 reduces the power consumption during standby of the user operation by projecting the infrared laser light only on the area 1302 of the projection image. Note that when projecting the region 1302, power consumption may be reduced by thinning out pixels and projecting. In the above-described processing, invisible light is regularly projected because the shape of the projection surface may change. However, the processing may be performed only once at the beginning of the processing. When no visible light is projected, only the invisible light may be projected. Note that, in the present embodiment, an example has been described in which invisible light is projected onto the upper, lower, left, and right end regions in the projected image as the outer frame of the projected image. However, the outer frame is invisible light in order to detect the presence or absence of the user. What is necessary is just to show the area | region where is projected. For example, invisible light may be projected around the UI image as an outer frame.

(Embodiment 5)
In the fourth embodiment described above, in the case of the second projection mode, an example has been described in which invisible light is projected onto the upper, lower, left, and right end regions in the projected image. In the present embodiment, an example will be described in which, in the case of the second projection mode, an infrared laser beam is projected outside the projected image as an outer frame of the projected image. The configuration of the projection device 400 according to the present embodiment is the same as that of the above-described fourth embodiment, and a description thereof is omitted.

  FIG. 14 is a conceptual diagram of a projection image and infrared laser light according to the present embodiment. The projection device 400 according to the present embodiment projects an infrared laser light region 1402 outside the projection image 1300. This is because by detecting (detecting) the user outside the projection image, the switching process from the second projection mode to the first projection mode is completed within the time until the user performs the UI operation.

  Further, the area 1402 can be projected using a portion of the scanning line on the scanning mirror 113 that is not used as an effective pixel. For example, since the scanning mirror 113 has a portion where the pixel density is not uniform at the left and right ends of the projection area, the area 1402 is projected outside the image projection plane by using the left and right ends of the projection area of the scanning mirror 113. May be.

(Embodiment 6)
In the first to fifth embodiments, the description has been given of the projection apparatus that scans the projection surface with the infrared laser light and the visible laser light (RGB). In a sixth embodiment, an apparatus that irradiates an object (operation object) on a projection surface with infrared laser light (predetermined light) and detects a user operation on the object will be described.

  FIG. 15 is a schematic diagram illustrating a detection device 1500 that irradiates infrared light onto an object and detects a user's operation on the object. The detection device 1500 irradiates an infrared laser beam to an object (button) 1501 provided on a wall surface and detects a reflected wave to detect a user operation on the object 1501. Since the detection method is the same as the method described in the first embodiment and the like, detailed description will be omitted.

  In response to detecting a user operation on object 1501, detecting device 1500 transmits information indicating the detection result to projector 1502. Projector 1502 executes processing according to the received information indicating the detection result. For example, the projector 1502 projects an image indicating additional information on the object 1501 for which a user operation has been detected. Further, projector 1502 changes an image that is previously projected on object 1501 for which a user operation has been detected, in response to the detection of the user operation.

  Note that the detection system illustrated in FIG. 15 is an example, and is not limited to the above-described example. The device that outputs the detection result is not limited to the projector, and may be any of a management server, a personal computer, a tablet, and a smartphone. The region irradiated with the infrared light is not limited to the wall surface as shown in FIG.

  FIG. 16 is a block diagram showing functional blocks of the detection device 1500. The detection device 1500 includes a CPU 1601, a memory 1602, an image generation unit 1603, an area determination unit 1605, a light source control unit 1606, a laser driver 1607, an infrared laser light source 1608, a mirror control unit 1612, and a scanning mirror 1613. The detection device 1500 includes an operation unit 1614, a timer 1615, a light receiving unit 1630, and a detection unit 1631. Each block of the detection device 1500 exhibits the same function as the block of the same name shown in the first to fifth embodiments. Hereinafter, differences from the blocks having the same names in the first to fifth embodiments will be described.

<Processing content>
FIG. 17 is a flowchart illustrating a detection process that is started in response to the detection device 1500 being instructed to execute a user operation detection process.

  The detection device 1500 is activated, and the detection process is started in response to the user operating the operation unit 1614 to instruct the execution of the detection process. Note that the execution of the detection process may be automatically started in response to the activation of the detection device 1500.

  In S1701, the detection device 1500 generates a reference distance image. This process is the same as the process of S301. Immediately after the detection process is started, it is assumed that there is no intruding object between the detection device 1500 and the detectable area including the object 1501. The area determining unit 1605 determines the entire area of the detectable area as an irradiation area (output area) of the infrared laser light. The laser driver 1607 and the mirror control unit 1612 control the operations of the infrared laser light source 1608 and the scanning mirror 1613 so that the entire area of the detectable area is irradiated with the infrared laser light. The CPU 1601 stores the time difference between the timing at which the infrared laser light is output and the timing at which the light receiving unit 1630 receives the reflected wave in the memory 1602 as a reference parameter for each irradiation position.

  In step S1702, the area determination unit 1605 determines the irradiation area of the infrared laser light as a monitoring area (first area) that is an area for detecting intrusion of a user. In the present embodiment, the area determination unit 1605 determines a frame-shaped area corresponding to four sides of the detectable area as a monitoring area.

  FIGS. 18A to 18E are schematic diagrams illustrating an irradiation region of infrared laser light. FIG. 18A is a schematic diagram illustrating a frame-shaped area corresponding to four sides of a detectable area set as an area for intrusion detection. Note that the monitoring area may have another shape. When the object 1501 is provided on a wall surface and the user is likely to enter the object 1501 from the left and right, areas corresponding to the two left and right sides of the detectable area may be set as the monitoring area. FIG. 18B is a schematic diagram illustrating a frame-shaped area corresponding to two left and right sides of a detectable area, which is set as an intrusion detection area. If the user is highly likely to invade the object 1501 from above and below, an area corresponding to the left and right sides of the detectable area may be set as the monitoring area. FIG. 18C is a schematic diagram showing a frame-shaped area corresponding to two upper and lower sides of the detectable area set as the area for intrusion detection. At least the monitoring area is an area smaller than the detectable area. The monitoring area is an area smaller than a detection area (second area) described later.

  In step S1703, the laser driver 1607 and the mirror control unit 1612 control the operations of the infrared laser light source 1608 and the scanning mirror 1613 so that the determined monitoring area is irradiated with the infrared laser light.

  In step S1704, the light receiving unit 1630 receives the infrared laser light. The detecting unit 1631 compares the time difference between the timing at which the light receiving unit 1630 receives the infrared laser light and the timing at which the infrared laser light is output, and the time difference acquired in S1701 to determine whether the user or object has entered. Is detected. Note that the processing in S1703 and S1704 is executed for each output of the infrared laser light.

  In step S1705, the CPU 1601 determines whether or not an object (pointing object) such as a user or a pointing rod for operation input has entered the monitoring area. Here, the method of determining intrusion is the same as in S1214 and S1215. When there is no intrusion, the process proceeds to S1715, and when there is an intrusion, the process proceeds to S1706. Note that the CPU 1601 can be regarded as a detection unit that detects intrusion of a person or an object by detecting invisible light.

  In step S1715, the CPU 1601 determines whether an instruction to end the detection processing has been input. For example, when a user inputs a termination instruction to the operation unit 1614 or a start / stop instruction of the detection device 1500, it is determined that a detection processing termination instruction has been input. Here, when the end instruction is input, the detection processing ends. If an instruction to end the detection process has not been input, the process advances to step S1703.

  In step S1706, the region determination unit 1605 determines the irradiation region of the infrared laser light as the detection region. The detection area (second area) is an area for detecting the presence / absence of a user operation, and is an area wider than the above-described monitoring area.

  FIG. 18D is a schematic diagram showing the entire detectable area determined as the detection area. If information indicating the position of the detection target object 1501 is stored in the memory 1602 in advance, the region determination unit 1605 determines a region including the detection target object 1501 as the detection region. FIG. 18E is a schematic diagram illustrating a region including the detection target object 1501 determined as the detection region. In this detection flow, the entire detectable area is set as the detection area.

In step S1707, the CPU 1601 resets the count of the timer 1615 and starts time measurement.

  In step S1708, the laser driver 1607 and the mirror control unit 1612 control the operations of the infrared laser light source 1608 and the scanning mirror 1613 so that the determined detection area is irradiated with the infrared laser light.

  In S1709, light receiving unit 1630 receives the infrared laser light. The detecting unit 1631 compares the time difference between the timing at which the light receiving unit 1630 receives the infrared laser light and the timing at which the infrared laser light is output, and the time difference acquired in S1701 to determine whether the user or object has entered. Is detected. Note that the processing of S1708 and S1709 is executed for each output of the infrared laser light.

  In step S1710, the CPU 1601 determines whether an operation has been performed on the detection area. When detecting an intrusion of any object into the area including the object 1501, the CPU 1601 determines that an operation on the object 1501 has been performed. If an operation has been detected (S1710-Yes), the flow proceeds to S1711; otherwise (S1710-No), the flow proceeds to S1712.

  In step S1711, the CPU 1601 outputs detection information corresponding to the object 1501 whose operation has been detected to another functional unit or an external device via the communication unit 1620. Then, the process returns to S1707.

  In step S1712, the CPU 1601 determines whether the elapsed time measured by the timer 1615 has exceeded a predetermined threshold. If the elapsed time exceeds the threshold, the process proceeds to S1713; otherwise, the process proceeds to S1714.

  In step S1713, the area determination unit 1605 determines the irradiation area of the infrared laser light as the monitoring area. The processing in S1713 is the same as the processing in S1702, and a description thereof will not be repeated. After the processing in S1713, the process proceeds to S1715. In this series of processing, after changing the irradiation area of the infrared laser light from the monitoring area to the detection area, if the operation on the detection area is not performed for a predetermined period, the irradiation area of the infrared laser light is monitored. This is a process of returning to the area and returning to the mode for detecting the intrusion of the user. This is a process for a so-called no operation timeout.

  In step S1714, the CPU 1601 determines whether an instruction to end the detection process has been input, as in step S1715. If it is determined in step S1714 that an instruction to end the detection process has been input, the detection process ends. If it is determined in step S1714 that an instruction to end the detection process has not been input, the process returns to step S1708.

  By the above-described series of processing, when the user and the pointing object have not entered the detectable area, the irradiation area of the infrared laser light is limited to the monitoring area, and the detection target is set in response to the detection of the intrusion. Can be irradiated with infrared laser light. This makes it possible to reduce the power consumption of the detection device without outputting infrared laser light in a wide range in a situation where the possibility of operation by the user and the pointing object is low.

  As described above, the preferred embodiments of the present invention have been described, but the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the gist.

(Modification)
When the projection image does not include the UI image, the projection device described above may project the infrared laser light onto the outer frame of the projection image regardless of the projection mode. This is particularly effective when the projected image is switched or when a moving image is projected. If the projection image does not include a UI image, the power consumption may be reduced by thinning out the lighting intervals of the infrared laser light. Further, in the projection image, when the UI image is first projected from a state where no UI image is present, the infrared laser light may be first projected onto the outer frame portion.

  In the above-described embodiment, an example in which the projected shape of the invisible light is rectangular has been described. However, the projected shape is not particularly limited, and may be, for example, a circle or an ellipse. In addition, the projection region of the invisible light is not particularly limited. For example, in the second projection mode, the invisible light is projected on only one of the upper end, the lower end, the left end, and the right end of the projected image, or only on the left and right ends. Is also good. When the projected image is rectangular, invisible light may be projected on a corner. Further, in the second projection mode, an invisible light such as a rectangle or an ellipse surrounding the UI image may be projected.

  In the above-described embodiment, an example has been described in which the projection device has a mode in which invisible light is projected on a UI image and a mode in which invisible light is projected on an outer frame (frame-shaped region) of the projected image. A projection mode for projecting invisible light onto the entire projection image may be provided. Thus, for example, the invisible light projection area can be switched to the outer frame of the projection image or the entire projection image, or the invisible light projection area can be switched to the UI image or the entire projection image. Further, the projection device may have all three modes.

  In the first to seventh embodiments, infrared laser light (invisible light) is used as light for detecting an operation, but the color (wavelength) of light used for detection is not limited to this. For example, laser light other than infrared light having a peak wavelength in the invisible light wavelength range (range) may be used, or laser light having a peak wavelength in the visible light wavelength range may be used.

  In addition, the idea of the present embodiment can be used in a detection device that irradiates an electromagnetic wave in a high frequency band called a so-called Radio Frequency and detects a user and a pointing object based on the reflected wave. More specifically, at least a part of the outer periphery of the detection area is set as an irradiation area (monitoring area) for intrusion detection. Then, the irradiation area is changed in response to detecting any intrusion of an object into the monitoring area while irradiating the monitoring area with the high-frequency electromagnetic waves. The irradiation area after the change may or may not include the monitoring area. The irradiation area after the change is an area including a predetermined detection target object. Before and after intrusion into the monitoring area, the wavelength of the light (electromagnetic wave) to be irradiated may be changed in addition to the irradiation area. For example, it is also possible to switch to irradiation of infrared light after detecting intrusion into the monitoring area by RF electromagnetic waves.

(Other)
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

1500: detection device 1601: CPU 1605: area determination unit 1608: infrared laser light source

Claims (24)

Determining means for determining a predetermined light output area,
Output means for outputting the predetermined light to the output area,
Detecting means for detecting intrusion of an object into the output area by detecting the predetermined light;
With
The determining unit changes the output region from a first region to a second region wider than the first region, according to a detection result of the detecting unit.
A control device characterized by the above-mentioned. The predetermined light is invisible light having a peak wavelength in a wavelength region of invisible light,
The control device according to claim 1, wherein: The first area is a part of an outer periphery of an area including the operation object,
The second area is an area including the operation object,
The control device according to claim 1 or 2, wherein: Further comprising projection means for projecting an image to a projection area of a projection surface using visible light,
The first region is a region corresponding to at least a part of an outer frame of the projection region,
The second area is an area of the projection area corresponding to the operation object included in the image.
The control device according to claim 1 or 2, wherein: Position acquiring means for acquiring a position of a UI (user interface) in the image projected by the projecting means,
The determining means determines an area on which the UI is projected as the second area,
The control device according to claim 4, wherein: The image processing apparatus further includes a generating unit configured to generate image data including the UI, wherein the position obtaining unit obtains the position of the UI based on the image data generated by the generating unit.
The control device according to claim 5, wherein: A receiving unit that receives image data from an external device,
The position acquisition unit acquires a position of the UI based on the image data.
The control device according to claim 5, wherein: A receiving unit that receives, from an external device, position information of the UI corresponding to the image data and the image data,
The position acquisition unit acquires position information received by the reception unit as position information of the UI.
The control device according to claim 5, wherein: A process of executing a predetermined process in response to the detection unit detecting an intrusion of an object into the second region while the output unit is outputting the predetermined light to the second region; Further comprising means,
The control device according to any one of claims 4 to 8, wherein: The detecting means outputs the predetermined light based on a time difference between a timing at which the output means outputs the predetermined light and a timing at which the detecting means detects the predetermined light. Detecting the intrusion of objects in the area where
The control device according to claim 1, wherein: The determining means, when the output means is outputting the predetermined light to the second area, if the detection means does not detect the intrusion of an object for a predetermined time or more, the output area is set to the output area. Change to the first area,
The control device according to claim 10, wherein: A determining step of determining a predetermined light output area,
An output step of outputting the predetermined light to the output area,
A detecting step of detecting an intrusion of an object into the output area by detecting the predetermined light;
Has,
In the determining step, the output area is changed from a first area to a second area wider than the first area according to a detection result of the detecting step.
A control method characterized in that: The predetermined light is invisible light having a peak wavelength in a wavelength region of invisible light,
The control method according to claim 12, wherein: The first area is a part of an outer periphery of an area including the operation object,
The second area is an area including the operation object,
The control method according to claim 12, wherein: Further comprising a projection step of projecting the image onto a projection area on a projection surface using visible light,
The first region is a region corresponding to at least a part of an outer frame of the projection region,
The second area is an area of the projection area corresponding to the operation object included in the image.
The control method according to claim 12, wherein: A position acquisition step of acquiring a position of a UI (user interface) among the images projected in the projection step;
In the determining step, an area on which the UI is projected is determined as the second area.
The control method according to claim 15, wherein: The method further includes a generation step of generating image data including the UI, wherein the position obtaining step obtains the position of the UI based on the image data generated in the generating step.
17. The control method according to claim 16, wherein: Further comprising a receiving step of receiving image data from an external device,
The position obtaining step obtains the position of the UI based on the image data,
17. The control method according to claim 16, wherein: A receiving step of receiving, from an external device, image data and the position information of the UI corresponding to the image data,
In the position obtaining step, the position information received in the receiving step is obtained as position information of the UI.
17. The control method according to claim 16, wherein: In the state where the predetermined light is being output to the second area, the method further includes a processing step of executing a predetermined process in response to detecting an intrusion of an object into the second area.
The control method according to any one of claims 15 to 19, wherein: In the detecting step, based on a time difference between a timing at which the predetermined light is output in the output step and a timing at which the predetermined light is detected in the detecting step, an object in an area outputting the predetermined light is detected. Detect intrusions,
The control method according to any one of claims 12 to 20, wherein: In the determining step, while outputting the predetermined light to the second area, if the intrusion of an object is not detected for a predetermined time or more in the detecting step, the output area is set to the first area. change,
22. The control method according to claim 21, wherein:   A program for causing a computer to execute each step of the control method according to any one of claims 12 to 22.   A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to any one of claims 12 to 22.