JP7224799B2 - IMAGE PROJECTION DEVICE, CONTROL METHOD FOR IMAGE PROJECTION DEVICE, AND PROGRAM - Google Patents

Description

The present invention relates to an image projection device, an image projection device control method, and a program.

In recent years, it is called an interactive system that enables control such as changing the projected presentation content (image) by holding an indicator such as the user's hand over the projected image projected by the projector (image projection device). things are known. In order to enable such control, the projector first uses a built-in camera or the like to detect that an indicator is held over a predetermined position of the projected image. The projector then instructs the PC to update the content to be output to the projector. This instruction may be communicated via a path different from the image transmission path between the PC and the projector, for example, via a USB (Universal Serial Bus) cable. Furthermore, in such a system, for the purpose of providing an instruction area (operation area) for detecting the pointer within the projection image (display image; projection screen), the content is reduced to create a margin, and the margin is filled with Lay out the indicated area. For example, as a technology of an interactive system that lays out an instruction area in a margin, Patent Document 1 discloses reducing content to generate a margin when it is detected that a user's shadow is superimposed on a projected image.

Further, in the above-described interactive system, the timing for providing the area for detecting the pointer may be triggered by the transition (start) of the operation mode in which the projector detects the pointer. According to this, since the area for detecting the pointer is displayed immediately after transition to the operation mode, it is possible to quickly control the content by holding the pointer over the projection image.

JP 2016-114626 A

However, when the area for detecting the pointer is displayed by the projector transitioning to the operation mode for detecting the pointer, the area may be displayed even when communication via the USB cable described above is not possible, for example. had been displayed. In other words, even when the projected image (content) cannot be controlled by holding the pointer over the projected image, the area is displayed. Therefore, in an image projection apparatus capable of controlling a displayed image according to a user's operation, the user may not be able to easily determine whether or not the image projection apparatus is in the controllable state.

SUMMARY OF THE INVENTION Accordingly, the present invention provides an image projection apparatus capable of controlling a displayed image in accordance with a user's operation and allowing the user to easily determine whether or not the controllable state exists. With the goal.

A first aspect of the present invention is
An image projection device including an operation mode for receiving a user operation corresponding to a user's operation on a display unit among predetermined user operations,
display control means for controlling display of an image projected on the display unit;
input means for inputting an instruction to start the operation mode;
determination means for determining whether or not the operation mode can be started;
has
The display control means is
controlling the display unit to display the image and an operation area for detecting the user's motion when the operation mode can be started;
controlling the display unit to display the image and not to display the operation area when the operation mode cannot be started;
When displaying the operation area, the display control means displays an icon indicating the predetermined user operation in the operation area,
The determination means further determines whether or not the display control means is no longer in a state in which the switching of the images can be controlled after the operation mode is started,
The display control means maintains the display of the operation area and changes the icon to a different form when it is no longer possible to control the switching of the images after the operation mode is started.
This image projection device is characterized by:

A second aspect of the present invention is
A control method for an image projection device including an operation mode for receiving a user operation corresponding to a user's operation on a display unit among predetermined user operations,
a display control step of controlling display of an image projected on the display unit;
a determination step of determining whether or not the operation mode can be started when an instruction to start the operation mode is input;
has
In the display control step,
When it is determined in the determination step that the operation mode can be started, the display unit is controlled to display the image and an operation area for detecting the user's motion. death,
if it is determined in the determination step that the operation mode cannot be started, displaying the image and controlling the display unit so as not to display the operation area;
In the display control step, when displaying the operation area, an icon indicating the predetermined user operation is displayed in the operation area;
In the determination step, further, after the operation mode is started, it is determined whether or not it is no longer possible to control switching of the images in the display control step;
In the display control step, when it is no longer possible to control the switching of the images after the operation mode is started, the display of the operation area is maintained and the icon is changed to a different form.
A control method for an image projection device characterized by:

According to the present invention, even in an image projection device capable of controlling a displayed image according to a user's operation, it is possible to allow the user to easily determine whether or not the control is possible.

Configuration diagram of a projector according to the first embodiment 1 is an external view of a display control system according to a first embodiment; FIG. Flowchart showing transition to bi-directional mode with conventional problems FIG. 4 is a diagram for explaining how to grasp the positions of operation icons according to the first embodiment; Flowchart showing transition to bidirectional mode according to the first embodiment Display example showing an error message according to the first embodiment Flowchart showing transition to bidirectional mode according to modification 1 Display example indicating that user operation cannot be executed according to modification 1

EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated in detail. The embodiment described below is an example for realizing the present invention, and should be appropriately modified or changed according to the configuration of the apparatus to which the present invention is applied and various conditions. It is not limited to the embodiment of Also, a part of each embodiment described later may be appropriately combined.

<Embodiment 1>
The configuration of the projector (image projection device) according to this embodiment and the general projection operation of the projector will be described below. Also, a process of starting (transitioning) an interactive mode (interactive mode; operation mode) that accepts an operation by a user's "hand holding" will be described. In the present embodiment, when the projector is not in a state in which it is possible to operate by "holding it over hand", an error message is displayed without generating an area for operation by "holding over hand" even if the mode transitions to the interactive mode. Notify users. As a result, the user can easily understand that the content cannot be controlled according to the operation by "holding over the hand". Here, "hand holding" indicates an action of the user holding a hand, which is an indicator, over a predetermined position in the projected image. Note that the pointing body is not limited to the hand, and may be a pointing stick or a writing utensil. Note that the user's motion is not limited to "holding up", and may be a motion of opening and closing the hand over the area for operation, or a motion of waving the hand left and right.

[Projector configuration]
FIG. 1 is a diagram showing the configuration of a projector 10 of this embodiment. The projector 10 has a control section 100 for controlling each functional section as a whole. The projector 10 has a liquid crystal unit 101, a liquid crystal driving unit 102, a light source 103, a projection optical system 104, a light source control unit 105, and an optical system control unit 106 for projecting an image onto a projection target. The projector 10 has an image acquisition unit 110, an image acquisition unit 111, a USB interface 112, a communication unit 113, and an operation unit 114 for transmitting and receiving data to and from the outside. The projector 10 includes a tilt sensor 120, a timer 121, a thermometer 122, a cooling unit 123, an infrared receiving unit 124, a focus detection unit 125, an imaging unit 126, and a screen for acquiring and adjusting information on the environment inside and outside the projector 10. It has a photometry unit 127 . The projector 10 has a notification unit 130 and a notification control unit 131 for notifying the user. The projector 10 has a power supply unit 140, a power input unit 141, and a battery 142 for supplying electric power to the inside. Projector 10 has internal memory 150 and RAM 151 for storing data. The projector 10 has an image processing section 160 and a file reproducing section 161 for performing image processing on image data. Note that the functional units described above are functional units for performing general projector operations, and include functional units that are not essential to the present embodiment. In this embodiment, a projector having a liquid crystal unit 101, a light source 103, and a projection optical system 104 will be described. effect can be obtained. An outline of the above functional units will be described below.

(Functional part for controlling each function part of the projector as a whole)
The control unit 100 is, for example, a CPU (Central Processing Unit) that controls a plurality of functional units of the projector 10 . The control unit 100 is connected to a plurality of functional units via a bus 170, and can access the functional units for control instructions, data transmission/reception, and the like. In this embodiment, the control unit 100 controls the liquid crystal unit 101 , the light source 103 , the projection optical system 104 , the operation unit 114 , the notification unit 130 , and the power input unit 141 among all the functional units of the projector 10 via the bus 170 . , and the battery 142 are directly controlled.

(Functional part for projecting an image onto a projection object)
The liquid crystal unit 101 is composed of one liquid crystal panel, three liquid crystal panels, or the like, and forms an image on the liquid crystal panel (display panel) by transmitting light emitted by the light source 103 . A reflective liquid crystal panel or DMD (Digital Micromirror Device) panel that reflects light may be used instead of the transmissive liquid crystal panel that transmits light.

The liquid crystal drive unit 102 causes the liquid crystal panel of the liquid crystal unit 101 to form an image based on the input image data. A light source 103 supplies light to the liquid crystal portion 101 . In this embodiment, the projector 10 has only one light source, and the amount of light projected onto the liquid crystal unit 101 does not change depending on the position within the image. Note that the number of light sources in the projector 10 according to this embodiment is not limited to one, and may be any number.

The projection optical system 104 projects an optical image obtained by transmitting the light emitted from the light source 103 through the liquid crystal unit 101 onto a projection object such as a screen. A light source control unit 105 controls the light amount of the light source 103 and the like. The optical system control unit 106 controls the operations of the zoom lens and focus lens of the projection optical system 104, and performs zoom magnification and focus adjustment. Note that the projection target on which the image is projected by the projector 10 is not limited to the screen, and may be, for example, a building or a balloon.

(Functional part for sending and receiving data to and from the outside of the projector)
The image acquisition unit 110 and the image acquisition unit 111 acquire image data including one or more content areas from a PC, Blu-ray (registered trademark) recorder, STB (Set Top Box), or the like. In this embodiment, the image acquisition unit 110 and the image acquisition unit 111 acquire image data via an HDMI (registered trademark; High-Definition Multimedia Interface) cable. The cable is not limited to the HDMI cable, but may be an SDI (Serial Digital Interface) cable, and the image acquisition units 110 and 111 may acquire image data through wireless communication.

Further, the image acquisition unit 110 and the image acquisition unit 111 hold an EDID (Extended Display Identification Data) indicating information about an image format that can be received by the projector 10 . The image acquisition unit 110 and the image acquisition unit 111 notify the EDID based on the request from the external device. Further, the image acquisition unit 110 and the image acquisition unit 111 are configured to have an HPD (Hot Plug Detect) capable of notifying the external device that the cable is connected in the connection line with the external device. . It is assumed that the connected external device outputs image data based on the information included in the EDID. Here, the information included in the EDID is information such as the resolution of receivable image data, signal timing information such as frame rate, and bit depth.

The USB interface 112 externally receives or externally writes various information data files such as image data (video data; image file). The USB interface 112 may also be connected to a pointing device, a keyboard, a USB flash memory, and the like. Note that the USB interface 112 supports various transfer modes defined by USB, and can also perform data communication in an interrupt transfer mode used in a bidirectional mode (interactive mode), which will be described later. The communication unit 113 transmits and receives various information data files such as image data and other command signals from the intranet and the Internet. For example, the communication unit 113 and the intranet or the Internet are connected via a wired LAN, wireless LAN, or the like. The operation unit 114 is a button, a switch, or the like, and receives operations such as power ON/OFF of the projector 10 from the user.

(Functional part for acquiring and adjusting information on the environment inside and outside the projector)
Tilt sensor 120 detects the physical tilt of projector 10 . The timer 121 detects the operation time of the projector 10, the operation time of each functional unit, and the like. A thermometer 122 measures the temperature of the light source 103 of the projector, the temperature of the liquid crystal unit 101, the outside air temperature, and the like. The cooling unit 123 cools the projector 10 by releasing the heat inside the projector 10 to the outside according to the temperature detected by the thermometer 122 . The cooling unit 123 is composed of, for example, a heat sink and a fan.

The infrared receiver 124 receives infrared rays from a remote controller attached to the projector 10 or other devices, and sends the received signal to the controller 100 . For example, the infrared receivers 124 are installed at a plurality of locations such as in the front-rear direction of the projector. The focus detection unit 125 detects the distance between the projector 10 and the screen, and detects the focal length. The imaging unit 126 images the direction of the screen.

A screen photometry unit 127 measures the amount of light reflected by the screen and the display brightness of the projection screen. Specifically, the screen photometry unit 127 calculates and stores the display luminance from the output light amount of the light source 103, the distance between the projector 10 and the screen, the zoom state of the projection optical system 104, and the like. For example, a distance sensor such as the focus detection unit 125 measures the distance from the projector 10 to the screen. The screen photometry unit 127 calculates the projected area from a pre-stored relationship of the projection size corresponding to the zoom state of the projection optical system 104 and the distance to the screen. Then, the screen photometry unit 127 calculates the display luminance of the projection screen based on the calculated projected area and the output light amount of the light source 103 .

(Functional part for notifying the user)
The notification unit 130 is arranged at a position where the user can see the projector 10, and displays the status of the projector 10, warnings, and the like to notify the user. Notification control unit 131 controls notification unit 130 .

(Functional part for supplying power to the projector)
The power supply unit 140 controls power supply to each functional unit of the projector 10 . The power supply input unit 141 receives AC power from the outside, rectifies it to a predetermined voltage, and supplies it to the power supply unit 140 . Battery 142 is a chargeable/dischargeable storage battery that supplies power to power supply unit 140 when power is not supplied from power supply input unit 141 .

(Functional part for storing data)
The internal memory 150 stores files of various information data such as image data and data related to user settings, and is composed of a semiconductor memory, a hard disk, or the like. The RAM 151 is used as a frame memory for developing programs stored in the internal memory 150 and images to be projected.

(Functional part for performing image processing on image data)
The image processing unit 160 converts the number of pixels (resolution) of the image data according to the number of pixels of the liquid crystal panel of the liquid crystal unit 101 . In order to alternately drive the liquid crystal panel, the image processing unit 160 doubles the number of frames of the input image data and performs processing such as correction suitable for image formation by the liquid crystal panel. Here, the AC driving of the liquid crystal panel is driving in which the direction of the voltage applied to the liquid crystal of the liquid crystal panel is alternately changed for display. It is used. At this time, the liquid crystal driving unit 102 needs to send the image data for the normal direction and the image data for the reverse direction one by one. I do. Based on the image data processed by the image processing unit 160, the liquid crystal driving unit 102 causes the liquid crystal panel of the liquid crystal unit 101 to form an image.

In addition, when the projector 10 projects an image obliquely onto the screen and the projection screen is distorted into a trapezoidal shape or the like, the image processing unit 160 adjusts the shape of the projected image so as to cancel out the trapezoidal distortion. perform keystone correction that transforms the Keystone correction changes the horizontal and vertical enlargement/reduction ratio of the projected image. In this way, the trapezoidal distortion of the projection screen and the distortion of the keystone-corrected image are canceled, and a nearly rectangular image with a normal aspect ratio is displayed on the screen. This keystone correction may be based on the tilt angle obtained by the tilt sensor 120, may be based on an image captured by the imaging unit 126, or may be performed by the user operating the operation unit 114 or the like. may The image processing unit 160 also performs image enlargement/reduction, display layout change, composition of two or more images, and OSD (On Screen Display) menu superimposition processing for accepting user instructions.

The file reproduction unit 161 restores (decodes), for example, file data encoded by an external device so that the data can be used in the projector 10 .

[Basic Projector Operation]
Here, an overview of the basic operation of the projector 10 will be described. Here, an overview of the flow from when the projector 10 is turned on to when it projects an image and when it is turned off will be described as an example.

When an instruction to turn on the power is input through the operation unit 114, the control unit 100 instructs the power supply unit 140 to supply power to each functional unit, thereby placing each functional unit in a standby state. After the power is turned on, the controller 100 instructs the light source controller 105 to start emitting light from the light source 103 .

Next, the control unit 100 instructs the optical system control unit 106 to adjust the projection optical system 104 based on the focus information obtained by the focus detection unit 125 . The optical system control unit 106 operates the zoom lens and the focus lens of the projection optical system 104 so as to form an image of projection light on the screen. In this way it is ready for projection.

Next, the image processing unit 160 synthesizes the image data input to the image acquisition unit 110 in a predetermined layout based on the instruction from the control unit 100 , and converts the image data into a resolution suitable for the liquid crystal unit 101 . Further, the image processing unit 160 applies gamma correction, correction to prevent luminance unevenness, and keystone correction to the image data based on instructions from the control unit 100 . The image data corrected by the image processing unit 160 is formed as an image on the liquid crystal unit 101 by the liquid crystal driving unit 102 . An image formed on the liquid crystal panel of the liquid crystal unit 101 is projected onto the screen from the projection optical system 104 by light emitted from the light source 103 .

Also, during projection, the control unit 100 detects the temperature of the light source 103 and the like with the thermometer 122 . Then, for example, when the temperature of the light source 103 changes to 40 degrees or more, the control unit 100 operates the cooling unit 123 to cool the light source 103 .

Then, when an instruction to turn off the power is input through the operation unit 114, the control unit 100 instructs each functional unit to perform end processing. Furthermore, when preparations for termination are completed, the power supply section 140 sequentially terminates power supply to each functional section based on an instruction from the control section 100 . The cooling unit 123 operates for a while after the power is turned off to cool the projector 10 .

[Construction of display control system and interactive mode]
Next, the configuration of the display control system 1 that controls an image projected using the projector 10 will be described. FIG. 2A is a diagram showing the appearance of the display control system 1 operating in the interactive mode (operation mode). A display control system 1 has a projector 10 , a PC 200 , a USB cable 300 , an HDMI cable 400 and a screen 500 .

PC 200 is an external device for transmitting image data to projector 10 . Therefore, any device such as a smart phone, a tablet terminal, an imaging device, a server, or the like may be used as long as the device is capable of transmitting image data, without being limited to a PC.

The USB cable 300 is a cable (transmission path) that connects the projector 10 and the PC 200 for data communication. It is assumed here that the USB cable 300 is connected to the USB interface 112 inside the projector 10 . Also, control data and the like for the projector 10 to control the PC 200 are communicated via the USB cable 300 . The communication of control data is not limited to being performed by the USB cable 300, and may be performed by, for example, wireless communication, a LAN (Local Area Network) cable, or a serial cable.

The HDMI cable 400 is a cable (transmission path) that connects the projector 10 and the PC 200 for image transmission. Here, it is assumed that the HDMI cable 400 is connected to the image acquisition unit 110 inside the projector 10 . Also, the projector 10 receives image data from the PC 200 via the HDMI cable 400 . Note that image data communication is not limited to being performed using an HDMI cable, and may be performed using, for example, an SDI cable.

The screen 500 is a display unit (projection object) that displays an image (projection image; display image) projected from the projector 10 (projection optical system 104) in a display area. In interactive mode in FIG. 2A, the projected image has content 600 and operation icon area 601 . That is, when the projector 10 is operating in the interactive mode, the reduced content 600 is displayed on a part of the display area of the screen 500, and the operation icon area 601 is displayed on a part other than the display area. . However, the present invention is not limited to this, and the content 600 may be displayed over the entire display area, and the operation icon area 601 may be displayed so as to be superimposed on the content 600 . Note that the projector 10 projects a projection image onto the screen 500 through processing in the image processing unit 160 , the liquid crystal unit 101 , and the projection optical system 104 .

The content 600 is an image (content) included in image data transmitted by the PC 200, and is an area displayed on the screen 500 by being output (transmitted) from the PC 200 to the projector 10 via the HDMI cable 400. .

An operation icon area 601 is an operation area (image) for displaying an operation icon (operation item) for accepting an operation by "hand holding", and has an operation icon 602 and an operation icon 603 in this embodiment. In other words, the operation icon area 601 is an area for detecting "hand holding". Note that the operation icon area 601 is an area generated by the projector 10 and added to the content 600 and is not included in the image data output by the PC 200 . In addition, the control unit 100 performs switching control of the content 600 when the presenter 700 performs an action such as holding the hand over the operation icon 602 or the operation icon 603 . Specifically, in the projector 10 , the control unit 100 detects “hand holding” from the image captured by the imaging unit 126 . Then, in response to the detection, the control unit 100 transmits control data for instructing switching of image data to the PC 200 via the USB cable 300 . In other words, the control unit 100 receives a user operation corresponding to the operation icon with the “hand over” operation icon, and performs control such as switching of the content 600 . For example, when the operation icon 602 is “held over”, the control unit 100 controls switching such that new content is displayed. Controls the toggle such that is displayed. Further, the control unit 100 controls switching to scroll the content 600 upward when the operation icon 602 is “held over”, and scrolls the content 600 downward when the operation icon 603 is “held over”. Directional scrolling switching may be controlled. Furthermore, the control unit 100 controls switching such that the display of the content 600 ends when the operation icon 602 is “held over”, and when the operation icon 603 is “held over”, the content 600 is displayed. may be controlled such that the display of is started.

As described above, in the present embodiment, the user operation can be said to be an operation for switching the content 600 or the like. Further, in the present embodiment, each operation icon can be said to indicate (display) a user operation to be accepted by "holding the hand over". Further, the operation icon 602 and the operation icon 603 have two roles: a role of displaying an acceptable operation (predetermined user operation) and a role of accepting a user operation. It should be noted that the control unit 100 may change the size of the content 600 or change the display position of the content 600 without being limited to switching the content 600 according to the “hand holding”. Note that although the present embodiment is described assuming that there are two operation icons, the number of operation icons may be one or three or more. In addition, in the present embodiment, the imaging unit 126 can be said to be a reception unit, since it receives a user operation corresponding to "hand holding" from a captured image.

On the other hand, FIG. 2B is a diagram showing the appearance of the display control system 1 in a normal mode (during normal projection), which is a non-interactive mode. In the normal mode, the operation icon area 601 is not displayed on the screen 500, and the content 600 is displayed over the entire display area. In the normal mode, when the user instructs the projector 10 to transition to the interactive mode, the control unit 100 reduces the content 600 and lays out the operation icon area 601 in the margin and projects it on the screen 500 .

[Regarding transition processing to bidirectional mode that may cause conventional problems]
The processing of the projector 10 when the user instructs the projector 10 to switch to the interactive mode in the normal mode will be described with reference to the flowchart shown in FIG. 3(A). Here, the projector 10 will be described without considering the case where the content 600 cannot be controlled by the user's “hand gesture”, such as a connection failure of the USB cable 300 . In other words, the processing of the projector 10 that may cause the conventional problem that the user cannot determine whether or not the content 600 can be controlled by the user operation will be described. Note that when the user instructs the projector 10 to switch to the interactive mode via the infrared reception unit 124 or the operation unit 114, this flowchart starts. Note that "transition to the bidirectional mode" can also be said to be "starting the bidirectional mode".

In S<b>101 , control unit 100 receives an instruction to switch to the interactive mode via infrared reception unit 124 or operation unit 114 . Therefore, it can be said that the infrared reception unit 124 and the operation unit 114 are input means for inputting an instruction to transition to the bidirectional mode.

In S<b>102 , the control unit 100 performs adjustment processing (calibration processing) for grasping the positions where the operation icons 602 and 603 are displayed on the screen 500 . Specific adjustment processing will be described later in detail.

In S103, the control unit 100 instructs the image processing unit 160 to reduce the image data (content 600) received from the PC 200 to a predetermined size. Note that the predetermined size may be a size preset by the user, or may be determined according to the size of the projection image projected onto the screen 500 by the control unit 100 .

In S104, that is, the control unit 100 controls the image processing unit 160 to change the display layout of the projection image on the screen 500 from the display layout before transitioning to the interactive mode. Specifically, the control unit 100 controls the image processing unit 160 to lay out the reduced image data so as to be displayed in the upper left corner of the projected image as the content 600 as shown in FIG. 2(A). Further, the control unit 100 reads out the images of the operation icon 602 and the operation icon 603 from the internal memory 150, controls the image processing unit 160, and superimposes and displays them at the positions grasped in S102. Therefore, when the processing in S104 ends, it can be said that the mode has changed to the bi-directional mode, since it is possible to accept the "hand over". Note that it is not always necessary to reduce the image data in S103. That is, for example, the content 600 may be displayed over the entire display area of the screen 500 in S104, and each operation icon may be superimposed on the content 600 and displayed. In addition, the control unit 100 can be said to be display control means since it controls the display on the screen, which is the display unit.

Although the content 600 is displayed in the upper left of the projected image, it may be displayed in the lower left, and the position of the content 600 in the projected image may be changed according to the position of the presenter 700 . For example, it is preferable from the viewpoint of user operability that the content 600 is displayed at a position distant from the presenter 700 and the operation icons 602 and 603 are displayed at positions close to the presenter 700 . Further, for example, when an instruction to transition to the bidirectional mode is given at the same time that the power of the projector 10 is turned on, the bidirectional mode may be started without going through the normal mode. That is, the content 600 may not be displayed on the entire display area of the screen 500, and the projection image may be displayed with the operation icon (operation icon area 601) from the start of projection.

In S<b>105 , the control unit 100 monitors (determines) whether or not the user has instructed to end the interactive mode. Control unit 100 receives an end instruction from the user via infrared reception unit 124 or operation unit 114 . If the user gives an end instruction, all the processes in this flow chart are finished, and if the user does not give an end instruction, the process of S105 is repeated. Further, while the processing of S105 is being repeated, the control unit 100 analyzes the projection image periodically captured by the image capturing unit 126, and the operation icon 602 or the operation icon 603 by the presenter 700 is “hand-held”. monitor whether there is When the control unit 100 detects "hand-held", the control unit 100 transmits control data corresponding to the operation icon "hand-held" to the PC 200 via the USB interface 112, thereby displaying the content 600. to control. For example, in the case of detecting "hand holding", the control unit 100 controls the PC 200 to display on the screen 500 by controlling the transition between a plurality of consecutive images such as the page number and shooting date and time. Controls (transitions) images.

(Detailed processing of S102)
The adjustment process for grasping the position of the operation icon in S102 described above will be described in detail below with reference to FIG. 3(B).

In S<b>201 , the control unit 100 instructs the image processing unit 160 to generate a test pattern and project it on the screen 500 . A test pattern 605 in FIG. 4 shows an example of a test pattern projected by the projector 10 . The test pattern 605 has a position icon 606 and a position icon 607 for grasping the positions of the operation icons. Here, a position icon 606 is an icon for grasping the position of the operation icon 602, and a position icon 607 is an icon for grasping the position of the operation icon 603. FIG.

In S<b>202 , the control unit 100 instructs the imaging unit 126 to image (shoot) the screen 500 . At this time, if the captured image is not suitable for grasping the positions of the position icons 606 and 607, the imaging parameters such as exposure may be changed and the image may be captured again so that a suitable image is obtained.

In S203, the control unit 100 reads out the zoom state of the projection optical system 107 of the optical system control unit 105 and the captured image captured by the imaging unit 126 to the RAM 151, and from these reads the position of the operation icon (position icon) in the captured image. Grasp. In this way, the control unit 100 captures the positions of the operation icons on the screen 500 by capturing the test pattern 605 having the position icons via the imaging unit 126 .

[Transition processing to interactive mode according to the present embodiment]
In the above description, transition processing between two-way modes has been described. However, since the projector 10 and the USB cable 300 are not connected, communication of control data is disabled and operation by "holding over the hand" cannot be performed. occurs. Even in such a case, since the operation icon 602 and the operation icon 603 are displayed on the screen 500, the user misunderstands that the operation can be performed by "holding over the hand" and is confused. There is a possibility of doing so.

Therefore, in view of the above-described problem, transition processing to the bidirectional mode in this embodiment will be described. FIG. 5A is a diagram showing a flowchart of processing of the projector 10 according to this embodiment. This flowchart describes the projector 10 that can solve the above-described problem by considering the possibility that the USB cable 300 is not properly connected. Note that S101 to S105 in this flowchart perform the same processing as the processing with the same reference numerals in the flowchart in FIG. 3A, so detailed description thereof will be omitted.

In S101 and S102, the control unit 100 receives an instruction to transition to the bidirectional mode and performs adjustment processing. When the process of S102 is completed, the process transitions to S106.

In S106, the control unit 100 determines whether or not the communication state (USB communication state) via the USB cable 300 is normal. For example, assuming that a cable has been forgotten to be inserted or removed, or that an internal wiring has been disconnected, information based on the result of the USB session establishment process, whether or not there is a response by communication, and the voltage state of Vbus, which is a power supply line, are used for this determination. Note that the control unit 100 may use the presence or absence of an interrupt in response to connection detection from the USB interface 112 to determine whether or not the USB communication state is normal. If the control unit 100 determines that the USB communication state is not normal, the process transitions to S107, and if it determines that the USB communication is normal, the process transitions to S103. That is, in S106, the control unit 100 determines whether or not the transition to the bidirectional mode is possible. Therefore, in the present embodiment, the control unit 100 is display control means for controlling the display of the content 600 by "holding the hand over" and determination means for determining whether transition to the interactive mode is possible. It can be said.

Note that the determination method in S106 is not limited to the method based on the USB communication state, and any method may be used as long as it can determine whether the control unit 100 can control the content 600 or not. That is, even when the control unit 100 cannot control the content 600 because the device status of the PC 200 is not normal, the process may proceed to S107. Furthermore, even when the control unit 100 cannot control the content 600 because the PC 200 does not support the interactive mode, the process may proceed to S107. Therefore, when the control unit 100 cannot control the content 600, the process may transition to S107 even if the USB communication state is normal.

In S107, the projector 10 performs processing when the USB communication state is not normal (abnormal).

(Detailed processing of S107)
The processing of S107 will be specifically described using the flowchart of FIG. 5B.
In S301, the control unit 100 reads out a pre-stored error message image from the internal memory 150, controls the image processing unit 160, and superimposes it on the image being projected. FIG. 6 is an external view showing how the error message image 604 is projected on the screen 500. As shown in FIG. Specifically, the error message image 604 notifies the user that the transition (start) to the interactive mode is impossible and that the connection of the USB cable 300 should be checked. Note that in this embodiment, if the USB communication state is abnormal, the user can recognize that the operation icon area 601 is not displayed and the transition to the interactive mode is not possible, so the error message image 604 is always displayed. No need. In this embodiment, an error message image is displayed to notify the user that the transition to the interactive mode is not possible. User may be notified.

In S302, while the error message image 604 is being projected, the control unit 100 waits until the USB communication state returns to normal. That is, when the transition to the interactive mode is not possible, the control unit 100 controls the image processing unit 160 to change the display layout of the projected image on the screen 500 from the display layout before transitioning to the interactive mode. It does not change. Specifically, only the content 600 is displayed in the display area of the screen 500, and the operation icon area 601 (operation area) is not displayed. When the control unit 100 determines (determines) that the USB communication state has returned to the normal state, the process proceeds to S303. It should be noted that in S302, if the standby state has elapsed for a predetermined period of time or more, the control unit 100 controls the image processing unit 160 to display to the user that the transition to the interactive mode was not possible. All the processing in the flowchart may be ended.

In S<b>303 , the control unit 100 instructs the image processing unit 160 to erase the error message image 604 . After the error message image 604 is deleted, the process transitions to S103.

Note that in the present embodiment, the user controls the projected image (content 600) by "holding the hand over" a predetermined position of the projected image. However, the display of the content 600 may be controlled by operating the infrared reception unit 124 or the operation unit 114, for example. In this case, the operation icons displayed on the screen 500 have the role of visualizing the operations that can be performed in controlling the content 600 .

In this embodiment, communication between the projector 10 and the PC 200 is performed via two transmission paths, the HDMI cable 400 and the USB cable 400, but may be performed via only one transmission path. For example, assume that the projector 10 and the PC 200 are connected by only one transmission line of a LAN cable, and mutual communication between the projector 10 and the PC 200 is possible under normal conditions. Even in such a case, although image data can be transmitted from the PC 200 to the projector 10, an abnormal state may occur in which the projector 10 cannot communicate to control the PC 200, so this embodiment is applicable. For the same reason, projector 10 and PC 200 may be connected via the Internet.

In this embodiment, an example using a projector has been described, but a smartphone, a tablet terminal, a liquid crystal display, or the like can also be applied as long as it is a display control device having a bidirectional mode. Specifically, when using a smart phone, for example, a touch operation on the touch panel may be accepted in the interactive mode, and the touch operation may not be accepted in the normal mode. Furthermore, the present embodiment can be applied as long as the content 600 is controlled by the smartphone instructing the external server to request image transmission.

[effect]
As described above, only when the content can be controlled according to the user operation, the interactive mode is entered and the operation icon area is displayed. If there is, it is possible to avoid erroneous recognition by the user. In addition, if the user is notified that the transition to the interactive mode is not possible when the transition is not possible, the user can more easily grasp the state of the projector.

[Modification 1]
In the first embodiment, the projector avoids misrecognition by the user by controlling the display of the content 600 based on the USB communication state between the projector 10 and the PC 200 when the user gives an instruction to transition to the interactive mode. 10 explained. In this modified example, after the transition to the bidirectional mode is completed (after the start), even if the "hand holding" operation cannot be performed due to the occurrence of a communication abnormality in the USB cable 300, the abnormal state is appropriately notified to the user. make it possible to recognize Hereinafter, transition processing to the bidirectional mode of this modification will be described using the flowchart shown in FIG. 7(A). Note that S101 to S107 in this flowchart perform the same processing as the processing with the same reference numerals in the flowchart shown in FIG. 5A, so detailed description thereof will be omitted.

The processes of S101 to S104 and S107 are performed in the same manner as the process of the flowchart shown in FIG. 5A. When the process of S104 is completed, the process transitions to S108.

In S108, the control unit 100 determines whether the USB communication state is normal. Here, consideration is given to the case where the USB communication is disabled due to, for example, removal of the USB cable 300 during the bidirectional mode (after the bidirectional mode is started). Since the specific processing in S108 is the same as that in S106, detailed description will be omitted. If it is determined that the USB communication state is abnormal, the process proceeds to S109, and if it is determined that the USB communication state is normal, the process proceeds to S110. In S108, as in S106, if the control unit 100 cannot control the content 600 (PC 200), the process may proceed to S109 even if the USB communication state is normal.

In S109, the control unit 100 performs processing when USB communication is in an abnormal state. Note that in this modification, after the operation icon 602 or the operation icon 603 is displayed on the screen 500 after transitioning to the interactive mode, the operation icon area 601 is not displayed even if the USB communication changes to an abnormal state. to maintain the display without This is because when the layout of projected images is executed depending on whether or not USB communication is possible, if the USB communication is unstable, the layout will change frequently every time the communication state changes, which may cause discomfort to the user. end up Therefore, by maintaining the display of the operation icon area 601, it is possible to prevent the user from feeling uncomfortable. Details of the processing of S109 will be described later.

In S110, the control unit 100 determines whether or not an instruction to terminate the interactive mode has been received, as in the processing of S105. If the instruction to end the bidirectional mode has not been received, the control unit 100 transitions to S108 and determines the USB communication state again. If an instruction to terminate the bidirectional mode has been received, all the processing in this flowchart is terminated.

(Detailed processing of S109)
Regarding the processing in S109, the flowchart of FIG. 7(B) and FIGS.
A detailed description will be given using (B). Note that S302 in the flowchart shown in FIG. 7B performs the same processing as the processing with the same reference numerals in the flowchart shown in FIG. 5B, so detailed description thereof will be omitted.

In S304, the control unit 100 reads from the internal memory 150 an image indicating that the operation by "holding over the hand" cannot be performed due to an abnormality in the USB communication state, and instructs the image processing unit 160 to display the image. FIG. 8A shows that the operation icon 602 and the operation icon 603 are grayed out and changed in form. On the other hand, FIG. 8B shows a form in which the shapes of the operation icon 602 and the operation icon 603 are changed from arrows to cross marks. In this way, the control unit 100 changes the form of the operation icon on the screen 500 to display that the user operation cannot be executed in both FIGS. 8A and 8B. In other words, it displays that the user's operation is not accepted. Note that the control unit 100 is not limited to changing the appearance of the operation icon, such as changing the arrow from the arrow to the cross, and may erase the display of the operation icon 602 and the operation icon 603 themselves. Further, the control unit 100 may display an error message image including, for example, confirmation of cable connection in the operation icon area 601 . In this way, even if the content 600 cannot be controlled according to the user's operation after transitioning to the interactive mode, the control unit 100 does not end the interactive mode and indicates that the user's operation cannot be executed. display. As a matter of course, while such display is being performed, the control unit 100 does not accept an operation by “holding the hand” through the imaging unit 126 .

In S305, the control unit 100 instructs the image processing unit 160 to cancel the processing performed in S304. That is, the control unit 100 controls the image processing unit 160 so that, for example, an operation icon changed to a cross is returned to an arrow, and a grayed-out operation icon is returned to a normal display state. When the process of S305 is completed, the process transitions to S110.

(effect)
As described above, even if the content (external device) becomes uncontrollable after transitioning to the interactive mode, by controlling the projected image, it is possible to tell from the projected image that the interactive mode is operating normally. It is possible to prevent erroneous recognition by the user.

Note that each functional unit in each of the above-described embodiments and modifications may or may not be separate hardware. Functions of two or more functional units may be implemented by common hardware. Each of the multiple functions of one functional unit may be implemented by separate hardware. Two or more functions of one functional unit may be realized by common hardware. Also, each functional unit may or may not be realized by hardware such as ASIC, FPGA, and DSP. For example, the device may have a processor and a memory (storage medium) in which a control program is stored. The functions of at least some of the functional units of the device may be realized by the processor reading out and executing the control program from the memory.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

10: Projector, 100: Control Unit, 110: Image Acquisition Unit, 114: Operation Unit

Claims (18)

An image projection device including an operation mode for receiving a user operation corresponding to a user's operation on a display unit among predetermined user operations,
display control means for controlling display of an image projected on the display unit;
input means for inputting an instruction to start the operation mode;
determination means for determining whether or not the operation mode can be started;
has
The display control means is
controlling the display unit to display the image and an operation area for detecting the user's motion when the operation mode can be started;
controlling the display unit to display the image and not to display the operation area when the operation mode cannot be started;
When displaying the operation area, the display control means displays an icon indicating the predetermined user operation in the operation area,
The determination means further determines whether or not the display control means is no longer in a state in which the switching of the images can be controlled after the operation mode is started,
The display control means maintains the display of the operation area and changes the icon to a different form when it is no longer possible to control the switching of the images after the operation mode is started.
An image projection device characterized by: The display control means is
In the operation mode, the image is displayed in a part of the display area of the display unit, and the operation area is displayed in a part other than the part of the display area;
if not in the operation mode, displaying the image over the entire display area;
2. The image projection device according to claim 1, wherein: The display control means is
In the operation mode, the image is displayed in the entire display area of the display unit, and the operation area is superimposed on the image and displayed;
if not in the operation mode, displaying the image over the entire display area;
2. The image projection device according to claim 1, wherein: When the operation mode cannot be started, the display control means displays on the display unit that the operation mode cannot be started.
4. The image projection device according to any one of claims 1 to 3, characterized in that: When the display control means controls the display unit so as not to display the operation area, the determination means determines whether or not the operation mode can be started thereafter.
5. The image projection device according to any one of claims 1 to 4, characterized in that: The display control means further controls switching of the image displayed on the display unit by a user operation according to the user's action.
6. The image projection device according to any one of claims 1 to 5, characterized in that: The determination means is
determining that the operation mode can be started if the image displayed on the display unit can be switched;
determining that the operation mode cannot be started if the image displayed on the display unit cannot be switched;
7. The image projection device according to claim 6, characterized in that: further comprising acquisition means for acquiring image data for displaying the image on the display unit;
The display control means controls switching of the image displayed on the display unit by controlling an external device that outputs the image data to the acquisition means.
8. The image projection device according to claim 6 or 7, characterized in that: The display control means controls switching of the images displayed on the display unit by controlling transitions between a plurality of consecutive images.
9. The image projection device according to claim 8, characterized by: The acquisition means acquires the image data from the external device via a first transmission line,
The display control means controls switching of images displayed on the display unit by controlling the external device via a second transmission line different from the first transmission line.
10. The image projection device according to claim 8 or 9, characterized in that: After starting the operation mode, if the display control means is no longer able to control switching of the images, the operation mode does not end.
11. The image projection device according to any one of claims 1 to 10, characterized in that: The display control means displays on the display unit that the predetermined user operation cannot be executed when the state in which the switching of the images cannot be controlled is lost after the operation mode is started.
12. The image projection device according to any one of claims 1 to 11, characterized in that: The display control means displays an error message on the display unit when it is no longer possible to control switching of the images after the operation mode is started .
13. The image projection device according to any one of claims 1 to 12, characterized in that : further comprising receiving means for receiving a user operation according to the user's action,
In the operation mode, the accepting means responds to the icon held up with the indicator by detecting the user's action of holding up the indicator over the icon displayed on the display unit. accepts user operations performed by
14. The image projection device according to any one of claims 1 to 13, characterized in that: The accepting means accepts a user operation corresponding to the icon over which the indicator is held by capturing an image of the display unit.
15. The image projection device according to claim 14 , characterized in that: further comprising projection means for projecting the image on the display unit;
The display unit is a projection target,
16. The image projection device according to any one of claims 1 to 15 , characterized in that: A control method for an image projection device including an operation mode for receiving a user operation corresponding to a user's operation on a display unit among predetermined user operations,
a display control step of controlling display of an image projected on the display unit;
a determination step of determining whether or not the operation mode can be started when an instruction to start the operation mode is input;
has
In the display control step,
When it is determined in the determination step that the operation mode can be started, the display unit is controlled to display the image and an operation area for detecting the user's motion. death,
if it is determined in the determination step that the operation mode cannot be started, displaying the image and controlling the display unit so as not to display the operation area;
In the display control step, when displaying the operation area, an icon indicating the predetermined user operation is displayed in the operation area;
In the determination step, further, after the operation mode is started, it is determined whether or not it is no longer possible to control switching of the images in the display control step;
In the display control step, when it is no longer possible to control the switching of the images after the operation mode is started, the display of the operation area is maintained and the icon is changed to a different form.
A control method for an image projection device, characterized by: A program for causing a computer to execute each step of the control method according to claim 17 .

Images