JP6364844B2 - Electronic device, display control method and program - Google Patents

Description

  The present invention relates to an electronic apparatus, a display control method, and a program suitable for a projector apparatus that performs presentation by connecting to a personal computer, for example.

  There has been proposed a technology that enables a projection device to perform horizontal wall surface (vertical surface) projection and vertical desktop (horizontal) projection. (For example, Patent Document 1)

JP 2011-158522 A

  It is widely and widely used to perform presentations using images from a projection device by connecting the projection device to a personal computer and executing a presentation program on the personal computer.

  In this type of usage environment, a person who gives a presentation (hereinafter referred to as a “presenter”) switches the projected page image while operating the keyboard of the personal computer and the mouse as a pointing device, or sets the pointer position in the image. Perform various operations such as moving.

  Consider a case where the presenter wants to move the position of the pointer in the image while looking at the projected image without using the display of the personal computer. In this case, the moving direction of the pointer in the image simply depends on the operation direction of the mouse, and has nothing to do with which direction the image projected by the presenter is viewed.

  Therefore, when the presenter is looking at the projected image from a direction other than the front, the direction in which the point is to be moved in the actually visible image is different from the direction in which the mouse is operated to move in that direction. Therefore, it is necessary for the presenter to operate the mouse after recognizing the difference in these directions.

  This is remarkable when, for example, a presentation is performed by connecting a personal computer to a projection apparatus that performs desktop (horizontal plane) projection in a vertical position described in the above-mentioned patent document. The projected image and the operation surface of the mouse Exist in the same plane. For this reason, for example, when the presenter is positioned in the upper direction of the projected image, the operation direction of the mouse and the direction of the pointer moved by the operation are completely opposite. Operation is required.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an electronic device and a display capable of operating a pointing device in an image without a sense of incongruity even for a user who is not facing the display image. It is to provide a control method and a program.

One aspect of the present invention is an acquisition unit that acquires reference angle information of a pointing device for an image based on image information, a detection unit that detects a moving operation direction of the pointing device, and reference angle information acquired by the acquisition unit And a correction unit that corrects the moving operation direction of the pointing device detected by the detection unit, and an output that outputs the pointer image superimposed on the image according to the moving operation direction of the pointing device corrected by the correction unit. And an imaging unit that captures the display state of the image output by the output unit, and the acquisition unit is configured to correlate the pointing device based on a correlation between the image captured by the imaging unit and the image based on the image information. The reference angle information is acquired .

  According to the present invention, even a user who is not facing the display image can operate the pointing device in the image without a sense of incongruity.

The figure which shows the installation environment of the presentation system which concerns on one Embodiment of this invention. 6 is an exemplary flowchart illustrating a part of an operation during presentation including manual setting of a moving operation direction of a mouse according to the embodiment. The figure which shows the example of the image for a presentation, etc. which are displayed on the display of the personal computer which concerns on the embodiment. 6 is an exemplary flowchart illustrating a part of an operation during presentation including automatic setting of a moving operation direction of a mouse according to the embodiment.

  Hereinafter, an embodiment in which the present invention is applied to a presentation system using a personal computer and a projector will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating an example of an installation environment of the system. Here, a notebook type personal computer 10 and a projector device 20 are placed on a table TB, and the personal computer 10 and the projector device 20 are connected by an RGB cable VC.

  The projector device 20 projects an image on the plate surface of the table TB by an image signal sent from the personal computer 10 to the projector device 20 via the RGB cable VC.

  The personal computer 10 is externally attached with a mouse 10A as a pointing device and a web camera 10B.

  The web camera 10B is detachably attached to the upper end of the display of the notebook type personal computer 10, and is mounted toward the inner side where the user (presenter) who operates the personal computer 10 is located. Can be photographed, while being mounted toward the outside as shown in the figure, the image projected by the projector device 20 onto the table TB can be photographed.

  Note that the configuration of each hardware circuit of the personal computer 10 and the projector device 20 is the same as that of a general hardware circuit of the same device, and illustration and description thereof are omitted.

Next, the operation of the above embodiment will be described.
The following operations are executed in the process of reproducing a document file created for the program by starting a presentation program installed in the personal computer 10 in advance. In this program, the presenter who is the user of the personal computer 10 can arbitrarily select whether to manually correct the moving operation direction with the mouse 10A or to automatically set using the web camera 10B by mode switching. Shall.

  In both manual setting and automatic setting operations, a CPU (not shown) in the personal computer 10 expands a presentation program read from a hard disk device or the like as a storage medium to a DRAM (dynamic RAM) as a main memory. Execute after memorizing.

  FIG. 2 is a flowchart showing a part of the motion during presentation including manual setting of the moving operation direction of the mouse 10A. At the beginning of the process, the CPU determines whether or not an instruction operation indicating manual setting of the moving operation direction of the mouse 10A has been selected (step S101).

  If it is determined that an instruction operation indicating manual setting has not been selected (No in step S101), the CPU next determines whether or not the mouse 10A has been moved (step S102).

  When it is determined that the mouse 10A has not been moved (No in step S102), the CPU determines whether or not the timing for switching the image signal output to the projector device 20 to be projected has arrived on the presentation program. A determination is made based on whether or not an operation has been performed (step S103).

  When it is determined that it is not yet time to switch the image signal to be projected (No in step S103), the process returns to step S101 again.

  In this way, while repeatedly performing the processes of steps S101 to S103, the process waits until the instruction operation indicating manual setting is selected, the mouse 10A is moved, or the timing of switching the image signal to be projected is reached.

  When it is determined in step S101 that the instruction operation indicating the manual setting of the moving operation direction of the mouse 10A is selected (Yes in step S101), the CPU uses the mouse 10A as the reference direction, for example, the mouse operated by the user of the personal computer 10 10A accepts an operation indicating as a rotation angle what angle relationship the 10A has with respect to the image projected by the projector device 20 (step S104), and then a predetermined angle determination operation, for example, a left click key operation is performed. It waits to be made (No in step S105).

  FIG. 3 shows an example of the presentation image displayed on the display DS of the personal computer 10 at this time and the mouse angle setting image MS at the end thereof, for example, the upper left corner.

  For example, when the projection image of the projector device 20 shown in FIG. 1 and the mouse 10A of the personal computer 10 are in a positional relationship, the projection image is rotated from the normal direction to the right rotation direction ( It can be seen in a state rotated approximately 90 ° clockwise.

  In this case, the front of the mouse 10A points to the left of the projected image. Therefore, the user of the personal computer 10 operates the mouse 10A to move the pointer PT on the display DS to the position of the mouse angle setting image MS, and the mouse angle setting image MS is left as shown in the figure. By performing the drag operation so as to face the direction, the direction of the mouse 10A under the projection situation at that time is correctly specified. In accordance with the operation of the mouse angle setting image MS displayed on the display DS, as shown in FIG. 3, a determination button ET appears and displayed at the bottom of the mouse angle setting image MS, and this determination button ET The determination in step S105 may be performed based on whether or not there is a click operation.

  It should be noted that a series of processing relating to manual setting of the moving operation direction of the mouse 10A is temporarily ended in accordance with the operation of the determination button ET, and the image display of the mouse angle setting image MS and the determination button ET on the display DS is stopped. To do.

  When it is determined that an angle determination operation has been performed (Yes in step S105), the information about the determined rotation angle is stored for setting (step S106), and then the process proceeds from step S102.

  As shown in FIG. 3 above, when the determination button ET is operated with the mouse angle setting image MS facing leftward on the display DS, for example, the sign of the clockwise direction (clockwise) is set to plus. Then, the state of the angle determined from the reference state in which the mouse is directed directly upward, for example, “minus 90 °”, is information on the rotation angle for correction.

  If it is determined in step S102 that the moving operation of the mouse 10A has been performed (Yes in step S102), the operation information of the mouse 10A by the operation is corrected by the rotation angle information set at that time (step S107).

  Then, after superimposing and processing the image of the pointer PT so as to move the position of the pointer PT in the image information output to the projector device 20 based on the correction result (step S108), the processing from the step S103 is performed. Transition to processing.

  If it is determined in step S103 that a page break operation has been performed on the presentation program and the timing for switching the image signal to be projected has come (Yes in step S103), the CPU reproduces the presentation program in response to the operation. A new image signal is generated from the document file being created, transmitted to the projector device 20 and projected (step S109), and the process proceeds to step S101 again.

  As described above, since the user of the personal computer 10 manually sets the operation direction of the mouse 10A to match the movement direction in the image actually projected by the projector device 20, the personal computer 10 Even when the user is not directly facing the projected image, the pointer position in the image can be reliably operated by operating the mouse 10A while looking at the projected image.

  In the present embodiment, as described above, it is possible to arbitrarily select whether the correction of the moving operation direction with the mouse 10A is performed manually or automatically using the web camera 10B by switching the mode.

FIG. 4 is a flowchart showing the processing contents of an operation for extracting a part of the presentation including automatic setting of the moving operation direction of the mouse 10A, which is executed instead of the processing of FIG.
At the beginning of the processing, the CPU determines whether or not the shooting timing (for example, once / 5 [seconds]) of the web camera 10B preset in the presentation program has come (step S201).

  If it is determined that the shooting timing has not come (No in step S201), the CPU next determines whether or not the mouse 10A has been moved (step S202).

  If it is determined that the mouse 10A has not been moved (No in step S202), the CPU determines whether or not the timing for switching the image signal to be output to the projector device 20 to be projected has been reached on the presentation program. A determination is made based on whether or not an operation has been performed (step S203).

  When it is determined that it is not yet time to switch the image signal to be projected (No in step S203), the process returns to step S201 again.

  In this way, while repeatedly performing the processes of steps S201 to S203, the process waits for the timing of shooting with the web camera 10B, the operation of moving the mouse 10A, or the timing of switching the image signal to be projected.

  In step S201, when it is determined that the timing for shooting with the web camera 10B has come (Yes in step S201), the CPU executes shooting with the web camera 10B (step S204).

  The CPU creates various images obtained by reducing, rotating, and deforming the image of the image signal that is output and projected to the projector device 20 at that time, and performs pattern matching processing on the image data obtained by shooting. (Step S205), it is determined whether or not the projected image by the projector device 20 is reflected in the captured image (Step S206).

  Here, if the corresponding pattern cannot be detected and it is determined that the projected image is not reflected in the captured image (No in step S206), the process proceeds to step S202 as it is.

At this time, for example, in the display DS of the personal computer 10, for example,
“I was unable to shoot the projected image.”
It is also possible to prompt the user to adjust the facing direction of the web camera 10B by giving a guide message using a character string such as
If the corresponding pattern is detected in step S206 and it is determined that the projected image is reflected in the photographed image (Yes in step S206), the projected image pattern is then extracted from the projected image pattern. Then, the angle at which the personal computer 10 is located is calculated (step S207).

  For example, in the case of the positional relationship between the personal computer 10 and the projector device 20 as shown in FIG. 1, the photographed image is photographed with the upper side of the projected image facing rightward. That is, when the projection image of the projector device 20 is used as a reference, the personal computer 10 is directed to the left of the projection image. For example, when the sign of the right rotation direction (clockwise direction) is plus, the personal computer 10 The direction directly above the mouse is “−90 °” from the reference in the projection image.

  After storing the calculated rotation angle information (step S208), the process proceeds to step S202.

  If it is determined in step S202 that the moving operation of the mouse 10A has been performed (Yes in step S202), the operation information of the mouse 10A by the operation is corrected by the rotation angle information set at that time (step S209).

  Then, after superimposing and processing the image of the pointer PT so as to move the position of the pointer PT in the image information output to the projector device 20 based on the correction result (step S210), the processing from step S203 is performed. Transition to processing.

  If it is determined in step S203 that a page break operation has been performed on the presentation program and the timing for switching the image signal to be projected has come (Yes in step S203), the CPU reproduces the presentation program in response to the operation. A new image signal is created from the document file being created, transmitted to the projector device 20 and projected (step S211), and then the process proceeds to step S201 again.

  As described above, the setting for matching the operation direction of the mouse 10A with the moving direction in the image actually projected by the projector device 20 according to the facing direction of the personal computer 10 with respect to the projection image by the projector device 20 is performed. Since it is automatically performed, even when the user of the personal computer 10 is not directly facing the projection image, the pointer position in the image can be easily and easily operated by operating the mouse 10A while looking at the projection image.

  As described above in detail, according to the present embodiment, even when the user of the personal computer 10 does not face the projection image of the projector device 20, the mouse 10 </ b> A can be operated comfortably while viewing the projection image.

  In the above embodiment, the case where the present invention is applied to a presentation system using a personal computer and a projector has been described. However, the present invention is not limited to this, and the operation system of a pointer and the position of an image for displaying the system are not limited. Any other pointing device, system, or device may be used. For example, an arcade game machine that surrounds a horizontally placed panel display and plays a game with a large number of people and its various other devices such as a controller thereof. It is also applicable to.

  In addition, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention in the implementation stage. Further, the functions executed in the above-described embodiments may be combined as appropriate as possible. The above-described embodiment includes various stages, and various inventions can be extracted by an appropriate combination of a plurality of disclosed constituent elements. For example, even if several constituent requirements are deleted from all the constituent requirements shown in the embodiment, if an effect is obtained, a configuration from which the constituent requirements are deleted can be extracted as an invention.

Hereinafter, the invention described in the scope of claims of the present application will be appended.
The invention according to claim 1 is an acquisition unit that acquires reference angle information of a pointing device for an image based on image information, a detection unit that detects a moving operation direction of the pointing device, and a reference angle acquired by the acquisition unit. Based on the information, a correction unit that corrects the movement operation direction of the pointing device detected by the detection unit, and a pointer image superimposed on the image according to the movement operation direction of the pointing device corrected by the correction unit is output. And an output unit.

  The invention of claim 2 further comprises an input unit for inputting image information according to claim 1, wherein the acquisition unit receives a moving operation direction of the pointing device with respect to an image output by the output unit, and performs pointing. The reference angle information of the device is acquired.

  The invention described in claim 3 further includes an input unit for inputting image information and a photographing unit for photographing a display state of the image output by the output unit in the first aspect, wherein the acquisition unit includes Reference angle information of the pointing device is acquired by correlation between an image captured by the imaging unit and an image based on the image information input by the input unit.

  According to a fourth aspect of the present invention, there is provided a display control method in an apparatus including a pointing device, an acquisition step of acquiring reference angle information of the pointing device with respect to an image based on image information, and a moving operation of the pointing device A detecting step for detecting a direction, a correcting step for correcting the moving operation direction of the pointing device detected in the detecting step based on the reference angle information acquired in the acquiring step, and a moving operation of the pointing device corrected in the correcting step And an output step of superimposing and outputting a pointer image on the image according to the direction.

  The invention according to claim 5 is a program executed by a computer built in an apparatus including a pointing device, wherein the acquisition unit acquires the reference angle information of the pointing device for an image based on the image information. Based on the reference angle information acquired by the acquisition unit, the detection unit that detects the movement operation direction of the pointing device, the correction unit that corrects the movement operation direction of the pointing device detected in the detection step, and the correction unit According to a moving operation direction of the pointing device, the pointer is superimposed on the image and output as an output unit.

10 ... Personal computer,
10A ... mouse,
10B ... Webcam,
20 ... Projector device,
DS ... display,
ET ... Decision button,
MS ... Mouse angle setting image,
PT ... pointer,
TB ... table,
VC: RGB cable.

Claims (3)

An acquisition unit that acquires reference angle information of a pointing device for an image based on the image information;
A detection unit for detecting a moving operation direction of the pointing device;
Based on the reference angle information acquired by the acquisition unit, a correction unit that corrects the moving operation direction of the pointing device detected by the detection unit;
An output unit that superimposes and outputs a pointer image on the image according to the movement operation direction of the pointing device corrected by the correction unit;
A photographing unit for photographing the display state of the image output by the output unit;
Equipped with a,
The electronic apparatus according to claim 1 , wherein the acquisition unit acquires reference angle information of the pointing device based on a correlation between an image captured by the imaging unit and an image based on the image information . A display control method in an apparatus including a pointing device , an output unit, and a photographing unit ,
An acquisition step of acquiring reference angle information of the pointing device for an image based on image information;
A detection step of detecting a moving operation direction of the pointing device;
Based on the reference angle information acquired in the acquisition step, a correction step for correcting the moving operation direction of the pointing device detected in the detection step;
According to the movement operation direction of the pointing device corrected in the correction step, an output step of superimposing a pointer image on the image and outputting the pointer image on the output unit ,
A photographing step of photographing the display state of the image output in the output step with the photographing unit;
Have,
The display control method according to claim 1, wherein the obtaining step obtains reference angle information of the pointing device based on a correlation between the image photographed in the photographing step and an image based on the image information . A program executed by a computer built in an apparatus including a pointing device , an output unit, and a photographing unit ,
An acquisition unit that acquires reference angle information of the pointing device for an image based on image information;
A detection unit for detecting a moving operation direction of the pointing device;
Based on the acquired reference angle information by the acquisition unit, the correction unit for correcting the movement operation direction of the pointing device detected by the detection unit,
A superimposing unit that superimposes the pointer image on the image according to the moving operation direction of the pointing device corrected by the correcting unit; and
A display unit for displaying an image superimposed by the superimposing unit on the output unit;
To function as,
The acquisition unit acquires reference angle information of the pointing device based on a correlation between an image captured by the imaging unit and an image based on the image information of the image displayed by the output unit. Program.

Images