JP6507905B2 - CONTENT DISPLAY CONTROL METHOD, CONTENT DISPLAY CONTROL DEVICE, AND CONTENT DISPLAY CONTROL PROGRAM - Google Patents

Description

  The present invention relates to a content display control method, a content display control device, and a content display control program.

  In recent years, a human computer interaction system constituted by an electronic pen, various sensors, a projector and the like, so-called interactive whiteboards has attracted great interest.

  In an interactive whiteboard using an electronic pen, a movement trajectory of the electronic pen in the board plane is detected by a sensor, and a drawn figure or character is displayed on the board plane. Further, a command to the computer is recognized from the flow line in the board plane of the electronic pen, and the operation of the computer is controlled (see, for example, Patent Documents 1 and 2).

JP, 2011-204059, A Japanese Patent Application Publication No. 2007-527573

  As described above, the operation with the electronic pen or the like in the conventional interactive whiteboard is limited to a two-dimensional one, and there is a problem that various operations can not be performed on the content displayed on the screen of the interactive whiteboard. The

  Then, the form of an indication aims at the improvement of the operativity to the content displayed on the screen as one side.

According to the disclosed embodiment, the content display control device displays an image of the content on the screen by the projector, detects the reflected light from the screen of the light beam emitted from the hand-held device to the screen, and detects the reflected light Based on the state of the reflected light which is set corresponding to the state of the reflected light which changes according to the distance between the screen and the hand-held device, the mode of the reflected light is selected from the mode selection until the light beam disappears. A process of detecting a user's operation and controlling display of an image of the content on the screen based on the detected operation is performed, and a process of selecting the mode includes light on the screen of the reflected light Select the mode associated with the combination of point size and center luminance value .

  The form of disclosure can improve the operability with respect to the content displayed on the screen.

It is a figure showing an example of composition of a system concerning one embodiment. It is a figure showing an example of the appearance of a system. It is a figure showing an example of composition of a pen device. It is a figure which shows the example of the operation assumed. It is a figure which shows the characteristic of the light spot on a screen. It is a figure which shows the example of the center brightness | luminance of the light spot with respect to the distance of a pen device and a screen. It is a figure which shows the example of the light spot size with respect to the distance of a pen device and a screen. FIG. 6 illustrates an example of light spot size and center luminance with respect to the distance between the pen device and the screen. It is a figure which shows the example of the relationship between the mode matched with distance, light spot size, and center luminance. It is a figure which shows the example of the procedure of a user's operation. It is a figure which shows the example of a mode and a matching table. It is a flow chart which shows an example of mode selection processing. It is a figure which shows the example of the display figure at the time of temporary decision / this decision, and a timer figure. It is a figure which shows the example of the parameter for every mode. It is a flowchart which shows the example of the parameter adjustment process in pan tilt mode. It is a flowchart which shows the example of the parameter adjustment process in a habit mode. It is a figure which shows the example of a handwriting mode. It is a figure (flat display-> 俯瞰 display) which shows the example of 俯瞰 mode. It is a figure which shows the example of pan tilt mode. It is a figure (俯瞰 display → plane display) showing an example of eyebrow mode. It is a figure showing an example of composition of a pen device. It is a figure which shows the specific structural example of a pen device. It is a figure which shows the circuit structural example of a pen device. It is a figure which shows the example of an internal structure of a controller. It is a flowchart which shows the process example of a controller. It is explanatory drawing of light quantity adjustment of a pen device.

  Hereinafter, preferred embodiments of the present invention will be described.

<Configuration>
FIG. 1 is a diagram showing an example of the configuration of a system according to an embodiment. In FIG. 1, the system includes a pen device 1 which is operated by a user by hand and which emits a light beam such as infrared light, and a screen (reflecting device) where the light is emitted from the pen device (handheld device) 1 2 and a light receiving sensor 3 for detecting reflected light (scattered reflected light) from the screen 2. The light receiving sensor 3 is, for example, an infrared camera, and in that case, the detection signal is a captured image. The control device 4 recognizes the user's operation based on the detection signal of the light receiving sensor 3 and generates a video signal of the content from the terminal device 6 inside or outside the device itself, and the video signal output from the control device 4 And a projector 5 for projecting the light onto the screen 2.

  The pen device 1 is provided with a light emitting element 11, switches 12 and 13, and a battery 14. When either the switch 12 or the switch 13 is turned on, power is supplied from the battery 14 to the light emitting element 11, and the light emitting element 11 is lit. Details will be described later.

  The control device 4 selects a mode selection unit 41 that selects a mode of operation based on a detection signal of the light receiving sensor 3, a parameter adjustment unit 42 that adjusts mode-specific parameters under the selected mode, and the adjusted parameters. And an image display control unit 43 that controls image display based on the information.

  FIG. 2 is a diagram showing an example of the appearance of the system. In FIG. 2, a projector 5 is provided, for example, below the screen 2, a projection lens 51 is directed to the screen 2, and a light receiving sensor with the screen 2 as a field of view so as to receive reflected light of light projected from the pen device 1. Three are provided. Further, the control device 4 is incorporated into the projector 5.

  The control device 4 has a hardware configuration of a general computer device. That is, the control device 4 includes a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a non-volatile random access memory (NVRAM), an auxiliary storage device, a wireless interface, and the like.

  FIG. 3 is a view showing a configuration example of the pen device 1. FIG. 3A shows a state in which both the switches 12 and 13 are off, and the light emitting element 11 at the tip is not lit. FIG. 3B shows a state in which the switch 12 is turned on by pressing the tip of the pen device 1 against the screen 2 and the light emitting element 11 is lit. FIG. 3C shows a state in which the light emitting element 11 is on by the switch 13 being turned on by the user.

<Mode and Light Spot Size, Center Brightness>
FIG. 4 is a diagram showing an example of the assumed operation. FIG. 4A is an operation of pressing the pen device 1 against the screen 2 to perform drawing. FIG. 4B is an operation of moving the pen device 1 back and forth (approaching and separating) in a state where the pen device 1 is released from the screen 2. FIG. 4C shows an operation of moving the tip of the pen device 1 to the left and right (air pan) or up and down (air tilt) in a state where the pen device 1 is released from the screen 2. The operation in the air by the pen device 1 is not limited to the illustrated one.

  FIG. 5 is a view showing the feature of the light spot on the screen 2. As shown in FIG. 5 (a), FIG. 5 (b) shows the luminance on the screen 2 from the center to the periphery of the light spot when the tip of the pen device 1 is separated from the screen 2 by a certain distance. is there. The center of the light spot has the highest brightness, and the brightness decreases toward the periphery. Further, a distance on the screen 2 at which the luminance reaches a predetermined threshold is set as a light spot size. The brightness at the center of the light spot is called the center brightness.

  FIG. 6 is a view showing an example of the center luminance of the light spot with respect to the distance between the pen device 1 and the screen 2. Although the central luminance is substantially constant from a distance of zero to a predetermined value, when the distance exceeds the central luminance, it becomes difficult for light rays to reach and the central luminance decreases.

  FIG. 7 is a view showing an example of the light spot size with respect to the distance between the pen device 1 and the screen 2. As shown in the lower left of FIG. 7, the light spot size on the screen 2 is small at a distance # 1 close to the screen 2, but as shown in the lower stage, the light spot size is large when moving away to a distance # 2. Become. Furthermore, as shown on the lower right, when the distance is increased to # 3, the luminance hardly reaches the threshold for obtaining the light spot size, and the light spot size starts to decrease. The upper part of FIG. 7 shows an example of the change of the light spot size according to the distance.

  FIG. 8 is a diagram showing an example of the light spot size and the center luminance with respect to the distance between the pen device 1 and the screen 2, where the light spot size is indicated by a solid line and the center luminance is indicated by a broken line. The light spot size alone can not specify the distance because two distances correspond to the same light spot size. Although there is a range in which the change is slow and the distance can not be specified with only the central brightness, the distance can be specified by combining both the intersection size and the central brightness.

  FIG. 9 is a diagram showing an example of the relationship between the mode associated with the distance, the light spot size, and the center luminance. FIG. 9A shows an example in which the distance from zero to the maximum is associated with five modes # 1 to # 5. The number of modes is arbitrary. FIG. 9B is an example of a correspondence table showing the relationship between the light spot size and the range of values of central luminance and the mode at that time. Therefore, by observing the light spot size and the center luminance, the mode can be specified from the correspondence table without the calculation of the distance.

<User operation>
FIG. 10 is a diagram showing an example of the procedure of the user's operation. In FIG. 10, when the user operates the pen device 1 and the light emitting element 11 is turned on (step S1), mode tentative determination is performed by observing a light spot on the screen 2 (step S2). As described above, the light emitting element 11 may be turned on either when the tip of the pen device 1 is pressed against the screen 2 or when the user turns it on purposely.

  When mode tentative determination is performed, clocking by the timer is started (step S3), and if the user moves the pen device 1 during that time (step S4), mode tentative determination is performed again (step S2). When the timer times out, mode finalization is performed (step S5). This is the mode selection operation.

  Next, when the user moves the pen device 1 (step S6), adjustment of parameters corresponding to the mode is performed (step S7), and if lighting continues, the mode is maintained and the movement according to the pen device 1 is performed Parameter adjustment is performed (steps S8, S6, S7). If the light is turned off, the process ends. After the mode selection up to here is the parameter adjustment operation.

<Mode selection process>
The details of the processing will be described below based on an example in which the mode is set to three modes of "handwriting mode", "pan tilt mode" and "wrinkle mode".

  FIG. 11 shows an example of the mode and the correspondence table. FIG. 11A shows a state in which “handwriting mode”, “pan tilt mode”, and “wrinkle mode” are assigned to the light spot size s and the center luminance i with respect to the distance d in ascending order of the distance d. The “handwriting mode” is a mode in which the pen device 1 is pressed against the screen 2 to perform drawing. The “pan-tilt mode” is a mode in which the viewpoint of the virtual camera is moved horizontally or vertically when an image is displayed in three dimensions. The “eyelid mode” is a mode in which an image is displayed in three dimensions and the viewpoint of the virtual camera is moved forward and backward.

  FIG. 11B shows an example of a correspondence table that defines the correspondence between the range of values of the light spot size and the value of the center luminance and the mode. The association table also defines the graphic displayed on the screen at the time of temporary decision of each mode and the main decision, and the length of the timer at the time of transition from temporary decision to the final decision.

  FIG. 12 is a flowchart showing an example of mode selection processing of the control device 4 corresponding to the mode selection operation of the user. In FIG. 12, when the mode selection process is started, the control device 4 performs image acquisition (image capture) from the image captured by the light receiving sensor 3 (step S101).

  Next, the control device 4 performs smoothing processing of the acquired image as pre-processing (step S102), performs binarization of the image (step S103), and performs contraction / expansion processing of the image (step S104). The expansion and contraction process is an expansion process after the image is shrunk and noise pixels can be effectively removed.

  Next, the control device 4 extracts a light spot from the preprocessed image (step S105). Extraction of a light spot is performed by extracting a pixel group with high luminance in the image. In addition, since only the infrared light and the like emitted from the pen device 1 are imaged by the light receiving sensor 3, the projection light from the projector 5 is not affected.

  Next, the control device 4 determines whether or not there is a light spot (step S106), and if there is not (step S106: no), the process returns to the image capture (step S101).

  If there is a light spot (yes in step S106), the control device 4 extracts the light spot size and the center luminance from the image (step S107).

  Next, the control device 4 compares the extracted light spot size and center luminance with the range of the light spot size and center luminance in the correspondence table shown in FIG. (Step S108).

  Next, the control device 4 determines whether or not there is one previously decided temporarily (step S109), and when it is not present (no in step S109), a timer is generated (step S110). The length of the timer uses the value set corresponding to the mode in the association table.

  Note that, when generating the above-described tentative determination and timer, the control device 4 displays a graphic as shown in the left column of FIG. 13 at the position of the center of the light spot on the screen 2. The upper part of FIG. 13 shows the pan tilt mode, and the lower part shows the mastic mode, and a donut graph timer graphic is displayed at the center of the temporary display graphic defined in the correspondence table. In this timer pattern, the gauge extends clockwise from the top as the timer counts down, and the gauge reaches full due to time-out. In addition, in the handwriting mode, the provisionally determined display figure is not defined in the correspondence table, and the timer length is also zero, so the provisionally displayed figure and the timer figure are not displayed, and this determination is immediately made. The final decision display figure is displayed.

  Returning to FIG. 12, if there is a previous temporary decision (yes in step S109), the control device 4 subsequently determines whether it is the same temporary decision as the previous one (step S111), and if it is not the same (step S111) No in S111 resets the timer (step S112).

  After generation of the timer (step S110), after resetting of the timer (step S112), or in the case where it is tentatively determined the same as the previous time (yes in step S111), the control device 4 counts down the timer (step S113). .

  Next, the control device 4 determines whether the timer has timed out (step S114), and if it has not timed out (no in step S114), the process returns to the image capture (step S101).

  If it has timed out (yes in step S114), the control device 4 determines the mode (step S115). At this time, as shown in the right column of FIG. 13, the control device 4 deletes the timer graphic, and changes the graphic for each mode from the provisional display graphic to the main display graphic. In the illustrated example, it is assumed that the figure is light in color at the time of tentative determination, and is in the form of dark color at the time of determination.

  Referring back to FIG. 12, after the main determination of the mode, if it is the handwriting mode (yes in step S116), the corresponding adjustment parameter does not exist, so the process returns to the image capture (step S101). In the case of the pan and tilt mode (yes in step S117) or in the case of the eyelid mode (yes in step S118), the process proceeds to the corresponding parameter adjustment process.

  FIG. 14 is a diagram showing an example of parameters for each mode. In the handwriting mode, as described above, it is assumed that there are no parameters to be adjusted. In the pan and tilt mode, the pan angle and the tilt angle of the virtual camera are to be adjusted. In the eyelid mode, the position in the front-rear direction of the virtual camera is the target of adjustment.

<Parameter adjustment process>
FIG. 15 is a flowchart showing an example of parameter adjustment processing in the pan and tilt mode. In FIG. 15, when the parameter adjustment process in the pan and tilt mode is started after the mode selection process, the control device 4 performs image acquisition (image capture) from the image captured by the light receiving sensor 3 (step S201).

  Next, the control device 4 performs smoothing processing of the acquired image as pre-processing (step S202), performs binarization of the image (step S203), and performs contraction / expansion processing of the image (step S204).

  Next, the control device 4 extracts a light spot from the preprocessed image (step S205).

  Next, the control device 4 determines whether there is a light spot (step S206). If there is a light spot (yes in step S206), the control device 4 detects the light spot center position from the image (step S207).

  Next, the control device 4 calculates the moving direction and moving width of the light spot center position (step S208), and adjusts (interlocks) the pan tilt angle of the virtual camera (step S209). Thereby, the screen display is updated. Then, the process returns to the image capture (step S201).

  If there is no light spot (no in step S206), the process returns to the mode selection process start (FIG. 12) (step S210).

  FIG. 16 is a flowchart showing an example of parameter adjustment processing in the eyebrow mode. In FIG. 16, when the parameter adjustment process in the eyelid mode is started after the mode selection process, the control device 4 performs image acquisition (image capture) from the image captured by the light receiving sensor 3 (step S211).

  Next, the control device 4 performs smoothing processing of the acquired image as pre-processing (step S212), performs binarization of the image (step S213), and performs contraction / expansion processing of the image (step S214).

  Next, the control device 4 extracts a light spot from the preprocessed image (step S215).

  Next, the control device 4 determines whether there is a light spot (step S216), and if there is (yes at step S216), the control device 4 extracts the light spot size and the center luminance from the image (step S217) .

  Next, the control device 4 estimates the proximity / retraction of the pen device 1 from the extracted light spot size and center luminance (step S218). This can be done using HMM (Hidden Markov Model).

  Next, when it is estimated that the user is separated (yes in step S219), the control device 4 changes the virtual camera position so as to approach the screen 2 (step S220). Thereby, the screen display is updated.

  Further, when it is estimated that the camera is in proximity (yes in step S221), the control device 4 changes the virtual camera position away from the screen 2 (step S222). Thereby, the screen display is updated.

  After the adjustment of the virtual camera position (steps S220 and S222), or when it is not estimated that both are separated as being close (no in step S221), the process returns to the image capture (step S211).

<Example of operation>
FIG. 17 is a diagram showing an example of the handwriting mode. FIG. 17A shows a state in which the user presses the pen device 1 against the screen 2 and lights it up to start drawing. In this state, the "handwriting mode" is selected from the light spot size and the center luminance.

  FIG. 17B shows a state where the user continues drawing with the pen device 1 and draws a rectangular frame.

  FIG. 17C shows a state in which the screen of the application is displayed with the frame drawn by the user as a window frame under control of the upper application.

  FIG. 18 is a view showing an example of the eyelid mode (planar display → eyelid display). FIG. 18A shows a state where the user turns on the pen device 1 at a little distance from the screen 2 to select “俯瞰 mode”, and moves the pen device 1 so as to be close to the screen 2. As a result, the image is pushed back and shifts to a three-dimensional display as shown in FIG. In the bird's-eye view mode, not only the screen in front of the user's eyes, but also the screen of the screen placed stereoscopically, such as right and left, can be stereoscopically displayed on the screen in front of the user's eyes.

  In FIG. 18B, by moving the pen device 1 forward while the user keeps lighting the pen device 1, the front-rear position of the virtual camera is adjusted in the direction away from it, and the image is pushed away. It will be in the state as shown in (c). .

  In FIG. 18C, the user further moves the pen device 1 forward while keeping the lighting of the pen device 1 on, resulting in the state of FIG. In (d) of FIG. 18, the user turns off the pen device 1 and finishes adjusting the depth of stereoscopic display.

  FIG. 19 is a diagram showing an example of the pan and tilt mode. FIG. 19A shows a state in which the “pan tilt mode” is selected by lighting the pen device 1 at such a distance that the user does not touch the screen 2.

  In FIG. 19B, by moving the pen device 1 to the right while the user keeps lighting the pen device 1, the viewpoint of the virtual camera is adjusted to the right, and the image is viewed from the right It shows the state of transition.

  In FIG. 19C, by moving the pen device 1 upward while the user keeps lighting the pen device 1, the viewpoint of the virtual camera is adjusted upward, and the image is viewed from above It shows the state of transition.

  FIG. 20 is a view showing an example of the eyelid mode (eyelid display → planar display). FIG. 20A shows a state in which the user turns on the pen device 1 at a little distance from the screen 2 and selects the “俯瞰 mode”.

  FIG. 20B shows a state where the image is pulled back to the front by moving the pen device 1 away from the screen 2 while the user keeps lighting the pen device 1. Finally, the three-dimensional display ends, and the display returns to the normal display as shown in FIG.

<Another Configuration Example of Pen Device>
As already described in FIG. 7 etc., the light spot size is linearly increased according to the distance until the distance between the pen device 1 and the screen 2 reaches a predetermined value (distance # 2) from the small side. And the distance can be determined from the light spot size. Further, in the configuration example of the pen device 1 shown in FIG. 3, power is directly supplied from the battery 14 to the light projecting element 11 through the switches 12 and 13, and the emission intensity of the light beam such as infrared light is substantially It is constant.

  Therefore, when the distance to the screen 2 is large, the reflected light is weak, and when the distance is small, the reflected light becomes strong. In particular, when the distance is short and the reflected light is strong, the light receiving sensor 3 for imaging the screen 2 may cause halation, and the light spot may not be detected or may be recognized as a light spot larger than the actual one. If an auto exposure function is provided on the light receiving sensor 3 side, the amount of light entering the light receiving sensor 3 can be controlled, but the load on the light receiving sensor 3 side becomes large, and the circuit becomes complicated and expensive. The software processing becomes heavy and the frame rate may be reduced.

  Therefore, here, a configuration example of the pen device 1 capable of accurately detecting the light spot without stabilizing the light amount and without providing the auto exposure function on the light receiving sensor 3 side without causing the halation will be described. Do.

  FIG. 21 is a view showing a configuration example of the pen device 1. In FIG. 21, the pen device 1 includes an infrared light generation unit 101, an infrared light amount detection unit 102, and an infrared light amount adjustment unit 103. The infrared light quantity adjustment unit 103 receives the reflected light of the light beam projected from the infrared light generation unit 101 by the infrared light quantity detection unit 102, and the light quantity of the light beam of the infrared light generation unit 101 is an appropriate value Adjust to

  FIG. 22 is a view showing a specific configuration example of the pen device 1. In FIG. 22, the pen device 1 is provided with a light emitting element 11 such as an infrared LED, a light receiving element 15 such as a phototransistor, and a switch 12 at the front. A switch 13 is provided on the side surface of the pen device 1. Inside the pen device 1, a battery 14 and a control circuit board 16 are provided.

  FIG. 23 is a view showing an example of the circuit configuration of the pen device 1. In FIG. 23, the switches SW1 and SW2 corresponding to the switches 12 and 13 are connected in parallel, and one end thereof is connected to the power supply VDD corresponding to the positive electrode of the battery 14. Between the other end of the power supply VDD and the ground (corresponding to the negative pole of the battery 14), a series circuit of a collector / emitter of a phototransistor PTr, a resistor R1, and a circuit of a power supply terminal / ground terminal of a controller (microcomputer) PIC , A resistor R4, an infrared LED (symbol in the figure: LED), and a series circuit of a collector and an emitter of the transistor Tr are connected.

  A capacitor C for noise removal is connected in parallel to the power supply terminal / ground terminal of the controller PIC. Further, the emitter of the phototransistor PTr is connected to the analog input terminal AD0 of the controller PIC via the resistor R2. The output terminal GP2 of the controller PIC is connected to the base of the transistor Tr via a resistor R3. A control signal of PWM (Pulse Width Modulation) having a predetermined duty is output from the output terminal GP2 of the controller PIC.

  A series circuit of a collector and an emitter of the phototransistor PTr and a resistor R1 corresponds to the infrared light amount detection unit 102, which detects the light amount received by the phototransistor PTr and converts it into a voltage. The circuit of the power supply terminal / ground terminal of the controller PIC corresponds to the infrared light amount adjustment unit 103, receives the voltage converted by the phototransistor PTr at the analog input terminal AD0 of the controller PIC, and generates PWM pulses according to the value. The width is adjusted by program and supplied to the transistor Tr. A series circuit of a resistor R4, an infrared LED (code: LED), and a collector-emitter of the transistor Tr corresponds to the infrared light generation unit 101, and corresponds to an infrared LED (code: LED) according to the PWM output of the controller PIC. Turn on / off the current flow.

  In the circuit example described above, the power supply VDD is applied to the phototransistor PTr and the controller PIC via the switches SW1 and SW2, but a separate power supply switch is provided and the phototransistor is not provided via the switches SW1 and SW2. It is also possible to supply power supply VDD to PTr and controller PIC.

  FIG. 24 shows an example of the internal configuration of the controller PIC. In FIG. 24, the controller PIC includes a central processing unit (CPU) 171, a clock transmitter 172, a random access memory (RAM) 173, a flash memory 174, various timers 175, an A / D converter 176 and an I / O port 177. Have. The CPU 171 executes the program using the RAM 173 as a work area based on the program and data stored in advance in the flash memory 174. The clock transmitter 172 supplies a clock signal to the CPU 171. The various timers 175 provide timer functions when timers are used in processing. The A / D converter 176 converts an analog signal input from an analog input terminal of the I / O port 177 into digital data.

  FIG. 25 is a flowchart showing an example of processing of the controller PIC. In FIG. 25, when the power is turned on to start the operation, the controller PIC sets the light amount target value Lt and the gain G (step S11).

Next, the current light amount Lc is read (step S12), and the control amount is calculated (step S13). That is, controller PIC is
Pt: PWM target duty
Pc: Current PWM target duty
Lt: Target of AD value of light quantity
Lc: AD value of current light amount
G: The following control formula, when using feedback gain
Pt = Pc + G * (Lt-Lc)
The PWM target duty Pt is calculated by

  Then, Pt is set as the control target value (step S14), and the current light amount Lc is read out, and the process returns to step S12. A PWM control signal corresponding to the control target value Pt is output to the output terminal GP2 of the controller PIC.

  FIG. 26 is an explanatory view of light amount adjustment of the pen device 1. FIG. 26A shows the case where the pen device 1 is at a position slightly away from the screen 2, and it lights up when the user presses the switch 13. When the distance to the screen 2 is large, the light reflected from the screen 2 is weak, so the light beam is adjusted to be strong.

  FIG. 26 (b) shows the case where the pen device 1 is positioned near the screen 2 without touching, and the user turns on when the switch 13 is pressed. When the distance to the screen 2 is small, the light reflected from the screen 2 becomes strong, so the light beam is adjusted to be weak.

  FIG. 26 (c) shows the case where the pen device 1 is in contact with the screen 2, and it lights up by pressing the switch 12 without pressing the switch 13. When the distance to the screen 2 is small, the light reflected from the screen 2 becomes strong, so the light beam is adjusted to be weak.

<Summary>
As described above, according to the present embodiment, it is possible to improve the operability with respect to the content displayed on the screen.

  Hereinabove, the preferred embodiments have been described. Although specific embodiments have been shown and described herein, it is apparent that various modifications and changes can be made to these embodiments without departing from the broad spirit and scope as defined in the claims. . That is, it should not be construed as being limited by the details of the specific example and the attached drawings.

  The control device 4 is an example of a content display control device. The projector 5 is an example of means for displaying an image of content. The light receiving sensor 3 is an example of means for detecting reflected light. The mode selection unit 41 is an example of means for selecting a mode. The parameter adjustment unit 42 is an example of a unit that detects a user's operation. The image display control unit 43 is an example of a unit that controls display of an image.

The following sections will be further disclosed regarding the above description.
(Supplementary Note 1) The content display control device
Display an image of the content on the screen by the projector,
Detecting the reflected light of the light beam emitted by the hand-held device onto the screen,
Based on the detected reflected light, a mode set corresponding to the state of the reflected light which changes according to the distance between the screen and the hand-held device is selected,
Detecting the user's operation from the state of the reflected light from the selection of the mode until the light beam disappears;
A content display control method, comprising: executing processing of controlling display of an image of the content on the screen based on the detected operation.
(Supplementary Note 2) In the content display control method according to supplementary note 1,
The content display control method, wherein the process of selecting the mode selects a mode associated with a combination of a light spot size on the screen of the reflected light and a value of center luminance.
(Supplementary Note 3) In the content display control method according to any one of supplementary notes 1 and 2,
In the process of selecting the mode, when the mode is selected based on the reflected light, the mode is temporarily decided, and when there is no change in the mode for a predetermined time set for each mode, transition to the final decision state is made. Content display control method characterized by
(Supplementary Note 4) In the content display control method according to supplementary note 3,
The content display control device
A graphic for temporary determination according to the mode is displayed in the temporary determination state centering on the center position of the light beam, and a process for displaying the main determination graphic according to the mode in the main determination state is executed. Content display control method.
(Supplementary Note 5) In the content display control method according to any one of supplementary notes 1 to 4,
The content display control method, wherein the mode includes a handwriting mode, a pan-tilt mode, and an eyebrow mode, in which the distances are set in ascending order.
(Supplementary Note 6) In the content display control method according to any one of supplementary notes 1 to 5,
The handheld device is
Irradiating the screen with a light beam;
Detecting the reflected light of the light beam emitted to the screen by the screen;
A content display control method, comprising: controlling the light beam so that the content display control device can detect the state of reflected light.
(Supplementary Note 7) A means for displaying an image of content on a screen by a projector,
Means for detecting the light reflected by the screen of the light beam emitted by the hand-held device onto the screen;
A means for selecting a mode set in association with the state of the reflected light which changes according to the distance between the screen and the hand-held device based on the detected reflected light;
Means for detecting the user's operation from the state of the reflected light from the selection of the mode until the light beam disappears;
And means for controlling display of the image of the content on the screen based on the detected operation.
(Supplementary Note 8) In the content display control device according to supplementary note 7,
The content display control device, wherein the means for selecting the mode selects a mode associated with a combination of a light spot size on the screen of the reflected light and a value of center luminance.
(Supplementary Note 9) In the content display control device according to any one of supplementary notes 7 or 8,
When the mode is selected based on the reflected light, the means for selecting the mode is in a provisionally decided state, and when there is no change in the mode over a predetermined time set for each mode, the mode is shifted to the final decided state. Content display control apparatus characterized by the above.
(Supplementary Note 10) In the content display control device according to supplementary note 9,
It has a means for displaying a figure for temporary determination according to the mode in the temporary determination state with the center position of the light beam as a center, and displaying the figure for main determination according to the mode in the main determination state. Content display control device.
(Supplementary Note 11) In the content display control device according to any one of supplementary notes 7 to 10,
The content display control device, wherein the mode includes a handwriting mode, a pan-tilt mode, and an eyebrow mode, in which the distances are set in ascending order.
(Supplementary Note 12) In the content display control device according to any one of supplementary notes 7 to 11,
The handheld device is
Irradiating the screen with a light beam;
Detecting the reflected light of the light beam emitted to the screen by the screen;
A content display control device, wherein the light beam is controlled so that the content display control device can detect the state of reflected light.
(Supplementary Note 13)
Display an image of the content on the screen by the projector,
Detecting the reflected light of the light beam emitted by the hand-held device onto the screen,
Based on the detected reflected light, a mode set corresponding to the state of the reflected light which changes according to the distance between the screen and the hand-held device is selected,
Detecting the user's operation from the state of the reflected light from the selection of the mode until the light beam disappears;
A content display control program that executes processing of controlling display of an image of the content on the screen based on the detected operation.
(Supplementary Note 14) In the content display control program according to supplementary note 13,
The content display control program, wherein the process of selecting the mode selects a mode associated with a combination of a light spot size on the screen of the reflected light and a value of center luminance.
(Supplementary Note 15) In the content display control program according to any one of supplementary notes 13 or 14,
The process of selecting the mode is a tentative state when the mode is selected based on the reflected light, and the content display is transitioned to the final state when there is no change in the mode for a predetermined time set for each mode. Control program.
(Supplementary Note 16) In the content display control program according to supplementary note 15,
On the computer
Content display control for displaying a temporary determination figure according to a mode in the temporary determination state centering on the center position of the light beam, and displaying a main determination figure according to the mode in the main determination state program.
(Supplementary Note 17) In the content display control program according to any one of supplementary notes 13 to 16,
The content display control program including the handwriting mode, the pan-tilt mode, and the eyebrow mode in which the modes are set in order of decreasing distance.
(Supplementary Note 18) In the content display control program according to any one of supplementary notes 13 to 17,
The handheld device is
Irradiating the screen with a light beam;
Detecting the reflected light of the light beam emitted to the screen by the screen;
The content display control program which controls the said light beam to the light emission quantity which the said content display control apparatus can detect the state of a reflected light.

Reference Signs List 1 pen device 11 light emitting element 12, 13 switch 14 battery 15 light receiving element 16 control circuit board 101 infrared light generating unit 102 infrared light amount detecting unit 103 infrared light amount adjusting unit 2 screen 3 light receiving sensor 4 control device 41 mode selecting unit 42 Parameter Adjustment Unit 43 Image Display Control Unit 5 Projector 51 Projection Lens 6 Terminal Device

Claims (14)

The content display control device
Display an image of the content on the screen by the projector,
Detecting the reflected light of the light beam emitted by the hand-held device onto the screen,
Based on the detected reflected light, a mode set corresponding to the state of the reflected light which changes according to the distance between the screen and the hand-held device is selected,
Detecting the user's operation from the state of the reflected light from the selection of the mode until the light beam disappears;
Executing a process of controlling display of the image of the content on the screen based on the detected operation ;
A content display control method, wherein the process of selecting the mode selects a mode associated with a combination of a light spot size on the screen of the reflected light and a value of center luminance . In the content display control method according to claim 1 ,
In the process of selecting the mode, when the mode is selected based on the reflected light, the mode is temporarily decided, and when there is no change in the mode for a predetermined time set for each mode, transition to the final decision state is made. Content display control method characterized by In the content display control method according to claim 2 ,
The content display control device
A graphic for temporary determination according to the mode is displayed in the temporary determination state centering on the center position of the light beam, and a process for displaying the main determination graphic according to the mode in the main determination state is executed. Content display control method. The content display control method according to any one of claims 1 to 3 .
The content display control method, wherein the mode includes a handwriting mode, a pan-tilt mode, and an eyebrow mode, in which the distances are set in ascending order. The content display control method according to any one of claims 1 to 4 .
The handheld device is
Irradiating the screen with a light beam;
Detecting the reflected light of the light beam emitted to the screen by the screen;
A content display control method, comprising: controlling the light beam so that the content display control device can detect the state of reflected light. The content display control device
Display an image of the content on the screen by the projector,
Detecting the reflected light of the light beam emitted by the hand-held device onto the screen,
Based on the detected reflected light, a mode set corresponding to the state of the reflected light which changes according to the distance between the screen and the hand-held device is selected,
Detecting the user's operation from the state of the reflected light from the selection of the mode until the light beam disappears;
Executing a process of controlling display of the image of the content on the screen based on the detected operation ;
The process of selecting the mode is temporarily decided when the mode is selected based on the reflected light, and shifts to the final decided state when there is no change in the mode for a predetermined time set for each mode. Content display control method characterized by In the content display control method according to claim 6 ,
The content display control device
A graphic for temporary determination according to the mode is displayed in the temporary determination state centering on the center position of the light beam, and a process for displaying the main determination graphic according to the mode in the main determination state is executed. Content display control method. In the content display control method according to claim 6 or 7 ,
The content display control method, wherein the mode includes a handwriting mode, a pan-tilt mode, and an eyebrow mode, in which the distances are set in ascending order. The content display control method according to any one of claims 6 to 8 .
The handheld device is
Irradiating the screen with a light beam;
Detecting the reflected light of the light beam emitted to the screen by the screen;
A content display control method, comprising: controlling the light beam so that the content display control device can detect the state of reflected light. The content display control device
Display an image of the content on the screen by the projector,
Detecting the reflected light of the light beam emitted by the hand-held device onto the screen,
Based on the detected reflected light, a mode set corresponding to the state of the reflected light which changes according to the distance between the screen and the hand-held device is selected,
Detecting the user's operation from the state of the reflected light from the selection of the mode until the light beam disappears;
Executing a process of controlling display of the image of the content on the screen based on the detected operation ;
The content display control method , wherein the mode includes a handwriting mode, a pan-tilt mode, and an eyebrow mode, in which the distances are set in ascending order . In the content display control method according to claim 10 ,
The handheld device is
Irradiating the screen with a light beam;
Detecting the reflected light of the light beam emitted to the screen by the screen;
A content display control method, comprising: controlling the light beam so that the content display control device can detect the state of reflected light. The content display control device
Display an image of the content on the screen by the projector,
Detecting the reflected light of the light beam emitted by the hand-held device onto the screen,
Based on the detected reflected light, a mode set corresponding to the state of the reflected light which changes according to the distance between the screen and the hand-held device is selected,
Detecting the user's operation from the state of the reflected light from the selection of the mode until the light beam disappears;
Executing a process of controlling display of the image of the content on the screen based on the detected operation ;
The handheld device is
Irradiating the screen with a light beam;
Detecting the reflected light of the light beam emitted to the screen by the screen;
A content display control method , wherein the light beam is controlled so that the content display control device can detect the state of the reflected light . Means for displaying an image of the content on the screen by the projector;
Means for detecting the light reflected by the screen of the light beam emitted by the hand-held device onto the screen;
A means for selecting a mode set in association with the state of the reflected light which changes according to the distance between the screen and the hand-held device based on the detected reflected light;
Means for detecting the user's operation from the state of the reflected light from the selection of the mode until the light beam disappears;
Means for controlling display of the image of the content on the screen based on the detected operation ;
The content display control device , wherein the process of selecting the mode selects a mode associated with a combination of the light spot size on the screen of the reflected light and the value of center luminance . On the computer
Display an image of the content on the screen by the projector,
Detecting the reflected light of the light beam emitted by the hand-held device onto the screen,
Based on the detected reflected light, a mode set corresponding to the state of the reflected light which changes according to the distance between the screen and the hand-held device is selected,
Detecting the user's operation from the state of the reflected light from the selection of the mode until the light beam disappears;
A process of controlling display of the image of the content on the screen is executed based on the detected operation ;
The process of selecting the mode selects a mode associated with a combination of the light spot size on the screen of the reflected light and the value of the central luminance .