JP2023033109A - Display device, display system, display control method, and program - Google Patents

Abstract

To ensure real-time performance of a display device which shares a display screen with an information terminal and receives operations or settings on the display screen from the information terminal.SOLUTION: A display device is provided, comprising a display unit configured to display a display screen based on a video signal input from an information terminal, an acquisition unit configured to acquire changes in an image showing the position on the display screen from the video signal, and a reception unit configured to receive operations on the display screen according to the changes in the image.SELECTED DRAWING: Figure 4

Description

The present invention relates to a display device, a display system, a display control method, and a program.

An electronic blackboard that has a large display with a touch panel and can display, for example, a display screen of a PC (Personal Computer) on the display and draw characters or figures on the display screen. It has been known.

In addition, in an environment where a wired connection is difficult, such as in a large conference room, there is known a technique in which a PC and an electronic blackboard are placed on the same network, the PC and the electronic blackboard share a remote screen, and the electronic blackboard is operated. (See, for example, Patent Document 1).

However, the technique disclosed in Patent Literature 1 has a problem that real-time performance cannot be ensured because screen sharing is performed between an information processing apparatus (PC) and an electronic blackboard via a network. Note that such a problem is not limited to electronic blackboards, but is present in various display devices that share a display screen with an information processing device and receive operations or settings on the display screen.

An embodiment of the present invention has been made in view of the above problems. to ensure

In order to solve the above problems, a display device according to an embodiment of the present invention includes a display unit that displays a display screen based on a video signal input from an information terminal, and a position on the display screen based on the video signal. and a reception unit that receives an operation on the display screen based on the change in the image.

According to an embodiment of the present invention, real-time performance can be ensured in a display device that shares a display screen with an information terminal and receives operations or settings on the display screen from the information terminal.

1 is a diagram illustrating an example of a system configuration of a display system according to one embodiment; FIG. It is a figure which shows the example of the hardware constitutions of the electronic blackboard which concerns on one Embodiment. 3 is a diagram illustrating an example of a hardware configuration of a projector according to one embodiment; FIG. It is a figure showing an example of functional composition of an electronic blackboard concerning one embodiment. 7 is a flowchart illustrating an example of operation acceptance processing according to the first embodiment; FIG. 7 is a diagram illustrating pointer detection processing according to the first embodiment; FIG. 4 is a diagram for explaining coordinate conversion processing according to the first embodiment; FIG. 7 is a diagram illustrating operation acceptance processing according to the first embodiment; FIG. 5 is a diagram illustrating an example of gestures according to one embodiment; 10 is a flowchart illustrating an example of operation acceptance processing according to the second embodiment; FIG. 12 is a flowchart illustrating an example of operation acceptance processing according to the third embodiment; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<System configuration>
FIG. 1 is a diagram illustrating an example of the system configuration of a display system according to one embodiment. The display system 1 includes, for example, an information terminal 10 and an electronic blackboard 100 connected to the information terminal 10 via a video signal cable 11, as shown in FIG.

The information terminal 10 is, for example, an information processing device such as a PC (Personal Computer) or a tablet terminal. The video signal cable 11 is, for example, a cable that transmits a digital video signal (wired signal) such as HDMI (High Definition Multimedia Interface, registered trademark), DisplayPort, USB (Universal Serial Bus) Type-C. Further, in the present embodiment, the video signal cable 11 may be a cable that transmits an analog video signal (wired signal) such as VGA, for example.

The electronic blackboard 100 is a touch sensor-equipped display, and is also called an IWB (Interactive White Board). The electronic blackboard 100 can directly write on the screen displayed on the display with, for example, a pen or a finger, and can save the contents displayed on the display as data.

The electronic blackboard 100 connects the information terminal 10 and the electronic blackboard 100 with the video signal cable 11, and displays the display screen of the information terminal 10 on the display 12 of the electronic blackboard 100 based on the video signal input from the information terminal 10. can be displayed.

When the operator writes on the display screen 13, the operator stands in front of the electronic blackboard 100 and performs operations. I often sit there. However, with the electronic blackboard 100, for example, when performing operations such as page switching, enlargement/reduction, screen saving, etc., the operation buttons 15 and the like on the display screen are operated before the electronic blackboard 100. .

Therefore, in the technology disclosed in Patent Document 1, an information terminal and an electronic blackboard are connected to the same network, and an operator can operate the electronic blackboard using a remote control application installed in the information terminal. I'm trying However, in this method, since it goes through a network, there is a problem that real-time performance cannot be ensured, for example, delays occur during video playback and the display of dropped frames, or the response to operations is slow. .

Therefore, the electronic whiteboard 100 according to the present embodiment displays the display screen 13 based on the video signal input from the information terminal 10 via the video signal cable 11, and displays the display screen 13 based on the movement of the pointer 14 on the display screen 13. to accept an operation or setting for the display screen 13 .

As a result, the electronic whiteboard 100 can receive an operation on the display screen 13 while displaying the display screen 13 in real time based on the video signal input from the information terminal 10 . Therefore, according to the present embodiment, a display device (such as the electronic whiteboard 100) that shares the display screen with the information terminal 10 and receives operations or settings for the display screen 13 from the information terminal 10 can ensure real-time performance. Become.

Note that the electronic blackboard 100 is an example of the display device according to the present embodiment. The electronic blackboard 100 may be called an electronic whiteboard, an electronic information board, or the like, for example. Also, the present embodiment can be suitably applied to various information processing apparatuses. The information processing device may be, for example, an output device such as digital signage, an industrial machine, a game machine, or the like.

In the present embodiment, the operator can operate the electronic blackboard 100 without using the touch sensors included in the electronic blackboard 100 . Therefore, the display device according to the present embodiment may be various display devices having a computer configuration, such as a display without a touch sensor or a projector.

<Hardware configuration>
Next, the hardware configuration of the display device according to this embodiment will be described.

(Hardware configuration of electronic blackboard)
FIG. 2 is a diagram illustrating an example of the hardware configuration of the electronic blackboard according to one embodiment. As shown in FIG. 2, the electronic blackboard 100 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an SSD (Solid State Drive) 104, a network I/ It has an F (Interface) 105, an external device connection I/F 106, and the like.

Among these, the CPU 101 is an arithmetic unit that controls the operation of the electronic blackboard 100 as a whole. The ROM 102 is a non-volatile memory that stores programs used to start the CPU 101, such as the CPU 101 and an IPL (Initial Program Loader). A RAM 103 is a volatile memory used as a work area for the CPU 101 . The SSD 104 is a large-capacity, nonvolatile storage device that stores various data such as programs for the electronic blackboard 100 .

A network I/F 105 is a communication interface for connecting the electronic blackboard 100 to a communication network and performing communication. The external device connection I/F 106 is an interface for connecting various external devices.

The electronic blackboard 100 also includes a capture device 111, a GPU (Graphics Processing Unit) 112, a display controller 113, a contact sensor 114, a sensor controller 115, an electronic pen controller 116, a near field communication circuit 119, and an antenna for the near field communication circuit 119. 119a, a power switch 117, selection switches 118, and the like.

Among these, the capture device 111 captures (obtains) a display screen or the like displayed on a display or the like of the external information terminal 10 as a still image or moving image. The GPU 112 is a semiconductor chip (processor) that specializes in graphics. A display controller 113 controls and manages screen display in order to output an output image from the GPU 112 to the display 12 or the like. The contact sensor 114 detects that the electronic pen 130 , the user's hand H, or the like touches the display 12 . A sensor controller 115 controls the processing of the contact sensor 114 .

The contact sensor 114 performs coordinate input and coordinate detection using, for example, an infrared shielding method. In this method of inputting coordinates and detecting coordinates, two light emitting/receiving devices installed at both ends of the upper side of the display 12 radiate a plurality of infrared rays parallel to the display 12, and are provided around the display 12. This is a method of receiving light that is reflected by a reflecting member and returns along the same optical path as that of light emitted by a light receiving element. The contact sensor 114 outputs to the sensor controller 115 the ID of the infrared rays emitted by the two light emitting/receiving devices blocked by the object, and the sensor controller 115 identifies the coordinate position, which is the contact position of the object. The electronic pen controller 116 communicates with the electronic pen 130 to determine whether or not the display 12 has been touched with the tip of the pen or the bottom of the pen. The short-range communication circuit 119 is a communication circuit that performs short-range wireless communication with another device via an antenna 119a.

The power switch 117 is a switch for switching on/off the power of the electronic blackboard 100 . The selection switches 118 are, for example, a group of switches for adjusting the brightness and color of the display on the display 12 .

Further, the electronic blackboard 100 has a bus line 120 . The bus line 120 includes an address bus, a data bus, various control signals, etc. for electrically connecting each component such as the CPU 101 shown in FIG.

Note that the contact sensor 114 is not limited to the infrared shielding type, but is a capacitive type touch panel that identifies the contact position by detecting changes in capacitance, and identifies the contact position by voltage changes of two opposing resistive films. A resistive touch panel or the like may be used. Further, the contact sensor 114 may use various detection means such as an electromagnetic induction type touch panel that detects electromagnetic induction generated when a contacting object touches the display unit and identifies the contact position. Further, the electronic pen controller 116 may determine whether or not the user grips the electronic pen 130 or other parts of the electronic pen as well as the tip and the pen butt of the electronic pen 130 .

(Hardware configuration of the projector)
FIG. 3 is a diagram illustrating an example of a hardware configuration of a projector according to one embodiment; Note that a projector is an example of a display device that does not have a touch sensor. As shown in FIG. 3, the projector 300 includes a CPU 301, a ROM 302, a RAM 303, a media I/F 305, an operation unit 306, a power switch 307, a network I/F 308, a sound output I/F 309, a capture device 310, a fan driving circuit 311, It includes a cooling fan 312, an LED (Light Emitting Diode) driving circuit 313, an LED light source 314, a projection device 315, a projection lens 316, a speaker 317, a bus line 318, and the like.

Among these, the CPU 301 controls the operation of the projector 300 as a whole. The ROM 302 stores programs used to drive the CPU 301 . A RAM 303 is used as a work area for the CPU 301 . A media I/F 305 controls reading or writing (storage) of data to a media 304 that is an external storage device.

An operation unit 306 has various keys, buttons, LEDs, etc., and is used by the user to perform various operations other than turning on and off the power of the projector 300 . The operation unit 306 receives an instruction operation such as a size adjustment operation of a projected image, a color tone adjustment operation, a focus adjustment operation, or a keystone adjustment operation, and outputs the contents of the received operation to the CPU 301 . A power switch 307 is a switch that accepts power-on and power-off operations of the projector 300 .

A network I/F 308 is an interface for connecting the projector 300 to a network such as a wireless LAN or a wired LAN. A sound output I/F 309 is a circuit for outputting sound from the speaker 317 under the control of the CPU 301 . The capture device 310 captures (acquires) the display screen or the like displayed on the display or the like of the external information terminal 10 as a still image or moving image.

A fan driving circuit 311 is connected to the CPU 301 and the cooling fan 312 , and drives and stops the cooling fan 312 based on a control signal from the CPU 301 . Cooling fan 312 rotates to exhaust the air inside projector 300 and cool the inside of projector 300 .

The LED driving circuit 313 controls lighting and extinguishing of the LED light source 314 and the like under the control of the CPU 301 . The LED light source 314 irradiates the projection device 315 with projection light when turned on under the control of the LED drive circuit 313 . The projection device 315 projects the modulated light obtained by modulating the projection light from the LED light source 314 by the spatial light modulation method based on the image data to be displayed onto the projection surface of the screen through the projection lens 316 as an image. do. As the projection device 315, for example, a liquid crystal panel or a DMD (Digital Micromirror Device) is used. The LED driving circuit 313, the LED light source 314, the projection device 315, and the projection lens 316 as a whole function as a projection section (projection means) that projects a projection image onto a projection surface based on image data.

The bus line 318 includes, in FIG. 3, an address bus, a data bus, various control signals, etc. for electrically connecting each component connected to the bus line 318 .

In the above configuration, when the power is supplied, the CPU 301 is activated according to a control program pre-stored in the ROM 302 and gives a control signal to the LED driving circuit 313 to light the LED light source 314 . The CPU 301 also gives a control signal to the fan driving circuit 311 to rotate the cooling fan 312 at a predetermined rated rotation speed. Furthermore, in the projector 300, when power supply is started, the projection device 315 becomes ready for image display, and power is supplied to various other components.

When the power switch 307 of the projector 300 is turned off, the power switch 307 sends a power off signal to the CPU 301. When the CPU 301 detects the power off signal, the CPU 301 gives a control signal to the LED driving circuit 313 to turn off the LED. Light source 314 is turned off. After that, after a predetermined time has passed, the CPU 301 gives a control signal to the fan driving circuit 311 to stop the cooling fan 312, terminates its own control processing, and finally gives an instruction to the power supply circuit to turn off the power supply. stop the supply of

<Functional configuration>
(Functional configuration of electronic blackboard)
FIG. 4 is a diagram illustrating an example of the functional configuration of the electronic blackboard according to one embodiment. The electronic blackboard (an example of the display device) 100 has an input unit 401, a display unit 402, an acquisition unit 403, a detection unit 404, a conversion unit 405, It implements a reception unit 406, a storage unit 407, and the like. It should be noted that at least part of the functional configurations described above may be realized by hardware.

The input unit 401 performs input processing for receiving an input of a video signal from the information terminal 10 . For example, the input unit 401 uses the capture device 111 to capture video signals input from the information terminal 10 .

The display unit 402 executes display processing for displaying a display screen based on the video signal input from the information terminal 10 to the input unit 401 . For example, the display unit 402 displays the screen displayed by the information terminal 10 and the display screen 13 for displaying the operation buttons 15 and the like on the display 12 of the electronic blackboard 100, as shown in FIG. Note that the operation button 15 is an example of a display element that receives an operation on the display screen 13 or electronic whiteboard 100 .

The acquisition unit 403 acquires changes in the image on the display screen 13 based on the video signal input to the input unit 401 . For example, the acquisition unit 403 acquires the trajectory of the pointer 14 such as the mouse pointer of the information terminal 10 indicating the position on the display screen 13 . Note that the locus of the pointer is an example of change in the image indicating the position on the display screen 13 .

The detection unit 404 executes detection processing for detecting a predetermined gesture from the trajectory of the pointer 14 acquired by the acquisition unit 403 . For example, the detection unit 404 detects a predetermined gesture representing an operation such as “select”, “cancel”, or “return” from the trajectory of the pointer 14 .

The conversion unit 405 executes a conversion process of converting the coordinates of the trajectory of the gesture detected by the detection unit 404 into coordinates on the electronic blackboard 100 . Note that the detection unit 404 , the conversion unit 405 , or the like may be included in the reception unit 406 .

The receiving unit 406 executes receiving processing for receiving an operation on the display screen 13 based on the trajectory of the pointer acquired by the acquiring unit 403 . As an example, when the gesture detected by the detection unit 404 represents a “select” operation, the reception unit 406 extracts the coordinates of the selected operation button 15 from the coordinates of the trajectory of the gesture converted by the conversion unit 405 into the coordinates on the electronic whiteboard 100 . is specified, and selection operation of the specified operation button 15 is accepted.

As another example, when the gesture detected by the detection unit 404 indicates an “undo” operation, the reception unit 406 receives an undo operation for undoing the immediately preceding operation. In this case, the conversion unit 405 may not (may) perform conversion processing for converting the coordinates of the trajectory of the gesture to the coordinates on the electronic blackboard 100 .

The storage unit 407 is implemented by, for example, a program executed by the CPU 101 and a memory such as the RAM 103 or SSD 104, and stores various data or information such as pointer coordinate data or predetermined gesture information.

(Functional configuration of projector)
The projector 300 implements a functional configuration similar to the functional configuration of the electronic whiteboard 100 shown in FIG. Since each functional configuration of the projector 300 is the same as each functional configuration of the electronic whiteboard 100, description thereof will be omitted here.

<Process flow>
Next, the processing flow of the display control method according to the first embodiment will be described.

(Operation reception processing)
FIG. 5 is a flowchart illustrating an example of operation reception processing according to the first embodiment. For example, in the display system 1 shown in FIG. An example of reception processing for the electronic blackboard 100 to receive the performed operation is shown.

In step S<b>501 , the acquisition unit 403 of the electronic whiteboard 100 detects the pointer 14 indicating the position on the display screen 13 from the video signal input to the input unit 401 . For example, as shown in FIG. 6, the acquisition unit 403 calculates the luminance difference between frames between frame 1 and frame 2, which is the frame next to frame 1, and binarizes it using a threshold value. As a result, for example, coordinates 601 before movement and coordinates 602 after movement of the pointer 14 can be obtained, as shown in an inter-frame difference 600 in FIG. In addition, the detection method of the pointer 14 is not limited to this, and any detection method may be used.

In step S502, when the pointer 14 can be detected, the acquisition unit 403 shifts the process to step S503. On the other hand, if the pointer 14 cannot be detected, the obtaining unit 403 returns the process to step S501.

After moving to step S<b>503 , the acquisition unit 403 acquires the detected trajectory of the pointer 14 . For example, the acquisition unit 403 acquires the detected coordinates of the pointer 14 and sequentially stores the acquired coordinates in the storage unit 407 for a predetermined period, thereby acquiring the trajectory of the coordinates of the pointer 14 .

In step S<b>504 , the detection unit 404 of the electronic blackboard 100 detects a predetermined gesture from the trajectory of the pointer 14 acquired by the acquisition unit 403 . As an example, the detection unit 404 stores trajectory information of a plurality of predetermined gestures in the storage unit 407, calculates the difference between the stored trajectory information and the trajectory of the pointer 14 acquired by the acquisition unit 403, A gesture with the smallest sum of all pixel values may be detected. As another example, the detection unit 404 may detect the gesture by learning the user's gesture motion in advance by machine learning and reading the trajectory of the pointer 14 acquired by the acquisition unit 403 . .

Alternatively, the detection unit 404 stores in advance the feature amount extracted from a plurality of predetermined gestures in the storage unit 407, and calculates the similarity with the feature amount extracted from the trajectory of the pointer 14 acquired by the acquisition unit 403. It is also possible to detect a gesture whose degree of similarity is equal to or greater than a threshold.

In step S505, if the predetermined gesture can be detected, the detection unit 404 causes the process to proceed to step S506. On the other hand, if the predetermined gesture cannot be detected, the detection unit 404 returns the processing to step S501.

In step S<b>506 , the conversion unit 405 of the electronic blackboard 100 converts the coordinates of the trajectory of the gesture detected by the detection unit 404 into coordinates on the electronic blackboard 100 . For example, as shown in FIGS. 7A and 7B, when the coordinate space of the information terminal 10 and the coordinate space of the electronic blackboard 100 are different, the conversion unit 405 converts the pointer 14 in the coordinate space of the information terminal 10 is transformed into a trajectory on the coordinate space of the electronic blackboard 100 . At this time, the conversion unit 405 may convert the entire trajectory of the pointer 14, or may convert the center coordinates of the gesture, etc., since the gesture has already been determined.

In step S<b>507 , the receiving unit 406 of the electronic whiteboard 100 detects the operation button 15 corresponding to the position of the gesture converted by the converting unit 405 . For example, assume that the display unit 402 displays a plurality of operation buttons 15a to 15e on the display screen 13 as shown in FIG. 8, and that there is a converted gesture 801 around the operation button 15a. In this case, the reception unit 406 detects the operation button 15a corresponding to the gesture 801 (surrounded by the gesture 801). Alternatively, the reception unit 406 may detect the operation button 15 a within a predetermined distance from the center coordinates of the gesture 801 as the operation button 15 corresponding to the gesture 801 .

In step S508, if the operation button can be detected, the reception unit 406 causes the process to proceed to step S509. On the other hand, if the operation button cannot be detected, the reception unit 406 returns the process to step S501.

When the process proceeds to step S509, an operation corresponding to the detected operation button 15 and gesture is accepted. FIG. 9 is a diagram illustrating examples of gestures according to one embodiment. As an example, the accepting unit 406 may accept a gesture of circling the detected operation button 15, such as the gesture A901 in FIG.

As another example, the accepting unit 406 may accept a gesture of checking the detected operation button 15 as a selection operation for selecting the detected operation button 15, such as the gesture B902 in FIG. Alternatively, the accepting unit 406 may accept a gesture of checking on the detected operation button, such as the gesture B902 in FIG.

As another example, the reception unit 406 performs a gesture of moving the pointer 14 to the left or right a predetermined number of times or more on the detected operation button 15, such as gesture C903 in FIG. may be accepted as Note that each operation described above is an example, and the gestures accepted by the accepting unit 406 may be arbitrary gestures.

5, the electronic blackboard 100 displays the display screen 13 based on the video signal input from the information terminal 10, and displays the display screen based on the movement of the pointer 14 on the display screen 13. 13 can be accepted.

Note that the display unit 402 of the electronic blackboard 100 displays operation buttons 15 for receiving operations or settings for the display screen 13 such as page switching, scaling, screen saving, etc., and the receiving unit 406 displays the display screen 13 by the operator. You may accept the operation for . Further, the display unit 402 of the electronic blackboard 100 displays the operation buttons 15 for receiving operations or settings for the electronic blackboard 100 such as brightness, contrast, aspect, volume, power saving function, etc., and the receiving unit 406 displays the An operation on the electronic blackboard 100 may be accepted.

5 has been described assuming that the display device is the electronic whiteboard 100, but since the processing in FIG. A similar process can also be applied to

[Second embodiment]
In the first embodiment, the reception unit 406 of the electronic blackboard 100 receives the operation corresponding to the operation button 15 and the gesture. You may further receive an operation corresponding to .

As an example, the reception unit 406 recognizes a gesture of moving the pointer 14 left or right a predetermined number of times or more, such as gesture C903 in FIG. You can accept it. As another example, the receiving unit 406 may receive, in any area, a gesture of drawing a "Z" shape, such as the gesture D in FIG. good.

<Process flow>
FIG. 10 is a flowchart illustrating an example of operation reception processing according to the second embodiment. In the display system 1 shown in FIG. 1, the operator who operates the information terminal 10 uses the mouse pointer or the like to perform this process on the operation buttons 15 displayed on the display screen 13 of the electronic blackboard 100. Another example of reception processing in which the electronic blackboard 100 receives an operation is shown.

10, steps S501 to S504 and S506 to S509 are the same as the operation reception processing according to the first embodiment described with reference to FIG. 5, and therefore descriptions thereof are omitted here. Also, the functional configuration of the electronic blackboard 100 according to the second embodiment may be the same as the functional configuration of the electronic blackboard 100 according to the first embodiment shown in FIG.

In step S505 of FIG. 10, when the predetermined gesture is detected, the detection unit 404 of the electronic whiteboard 100 shifts the process to step S1001.

In step S<b>1001 , the reception unit 406 of the electronic blackboard 100 determines whether the gesture detected by the detection unit 404 is a gesture dependent on the operation button 15 . For example, the reception unit 406 classifies predetermined gestures into gestures that depend on the operation button 15 and gestures that do not depend on the operation button 15, and stores them in the storage unit 407. 15 can be determined.

If the detected gesture is a gesture dependent on the operation button 15, the reception unit 406 causes the process to proceed to step S506. On the other hand, if the detected gesture is a gesture that does not depend on the operation button 15, the reception unit 406 causes the process to proceed to step S1002.

After moving to step S1002, the receiving unit 406 receives an operation corresponding to the detected gesture. As an example, the reception unit 406 recognizes a gesture of moving the pointer 14 left or right a predetermined number of times or more, such as gesture C903 in FIG. You can accept it.

As another example, the receiving unit 406 may receive, in any area, a gesture of drawing a "Z" shape, such as the gesture D in FIG. good. Each gesture described above is an example, and the gesture received by the receiving unit 406 may be any gesture.

10, the electronic whiteboard 100 displays the display screen 13 based on the video signal input from the information terminal 10, and displays the display screen 13 based on the movement of the pointer 14 on the display screen 13. 13 can be accepted.

10, the display device is the electronic whiteboard 100, but the process of FIG. 10 does not use a touch sensor. A similar process can also be applied to

[Third embodiment]
In the second embodiment, the accepting unit 406 of the electronic blackboard 100 accepts operations corresponding to gestures that depend on the operation buttons 15 and gestures that do not depend on the operation buttons 15 . It may accept only gestures that do not depend on

<Functional configuration>
The electronic blackboard 100 according to the third embodiment has, as an example, the functional configuration of the electronic blackboard 100 according to the first embodiment described with reference to FIG. As another example, the electronic blackboard 100 according to the third embodiment has a functional configuration obtained by omitting the detection unit 404 and the conversion unit 405 from the functional configuration of the electronic blackboard 100 according to the first embodiment described with reference to FIG. may have

<Process flow>
FIG. 11 is a flowchart illustrating an example of operation reception processing according to the third embodiment. In the display system 1 shown in FIG. 1, the operator who operates the information terminal 10 uses the mouse pointer or the like to perform this process on the operation buttons 15 displayed on the display screen 13 of the electronic blackboard 100. Another example of reception processing in which the electronic blackboard 100 receives an operation is shown. Note that the processing of steps S501 to S504 in the processing shown in FIG. 11 is the same as the operation reception processing according to the first embodiment described with reference to FIG. 5, so description thereof will be omitted here.

In step S505 of FIG. 11, when the predetermined gesture is detected, the detection unit 404 of the electronic blackboard 100 shifts the process to step S1101.

After shifting to step S1101, the receiving unit 406 receives an operation corresponding to the detected gesture. As an example, the reception unit 406 recognizes a gesture of moving the pointer 14 left or right a predetermined number of times or more, such as gesture C903 in FIG. You can accept it.

As another example, the receiving unit 406 may receive, in any area, a gesture of drawing a "Z" shape, such as the gesture D in FIG. good. Note that the gestures described above are merely examples, and the gestures accepted by the accepting unit 406 may be arbitrary gestures.

11, the electronic whiteboard 100 displays the display screen 13 based on the video signal input from the information terminal 10, and displays the display screen 13 based on the movement of the pointer 14 on the display screen 13. 13 can be accepted.

In FIG. 11, the explanation has been given assuming that the display device is the electronic whiteboard 100. However, since the process of FIG. A similar process can also be applied to

As described above, the display device according to each embodiment of the present invention displays the display screen 13 in real time based on the video signal input from the information terminal 10, and can accept operations or settings for the display screen 13. can. Therefore, according to the present embodiment, the display device (the electronic blackboard 100, the projector 300, or the like) that shares the display screen with the information terminal 10 and receives the operation on the display screen 13 from the information terminal 10 can ensure real-time performance. become.

<Supplement>
Each function of each embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in this specification means a processor programmed by software to perform each function, such as a processor implemented by an electronic circuit, or a processor designed to perform each function described above. ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays) and devices such as conventional circuit modules.

<Appendix>
This specification discloses a communication device, a communication method, and a program for each of the following items.
(Section 1)
a display unit that displays a display screen based on a video signal input from an information terminal;
an acquisition unit that acquires a change in an image indicating a position on the display screen from the video signal;
a reception unit that receives an operation on the display screen based on the change in the image;
A display device.
(Section 2)
2. The display device according to claim 1, wherein the display unit displays, on the display screen, a display element for accepting an operation or setting for the display screen.
(Section 3)
2. The display device according to claim 1, wherein the display unit displays, on the display screen, display elements for accepting operations or settings for the display device.
(Section 4)
4. The display device according to claim 2 or 3, wherein the reception unit receives an operation on the display element according to a change in the image corresponding to the position of the display element.
(Section 5)
5. The display device according to any one of items 1 to 4, wherein when the change in the image indicates a predetermined gesture, the accepting unit accepts an operation corresponding to the gesture.
(Section 6)
6. The display device of Claim 5, wherein the gestures include gestures corresponding to one or more of select, cancel, and return.
(Section 7)
5. The display device according to any one of items 1 to 4, wherein the video signal is a digital or analog wired signal.
(Section 8)
8. The display device according to any one of items 1 to 7, wherein the change in the image includes a trajectory of a pointer indicating a position on the display screen.
(Section 9)
a display unit that displays a display screen based on a video signal input from an information terminal;
an acquisition unit that acquires a change in an image indicating a position on the display screen from the video signal;
a reception unit that receives an operation on the display screen based on the change in the image;
a display system.
(Section 10)
A process of displaying a display screen based on a video signal input from an information terminal;
A process of acquiring a change in an image indicating a position on the display screen from the video signal;
a process of receiving an operation on the display screen based on the change in the image;
A computer-implemented display control method.
(Section 11)
A program that causes a computer to execute the display control method according to claim 10.
(Section 12)
a display unit that displays a display screen based on a video signal input from an information terminal;
an acquisition unit that acquires a trajectory of a pointer indicating a position on the display screen from the video signal;
a reception unit that receives an operation on the display screen based on the trajectory of the pointer;
A display device.
(Section 13)
13. The display device according to claim 12, wherein the display unit displays, on the display screen, display elements for accepting operations or settings for the display screen.
(Section 14)
13. The display device according to claim 12, wherein the display unit displays, on the display screen, display elements for receiving operations or settings for the display device.
(Section 15)
15. The display device according to Item 13 or 14, wherein the receiving unit receives an operation on the display element according to a trajectory of the pointer corresponding to the position of the display element.
(Section 16)
16. The display device according to any one of items 12 to 15, wherein when the trajectory of the pointer indicates a predetermined gesture, the accepting unit accepts an operation corresponding to the gesture.
(Section 17)
17. The display device of Claim 16, wherein the gestures include gestures corresponding to one or more of select, cancel, and return.
(Section 18)
16. The display device according to any one of items 12 to 15, wherein the video signal is a digital or analog wired signal.
(Section 19)
a display unit that displays a display screen based on a video signal input from an information terminal;
an acquisition unit that acquires a trajectory of a pointer indicating a position on the display screen from the video signal;
a reception unit that receives an operation on the display screen based on the trajectory of the pointer;
a display system.
(Section 20)
A process of displaying a display screen based on a video signal input from an information terminal;
a process of acquiring a trajectory of a pointer indicating a position on the display screen from the video signal;
a process of receiving an operation on the display screen based on the trajectory of the pointer;
A computer-implemented display control method.
(Section 21)
A program that causes a computer to execute the display control method according to claim 20.

Although the embodiments of the present invention have been described above, the present invention is not limited to such specific embodiments, and various modifications and Application is possible.

1 display system 10 information terminal 13 display screen 15 operation button (an example of display elements)
100 electronic blackboard (an example of a display device)
300 projector (an example of a display device)
402 display unit 403 acquisition unit 406 reception unit 901 gesture A
902 Gesture B
903 Gesture C
904 Gesture D

JP 2015-70345 A

Claims (11)

a display unit that displays a display screen based on a video signal input from an information terminal;
an acquisition unit that acquires a change in an image indicating a position on the display screen from the video signal;
a reception unit that receives an operation on the display screen based on the change in the image;
A display device. 2. The display device according to claim 1, wherein said display unit displays, on said display screen, a display element for accepting an operation or setting for said display screen. 2. The display device according to claim 1, wherein said display unit displays, on said display screen, display elements for receiving operations or settings for said display device. 4. The display device according to claim 2, wherein said accepting unit accepts an operation on said display element according to a change in said image corresponding to the position of said display element. The display device according to any one of claims 1 to 3, wherein, when the change in the image indicates a predetermined gesture, the reception unit receives an operation corresponding to the gesture. 6. The display device of claim 5, wherein the gestures include gestures corresponding to one or more of select, cancel, and return. 4. The display device according to any one of claims 1 to 3, wherein the video signal is a digital or analog wired signal. 4. The display device according to any one of claims 1 to 3, wherein the change in the image includes a locus of a pointer indicating a position on the display screen. a display unit that displays a display screen based on a video signal input from an information terminal;
an acquisition unit that acquires a change in an image indicating a position on the display screen from the video signal;
a reception unit that receives an operation on the display screen based on the change in the image;
a display system. A process of displaying a display screen based on a video signal input from an information terminal;
A process of acquiring a change in an image indicating a position on the display screen from the video signal;
a process of receiving an operation on the display screen based on the change in the image;
A computer-implemented display control method. A program that causes a computer to execute the display control method according to claim 10 .

Images