JP7302640B2 - Display device operation method and display device - Google Patents

Description

The present invention relates to a method of operating a display device and a display device.

Japanese Unexamined Patent Application Publication No. 2002-200003 describes a display device that displays an image represented by image data supplied from an image data supply device such as a PC (Personal Computer).
The display device described in Patent Document 1 stores the first image data in the first buffer upon receiving the first image data from the image data supply device. Subsequently, the display device displays the first image indicated by the first image data stored in the first buffer on the second area of the display surface. When the display device subsequently receives the second image data from the image data supply device, it stores the second image data in the second buffer. Subsequently, the display device displays the second image indicated by the second image data stored in the second buffer in the second area instead of the first image, and displays the second image data stored in the first buffer. A first image represented by one image data is displayed in a first area on the display surface.

JP 2008-225175 A

In the display device described in Patent Document 1, even if a line or the like is written on the first image in the second area in response to an operation of an indicator such as an electronic pen before receiving the second image data, the When the second image data is received, the first image indicated by the first image data stored in the first buffer, that is, the first image in which the writing is not reflected is displayed in the first area. For this reason, the display device described in Patent Document 1 cannot display the contents of the writing in the first area even if it is desired to leave the display of the contents of writing executed in the second area.

One aspect of the operation method according to the present invention is an operation method executed by a display device that displays an image on a display surface, wherein a first image is displayed on a first region of the display surface, and A third image generated by displaying a second image in a second area different from the first area and superimposing a writing image based on a writing operation on the second image on the second image, determining whether a condition for displaying the image in the second area instead of the second image and changing the image displayed in the first area and the image displayed in the second area is satisfied; In a situation where the first image is displayed in the first area and the third image is displayed in the second area, if it is determined in the determination that the condition is satisfied, the image displayed in the first area. is changed from the first image to the third image, and the image displayed in the second area is changed from the third image to the fourth image.

One aspect of the display device according to the present invention is a display device that displays an image on a display surface, wherein a first image is displayed on a first region of the display surface, and the first image is displayed on the first region of the display surface. a display unit that displays a second image in a second area different from the area; a display control unit that controls the display unit; and an image displayed in the first area and an image displayed in the second area. a determination unit that determines whether a change condition is satisfied, wherein the display control unit generates a second image generated by superimposing a writing image based on a writing operation on the second image on the second image. causing the display unit to perform an operation of displaying three images in the second area instead of the second image, the first image being displayed in the first area, and the third image being displayed in the second area; In a situation where an image is displayed, if it is determined that the condition is satisfied, the image displayed in the first area is changed from the first image to the third image and displayed in the second area. The display unit is caused to change the image from the third image to the fourth image.

1 is a diagram showing a projector system 1000 including a projector 1 according to a first embodiment; FIG. 1 shows a projector system 1000; FIG. 1 shows a projector system 1000; FIG. 1 shows a projector system 1000; FIG. 4A and 4B are diagrams showing an example of an indicator 2; FIG. 1 is a diagram showing an example of a projector 1; FIG. 4A and 4B are diagrams for explaining an operation sequence of the indicator 2; FIG. 4 is a flowchart for explaining an example of a write operation; 4 is a flowchart for explaining an example of an image switching operation; It is a figure for demonstrating the 3rd modification. FIG. 16 is a flowchart for explaining an example of an image switching operation when the change condition is the sixth condition; FIG.

A: First Embodiment A1: Overview of Projector System 1000 FIG. 1 is a diagram showing a projector system 1000 including a projector 1 according to the first embodiment. A projector system 1000 includes a projector 1 and an indicator 2 .

The projector 1 is installed on a portion of the wall above the upper end 4 a of the projection surface 4 . The projector 1 may be placed, for example, on a desk, table, or floor instead of being installed on the wall, or may be hung from the ceiling. The projection surface 4 is, for example, a whiteboard. The projection surface 4 is not limited to a whiteboard, but may be, for example, a screen fixed to a wall, a portion of a wall, or a door. The projection surface 4 is an example of a display surface.

The projector 1 receives image data from the PC3. The projector 1 receives image data from the PC 3 by wire. The projector 1 may receive image data wirelessly from the PC 3 .

The image data is, for example, data representing an image that serves as material for class. The image data is not limited to data representing an image that serves as material for a class, and may be data representing an image that serves as a presentation material, for example. Each image data represents the content of one page in the document. Image data is provided from the PC 3 according to the ascending order of pages in the material.

PC3 is an example of an image data supplier. A source of image data may also be referred to as an image supplier. The image data supply source is not limited to the PC 3, and may be, for example, a tablet terminal, a smartphone, or a so-called document camera.

The projector 1 displays an image on the projection surface 4 by projecting the image onto the projection surface 4 . FIG. 1 shows a mode in which the projector 1 displays an image G on the projection surface 4. As shown in FIG. The projector 1 is an example of a display device. The display device is not limited to the projector 1, and may be a display such as an FPD (Flat Panel Display). FPDs are, for example, liquid crystal displays, plasma displays, or organic EL (Electro Luminescence) displays. Hereinafter, the area on which the image is projected on the projection surface 4 will be referred to as a "projection area R". The shape of the projection area R is the shape of the image projected from the projector 1 .

Here, the image projected from the projector 1 will be described using the image G. FIG.
The image G has a horizontally long shape. The image G includes a first image G1 and a second image G2. The first image G1 and the second image G2 are arranged in the horizontal direction. The size of the first image G1 is equal to the size of the second image G2. The size of the first image G1 may differ from the size of the second image G2. For example, the first image G1 may be larger than the second image G2. The first image G1 may be smaller than the second image G2. The first image G1 and the second image G2 may be in contact with each other or may be separated from each other.

The projector 1 displays the first image G1 in the first region R1 located on the left side of the projection region R, and displays the second image G2 in the second region R2 located on the right side of the projection region R.

When the projector 1 is used in a class at school, the teacher advances the class using the image displayed in the second area R2. The teacher uses the image displayed in the first area R1 as an auxiliary. For example, the image already displayed in the second region R2 is displayed in the first region R1. That is, even if the image displayed in the second region R2 is switched, the image displayed in the second region R2 before the image is switched in the second region R2 is displayed in the first region R1.
For this reason, even if the image displayed in the second region R2 changes before the student writes in the notebook the image that was displayed in the second region R2 before the image changes in the second region R2, the student can The image displayed in the second area R2 before the image is switched in the second area R2 can be written in a notebook.

The first image G1 is an image represented by the first image data supplied from the PC 3 to the projector 1 . In FIG. 1, an image showing "AB" is shown as an example of the first image G1. The first image G1 is not limited to the image showing "AB" and can be changed as appropriate.
The second image G2 is an image indicated by the second image data supplied to the projector 1 from the PC3. In FIG. 1, an image showing "F" is shown as an example of the second image G2. The second image G2 is not limited to the image showing "F" and can be changed as appropriate. The second image data is supplied from the PC 3 to the projector 1 after the first image data.

An update button 6 for updating the display of the image is superimposed on the second image G2. In FIG. 1, an image showing a “rhombus figure” is shown as an example of the update button 6 . The shape of the update button 6 is not limited to a rhombus, and may be circular or triangular, for example.

The indicator 2 is, for example, a pen-shaped indicator. The shape of the indicator 2 is not limited to the shape of a pen, and may be, for example, a cylinder, a prism, a cone, or a pyramid. The user operates the image projected by the projector 1 by using the indicator 2 . For example, the user holds the shaft portion 2 b of the indicator 2 and moves the indicator 2 on the projection surface 4 while keeping the tip 2 a in contact with the projection surface 4 .

The projector 1 generates imaging data by imaging an area including the projection area R with the camera 15 . The projector 1 identifies the position of the indicator 2 , that is, the operation position by the indicator 2 by analyzing the imaging data. The projector 1 projects, for example, a line corresponding to the trajectory of the operation position of the pointer 2 onto the projection surface 4 . Therefore, a user can perform a writing operation by using the pointer 2 . A line corresponding to the trajectory of the operation position of the pointer 2 is an example of a written image based on the writing operation.

The color of the line corresponding to the trajectory of the operation position of the pointer 2 may be set in advance, or may be appropriately changed according to the operation of a color selection button (not shown) or a color selection icon (not shown).

FIG. 2 shows an image representing an ellipse as an example of the writing image 5 . In the example shown in FIG. 2, a third image G3 generated by superimposing the written image 5 on the second image G2 is displayed in the second area R2 instead of the second image G2.
The written image 5 is not limited to an image showing an ellipse and can be changed as appropriate. The number of writing images 5 included in the third image G3 may be more than one. Note that the projector 1 can also superimpose the writing image 5 on the first image G1.

When the projector 1 detects that the update button 6 has been operated by the indicator 2 by analyzing the imaging data, the projector 1 issues a request instruction requesting subsequent image data to be supplied to the projector 1 temporally later than the second image data. It is determined that it has been received from the user. In this embodiment, the subsequent image data is different from the image data representing the second image G2, and the fourth image G4 represented by the subsequent image data is different from the second image G2.

Upon receiving a request instruction from the user, the projector 1 requests the PC 3 for subsequent image data. Upon receiving the request for subsequent image data, the PC 3 supplies the subsequent image data to the projector 1 .

When the projector 1 receives subsequent image data in a situation where the first image G1 is displayed in the first region R1 and the third image G3 is displayed in the second region R2 as shown in FIG. is changed from the first image G1 to the third image G3, and the image displayed in the second region R2 is changed from the third image G3 to the fourth image G4. Therefore, as shown in FIG. 3, the third image G3 including the writing image 5 moves from the second area R2 to the first area R1, and the fourth image G4 is displayed in the second area R2.

On the other hand, when the projector 1 receives subsequent image data in a situation where the first image G1 is displayed in the first region R1 and the second image G2 is displayed in the second region R2 as shown in FIG. The image displayed in the region R1 is changed from the first image G1 to the second image G2, and the image displayed in the second region R2 is changed from the second image G2 to the fourth image G4. Therefore, as shown in FIG. 4, the second image G2 moves from the second area R2 to the first area R1, and the fourth image G4 is displayed in the second area R2.

A2. Example of Indicator 2 FIG. 5 is a diagram showing an example of the indicator 2 . The indicator 2 includes a power source 21 , a first communication section 22 , a first light source 23 , a switch 24 , an indicator storage section 25 and an indicator control section 26 .

The power supply 21 supplies power to the first communication section 22 , the first light source 23 , the switch 24 , the pointer storage section 25 and the pointer control section 26 . In FIG. 5, power lines used by the power supply 21 to supply power are omitted. The power supply 21 starts supplying power when a power button (not shown) provided on the pointer 2 is turned on. The power supply 21 stops supplying power when the power button is turned off.

The first communication unit 22 wirelessly communicates with the projector 1 via Bluetooth. Bluetooth is a registered trademark. Bluetooth is an example of a short-range wireless system. The short-range wireless system is not limited to Bluetooth, and may be, for example, an infrared communication system or Wi-Fi. Wi-Fi is a registered trademark. The communication method for wireless communication between the first communication unit 22 and the projector 1 is not limited to the short-range wireless method, and may be another communication method.
The first communication unit 22 receives a synchronization signal from the projector 1, for example. The synchronization signal is used to synchronize the light emission timing of the indicator 2 with the imaging timing of the camera 15 in the projector 1 .

The first light source 23 is an LED (Light Emitting Diode) that emits infrared light. The first light source 23 is not limited to an LED, and may be, for example, an LD (Laser Diode) that emits infrared light. The first light source 23 emits infrared light to allow the projector 1 to recognize the position operated by the pointer 2 .

The switch 24 turns on when pressure is applied to the tip 2a of the indicator 2, and turns off when the pressure applied to the tip 2a is released. The switch 24 functions as a sensor that detects whether the tip 2a is in contact with the projection surface 4 or not.

The pointer storage unit 25 is, for example, a non-volatile semiconductor memory such as flash memory. The pointer storage unit 25 stores a control program executed by the pointer controller 26 and various data used by the pointer controller 26 .

The indicator control unit 26 is configured by, for example, one or more processors. For example, the pointer control unit 26 is configured by one or more CPUs (Central Processing Units). Some or all of the functions of the indicator control unit 26 are configured by circuits such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). good too. The pointer control unit 26 executes various processes in parallel or sequentially.

The indicator control unit 26 implements various functions by executing control programs stored in the indicator storage unit 25 .

For example, when the switch 24 is on, the indicator control unit 26 lights the first light source 23 at the timing specified based on the reception timing of the synchronization signal.

A3. Example of Projector 1 FIG. 6 is a diagram showing an example of the projector 1 . Projector 1 includes operation unit 11 , light receiving unit 12 , second communication unit 13 a , image data receiving unit 13 b , projection unit 14 , camera 15 , storage unit 16 , and processing unit 17 .

The operation unit 11 is, for example, various operation buttons, operation keys, or a touch panel. The operation unit 11 is provided on the housing of the projector 1 . The operation unit 11 receives a user's input operation.

The light receiving unit 12 receives an infrared signal from a remote controller (not shown) based on an input operation to the remote controller. The remote controller includes various operation buttons, operation keys, or a touch panel for receiving input operations.

The second communication section 13a performs wireless communication with the first communication section 22 of the indicator 2 by Bluetooth. As described above, the wireless communication method is not limited to Bluetooth, and may be infrared communication or Wi-Fi, for example.

The image data receiver 13b is connected to the PC 3, for example, via a wired LAN (Local Area Network). The connection between the image data receiving unit 13b and the PC 3 is not limited to wired LAN, and can be changed as appropriate. For example, the image data receiver 13b may be connected to the PC 3 via a wireless LAN, USB (Universal Serial Bus) cable, HDMI (High Definition Multimedia Interface) cable, or VGA (Video Graphics Array) cable. USB is a registered trademark. HDMI is a registered trademark.

The projection unit 14 displays an image on the projection plane 4 by projecting the image onto the projection plane 4 . For example, the projection unit 14 displays the first image G1 in the first region R1 and displays the second image G2 in the second region R2. The projection unit 14 is an example of a display unit. The projection unit 14 includes an image processing unit 141, a frame memory 142, a light valve driving unit 143, a second light source 144, a red liquid crystal light valve 145R, a green liquid crystal light valve 145G, and a blue liquid crystal light valve. 145B and a projection optical system 146. Hereinafter, the red liquid crystal light valve 145R, the green liquid crystal light valve 145G, and the blue liquid crystal light valve 145B are referred to as "liquid crystal light valves 145" when there is no need to distinguish them from each other.

The image processing unit 141 is configured by circuits such as one or more image processors. The image processing unit 141 receives image data from the processing unit 17 . For example, the image processor 141 receives two pieces of image data from the processor 17 .

The image processing unit 141 develops the image data in the frame memory 142 .
When the image processing unit 141 receives the two image data from the processing unit 17, the image processing unit 141 develops the two image data in the frame memory 142 so as not to overlap each other, thereby producing image data representing an image to be projected onto the projection area R. to generate
For example, when the image processing unit 141 receives the first image data and the second image data from the processing unit 17, the image processing unit 141 uses the frame memory 142 to generate image data representing the image G. FIG.
The frame memory 142 is configured by a storage device such as a RAM (Random Access Memory), for example. The image processing unit 141 generates an image signal by performing image processing on the image data developed in the frame memory 142 .

The image processing executed by the image processing unit 141 includes, for example, a geometric correction process for correcting trapezoidal distortion of the image projected by the projection unit 14, and an OSD (On Screen Display) image on the image indicated by the image data provided from the PC 3. Perform superimposed OSD processing. An example of the OSD image is the update button 6 shown in FIG.

The light valve drive unit 143 is configured by a circuit such as a driver, for example. The light valve driving section 143 drives the liquid crystal light valve 145 based on the image signal provided from the image processing section 141 .

The second light source 144 is, for example, an LED. The second light source 144 is not limited to an LED, and may be, for example, a xenon lamp, an ultra-high pressure mercury lamp, or a laser light source. The light emitted from the second light source 144 has its brightness distribution reduced by an integrator optical system (not shown), and is then passed through a color separation optical system (not shown). separated into The red color light component enters the red liquid crystal light valve 145R. The green light component enters the green liquid crystal light valve 145G. The blue color light component enters the blue liquid crystal light valve 145B.

The liquid crystal light valve 145 is composed of a liquid crystal panel or the like in which liquid crystal is present between a pair of transparent substrates. The liquid crystal light valve 145 has a rectangular pixel region 145a including a plurality of pixels 145p arranged in a matrix. In the liquid crystal light valve 145, a driving voltage is applied to the liquid crystal for each pixel 145p. When the light valve drive unit 143 applies a drive voltage based on the image signal to each pixel 145p, each pixel 145p is set to have a light transmittance based on the drive voltage. Light emitted from the second light source 144 is modulated by passing through the pixel region 145a, and an image based on the image signal is formed for each color light. The liquid crystal light valve 145 is an example of an optical modulator.

Images of respective colors are synthesized for each pixel 145p by a color synthesizing optical system (not shown) to generate a color image. A color image is projected through projection optics 146 .

The camera 15 generates imaging data by imaging the projection area R. FIG. The camera 15 includes a light-receiving optical system 151 such as a lens, and an imaging device 152 that converts light collected by the light-receiving optical system 151 into an electrical signal. The imaging element 152 is, for example, a CCD (Charge Coupled Device) image sensor that receives light in the infrared region and the visible light region. The imaging element 152 is not limited to a CCD image sensor, and may be, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor that receives light in the infrared region and visible light region.

The camera 15 may include a filter that blocks part of the light that enters the imaging device 152 . For example, when the imaging device 152 receives infrared light, the camera 15 arranges a filter that mainly transmits light in the infrared region in front of the imaging device 152 .

The camera 15 may be provided separately from the projector 1 . In this case, the camera 15 and the projector 1 may be connected to each other through a wired or wireless interface so that data can be transmitted and received.

When the camera 15 captures an image using visible light, for example, an image projected onto the projection surface 4 by the projection unit 14 is captured. The imaging data generated by imaging with the camera 15 using visible light is hereinafter referred to as "visible light imaging data". Visible light imaging data is used, for example, in calibration, which will be described later.

When the camera 15 captures an image using infrared light, captured image data representing the infrared light emitted by the pointer 2 is generated, for example. In the following, the imaging data generated by the imaging by the camera 15 using infrared light will be referred to as "infrared light imaging data". The infrared light imaging data is used, for example, to detect the operation position of the pointer 2 on the projection plane 4 .

The storage unit 16 is a recording medium readable by the processing unit 17 . Storage unit 16 includes, for example, nonvolatile memory 161 and volatile memory 162 . Examples of the nonvolatile memory 161 include ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory). Volatile memory 162 includes, for example, RAM.
Volatile memory 162 includes a first buffer 162a, a second buffer 162b, a third buffer 162c, and a fourth buffer 162d.

The first buffer 162a stores image data representing an image displayed in the first region R1. The first buffer 162a stores, for example, first image data.

The second buffer 162b stores image data representing an image displayed in the second region R2. The second buffer 162b stores, for example, second image data.

The third buffer 162c stores image data representing a superimposed image in which the written image is superimposed on the image displayed in the second region R2. The third buffer 162c stores, for example, third image data representing the third image G3.

The fourth buffer 162d stores image data representing a superimposed image in which the written image is superimposed on the image displayed in the first region R1.

The processing unit 17 is composed of, for example, one or more processors. For example, the processing unit 17 is configured by one or more CPUs. A part or all of the functions of the processing unit 17 may be configured by a circuit such as DSP, ASIC, PLD or FPGA. The processing unit 17 executes various processes in parallel or sequentially.

The processing unit 17 functions as an operation control unit 171 , a display control unit 172 and a determination unit 173 by reading and executing a control program from the storage unit 16 .

The operation control section 171 controls various operations of the projector 1 . For example, the operation control section 171 establishes communication between the indicator 2 and the second communication section 13a.

The operation control unit 171 also performs calibration. Calibration is a process of associating the coordinates on the frame memory 142 with the coordinates on the imaging data. Coordinates on the frame memory 142 correspond to positions on the image projected onto the projection plane 4 . By associating the position on the frame memory 142 with the position on the imaging data, for example, in the image projected on the projection surface 4, the portion corresponding to the operation position of the pointer 2 on the projection surface 4 is specified. be able to.

The calibration will be described below.
The operation control section 171 reads the calibration image data from the storage section 16 . Note that the operation control unit 171 may generate calibration image data according to a control program. The operation control section 171 provides calibration image data to the image processing section 141 .
The image processing unit 141 develops the calibration image data on the frame memory 142 . The image processing unit 141 performs geometric correction processing and the like on the calibration image data to generate an image signal. When the image processing unit 141 provides an image signal to the light valve driving unit 143 , a calibration image in which marks of a preset shape are arranged at intervals is projected onto the projection surface 4 .

Subsequently, the operation control unit 171 causes the camera 15 to capture a calibration image with visible light. The camera 15 generates visible light image data by capturing a calibration image with visible light. Subsequently, the operation control unit 171 acquires visible light imaging data from the camera 15 . The operation control unit 171 detects the mark represented by the visible light imaging data. The motion control unit 171 identifies the center-of-gravity position of each mark as the coordinates of each mark in the imaging data.

Subsequently, the operation control unit 171 associates the coordinates of the mark detected from the visible light imaging data with the coordinates of the mark on the frame memory 142 . The operation control unit 171 generates calibration data that associates the coordinates on the imaging data with the coordinates on the frame memory 142 by this association. The operation control section 171 stores the calibration data in the storage section 16 .
The above is the description of the calibration.

After completing the calibration, the operation control unit 171 causes the camera 15 to perform imaging with infrared light at regular time intervals, thereby generating infrared light image data. In addition, the operation control section 171 transmits a synchronization signal synchronized with the imaging timing of the camera 15 from the second communication section 13a to the indicator 2 .

The display control section 172 controls the projection section 14 . The display control unit 172 controls the image projected onto the projection region R by the projection unit 14 by controlling the projection unit 14 .

The determination unit 173 determines whether or not a condition for changing the image displayed in the first region R1 and the image displayed in the second region R2 is satisfied. Hereinafter, such conditions are referred to as "change conditions".
The change condition is the first condition that the projector 1 receives subsequent image data. Note that the change condition is not limited to the first condition and can be changed as appropriate.
The determination result of the determination unit 173 is used by the display control unit 172 .
For example, when the determination result of the determination unit 173 is affirmative in a situation where the third image G3 is displayed in the second region R2, the display control unit 172 displays the image displayed in the first region R1 as the first image. The image G1 is changed to the third image G3, and the image displayed in the second region R2 is changed from the third image G3 to the fourth image G4.

A4. Operation Example for Detecting Position of Indicator 2 FIG. 7 is a diagram for explaining an operation sequence of the indicator 2 .
In the sequence shown in FIG. 7, each cycle from the first cycle to the third cycle comprises four phases from the first phase PH1 to the fourth phase PH4. The first phase PH1 to fourth phase PH4 are repeated in order.

The first phase PH1 is the synchronization phase.
The indicator control unit 26 recognizes the start timing of the first phase PH1 by receiving the synchronization signal from the projector 1 via the first communication unit 22 . Each time of the first phase PH1 to the fourth phase PH4 is set to be the same. Therefore, by recognizing the start timing of the first phase PH1, the pointer control unit 26 also recognizes the start timings of the second phase PH2 to the fourth phase PH4.

The second phase PH2 and the fourth phase PH4 are position detection phases.
The pointer control unit 26 causes the first light source 23 to emit light in each of the second phase PH2 and the fourth phase PH4. The projector 1 performs imaging by the camera 15 in accordance with the light emission timing of the pointer 2 . The imaging data generated by imaging by the camera 15 shows the light emission of the pointer 2 as a bright spot.

The third phase PH3 is a contact determination phase.
The switch 24 is turned on in response to pressure on the tip 2a. In the third phase PH3, the first light source 23 emits light when the switch 24 is on. When the first light source 23 emits light in the third phase PH3, the image data of the camera 15 in the third phase PH3 shows the light emission of the pointer 2 as a bright spot.

The projector 1 determines the position of the indicator 2 in each of the second phase PH2 and the fourth phase PH4 by using the imaging data generated in each of the second phase PH2 and the fourth phase PH4 and the calibration data. Identify the position coordinates to show.
The projector 1 is located at a position close to the position coordinates specified by using the imaging data and the calibration data generated in the third phase PH3 among the position coordinates specified in each of the second phase PH2 and the fourth phase PH4. The coordinates are determined as the position where the pointer 2 contacts the projection surface 4, that is, the operation position.

A5. Example of Write Operation FIG. 8 is a flowchart for explaining an example of a write operation. The operation shown in FIG. 8 is repeatedly executed. In the explanation using FIG. 8, it is assumed that the pointer 2 and the projector 1 operate in a synchronized state according to the sequence shown in FIG. Assume that calibration has already been performed and calibration data is stored in the storage unit 16 .

In step S101, the camera 15 generates imaging data by imaging infrared light in each of the second phase PH2, the third phase PH3, and the fourth phase PH4.

In step S102, the display control unit 172 detects the position of the indicator 2 in the image indicated by the imaging data by analyzing the imaging data. The display control unit 172 converts the position of the indicator 2 in the imaging data into position coordinates indicating the position on the frame memory 142 using the calibration data.

In step S103, the display control unit 172 detects the light emission pattern of the pointer 2 by analyzing the imaging data.

In step S<b>104 , the display control unit 172 determines whether or not the indicator 2 has come into contact with the projection surface 4 based on the light emission pattern of the indicator 2 .

When the light emission pattern of the pointer 2 is a light emission pattern indicating contact between the pointer 2 and the projection surface 4, the display control unit 172 controls the second phase PH2 and the fourth phase PH4 in step S105. , and the position coordinates that are close to the position coordinates specified in the third phase PH3 are detected as the operation position. Then, the display control unit 172 writes a writing image 5, which is a line indicating the trajectory of the operation position, in the second image G2, as shown in FIG. 2, for example.
When the light emission pattern of the indicator 2 is a light emission pattern indicating that the indicator 2 is out of contact with the projection surface 4, the display control unit 172 ends the operation shown in FIG.

A6. Example of Image Switching Operation FIG. 9 is a flowchart for explaining an example of an image switching operation by the projector 1 . The operation shown in FIG. 9 is repeatedly executed. In the explanation using FIG. 9, it is assumed that the projector 1 displays on the projection surface 4 an image generated using two pieces of image data, for example, the image G shown in FIG.

In step S201, when the display control unit 172 detects that the update button 6 has been operated by the pointer 2 by analyzing the imaging data, it determines that a request instruction requesting subsequent image data has been received from the user.
If the display control unit 172 cannot detect the operation of the update button 6 by the pointer 2 by analyzing the imaging data, it determines that the request instruction has not been received from the user. If no request instruction has been received from the user, the operation shown in FIG. 9 ends.

Upon receiving the request instruction from the user, the display control unit 172 requests the subsequent image data from the PC 3 in step S202. The PC 3 transmits subsequent image data to the projector 1 in response to the subsequent image data request.

In step S203, the determination unit 173 determines whether or not the projector 1 receives subsequent image data. Here, in the present embodiment, the fact that the projector 1 receives the subsequent image data is used as the first condition, more specifically, as the change condition. Therefore, in step S203, the determination unit 173 substantially determines whether or not the change condition is satisfied.
Note that in step S203, when the projector 1 receives the subsequent image data within a predetermined time after receiving the request instruction from the user, the determination unit 173 determines that the projector 1 receives the subsequent image data. The predetermined time is, for example, 5 seconds. The predetermined time is not limited to 5 seconds and may be longer or shorter than 5 seconds.
Further, in step S203, when the projector 1 does not receive the subsequent image data within a predetermined time after receiving the request instruction from the user, the determination unit 173 determines that the projector 1 does not receive the subsequent image data.

When the determination unit 173 determines that the projector 1 has received the subsequent image data, that is, when it determines that the change condition is satisfied, in step S204, the display control unit 172 sets the first region R1 and the second region R2. , the image is switched.

In step S204, for example, in a situation where the first image G1 is displayed in the first region R1 and the third image G3 is displayed in the second region R2, if the result of determination by the determination unit 173 is affirmative, display control is performed. The unit 172 changes the image displayed in the first region R1 from the first image G1 to the third image G3, and changes the image displayed in the second region R2 from the third image G3 to the fourth image G4. change to

For example, the display control unit 172 first generates the third image data by capturing the third image G3. Subsequently, the display control unit 172 changes the image data stored in the first buffer 162a from the first image data to the third image data.
Note that when the third image data is stored in the third buffer 162c, the display control unit 172 stores the image data stored in the first buffer 162a as the first image data without capturing the third image G3. , to the third image data stored in the third buffer 162c.
Subsequently, the display control unit 172 changes the image data stored in the second buffer 162b from the second image data to the succeeding image data. Subsequently, the display control section 172 reads the third image data from the first buffer 162 a and outputs the third image data to the image processing section 141 . Subsequently, the display control unit 172 reads subsequent image data from the second buffer 162 b and outputs the subsequent image data to the image processing unit 141 .
When the image processing unit 141 receives the third image data and the subsequent image data, as shown in FIG. 3, the image processing unit 141 generates an image in which the third image G3 is positioned in the first region R1 and the fourth image G4 is positioned in the second region R2. is generated using the frame memory 142 . The image processing unit 141 outputs the image signal to the light valve driving unit 143, so that the projection unit 14 positions the third image G3 in the first region R1 and the fourth image G4 in the second region R2. An image is projected onto the projection surface 4 .

Further, in step S204, for example, in a situation where the first image G1 is displayed in the first region R1 and the second image G2 is displayed in the second region R2, if the determination result of the determination unit 173 is affirmative, The display control unit 172 changes the image displayed in the first region R1 from the first image G1 to the second image G2, and changes the image displayed in the second region R2 from the second image G2 to the fourth image G2. Change to image G4.

For example, the display control unit 172 first generates the second image data by capturing the second image G2. Subsequently, the display control unit 172 changes the image data stored in the first buffer 162a from the first image data to the second image data.
Note that the display control unit 172 changes the image data stored in the first buffer 162a from the first image data to the second image data stored in the second buffer 162b without capturing the second image G2. You can change it.
Subsequently, the display control unit 172 changes the image data stored in the second buffer from the second image data to the succeeding image data. Subsequently, the display control section 172 reads the second image data from the first buffer 162 a and outputs the second image data to the image processing section 141 . Subsequently, the display control unit 172 reads subsequent image data from the second buffer 162 b and outputs the subsequent image data to the image processing unit 141 .
When the image processing unit 141 receives the second image data and the subsequent image data, the image processing unit 141 generates an image in which the second image G2 is positioned in the first region R1 and the fourth image G4 is positioned in the second region R2, as shown in FIG. is generated using the frame memory 142 . The image processing unit 141 outputs the image signal to the light valve driving unit 143, so that the projection unit 14 positions the second image G2 in the first region R1 and the fourth image G4 in the second region R2. An image is projected onto the projection surface 4 .

If the determination unit 173 determines in step S203 that the projector 1 has not received subsequent image data, that is, if it determines that the change condition is not satisfied, the operation of FIG. 9 ends.

A7. Summary of First Embodiment The method of operating the display device and the display device according to the present embodiment described above include the following aspects.

The projection unit 14 displays the first image G1 in the first region R1 and displays the second image G2 in the second region R2. The display control unit 172 controls the projection unit 14 to display the third image G3 generated by superimposing the writing image 5 based on the writing operation on the second image G2 on the second image G2. is displayed in the second region R2 instead of . The determination unit 173 determines whether or not the change condition is satisfied. When the change condition is satisfied in a situation where the first image G1 is displayed in the first region R1 and the third image G3 is displayed in the second region R2, the display control unit 172 causes the image to be displayed in the first region R1. the image displayed in the second region R2 from the first image G1 to the third image G3, and the image displayed in the second region R2 from the third image G3 to the fourth image G4. Let

According to this aspect, even if the image displayed in the second region R2 is changed from the third image G3 to the fourth image G4, the third image G3 is displayed in the first region R1. Since the third image G3 is an image generated by superimposing the written image 5 on the second image G2, it is possible to display the contents of the writing executed in the first region R1 in the second region. .
Thus, for example, the student can keep track of important information written by the teacher.

The fourth image G4 is an image represented by subsequent image data supplied from the PC 3 to the projector 1 after the second image data representing the second image G2. Therefore, among the images displayed in the second region R2, the image indicated by the image data supplied from the PC3 can be updated according to the supply order of the image data supplied from the PC3.

When the projector 1 receives a request instruction requesting subsequent image data from the user, the projector 1 requests the subsequent image data from the PC 3, which is the supply source of the subsequent image data, and receives the subsequent image data supplied by the PC 3 in response to the request. Therefore, the user can cause the projector 1 to acquire subsequent image data by issuing a request instruction as necessary.

The change condition is the first condition that the projector 1 receives subsequent image data. Therefore, when the display control unit 172 receives subsequent image data in a situation where the third image G3 is displayed in the second region R2, the display control unit 172 changes the image displayed in the first region R1 from the first image G1 to the third image G3. It is possible to cause the projection unit 14 to perform an operation of changing to the image G3 and changing the image displayed in the second region R2 from the third image G3 to the fourth image G4.

B. Modifications Modifications of the above-exemplified embodiments are exemplified below. Two or more aspects arbitrarily selected from the following examples may be combined as appropriate within a mutually consistent range.

B1. First Modification In the first embodiment, if it is unclear whether the second image data and the subsequent image data are different from each other, the change condition is that the second image data and the subsequent image data are different from each other. A second condition may be used. In this case, the determination unit 173 determines whether the second image data and the subsequent image data are different from each other by comparing the second image data and the subsequent image data stored in the second buffer 162b. . When the change condition is the second condition, the display control unit 172 executes step S204 shown in FIG. 9 when the determination unit 173 determines that subsequent image data showing an image different from the second image G2 has been received. .
According to the first modification, the image displayed in the first area R1 and the image displayed in the second area R2 can be changed according to the update of the image data supplied from the PC3. Also, the projector 1 can determine whether or not the image data supplied from the PC 3 has been updated.

B2. Second Modified Example In the first embodiment, when the PC 3 transmits the supply signal indicating the supply of the subsequent image data when the subsequent image data is supplied, the change condition is that the projector 1 receives the supply signal. may be a condition. In this case, the determination unit 173 determines whether or not the projector 1 has received the supply signal. When the change condition is the third condition, the display control unit 172 executes step S204 shown in FIG. 9 when the determination unit 173 determines that the projector 1 has received the supply signal.
According to the second modification, the determination by the determination unit 173 can be made easier than, for example, determining whether or not the second image data and the subsequent image data are different from each other.
The source of the supply signal is not limited to the PC3, and may be, for example, a control device (not shown) that controls the PC3.

B3. Third Modification In the first embodiment, the change condition may be the sixth condition that the fourth condition is satisfied and the fifth condition is satisfied. The fourth condition is, for example, any one of the above-described first condition, the above-described second condition, and the above-described third condition. The fifth condition is, for example, that the projector 1 has not received from the user a maintenance instruction to maintain the image displayed in the first region R1.

As an example of the maintenance instruction, for example, as shown in FIG. 10, the object 7 for maintaining the image displayed in the first region R1 by prohibiting the change of the image displayed in the first region R1 is A state located in the first region R1 can be mentioned. The object 7 has, for example, a magnet, and is positioned on the projection surface 4 such as a whiteboard by the magnetic force of this magnet. Note that the object 7 may be attached to the projection surface 4 with a sticky substance.

The object 7 has, for example, a light source that emits infrared light. The light source of object 7 is larger than the first light source 23 of indicator 2 . Therefore, the determination unit 173 distinguishes between the object 7 and the indicator 2 based on the difference in the size of the bright spots indicated by the imaging data.

The determination unit 173 determines that the projector 1 has received a maintenance instruction from the user when the bright point corresponding to the object 7 is located in the first region R1. In this case, the fifth condition that the projector 1 has not received a maintenance instruction from the user is not satisfied.

The determination unit 173 determines that the projector 1 has not received a maintenance instruction from the user when the bright point corresponding to the object 7 is not located in the first region R1. In this case, the fifth condition is satisfied.

The shape of the object 7 is not limited to the shape shown in FIG. 10 and can be changed as appropriate. The shape of the object 7 may be, for example, a cylinder, square prism, triangular prism or hemisphere.
Also, on the surface of the object 7, a picture or characters of a fixture such as a pin for fixing the position of the paper medium may be written. In this case, the user intuitively recognizes the function of the object 7, that is, the function of maintaining the image displayed in the first region R1 by prohibiting the change of the image displayed in the first region R1. can. Therefore, the user can intuitively operate the object 7 .

FIG. 11 is a flowchart for explaining an example of the image switching operation when the change condition is the sixth condition. In FIG. 11, the same reference numerals are given to the same processes as those shown in FIG. Hereinafter, among the processes shown in FIG. 11, the processes different from the processes shown in FIG. 9 will be mainly described.

When the subsequent image data is received in step S203, in step S301 the determination unit 173 uses the imaging data and the calibration data to determine whether or not the bright point corresponding to the object 7 exists in the first region R1. do. If the bright point corresponding to the object 7 does not exist in the first region R1, the determination unit 173 determines that the maintenance instruction has not been received. The determination unit 173 determines that the maintenance instruction is received when the bright point corresponding to the object 7 exists in the first region R1.

If it is determined in step S301 that the maintenance instruction has not been received, step S204 is executed.

When it is determined in step S301 that a maintenance instruction has been received, in step S302 the display control unit 172 switches the image of only the second region R2 out of the first region R1 and the second region R2.

In step S302, for example, in a situation where the first image G1 is displayed in the first region R1 and the third image G3 is displayed in the second region R2, the display control unit 172 causes the image to be displayed in the first region R1. The image is maintained as the first image G1, and the image displayed in the second region R2 is changed from the third image G3 to the fourth image G4.

Further, in step S302, for example, in a situation where the first image G1 is displayed in the first region R1 and the second image G2 is displayed in the second region R2, the display control unit 172 causes the image to be displayed in the first region R1. The image to be displayed is maintained as the first image G1, and the image displayed in the second region R2 is changed from the second image G2 to the fourth image G4.

In this way, in step S302, the display control unit 172 displays the image in the second region R2 without changing the image displayed in the first region R1 when the subsequent image data is received while the maintenance instruction is being received. change the displayed image to the fourth image G4.
Therefore, in the third modified example, the user can intentionally prohibit the change of the image displayed in the first region R1.
Therefore, while changing the image displayed in the second area R2, it is possible to continue displaying information important in class, such as formulas, in the first area R1. Therefore, it becomes possible to proceed with the lesson effectively.
Further, by removing the object 7 from the first region R1, the user can update the image displayed in the first region R1 in synchronization with the update timing of the image displayed in the second region R2.

B4. Fourth Modification In the first embodiment and the first to third modifications, the third image data consists of the second image data and the written image data representing the written image 5 separately, for example, in a layered structure. may have.
In this case, it is possible to easily delete the written image 5 from the third image G3 displayed in the first area R1.

B5. Fifth Modification In the first embodiment and the first to fourth modifications, the projector 1 may output the third image data to an external device via a USB cable, for example.

B6. Sixth Modified Example In the first embodiment and the first to fifth modified examples, the fourth image G is not limited to the image indicated by the subsequent image data, but is the second image data or the second image data. It may also be an image represented by previously supplied image data, for example the first image data.
In this case, for example, an image before writing, such as the second image G2, and an image after writing, such as the third image G3, can be displayed side by side.

B7. Seventh Modification In the first embodiment and the first to sixth modifications, the pointer 2 may be an object that does not emit infrared light, such as a human finger. In this case, a light output device that planarly emits infrared light along the projection surface 4 is provided above the upper end 4 a of the projection surface 4 .
In the projector 1 , the camera 15 captures an image of the infrared light emitted from the light output device and reflected by the indicator 2 on the projection surface 4 .
The projector 1 identifies the operation position of the pointer 2 by analyzing the imaging data generated by the imaging by the camera 15 .

B8. Eighth Modification In the first embodiment and the first to seventh modifications, the operation unit 11 may be provided with a physical update button for inputting a request instruction. Also, the operation unit 11 may be provided with a physical maintenance instruction button for inputting a maintenance instruction.

B9. Ninth Modification In the first embodiment and the first to eighth modifications, the projector 1 operates in a writing mode in which writing is performed using the pointer 2 and a mouse mode in which the pointer 2 is used as a so-called mouse. and may have
The operation control unit 171 switches the operation mode by, for example, mode switching input to the operation unit 11 .
In this case the writing image 5 is created in writing mode and the operation to the update button 6 is performed in mouse mode.

B10. Tenth Modification In the first embodiment and the first to ninth modifications, the liquid crystal light valve 145 was used as an example of the light modulation device, but the light modulation device is not limited to the liquid crystal light valve and can be changed as appropriate. be. For example, the light modulation device may be configured using three reflective liquid crystal panels. Further, the optical modulation device may have a structure using one liquid crystal panel, a method using three digital mirror devices (DMD), a method using one digital mirror device, or the like. If only one liquid crystal panel or DMD is used as the light modulator, members corresponding to the color separating optical system and the color synthesizing optical system are unnecessary. In addition to the liquid crystal panel and DMD, a configuration capable of modulating the light emitted by the second light source 144 can be employed as the light modulation device.

B11. Eleventh Modification In the first embodiment and the first to tenth modifications, when an FPD is used instead of the projector 1 as the display device, the FPD as the display device is, for example, an electronic blackboard or an electronic conference system. The FPD used may be used.

DESCRIPTION OF SYMBOLS 1... Projector 2... Indicator 5... Written image 11... Operation part 12... Light receiving part 13a... Second communication part 13b... Image data receiving part 14... Projection part 15... Camera 16... Memory Section 17 ... Processing section 171 ... Operation control section 172 ... Display control section 173 ... Judgment section.

Claims (3)

A method of operation performed by a display device for displaying an image on a display surface, comprising:
displaying a first image in a first region of the display surface and displaying a second image in a second region different from the first region of the display surface;
displaying a third image generated by superimposing a writing image based on a writing operation on the second image on the second image in place of the second image in the second area;
Determining whether a condition for changing the image displayed in the first area and the image displayed in the second area is satisfied,
When determining that the condition is satisfied in the determination in a situation where the first image is displayed in the first area and the third image is displayed in the second area,
changing an image displayed in the first area from the first image to the third image, and changing an image displayed in the second area from the third image to a fourth image;
the condition is a sixth condition that the fourth condition is satisfied and the fifth condition is satisfied;
The fourth condition is
a condition that the display device receives subsequent image data provided to the display device temporally later than image data representing the second image;
a condition that the display device receives a supply signal indicating the supply of the subsequent image data;
either the condition that the image data representing the second image and the subsequent image data are different from each other;
The fifth condition is
The condition is that the display device has not received a maintenance instruction from the user to maintain the image displayed in the first area ,
The maintenance instructions are
an object having a light source that emits infrared light is attached to the first region;
determining that the display device has not received the maintenance instruction from the user when the bright point corresponding to the object is not located in the first region;
How it works. determining that the display device has received the maintenance instruction from the user when the bright point corresponding to the object is located in the first region;
An image displayed in the second area without changing the image displayed in the first area when the display apparatus receives the subsequent image data in a situation where the display apparatus receives the maintenance instruction from the user. to the fourth image;
A method of operation according to claim 1 . A display device for displaying an image on a display surface,
a display unit that displays a first image on a first area of the display surface and displays a second image on a second area that is different from the first area on the display surface;
a display control unit that controls the display unit;
a determination unit that determines whether a condition for changing the image displayed in the first area and the image displayed in the second area is satisfied;
including
The display control unit
an operation of displaying a third image generated by superimposing a writing image based on a writing operation on the second image on the second image in the second area instead of the second image; let it run,
When determining that the condition is satisfied in a situation where the first image is displayed in the first area and the third image is displayed in the second area,
The operation of changing the image displayed in the first area from the first image to the third image and changing the image displayed in the second area from the third image to the fourth image, Let the display unit execute
the condition is a sixth condition that the fourth condition is satisfied and the fifth condition is satisfied;
The fourth condition is
a condition that the display device receives subsequent image data provided to the display device temporally later than image data representing the second image;
a condition that the display device receives a supply signal indicating the supply of the subsequent image data;
either the condition that the image data representing the second image and the subsequent image data are different from each other;
The fifth condition is
The condition is that the display device has not received a maintenance instruction from the user to maintain the image displayed in the first area ,
The maintenance instructions are
an object having a light source that emits infrared light is attached to the first region;
The determination unit is
determining that the display device has not received the maintenance instruction from the user when the bright point corresponding to the object is not located in the first region;
display device.

Images