JP2024054506A - Display method, program, and projector - Google Patents

Abstract

A projection position, shape, and size of a projected image are adjusted using a single marker.
[Solution] The display method of the projector 100 includes, when a marker MK is placed at a first position P1, acquiring a first captured image PM1, determining a first coordinate axis AX1 parallel to the first side SD1 and a second coordinate axis AY1 parallel to the second side SD2, and storing the coordinates of the four corners of the first projected image PB1 as first information JP1; when a marker MK is placed at a second position P2, acquiring a second captured image PM2, determining a third coordinate axis AX2 parallel to the first side SD1 and a fourth coordinate axis AY2 parallel to the second side SD2, determining the coordinates of the four corners of the second projected image PB2 as second information JP2, and adjusting the projection position, shape, and size of the second projected image PB2 based on the first information JP1 and the second information JP2.
[Selected figure] Figure 7

Description

The present invention relates to a display method, a program, and a projector.

Patent document 1 describes a method of detecting multiple markers or a single marker arranged on a display surface, and determining the position and size of a projected image based on the positions indicated by those markers.

JP 2020-92337 A

In the technology described in Patent Document 1, when one marker is used to indicate the size of the projected image, it is necessary to prepare a marker that matches the size of the projected image, which makes it difficult to handle when displaying a large projected image. When multiple markers are used to indicate the size of the projected image, the markers must be arranged at intervals that indicate one side of the size of the projected image, so when a large projected image is displayed, the distance between the markers becomes long, making it difficult to arrange the markers.

A display method according to one aspect of the present disclosure includes, when a marker having a first side and a second side intersecting the first side is placed at a first position, acquiring a first captured image including the marker and a first projection image projected by a projector, identifying the first side and the second side of the marker, a first length of the first side, and a second length of the second side based on the first captured image, determining a first coordinate axis that is parallel to the first side and has the first length as a unit length, determining a second coordinate axis that is parallel to the second side and has the second length as a unit length, storing first information including coordinates of the four corners of the first projection image in coordinates defined by the first coordinate axis and the second coordinate axis, and storing the marker at a second position different from the first position. When the marker is placed, the method includes acquiring a second captured image including the marker and a second projected image projected by the projector that is different from the first projected image; determining the first side and the second side of the marker, a third length of the first side, and a fourth length of the second side based on the second captured image; determining a third coordinate axis that is parallel to the first side and has the third length as a unit length; determining a fourth coordinate axis that is parallel to the second side and has the fourth length as a unit length; determining second information including coordinates of the four corners of the second projected image in coordinates defined by the third coordinate axis and the fourth coordinate axis; and adjusting the projection position, shape, and size of the second projected image based on the first information and the second information.

A program according to another aspect of the present disclosure includes, when a marker having a first side and a second side intersecting the first side is placed at a first position, acquiring a first captured image including the marker and a first projection image projected by a projector, identifying the first side and the second side of the marker, a first length of the first side, and a second length of the second side based on the first captured image, determining a first coordinate axis that is parallel to the first side and has the first length as a unit length, determining a second coordinate axis that is parallel to the second side and has the second length as a unit length, storing first information including coordinates of the four corners of the first projection image in coordinates defined by the first coordinate axis and the second coordinate axis, and storing first information when the marker is placed at a second position different from the first position. The projector's processor executes the following operations: acquiring a second captured image including the marker and a second projected image projected by the projector, the second captured image being different from the first projected image; identifying the first side and the second side of the marker, a third length of the first side, and a fourth length of the second side based on the second captured image; determining a third coordinate axis that is parallel to the first side and has the third length as a unit length; determining a fourth coordinate axis that is parallel to the second side and has the fourth length as a unit length; determining second information including coordinates of the four corners of the second projected image in coordinates defined by the third coordinate axis and the fourth coordinate axis; and adjusting the projection position, shape, and size of the second projected image based on the first information and the second information.

A projector according to another aspect of the present disclosure includes a camera that captures a projection image, a memory, and at least one processor, and when a marker having a first side and a second side intersecting the first side is placed at a first position, the at least one processor acquires a first captured image including the marker and a first projection image projected by the projector, identifies the first side and the second side of the marker, a first length of the first side, and a second length of the second side based on the first captured image, determines a first coordinate axis that is parallel to the first side and has the first length as a unit length, determines a second coordinate axis that is parallel to the second side and has the second length as a unit length, and acquires first information including coordinates of four corners of the first projection image in coordinates defined by the first coordinate axis and the second coordinate axis. and when the marker is placed at a second position different from the first position, acquiring a second captured image including the marker and a second projected image projected by the projector that is different from the first projected image, determining the first side and the second side of the marker, a third length of the first side, and a fourth length of the second side based on the second captured image, determining a third coordinate axis that is parallel to the first side and has the third length as a unit length, determining a fourth coordinate axis that is parallel to the second side and has the fourth length as a unit length, determining second information including coordinates of the four corners of the second projected image in coordinates defined by the third coordinate axis and the fourth coordinate axis, and adjusting the projection position, shape, and size of the second projected image based on the first information and the second information.

FIG. 1 is a diagram showing an example of the configuration of a projector according to an embodiment. FIG. 2 is a diagram showing an example of the configuration of a control unit of the projector. FIG. 4 is an image diagram showing an example of a first captured image. FIG. 11 is an image diagram showing an example of a second captured image. 10A to 10C are image diagrams showing an example of a method for adjusting the projection position of the second projected image. 10A to 10C are image diagrams showing an example of a method for adjusting the shape and size of a second projected image. 10 is a flowchart showing an example of processing of a control unit.

The following describes the embodiment with reference to the drawings.

[1. Projector configuration]
Next, the configuration of a projector 100 will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of a projector 100 according to this embodiment.
1, the projector 100 includes a projection unit 110 and a drive unit 120 that drives the projection unit 110. The projection unit 110 forms an optical image and displays a projection image PB on a projection surface SC. The projection surface SC is, for example, a screen.
The projection unit 110 includes a light source unit 111, a light modulation device 112, and a projection optical system 113. The drive unit 120 includes a light source drive unit 121 and a light modulation device drive unit 122.

In this embodiment, a case will be described in which the projector 100 displays the projection image PB on a screen as the projection surface SC, but this is not limited to the case. The projector 100 may display the projection image PB on, for example, a wall surface. In other words, the projection surface SC may be, for example, a wall surface.

The light source unit 111 includes a lamp such as a halogen lamp, a xenon lamp, or an extra-high pressure mercury lamp, or a solid-state light source such as an LED (Light Emitting Diode) or a laser light source.
The light source unit 111 may also include a reflector and an auxiliary reflector that guide the light emitted by the light source to the light modulation device 112. Furthermore, the light source unit 111 may also include a lens group for improving the optical characteristics of the projected light, a polarizing plate, or a dimming element that reduces the amount of light emitted by the light source on the path leading to the light modulation device 112.
The light source driving unit 121 is connected to the internal bus 107 , and turns on and off the light source of the light source unit 111 according to instructions from a control unit 150 also connected to the internal bus 107 .

The light modulation device 112 includes, for example, three liquid crystal panels 115 corresponding to the three primary colors of R, G, and B. R indicates red, G indicates green, and B indicates blue. That is, the light modulation device 112 includes a liquid crystal panel 115 corresponding to R light, a liquid crystal panel 115 corresponding to G light, and a liquid crystal panel 115 corresponding to B light.
The light emitted by the light source unit 111 is separated into three color lights of RGB, and each color light is incident on a corresponding liquid crystal panel 115. Each of the three liquid crystal panels 115 is a transmissive liquid crystal panel, and generates image light PL by modulating the light passing through the liquid crystal panel 115. The image light PL modulated after passing through each liquid crystal panel 115 is combined by a combining optical system such as a cross dichroic prism, and is output to the projection optical system 113.

The light modulation device 112 is driven by a light modulation device driving section 122. The light modulation device driving section 122 is connected to an image processing section 145.
Image data corresponding to each of the primary colors R, G, and B is input to the light modulation device driving unit 122 from the image processing unit 145. The light modulation device driving unit 122 converts the input image data into a data signal suitable for the operation of the liquid crystal panel 115. The light modulation device driving unit 122 applies a voltage to each pixel of each liquid crystal panel 115 based on the converted data signal, and draws an image corresponding to the projection image PB on each liquid crystal panel 115.
Instead of the liquid crystal panel 115, the light modulation device 112 may include a reflective liquid crystal panel, a liquid crystal on silicon (LCOS) liquid crystal panel, or a light modulation panel other than a liquid crystal panel, such as a digital micromirror device (DMD).

The projection optical system 113 includes lenses, mirrors, etc. that form an image of the incident image light PL on the projection surface SC. The projection optical system 113 may also include a zoom mechanism that enlarges or reduces the image projected onto the projection surface SC, a focus adjustment mechanism that adjusts the focus, etc.
The projection optical system 113 also includes a projection lens 113A that forms an image of the incident image light PL on a projection surface SC.

The projector 100 further includes an operation unit 131, a remote control receiver 133, an input interface 135, a storage unit 137, a communication interface 141, a frame memory 143, an image processing unit 145, a control unit 150, and a camera 170. The input interface 135, the storage unit 137, the communication interface 141, the image processing unit 145, the control unit 150, and the camera 170 are connected to each other via an internal bus 107 so as to be able to communicate data with each other.

The operation unit 131 has various buttons and switches provided on the surface of the housing of the projector 100, generates operation signals corresponding to the operation of these buttons and switches, and outputs them to the input interface 135. The input interface 135 outputs the operation signals input from the operation unit 131 to the control unit 150.

The remote control light receiving unit 133 receives an infrared signal transmitted from the remote control 5, and decodes the received infrared signal to generate an operation signal. The remote control light receiving unit 133 outputs the generated operation signal to the input interface 135. The input interface 135 outputs the operation signal input from the remote control light receiving unit 133 to the control unit 150.

The storage unit 137 is, for example, a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The storage unit 137 stores programs executed by the control unit 150, data processed by the control unit 150, image data, etc.

The communication interface 141 includes a connector and an interface circuit, and is communicatively connected to the image supply device 200. In this embodiment, the communication interface 141 is an interface for communicating with the image supply device 200 in accordance with, for example, the Ethernet (registered trademark) standard. The communication interface 141 receives an image corresponding to the projection image PB from the image supply device 200.
In this embodiment, a case will be described in which the projector 100 projects an image received from the image supply device 200 onto the projection surface SC as the projection image PB, but is not limited to this. The projector 100 may project, for example, an image stored in advance in the storage unit 137 or the memory 150B onto the projection surface SC as the projection image PB.

The image supply device 200 is configured, for example, by a personal computer. The image supply device 200 includes, for example, a DVD (Digital Versatile Disc) player. The DVD player plays a DVD and supplies the image stored on the DVD to the projector 100.

In this embodiment, a case will be described in which the image supply device 200 is a personal computer, but the present invention is not limited to this. It is sufficient that the image supply device 200 supplies an image to the projector 100. The image supply device 200 may be, for example, a DVD player. The image supply device 200 may also be, for example, a tablet terminal or a smartphone.

The camera 170 captures the projection image PB displayed on the projection surface SC in accordance with an instruction from the control unit 150, and generates a captured image PM. The captured image PM includes a marker MK and a projection image PB. The captured image PM also includes a first captured image PM1 and a second captured image PM2. For example, when the marker MK is placed at a first position P1 on the first projection surface SC1, the camera 170 generates the first captured image PM1 in accordance with an instruction from the control unit 150. Then, the camera 170 outputs the first captured image PM1 to the control unit 150. Also, for example, when the marker MK is placed at a second position P2 on the second projection surface SC2, the camera 170 generates the second captured image PM2 in accordance with an instruction from the control unit 150. Then, the camera 170 outputs the second captured image PM2 to the control unit 150.
The second position P2 is a position different from the first position P1. The second projection surface SC2 may be the same screen as the first projection surface SC1, or may be a screen different from the first projection surface SC1.
The first position P1, the marker MK, and the first captured image PM1 will be further described with reference to FIG.
The second position P2 and the second captured image PM2 will be further described with reference to FIG.

The control unit 150 includes a processor 150A and a memory 150B.
The memory 150B is a storage device that non-volatilely stores programs and data executed by the processor 150A. The memory 150B is configured with a semiconductor storage element such as a magnetic storage device or a flash ROM (Read Only Memory), or other types of non-volatile storage devices. The memory 150B may also include a RAM (Random Access Memory) that configures a work area for the processor 150A. The memory 150B may also include a non-volatile storage device such as an HDD or SSD.
The memory 150B stores data processed by the control unit 150 and the control program PG executed by the processor 150A.

The processor 150A may be configured as a single processor, or may be configured with multiple processors functioning as the processor 150A. The processor 150A executes the control program PG to control each part of the projector 100. For example, the processor 150A outputs to the image processing unit 145 an instruction to execute image processing corresponding to an operation received from the operation unit 131 or the remote control 5, and parameters used in this image processing. The parameters include, for example, geometric correction parameters for correcting geometric distortion of the projection image PB projected onto the projection surface SC. The processor 150A also controls the light source driving unit 121 to control the turning on and off of the light source unit 111.

Each of the image processing unit 145 and the control unit 150 can be configured, for example, by an integrated circuit. Integrated circuits include LSIs, ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). PLDs include, for example, FPGAs (Field-Programmable Gate Arrays). Furthermore, an analog circuit may be included as part of the configuration of an integrated circuit, or a combination of a processor and an integrated circuit may be used. The combination of a processor and an integrated circuit is called a microcontroller (MCU), SoC (System-on-a-chip), system LSI, chipset, etc.

The image processing unit 145 expands the image data stored in the memory 150B or the storage unit 137 into the frame memory 143. The frame memory 143 has a plurality of banks. Each bank has a storage capacity capable of writing one frame's worth of image data. The frame memory 143 is composed of, for example, a Synchronous Dynamic Random Access Memory (SDRAM).

The image processing unit 145 performs image processing such as resolution conversion or resizing, distortion correction, shape correction, digital zooming, and adjustment of the color tone and brightness of the image on the image data expanded in the frame memory 143 in accordance with instructions from the control unit 150.

The processor 150A of the control unit 150 executes the control program PG to perform, for example, the following processing.
The processor 150A causes the communication interface 141 to receive an image corresponding to the projection image PB from the image supply device 200. Then, the processor 150A causes the liquid crystal panel 115 to display the projection image PB via the light modulation device drive unit 122. Furthermore, the processor 150A causes the projection unit 110 to project image light PL representing the projection image PB onto the projection surface SC via the drive unit 120. In this way, the projector 100 displays the projection image PB on the projection surface SC.

[2. Configuration of the control unit]
Next, the configuration of the control unit 150 of the projector 100 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the control unit 150 of the projector 100. As shown in Fig. 2, the control unit 150 includes an acquisition unit 151, an identification unit 152, a determination unit 153, a processing unit 154, a notification unit 155, an adjustment unit 156, an image storage unit 157, and an information storage unit 158.
Specifically, the processor 150A of the control unit 150 executes the control program PG stored in the memory 150B, thereby functioning as an acquisition unit 151, an identification unit 152, a determination unit 153, a processing unit 154, a notification unit 155, and an adjustment unit 156. The processor 150A of the control unit 150 executes the control program PG stored in the memory 150B, thereby causing the memory 150B to function as an image storage unit 157 and an information storage unit 158.
The control program PG corresponds to an example of a "program".
The processor 150A corresponds to an example of "at least one processor."

The image storage unit 157 stores the first captured image PM1 and the second captured image PM2. The first captured image PM1 is generated by the camera 170 and acquired by the acquisition unit 151 when the marker MK is placed at the first position P1 on the first projection surface SC1. The first captured image PM1 is stored in the image storage unit 157 by the acquisition unit 151.
When the marker MK is placed at the second position P2 on the second projection surface SC2, the second captured image PM2 is generated by the camera 170 and acquired by the acquisition unit 151. In addition, the second captured image PM2 is stored in the image storage unit 157 by the acquisition unit 151.

The information storage unit 158 stores the first information JP1 and the second information JP2.
The first information JP1 includes coordinates of the four corners of the first projection image PB1 and coordinates of a center CT1 of the first projection image PB1. The first projection image PB1 is projected by the projector 100 onto the first projection surface SC1 when the marker MK is placed at a first position P1 on the first projection surface SC1. The second information JP2 includes coordinates of the four corners of the second projection image PB2 and coordinates of a center CT2 of the second projection image PB2. The second projection image PB2 is projected by the projector 100 onto the second projection surface SC2 when the marker MK is placed at a second position P2 on the second projection surface SC2.
The first information JP1 is generated by the processing unit 154 when the marker MK is placed at a first position P1 on the first projection surface SC1. The first information JP1 is stored in the information storage unit 158 by the processing unit 154. The second information JP2 is generated by the processing unit 154 when the marker MK is placed at a second position P2 on the second projection surface SC2. The second information JP2 is stored in the information storage unit 158 by the processing unit 154.

When the marker MK is placed at a first position P1 on the first projection surface SC1, the acquisition unit 151 acquires a first captured image PM1 from the camera 170. The acquisition unit 151 also stores the first captured image PM1 in the image storage unit 157. When the marker MK is placed at a second position P2 on the second projection surface SC2, the acquisition unit 151 acquires a second captured image PM2 from the camera 170. The acquisition unit 151 also stores the second captured image PM2 in the image storage unit 157.

The determination unit 152 determines a first direction DX1, a second direction DY1, a first length LX1, and a second length LY1 based on the first captured image PM1. The first direction DX1 is the direction of a first side SD1 of the marker MK in the first captured image PM1. The second direction DY1 is the direction of a second side SD2 of the marker MK in the first captured image PM1. The first length LX1 is the length of the first side SD1 of the marker MK in the first captured image PM1. The second length LY1 is the length of the second side SD2 of the marker MK in the first captured image PM1.
The first length LX1 corresponds to an example of a "first length."
The second length LY1 corresponds to an example of a "second length."
The first direction DX1, the second direction DY1, the first length LX1, and the second length LY1 will be further described with reference to FIG.
The determination unit 152 also determines a first direction DX2, a second direction DY2, a third length LX2, and a fourth length LY2 based on the second captured image PM2. The first direction DX2 is the direction of the first side SD1 of the marker MK in the second captured image PM2. The second direction DY2 is the direction of the second side SD2 of the marker MK in the second captured image PM2. The third length LX2 is the size of the first side SD1 of the marker MK in the second captured image PM2. The fourth length LY2 is the size of the second side SD2 of the marker MK in the second captured image PM2.
The third length LX2 corresponds to an example of a "third length."
The fourth length LY2 corresponds to an example of a "fourth length."
The first direction DX2, the second direction DY2, the third length LX2, and the fourth length LY2 will be further described with reference to FIG.

The determination unit 153 determines a first coordinate axis AX1 that is parallel to the first direction DX1 and has a unit length equal to the first length LX1, determines a second coordinate axis AY1 that is parallel to the second direction DY1 and has a unit length equal to the second length LY1, determines a third coordinate axis AX2 that is parallel to the first direction DX2 and has a unit length equal to the third length LX2, and determines a fourth coordinate axis AY2 that is parallel to the second direction DY2 and has a unit length equal to the fourth length LY2.
The first coordinate axis AX1 and the second coordinate axis AY1 will be further described with reference to FIG.
The third coordinate axis AX2 and the fourth coordinate axis AY2 will be further described with reference to FIG.

The processing unit 154 obtains coordinates defined by the first coordinate axis AX1 and the second coordinate axis AY1 for each of the four corners of the first projection image PB1, and stores the coordinates of the four corners of the first projection image PB1 as first information JP1 in the information storage unit 158. The four corners of the first projection image PB1 correspond to the four vertices of the first projection image PB1. The processing unit 154 also obtains coordinates defined by the first coordinate axis AX1 and the second coordinate axis AY1 for the center CT1 of the first projection image PB1, and stores the coordinates of the center CT1 of the first projection image PB1 as first information JP1 in the information storage unit 158.
The processing unit 154 also obtains coordinates defined by the third coordinate axis AX2 and the fourth coordinate axis AY2 for each of the four corners of the second projection image PB2, and stores the coordinates of the four corners of the second projection image PB2 as second information JP2 in the information storage unit 158. The four corners of the second projection image PB2 correspond to the four vertices of the second projection image PB2. The processing unit 154 also obtains coordinates defined by the third coordinate axis AX2 and the fourth coordinate axis AY2 for the center CT2 of the second projection image PB2, and stores the coordinates of the center CT2 of the second projection image PB2 as second information JP2 in the information storage unit 158.
The four vertices and center CT1 of the first projected image PB1 will be further described with reference to FIG.
The four vertices and center CT2 of the second projected image PB2 will be further described with reference to FIG.

When the second captured image PM2 does not include at least one of the four corners of the second projected image PB2, the notification unit 155 notifies that the second captured image PM2 is inappropriate. The notification unit 155 displays, for example, on the LCD (Liquid Crystal Display) of the remote control 5 that the second captured image PM2 is inappropriate. The notification unit 155 also displays, for example, on the projector 100 as an OSD (On Screen Display) that the second captured image PM2 is inappropriate.

The adjustment unit 156 adjusts the projection position, shape, and size of the second projection image PB2 based on the first information JP1 and the second information JP2. For example, the adjustment unit 156 adjusts the projection position of the second projection image PB2 so that the coordinates of the center CT2 of the second projection image PB2 match the coordinates of the center CT1 of the first projection image PB1. For example, the adjustment unit 156 adjusts the projection position of the second projection image PB2 by adjusting the direction of the projection light PL of the projection lens 113A.
The adjustment unit 156 also adjusts the shape and size of the second projection image PB2 so that the coordinates of the four corners of the second projection image PB2 match the coordinates of the four corners of the first projection image PB1. The adjustment unit 156 adjusts the shape of the second projection image PB2 by, for example, adjusting the shape of the second projection image PB2 drawn on the liquid crystal panel 115. The adjustment unit 156 also adjusts the size of the second projection image PB2 by, for example, adjusting the zoom mechanism of the projection optical system 113.

In this embodiment, the adjustment unit 156 adjusts the shape and size of the second projection image PB2 so that the coordinates of the four corners of the second projection image PB2 match the coordinates of the four corners of the first projection image PB1, but is not limited to this. The adjustment unit 156 may adjust the projection position, shape, and size of the second projection image PB2 based on the first information JP1 and the second information JP2.
The adjustment unit 156 may adjust the size of the second projection image PB2 so that the coordinates of the four corners of the second projection image PB2 match a predetermined multiple of the coordinates of the four corners of the first projection image PB1. The predetermined multiple is, for example, 1/2. In this case, the size of the second projection image PB2 based on the third length LX2 of the marker MK included in the second captured image PM2 is adjusted to 1/2 the size of the first projection image PB1 based on the first length LX1 of the marker MK included in the first captured image PM1.

3. Specific Examples of the First Captured Image
Next, a specific example of the first captured image PM1 will be described with reference to Fig. 3. Fig. 3 is an image diagram showing the first captured image PM1.
3, the first captured image PM1 includes an image of the marker MK and a first projected image PB1. An imaging area AR1 indicated by a dashed line is an imaging area of the camera 170 when generating the first captured image PM1. Note that in this embodiment, a case will be described in which the projector 100 is disposed in the normal direction of the imaging area AR1 on the first projection surface SC1.
In this embodiment, the image of the marker MK may be simply referred to as a "marker MK." The marker MK is a plate-like member formed in an L-shape. The marker MK is made of, for example, resin, aluminum, etc. The marker MK is disposed at a first position P1. The marker MK is fixed to the first position P1 on the first projection surface SC1 by, for example, a magnet. The marker MK may also be fixed to the first position P1 on the first projection surface SC1 by, for example, an adhesive.

The marker MK has a first side SD1 and a second side SD2. The first side SD1 and the second side SD2 intersect perpendicularly at an intersection TS1. That is, one end point of the first side SD1, here the left end point, and one end point of the second side SD2, here the upper end point, are located at the intersection TS1. In other words, the first side SD1 is a side that extends rightward from the intersection TS1, and the second side SD2 is a side that extends downward from the intersection TS1.
The first length LX1 is the length of the first side SD1 in the first captured image PM1, and the second length LY1 is the length of the second side SD2 in the first captured image PM1. In this embodiment, the first length LX1 is greater than the second length LY1.

The first coordinate axis AX1 is parallel to the first direction DX1 and has a unit length of the first length LX1. The first coordinate axis AX1 has the intersection point TS1 as the origin RG1 and extends in a direction parallel to the first direction DX1. The coordinate X1 defined by the first coordinate axis AX1 indicates how many times the distance from the origin RG1 is the first length LX1.
The second coordinate axis AY1 is parallel to the second direction DY1 and has the second length LY1 as a unit length. The second coordinate axis AY1 has the intersection point TS1 as the origin RG1 and extends in a direction parallel to the second direction DY1. The coordinate Y1 defined by the second coordinate axis AY1 indicates how many times the distance from the origin RG1 is the second length LY1.
The first coordinate axis AX1 and the second coordinate axis AY1 constitute a first coordinate system CD1. The processing unit 154 obtains coordinates in the first coordinate system CD1 for each of the four corners of the first projection image PB1 and the center CT1 of the first projection image PB1. The four corners of the first projection image PB1, i.e., the four vertices of the first projection image PB1, are vertices VT11, VT12, VT13, and VT14. The processing unit 154 obtains coordinates of each of the vertices VT11, VT12, VT13, and VT14 and the center CT1 in the first coordinate system CD1.

3, the first projected image PB1 is a rectangle with a long side SD3 parallel to the first coordinate axis AX1 and a short side SD4 parallel to the second coordinate axis AY1. The long side SD3 is a side connecting the vertices VT11 and VT12, and a side connecting the vertices VT13 and VT14. The short side SD4 is a side connecting the vertices VT11 and VT14, and a side connecting the vertices VT12 and VT13. The length of the long side SD3 is, for example, a first predetermined number multiple of the first length LX1, and the length of the short side SD4 is a second predetermined number multiple of the second length LY1. For example, each of the first predetermined number multiple and the second predetermined number multiple is "5 times".
The center CT1 coincides with the intersection of two diagonals in the first projection image PB1. The two diagonals are the diagonal line connecting the vertices VT11 and VT13, and the diagonal line connecting the vertices VT12 and VT14.
The long side SD3 corresponds to an example of a "third side."
The short side SD4 corresponds to an example of a "fourth side."
The rectangle corresponds to an example of a "rectangle."

As described with reference to FIG. 3, the first coordinate axis AX1 is a coordinate axis parallel to the first direction DX1 and has a unit length of the first length LX1, and the second coordinate axis AY1 is a coordinate axis parallel to the second direction DY1 and has a unit length of the second length LY1. In addition, the processing unit 154 determines the coordinates in the first coordinate system CD1 consisting of the first coordinate axis AX1 and the second coordinate axis AY1 for each of the four corners of the first projection image PB1 and the center CT1 of the first projection image PB1. Therefore, the projection position, shape, and size of the first projection image PB1 included in the first captured image PM1 can be determined by the image of the marker MK included in the first captured image PM1.

4. Specific Examples of Second Captured Image
Next, a specific example of the second captured image PM2 will be described with reference to Fig. 4. Fig. 4 is an image diagram showing the second captured image PM2.
4, the second captured image PM2 includes an image of the marker MK and a second projected image PB2. An imaging area AR2 indicated by a dashed line is an imaging area of the camera 170 when generating the second captured image PM2. Note that in this embodiment, a case will be described in which the projector 100 is disposed in the normal direction of the imaging area AR2 on the second projection surface SC2.
The marker MK is disposed at the second position P2. The marker MK is fixed to the second position P2 on the second projection surface SC2 by, for example, a magnet. The marker MK may also be fixed to the second position P2 on the second projection surface SC2 by, for example, an adhesive.

The marker MK has a first side SD1 and a second side SD2. The first side SD1 and the second side SD2 intersect perpendicularly at an intersection TS1. That is, one end point of the first side SD1, here the left end point, and one end point of the second side SD2, here the upper end point, are located at the intersection TS1. In other words, the first side SD1 is a side that extends rightward from the intersection TS1, and the second side SD2 is a side that extends downward from the intersection TS1.
The third length LX2 is the length of the first side SD1 in the second captured image PM2, and the fourth length LY2 is the length of the second side SD2 in the second captured image PM2. The third length LX2 is greater than the fourth length LY2.
Also, the third length LX2 shown in Fig. 4 is smaller than the first length LX1 shown in Fig. 3, and the fourth length LY2 shown in Fig. 4 is smaller than the second length LY1 shown in Fig. 3. Also, since the marker MK shown in Fig. 4 and the marker MK shown in Fig. 3 are the same marker MK, the ratio of the third length LX2 to the fourth length LY2 shown in Fig. 4 matches the ratio of the first length LX1 to the second length LY1 shown in Fig. 3. In other words, the image of the marker MK in the second captured image PM2 is similar to the image of the marker MK in the first captured image PM1 shown in Fig. 3.

The third coordinate axis AX2 is parallel to the first direction DX2 and has a unit length of the third length LX2. The third coordinate axis AX2 has an origin RG2 at the intersection TS2 and extends in a direction parallel to the first direction DX2. The coordinate X2 defined by the third coordinate axis AX2 indicates how many times the distance from the origin RG2 is the third length LX2.
The fourth coordinate axis AY2 is parallel to the second direction DY2 and has a fourth length LY2 as a unit length. The fourth coordinate axis AY2 has the intersection point TS2 as the origin RG2 and extends in a direction parallel to the second direction DY2. The coordinate Y2 defined by the fourth coordinate axis AY2 indicates how many times the distance from the origin RG2 is relative to the fourth length LY2.
The third coordinate axis AX2 and the fourth coordinate axis AY2 constitute a second coordinate system CD2. The processing unit 154 obtains the coordinates of each of the four corners of the second projection image PB2 and the center CT2 of the second projection image PB2 in the second coordinate system CD2. The four corners of the second projection image PB2, i.e., the four vertices of the second projection image PB2, are vertices VT21, VT22, VT23, and VT24. The processing unit 154 obtains the coordinates of each of the vertices VT21, VT22, VT23, and VT24 and the center CT2 in the second coordinate system CD2.

In FIG. 4, the second projected image PB2 is a rectangle.
The center CT2 coincides with the intersection of two diagonals in the second projection image PB2. The two diagonals are the diagonal line connecting the vertices VT21 and VT23, and the diagonal line connecting the vertices VT22 and VT24.

As described with reference to FIG. 4, the third coordinate axis AX2 is a coordinate axis parallel to the first direction DX2 and has a unit length of the third length LX2, and the fourth coordinate axis AY2 is a coordinate axis parallel to the second direction DY2 and has a unit length of the fourth length LY2. In addition, the processing unit 154 determines the coordinates in the second coordinate system CD2 consisting of the third coordinate axis AX2 and the fourth coordinate axis AY2 for each of the four corners of the second projection image PB2 and the center CT2 of the second projection image PB2. Therefore, the projection position, shape, and size of the second projection image PB2 included in the second captured image PM2 can be specified by the image of the marker MK included in the second captured image PM2.

5. Specific Example of Method for Adjusting Second Projected Image
Next, a method for adjusting the second projection image PB2 will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is an image diagram showing an example of a method for adjusting the projection position of the second projection image PB2. Fig. 6 is an image diagram showing an example of a method for adjusting the shape and size of the second projection image PB2.
First, a method for adjusting the projection position of the second projection image PB2 will be described with reference to Fig. 5. The second captured image PM2A shown in Fig. 5 is a captured image PM obtained by capturing the second projection image PB2A by the camera 170 while maintaining the positions of the marker MK and the projector 100 in the same state as when the second captured image PM2 shown in Fig. 4 was captured. The second projection image PB2A is the projection image PB after the projection position of the second projection image PB2 is adjusted.

As shown by arrow V1 in Figure 5, the adjustment unit 156 moves the second projection image PB2 so that the coordinates in the second coordinate system CD2 of the center CT2 of the second projection image PB2 shown in Figure 4 coincide with the coordinates in the first coordinate system CD1 of the center CT1 of the first projection image PB1 shown in Figure 3.
The center CT2 is the center of the second projected image PB2. The center CT2A is the center of the second projected image PB2A. The coordinates of the center CT2A in the second coordinate system CD2 match the coordinates of the center CT1 in the first coordinate system CD1 shown in FIG.
Vertex VT21A, vertex VT22A, vertex VT23A, and vertex VT24A are the four vertices of the second projection image PB2A. Vertex VT21A corresponds to vertex VT21 of the second projection image PB2. Vertex VT22A corresponds to vertex VT22 of the second projection image PB2. Vertex VT23A corresponds to vertex VT23 of the second projection image PB2. Vertex VT24A corresponds to vertex VT24 of the second projection image PB2.

As described with reference to FIG. 5, the adjustment unit 156 adjusts the projection position of the second projection image PB2 so that the coordinates of the center CT2 of the second projection image PB2 match the coordinates of the center CT1 of the first projection image PB1. Therefore, the projection position of the second projection image PB2 can be appropriately adjusted.

Next, a method for adjusting the shape and size of the second projection image PB2 will be described with reference to Fig. 6. Note that, as described with reference to Fig. 5, the adjustment unit 156 translates the second projection image PB2, and then performs the adjustment process for the shape and size of the second projection image PB2A.
The second captured image PM2B shown in Fig. 6 is a captured image PM obtained by capturing the second projected image PB2B by the camera 170 while maintaining the positions of the markers MK and the projector 100 in the same states as when the second captured image PM2 shown in Fig. 4 was captured. The second projected image PB2B is a projected image PB after adjusting the shape and size of the second projected image PB2A.

Vertex VT21B, vertex VT22B, vertex VT23B, and vertex VT24B are the four vertices of the second projected image PB2B. Center CT2B is the center of the second projected image PB2B. The coordinates of center CT2B in the second coordinate system CD2 match the coordinates of center CT2A shown in FIG. 5 in the second coordinate system CD2, and match the coordinates of center CT1 shown in FIG. 3 in the first coordinate system CD1.

As shown by the arrow V21 in Fig. 6, the adjustment unit 156 moves the position of the vertex VT21A. That is, the position of the vertex VT21A is moved so that the coordinates in the second coordinate system CD2 of the vertex VT21A of the second projection image PB2A shown in Fig. 5 match the coordinates in the first coordinate system CD1 of the vertex VT11 of the first projection image PB1 shown in Fig. 3. The vertex VT21B indicates the position after the vertex VT21A is moved. The coordinates in the second coordinate system CD2 of the vertex VT21B match the coordinates in the first coordinate system CD1 of the vertex VT11 shown in Fig. 3.
Also, as indicated by the arrow V22 in Fig. 6, the adjustment unit 156 moves the position of the vertex VT22A. That is, the position of the vertex VT22A is moved so that the coordinates in the second coordinate system CD2 of the vertex VT22A of the second projection image PB2A shown in Fig. 5 match the coordinates in the first coordinate system CD1 of the vertex VT12 of the first projection image PB1 shown in Fig. 3. The vertex VT22B indicates the position after the vertex VT22A is moved. The coordinates in the second coordinate system CD2 of the vertex VT22B match the coordinates in the first coordinate system CD1 of the vertex VT12 shown in Fig. 3.

Also, as indicated by the arrow V23 in Fig. 6, the adjustment unit 156 moves the position of the vertex VT23A. That is, the position of the vertex VT23A is moved so that the coordinates in the second coordinate system CD2 of the vertex VT23A of the second projection image PB2A shown in Fig. 5 match the coordinates in the first coordinate system CD1 of the vertex VT13 of the first projection image PB1 shown in Fig. 3. The vertex VT23B indicates the position after the vertex VT23A is moved. The coordinates in the second coordinate system CD2 of the vertex VT23B match the coordinates in the first coordinate system CD1 of the vertex VT13 shown in Fig. 3.
Also, as indicated by the arrow V24 in Fig. 6, the adjustment unit 156 moves the position of the vertex VT24A. That is, the position of the vertex VT24A is moved so that the coordinates in the second coordinate system CD2 of the vertex VT24A of the second projection image PB2A shown in Fig. 5 match the coordinates in the first coordinate system CD1 of the vertex VT14 of the first projection image PB1 shown in Fig. 3. The vertex VT24B indicates the position after the vertex VT24A is moved. The coordinates in the second coordinate system CD2 of the vertex VT24B match the coordinates in the first coordinate system CD1 of the vertex VT14 shown in Fig. 3.

6, the shape and size of the second projection image PB2 are adjusted so that the coordinates of the four corners of the second projection image PB2 match the coordinates of the four corners of the first projection image PB1, so that the shape and size of the second projection image PB2 can be appropriately adjusted.
In this embodiment, the second projection image PB2 is a rectangle with its long side parallel to the third coordinate axis AX2 and its short side parallel to the fourth coordinate axis AY2. The shape of the second projection image PB2 is similar to the shape of the first projection image PB1 shown in Fig. 3. The similarity ratio of the second projection image PB2 to the first projection image PB1 is the same as the similarity ratio of the markers MK in the second captured image PM2 to the markers MK in the first captured image PM1.

[6. Processing of the control unit]
Next, an example of the process of the control unit 150 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the process of the control unit 150.
7, in step S101, the control unit 150 determines whether or not the marker MK is placed at a first position P1. The control unit 150 determines whether or not the marker MK is placed at the first position P1 based on, for example, a user operation from the operation unit 131.
When the control unit 150 determines that the marker MK is not located at the first position P1 (step S101; NO), the process goes to a standby state. When the control unit 150 determines that the marker MK is located at the first position P1 (step S101; YES), the process goes to step S103.
Then, in step S103, the acquisition unit 151 acquires the first captured image PM1 from the camera 170. The acquisition unit 151, for example, causes the camera 170 to generate the first captured image PM1 and acquires the generated first captured image PM1 from the camera 170.
Next, in step S105, the determination unit 152 determines a first direction DX1 and a first length LX1 based on the first captured image PM1. The first direction DX1 is the direction of the first side SD1 of the marker MK in the first captured image PM1. The first length LX1 is the length of the first side SD1 of the marker MK in the first captured image PM1.
Next, in step S107, the determination unit 152 determines a second direction DY1 and a second length LY1 based on the first captured image PM1. The second direction DY1 is the direction of the second side SD2 of the marker MK in the first captured image PM1. The second length LY1 is the length of the second side SD2 of the marker MK in the first captured image PM1.

Next, in step S109, the determination unit 153 determines a first coordinate axis AX1 that is parallel to the first direction DX1 and has the first length LX1 as its unit length, and determines a second coordinate axis AY1 that is parallel to the second direction DY1 and has the second length LY1 as its unit length.
Next, in step S111, the processing unit 154 obtains coordinates defined by the first coordinate axis AX1 and the second coordinate axis AY1 for each of the four corners of the first projection image PB1, and stores the coordinates of the four corners of the first projection image PB1 as first information JP1 in the information storage unit 158. The processing unit 154 also obtains coordinates defined by the first coordinate axis AX1 and the second coordinate axis AY1 for the center CT1 of the first projection image PB1, and stores the coordinates of the center CT1 of the first projection image PB1 as first information JP1 in the information storage unit 158.

Next, in step S113, the control unit 150 determines whether or not the marker MK is located at the second position P2. The control unit 150 determines whether or not the marker MK is located at the second position P2 based on, for example, a user operation from the operation unit 131.
If the control unit 150 determines that the marker MK is not located at the second position P2 (step S113; NO), the process goes to a standby state. If the control unit 150 determines that the marker MK is located at the second position P2 (step S113; YES), the process proceeds to step S115.
Then, in step S115, the acquisition unit 151 acquires the second captured image PM2 from the camera 170. The acquisition unit 151, for example, causes the camera 170 to generate the second captured image PM2 and acquires the generated second captured image PM2 from the camera 170.
Next, in step S117, the determination unit 152 determines a first direction DX2 and a third length LX2 based on the second captured image PM2. The first direction DX2 is the direction of the first side SD1 of the marker MK in the second captured image PM2. The third length LX2 is the length of the first side SD1 of the marker MK in the second captured image PM2.
Next, in step S119, the determination unit 152 determines a second direction DY2 and a fourth length LY2 based on the second captured image PM2. The second direction DY2 is the direction of the second side SD2 of the marker MK in the second captured image PM2. The fourth length LY2 is the length of the second side SD2 of the marker MK in the second captured image PM2.

Next, in step S121, the determination unit 153 determines a third coordinate axis AX2 that is parallel to the first direction DX2 and has the third length LX2 as its unit length, and determines a fourth coordinate axis AY2 that is parallel to the second direction DY2 and has the fourth length LY2 as its unit length.
Next, in step S123, the processing unit 154 obtains coordinates defined by the third coordinate axis AX2 and the fourth coordinate axis AY2 for each of the four corners of the second projection image PB2, and stores the coordinates of the four corners of the second projection image PB2 as second information JP2 in the information storage unit 158. The processing unit 154 also obtains coordinates defined by the third coordinate axis AX2 and the fourth coordinate axis AY2 for the center CT2 of the second projection image PB2, and stores the coordinates of the center CT2 of the second projection image PB2 as second information JP2 in the information storage unit 158.

Next, in step S125, the adjustment unit 156 adjusts the projection position of the second projection image PB2 so that the coordinates of the center CT2 of the second projection image PB2 match the coordinates of the center CT1 of the first projection image PB1.
Next, in step S127, the adjustment unit 156 adjusts the shape and size of the second projection image PB2 so that the coordinates of the four corners of the second projection image PB2 match the coordinates of the four corners of the first projection image PB1. Then, the process ends.

Step S103 corresponds to an example of "obtaining a first captured image". Steps S105 and S107 correspond to an example of "determining a size based on a first captured image". Step S109 corresponds to an example of "determining a first coordinate axis" and "determining a second coordinate axis". Step S111 corresponds to an example of "storing first information including the coordinates of the four corners of a first projected image". Step S115 corresponds to an example of "obtaining a second captured image". Steps S117 and S119 correspond to an example of "determining a size based on a second captured image". Step S121 corresponds to an example of "determining a third coordinate axis" and "determining a fourth coordinate axis". Step S123 corresponds to an example of "determining second information including the coordinates of the four corners of a second projected image". Steps S125 and S127 correspond to an example of "adjusting a projection position, shape, and size of a second projected image".

[7. This embodiment and its effects]
As described above with reference to FIGS. 1 to 7, the display method according to the present embodiment includes the steps of: acquiring a first captured image PM1 including the marker MK and a first projected image PB1 projected by the projector 100 when a marker MK having a first side SD1 and a second side SD2 intersecting with the first side SD1 is placed at a first position P1; and acquiring a first captured image PM1 including the marker MK and a first projected image PB1 projected by the projector 100 based on the first captured image PM1. specifying a first length LX1 and a second length LY1 of a second side SD2; determining a first coordinate axis AX1 that is parallel to the first side SD1 and has the first length LX1 as a unit length; determining a second coordinate axis AY1 that is parallel to the second side SD2 and has the second length LY1 as a unit length; storing first information JP1 including coordinates of four corners of a first projection image PB1 in coordinates defined by the first coordinate axis AX1 and the second coordinate axis AY1; When a marker MK is disposed at a second position P2, a second captured image PM2 including the marker MK and a second projected image PB2 projected by the projector 100 and different from the first projected image PB1 is acquired, and a first side SD1 and a second side SD2 of the marker MK, a third length LX2 of the first side SD1 and a fourth length LY2 of the second side SD2 are specified based on the second captured image PM2, and a third length LX2 of the second side SD2 is specified. determining a third coordinate axis AX2 having a unit length of LY2; determining a fourth coordinate axis AY2 parallel to the second side SD2 and having a unit length of LY2; determining second information JP2 including coordinates of the four corners of the second projected image PB2 in the coordinates defined by the third coordinate axis AX2 and the fourth coordinate axis AY2; and adjusting the projection position, shape, and size of the second projected image PB2 based on the first information JP1 and the second information JP2.

According to this configuration, when the marker MK is disposed at the first position P1, a first coordinate axis AX1 is determined based on the first captured image PM1, the first coordinate axis AX1 being parallel to the first side SD1 and having a unit length of the first length LX1, and a second coordinate axis AY1 is determined based on the second side SD2 and having a unit length of the second length LY1. Then, first information JP1 including coordinates of the four corners of the first projected image PB1 in the coordinates defined by the first coordinate axis AX1 and the second coordinate axis AY1 is stored. When the marker MK is disposed at the second position P2, a third coordinate axis AX2 is determined based on the second captured image PM2, the third coordinate axis AX2 is determined based on the first side SD1 and having a unit length of the third length LX2, and a fourth coordinate axis AY2 is determined based on the second captured image PM2, the fourth coordinate axis AY2 is determined based on the second captured image PM2, the third coordinate axis AX2 is determined based on the first side SD1 and having a unit length of the third length LX2, and a fourth coordinate axis AY2 is determined based on the second side SD2 and having a unit length of the fourth length LY2. Then, second information JP2 including the coordinates of the four corners of the second projection image PB2 in the coordinate system defined by the third coordinate axis AX2 and the fourth coordinate axis AY2 is determined. Furthermore, the projection position, shape, and size of the second projection image PB2 are adjusted based on the first information JP1 and the second information JP2.
Therefore, the first coordinate axis AX1 in the first captured image PM1 corresponds to the third coordinate axis AX2 in the second captured image PM2, and the second coordinate axis AY1 in the first captured image PM1 corresponds to the fourth coordinate axis AY2 in the second captured image PM2. Furthermore, the coordinates of the four corners of the first projected image PB1 included in the first information JP1 are defined by the first coordinate axis AX1 and the second coordinate axis AY1, and the coordinates of the four corners of the second projected image PB2 included in the second information JP2 are defined by the third coordinate axis AX2 and the fourth coordinate axis AY2. Therefore, the projection position, shape, and size of the second projected image PB2 can be appropriately adjusted based on the coordinates of the four corners of the first projected image PB1 included in the first information JP1 and the coordinates of the four corners of the second projected image PB2 included in the second information JP2. For example, by adjusting the second projected image PB2 so that each of the coordinates of the four corners of the second projected image PB2 matches each of the coordinates of the four corners of the first projected image PB1, the projection position, shape, and size of the second projected image PB2 can be appropriately adjusted.

In the display method of the projector 100 according to this embodiment, the first information JP1 includes the coordinates of the center CT1 of the first projected image PB1, and the second information JP2 includes the coordinates of the center CT2 of the second projected image PB2.
Therefore, for example, by adjusting the position of the second projected image PB2 so that the coordinates of the center CT2 of the second projected image PB2 match the coordinates of the center CT1 of the first projected image PB1, the projection position of the second projected image PB2 can be appropriately adjusted.

Furthermore, in the display method according to this embodiment, adjusting the projection position, shape, and size of the second projection image PB2 includes adjusting the projection position of the second projection image PB2 so that the coordinates of the center CT2 of the second projection image PB2 match the coordinates of the center CT1 of the first projection image PB1, and adjusting the shape and size of the second projection image PB2 so that the coordinates of the four corners of the second projection image PB2 match the coordinates of the four corners of the first projection image PB1.
Therefore, the projection position of the second projection image PB2 is adjusted so that the coordinates of the center CT2 of the second projection image PB2 match the coordinates of the center CT1 of the first projection image PB1, and therefore the projection position of the second projection image PB2 can be appropriately adjusted. Also, the shape and size of the second projection image PB2 are adjusted so that the coordinates of each of the four corners of the second projection image PB2 match the coordinates of each of the four corners of the first projection image PB1, and therefore the shape and size of the second projection image PB2 can be appropriately adjusted. Therefore, the projection position, shape, and size of the second projection image PB2 can be appropriately adjusted.

In addition, in the display method according to this embodiment, the first side SD1 and the second side SD2 of the marker MK are perpendicular to each other.
As described above, the first coordinate axis AX1 is determined parallel to the first side SD1 and the second coordinate axis AY1 is determined parallel to the second side SD2 based on the first captured image PM1, and the third coordinate axis AX2 is determined parallel to the first side SD1 and the fourth coordinate axis AY2 is determined parallel to the second side SD2 based on the second captured image PM2.
Therefore, the first coordinate axis AX1 and the second coordinate axis AY1 are determined so that the first coordinate axis AX1 and the second coordinate axis AY1 are orthogonal to each other, and the third coordinate axis AX2 and the fourth coordinate axis AY2 are determined so that the third coordinate axis AX2 and the fourth coordinate axis AY2 are orthogonal to each other. That is, the first coordinate axis AX1 and the second coordinate axis AY1 form an orthogonal coordinate system, and the third coordinate axis AX2 and the fourth coordinate axis AY2 form an orthogonal coordinate system. Therefore, the first coordinate axis AX1 and the second coordinate axis AY1, and the third coordinate axis AX2 and the fourth coordinate axis AY2 can be properly determined.

Furthermore, in the display method according to this embodiment, determining the first coordinate axis AX1 and the second coordinate axis AY1 includes setting the position of the intersection point TS1 between the first side SD1 and the second side SD2 in the first captured image PM1 to the position of the origin RG1 of the first coordinate axis AX1 and the second coordinate axis AY1, and determining the third coordinate axis AX2 and the fourth coordinate axis AY2 includes setting the position of the intersection point TS2 between the first side SD1 and the second side SD2 in the second captured image PM2 to the position of the origin RG of the third coordinate axis AX2 and the fourth coordinate axis AY2.
Therefore, the position of the origin RG1 of the first coordinate axis AX1 and the second coordinate axis AY1 coincides with the position of the intersection TS1 between the first side SD1 and the second side SD2 in the first captured image PM1. Also, the position of the origin RG of the third coordinate axis AX2 and the fourth coordinate axis AY2 coincides with the position of the intersection TS2 between the first side SD1 and the second side SD2 in the second captured image PM2. Therefore, the position of the origin RG1 of the first coordinate axis AX1 and the second coordinate axis AY1 and the position of the origin RG of the third coordinate axis AX2 and the fourth coordinate axis AY2 can be properly determined.

In addition, in the display method according to this embodiment, the first projection image PB1 has a rectangular shape including a third side SD3 parallel to the first side SD1 and a fourth side SD4 parallel to the second side SD2.
Therefore, for example, when the first side SD1 and the second side SD2 are perpendicular to each other, the first projection image PB1 and the second projection image PB2 can each be projected in a rectangular shape.

In addition, the display method according to this embodiment includes notifying that the second captured image PM2 is inappropriate if the second captured image PM2 does not include at least one of the four corners of the second projection image PB2.
Therefore, when the second captured image PM2 does not include at least one of the four corners of the second projected image PB2, it is notified that the second captured image PM2 is not appropriate. Also, when the second captured image PM2 does not include at least one of the four corners of the second projected image PB2, the coordinates of the four corners of the second projected image PB2 cannot be obtained, and therefore the projection position, shape, and size of the second projected image PB2 cannot be adjusted. Therefore, the user can confirm that the second captured image PM2 is not appropriate.

Furthermore, the control program PG of the projector 100 according to this embodiment acquires a first captured image PM1 including the marker MK and a first projection image PB1 projected by the projector 100 when a marker MK having a first side SD1 and a second side SD2 intersecting with the first side SD1 is placed at a first position P1, and acquires, based on the first captured image PM1, the first side SD1 and the second side SD2 of the marker MK, a first length LX1 and specifying a second length LY1 of the second side SD2; determining a first coordinate axis AX1 that is parallel to the first side SD1 and has the first length LX1 as a unit length; determining a second coordinate axis AY1 that is parallel to the second side SD2 and has the second length LY1 as a unit length; storing first information JP1 including coordinates of four corners of the first projection image PB1 in the coordinates defined by the first coordinate axis AX1 and the second coordinate axis AY1; When the marker MK is placed, a second captured image PM2 including the marker MK and a second projected image PB2 projected by the projector 100 and different from the first projected image PB1 is acquired; based on the second captured image PM2, a first side SD1 and a second side SD2 of the marker MK, a third length LX2 of the first side SD1, and a fourth length LY2 of the second side SD2 are specified; and a third coordinate axis AX2 that is parallel to the first side SD1 and has the third length LX2 as a unit length is determined. The processor 150A of the projector 100 is caused to execute the following operations: determine a fourth coordinate axis AY2 that is parallel to the second side SD2 and has a fourth length LY2 as a unit length; determine second information JP2 that includes the coordinates of the four corners of the second projected image PB2 in the coordinates defined by the third coordinate axis AX2 and the fourth coordinate axis AY2; and adjust the projection position, shape, and size of the second projected image PB2 based on the first information JP1 and the second information JP2.
According to this configuration, the control program PG of the projector 100 according to this embodiment achieves the same effects as the display method according to this embodiment.

The projector 100 according to this embodiment includes a camera 170 that captures a projection image PB, a memory 150B, and a processor 150A. When a marker MK having a first side SD1 and a second side SD2 intersecting with the first side SD1 is placed at a first position P1, the processor 150A acquires a first captured image PM1 including the marker MK and a first projection image PB1 projected by the projector 100, and calculates a marker based on the first captured image PM1. determining a first coordinate axis AX1 that is parallel to the first side SD1 and has the first length LX1 as a unit length; determining a second coordinate axis AY1 that is parallel to the second side SD2 and has the second length LY1 as a unit length; and generating first information JP including coordinates of four corners of a first projection image PB1 in the coordinate system defined by the first coordinate axis AX1 and the second coordinate axis AY1. and storing a second captured image PM2 including the marker MK and a second projected image PB2 projected by the projector 100 and different from the first projected image PB1 when the marker MK is arranged at a second position P2 different from the first position P1. Based on the second captured image PM2, the first side SD1 and the second side SD2 of the marker MK, a third length LX2 of the first side SD1 and a fourth length LY2 of the second side SD2 are specified. , determining a third coordinate axis AX2 having a unit length of the third length LX2, determining a fourth coordinate axis AY2 parallel to the second side SD2 and having a unit length of the fourth length LY2, determining second information JP2 including the coordinates of the four corners of the second projected image PB2 in the coordinates defined by the third coordinate axis AX2 and the fourth coordinate axis AY2, and adjusting the projection position, shape, and size of the second projected image PB2 based on the first information JP1 and the second information JP2.

With this configuration, the projector 100 according to this embodiment achieves the same effects as the display method according to this embodiment.

8. Other embodiments
The above-described embodiment is a preferred embodiment, but is not limited to the above-described embodiment and various modifications are possible without departing from the spirit and scope of the present invention.

In this embodiment, the adjustment unit 156 adjusts the projection position of the second projection image PB2 and adjusts the shape and size of the second projection image PB2, but this is not limited to the above. It is sufficient to adjust the projection position, shape, and size of the second projection image PB2. For example, the adjustment unit 156 may adjust the projection position, shape, and size of the second projection image PB2 so that the coordinates of the four corners of the second projection image PB2 match the coordinates of the four corners of the first projection image PB1.

In addition, in this embodiment, a case will be described in which the first information JP1 includes the coordinates of each of the four corners and the center CT1 of the first projection image PB1, and the second information JP2 includes the coordinates of each of the four corners and the center CT1 of the second projection image PB2, but this is not limited to the above. It is sufficient that the first information JP1 includes the coordinates of each of the four corners of the first projection image PB1, and the second information JP2 includes the coordinates of each of the four corners of the second projection image PB2. In other words, for example, the adjustment unit 156 may calculate the coordinates of the center CT1 of the first projection image PB1 from the coordinates of each of the four corners of the first projection image PB1, and calculate the coordinates of the center CT2 of the second projection image PB2 from the coordinates of each of the four corners of the second projection image PB2.

In addition, in this embodiment, the processing unit 154 stores the second information JP2 in the information storage unit 158, but this is not limiting. The processing unit 154 only needs to determine the second information JP2. In other words, the processing unit 154 only needs to determine the coordinates defined by the third coordinate axis AX2 and the fourth coordinate axis AY2 for each of the four corners and the center CT2 of the second projection image PB2.

In addition, in this embodiment, the marker MK is described as having a first side SD1 and a second side SD2 that are perpendicular to each other, but this is not limited to the above. The marker MK may have a first side SD1 and a second side SD2 that intersect at a predetermined angle. The predetermined angle is, for example, 60 degrees. The predetermined angle is also, for example, 120 degrees.

In addition, in this embodiment, a case will be described in which the determination unit 153 sets the position of the intersection point TS1 to the position of the origin RG1 of the first coordinate axis AX1 and the second coordinate axis AY1, and sets the position of the intersection point TS2 to the position of the origin RG of the third coordinate axis AX2 and the fourth coordinate axis AY2, but is not limited to this. The determination unit 153 may determine the position of the origin RG1 of the first coordinate axis AX1 and the second coordinate axis AY1 based on the image of the marker MK in the first captured image PM1. The determination unit 153 may determine the position of the origin RG2 of the third coordinate axis AX2 and the fourth coordinate axis AY2 based on the image of the marker MK in the second captured image PM2.
For example, the determination unit 153 may determine the position of the right end of the first side SD1 of the marker MK in the first captured image PM1 to be the position of the origin RG1 of the first coordinate axis AX1 and the second coordinate axis AY1. In this case, the determination unit 153 determines the position of the right end of the first side SD1 of the marker MK in the second captured image PM2 to be the position of the origin RG2 of the third coordinate axis AX2 and the fourth coordinate axis AY2.
Furthermore, the determination unit 153 may determine, for example, the position of the bottom end of the second side SD2 of the marker MK in the first captured image PM1 to be the position of the origin RG1 of the first coordinate axis AX1 and the second coordinate axis AY1. In this case, the determination unit 153 determines the position of the bottom end of the second side SD2 of the marker MK in the second captured image PM2 to be the position of the origin RG2 of the third coordinate axis AX2 and the fourth coordinate axis AY2.

In addition, in this embodiment, the first projection image PB1 is described as having a rectangular shape including a third side SD3 parallel to the first side SD1 and a fourth side SD4 parallel to the second side SD2, but is not limited to this. The first projection image PB1 may have any shape as long as it is a quadrangle. The first projection image PB1 may have, for example, a parallelogram shape. In this case, the second projection image PB2 is adjusted to have a parallelogram shape similar to the first projection image PB1. The first projection image PB1 may also have, for example, a trapezoid shape. In this case, the second projection image PB2 is adjusted to have a trapezoid shape similar to the first projection image PB1.

In addition, in this embodiment, the marker MK is a plate-like member formed in an L-shape, but is not limited to this. The marker MK only needs to have a first side SD1 and a second side SD2 that intersects with the first side SD1. For example, the marker MK may be a plate-like member formed in a triangular shape. For example, the marker MK may be a plate-like member formed in a right-angled triangle or rectangular shape. In this case, it is preferable that the two sides that sandwich the right angle are the first side SD1 and the second side SD2. In this case, the first side SD1 and the second side SD2 are perpendicular to each other.

Furthermore, each functional unit shown in FIG. 1 and FIG. 2 indicates a functional configuration, and the specific implementation form is not particularly limited. In other words, it is not necessarily necessary to implement hardware that corresponds to each functional unit individually, and it is of course possible to implement a configuration in which one processor executes a program to realize the functions of multiple functional units. Furthermore, some of the functions realized by software in the above embodiment may be realized by hardware, or some of the functions realized by hardware may be realized by software. In addition, the specific detailed configurations of each of the other units of projector 100 may also be changed as desired without departing from the spirit of the projector.

The processing units in the flowchart shown in FIG. 7 are divided according to the main processing content in order to make the processing of the control unit 150 easier to understand. There is no limitation to the manner in which the processing units are divided or the names shown in the flowchart in FIG. 7, and they can be divided into even more processing units depending on the processing content, or one processing unit can be divided to include even more processes. Furthermore, the processing order of the above flowchart is not limited to the example shown.

The display method according to the present disclosure can be realized by causing the processor 150A of the projector 100 to execute a control program PG corresponding to the display method. This control program PG can also be recorded on a computer-readable recording medium. As the recording medium, a magnetic or optical recording medium or a semiconductor memory device can be used.
Specifically, examples of the recording medium include portable or fixed recording media such as a flexible disk, HDD, CD-ROM (Compact Disk Read Only Memory), DVD, Blu-ray (registered trademark) Disc, magneto-optical disk, flash memory, card-type recording medium, etc. The recording medium may also be a non-volatile storage device such as a RAM, ROM, or HDD, which is an internal storage device provided in the image processing device.
Moreover, the control program PG corresponding to the display method can be stored in a server device or the like, and the display method can be realized by downloading the control program PG from the server device to the projector 100 .

9. Summary of the Disclosure
The following is a summary of this disclosure.
(Appendix 1)
a first captured image including the marker and a first projection image projected by a projector when the marker is placed at a first position, the first captured image including the marker and a first projection image projected by a projector; specifying the first side and the second side of the marker, a first length of the first side, and a second length of the second side based on the first captured image; determining a first coordinate axis that is parallel to the first side and has the first length as a unit length; determining a second coordinate axis that is parallel to the second side and has the second length as a unit length; a display method including: acquiring a second captured image including the marker and a second projected image projected by the projector, the second projected image being different from the first projected image; identifying the first side and the second side of the marker, a third length of the first side, and a fourth length of the second side based on the second captured image; determining a third coordinate axis that is parallel to the first side and has the third length as a unit length; determining a fourth coordinate axis that is parallel to the second side and has the fourth length as a unit length; determining second information including coordinates of four corners of the second projected image in coordinates defined by the third coordinate axis and the fourth coordinate axis; and adjusting a projection position, shape, and size of the second projected image based on the first information and the second information.

This allows the projection position, shape, and size of the second projected image to be appropriately adjusted based on the coordinates of the four corners of the first projected image contained in the first information and the coordinates of the four corners of the second projected image contained in the second information. For example, by adjusting the second projected image so that each of the coordinates of the four corners of the second projected image matches each of the coordinates of the four corners of the first projected image, the projection position, shape, and size of the second projected image can be appropriately adjusted. This reduces the effort required to place markers.

(Appendix 2)
The display method according to claim 1, wherein the first information includes coordinates of a center of the first projection image, and the second information includes coordinates of a center of the second projection image.

This allows the projection position of the second projected image to be appropriately adjusted, for example, by adjusting the position of the second projected image so that the coordinates of the center of the second projected image match the coordinates of the center of the first projected image.

(Appendix 3)
The display method described in Appendix 2, wherein adjusting the projection position, shape, and size of the second projection image includes adjusting the projection position of the second projection image so that the coordinates of the center of the second projection image match the coordinates of the center of the first projection image, and adjusting the shape and size of the second projection image so that the coordinates of each of the four corners of the second projection image match the coordinates of each of the four corners of the first projection image.

This allows the shape and size of the second projected image to be adjusted so that the coordinates of each of the four corners of the second projected image match the coordinates of each of the four corners of the first projected image, thereby allowing the shape and size of the second projected image to be appropriately adjusted.

(Appendix 4)
The display method according to any one of Supplementary Note 1 to Supplementary Note 3, wherein the first side and the second side of the marker are perpendicular to each other.

As a result, the first and second coordinate axes are determined so that they are orthogonal to each other, and the third and fourth coordinate axes are determined so that they are orthogonal to each other. Therefore, the first and second coordinate axes, and the third and fourth coordinate axes can be determined appropriately.

(Appendix 5)
The display method of any one of Appendix 1 to Appendix 4, wherein determining the first coordinate axis and the second coordinate axis includes setting a position of an intersection of the first side and the second side in the first captured image as a position of an origin of the first coordinate axis and the second coordinate axis, and determining the third coordinate axis and the fourth coordinate axis includes setting a position of an intersection of the first side and the second side in the second captured image as a position of an origin of the third coordinate axis and the fourth coordinate axis.

As a result, the position of the origin of the first coordinate axis and the second coordinate axis coincides with the position of the intersection of the first side and the second side in the first captured image. In addition, the position of the origin of the third coordinate axis and the fourth coordinate axis coincides with the position of the intersection of the first side and the second side in the second captured image. Therefore, the position of the origin of the first coordinate axis and the second coordinate axis and the position of the origin of the third coordinate axis and the fourth coordinate axis can be properly determined.

(Appendix 6)
The display method according to any one of Supplementary Note 1 to Supplementary Note 5, wherein the first projection image has a rectangular shape including a third side parallel to the first side and a fourth side parallel to the second side.

As a result, for example, when the first side and the second side are perpendicular to each other, the first projection image and the second projection image can each be projected into a rectangular shape.

(Appendix 7)
The display method according to any one of Appendix 1 to Appendix 6, further comprising notifying that the second captured image is not appropriate when the second captured image does not include at least one of the four corners of the second projected image.

This allows the user to confirm that the second captured image is not correct.

(Appendix 8)
a first captured image including the marker and a first projection image projected by a projector when the marker is placed at a first position, the first captured image including the marker and a first projection image projected by a projector; specifying the first side and the second side of the marker, a first length of the first side, and a second length of the second side based on the first captured image; determining a first coordinate axis that is parallel to the first side and has the first length as a unit length; determining a second coordinate axis that is parallel to the second side and has the second length as a unit length; A program that causes a processor of a projector to execute the following steps: acquire a second captured image including a second projected image projected by the projector, the second projected image being different from a first projected image; identify the first side and the second side of the marker, a third length of the first side, and a fourth length of the second side based on the second captured image; determine a third coordinate axis that is parallel to the first side and has the third length as a unit length; determine a fourth coordinate axis that is parallel to the second side and has the fourth length as a unit length; determine second information including coordinates of four corners of the second projected image in coordinates defined by the third coordinate axis and the fourth coordinate axis; and adjust the projection position, shape, and size of the second projected image based on the first information and the second information.

As a result, the program described in Appendix 8 achieves the same effect as the display method described in Appendix 1.

(Appendix 9)
The at least one processor includes a camera that captures a projection image, a memory, and at least one processor, and when a marker having a first side and a second side intersecting the first side is placed at a first position, the at least one processor acquires a first captured image including the marker and a first projection image projected by a projector, specifies the first side and the second side of the marker, a first length of the first side, and a second length of the second side based on the first captured image, determines a first coordinate axis that is parallel to the first side and has the first length as a unit length, determines a second coordinate axis that is parallel to the second side and has the second length as a unit length, stores first information including coordinates of four corners of the first projection image in coordinates defined by the first coordinate axis and the second coordinate axis, and stores information regarding the first position. a fourth length of the second side of the marker based on the second captured image; determining a third coordinate axis that is parallel to the first side and has the third length as a unit length; determining a fourth coordinate axis that is parallel to the second side and has the fourth length as a unit length; determining second information including coordinates of four corners of the second projected image in coordinates defined by the third coordinate axis and the fourth coordinate axis; and adjusting a projection position, a shape, and a size of the second projected image based on the first information and the second information.

As a result, the projector described in Appendix 9 achieves the same effect as the display method described in Appendix 1.

100...projector, 110...projection unit, 112...light modulation device, 113...projection optical system, 115...liquid crystal panel, 120...drive unit, 121...light source drive unit, 122...light modulation device drive unit, 141...communication interface, 145...image processing unit, 150...control unit, 150A...first processor (at least one processor), 150B...first memory, 151...acquisition unit, 152...identification unit, 153...determination unit, 154...processing unit, 155...notification unit, 156...adjustment unit, 157...image storage unit, 158...information storage unit, 170...camera, 200...image supply device, AR1, AR2...imaging area, AX1...first coordinate axis, AX2...third coordinate axis, AY1...second coordinate axis, AY2...fourth coordinate axis, CD1...first coordinate system, CD2...second Coordinate system, CT1, CT2, CT2A...center, DX1, DX2...first direction, DY1, DY2...second direction, JP1...first information, JP2...second information, LX1...first length (first length), LY1...second length (second length), LX2...third length (third length), LY2...fourth length (fourth length), MK...marker, P1...first position, P2...second position, PB...throw projected image, PB1...first projected image, PB2, PB2A, PB2B...second projected image, PG...control program (program), PM...captured image, PM1...first captured image, PM2, PM2A, PM2B...second captured image, RG1, RG2...origin, SD1...first side, SD2...second side, SD3...long side (third side), SD4...short side (fourth side), TS1, TS2...intersection.

Claims (9)

When a marker having a first side and a second side intersecting the first side is placed at a first position,
acquiring a first captured image including the marker and a first projected image projected by a projector;
Identifying the first side and the second side of the marker, a first length of the first side, and a second length of the second side based on the first captured image;
determining a first coordinate axis that is parallel to the first side and has the first length as a unit length;
determining a second coordinate axis that is parallel to the second side and has the second length as a unit length;
storing first information including coordinates of four corners of the first projection image in coordinates defined by the first coordinate axis and the second coordinate axis;
When the marker is placed at a second position different from the first position,
acquiring a second captured image including the marker and a second projected image projected by the projector, the second projected image being different from the first projected image;
Identifying the first side, the second side, a third length of the first side, and a fourth length of the second side of the marker based on the second captured image;
determining a third coordinate axis that is parallel to the first side and has the third length as a unit length;
determining a fourth coordinate axis that is parallel to the second side and has the fourth length as a unit length;
determining second information including coordinates of four corners of the second projected image in coordinates defined by the third coordinate axis and the fourth coordinate axis;
adjusting a projection position, a shape, and a size of the second projection image based on the first information and the second information;
Including, how to display. the first information includes coordinates of a center of the first projection image;
The second information includes coordinates of a center of the second projection image.
The display method according to claim 1 . Adjusting the projection position, shape, and size of the second projection image includes:
adjusting a projection position of the second projection image such that coordinates of a center of the second projection image coincide with coordinates of a center of the first projection image;
adjusting a shape and a size of the second projected image such that coordinates of four corners of the second projected image match coordinates of four corners of the first projected image;
The display method according to claim 1 or 2, comprising: The marker is
The display method according to claim 1 , wherein the first side and the second side are perpendicular to each other. Determining the first coordinate axis and the second coordinate axis includes:
determining a position of an intersection between the first side and the second side in the first captured image as a position of an origin of the first coordinate axis and the second coordinate axis;
Including,
Determining the third coordinate axis and the fourth coordinate axis includes:
determining a position of an intersection between the first side and the second side in the second captured image as a position of an origin of the third coordinate axis and the fourth coordinate axis;
The display method according to claim 4 , comprising: the first projected image has a rectangular shape including a third side parallel to the first side and a fourth side parallel to the second side;
The display method according to claim 4 or 5. when the second captured image does not include at least one of the four corners of the second projected image, notifying that the second captured image is not appropriate;
The display method of claim 1 , comprising: When a marker having a first side and a second side intersecting the first side is placed at a first position,
acquiring a first captured image including the marker and a first projected image projected by a projector;
Identifying the first side and the second side of the marker, a first length of the first side, and a second length of the second side based on the first captured image;
determining a first coordinate axis that is parallel to the first side and has the first length as a unit length;
determining a second coordinate axis that is parallel to the second side and has the second length as a unit length;
storing first information including coordinates of four corners of the first projection image in coordinates defined by the first coordinate axis and the second coordinate axis;
When the marker is placed at a second position different from the first position,
acquiring a second captured image including the marker and a second projected image projected by the projector, the second projected image being different from the first projected image;
Identifying the first side, the second side, a third length of the first side, and a fourth length of the second side of the marker based on the second captured image;
determining a third coordinate axis that is parallel to the first side and has the third length as a unit length;
determining a fourth coordinate axis that is parallel to the second side and has the fourth length as a unit length;
determining second information including coordinates of four corners of the second projected image in coordinates defined by the third coordinate axis and the fourth coordinate axis;
adjusting a projection position, a shape, and a size of the second projection image based on the first information and the second information;
A program that causes the projector's processor to execute the above. A camera for capturing a projected image;
Memory and
at least one processor;
The at least one processor
When a marker having a first side and a second side intersecting the first side is placed at a first position,
acquiring a first captured image including the marker and a first projected image projected by a projector;
Identifying the first side and the second side of the marker, a first length of the first side, and a second length of the second side based on the first captured image;
determining a first coordinate axis that is parallel to the first side and has the first length as a unit length;
determining a second coordinate axis that is parallel to the second side and has the second length as a unit length;
storing first information including coordinates of four corners of the first projection image in coordinates defined by the first coordinate axis and the second coordinate axis;
When the marker is placed at a second position different from the first position,
acquiring a second captured image including the marker and a second projected image projected by the projector, the second projected image being different from the first projected image;
Identifying the first side, the second side, a third length of the first side, and a fourth length of the second side of the marker based on the second captured image;
determining a third coordinate axis that is parallel to the first side and has the third length as a unit length;
determining a fourth coordinate axis that is parallel to the second side and has the fourth length as a unit length;
determining second information including coordinates of four corners of the second projected image in coordinates defined by the third coordinate axis and the fourth coordinate axis;
adjusting a projection position, a shape, and a size of the second projection image based on the first information and the second information;
Run the projector.

Images