JP2019192996A - Projection control apparatus and projection control method - Google Patents

Abstract

To provide a projection control apparatus capable of easily aligning projection images of a plurality of projection apparatuses.SOLUTION: The projection control apparatus controls a plurality of projection apparatuses including a first projection apparatus for projecting a first projection image a second projection apparatus for projecting a second projection image. The projection control apparatus includes: control means for controlling the first projection apparatus and the second projection apparatus; and reception means for receiving a user's operation. The control means controls the first projection apparatus to project images 1401 to 1404 representing a common area where the first projection image and the second projection image are deformable and the second projection apparatus to project images 1406 to 1409 for designating the deformed shapes of the first projection image and the second projection image. The reception means accepts an operation for moving the images 1406 to 1409.SELECTED DRAWING: Figure 14

Description

  The present invention relates to a projection control apparatus and a projection control method for controlling a plurality of projection apparatuses.

  A projection method (stack projection) in which projection positions of a plurality of projectors (projectors) are overlapped is known. In the case of stack projection, since projection alignment between projectors is necessary, a projector having a function for facilitating alignment is also known.

  Except when projecting from a directly-facing position where the optical axis of the projector and the projection plane are orthogonal, the shape of the projected image is distorted (trapezoidal distortion). A keystone correction function is known as a function for correcting this trapezoidal distortion without changing the position of the projector. The keystone correction function can be realized by deforming the image of the liquid crystal panel so as to cancel the trapezoidal distortion.

  A technique is known in which the projection positions of a plurality of projectors are overlapped by applying this keystone correction function.

  By the way, in the keystone correction, there is a case where a deformable range of the projected image is determined due to restrictions of hardware and software. The technique disclosed in Patent Document 1 shows a user how to correct trapezoidal distortion by superimposing an image representing a deformable range of keystone correction on a projected image.

JP 2009-200557 A

  In order to realize stack projection, a keystone correction function is often used for each of a plurality of projectors. Since the optical axes of a plurality of projectors are often not parallel to each other, the amount of deformation for keystone correction of the plurality of projectors is different. In addition, the target projection position (for example, the corner of the screen) needs to be within the deformable range of all projectors used for stack projection.

  From the above, the alignment becomes very complicated unless the deformable range common to the plurality of projectors is known in advance.

  However, Patent Document 1 does not consider the case of using a plurality of projectors.

  Accordingly, an object of the present invention is to provide a projection control device that can facilitate alignment of projection images of a plurality of projection devices.

  In order to achieve the object, a projection control device according to one aspect of the present invention includes a plurality of projection devices including a first projection device that projects a first projection image and a second projection device that projects a second projection image. A projection control apparatus for controlling the first projection apparatus and the second projection apparatus, and a reception means for receiving a user operation, wherein the control means includes the first projection. An image for projecting an image representing a common area where the image and the second projection image can be deformed onto the first projection device, and designating the deformed shape of the first projection image and the second projection image Is projected onto the second projection device, and the accepting means accepts an operation of moving an image for designating the shape.

  According to the projection control apparatus as one aspect of the present invention, it is possible to easily align the projection images of a plurality of projection apparatuses.

  Other aspects of the present invention will be clarified in the embodiments described below.

FIG. 2 is a schematic diagram illustrating a configuration example of a projection system that performs stack projection according to the first embodiment. FIG. 3 is a block diagram illustrating an example of a functional configuration of the projection system according to the first embodiment. Illustration about keystone correction Diagram of the deformable amount of keystone correction The flowchart regarding the outline | summary of the automatic alignment process which concerns on Embodiment 1. FIG. FIG. 6 is a diagram illustrating an example of a projector selection GUI according to the first embodiment. 6 is a diagram illustrating an example of a camera selection GUI according to the first embodiment. FIG. FIG. 5 is a diagram illustrating an example of a camera parameter setting GUI according to the first embodiment. FIG. 5 is a diagram illustrating an example of an alignment mode selection GUI according to the first embodiment. Flowchart relating to automatic alignment processing according to Embodiment 1 The figure which shows an example of a camera coordinate plane and a projector coordinate plane The flowchart regarding the projection shape designation | designated process which concerns on Embodiment 1. FIG. Diagram of deformable area in each plane The figure which shows an example of the post-deformation shape designation | designated marker and deformable area | region marker which concern on Embodiment 1. FIG. 5 is a diagram showing an example of a projection shape designation GUI according to the first embodiment. Flowchart for projection shape designation processing according to the second embodiment The figure which shows an example of the marker for a deformable area | region detection which concerns on Embodiment 2. FIG. Flowchart for projection shape designation processing according to Embodiment 3

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited to embodiment described. In addition, all of the constituent elements described in the embodiments are not necessarily essential to the present invention. Individual functional blocks in the embodiments can be realized by hardware, software, or a combination of hardware and software. One functional block may be realized by a plurality of hardware. One hardware may realize a plurality of functional blocks. In addition, the one or more functional blocks may be realized by executing a computer program loaded into at least one memory by at least one programmable processor (CPU, MPU, etc.). When at least one functional block is realized by hardware, it can be realized by an integrated circuit such as a discrete circuit, FPGA, or ASIC.

  In the following embodiments, a configuration in which the present invention is applied to a stand-alone type projector (projector) will be described. However, the present invention can also be applied to projectors incorporated in general electronic devices such as personal computers, smartphones, tablet terminals, game machines, and digital (video) cameras.

  In the description of the embodiments, GUIs may be used. However, these are only examples, and GUI layout, component types, screen transitions, and the like are omitted or replaced without departing from the gist of the present invention. Can make changes.

[System configuration]
FIG. 1 is a schematic diagram illustrating an example of a projection system according to an embodiment of the present invention. The projection system 10 performs stack projection in which projection areas of a plurality of projectors are overlapped and matched on the projection plane 500 in order to expand the dynamic range of the projection image, improve the luminance, or perform 3D display. FIG. 1 shows a projection system that matches the projection areas A and B of the two projectors 100a and 100b, but the present invention is applied to a projection system that matches the projection areas of three or more projectors. Also good.

  All projectors included in the projection system 10 are connected to a personal computer (PC) 200 functioning as a projection control device so as to be able to communicate with each other. In the present embodiment, the use of a PC as an example of the projection control apparatus is exemplified, but other information processing apparatuses such as a smartphone and a tablet may be used. Further, communication between the plurality of projectors and the projection control apparatus may be wired communication or wireless communication, and there is no particular limitation on the communication protocol. In this embodiment, as an example, it is assumed that communication between devices is performed in a local area network (LAN) using TCP / IP as a communication protocol.

  In addition, the PC 200 can control the operations of the projectors 100a and 100b by transmitting predetermined commands to the projectors 100a and 100b. Projectors 100a and 100b perform an operation in accordance with a command received from PC 200, and transmit the operation result to PC 200.

  The projection system 10 further includes an imaging device 300 as an imaging unit. The imaging device 300 is, for example, a digital camera, a web camera, or a network camera. Alternatively, an imaging device built in the projector 100 or the PC 200 may be used. The imaging device is installed so as to include the entire projection plane as an imaging range. When using an imaging device external to the PC 200, the PC 200 is communicably connected to the PC 200 directly or through a LAN. The PC 200 can control the operation of the imaging apparatus 300 by transmitting a predetermined command to the imaging apparatus 300. For example, the imaging apparatus 300 can perform imaging in response to a request from the PC 200, and can transmit the obtained captured image data to the PC 200.

[Configuration of Projector 100]
FIG. 2 is a block diagram illustrating a functional configuration example of the projector 100 and the PC 200 included in the projection system 10. The projector 100 includes a CPU 101, a RAM 102, a ROM 103, a projection unit 104, a projection control unit 105, a VRAM 106, an operation unit 107, a network IF 108, an image processing unit 109, and a video input 110. These functional blocks are communicably connected via an internal bus 111.

  The CPU 101 is an example of a programmable processor, and implements the operation of the projector 100 by, for example, reading a program stored in the ROM 103 into the RAM 102 and executing it.

  The RAM 102 is used as a work memory when the CPU 101 executes a program. The RAM 102 stores a program and variables used for executing the program. The RAM 102 may be used for other purposes (for example, as a data buffer).

  The ROM 103 may be rewritable. The ROM 103 stores a program executed by the CPU 101, various setting values such as GUI data used for displaying a menu screen, and the like.

  The projection unit 104 includes a projection optical system including a light source and a lens, and projects an optical image based on the projection image supplied from the projection control unit 105. In this embodiment, a liquid crystal panel is used as an optical modulation element, and an optical image based on the projection image is generated by controlling the reflectance or transmittance of light from the light source according to the projection image. Project to.

  The projection control unit 105 supplies the projection image data supplied from the image processing unit 109 to the projection unit 104.

  The VRAM 106 is a video memory that stores projection image data received from the outside (for example, a PC or a media player).

  The operation unit 107 has an input device such as a key button, a switch, or a touch panel, and is a reception unit that receives an instruction from the user to the projector 100. The CPU 101 monitors the operation of the operation unit 107. When the operation of the operation unit 107 is detected, the CPU 101 executes processing according to the detected operation. When the projector 100 has a remote controller, the operation unit 107 notifies the CPU 101 of an operation signal received from the remote controller.

  A network IF 108 is an interface for connecting the projector 100 to a communication network, and has a configuration conforming to the standard of the supported communication network. In the present embodiment, the projector 100 is connected to a local network common to the PC 200 through the network IF 108. Therefore, communication between the projector 100 and the PC 200 is executed through the network IF 108.

  The image processing unit 109 applies various image processing to the video signal supplied to the video input unit 110 and stored in the VRAM 106 as necessary, and supplies the image signal to the projection control unit 105. The image processing unit 109 may be a microprocessor for image processing, for example. Alternatively, a function corresponding to the image processing unit 109 may be realized by the CPU 101 executing a program stored in the ROM 103.

  Image processing that can be applied by the image processing unit 109 includes frame thinning processing, frame interpolation processing, resolution conversion processing, OSD overlap processing such as a menu screen, keystone correction processing, and edge blending processing. It is not limited to.

  The video input unit 110 is an interface that directly or indirectly receives a video signal output from an external device (PC 200 in the present embodiment), and has a configuration corresponding to the video signal to be supported. The video input unit 110 includes, for example, one or more of a composite terminal, an S video terminal, a D terminal, a component terminal, an analog RGB terminal, a DVI-I terminal, a DVI-D terminal, and an HDMI (registered trademark) terminal. When receiving an analog video signal, the video input unit 110 converts the analog video signal into a digital video signal and stores the digital video signal in the VRAM 106.

[Configuration of PC 200]
Next, the functional configuration of the PC 200 will be described. The PC 200 may be a general-purpose computer to which an external display can be connected, and thus has a functional configuration according to the general-purpose computer. The PC 200 includes a CPU 201, a RAM 202, a ROM 203, an operation unit 204, a display unit 205, a network IF 206, a video output unit 207, and a communication unit 208. These functional blocks are communicably connected via an internal bus 209.

  The CPU 201 is an example of a programmable processor, and realizes the operation of the PC 200 by, for example, reading a program (OS or application program) stored in the ROM 203 into the RAM 202 and executing it.

  The RAM 202 is used as a work memory when the CPU 201 executes a program. The RAM 202 stores a program and variables used for executing the program. The RAM 202 may be used for other purposes (for example, as a data buffer).

  The ROM 203 may be rewritable. The ROM 203 stores programs executed by the CPU 201, GUI data used for displaying menu screens, various setting values, and the like. The PC 200 may have a storage device (HDD or SSD) having a larger capacity than the ROM 203. In this case, a program having a large capacity such as an OS or an application program may be stored in the storage device.

  The operation unit 204 includes input devices such as a keyboard, a pointing device (such as a mouse), a touch panel, and a switch, and receives instructions from the user to the PC 200. The keyboard may be a software keyboard. The CPU 201 monitors the operation of the operation unit 204. When the operation of the operation unit 204 is detected, processing according to the detected operation is executed.

  The display unit 205 is, for example, a liquid crystal panel or an organic EL panel. The display unit 205 displays a screen provided by the OS or application program. The display unit 205 may be an external device. The display unit 205 may be a touch display.

  A network IF 206 is an interface for connecting the PC 200 to a communication network, and has a configuration conforming to the standard of the supported communication network. In the present embodiment, the PC 200 is connected to a local network common to the projector 100 through the network IF 206. Therefore, communication between the PC 200 and the projector 100 is executed through the network IF 206.

  The video output unit 207 is an interface that transmits a video signal to an external device (in this embodiment, the projector 100 or the video distributor 300), and has a configuration corresponding to the video signal to be supported. The video output unit 207 includes, for example, one or more of a composite terminal, an S video terminal, a D terminal, a component terminal, an analog RGB terminal, a DVI-I terminal, a DVI-D terminal, and an HDMI (registered trademark) terminal.

  In this embodiment, the UI screen of the projection control application program having the projection area adjustment function of the projector 100 is displayed on the display unit 205. However, the UI screen is displayed on an external device connected to the video output unit 207. It may be displayed.

  The communication unit 208 is a communication interface for performing serial communication with an external device, for example, and is typically a USB interface, but may have a configuration conforming to another standard such as RS-232C. In this embodiment, it is assumed that the imaging apparatus 300 is connected to the communication unit 208. However, there is no particular limitation on the communication method between the imaging apparatus 300 and the PC 200, and communication conforming to any standard supported by both is performed. be able to.

[Keystone correction]
Next, keystone correction will be described with reference to FIG. Keystone correction geometrically transforms the original image to cancel the trapezoidal distortion that occurs in the projected image according to the deviation between the normal direction of the projection plane and the projection direction (typically the optical axis of the projection optical system). This is correction (geometric correction) to be (deformed). Since the geometric transformation of the image can be realized by projective transformation, the keystone correction is equivalent to the determination of the parameter of the projective transformation that is the correction amount of the geometric correction. For example, the CPU 101 can determine projection transformation parameters based on the movement amount and movement direction of each vertex of the original image on the rectangle, and can provide the parameters to the image processing unit 109.

  For example, assuming that the coordinates of the original image are (xs, ys), the coordinates (xd, yd) of the image after deformation by projective transformation are expressed by the following Equation 1.

  Here, M is a 3 × 3 matrix, which is a projective transformation matrix from the original image to the transformed image. Further, xso and yso are the coordinates of the upper left vertex of the original image indicated by the solid line in FIG. 3, and xdo and ydo are the vertexes (xso, yso) of the original image in the transformed image indicated by the dashed line in FIG. ) Is the coordinate value of the vertex corresponding to.

The CPU 101 supplies the matrix M of Expression 1 and its inverse matrix M ⁻¹ to the image processing unit 109 as offset parameters (xso, yso) and (xdo, ydo) as parameters for keystone correction. The image processing unit 109 can obtain the coordinates (xs, ys) of the original image corresponding to the coordinate values (xd, yd) after the keystone correction according to the following Expression 2.

  If the coordinates xs and ys of the original image obtained by Expression 2 are both integers, the image processing unit 109 uses the pixel values of the original image coordinates (xs, ys) as they are as the coordinates (xd, yd) of the image after keystone correction. ) Pixel value. On the other hand, when the coordinates of the original image obtained by Expression 2 are not integers, the image processing unit 109 calculates the pixel value corresponding to the original image coordinates (xs, ys) by interpolation using the values of a plurality of surrounding pixels. Can be sought. The interpolation calculation can be performed using any of known interpolation calculations such as bilinear and bicubic. When the coordinates of the original image obtained by Expression 2 are the coordinates of the external area of the original image, the image processing unit 109 sets the pixel value of the coordinates (xd, yd) of the image after keystone correction to black (0 ) Or the background color set by the user. In this way, the image processing unit 109 can obtain pixel values for all coordinates of the image after the keystone correction, and create a converted image.

Here, it is assumed that both the matrix M and its inverse matrix M ⁻¹ are supplied from the CPU 101 of the projector 100 to the image processing unit 109, but only one of the matrices is supplied, and the other matrix is image processing. The part 109 may obtain it.

  Note that the coordinates of the vertices of the image after the keystone correction are obtained, for example, by causing the user to input a movement amount through the operation unit 107 so that the vertices are projected at a desired position for each vertex of the projection image. Can do. At this time, in order to support the input of the movement amount, the CPU 201 may cause the projector 100 to project the test pattern using the function of the projection control application program.

  By the way, in the keystone correction, a deformable area may be defined due to hardware or software limitations. FIG. 4 is a schematic diagram showing the deformable amount of keystone correction. 4 in FIG. 4 represents a panel plane of the projector. A hatched area 401 in FIG. 4 represents a deformable area where the upper left vertex is movable when performing keystone correction. The hatched areas 401 to 404 represent the deformable areas of the four vertices of the rectangular panel. In FIG. 4, Δx_max represents the maximum deformable amount in the x-axis direction, and Δy_max represents the maximum deformable amount in the y-axis direction.

  Note that Δx_max and Δy_max may differ depending on the hardware and software configuration of the projector. Δx_max and Δy_max may be set so that each deformable region is similar to the maximum pixel region of the panel, as in Patent Document 1. Further, for example, Δx_max and Δy_max may be set to 500 pixels so that each deformable region is a square.

[Automatic alignment processing]
A flowchart of FIG. 5 shows an outline of the automatic alignment processing realized by the PC 200 of the present embodiment executing the projection control application program.

  In step S <b> 501, the CPU 201 of the PC 200 selects a plurality of projectors to be subjected to automatic alignment processing from the projectors 100 with which the PC 200 can communicate.

  FIG. 6 is a diagram illustrating an example of a GUI screen 600 displayed on the display unit 205 when the CPU 201 executes the projection control application program. The user operates the screen 600 through the operation unit 204 of the PC.

  When the CPU 201 detects that the “search” button 601 has been pressed by the user, the CPU 201 broadcasts a predetermined command requesting information on the projector name and the IP address via the network IF 206 on the network. Note that the information requested at this time is not limited to the projector name and the IP address, and for example, the keystone deformable amount may be requested in advance.

  When the CPU 101 of each projector 100 connected to the network receives a command via the network IF 108, the CPU 101 transmits data including information indicating its projector name and IP address to the PC 200. The CPU 201 of the PC 200 receives the data transmitted in response to the command, extracts information included in the data, and displays it on the list view 602. Note that the arrangement order of the projectors displayed in the list view 602 may be the detected order, or may be sorted based on a specific rule.

  For example, the user selects check boxes 603 to 606 to select a projector to be aligned. Screen 600 is an example when there are four projectors 100 connected to PC 200.

  When the projectors to be aligned are the projector 100a (Projector 1) and the projector 100b (Projector 2), the check box 604 and the check box 606 may be checked.

  Information (for example, projector name, IP address, etc.) relating to the projector whose check box is checked is stored in the RAM 202 of the PC 200.

  When the operation of the “test pattern ON” button 607 in FIG. 6 is detected, the CPU 201 transmits a command for instructing the display of the test pattern through the network IF 206 to each of the plurality of projectors whose check boxes are checked. . This corresponds to S502 of the processing flow in FIG. The test pattern to be displayed according to the operation of the button 606 is a test pattern for making it easy to confirm the size and position of the display area of each projector 100, and displays, for example, a grid image. Regarding the test pattern, a command for instructing display of the test pattern may be transmitted from the PC 200 to each projector 100 in association with each other, or a plurality of commands for causing the projector to draw an arbitrary straight line, figure, character string, or the like. You may transmit in combination.

  When the operation of the “test pattern OFF” button 608 in FIG. 6 is detected, the CPU 201 transmits a command instructing non-display of the test pattern through the network IF 206 to each projector whose check box is checked.

  When the operation of the “Next” button 609 in FIG. 6 is detected, the screen transitions to the screen 700 shown in FIG. 7 (corresponding to the transition from S502 to S503 in FIG. 5).

  In S503 of FIG. 5, the camera used for the automatic alignment process is selected. Reference numeral 700 in FIG. 7 represents a camera selection screen. When the operation of the “Search” button 701 in FIG. 7 is detected, the CPU 201 of the PC 200 acquires information (for example, the product name of the camera) of the camera connected to itself (USB connection, network connection, etc.), and performs drop-down. It is displayed in a list 702. The user selects one desired camera from the plurality of searched cameras by using the drop-down list 702. Note that the number of cameras used for the automatic alignment process is not limited to one, and two or more cameras may be selected.

  An image area 703 shown in FIG. 7 is an area for displaying an image captured by the camera selected from the drop-down list 702. For example, the CPU 201 of the PC 200 can be realized by transmitting a command for instructing shooting to the selected camera 300 and pasting the acquired captured image in the image area 703. The image displayed here is preferably a live view image, but may be a still image.

  By observing the image area 703, the user can easily perform camera installation and zoom adjustment so that the projection images (test patterns) of the projector are all within the imaging range of the camera.

  When the operation of the “Back” button 705 in FIG. 7 is detected, the screen returns to the previous screen (screen shown in FIG. 6). Although not shown in the flowchart of FIG. 5, the process proceeds from S503 to S501.

  A check box 704 in FIG. 7 is used to set whether or not the PC 200 automatically sets camera parameters. When transitioning to the next screen with the “Next” button 706, whether or not to automatically set camera parameters can be switched depending on whether or not the check box 704 is checked. Although detailed description is omitted here, the parameters of the camera can be obtained by performing photometry with the camera in a state where the test patterns of the plurality of projectors 100 are projected.

  When the operation of the “Next” button 706 in FIG. 7 is detected, the screen transitions to a screen indicated by 800 in FIG. 8 (corresponding to the transition from S503 to S504 in FIG. 5).

  In S504 of FIG. 5, camera parameters (shutter speed, ISO sensitivity, aperture value, etc.) are set. 8 represents a camera parameter setting screen.

  Reference numerals 801, 802, and 803 in FIG. 8 are drop-down lists for setting the camera shutter speed, ISO sensitivity, and aperture value, respectively. Note that the camera parameters that can be set are not limited to this, and for example, white balance, photometry method, and the like may be set.

  When the operation of the “test shooting” button 804 in FIG. 8 is detected, the PC 200 transmits a command for prompting shooting to the selected camera 300, acquires the shot image, and displays it on the image display area 805. . Since the image displayed at this time is for confirming whether parameters such as the shutter speed have been set correctly, it is preferably a still image, but it may be a live view image.

  When the operation of the “Back” button 806 in FIG. 8 is detected, the screen returns to the previous screen (the screen shown in FIG. 7). Although not shown in the flowchart of FIG. 5, the process proceeds from S504 to S503.

  When the operation of the “Next” button 807 in FIG. 8 is detected, the screen transitions to a screen indicated by 900 in FIG. 9 (corresponding to the transition from S504 to S505 in FIG. 5).

  In S505 of FIG. 5, the alignment mode is selected. As the alignment mode, one of “4-point designated adjustment” and “Align with reference projector” is selected.

  “Four-point designation adjustment” is a mode in which a keystone correction amount is automatically determined such that the vertices of individual projection areas are aligned with four predetermined points. The four-point designation adjustment is useful when the projection target position is clear, for example, when the projection surface is a screen with a frame. Note that the number of points whose coordinates can be adjusted may be less than 4, or may be 5 points or more including coordinates other than the vertexes.

  “Adjust to reference projector” is a mode in which one projector is used as a reference projector and a keystone correction amount is automatically determined so that the projection area of another projector matches the projection area of the reference projector. The automatic alignment in this mode is executed when the position of the projection area of the reference projector is adjusted to the designated position. A keystone correction amount for automatically matching the projection area of the projector other than the reference projector with the projection area of the reference projector is determined. This is an effective function when the projection target position is not clear (for example, projection onto a wall surface, etc.), unlike “4-point designation adjustment”.

  FIG. 9 shows an alignment mode selection screen 900. Depending on which one of the radio button 901 and the radio button 902 is selected, the CPU 201 of the PC 200 switches which alignment mode is used for adjustment. Information regarding the selected alignment mode is stored in its own RAM 202 by the CPU 201 of the PC 200.

  A drop-down list 903 in FIG. 9 is for selecting a reference projector. The projector that can be selected here is the projector that has been adjusted in S501 in FIG. The user selects one desired projector from the drop-down list, and the CPU 201 of the PC 200 stores the selected projector in its own RAM 202 as the reference projector.

  When the operation of the “confirm reference projector” button 904 in FIG. 9 is detected, the CPU 201 of the PC 200 issues a command for displaying a specific test pattern for each of the projectors selected in S501 of FIG. Send. At this time, it is possible to easily confirm which projection image on the projection plane is the reference projector by projecting an image whose color, brightness, or shape is different from other projectors only on the reference projector. Can do.

  When the operation of the “Back” button 905 in FIG. 9 is detected, the screen returns to the previous screen (the screen shown in FIG. 8). Although not shown in the flowchart of FIG. 5, the process proceeds from S505 to S504.

  When the operation of the “Next” button 906 in FIG. 9 is detected, the CPU 201 of the PC 200 starts an automatic adjustment process (corresponding to the transition from S504 to S505 in FIG. 5).

  FIG. 10 is a detailed flowchart of the automatic alignment process S506. By repeating S1002 to S1004 for each projector, the projective transformation matrices of each projector and camera to be aligned are acquired.

  A method of calculating the projective transformation matrix will be described with reference to FIG. FIG. 11A shows the camera coordinate plane (the coordinate system of the captured image), and FIG. 11B shows the projector coordinate plane (the coordinate system of the projector panel). When the coordinates on the projector coordinate plane are (xi, yi) and the coordinates on the camera coordinate plane are (Xi, Yi), the projective transformation formulas are expressed by Formula 3 and Formula 4 (i is a natural number). At this time, the variables having the same i correspond to each other.

  Here, a to h are predetermined constants.

  Formula 5 is a formula obtained by modifying Formula 3 and Formula 4 above and expressing the result in a matrix.

  M at this time is a projective transformation matrix. This projective transformation matrix has four corresponding points (x1, y1, X1, Y1), (x2, y2, X2, Y2), (x3, y3, X3, Y3) with respect to Equations 3 and 4. , (X4, y4, X4, Y4) can be respectively calculated. That is, if a correspondence relationship between at least four coordinates of planes is known, a projective transformation matrix can be calculated.

  For example, if the projector projects a quadrangle (the shaded area in FIG. 11B) and the vertex coordinates of the quadrangle projected from the projector in the captured image can be detected, the projective transformation matrix can be calculated. Description of the vertex coordinate detection method will be omitted because a known technique may be used.

  In FIG. 11, P_a, P_b, P_c, and P_d on the camera coordinate plane correspond to p_a, p_b, p_c, and p_d on the projector coordinate plane, respectively. A projective transformation matrix is calculated from the correspondence of these four known points. By using this calculated projective transformation matrix, projective transformation of unknown coordinate points can be executed. For example, the corresponding point p_e on the projector coordinate plane can be calculated by multiplying the point P_e on the camera coordinate plane by the calculated projective transformation matrix. Also, by multiplying p_e on the projector coordinate plane by the inverse matrix of the calculated projective transformation matrix, the corresponding point P_e on the camera coordinate plane can be calculated.

  Although the projective transformation matrix can be calculated by the above-described method, an image projected by the projector when shooting with the imaging apparatus 300 may not be a quadrangle. Any image may be used as long as it can obtain the correspondence of at least four coordinates between the camera coordinate plane and the projector coordinate plane.

  Returning to FIG. 10, the automatic alignment process will be described. The CPU 201 of the PC 200 repeats S1002 to S1004, calculates the projective transformation matrix of each projector, and stores it in its own RAM 202.

  In step S <b> 1005, the CPU 201 switches the deformation parameter calculation method according to the alignment mode information specified by the user stored in the RAM 202. In the present embodiment, the description of the “match with reference projector” mode modification (S1009) is omitted.

  If it is determined in S1005 in FIG. 10 that the “four-point designation adjustment” mode has been selected by the user, the process proceeds to “projection shape designation processing” in S1006.

  A detailed flowchart of “projection shape designation processing (S1006)” in FIG. 12 will be described. First, the CPU 201 of the PC 200 acquires the deformable area of each projector on the camera coordinate plane by repeating the processing of S1202 to S1203 for each projector. In step S <b> 1202, the CPU 201 of the PC 200 transmits a command requesting the keystone deformable amount to the projector 100 via the network IF 206. When the projector 100 receives a command requesting the deformable amount of keystone, it returns information including its own deformable amount of keystone to the PC. The PC 200 stores the keystone deformable amount received from the projector 100 in its own RAM 202.

Next, in step S1203, the CPU 201 of the PC 200 acquires a keystone deformable area on the projector coordinate plane, and obtains a deformable area on the camera coordinate plane by projective transformation. The keystone deformable area in the projector 100 is shown in FIGS. 13 (a) and 13 (b). 13A represents the keystone deformable region of the projector 100a, and the hatched portions 1311 to 1314 of FIG. 13B represent the keystone deformable region of the projector 100b. This keystone deformable area can be obtained from the panel resolution of each projector and the keystone deformable amount (Δx_max, Δy_max) of each projector. Taking the coordinates of the deformable area 1301 in the upper left as an example,
Upper left vertex coordinate of upper left deformable region 1301 = (0, 0),
Upper right vertex coordinates of the upper left deformable region 1301 = (Δx_max, 0),
Lower right vertex coordinates of upper left deformable region 1301 = (Δx_max, Δy_max),
Lower left vertex coordinates of upper left deformable area 1301 = (0, Δy_max)
It becomes.

  The CPU 201 of the PC 200 obtains the coordinates of the deformable area for all the projectors 100 to be aligned. Next, the CPU 201 of the PC 200 reads out the projective transformation matrix acquired in S1004 of FIG. 10 from the RAM 202, and projects the vertex coordinates of the convertible region onto the camera coordinate plane using the projective transformation matrix. FIG. 13C shows the result of projective transformation of the deformable regions of the projector 100a and the projector 100b on the camera coordinate plane. The hatched portions 1331-1334 represent the result of projective transformation of the keystone deformable region of the projector 100a on the camera coordinate plane, and the hatched portions 1341-1344 are the projective transform of the keystone deformable region of the projector 100b on the camera coordinate plane. Represents the result.

  In step S1204, the CPU 201 of the PC 200 acquires a common deformable area of a plurality of projectors on the camera coordinate plane. The common deformable area is calculated by applying a known mathematical method to the vertex coordinates of the polygon representing the deformable area of each projector. Shaded portions 1351 to 1354 in FIG. 13D are common deformable regions of a plurality of projectors on the camera coordinate plane. Specifically, a common area included in both the deformable area 1331 (1332-1334) of the projector 100a and the deformable area 1341 (1342-1344) of the projector 100b is a common deformable area 1351 (1352). 1354). In S1204, when the CPU 201 of the PC 200 determines that at least one of the common deformable areas does not exist, a warning message may be displayed on its own display unit 205. The warning message may display a specific work instruction (for example, “zoom the projector”) for the user. Furthermore, a warning message may be displayed, and the CPU 201 of the PC 200 may optically control (optical zoom or lens shift) each projector so that a common deformable area exists on the projection surface.

  Next, in S1205, the CPU 201 of the PC 200 selects any one of the alignment target projectors, and projects the common deformable area obtained in S1204 onto the coordinate plane of the selected projector.

  In S1206, the CPU 201 of the PC 200 generates a marker representing the common deformable area, and projects the marker on the projector selected in S1205.

  At this time, the CPU 201 of the PC 200 transmits a command via the network IF 206 so that projectors other than the common deformable area projection projector are in a non-projection state. As a result, it is possible to prevent other projectors from inhibiting the projection of the common deformable region onto the projection plane. FIGS. 14A and 14B show examples of markers projected onto the projection plane 1400. FIG. The marker representing the common deformable area is an image such as 1401 to 1404 in FIGS. 14A and 14B and is displayed as a rectangular hatched area. Note that any color, line thickness, pattern, or the like of the marker representing the common deformable region may be used. For example, any of the markers in the display form described in Patent Document 1 is used. May be used.

  In S1207, the CPU 201 of the PC 200 generates a marker for designating the deformed shape, and further projects it on the projector selected in S1205. The marker for designating the deformed shape is an image as shown in FIG. FIG. 14 shows a marker having four L-shaped markers (1406 to 1409) and four line segments (1411 to 1414) connecting the L-shaped markers. The projection area (projection image) of the projector 100a and the projection area (projection image) of the projector 100b after the automatic alignment process (after deformation) are aligned with each other so that one vertex is aligned. Form. The L-shaped marker is an image that represents the vertex of the superimposed area after the automatic alignment process. The shape and color of the marker may be any as long as it can specify at least four coordinates on the projection plane. In addition, as shown in FIG. 14A, the initial positions of the markers for designating the deformed shape are markers (1401 to 1404) in which L-shaped markers (1406 to 1409) represent common deformable regions. Is determined to be located inside. The position at this time may be, for example, the center of gravity of the common deformable region, or may be on a predetermined vertex of the common deformable region. Alternatively, it may be a characteristic position inside the common deformable region in the camera coordinate plane (for example, an intersection of a screen corner or a line segment projected by the laser marking device). In FIG. 14A, the L-shaped marker is arranged so that the entire L-shaped marker is included in the common deformable region, but corresponds to the apex of the overlapping region of the L-shaped marker. An L-shaped marker may be arranged so that only the portion is included in the common deformable region.

  Note that an image generated by the CPU 201 of the PC 200 may be transmitted via the network IF 206 or the video output unit 207 as a marker for indicating the common deformable region and a marker for designating the deformed shape. Alternatively, the CPU 201 of the PC 200 may transmit a command for causing the projector 100 to draw an arbitrary line segment or polygon via the network IF 206, and the CPU 101 of the projector 100 may draw the marker image based on the result of interpreting the received command. .

  When the processing of S1207 is completed, the CPU 201 of the PC 200 makes a transition to the GUI screen 1500 for designating the deformed shape shown in FIG.

  In FIG. 15, reference numerals 1501 to 1504 denote buttons for operating the marker 1406 at the upper left vertex. Further, reference numerals 1505 to 1508 denote buttons for operating the marker 1407 at the upper right vertex, reference numerals 1509 to 1512 denote buttons for operating the marker 1408 at the lower right vertex, and reference numerals 1513 to 1516 operate the marker 1409 at the lower left vertex. It is a button to do. These operation buttons are buttons for receiving an instruction to move the markers 1406 to 1409 in the direction of the arrow displayed on the button.

  An image area 1517 in FIG. 15 is an area for displaying a live view image taken and acquired by instructing the camera 300 to which the PC 200 is connected. The user directly operates the post-deformation shape designation marker by directly observing the projection surface or by operating one of the operation buttons 1501 to 1516 while viewing the image area 1517.

  When detecting the operation of the operation buttons 1501 to 1516, the CPU 201 of the PC 200 updates the position coordinates of the marker for designating the deformed shape on the camera coordinate plane stored in its own RAM 202 (S1210). Further, the CPU 201 regenerates the marker image of the updated position coordinate, and re-projects the regenerated marker image on the projector selected in S1205 (S1211). FIG. 14B shows a result of the user pressing the operation button 1503 and the operation button 1504 several times from the initial state shown in FIG. On the projection surface 1400 of FIG. 14B, the positions of the L-shaped marker portion 1406 and two line segments (1411, 1414) adjacent thereto are changed and displayed.

  Until the “execute” button 1519 in FIG. 15 is operated by the user, the operation buttons 1501 to 1516 can be operated. When the “execute” button 1519 in FIG. 15 is operated by the user (S1209—YES), the flowchart shown in FIG. 12 is terminated, and the process proceeds to S1007 in FIG.

  Returning to FIG. 10, the automatic alignment process will be described.

  In step S <b> 1007, the CPU 201 of the PC 200 acquires a keystone correction amount for forming a single superimposed image by completely overlapping the projection images of a plurality of projectors on the projection surface. The CPU 201 calculates the keystone correction amount of each projector by projecting the coordinates of the deformed shape in the camera coordinate plane stored in its own RAM 202 onto each projector plane using the projection transformation matrix acquired in S1004. To do.

  In step S1008, the CPU 201 of the PC 200 transmits the keystone correction amount calculated in step S1007 to each projector 100 via the network IF 206. When the projector 100 receives the keystone correction amount from the PC, the projector 100 transmits the received keystone correction amount to its own image processing unit 109 to perform shape correction of the input image.

  According to the present embodiment, when the keystone correction is performed on a plurality of projectors, the deformable range common to all projectors can be visualized, so that the convenience for the user can be improved.

  Even if a plurality of projectors of Patent Document 1 are prepared and the color or pattern of an image representing the deformable range is changed for each projector (for example, as shown in FIG. 13C), it is common to each projector for the user. A deformable range can be indicated. However, with this method, when the number of projectors used for stack projection is three or more, it becomes very difficult to identify the deformable range common to all projectors. However, in this embodiment, a deformable range common to all projectors is obtained in advance, and only the common deformable range is projected onto the projection plane as shown in FIG. Identification of the deformable range is facilitated.

  In the embodiment shown in Example 1, the common deformable area of each projector is acquired using a mathematical method, but the acquisition method is not limited to this. In the present embodiment, a method for acquiring a common deformable region of each projector using image processing will be described.

  Except for the method for acquiring the common deformable area of each projector, the configuration is the same as that of the first embodiment, and thus the description other than the method for acquiring the common deformable area of each projector is omitted.

  FIG. 16 is a detailed flowchart of S1006 of FIG. 10 shown in the first embodiment.

  First, the CPU 201 of the PC 200 generates a marker image based on the deformable amount by repeating S1601 to S1602. FIGS. 17A and 17B show examples of marker images based on the deformable amount. Reference numeral 1701 in FIG. 17 denotes a deformable area of the projector 100a. Reference numeral 1702 in FIG. 17 denotes a deformable area of the projector 100b. When generating the marker image of each projector, the CPU 201 of the PC 200 either changes the color of the deformable area for each projector or makes the color and brightness of the deformable area of each projector the same.

  By devising the marker image as described above, when the projectors simultaneously project the marker image, the following characteristics appear on the projection surface and the captured image.

  FIG. 17C shows a state in which each projector projects a marker image at the same time and photographs it. In FIG. 17C, reference numeral 1703 denotes an area where only the projector 100a projects the deformable area, and 1704 denotes an area where only the projector 100b projects the deformable area. Reference numeral 1705 denotes a region where both the projector 100a and the projector 100b project a deformable region, that is, a region included in both the region 1703 and the region 1704. When the color is changed for each projector, for example, if the projector 100a projects a red marker and the projector 100b projects a green marker, the region 1705, that is, a common deformable region of a plurality of projectors, is a mixed color yellow. In the case of the same color / brightness, only the area 1705 has a higher brightness than the other areas.

  By using these features, it is possible to acquire a common deformable region of each projector on the camera coordinate plane. When color information is used, a common deformable region can be obtained by applying a determination process for determining whether or not the hue of a pixel is close to a predetermined hue for all pixels of the captured image. In addition, when using luminance information, a common deformable region can be obtained by applying a determination process for determining whether or not the luminance of a pixel is equal to or higher than a predetermined threshold value for all pixels of the captured image. Here, the process using the hue and luminance is merely an example, and other pixel information such as brightness and saturation may be used.

  The CPU 201 of the PC 200 can calculate the common deformable area of each projector on the camera coordinate plane by executing the above processing in steps S1604 to S1606.

  The processing from S1607 to S1613 in FIG. 16 is the same as S1205 to S1211 in FIG.

  According to the present embodiment, as in the first embodiment, when the keystone correction is performed on a plurality of projectors, the deformable amount common to all projectors can be visualized, so that the convenience for the user is improved. Can do.

  Consider the case where the marker image drawing described in the “projection shape designation process” shown in FIG. 12 of the first embodiment is performed by a network command from the PC to the projector.

  In this process, first, the CPU 201 of the PC 200 transmits a command for causing the projector 100 to draw an arbitrary line segment or polygon via the network IF 206. Then, the CPU 101 of the projector 100 interprets the command received via the network IF 108 and stores the line segment or polygon according to the content in the VRAM 106.

  Here, in the first embodiment, the marker representing the common deformable area is projected by the projector selected in S1205 in S1206 of FIG. In step S1207, a marker for designating the deformed shape is projected by the same projector. However, each marker may be projected by a separate projector. In this embodiment, the method will be described.

  First, the difference between the markers 1401 to 1404 representing the common deformable region in FIG. 14 and the markers 1406 to 1407 for designating the deformed shape will be described. The markers 1401 to 1404 representing the common deformable area are determined by the projection position of each projector and the deformable area of each projector. That is, it is a marker whose position does not change from the start to the end of the “projection shape designation process”, and there is no need for successive updates. On the other hand, the markers 1406 to 1407 for designating the deformed shape are markers that need to be updated by a user operation as shown in S1209, S1210, and S1211 of FIG.

  If markers with different requirements are drawn on the CPU of a single projector in this way, it may take a long time to draw markers that do not need to be updated sequentially, and it may not be possible to draw markers that require immediate response with good response. There is. A method for solving this problem will be described with reference to FIG.

  FIG. 18 is a modification of the “projection shape designation process” of FIG. 12 shown in the first embodiment. First, when the projection shape designation process is started, the same processes as those shown in S1201 to S1204 in FIG. 12 are performed. Since this has already been described, details are omitted.

  In step S <b> 1801, the CPU 201 of the PC 200 transmits a command for acquiring marker drawing performance to each projector via the network IF 206. Receiving this, the projector 100 returns information indicating the marker drawing performance to the PC. This is, for example, the operating frequency of the CPU 101 of the projector 100. Then, when the CPU 201 of the PC 200 acquires the drawing performance of the marker from all the projectors, it determines the projector (hereinafter referred to as “PJ-A”) having the highest drawing performance from among the drawing performance.

  In step S1802, the CPU 201 of the PC 200 selects one of the projectors (hereinafter, PJ-B) other than the PJ-A determined in step S1801, and sets the common deformable area obtained in step S1204 for the coordinate plane of the projector. Project.

  In step S1803, the CPU 201 of the PC 200 generates a marker representing the common deformable area, and projects the marker onto the PJ-B selected in step S1802.

  In step S1804, the CPU 201 of the PC 200 generates a marker for designating the post-deformation shape, and projects the marker onto the PJ-A selected in step S1801.

  The subsequent steps are the same as S1208 to S1211 in FIG.

  According to the present embodiment, drawing of a “marker for designating a projection shape” that requires quick response is performed by a projector having the highest drawing performance, and “a marker representing a common deformable region” that does not need to be sequentially updated. This drawing can be performed by another projector. This makes it possible to present the update of the marker with high responsiveness to the user who operates the “marker for designating the projection shape”.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

  In addition, the technical elements described in the present specification or drawings exhibit technical usefulness alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. In addition, the technology illustrated in the present specification or the drawings achieves a plurality of objects at the same time, and has technical utility by achieving one of the objects.

100a Projector (first projection device)
100b Projector (second projection device)
200 PC (projection control device)
201 CPU (acquisition means, control means)
204 Operation unit (accepting means)
300 Imaging device (photographing means)
1331 to 1334 Deformable region (first region) of projector 100a
1341-1344 Deformable region (second region) of projector 100b
1351-1354 Common deformable area (common area)

Claims (9)

A projection control device that controls a plurality of projection devices including a first projection device that projects a first projection image and a second projection device that projects a second projection image,
Control means for controlling the first projection device and the second projection device;
Receiving means for receiving a user operation,
The control means causes the first projection device to project an image representing a common area in which the first projection image and the second projection image can be deformed, and transforms the first projection image and the second projection image. Projecting an image for designating a later shape on the second projection device,
The projection control apparatus, wherein the reception unit receives an operation of moving an image for designating the shape. The common area includes a first area in which the vertex of the first projection image is movable when the first projection image is deformed, and a vertex of the second projection image is movable when the second projection image is deformed. The projection control apparatus according to claim 1, wherein the projection control apparatus is an area included in both of the second areas. The image for designating the shape is an image representing a vertex of a superimposed image formed by superimposing the deformed first projection image and the deformed second projection image. The projection control apparatus according to claim 1 or 2. The said control means changes the position which projects the image for designating the said shape to the said 2nd projection apparatus based on the said operation, The Claim 1 thru | or 3 characterized by the above-mentioned. Projection control device. The projection control apparatus according to claim 1, wherein the second projection apparatus has higher drawing performance than the first projection apparatus. A projection apparatus comprising the projection control apparatus according to claim 1. A projection control apparatus according to any one of claims 1 to 5;
And a plurality of the projection devices. A projection control method for controlling a plurality of projection devices including a first projection device that projects a first projection image and a second projection device that projects a second projection image,
Projecting an image representing a common area in which the first projection image and the second projection image are deformable onto the first projection device;
Projecting an image for designating a deformed shape of the first projection image and the second projection image on the second projection device;
And a step of receiving an operation of moving an image for designating the shape.   The program for making a computer perform each step of the projection control method of Claim 8.

Images