JP2019110408A - Projection control device, control method of the same, and projection system - Google Patents

Abstract

To provide a projection control device capable of easily executing test imaging for automatic alignment processing at a time of multi-projection in a method according to a purpose, a control method of the same, and a projection system.SOLUTION: The projection control device that controls projection using a plurality of projection devices includes control means for controlling a first test shooting for confirming that a projection area where each of the plurality of projection devices projects an optical image on a projection plane is within a shooting range and second test shooting for determining shooting conditions when shooting the projection surface in order to automatically align the projection areas of a plurality of projection devices. The control means causes at least one of the number and timing of the projection devices to be turned on among the plurality of projection devices to be different between the first test shooting and the second test shooting.SELECTED DRAWING: Figure 10

Description

  The present invention relates to a projection control device, a control method therefor, and a projection system, and more particularly to a technique for adjusting a projection position.

  There is known a projection method (multi-projection) in which projection positions of a plurality of projection apparatuses are overlapped. In multi-projection, since projection position alignment between projection devices is necessary, a projection device having a function of facilitating alignment is also known. In Patent Document 1, in an image display apparatus having a plurality of image display means, one optical image is used as a reference image, and the formation position of the optical image is set so that the pixel position of the remaining optical image matches the pixel position of the reference image. One approach to automatic adjustment is disclosed.

JP 2008-225297 A

  In the case of automatically aligning the optical image, the projection plane may be photographed to recognize the projection area of each projector. In this case, if the imaging condition is inappropriate, the projection area may not be recognized from the image obtained by photographing the projection plane, or the recognition accuracy may be degraded. Therefore, before performing alignment, it is conceivable to perform test shooting for determining shooting conditions such as exposure conditions (test shooting for the purpose of checking shooting conditions).

  On the other hand, in order to recognize the projection area from the image obtained by photographing the projection plane, the projection area needs to be contained (included) in the imaging range (field angle). Therefore, it is conceivable to perform test shooting to confirm that the projection ranges of all the projectors performing alignment are within the shooting range (test shooting for the purpose of confirming the angle of view).

  In order to check the angle of view, it is sufficient to shoot the projection plane once while all the projectors are projecting. On the other hand, in order to confirm appropriate shooting conditions for individual projectors, it is necessary to shoot a projection plane while only one projector is projecting. Conventionally, no mechanism has been provided to easily switch and carry out such test imaging.

  The present invention has been made in view of the problems of the prior art as described above, and a projection control apparatus capable of easily executing test imaging for automatic alignment processing at the time of multi projection by a method according to the purpose It aims to provide a control method and a projection system.

  The above-mentioned object is a projection control device for controlling projection using a plurality of projection devices, in which a projection area for projecting an optical image on the projection surface of each of the plurality of projection devices is within the imaging range A first test shooting for confirming the image quality, and a second test shooting for determining shooting conditions at the time of shooting the projection surface in order to automatically align the projection areas of the plurality of projection apparatuses. It has control means to control, and the control means controls at least one of the number and timing of the number of the projection devices for which the projection state is turned on among the plurality of projection devices in the first test photographing and the second test photographing. This is achieved by a projection control device characterized in that it is different.

  According to the present invention, it is possible to provide a projection control device and its control method, and a projection system capable of easily executing test imaging for automatic alignment processing at the time of multi-projection in a method according to the purpose.

The schematic diagram which shows the structural example of the projection system which performs stack | stuck projection based on embodiment The schematic diagram which shows the structural example of the projection system which performs multi-screen projection based on embodiment Block diagram showing an example of functional configuration of the projection system according to the embodiment Diagram of keystone correction Flowchart of Outline of Automatic Registration Processing According to Embodiment A diagram showing an example of a GUI screen of a projection control application according to an embodiment A diagram showing an example of a GUI screen of a projection control application according to an embodiment A figure showing an example of a test pattern concerning an embodiment A diagram showing an example of a GUI screen for remote setting of a projection control application according to an embodiment Flow chart on test photographing processing according to the embodiment A figure showing an example of a test pattern concerning an embodiment Diagram of edge blending process Flowchart of automatic registration processing according to the embodiment Flowchart on reference projector confirmation processing according to the embodiment The figure which shows the example of the dialog which can be displayed by the reference | standard projector confirmation process which concerns on embodiment

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. The present invention is not limited to the embodiments described. Further, not all of the components described in the embodiments are necessarily essential to the present invention. Individual functional blocks in the embodiment can be realized by hardware, software, or a combination of hardware and software. Also, one functional block may be realized by a plurality of hardware. Also, one hardware may implement a plurality of functional blocks. Also, one or more functional blocks may be realized by one or more programmable processors (CPU, MPU, etc.) executing a computer program read into the memory. When one or more functional blocks are realized by hardware, they can be realized by discrete circuits or integrated circuits such as FPGAs and ASICs.

  In the following embodiments, a configuration in which the present invention is applied to a stand-alone type projector (projector) will be described. However, the present invention is also applicable to a projector incorporated in a general electronic device such as a personal computer, a smartphone, a tablet terminal, a game machine, and a digital (video) camera.

[System configuration of this embodiment]
FIG. 1 is a schematic view showing an example of a projection system according to an embodiment of the present invention. The projection system 10 performs stack projection in which projection areas on a projection surface of a plurality of projection devices (hereinafter, projectors) are aligned in order to expand the dynamic range of an optical image, improve the luminance, or display in 3D. Although FIG. 1 shows the projection system having the minimum number (two) of projectors 100a and 100b and matching the projection areas A and B, it may have three or more projectors.

  All projectors included in the projection system 10 are communicably connected to a personal computer (PC) 200 that functions as a projection control device. Communication between the plurality of projectors and the projection control apparatus may be wired communication or wireless communication, and the communication protocol is not particularly limited. In the present embodiment, as an example, communication between devices is performed in a local area network (LAN) using TCP / IP as a communication protocol. Further, the PC 200 can control the operation of the projectors 100a and 100b by transmitting predetermined commands to the projectors 100a and 100b. The projectors 100 a and 100 b operate according to the command received from the PC 200, and transmit the result of the operation to the PC 200.

  The video distributor 300 distributes the video signal output from the PC 200 to the projectors 100a and 100b. The video distributor 300 outputs the same video signal to all connected projectors. Here, the configuration at the time of adjustment before projection for viewing is shown, and the images projected by the individual projectors for viewing are separately supplied to the individual projectors from a reproduction apparatus or the like. Note that the video signal may be directly supplied from the PC 200 to the projectors 100a and 100b. Note that the video signal can be transmitted according to the standard of the display interface generally used. Examples of standards that can be used include HDMI (registered trademark), DVI, and VGA.

  The projection system 10 further comprises an imaging device 400, for example a digital camera. It is assumed that the imaging device 400 is grounded so as to include the entire projection plane as a shooting range at a position facing the projection plane. The imaging device 400 is communicably connected to the PC 200 directly or through a LAN. The PC 200 can control the operation of the imaging device 400 by transmitting a predetermined command to the imaging device 400. For example, the imaging device 400 can perform imaging in response to a request from the PC 200, and can transmit the obtained image data to the PC 200.

FIG. 2 is a schematic view showing another configuration example of the projection system according to the embodiment of the present invention, and the same components as in FIG. 1 are given the same reference numerals. The projection system 11 performs multi-screen projection which realizes an optical image of a large resolution (number of pixels) which can not be projected by one projector by arranging optical images projected by the respective projectors on a projection plane. Also in the configuration of FIG. 2, the video distributor 300 outputs the same video signal to all connected projectors. Images projected by the individual projectors for viewing are separately supplied to the individual projectors from a reproduction device or the like.
Although FIG. 2 shows a projection system having four projectors 100a to 100d, it may have more projectors. When multi-screen projection is performed, adjacent projection areas of the projection areas 1 to 4 of the projectors 100a to 100d are partially overlapped in order to make joints of individual optical images inconspicuous. In addition, in order to make the luminance increase in the overlapping portion inconspicuous, light reduction processing (edge blending processing) is performed. In the following description, “projector 100” represents all or any one of a plurality of projectors.

Further, terms used in the present specification are defined as follows.
"Projection area" An area occupied by the optical image projected by the projector 100 on the projection plane "Projected image" An optical image projected on the projection area "Image for projection" An image represented by a video signal or image data output from the PC 200 "Multi Projection "Projection using multiple projection devices" Stacked projection "Multi projection where the projection areas coincide or the projected image is completely superimposed" Multi screen projection "The projection areas are arranged so that some of the adjacent projection areas overlap Multi projection "Projector (projector)" A device that modulates the light from the light source based on the projection image and forms the projection image on the projection surface by projecting onto the projection surface or scanning on the projection surface

[Configuration of Projector 100]
FIG. 3 is a block diagram showing an example of the functional configuration of the projector 100 and the PC 200 included in the projection system 10 or 11. The projector 100 includes a CPU 101, a RAM 102, a ROM 103, a projection unit 104, a projection control unit 105, a VRAM 106, an operation unit 107, a network IF 108, an image processing unit 109, and an image input unit 110. These functional blocks are communicably connected by an internal bus 111.

The CPU 101 is an example of a programmable processor, and implements an operation of the projector 100 by, for example, loading a program stored in the ROM 103 into the RAM 102 and executing the program.
The RAM 102 is used as a work memory when the CPU 101 executes a program. The RAM 102 stores programs and variables used for executing the programs. Also, the RAM 102 may be used for other applications (for example, as a data buffer).

  The ROM 103 may be rewritable. The ROM 103 stores programs executed by the CPU 101, GUI data used for displaying menu screens and the like, test pattern data used for keystone correction, alignment processing, and the like, various setting values, and the like.

  The projection unit 104 includes a light source, a projection optical system, and the like, and projects an optical image based on the projection image supplied from the projection control unit 105. In this embodiment, a liquid crystal panel is used as an optical modulation element, and the reflectance or transmittance of light from the light source is controlled in accordance with the projection image to generate an optical image based on the projection image, and the projection optical system Project to

The projection control unit 105 supplies data of the projection image supplied from the image processing unit 109 to the projection unit 104.
The VRAM 106 is a video memory for storing data of a projection image received from the PC 200.

  The operation unit 107 includes input devices such as keys, buttons, switches, and a touch panel, and receives an instruction from the user to the projector 100. The CPU 101 monitors the operation of the operation unit 107, and when detecting the operation of the operation unit 107, executes processing according to the detected operation. When the projector 100 has a remote controller, the operation unit 107 notifies the CPU 101 of an operation signal received from the remote controller.

  A network IF 108 is an interface for connecting the projector 100 to a communication network, and has a configuration conforming to the standard of the communication network to be supported. In the present embodiment, the projector 100 is connected to a common local network with the PC 200 through the network IF 108. Therefore, communication between the projector 100 and the PC 200 is performed through the network IF 108.

The image processing unit 109 applies various image processing to the video signal supplied to the video input unit 110 and stored in the VRAM 106 as necessary, and supplies the same to the projection control unit 105. The image processing unit 109 may be, for example, a microprocessor for image processing. Alternatively, the function corresponding to the image processing unit 109 may be realized by the CPU 101 executing a program stored in the ROM 103.
Image processing applicable to the image processing unit 109 includes frame thinning processing, frame interpolation processing, resolution conversion processing, processing for overlapping OSDs such as a menu screen, keystone correction processing, edge blending processing, etc. It is not limited to.

  The video input unit 110 is an interface that directly or indirectly receives a video signal output from an external device (in this embodiment, the PC 200), and has a configuration according to the video signal to be supported. The video input unit 110 includes, for example, one or more of a composite terminal, an S video terminal, a D terminal, a component terminal, an analog RGB terminal, a DVI-I terminal, a DVI-D terminal, and an HDMI (registered trademark) terminal. When the video input unit 110 receives an analog video signal, the video input unit 110 converts it into a digital video signal and stores it in the VRAM 106.

[Configuration of PC 200]
Next, the functional configuration of the PC 200 will be described. The PC 200 may be a general purpose computer to which an external display can be connected, and thus has a functional configuration similar to the general purpose computer. The PC 200 includes a CPU 201, a RAM 202, a ROM 203, an operation unit 204, a display unit 205, a network IF 206, a video output unit 207, and a communication unit 208. Also, these functional blocks are communicably connected by an internal bus 209.

The CPU 201 is an example of a programmable processor, and implements operations of the PC 200 by, for example, reading a program (OS or application program) stored in the ROM 203 into the RAM 202 and executing the program.
The RAM 202 is used as a work memory when the CPU 201 executes a program. The RAM 202 stores programs and variables used to execute the programs. Also, the RAM 202 may be used for other applications (for example, as a data buffer).

  The ROM 203 may be rewritable. The ROM 203 stores programs executed by the CPU 201, GUI data used for displaying menu screens and the like, various setting values, and the like. Note that the PC 200 may have a storage device (HDD or SSD) having a larger capacity than the ROM 203, and in this case, a large-capacity program such as an OS or an application program may be stored in the storage device.

  The operation unit 204 includes input devices such as a keyboard, a pointing device (such as a mouse), a touch panel, and a switch, and receives an instruction from the user to the PC 200. The keyboard may be a software keyboard. The CPU 201 monitors the operation of the operation unit 204, and when detecting the operation of the operation unit 204, executes processing according to the detected operation.

  The display unit 205 is, for example, a liquid crystal panel or an organic EL panel. The display unit 205 displays a screen provided by the OS or the application program. The display unit 205 may be an external device. Also, the display unit 205 may be a touch display.

  A network IF 206 is an interface for connecting the PC 200 to a communication network, and has a configuration compliant with the standard of the supported communication network. In the present embodiment, the PC 200 is connected to a common local network with the projector 100 through the network IF 206. Therefore, communication between PC 200 and projector 100 is performed through network IF 206.

  The video output unit 207 is an interface for transmitting a video signal to an external device (in the present embodiment, the projector 100 or the video distributor 300), and has a configuration according to the video signal to be supported. The video output unit 207 includes, for example, one or more of a composite terminal, an S video terminal, a D terminal, a component terminal, an analog RGB terminal, a DVI-I terminal, a DVI-D terminal, and an HDMI (registered trademark) terminal.

  In the present embodiment, the UI screen of the projection control application program having the function of adjusting the projection area of the projector 100 is displayed on the display unit 205. However, the UI screen is displayed on the external device connected to the video output unit 207. It may be displayed.

  The communication unit 208 is, for example, a communication interface for performing serial communication with an external device, and is typically a USB interface, but may have a configuration conforming to another standard such as RS-232C. In the present embodiment, the imaging apparatus 400 is connected to the communication unit 208, but the communication method between the imaging apparatus 400 and the PC 200 is not particularly limited, and communication conforming to any standard supported by both is performed. be able to.

[Video distributor 300]
In the present embodiment, the PC 200, which is a projection control apparatus, performs alignment of individual projectors before multi-projecting an ornamental image. Accordingly, the video signal transmitted from the PC 200 to the individual projectors is a video signal for test (test pattern). Video signals projected for viewing are separately supplied to individual projectors. In the present embodiment, the video distributor 300 outputs the same video signal in parallel to all connected projectors.

[About keystone correction]
Next, keystone correction will be described with reference to FIG. The keystone correction geometrically transforms the original image so as to cancel out the trapezoidal distortion generated in the projection image according to the deviation between the normal direction of the projection surface and the projection direction (generally the optical axis of the projection optical system) It is a correction (geometrical correction) to be (deformed). Since the geometric transformation of the image can be realized by projective transformation, the keystone correction is equal to the determination of the parameters of the projective transformation which is the correction amount of the geometric correction. For example, the CPU 101 can determine a parameter of projective transformation based on the movement amount and movement direction of each vertex of the rectangular original image, and can provide the image processing unit 109.

ここで、Ｍは３×３行列で、元画像から変形後の画像への射影変換行列である。また、ｘｓｏ、ｙｓｏは、図４に実線で示す元画像の左上の頂点の座標であり、ｘｄｏ、ｙｄｏは、図４に一点鎖線で示す変形後の画像において、元画像の頂点（ｘｓｏ，ｙｓｏ）に対応する頂点の座標値である。 For example, assuming that the coordinates of the original image are (xs, ys), the coordinates (xd, yd) of the image after deformation by projective transformation are expressed by Equation 1 below.

Here, M is a 3 × 3 matrix, and is a projection transformation matrix from the original image to the image after deformation. Further, xso and yso are the coordinates of the upper left vertex of the original image shown by the solid line in FIG. 4, and xdo and ydo are the vertices (xso, yso) of the original image in the image after deformation shown by the dashed dotted line in FIG. It is the coordinate value of the vertex corresponding to).

The CPU 101 supplies the matrix M of Expression 1 and its inverse matrix M ⁻¹ together with the offsets (xso, yso) and (xdo, ydo) to the image processing unit 109 as parameters of keystone correction. The image processing unit 109 can obtain the coordinates (xs, ys) of the original image corresponding to the coordinate values (xd, yd) after the keystone correction in accordance with Equation 2 below.

  If the coordinates xs and ys of the original image obtained by Equation 2 are both integers, the image processing unit 109 directly coordinates the pixel value of the original image coordinates (xs, ys) as the coordinates (xd, yd) of the image after keystone correction. And the pixel value of. On the other hand, when the coordinates of the original image obtained by Equation 2 do not become integers, the image processing unit 109 interpolates the pixel value corresponding to the original image coordinates (xs, ys) using the values of a plurality of peripheral pixels. It can be asked. The interpolation operation can be performed using any of known interpolation operations such as bilinear and bicubic. If the coordinates of the original image obtained by Equation 2 are the coordinates of the external area of the original image, the image processing unit 109 sets the pixel value of the coordinates (xd, yd) of the image after the keystone correction to black (0 ) Or the background color set by the user. In this manner, the image processing unit 109 can obtain pixel values for all coordinates of the image after keystone correction, and create an image after conversion.

Here, although both the matrix M and its inverse matrix M ⁻¹ are supplied from the CPU 101 of the projector 100 to the image processing unit 109, only one of the matrices is supplied, and the other matrix is subjected to image processing. The part 109 may ask for it.

  Since keystone correction usually involves pixel interpolation, pixel information (such as RGB values) of the original image is lost particularly when the amount of deformation is large. Therefore, a smaller keystone correction amount (geometric correction correction amount) is advantageous in terms of image quality.

  The coordinates of the vertices of the image after the keystone correction are obtained, for example, by inputting the movement amount from the user through the operation unit 107 so that the vertices are projected at the desired positions for individual vertices of the projection image. Can. At this time, in order to support the input of the movement amount, the CPU 201 may cause the projector 100 to project a test pattern using the function of the projection control application program.

[Overview of Automatic Registration Processing]
An outline of the automatic alignment processing realized by the PC 200 of the present embodiment executing the projection control application program is shown in the flowchart of FIG.

  In S100, the CPU 201 selects, from among the projectors 100 with which the PC 200 can communicate, a plurality of projectors to be targets for adjustment processing (targets for adjustment). The plurality of projectors to be selected are one projector for projecting a reference image and one or more projectors for projecting an optical image to be aligned with the reference image. For example, as described later, a list of communicatable projectors may be displayed on the display unit 205 in a selectable manner, and the user may be allowed to select a plurality of projectors to be targets of automatic alignment processing. Also, the reference projector may be selected explicitly by the user or automatically. In the case of automatically selecting, for example, among the selected projectors, it may be considered that the display position in the list display is the top projector as the reference projector. For example, when an instruction to complete selection is input from the user, the CPU 201 advances the process to step S200.

  In S200, the CPU 201 transmits a command instructing to project a predetermined test pattern to each of the projectors 100 selected in S100 through the network IF 206. The CPU 101 of the projector 100 having received the command reads the test pattern data from the ROM 103 and causes the projection control unit 105 to project the optical image of the test pattern through the projection control unit 105. The test pattern to be projected here is a test pattern for allowing the user to grasp the positional relationship of the projection areas of the individual projectors 100, distortion of the projection image, and the like. For example, it may be a grid (mesh) pattern or the like.

  The user can grasp whether or not the projection area of the selected projector 100 is approximately at an appropriate position from the test pattern thus projected. Since the adjustable range is limited by the automatic alignment function, the user adjusts, for example, the installation position of the projector 100, the projection magnification, and the like so that the projection area of each projector 100 is approximately at the desired position at this time. Do.

  On the other hand, in step S300, the CPU 201 causes the display unit 205 to selectably display an imaging device connected to the PC 200 together with a message prompting the user to make a selection. Here, since only the imaging device 400 is usable, it is displayed in a state where the imaging device 400 is selected. When a test imaging instruction is detected through the operation unit 204 in a state in which the imaging apparatus is selected, the CPU 201 advances the process to step S400.

  In S400, the CPU 201 sets shooting conditions (angle of view, exposure conditions, white balance, etc.) for the imaging device 400 selected in S300. Exposure conditions and white balance settings can be made manually or automatically. The user can set directly or remotely from the PC 200 by operating the imaging apparatus 400 or operating the GUI provided by the projection control application using the operation unit 204. In the case of manual setting, common exposure conditions and white balance are used for all the projectors. The change of the angle of view of the imaging device 400 can be performed similarly to the manual setting of the exposure condition and the white balance. The process of S400 will be described in detail later.

  In step S500, the CPU 201 causes the display unit 205 to display a list of automatic alignment processing in a selectable manner. Details will be described later. When the execution instruction of the automatic alignment process is detected through the GUI operation provided by the projection control application, the CPU 201 advances the process to S600.

  In S600, the CPU 201 executes the selected automatic alignment process. The CPU 201 executes, for example, a process of automatically aligning the projection area of the projector selected in S300 for stack projection. Details of the process of automatically aligning the projection area of the projector selected in S300 will be described later.

  In addition, the execution order of S100 to S600 mentioned above may be different from FIG. For example, in the process of automatically aligning the projection area of the projector, the selection of the target projector, the selection of the imaging device 400, and the setting of the imaging conditions are completed when the start instruction of the automatic alignment process is issued. Just do it. For example, after performing the processing (S300, S400) related to the imaging device 400, the setting of the projector (S100, S200) may be performed.

  FIG. 6 is a view showing an example of the GUI screen 600 displayed on the display unit 205 when the CPU 201 executes the projection control application program. The user operates the GUI screen 600 through the operation unit 204 of the PC. 6 (a) and 6 (b) show a case where display related to stack projection is performed on a part of the lower side of the GUI screen 600 and a case where display related to multi-screen projection is performed. Are common except for

  A list view 601 is an area for displaying a list of information of the projector 100 connected to the PC 200 in a selectable manner. In the present embodiment, a list view 601 displays a list of projector names, IP addresses, and whether or not keystone correction is being applied. These pieces of information can be acquired from the projector 100 by transmitting an information acquisition command from the CPU 201 to each of the projectors 100. Further, in the present embodiment, the projector to which the keystone correction is applied is displayed as "deformed", and the projector to which the keystone correction is not applied is displayed as "deformed" or nothing is displayed.

  When the operation of the search button 602 is detected, the CPU 201 of the PC 200 broadcasts a predetermined command requesting information on the projector name, the IP address, and the application status of the keystone correction via the network IF 206 on the network. When the CPU 101 of each projector 100 connected to the network receives the command via the network IF 108, the data including the projector name, the IP address, and the keystone correction information is sent to the PC 200. Send. The CPU 201 of the PC 200 receives the transmitted data in response to the command, extracts the information contained in the data, and displays a list on the list view 601.

  The list view 603 is an area for displaying a list of projectors selected as targets of automatic alignment among the projectors listed and displayed in the list view 601. For example, when an operation of dragging and dropping the user on the list view 603 is detected for one or more of the elements listed in the list view 601, the CPU 201 changes the element targeted for the operation to the list view 603. to add. The CPU 201 manages the information of the projector displayed in the list view 603 on the RAM 202. The CPU 201 adds an element to the list view 603 even when an add button 605 described later is operated.

  A text box 604 and an add button 605 are GUIs for designating by the IP address and adding a projector that the user wants to add as a target of automatic alignment. The CPU 201 adds the projector having the IP address input to the text box 604 when the operation of the addition button 605 is detected to the list view 603 and the list of projectors managed by the RAM 202. The process of adding an element to the list view 603 corresponds to the process of S100 described above.

  In the present embodiment, when the projector added to the list view 603 or the projector selected by the list view 603 is applying the keystone correction, it can notify the user of it. The notification that the selected projector is applying the keystone correction can be performed, for example, by displaying a warning screen 700 shown in FIG. 7 on the display unit 205. That is, in the case where geometric correction such as keystone correction is applied to the projector selected as the target of automatic alignment, the warning screen 700 allows the user to select whether to cancel the application of the geometric correction. Notification screen. In addition to the display of the warning screen 700, the CPU 201 may transmit a command for projecting a predetermined test pattern to the corresponding projector when making a notification. As a result, a test pattern is projected from the projector 100 that is the target of notification. The user can confirm the position of the notified projection area of the projector by the projected test pattern.

  If the operation of the “Yes” button 701 on the warning screen 700 is detected, the CPU 201 requests the corresponding projector 100 via the network IF 206 to issue a command for requesting a correction amount of keystone correction (geometric correction) being applied. Send. In response to the command, the CPU 101 acquires the currently applied keystone correction amount from, for example, the RAM 102 and transmits the keystone correction amount to the PC 200. When receiving the correction amount, the CPU 201 stores the correction amount as information on the corresponding projector in the list managed by the RAM 202. Then, the CPU 201 further transmits, to the projector 100, a command instructing cancellation of the keystone correction. When receiving the command instructing cancellation of the keystone correction, the CPU 101 of the projector 100 instructs the image processing unit 109 to cancel the keystone correction. When the CPU 201 transmits a command instructing cancellation of the keystone correction, the warning screen 700 is closed. Then, the CPU 201 cancels (or changes to “no deformation” display) the “deformed” display displayed in the list views 601 and 603 for the projector for which the keystone correction has been canceled. In addition, the CPU 201 also updates information on whether or not keystone correction is applied to the list of projectors managed by the RAM 202.

  On the other hand, when the operation of the “No” button 702 on the warning screen 700 is detected, the CPU 201 closes the warning screen 700 without communicating with the corresponding projector 100. When the operation of the “No” button 702 on the warning screen 700 is detected, the CPU 201 may store the selected projector in the RAM 202 as a reference projector candidate. Therefore, the display of the list views 601 and 603 and the projector list in the RAM 202 are not changed. The display of the warning screen 700 when the projector is added to the list view 603 and the operation of canceling the keystone correction are not essential.

  Returning to FIG. 6, when the operation of the “test pattern display” button 606 is detected, the CPU 201 transmits a command instructing display of a test pattern to each of the projectors 100 displayed in the list view 603 through the network IF 206. Do. This corresponds to the process in S200 of FIG. The test pattern displayed according to the operation of the button 606 is a test pattern for making it easy to confirm the size and position of the display area of each projector 100. For example, the test shown in FIGS. 8A and 8B It may be a pattern. The two test patterns have different display (for example, color) of the rectangular portions 801 and 802 at the four corners.

  The test pattern may be transmitted from the PC 200 in association with a command instructing display of the test pattern to each projector 100, or may be generated by the CPU 101 of the projector 100.

  The test pattern can be generated so as to represent the upper limit value of the keystone correction in the projector depending on the size of the rectangular portion 801 or 802 of the test pattern displayed by each projector. For example, it is assumed that the upper limit value of the keystone correction of a certain projector 100 is 250 pixels in the X direction and 200 pixels in the Y direction. In this case, the test pattern can be generated such that the horizontal width of the rectangular portion 801 of the test pattern displayed by the projector 100 is 250 pixels and the vertical width is 200 pixels.

  When such a test pattern is generated by the PC 200, the CPU 201 acquires the upper limit value of the keystone correction from each of the projectors 100 displayed in the list view 603. Note that instead of directly acquiring the upper limit value of the keystone correction, other information (for example, a firmware version and a model name) that can be converted to the upper limit value of the keystone correction amount may be acquired. In this case, information (for example, a look-up table) for converting the information to be acquired into the upper limit value of the keystone correction amount is stored in the ROM 203 of the PC 200. Then, the CPU 201 generates image data representing the above-described test pattern based on the acquired upper limit value of the keystone correction.

  On the other hand, when a test pattern is generated by each projector 100, the CPU 101 generates image data representing the test pattern described above based on its own information stored in the ROM 103, for example. The test pattern described above may be stored in the ROM 103 in advance.

  For example, by making the representation (for example, color, fill pattern, etc.) of the rectangular portion in the test pattern projected by each projector different, is it possible to perform automatic alignment with the current projector arrangement based on the projected test pattern? It can support judgment of whether or not. For example, the rectangular portion 801 of the test pattern of FIG. 8A is red, and the rectangular portion 802 of the test pattern of FIG. 8B is green. Then, it is assumed that the projectors 100a and 100b project different test patterns.

  In this case, when the two test patterns are projected in a positional relationship in which the rectangular portions overlap, the overlapping area of the rectangular portions appears yellow. The overlapping area of the rectangular part of the corresponding position among the rectangular parts of the test pattern is an area in which any projector can move the vertex by keystone correction. Therefore, if the projected images of the test pattern have overlapping areas in all the rectangular parts at the same position (upper left, upper right, lower left, lower right), it is possible to match the projection areas of the projectors projecting those test patterns It can be seen that (that is, stack projection is possible). In addition, the upper limit of the keystone correction of the projector on which the test pattern is projected can be grasped from the rectangular part of the projected image of the test pattern.

  For example, if the projected images of the test pattern projected by the projectors 100a and 100b have a positional relationship as shown in FIG. 8C, it is understood that the projection areas of the projectors 100a and 100b can be automatically aligned. However, if the projected images of the test pattern have a positional relationship as shown in FIG. 8D, the projection areas can not be automatically aligned with the current arrangement of the projectors 100a and 100b. In this case, the user can move at least one of the projectors 100a and 100b such that all of the corresponding rectangular portions in the projected image of the test pattern have overlapping regions (yellow regions). Also, the user may move the projection position of at least one of the projectors 100a and 100b by the lens shift function so that all of the corresponding rectangular portions in the projected image of the test pattern have overlapping regions (yellow regions). it can.

  Returning to FIG. 6, the list view 607 of the GUI screen 600 displays a list of one of the imaging devices currently connected to the PC 200 in a selectable manner. The imaging device selected here is used for automatic alignment. Although four imaging devices are displayed in the list view 607 in the example of FIG. 6, in the present embodiment, only the imaging device 400 is connected, so the state in which the imaging device 400 is selected (highlighted state) It is displayed in).

  The CPU 201 establishes communication for remotely controlling the imaging device from the PC 200 through the communication unit 208 through the imaging device selected in the list view 607 (corresponding to the processing for selecting the imaging device in S300). As a result, it becomes possible to acquire various types of information from the selected imaging device, instruct photographing, acquire image data acquired by photographing, and set (change) photographing conditions. . The CPU 201 also stores information acquired from the imaging device in the RAM 202.

  Returning to FIG. 6, when the operation of the “set camera detail” button 608 is detected, the CPU 201 displays a GUI screen for remotely setting the imaging conditions of the imaging device selected in the list view 607. Displayed on 205. FIG. 9 shows an example of the remote control GUI screen 910 of the imaging apparatus. The remote control GUI screen 910 includes operation buttons corresponding to the type of imaging device, and the current setting values are displayed on each operation button. The aperture value button 912, the photographing sensitivity button 913, and the shutter speed button 914 each include a pull-down list, and when selected, a list of settable values is displayed. When a value is selected from the list, the CPU 201 transmits a command to change the exposure condition corresponding to the type of the button to the selected value to the imaging device through the communication unit 208. When the imaging instruction button 911 is operated, the CPU 201 transmits a command instructing imaging via the communication unit 208 to the imaging apparatus.

  When the operation of the “shooting condition automatic setting” button 609 is detected, the CPU 201 sets the shooting conditions (aperture value, exposure condition, white balance) of the imaging device selected in the list view 607 to values suitable for automatic alignment. Execute the process to set automatically. The details of this process will be described later together with the description of the test imaging operation.

  When the operation of the “test imaging” button 610 is detected, the CPU 201 executes test imaging processing by the imaging device selected in the list view 607. In the present embodiment, different processes are executed for the first test shooting for the purpose of confirming the angle of view of the imaging device and the second test shooting for the purpose of checking or automatically setting the shooting conditions for each projector.

  The test imaging process will be described with reference to the flowchart of FIG. When the CPU 201 detects the operation of the “test shooting” button 610 on the GUI screen 600 of the projection control application, it starts test shooting processing.

  In step S1501, the CPU 201 selects a radio button 627 as to whether the test shooting (first test shooting) for the purpose of confirming the angle of view or the test shooting (second test shooting) for the purpose of checking the shooting conditions. The determination is made based on the selection state of 628. Specifically, if the radio button 627 is selected, the CPU 201 determines that the purpose of the test shooting is the confirmation of the angle of view, and advances the process to step S1003. Also, if the radio button 628 is selected, the CPU 201 determines that the purpose of the test imaging is the imaging condition confirmation, and advances the process to S1021. Note that the photographing condition automatic setting process executed when the operation of the “photographing condition automatic setting” button 609 is detected corresponds to the process from S1021.

First, a test imaging operation for the purpose of confirming the angle of view will be described.
In step S <b> 1003, the CPU 201 acquires the keystone correction amount currently applied from each of the projectors to be adjusted (projectors displayed in the list view 603). If the keystone correction amount is already obtained when displaying the information of the projector connected to the list view 601, the keystone correction amount can be obtained from the information of the projector managed in the RAM 202. In this case, it is not necessary to transmit a command for requesting the keystone correction amount to the projector again in S1003.

  In step S1005, the CPU 201 transmits, via the network IF 206, a command instructing cancellation of keystone correction to each of the projectors to be adjusted. The CPU 101 of the projector that has received the command instructs the image processing unit 109 to cancel the keystone correction. Thereafter, the image processing unit 109 does not apply the keystone correction to the projection image data.

  In step S1007, the CPU 201 transmits a command instructing to turn on the projection state to each of the projectors to be adjusted through the network IF 206. This command may be, for example, a command to turn on the light source or a command to cancel projection of a blank (black screen).

  In step S1009, the CPU 201 transmits a command instructing projection of the test pattern to each of the projectors to be adjusted through the network IF 206. When the test pattern is supplied from the PC 200, the CPU 201 reads the image data of the test pattern from the ROM 203, for example, together with the command and transmits the image data to the projector. When using a test pattern that each projector has, the command can include information specifying the test pattern or information indicating the purpose of the test pattern. The test pattern for confirming the angle of view only needs to be able to confirm at least the outer edge of the projection area in the projected image. It may be an image in which the entire surface is filled, or it may be a frame-shaped image in which the outer edge is indicated by a solid line. Alternatively, for example, a test pattern having a rectangular area representing the upper limit value of the keystone correction amount of each projector as described with reference to FIGS. 8A and 8B may be used. Thus, the user can grasp not only whether the projection areas of all the projectors to be adjusted are within the angle of view (shooting range) but also whether or not alignment is possible by correction. .

  The CPU 101 of the projector that has received the command stores the test pattern image supplied or read out from the ROM 203 in the VRAM 106 and instructs the image processing unit 109 to start projection. Accordingly, the image processing unit 109 starts supplying the image data of the test pattern to the projection control unit 105. The projection control unit 105 controls the transmittance or reflectance of the optical modulation element of the projection unit 104 based on the image data of the test pattern, and projects the optical image of the test pattern.

  In step S1011, the CPU 201 instructs the imaging apparatus 400 to execute imaging via the communication unit 208. Thereby, the imaging apparatus 400 performs imaging and transmits the obtained image data to the PC 200.

  In step S1013, the CPU 201 stores, for example, the image data acquired from the imaging apparatus 400 in the RAM 202. Then, the CPU 201 displays the photographed image on the area 611 of the GUI screen 600 of FIG. From the image displayed in the area 611, the user can confirm whether all the projection areas of the projector to be adjusted fall within the angle of view of the imaging device 400. Note that the CPU 201 may automatically execute determination as to whether or not all the projection areas of the projector to be adjusted fall within the angle of view of the imaging device 400 by analyzing the image data obtained by shooting. . When it is determined that there is a projector whose projection area is not included in the angle of view of the imaging device 400, the CPU 201 displays a warning screen or the like to prompt the user to change the arrangement of the projector or the imaging device. It is also good.

  When the CPU 201 displays an image in the area 611 in S1013, the process proceeds to S1015. In S1015, the CPU 201 transmits a command instructing application of keystone correction to each of the projectors to be adjusted through the network IF 206. At this time, the CPU 201 transmits a command specifying the keystone correction amount acquired earlier, for each projector as a command transmission destination. As a result, each projector returns to the projection state before releasing the keystone correction in S1004. Here, it is assumed that the projector does not have the function of holding the keystone correction amount. However, if the projector has a function of holding the keystone correction amount, a command to enable keystone correction may be simply transmitted in S1015.

  This is the end of the test shooting process for the purpose of confirming the angle of view. When there is a projector whose projection area does not fit within the angle of view of the imaging device 400, the user adjusts at least one of the angle of view of the imaging device 400 and the position of the projector, and then performs test shooting for the purpose of checking the angle of view. Run again. Then, adjustment and test shooting are repeatedly performed until it can be confirmed that the projection areas of all the projectors to be adjusted fall within the angle of view of the imaging device 400.

  In addition, in order to perform automatic alignment correctly, it is desirable for the projection area of the projector of adjustment object to be accommodated in the view | field angle of the imaging device 400 by a certain size. Therefore, for example, the user adjusts the angle of view of the imaging device 400 and the position of the projector such that the size of the projection area occupied in the image obtained by photographing the projection plane is approximately 25% or more. Note that 25% is an example, and can be appropriately determined in consideration of the accuracy of automatic alignment and the like.

Next, a test shooting operation for the purpose of checking the shooting conditions (or automatically setting the shooting conditions) after S1021 will be described.
First, in step S1021, the CPU 201 transmits, through the network IF 206, a command instructing to turn on the projection state to any one of the projectors to be adjusted that has not been confirmed for the shooting conditions. This command may be the same as that transmitted in S1007. In test shooting for the purpose of checking shooting conditions, it is necessary to shoot while only one projector is projecting in order to check the shooting conditions for each projector. Therefore, unlike the test shooting for the purpose of confirming the angle of view, among the projectors to be adjusted, the projection state is turned on only to one of the shooting conditions.

  Although the case of confirming the shooting conditions one by one for all the projectors to be adjusted will be described here, the shooting conditions may be confirmed for only one of the projectors of the target conditions selected by the user. . In this case, a GUI for selecting one of the projectors to be adjusted (for example, a GUI for selecting one of the projectors displayed in the list view 603) is provided, and only for the selected one of the projectors. , S1021 and subsequent steps may be executed.

  In step S1023, the CPU 201 transmits a command instructing projection of the test pattern to the projector whose projection state has been turned on in step S1021 through the network IF 206. This command may be the same as the command transmitted in S1009. However, test patterns shall be the same as those projected during automatic alignment. For example, a different test pattern may be used for each projector, or the same image such as a full white image may be used as a test pattern.

  In step S1025, the CPU 201 transmits a command through the communication unit 208, and sets imaging conditions in the imaging apparatus 400. For example, when the shooting conditions for each of the projectors to be adjusted have already been determined by the operation of the “shooting condition automatic setting” button 608, the CPU 201 can read out and set the corresponding shooting conditions from the RAM 202. Alternatively, the CPU 201 may set the imaging device 400 to the shutter speed priority AE mode, and may further set the shutter speed to a value corresponding to the frame rate of the projector. In this case, the shutter speed can be set to, for example, a shutter speed (1 / frame rate [seconds]) equal to the reciprocal of the frame rate [frames / seconds].

  In step S1027, the CPU 201 instructs the imaging apparatus 400 to execute imaging via the communication unit 208. Thereby, the imaging apparatus 400 performs imaging and transmits the obtained image data to the PC 200.

  In step S1029, the CPU 201 stores, for example, the image data acquired from the imaging apparatus 400 in the RAM 202. Then, the CPU 201 causes the photographed image to be displayed in the area 611 of the GUI screen 600 of FIG. 6, and advances the processing to step S1031. When it is detected that the mouse pointer or the cursor is positioned on the image displayed in the area 611, the CPU 201 may display the luminance value or the like of the corresponding pixel.

  In step S1031, the CPU 201 extracts an image area corresponding to the projection area from the image data. Then, the CPU 201 generates information (statistics data) for determining imaging conditions from the image data in the extracted area, and stores the information in the RAM 202. The statistic data may be, for example, a histogram of luminance, an average luminance, a value of a white pixel, etc., which is generally used for automatic exposure control or auto white balance adjustment. The CPU 201 may determine whether the projection area is within the angle of view (shooting range), and may issue a warning if it is determined that the projection area is not within the angle of view (shooting range).

  In step S1033, the CPU 201 determines whether to automatically set imaging conditions. For example, when the processing from S1021 is being executed by the detection that the “shooting condition automatic setting” button 609 in FIG. 6 is operated, the CPU 201 determines that the shooting conditions are automatically set. On the other hand, when the test shooting for the purpose of checking the shooting conditions is being performed, the CPU 201 determines that automatic setting of the shooting conditions is not performed. Alternatively, the CPU 201 may inquire the user whether or not to automatically set shooting conditions for a projector whose projection state is currently on (projecting), and may branch the process according to the response result. For example, the user can be inquired by displaying on the display unit 205 an inquiry screen capable of selecting whether to perform automatic setting. In addition, when the test shooting for the purpose of automatic setting of the shooting conditions is being executed, the CPU 201 at S1033 applies the shooting conditions determined most recently to all the projectors and ends the test shooting. The user may be asked. If common shooting conditions can be applied to all the projectors, the accuracy may not be required to determine the shooting conditions separately for each projector.

  If it is determined that the automatic setting is to be performed (or instructed by the user), the process proceeds to step S1035. If it is determined that the automatic setting is not to be performed (or instructed by the user), the process proceeds to step S1037.

  In step S1035, the CPU 201 determines imaging conditions (exposure conditions and white balance) by a known method using the statistic data generated in step S1031. Then, the CPU 201 sets the determined imaging condition in the imaging apparatus 400 through the communication unit 208. Further, the CPU 201 stores the determined shooting condition in the RAM 202 as a shooting condition for a projector whose projection state is currently on (projecting). At this time, the CPU 201 updates the imaging conditions determined in the past, if any. Then, the CPU 201 advances the process to step S1039.

  On the other hand, in step S1037, the CPU 201 displays the statistic data generated in step S1031 in the GUI screen 600 or in another screen, and advances the process to step S1039. The process may proceed to step S1039 in accordance with the detection of the confirmation input that the process for the projector currently being projected may be completed. For example, the user can manually set the shooting conditions for the currently projected projector based on the statistic data displayed in S1037. The manual setting may be performed by directly operating the imaging apparatus 400 as described above, or may be performed remotely through a GUI screen displayed on the display unit 205.

  In step S1039, the CPU 201 determines whether confirmation (or determination and setting of the photographing conditions) of the photographing conditions has been completed for all of the projectors to be adjusted. If not, the process proceeds to S1041.

  In step S1041, the CPU 201 determines whether an instruction to end the process has been detected. If it is determined that the instruction is detected, the process ends. If it is not determined that the instruction is detected, the process returns to step S1021 and one of the remaining projectors Do the processing for Here, the end instruction may be an instruction to interrupt or forcibly end the processing through the operation unit 204. When there are a large number of projectors to be adjusted, the process of confirming (or determining and setting) the photographing conditions for all the projectors takes time. Therefore, the interruption of the process is accepted.

  The test imaging process described above is executed as part of the imaging condition setting process in S400. The number of projectors to be projected at the same time and the timing of projection are different when the purpose of the test shooting is the confirmation of the angle of view and the case of the shooting condition check. In this embodiment, only by specifying the purpose of the test shooting and instructing execution of the test shooting, the test shooting suitable for each purpose is performed, so that the appropriate test shooting can be performed while saving time and effort. Can.

  Returning to FIG. 6, the lower half of the GUI screen 600 of the projection control application will be described. The contents of automatic alignment differ between stack projection (FIG. 1) and multiscreen projection (FIG. 2). Therefore, the GUI screen 600 is configured to selectively display a setting area according to the type of multi-projection. Specifically, the CPU 201 displays a setting area for stack projection when a selection operation of the tab 613 is detected, and displays a setting area for multiscreen projection when a selection operation of the tab 614 is detected. FIG. 6A shows the setting area for stack projection, and FIG. 6B shows the setting area for multi-screen projection being displayed.

[Setting area for stack projection]
As described with reference to FIG. 1, stack projection is multi-projection in which a plurality of projection areas are projected to coincide with each other. Since a plurality of projection optical systems can not be arranged so that the optical axes become the same, at most one projector does not require keystone correction. In many cases, it is not easy to arrange the projector at a position directly facing the projection plane, so in practice, keystone correction is often performed on all the projectors. The alignment in the stack projection is a process of automatically determining the keystone correction amount of each projector so that the projection areas coincide with each other.

  In the stack projection setting area shown in FIG. 6A, reference numerals 615, 616, and 617 are radio buttons for exclusively selecting the automatic alignment mode. In the present embodiment, it is possible to select one of “four-point designation adjustment”, “automatic shape determination”, and “fit to a reference projector” as the automatic alignment mode for stack projection.

  The “four-point specification adjustment” is a mode in which a keystone correction amount is automatically determined such that the apexes of the individual projection areas are aligned with four predetermined points. For example, the CPU 201 movably displays adjustment markers 622, 623, 624, and 625 respectively corresponding to the upper left, upper right, lower right, and lower left corners of the projection area on the four-point adjustment area 621. The user can specify the coordinates of each vertex of the projection area by moving each adjustment marker, for example, by a drag and drop operation or a combination of a selection operation and a cursor operation. The four-point specification adjustment is useful, for example, when the projection target position is clear, as in the case where the projection plane is a framed screen. The number of adjustable points of coordinates may be smaller than four, or five or more including coordinates other than vertices.

  "Automatic shape determination" is a mode in which a keystone correction amount is automatically determined such that each projected image is rectangular. In this mode, the CPU 201 determines four points which are targets of alignment based on the photographed image of the projection area. Then, the CPU 201 determines a keystone correction amount that brings the apexes of the individual projection areas into alignment with the determined four points. Therefore, the user does not need to specify four points. Automatic shape determination is useful when the target position of projection is not clear (for example, projection to a wide wall surface).

  The “four-point specification adjustment” and “automatic shape determination” are processes for automatically determining a keystone correction amount to be matched with the projection areas of all the projectors in the projection areas predetermined by the user or the CPU 201. On the other hand, “adjust to the reference projector” is a mode in which one projector is used as a reference projector and a keystone correction amount is automatically determined such that the projection area of the reference projector matches the projection area of the other projector. The automatic alignment in this mode is performed when the position of the projection area of the reference projector is adjusted to the designated position. A keystone correction amount for causing a projection area of a projector other than the reference projector to coincide with a projection area of the reference projector is automatically determined.

  When detecting the selection operation of the radio button 617 of “Fit to standard projector”, the CPU 201 copies the information of the projector displayed in the list view 603 to the list view 618. The selection of the reference projector may be made automatically or may be made selectable by the user. For example, the projector listed at the top of the list view 618 may be set as a reference projector, and the order in the list may be changeable by a drag and drop operation or the like, but any other configuration may be adopted. Also, with reference to the list managed by the CPU 201 in the RAM 202, a projector with the smallest keystone correction amount can be automatically set as the reference projector. Furthermore, as described above, a projector whose keystone correction is not canceled may be used as a reference projector in response to a warning when selected in the list view 603.

  Note that, in order to support the user's selection of the reference projector, for example, with respect to the projector displayed in the list view 618, the projection indicating the projection area may be performed. For example, when detecting the operation of the “test pattern display” button 619, the CPU 201 projects a pattern image as shown in FIG. 11A, for example, for the reference projector among the projectors displayed in the list view 618. Further, for the remaining projectors displayed in the list view 618, the CPU 201 projects, for example, a pattern image as shown in FIG. Basically, any pattern image can be used as long as the outer edge of the projection area of all the projectors can be grasped and the projection area of one selected projector can be specified.

  FIG. 11C is a view schematically showing a projection image when information of one of the projectors is selected in a state where two projectors are displayed in the list view 618. The user can easily determine the projection area of the currently selected projector from this projection image. In addition, the relationship between the projection area of the currently selected projector and the projection areas of other projectors can be easily grasped. The pattern image may be transmitted from the PC 200 to the projector 100 as image data through the network IF 206, or may be transmitted as a video signal through the video output unit 207.

  When the operation of the “automatic adjustment start” button 620 is detected, the CPU 201 starts automatic alignment processing according to the selected radio button. Details of the automatic alignment process will be described later.

[Setting area for multi-screen projection]
As described with reference to FIG. 2, the multi-screen projection is a multi-projection in which a plurality of projection areas are arranged and arranged. Alignment in multi-screen projection is a process of automatically determining the keystone correction amount of each projector so as not to cause a shift of the overlapping portion of adjacent projection areas.

  Descriptions of matters common to stack projection will be omitted, and matters unique to multi-screen projection will be described. First, in the case of multi-screen projection, there is no “fit to the reference projector” in the automatic alignment mode. When the CPU 201 displays the setting area for multi-screen projection by detecting the selection operation of the tab 614, information of the projector displayed in the list view 603 is copied and displayed in the list view 631.

  The list view 629 provides a choice of arrangement pattern of the projection area of the multiscreen projection. The user can select one of the options. “2 × 2” represents an arrangement of 2 rows and 2 columns, “1 × 4” represents an arrangement of 1 rows and 4 columns, and “4 × 1” represents an arrangement of 4 rows and 1 column. The selectable arrangement pattern options change depending on the number of projectors displayed in the list view 631.

  An area 630 is a display schematically showing an arrangement pattern selected by the user in the list view 629. When the arrangement pattern is selected in the list view 629, the CPU 201 causes the area 630 to display an image corresponding to the selected arrangement pattern. By visualizing the arrangement pattern, it is possible to prevent, for example, the 1x4 arrangement from being erroneously arranged into the 4x1 arrangement.

  The list view 631 is an area in which the correspondence between the projection area constituting the arrangement pattern and the projector is changeably displayed. ID of the projection area (1 to 4 in FIG. 6B) included in the image of the arrangement pattern displayed in the area 630, and the text box 632 displayed in association with the information of each projector in the list view 631 From the contents, the user can grasp the correspondence. The contents of the text box 632 can be changed by the user. For example, when projecting the upper left area with “projector A 192.168.254.1”, “1” is input to the corresponding text box 632 . The ID of the projection area may be any character, symbol, mark or the like other than a number, as long as the same one is not assigned to a different projection area. The CPU 201 stores and manages an arrangement pattern and information indicating which projection area of each arrangement pattern is responsible for each projector.

  Text boxes 633 and 634 are areas for inputting the edge blend width in the vertical direction and the horizontal direction as the number of pixels. The CPU 201 stores the values input in the text boxes 633 and 634 in the RAM 202 as edge blend width information. The text boxes 633 and 634 can be displayed with default values input.

Edge blending processing will be described with reference to FIG. Here, the edge blending processing in the horizontal direction will be described, but the principle is the same in the vertical direction.
12A and 12B show projection images 1100a and 1100b of the first and second projectors. The projected image 1100a is composed of a non-overlapping area 1110a and an overlapping area 1120a. The projected image 1100b is also composed of a non-overlap area 1110b and an overlap area 1120b. The horizontal size of the overlapping area 1120 a, 1120 b corresponds to the edge blend width specified in the text box 634.

  FIG. 12C shows the relationship between the magnitude of the gain controlled by the edge blending process and the position in the horizontal direction of the image. Gains 1130 a and 1130 b are gains to be applied to the image processing unit 109 of the first projector and the second projector, respectively. In the non-overlapping areas 1110a and 1110b, since the increase in the brightness due to the image combination does not occur, the gain is set to 1.0 and the brightness of the image is not changed. On the other hand, if the gains of both the overlapping areas 1120a and 1120b are increased by 1.0, the luminance is increased and the overlapping areas of the projected images become noticeable. Therefore, for the overlap regions 1120a and 1120b, the gain is lowered linearly to 0 toward the edge of the image. At this time, the relationship of 1130a = 1.0-1130b is satisfied. The gain in the overlap region may be changed non-linearly.

FIG. 12D shows a projection image. In the overlapping area 1140 of the projection image, both the overlapping areas 1120 a and 1120 b are projected. For example, when projecting a uniform image, the luminance of the overlapping area 1140 is equal to the luminance of the non-overlapping areas 1110a and 1110b, so the boundary between the two projected images becomes inconspicuous. Note that edge blending processing is applied to both of the overlapping regions existing on the left and right (or upper and lower) with respect to the image projected on the projection region having the adjacent projection regions on both left and right or both up and down.
The operation according to the operation on the setting area for stack projection and the setting area for multiscreen projection described above is performed as part of the process of S500.

[Auto alignment process]
Next, the details of the automatic alignment process (S600) of FIG. 5 will be described using FIGS. The automatic alignment processing is started in response to the detection of the operation of the “automatic adjustment start” button 620 by the CPU 201.

In step S601, the CPU 201 performs confirmation processing of a reference projector. The confirmation process of the reference projector instructs the user with regard to performing the substantial alignment process (adjustment process) of S603 or later for the current reference projector when the “match to the reference projector” automatic alignment process is set. It is a process for asking for confirmation. Further, in the confirmation process, when the “match to the reference projector” automatic alignment process is set, the projectors other than the reference projector cancel the geometric correction currently applied. When the automatic alignment processing other than “match to the reference projector” is set, each projector cancels the geometric correction currently applied.
In step S602, the CPU 201 determines whether to execute the adjustment process in step S603 or later according to the result of the confirmation process of the reference projector.
Here, the confirmation processing of the reference projector in S601 will be described using the flowchart of FIG.
In step S1301, the CPU 201 determines whether the setting of the automatic alignment mode stored in the RAM 202 is “match to the reference projector”. If the CPU 201 determines that the setting is “match to the reference projector”, the process proceeds to step S1303. If not determined, the process proceeds to step S1302.

  In step S1302, the CPU 201 transmits a command instructing cancellation of keystone correction to all projectors to be adjusted (projectors displayed in the list view 603 in FIG. 6) through the network IF 206. Then, the CPU 201 ends the confirmation process of the reference projector. In this case, the return value of the process is "process start". The CPU 101 of the projector that has received the command instructs the image processing unit 109 to cancel the keystone correction.

  In step S1303, the CPU 201 transmits a command to each of the projectors to be adjusted (projectors displayed in the list view 618) through the network IF 206, and acquires the currently applied keystone correction amount. In addition, when displaying information of the projector connected to the list view 601, or if the keystone correction amount has already been acquired at the time of test shooting, obtain the keystone correction amount from the information of the projector managed in the RAM 202. Can. In this case, it is not necessary to transmit a command for requesting the keystone correction amount to the projector again in S1303.

  In step S1304, the CPU 201 transmits a command instructing cancellation of keystone correction to each of projectors to be adjusted other than the reference projector through the network IF 206. Thereby, the keystone correction is canceled in the target projectors other than the reference projector. The reference projector is selected by the user or automatically as described above.

  In step S1305, the CPU 201 causes the reference projector to project the pattern image shown in FIG. 11A, and causes the other target projectors to project the pattern image shown in FIG. As a result, the user can confirm the currently set reference projector and the projection area of another target projector. The pattern image may be projected only to the reference projector.

  In step S1306, the CPU 201 requests the user to approve the execution of the automatic alignment processing to align with the currently set reference projector. The CPU 201 can request approval by displaying a dialog message as shown in FIG. 15 on the display unit 205, for example. The CPU 201 waits until the operation of the “Yes” button 1401 or the “No” button 1402 included in the dialog message is detected.

  In step S1307, the CPU 201 determines whether or not the user has instructed that execution of the automatic alignment processing may be started. When detecting the operation of the “Yes” button 1401, the CPU 201 determines that the user has instructed that the execution of the automatic alignment processing may be started, and ends the reference projector confirmation processing. In this case, the return value of the process is "process start".

  On the other hand, when detecting the operation of the “No” button 1402, the CPU 201 determines that the user has instructed that the execution of the automatic alignment process is not possible, and advances the process to S1308. In step S1308, the CPU 201 transmits a command instructing application of keystone correction to each of the projectors other than the reference projector among the projectors to be adjusted through the network IF 206. At this time, the CPU 201 transmits a command specifying the keystone correction amount acquired earlier, for each projector as a command transmission destination. Thus, each projector returns to the projection state before releasing the keystone correction in S1304. Here, it is assumed that the projector does not have the function of holding the keystone correction amount. However, if the projector has a function of holding the keystone correction amount, in S1308, a command to enable keystone correction may be simply transmitted. After the process of step S1308, the CPU 201 ends the process of confirming the reference projector. In this case, the return value of the process is "process end".

  The inquiry to the user as to whether or not the automatic alignment processing may be executed is not limited to the one using the dialog message shown in FIG. In addition, after the end of S1307, the projection of the pattern image projected in S1305 may be ended.

  Before execution of automatic alignment processing for aligning the projection area (or projection image) of another projector with the projection area (or projection image) of the reference projector by confirmation processing of the reference projector, the keystone of a projector other than the reference projector Cancel the correction. Therefore, it is possible to eliminate the alignment to the wrong projection position due to the cancellation of the keystone correction of the reference projector performed in order to determine the projection position strictly, and the time and effort of the keystone correction again for the reference projector . On the other hand, when performing automatic alignment processing such as 4-point specification adjustment or automatic shape determination in which the reference projector is not set, keystone correction of all projectors to be aligned is performed before execution of the automatic alignment processing. To release. Therefore, it is possible to minimize the keystone correction amount or the number of corrections with respect to the original image, and it is possible to suppress the deterioration of the image quality of the projection image after the correction and the image quality of the projection image.

  In addition, before execution of the automatic alignment processing, a test pattern representing a projection area of the reference projector is projected to allow the user to confirm the setting of the reference projector. Therefore, for example, when the reference projector is automatically set, it is possible to prevent the execution of the alignment processing using the projector different from the user's intention as the reference projector. Also, before starting the automatic alignment process, it can be confirmed that the intended reference projector is set.

  Referring back to FIG. 13, in step S602, the CPU 201 determines whether the result (return value) of the confirmation processing of the reference projector is “processing start” or “processing end”, and is determined to be “processing start”. For example, the processing proceeds to step S603, and if it is determined that "processing is completed", the alignment processing is ended. Thus, the "match to reference projector" alignment process is not performed if the user does not approve the current reference projector.

  In step S603, the CPU 201 starts substantial automatic alignment processing (adjustment processing). The CPU 201 determines whether or not the adjustment test pattern has been projected and photographed for all of the projectors to be adjusted (including the reference projector, if any), and if it is determined that it has been determined, the process proceeds to S606. If not, the process proceeds to S604.

  In step S604, the CPU 201 projects a test pattern for alignment on one of the projectors to be adjusted. As described above, the projection of the test pattern may be instructed using a command to be transmitted through the network IF 206, or may be realized by supplying a video signal through the video output unit 207 and the video distributor 300. Note that the CPU 201 causes a projector other than the projector on which the test pattern is projected to project a black image or turns off the projection so that the projection area of the projector projecting the test pattern can be recognized from the photographed image on the projection surface Send a command The test pattern for alignment may or may not be common to all the projectors. For example, the test pattern may be made different in consideration of the installation position of the projector, the projection position of the projector in multi-screen projection, and the like.

  In step S605, the CPU 201 transmits a command instructing shooting to the imaging apparatus 400 used for alignment through the communication unit 208. When receiving the command, the imaging apparatus 400 performs imaging, and transmits the obtained image data to the PC 200. If necessary, a command for transmitting image data to the PC 200 may be separately transmitted from the CPU 201 to the imaging device 400. The CPU 201 stores the received image data in the RAM 202.

  In step S606, the CPU 201 as a recognition unit recognizes, from the image data stored in the RAM 202, the apex of the projection area of the projector currently projecting the test pattern and the coordinates thereof. Then, the CPU 201 as a control unit determines the keystone correction amount for the projector projecting the test pattern by a method according to the setting of the automatic alignment mode.

  For example, in the case of the automatic alignment processing for stack projection, the CPU 201 determines the keystone correction amount as follows. In the case of “four-point designation adjustment”, the CPU 201 determines a keystone correction amount for aligning the coordinates of the apex of the projection area with the coordinates of the four apexes designated by the user. Further, in the case of “automatic shape determination”, the CPU 201 determines the coordinates of the four vertices included in the range where the keystone correction is possible in all the projectors, and a keystone for aligning the vertices of the projection area with the coordinates. Determine the correction amount. In the case of “match to the reference projector”, the CPU 201 recognizes the four vertices of the projection area and their coordinates with respect to the reference projector, and does not determine the keystone correction amount. For projectors other than the reference projector, a keystone correction amount for adjusting the coordinates of the four apexes of the recognized projection area to the coordinates of the four apexes of the projection area of the reference projector is determined. The keystone correction amount can be determined by a known method, and the present invention does not depend on the determination method of the keystone correction amount. Therefore, the description of the determination method is omitted.

  In step S607, the CPU 201 transmits a command instructing application of keystone correction to each projector 100 through the network IF 206. This command contains the keystone correction amount of each vertex determined for the projector of the destination. Note that, in the case of “match to reference projector”, the CPU 201 does not transmit a command instructing application of keystone correction to the reference projector.

  The CPU 101 of the projector 100 that has received the command extracts the keystone correction amount from the command, and instructs the image processing unit 109 to apply the keystone correction to the projection image.

  In the present embodiment, when performing automatic alignment processing for aligning the projection area of another projector with the projection area of the reference projector, only the projectors other than the reference projector are keystone before performing imaging for recognizing the projection area. I made it to cancel the correction. Therefore, it is possible to eliminate the alignment to the wrong projection position due to the cancellation of the keystone correction of the reference projector performed in order to determine the projection position strictly, and the time and effort of the keystone correction again for the reference projector .

  Further, in the present embodiment, the projector for projecting and the timing of the projection are controlled so that the appropriate test imaging is performed according to the purpose of the test imaging. Therefore, the user can execute appropriate test imaging simply by specifying the purpose of the test imaging and instructing execution, and the time and effort can be largely saved.

Other Embodiments
In the above-described embodiment, assuming that the test pattern used at the time of alignment is not common to the projectors, imaging and imaging conditions are determined separately for each projector. However, when using the same test pattern for all the projectors, when performing test shooting for automatically setting the shooting conditions, typically only one projector is shot and the determined shooting conditions are all The present invention may be applied to a projector. Further, in the above-described embodiment, only the keystone correction for correcting trapezoidal distortion is described as the geometric correction to be applied by the projector, but in the case where the geometric correction for correcting barrel distortion or pincushion distortion is applied Can be treated the same as keystone correction.

  The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or computer readable storage medium, and one or more processors in a computer of the system or apparatus It can also be realized by a process of reading and executing a program. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

100a, 100b, 100c, 100d ... projector, 200 ... PC, 101, 201 ... CPU, 102, 202 ... RAM, 103, 203 ... ROM, 400 ... imaging device

Claims (12)

A projection control device that controls projection using a plurality of projection devices.
A first test shooting for confirming that the projection area on which the plurality of projection apparatuses project the optical image on the projection surface is within the imaging range, and the projection areas of the plurality of projection apparatuses Control means for controlling a second test shooting for determining a shooting condition at the time of shooting the projection plane in order to automatically align the
The control means makes at least one of the number and timing of the projection apparatuses for which the projection state is on be different among the plurality of projection apparatuses between the first test imaging and the second test imaging. Projection control device to be characterized.   In the first test shooting, the control means shoots with the projection state on for all of the plurality of projection devices, and in the second test shooting, turns on one projection state of the plurality of projection devices. The projection control apparatus according to claim 1, wherein the test shooting is controlled to perform shooting.   In the second test shooting, the control means controls the test shooting so that shooting with one projection state of the plurality of projection devices turned on is performed for each of the plurality of projection devices. The projection control device according to claim 2.   The projection according to any one of claims 1 to 3, wherein the control means further determines photographing conditions of the plurality of projection devices based on an image photographed in the second test photographing. Control device.   5. The projection control apparatus according to claim 4, wherein the control unit determines a photographing condition for one of the plurality of projection apparatuses, the projection state of which is on when the photographing is performed.   The control means sets, as the photographing condition common to the plurality of projection devices, the photographing condition determined for one of the plurality of projection devices whose projection state was on when the photographing was performed. The projection control device according to claim 5 characterized by the above.   The control means separately determines each of the plurality of projection apparatuses by determining an imaging condition for one of the plurality of projection apparatuses whose projection state was on when the photographing was performed. 6. The projection control apparatus according to claim 5, wherein a photographing condition is determined.   8. The image forming apparatus according to claim 1, wherein the control means causes the projection device, which turns on the projection state in the second test shooting, to project the same image as the image projected by the projection device in the automatic alignment processing. The projection control device according to any one of the above. With multiple projection devices,
A projection control apparatus according to any one of claims 1 to 8.
An imaging device for imaging the projection plane;
The projection system characterized by having. A control method of a projection control apparatus for controlling projection using a plurality of projection devices, comprising:
A first test shooting for confirming that the projection area on which the plurality of projection apparatuses project the optical image on the projection surface is within the imaging range, and the projection areas of the plurality of projection apparatuses Control step of controlling a second test shooting for determining a shooting condition at the time of shooting the projection plane in order to automatically align the
In the control step, at least one of the number and timing of the projection apparatuses for which the projection state is turned on among the plurality of projection apparatuses is made different between the first test imaging and the second test imaging. The control method of the projection control device to be characterized.   The program for functioning a computer as each means of the projection control apparatus of any one of Claim 1 to 8.   A computer readable storage medium storing a program for causing a computer to function as each means of the projection control device according to any one of claims 1 to 8.

Images