JP2016042653A - Image processing system, and image projection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system operable to project a predetermined image with image projection devices, of which the installation, setting, etc. are facilitated while holding down the cost.SOLUTION: An image processing system is arranged to project a predetermined image on a projection plane with image projection devices each including projection means for projecting an image on the projection plane, and image-taking means for taking the image projected on the projection plane. The image processing system comprises: control means for controlling the projection by the projection means of the image projection devices; measurement means for measuring a three-dimensional position of the projection plane for each image projection device on the basis of an image taken under the control, and measurement parameters of three-dimensional measurement by the projection means and the image-taking means of the image projection devices; and integration means for integrating the measured three-dimensional positions. The predetermined image is projected on the projection plane on the basis of the three-dimensional position of the projection plane resulting from the integration.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an image processing system and an image projection apparatus.

  Multi-projection (multi-projection) is a technique for projecting a large screen using a plurality of projectors (image projection devices). Compared to displaying a large screen using a single display, a large screen can be produced at a lower cost. There is a merit that it can be realized.

  As a technology related to this, there is known an integrated device that uses a plurality of projectors and a plurality of stereo cameras to display images from a plurality of projectors on a quadratic curve screen without distortion from an arbitrary starting point (for example, Patent Documents). 1).

  The technique disclosed in Patent Document 1 is disadvantageous in terms of cost because it uses a plurality of stereo cameras, and since a plurality of projectors and a plurality of stereo cameras are separate systems, system installation and setting It was accompanied by difficulties.

  An embodiment of the present invention has been made in view of the above problems, and in an image processing system that projects a predetermined image using a plurality of image projection apparatuses, the system is installed while suppressing the cost of the system. An object of the present invention is to provide an image processing system that facilitates setting and the like.

  In order to solve the above problems, an image processing system according to an embodiment of the present invention is an image processing system that projects a predetermined image on a projection surface using a plurality of image projection devices, and the plurality of image projection devices. Each of which includes a projecting unit that projects an image on the projection surface, and a photographing unit that is provided at a predetermined position with respect to the projection unit and captures the image projected on the projection surface. The processing system includes: a control unit that controls projection by the projection unit of the plurality of image projection devices; and shooting by the shooting unit of the plurality of image projection devices; an image shot by the control; A measuring hand that measures a three-dimensional position of the projection plane with respect to each of the plurality of image projection devices based on measurement parameters of three-dimensional measurement by the projection unit and the photographing unit of the image projection device. If, anda integration means for integrating a plurality of three-dimensional position where the measured, on the basis of the three-dimensional position of the integrated the projection plane, it projects the predetermined image on the projection plane.

  According to an embodiment of the present invention, in an image processing system that projects a predetermined image using a plurality of image projection apparatuses, an image processing system that facilitates installation, setting, etc. while suppressing system cost is provided. can do.

1 is a diagram illustrating a configuration example of an image processing system according to an embodiment. It is a figure which shows the hardware structural example of the projector which concerns on one Embodiment. It is a figure which shows the hardware structural example of PC which concerns on one Embodiment. 1 is a functional configuration diagram of an image processing system according to a first embodiment. 6 is a flowchart illustrating a flow of a calibration image photographing process according to the first embodiment. It is a figure for demonstrating imaging | photography of the image for a calibration which concerns on 1st Embodiment. It is a flowchart of an example of the calibration process which concerns on 1st Embodiment. It is a figure for demonstrating the definition of the symbol which concerns on 1st Embodiment. It is a figure for demonstrating calculation of the three-dimensional coordinate which concerns on 1st Embodiment. It is a figure which shows the example of the synthesized pattern image which concerns on 1st Embodiment. It is a flowchart of an example of the image projection process which concerns on 1st Embodiment. It is a figure which shows the example of the connection form of the image processing system which concerns on 1st Embodiment. It is a functional block diagram of the image processing system which concerns on 2nd Embodiment. It is a flowchart of an example of the calibration process which concerns on 2nd Embodiment. It is a flowchart of an example of the image projection process which concerns on 2nd Embodiment. It is a functional block diagram of the image processing system which concerns on 3rd Embodiment. 10 is a flowchart illustrating an example of a calibration image capturing process according to a third embodiment. It is a flowchart which shows the example of the calibration process which concerns on 3rd Embodiment. 10 is a flowchart illustrating an example of image processing according to a third embodiment. It is a figure which shows the example of the connection form of the image processing system which concerns on 3rd Embodiment. It is a figure which shows the structural example of the image processing system which concerns on 4th Embodiment. It is a figure for demonstrating the area | region where the projection image which concerns on 4th Embodiment is produced | generated.

  Embodiments of the present invention will be described below with reference to the accompanying drawings.

<System configuration>
FIG. 1 is a diagram illustrating a configuration example of an image processing system according to an embodiment. The image processing system 100 includes a plurality of projectors (image projection apparatuses) 101-1, 101-2, 101-3, a PC (Personal Computer) 102, and the like. In the following description, “projector 101” is used to indicate an arbitrary projector among the plurality of projectors 101-1 to 101-3. Further, the number of the plurality of projectors 101-1 to 101-3 is an example, and may be another number of two or more.

  The projector 101 includes a projecting unit 105 that projects a projection image onto the projection surface 104, and an imaging unit 106 that is provided at a predetermined position with respect to the projection unit 105 and photographs the projection image projected onto the projection surface 104. Preferably, the projector 101 includes a storage unit that stores in advance measurement parameters for three-dimensional measurement by the projection unit 105 and the photographing unit 106.

  Further, for example, as shown in FIG. 1, the projector 101-1 is arranged so that the projection area 103-1 of the projector 101-1 overlaps a part of the projection area 103-2 of the projector 101-2. . Similarly, as shown in FIG. 1, the projector 101-3 is arranged so that the projection area 103-3 of the projector 101-3 overlaps a part of the projection area 103-2 of the projector 101-2. . That is, each of the plurality of projectors 101-1 to 101-3 is arranged so that at least a part of the projection area overlaps with the adjacent projector 101. With this arrangement, each projector 101 projects a predetermined image to be projected onto the projection surface 104 in cooperation with the other projectors 101.

  The PC 102 is connected to a plurality of projectors 101-1 to 101-3 using, for example, a wired / wireless LAN (Local Area Network), short-range wireless, HDMI (registered trademark), USB (Universal Serial Bus), etc. Control signals, image data, etc. can be transmitted and received. The PC 102 uses a plurality of projectors 101-1 to 101-3, for example, to control multi-projection (multi-projection) in which a projection screen larger than the projection area of each projector 101 is projected onto the projection surface 104.

  In the above configuration, the PC 102 controls projection of images by the plurality of projectors 101-1 to 101-3 and shooting of the projected images. Further, the PC 102 calculates the three-dimensional position of the projection plane 104 with respect to each projector 101 based on the photographed image and the measurement parameters of the three-dimensional measurement stored in advance by each projector 101. Further, the PC 102 integrates the calculated three-dimensional position of the projection plane 104 with respect to each projector 101 into one coordinate system.

  Further, the PC 102 calculates image processing parameters of each projector 101 based on the integrated three-dimensional position of the projection plane 104 and notifies the calculated parameters to each projector 101. Based on the notified image processing parameters, each projector 101 performs image processing (for example, extraction of a projected portion, deformation, brightness adjustment) on a predetermined image to be projected, and projects the image after image processing Project onto the surface 104.

  Through the above processing, a predetermined image to be projected is multi-projected on the projection plane 104.

  As described above, the image processing system 100 according to the present embodiment includes the projection unit 105 of each projector 101, the photographing unit 106, and the projection surface 104 of each projector 101 based on the measurement parameters of the three-dimensional measurement stored in advance. A three-dimensional position is calculated. Further, the image processing system 100 identifies the three-dimensional position of the projection plane 104 by integrating the three-dimensional position of the projection plane 104 with respect to each projector into one coordinate system.

  Thereby, the image processing system 100 according to the present embodiment is based on expensive shooting means such as a stereo camera, a wide-angle and high-resolution camera that can shoot the entire projection surface 104, and operations such as setting and adjustment of the camera. In addition, the three-dimensional position of the projection plane 104 can be specified. Therefore, according to the image processing system 100 according to the present embodiment, in the image processing system 100 that projects a predetermined image using a plurality of projectors (image projection apparatuses) 101, installation and setting are performed while suppressing the cost of the system. Etc. can be facilitated.

  Note that the system configuration in FIG. 1 is merely an example, and does not limit the scope of the present invention. For example, the image processing system 100 may perform stack projection in which projection images of a plurality of projectors 101 are superimposed and a high-brightness screen brighter than the projection screens of the projectors 101 is projected.

  Further, the measurement parameters of the three-dimensional measurement by the projection unit 105 and the imaging unit 106 of each projector 101 may be stored in advance by the PC 102, an external server, an external storage device, or the like, for example.

  Furthermore, the function of the PC 102 may be realized by a dedicated control device or the like, or may be realized by one or more projectors 101.

<Hardware configuration>
(Hardware configuration of projector)
FIG. 2 is a diagram illustrating a hardware configuration example of the projector according to the embodiment. The projector 101 includes a general computer configuration, and includes, for example, a CPU (Central Processing Unit) 201, a memory 202, a storage unit 203, a communication I / F (Interface) 204, an external I / F 205, a light source 206, and a display element 207. A projection lens 208, a motor 209, an imaging lens 210, an imaging element 211, a bus 212, and the like. Further, the projector 101 may include an image processing unit 21 that speeds up image processing.

  The CPU 201 is an arithmetic device that implements each function of the projector 101 by reading a program and data from the memory 202, the storage unit 203, and the like and executing processing. The memory 202 includes, for example, a RAM (Random Access Memory) used as a work area of the CPU 201, a ROM (Read Only Memory) that stores a startup program of the projector 101, and the like.

  The storage unit 203 is a storage unit that stores a program of the projector 101, various data, and the like, and includes, for example, a flash ROM, an SSD (Solid State Drive), an HDD (Hard Disk Drive), or the like.

  The communication I / F 204 is an interface for communicating with another projector 101, the PC 102, and the like, and includes, for example, a communication interface such as a wired / wireless LAN, short-range wireless communication, and LTE (Long Term Evolution).

  An external I / F 205 is an interface for transmitting / receiving image data and the like to / from another projector 101, the PC 102, and the like. The external I / F 205 includes interfaces such as USB (Universal Serial Bus) and HDMI, for example.

  The light source 206 is a light source for the projector 101 to project an image, and includes, for example, a light source such as a lamp, a laser, and an LED (Light Emitting Diode).

  The display element 207 includes, for example, a DMD (Digital Mirror Device) 214, a color wheel 215, and the like. The DMD 214 is a display element in which a large number of micromirror surfaces (micromirrors) are arranged on a plane. The color wheel 215 performs coloring by rotating and transmitting a disk or the like color-coded into three colors of red, blue, and green at high speed, for example.

  The light emitted from the light source 206 is reflected by the DMD 214 whose direction is controlled for each pixel according to the image to be projected, is colored by the color wheel 215, and is irradiated onto the projection plane 104 through the projection lens 208 to form an image. A motor 209 is a motor that drives the projection lens 208 and performs adjustments such as zoom and focus. The DMD 214 and the color wheel 215 are examples of the display element 207. The display element 207 may be another display element such as an LCD (Liquid Crystal Display).

  The image projected on the projection surface 104 is picked up by the image pickup lens 210 and the image pickup device 211, and the picked-up image is stored in the memory 202, the storage unit 203, and the like.

  In addition, the projector 101 may include an image processing unit 213 for performing image processing of an image projected by the projector 101. The image processing unit 213 is realized by, for example, an image processing processor, a DSP (Digital Signal Processor), or dedicated hardware. Further, the projector 101 may not perform the image processing unit 213 but may perform image processing of an image to be projected by, for example, a program operating on the CPU 201.

  The bus 212 is connected to each of the above components and transmits an address signal, a data signal, various control signals, and the like.

(PC hardware configuration)
FIG. 3 is a diagram illustrating a hardware configuration example of a PC according to an embodiment. The PC 102 has a general computer configuration, and includes, for example, a CPU 301, a RAM 302, a ROM 303, a storage unit 304, a communication I / F 305, an external I / F 306, a display device 307, an input device 308, a bus 309, and the like. .

  The CPU 301 is an arithmetic device that reads programs and data from the ROM 303, the storage unit 304, and the like onto the RAM 302 and executes processing. A RAM 302 is a non-volatile memory used as a work area for the CPU 301. The ROM 303 is a non-volatile memory that stores, for example, a basic input / output system (BIOS) executed when the PC 102 is activated, various settings, and the like, and is configured by, for example, a flash ROM.

  The storage unit 304 is a non-volatile storage device that stores, for example, an OS (Operating System), application programs, various data, and the like, and includes, for example, an HDD, an SSD, or the like.

  A communication I / F 305 is an interface for performing communication by connecting the PC 102 to a plurality of projectors 101-1 to 101-3, for example, wired / wireless LAN, near field communication, LTE (Long Term Evolution), etc. Communication interface.

  The external I / F 306 is an interface for transmitting / receiving image data and the like to / from a plurality of projectors 101-1 to 101-3. The external I / F 306 includes interfaces such as USB (Universal Serial Bus) and HDMI, for example.

  The display device 307 is realized by, for example, an LCD display and displays a processing result by the PC 102 and the like. The input device 308 is realized by, for example, a keyboard, a mouse, a touch panel, and the like, and is used for inputting each operation signal to the PC 102. A bus 309 is connected to each of the above components and transmits an address signal, a data signal, various control signals, and the like.

<Functional configuration>
[First Embodiment]
FIG. 4 is a functional configuration diagram of the image processing system according to the first embodiment.

(Functional configuration of projector)
In FIG. 4, a plurality of projectors 101-1 to 101-3 have the same configuration. Each projector 101 includes a projection unit 105, a photographing unit 106, a storage unit 401, an image processing unit 402, a communication unit 403, and the like.

  The projection unit 105 is a unit for projecting an image on the projection plane 104, and includes, for example, the light source 206, the display element 207, the projection lens 208, and the like shown in FIG.

  The photographing unit 106 is a unit for photographing an image projected on the projection plane 104, and includes, for example, the imaging lens 210 and the imaging element 211 shown in FIG. In the present embodiment, the photographing unit 106 is provided at a predetermined position with respect to the projection unit 105. Further, the photographing unit 106 may not be able to photograph the entire projection surface 104 as long as the image projected by the projection unit 105 onto the projection surface 104 can be photographed. In the present embodiment, a monocular camera is assumed as the photographing means 106, but the scope of the present invention is not limited.

  The storage unit 401 is a storage unit that stores in advance measurement parameters of the three-dimensional measurement performed by the projection unit 105 and the imaging unit 106, and is included in, for example, the storage unit 203 and the memory 202 in FIG. Measurement parameters for three-dimensional measurement will be described later.

  For example, the image processing unit 402 performs image processing (processing) on an image projected by the projector 101 using the image processing parameter stored in the storage unit 401 or the image processing parameter notified from the PC 102 or another projector 101. Means to do. The image processing unit 402 is realized by, for example, the image processing unit 213 in FIG. 2 or a program that operates on the CPU 201 in FIG.

  The communication unit 403 is a unit for transmitting / receiving control data, image data, and the like to / from the PC 102, another projector 101, and the like, and includes, for example, the communication I / F 204 and the external I / F 205 in FIG.

  With the above configuration, the projector 101 can project an image onto the projection surface 104 by the projection unit 105 and can capture the projected image with the imaging unit 106. The photographing unit 106 is provided at a predetermined position with respect to the projecting unit 105, and the storage unit 401 stores measurement parameters for three-dimensional measurement by the projecting unit 105 and the photographing unit 106. Accordingly, the image processing system 100 measures the three-dimensional position (for example, coordinates) of the projection surface 104 with respect to the projector 101 based on the measurement parameters stored in advance in the projection unit 105, the imaging unit 106, and the storage unit 401. be able to.

(PC functional configuration)
The PC 102 includes a control unit 404, a three-dimensional position measurement unit 405, a three-dimensional position integration unit 406, a parameter calculation unit 407, a communication unit 408, a storage unit 409, and the like.

  The control unit 404 controls projection of images by the plurality of projectors 101-1 to 101-3 and shooting of images projected on the projection plane 104 by the plurality of projectors 101-1 to 101-3. In addition, the control unit 404 performs control to project a predetermined image (for example, a large screen image) on the projection surface 104 using the plurality of projectors 101-1 to 101-3.

  A three-dimensional position measuring unit (measuring unit) 405 projects to each projector 101 based on a plurality of images taken by the control of the control unit 404 and measurement parameters stored in advance in the storage unit 401 of each projector 101. The three-dimensional position of the surface 104 is measured. The measurement parameters stored in advance in the storage unit 401 include, for example, information corresponding to the baseline length of triangulation by the projection unit 105 and the imaging unit 106, and the three-dimensional position measurement unit 405 includes information corresponding to the baseline length. Based on the above, the three-dimensional coordinates of the projection surface 104 are measured.

  A three-dimensional position integration unit (integration unit) 406 integrates a plurality of three-dimensional positions measured by the three-dimensional position measurement unit 405. For example, the three-dimensional position integration unit 406 integrates the three-dimensional position of the projection plane 104 with respect to each projector 101 into one coordinate system.

  A parameter calculation unit (calculation unit) 407 calculates an image processing parameter (correction parameter) of an image projected by each projector 101 based on the three-dimensional position of the projection plane 104 integrated by the three-dimensional position integration unit 406.

  The communication unit 408 is a unit that transmits the image processing parameters calculated by the parameter calculation unit 407 to each projector 101, and includes, for example, the communication I / F 305 and the external I / F 306 in FIG. The communication unit 408 also transmits / receives a control command to / from each projector 101, receives an image captured by each projector 101, transmits an image to be projected to each projector 101, and the like.

  The storage unit 409 includes, for example, a plurality of images captured by each projector 101 under the control of the control unit 404, the three-dimensional position of the projection plane 104 integrated by the three-dimensional position integration unit 406, and the image processing parameter calculated by the parameter calculation unit 407. Etc. are stored.

  Note that the control unit 404, the three-dimensional position measurement unit 405, the three-dimensional position integration unit 406, the parameter calculation unit 407, the storage unit 409, and the like are realized by, for example, a program that operates on the CPU 301 in FIG.

<Process flow>
(Capturing image for calibration)
FIG. 5 is a flowchart illustrating the flow of the calibration image capturing process according to the first embodiment. For example, the image processing system 100 starts a calibration image capturing process in response to a user operation on the PC 102 or the like.

  Before starting the shooting process, the plurality of projectors 101-1 to 101-3 are arranged so that, for example, a part of the projection area of the adjacent projector 101 overlaps as shown in FIG. Shall. Further, the positional relationship of the plurality of projectors 101-1 to 101-3, for example, the projector 101-1, the projector 101-2, and the projector 101-3 arranged in order from the left is set or detected in advance. It shall be.

  When the calibration processing image capturing process of the image processing system 100 is started, the PC 102 transmits a first capturing instruction to instruct the start of the calibration image capturing process to the plurality of projectors 101-1 to 101-3 ( Step S501).

  In response to the first shooting instruction, the projector 101-1 projects a pattern image for calibration (step S502), and the projectors 101-2 and 101-3 turn off the light source 206 (steps S503 and S504).

  An example of the pattern image projected onto the projection surface 104 at this time is shown in FIG. There are various pattern images used for parameter correction. Here, it is assumed that a pattern image 601 in which circular patterns are arranged vertically and horizontally as shown in FIG. 6A is used.

  In the state of FIG. 6A, the projector 101-1 captures the projection surface 104 on which the pattern image 601 is projected by the projection unit 105 of the projector 101-1 (step S505). At this time, the image captured by the projector 101-1 includes a pattern image 601 projected by the projector 101-1. The projector 101-1 transmits the image captured in step S505 to the PC 102 (step S506).

  In addition, the projector 101-2 captures the projection surface on which the pattern image 601 is projected by the projection unit 105 of the projector 101-1 (step S507). At this time, the image photographed by the projector 101-1 includes the calibration pattern of the common area 602 of the projector 101-1 and the projector 101-1, among the pattern images 601 projected by the projector 101-1. The projector 101-2 transmits the image captured in step S507 to the PC 102 (step S508). At this time, since the calibration pattern image 601 is not included in the photographing area of the projector 101-3, the projector 101-3 does not have to photograph the projection plane 104.

  When the PC 102 receives images from the projector 101-1 and the projector 101-2 (step S509), the PC 102 transmits a second photographing instruction to the plurality of projectors 101-1 to 101-3 (step S510).

  In response to the second shooting instruction, the projector 101-2 projects a calibration pattern image (step S512), and the projectors 101-1 and 101-3 turn off the light source 206 (steps S511 and S513).

  An example of the pattern image projected onto the projection plane 104 at this time is shown in FIG. In the state of FIG. 6B, the projector 101-1 captures the projection surface 104 on which the pattern image 601 is projected by the projection unit 105 of the projector 101-2 (step S514). At this time, the image photographed by the projector 101-1 includes the calibration pattern of the common area 602 of the projector 101-1 and the projector 101-1 in the pattern image 601 projected by the projector 101-2. The projector 101-1 transmits the image captured in step S514 to the PC 102 (step S515).

  In addition, the projector 101-2 captures the projection surface on which the pattern image 601 is projected by the projection unit 105 of the projector 101-2 (step S516). At this time, the image captured by the projector 101-2 includes a pattern image 601 projected by the projector 101-2. The projector 101-2 transmits the image captured in step S516 to the PC 102 (step S517).

  Further, the projector 101-3 captures the projection surface 104 on which the pattern image 601 is projected by the projection unit 105 of the projector 101-2 (step S518). At this time, the image photographed by the projector 101-3 includes the calibration pattern of the common area 603 of the projector 101-2 and the projector 101-3 in the pattern image 601 projected by the projector 101-2. The projector 101-3 transmits the image captured in step S518 to the PC 102 (step S519).

  When the PC 102 receives images from the projectors 101-1 to 101-3 (step S520), the PC 102 transmits a third shooting instruction to the plurality of projectors 101-1 to 101-3 (step S521).

  In response to the third shooting instruction, the projectors 101-1 and 101-2 turn off the light source 206 (steps S <b> 522 and S <b> 523), and the projector 101-3 projects a calibration pattern image (step S <b> 524).

  An example of the pattern image projected on the projection surface 104 at this time is shown in FIG. In the state of FIG. 6C, the projector 101-2 captures the projection surface 104 on which the pattern image 601 is projected by the projection unit 105 of the projector 101-3 (step S525). At this time, the image captured by the projector 101-2 includes a calibration pattern of the common area 603 of the projector 101-2 and the projector 101-3 in the pattern image 601 projected by the projector 101-3. The projector 101-2 transmits the image captured in step S525 to the PC 102 (step S526).

  In addition, the projector 101-3 captures a projection surface on which the pattern image 601 is projected by the projection unit 105 of the projector 101-3 (step S527). At this time, the image captured by the projector 101-3 includes a pattern image 601 projected by the projector 101-3. The projector 101-3 transmits the image captured in step S527 to the PC 102 (step S528). At this time, since the calibration pattern image 601 is not included in the photographing area of the projector 101-1, the projector 101-1 does not have to photograph the projection plane 104.

  The PC 102 receives images from the projectors 101-1 and 102-3 (step S <b> 529), and ends the calibration image capturing process.

(Calibration process)
The image processing system 100 performs the calibration process when the calibration image capturing process is completed.

  FIG. 7 is a flowchart of an example of a calibration process according to the first embodiment.

  When the calibration process is started, the three-dimensional position measurement unit 405 performs the three-dimensional projection surface 104 for each projector 101 based on the image acquired in the process of FIG. 5 and the measurement parameters stored in advance by each projector 101. The position (coordinates) is calculated (step S701). A method for calculating the three-dimensional position will be described later.

  Next, the three-dimensional position integration unit 406 calculates the correspondence between the coordinate systems of adjacent projectors (step S702). For example, the three-dimensional position integration unit 406 calculates a correspondence relationship between the coordinate system of the projector 101-1 and the coordinate system of the projector 101-2. Further, the three-dimensional position integration unit 406 calculates a correspondence relationship between the coordinate system of the projector 101-2 and the coordinate system of the projector 101-3.

  Further, the three-dimensional position integration unit 406 of the PC 102 integrates a plurality of three-dimensional positions of the projection plane 104 into one coordinate system based on the correspondence relationship calculated in step S802 (step S703).

  Subsequently, the parameter calculation unit 407 calculates image processing parameters of an image projected by each of the plurality of projectors 101-1 to 101-3 based on the three-dimensional position integrated in step S803 (step S704). Further, the PC 102 transmits the calculated image processing parameters to the plurality of projectors 101-1 to 101-3 via the communication unit 408 (step S705).

  Each of the plurality of projectors 101-1 to 101-3 stores the received image processing parameters in the storage unit 401 (steps S706 to S708), and ends the calibration process.

(Measurement of 3D coordinates)
Here, the measurement of the three-dimensional position by the three-dimensional position measuring means 405 will be described. Prior to explanation, symbols are defined.

  FIG. 8 is a diagram for explaining the definition of symbols according to the first embodiment.

  In FIG. 8A, among the three-dimensional coordinates of the center of gravity of each circular pattern that is three-dimensionally measured by the system of the projector 101-1, a coordinate group of points photographed only by the projector 101-1 is P1s (i). . In addition, among the three-dimensional coordinates of the center of gravity of each circular pattern that is three-dimensionally measured by the system of the projector 101-1, a coordinate group of points of the common area 602 that are photographed together by the projectors 101-1 and 101-2 is P1m ( j).

  Also, in FIG. 7B, among the three-dimensional coordinates of the center of gravity of each circular pattern that is three-dimensionally measured by the projector 101-2 system, a coordinate group of points that are photographed only by the projector 101-2 is P2s (k). And In addition, among the three-dimensional coordinates of the center of gravity of the pattern that is three-dimensionally measured by the projector 101-2 system, the coordinate group of the points in the common area 603 that are photographed together by the projectors 101-1 and 101-2 is P2m (l). To do.

  Further, in FIG. 7A, let P2m (j) be a coordinate group of points in the common area 602 projected by the projector 101-1, expressed in the coordinate system of the projector 101-2. Similarly, in FIG. 8B, a coordinate group of the points of the common area 602 projected by the projector 101-2 expressed by the coordinate system of the projector 101-1 is P1m (l).

  That is, P1m (j) and P2m (j) represent the same point group in different coordinate systems, and P1m (l) and P2m (l) also represent the same point group in different coordinate systems. Is.

  The three-dimensional position measurement unit 405 calculates (measures) the three-dimensional position of the projection surface 104 with respect to the projector 101 using the pattern image projected by the projection unit 105 of the projector 101 and the image photographed by the photographing unit 106 of the projector 101. To do. For example, the three-dimensional position measurement unit 405 uses the above-described P1s (i) and P1m (j) by using the pattern image projected by the projection unit 105 of the projector 101-1 and the image captured by the imaging unit of the projector 101-1. ) The three-dimensional coordinates of the pattern image projected on the projection plane 104 are an example of information indicating the three-dimensional position of the projection plane 104.

  The three-dimensional coordinates of the pattern image can be obtained by triangulation from the difference between the projection light from the projection unit 105 of the projector 101-1 and the angle at which the reflected light from the projection surface is received by the photographing unit 106. Here, it is assumed that the coordinate origin is set at the center of the projector 101-1, and the three-dimensional coordinates are expressed by a coordinate system in which the direction of the center of the line of sight of the projection unit 105 is the z axis.

  Similarly, a pattern image projected by the projection unit 105 of the projector 101-2 is used as an image captured by the imaging unit 106 of the projector 101-2. (K), P2m (l) is obtained. At this time, the coordinate origin is set at the center of the projector 101-2, and the coordinate system has the z-axis as the direction of the center of the line of sight of the projection unit 105.

  Further, the three-dimensional coordinates of the projection plane 104 with respect to another projector 101, for example, the projector 101-3 are also calculated by the same method. At this stage, the coordinate relationship between the calibration patterns projected by the projectors 101 is unknown.

  FIG. 9 is a diagram for explaining calculation of three-dimensional coordinates according to the first embodiment. The storage unit 401 of the projector 101 stores measurement parameters for three-dimensional measurement by the projection unit 105 and the photographing unit 106 of the projector 101 in advance. The measurement parameters include, for example, internal parameters of the projection optical system, internal parameters of the imaging optical system, external parameters of the projection unit 105 and the imaging unit 106, and the like.

  For example, as the internal parameters of the projection optical system, the distance between the display element 207 and the projection lens 208, the relationship between each circular pattern of the pattern image to be projected and the physical position on the display element 207, etc. are measured in advance. And stored in the storage unit 401.

  Further, as internal parameters of the imaging optical system, the distance between the imaging lens 210 and the imaging element 211, the relationship between the coordinates of the captured image and the physical position on the imaging element 211, and the like are obtained in advance by actual measurement and stored. Stored in the means 401.

  Further, information corresponding to the baseline length of triangulation such as the distance between the display element 207 and the imaging element 211 is obtained in advance by actual measurement or the like and stored in the storage unit 401 as an external parameter.

  The three-dimensional position measurement unit 405 is configured to output 3 of the projection plane 104 with respect to the projector 101 based on information corresponding to the internal parameters of the projection optical system, the internal parameters of the imaging optical system, and the baseline length of triangulation stored in the storage unit 401. Measure dimensional coordinates.

  The distance to the projection surface 104 is such that the light beam passing through the center of the projection lens 208 from the display element 207, the light beam whose reflected light reaches the image sensor 211 through the center of the image pickup lens 210, the display element 207, the image sensor 211, It can be measured by the line between and the triangle. For example, by triangulation using the position on the display element 207 of the center of gravity of the circular pattern of the pattern image, the position on the image sensor 211 of the center of gravity of the corresponding circular pattern, and information corresponding to the above-described baseline length, The distance to the projection surface 104 can be measured.

  In the present embodiment, since the photographing unit 106 is provided at a predetermined position with respect to the projection unit 105, the measurement parameters of the three-dimensional survey are measured or calculated in advance, for example, before the factory shipment or the like. It can be stored in the storage unit 401.

(About integration of 3D position)
The three-dimensional position integration unit 406 uses calibration patterns that are photographed in common by adjacent projectors 101, such as P1m (j) and P2m (j) and P1m (l) and P2m (l) in FIG. Then, the correspondence relationship of the coordinate system between the adjacent projectors 101 is calculated.

  For example, consider the case where P1m (j) in FIG. 8A is represented by P2m (j) which is the coordinate system representation of the projector 101-2. In this case, the pattern P1m (j) projected by the projector 101-1 is calculated by SfM (Structure from Motion) using the image photographed by the projector 101-1 and the image photographed by the projector 101-2. be able to. SfM is a known technique for estimating the three-dimensional structure of a scene using images taken from multiple viewpoints (see, for example, Non-Patent Document 1). Accordingly, P1m (j) in FIG. 8A projected by the projector 101-1 can be represented by P2m (j) which is a coordinate system representation of the projector 101-2.

  Similarly, consider the case where P2m (l) in FIG. 8B is represented by P1m (l) which is the coordinate system representation of the projector 101-1. In this case, P1m (l) can be calculated by SfM from the image captured by the projectors 101-1 and 101-2 for the pattern P2m (l) projected by the projector 101-2. Thus, P2m (l) in FIG. 8B projected by the projector 101-2 can be represented by P1m (l) which is a coordinate system representation of the projector 101-1.

  Next, the three-dimensional position integration unit 406 integrates the three-dimensional positions of the individual coordinate systems into one coordinate system. For example, since P2m (j) and P1m (j) calculated by SfM and P2m (l) and P2m (l) calculated by SfM are the same point, the rotation vector R representing the relationship between them is A translation vector t can be obtained (see, for example, Non-Patent Document 2).

  If R and t are obtained, all three-dimensional points can be expressed in the same coordinate system. Since these three-dimensional points are on the same plane, the plane can be estimated. For example, if one of the three-dimensional points is (x0, y0, z0), the distance to the plane ax + by + cz + d = 0 is | ax0 + by0 + cz0 + d | / sqrt (a ^ 2 + b ^ 2 + c ^ 2). Therefore, a, b, c, d that minimizes the sum of squares of the distances of all points is a plane expression to be obtained.

  Here, the case where there are two projectors 101 has been described. However, even when the number of projectors is increased to three, coordinate systems can be integrated by the same method.

  Pattern images (two-dimensional calibration pattern images) individually projected from the viewpoint from the normal direction of the plane obtained by the above processing can be synthesized.

  FIG. 10 is a diagram illustrating an example of a combined pattern image according to the first embodiment. FIG. 10A shows a combined pattern image. FIG. 10B is a diagram in which the pattern 1001 projected by the projector 101-1, the pattern 1002 projected by the projector 101-2, and the pattern 1003 projected by the projector 101-3 are distinguished and displayed.

  The parameter calculation unit 407 calculates image processing parameters based on the synthesized pattern image.

  For example, the parameter calculation unit 407 linearly extrapolates each circular pattern of the calibration image for each projector 101, and calculates the outer peripheral coordinates of the projectable area of each projector 101.

  Further, the parameter calculation unit 407 obtains a projectable area by the plurality of projectors 101-1 to 101-3 based on the logical sum of the projectable areas of each projector 101.

  Furthermore, the parameter calculation unit 407 maps a predetermined image to be projected onto a region that can be projected by the plurality of projectors 101-1 to 101-3. For example, the parameter calculation unit 407 maximizes the predetermined image to be projected in the region that can be projected by the plurality of projectors 101-1 to 101-3 while maintaining the aspect ratio of the predetermined image to be projected. Now mapping. For example, the parameter calculation unit 407 performs geometric correction and arrangement so that a predetermined image to be projected falls within the area 1004 in FIG. 10B while maintaining the aspect ratio.

  Furthermore, the parameter calculation unit 407 calculates an overlapping area where the projection areas overlap between the adjacent projectors 101 from the projectable area of each projector 101, and calculates a brightness correction parameter for the overlapping area. This is because if the images of the projectors 101-1 to 101-3 are simply superimposed, the overlapping area becomes bright.

  For example, through the processing described above, the parameter calculation unit 407 performs image processing necessary for extraction of a region projected by each projector 101, geometric correction, brightness correction of an overlapping region, etc., from a predetermined image to be projected. Calculate the parameters.

  Here, the case of synthesizing pattern images has been described. However, the parameter calculation unit 407 may calculate the image processing parameters using the barycentric coordinates of the circular pattern.

(Projection processing)
Next, the projection process will be described. For example, an image to be projected is transmitted from the PC 102 to each projector 101, and the image received by each projector 101 using the image processing parameters stored in the storage unit 401 is processed and projected to project an integrated image. It can be projected onto the surface 104.

  FIG. 11 is a flowchart of an example of the image projection processing according to the first embodiment.

  For example, when the projection process is started by a user operation or the like on the PC 102, the PC 102 instructs the plurality of projectors 101-1 to 101-3 to start multi-projection (step S1101). Each of the plurality of projectors 101-1 to 101-3 that has received the instruction reads the image processing parameters stored in the storage unit 401 (steps S1102 to S1104).

  Each of the projectors 101-1 to 101-3 enters a parameter application mode for processing an image to be projected with the read image processing parameters (steps S1105 to S1107). In the parameter application mode, each projector 101 extracts and transforms a necessary part from the received landscape image according to the read image processing parameter, and projects an image with brightness and color correction.

  The PC 102 transmits the same content image (for example, a horizontally long image) to the plurality of projectors 101-1 to 101-3 (step S1108). Each of the plurality of projectors 101-1 to 101-3 applies a parameter to the content image received from the PC 102, performs image processing (steps S1109 to S1111), and projects the image after image processing (steps S1112 to S1114). ).

  The transmission and projection of the content image are repeatedly performed at, for example, 60 fps (frame per second) until the multi-projection is completed (step S1115). When the multi-projection is completed, the PC 102 transmits a multi-projection end instruction to the plurality of projectors 101-1 to 101-3 (step S1116). Each of the plurality of projectors 101-1 to 101-3 receives the multi-projection end instruction and shifts to the simple projection mode (steps S1117 to S1119), and ends the multi-projection process.

(Connection type)
FIG. 12 is a diagram illustrating an example of a connection form of the image processing system according to the first embodiment. As long as the image processing system 100 can communicate a control signal and a video signal between the PC 102 and each projector 101, the connection form may be arbitrary.

  For example, in the connection form of FIG. 12A, the PC 102 transmits the video signal to each projector 101 via the branching device 1202 (for example, a USB hub) using the external video card 1201, and the control signal is exchanged. , Using a wireless LAN.

  In the connection mode of FIG. 12B, the PC 102 outputs a video signal using an image interface such as HDMI, and the video distributor 1203 distributes the video signal to each projector. In addition, the exchange of control signals is performed by a wired connection such as a wired LAN.

  In the connection form of FIG. 12C, an example in which the video signal and the control signal are communicated by wireless communication (for example, wireless LAN) is shown.

  Each connection form in FIG. 12 is merely an example, and the connection form of the image processing system 100 can be various combinations other than each connection form in FIG.

<Summary>
As described above, the projector 101 according to the present embodiment is provided with the photographing unit 106 (camera or the like) at a predetermined position with respect to the projection unit 105, and the measurement parameters of the three-dimensional measurement by the projection unit 105 and the photographing unit 106 are stored. Stored in the means 401 in advance. Thereby, the three-dimensional coordinates of the projection plane 104 with respect to the projector 101 can be easily calculated without using the camera installation, the measurement of the baseline length, or the like, or the stereo camera. However, the positional relationship of the coordinate system with other projectors 101 is unknown only by the three-dimensional measurement of each projector 101 alone.

  On the other hand, in the three-dimensional measurement using only SfM, it is necessary to photograph many circular patterns in order to obtain accuracy, and it is desirable to use a camera with a wide angle and a wide photographing range. However, a camera having such a lens is expensive, and a high-resolution image sensor is required, which is disadvantageous in terms of cost.

  Therefore, in this embodiment, the combination of the low-cost imaging means built in each projector 101 and two three-dimensional measurements such as triangulation, SfM, etc., installation, setting, etc., while suppressing the system cost. An image processing system 100 that facilitates the above is realized.

[Second Embodiment]
In the functional configuration of the first embodiment shown in FIG. 4, the projector 101 has been described as having the image processing unit 402, but the image processing on the projection image is performed collectively by the PC 102. May be.

  FIG. 13 is a functional configuration diagram of the image processing system according to the second embodiment. In the present embodiment, the PC 102 has an image processing unit 1301 that performs image processing of an image projected by each of the plurality of projectors 101-1 to 101-3. Since the other configuration is the same as that of the first embodiment, the difference from the first embodiment will be mainly described here.

  In the present embodiment, the calibration image shooting process is performed in the same manner as the shooting process of the first embodiment shown in FIG.

  FIG. 14 is a flowchart of an example of a calibration process according to the second embodiment. The processing from step S701 to S704 is the same as that in the first embodiment. In step S1401, the image processing parameters are transmitted to each projector 101 in the first embodiment. However, in this embodiment, the PC 102 stores the calculated image processing parameters.

  FIG. 15 is a flowchart of an example of an image projection process according to the second embodiment. For example, when the projection processing is started by a user operation or the like on the PC 102, the PC 102 applies the image processing parameters saved in step S1401 in FIG. 14 to the content image to be projected (step S1501). For example, the image processing unit 1301 of the PC 102 extracts and deforms a portion projected by the projector 101-1 from the horizontally long content image using image processing parameters applied to the projector 101-1, and corrects brightness and color. I do.

  Similarly, the image processing unit 1301 extracts and deforms a portion projected by the projector 101-2 from the horizontally long content image using image processing parameters applied to the projector 101-2, and corrects brightness and color. Do. Further, the image processing unit 1301 extracts and deforms a portion projected by the projector 101-3 from the horizontally long content image using image processing parameters applied to the projector 101-3, and corrects brightness and color. .

  Next, the PC 102 transmits the image subjected to the image processing in step S1501 to the plurality of projectors 101-1 to 101-3 (step S1502).

  Each of the plurality of projectors 101-1 to 101-3 updates the projection image using the image received from the PC 102 (steps S1503 to 1505).

  The above process is repeated, for example, at 60 fps until the multi-projection is completed (step S1506).

  In this embodiment, since the image transmitted by the PC 102 differs for each projector 101, the connection form shown in FIG. 12B cannot be used. In the present embodiment, it can be realized using, for example, the connection forms shown in FIGS.

  As described above, according to the present embodiment, each projector 101 does not require an image processing unit, and the cost or load of the projector 101 can be reduced.

[Third Embodiment]
In the first and second embodiments, the case where the image processing system 100 is controlled using the PC 102 has been described. However, the image processing system 100 may be configured by a plurality of projectors without using the PC 102. Is possible.

<Functional configuration>
FIG. 16 is a functional configuration diagram of an image processing system according to the third embodiment. In addition to the configuration of the projector 101 according to the first embodiment, the projector 101 according to the present embodiment includes an image acquisition unit 1601, a three-dimensional position measurement unit 1602, a three-dimensional position integration unit 1603, a parameter calculation unit 1604, and a control unit. 1605, synchronization means 1606, and the like.

  The image acquisition unit 1601 is a unit that acquires a content image to be projected onto the projection plane 104. For example, the image acquisition unit 1601 acquires the same content to be projected from an external storage such as a USB memory connected to the external I / F 205 in FIG. Alternatively, the image acquisition unit 1601 may be one that acquires the content to be projected from the server device or the like using the communication I / F 204 or the like.

  A three-dimensional position measuring unit (measuring unit) 1602 is a projection plane for each projector 101 based on an image photographed under the control of the control unit 1605 and the like and a measurement parameter stored in advance in the storage unit 401 of each projector 101. The three-dimensional position 104 is measured.

  A three-dimensional position integration unit (integration unit) 1603 integrates a plurality of three-dimensional positions measured by the three-dimensional position measurement unit 1602 of each projector 101. The three-dimensional position integration process may be performed in a distributed manner by a plurality of projectors 101, or may be performed collectively by a single projector 101.

  A parameter calculation unit (calculation unit) 1604 calculates image processing parameters (correction parameters) of images projected by the projectors 101 based on the three-dimensional position of the projection plane 104 integrated by the three-dimensional position integration unit 1603 or the like. .

  The control unit 1605 controls the projection of images by the plurality of projectors 101-1 to 101-3 and the photographing of images projected on the projection plane 104 by the plurality of projectors 101. In addition, the control unit 404 uses a plurality of projectors 101-1 to 101-3 to control multi-projection for projecting a predetermined image (large screen image).

  The synchronization unit 1606 is a unit for synchronizing the image projection timings and the like of the plurality of projectors 101-1 to 101-3.

  Other configurations are the same as those in the first embodiment.

<Process flow>
(Capturing image for calibration)
FIG. 17 is a flowchart illustrating the flow of the calibration image capturing process according to the third embodiment. The image processing system 100 starts a calibration image capturing process in response to, for example, a user operation on the projector 101-1.

  Before starting the shooting process, the plurality of projectors 101-1 to 101-3 are arranged so that, for example, a part of the projection area of the adjacent projector 101 overlaps as shown in FIG. Shall. Further, the positional relationship of the plurality of projectors 101-1 to 101-3, for example, that the projectors 101-1, 101-2, and 101-3 are arranged in order from the left is set or detected in advance. Shall.

  When the start operation of the calibration image capturing process is performed (step S1701), the projector 101-1 instructs the other projectors 101-2 and 101-3 to start the calibration image capturing process. One shooting instruction is transmitted (step S1702).

  Further, the projector 101-1 projects a pattern image for calibration (step S1703), and the projectors 101-2 and 101-3 turn off the light source 206 in response to the first photographing instruction (steps S1704 and S1705). .

  The projector 101-1 captures the projection surface 104 onto which the pattern image is projected by the projection unit 105 of the projector 101-1 (step S 1706), and saves (stores) the captured image in the storage unit 401 of the own apparatus (step S 1706). S1707).

  In addition, the projector 101-2 captures the projection plane on which the pattern image is projected by the projection unit 105 of the projector 101-1 (step S1708), and stores the captured image in the storage unit 401 of the own apparatus (step S1709). At this time, since the calibration pattern image is not included in the photographing area of the projector 101-3, the projector 101-3 does not have to photograph the image.

  Next, the projector 101-1 transmits a second shooting instruction to the other projectors 101-2 and 101-3 (step S1710).

  The projector 101-1 turns off the light source 206 (step S1711). In response to the second imaging instruction, the projector 101-2 projects a calibration pattern image (step S1712), and the projector 101-3 turns off the light source 206 (step S1713).

  Next, the projector 101-1 captures the projection surface 104 on which the pattern image is projected by the projection unit 105 of the projector 101-2 (step S1714), and saves the captured image in the storage unit 401 of the own apparatus (step S1714). Step S1715).

  Further, the projector 101-2 captures the projection surface on which the pattern image is projected by the projection unit 105 of the projector 101-2 (step S1716), and stores the captured image in the storage unit 401 of the own apparatus (step S1717).

  The projector 101-3 captures the projection surface 104 on which the pattern image is projected by the projection unit 105 of the projector 101-2 (step S1718), and stores the captured image in the storage unit 401 of the own apparatus (step S1719).

  Next, the projector 101-1 transmits a third shooting instruction to the other projectors 101-2 and 101-3 (step S1720).

  The projector 101-1 turns off the light source 206 (step S1721). Further, in response to the second photographing instruction, the projector 101-2 turns off the light source 206 (step S1722), and the projector 101-3 projects a calibration pattern image (step S1723).

  Next, the projector 101-2 captures the projection surface 104 on which the pattern image is projected by the projection unit 105 of the projector 101-3 (step S1724), and stores the captured image in the storage unit 401 of the own apparatus (step S1724). S1725).

  The projector 101-3 captures the projection surface on which the pattern image is projected by the projection unit 105 of the projector 101-3 (step S1726), and stores the captured image in the storage unit 401 of the own apparatus (step S1727). The process is terminated. At this time, since the calibration pattern image is not included in the photographing area of the projector 101-1, the projector 101-1 does not have to photograph the image.

(Calibration process)
The image processing system 100 performs the calibration process when the calibration image capturing process is completed.

  FIG. 18 is a flowchart of an example of a calibration process according to the third embodiment.

  When the calibration process is started, the projector 101-1 transmits a configuration start instruction to the other projectors 101-2 and 101-3 (step S1801).

  Next, the projector 101 calculates the three-dimensional position (coordinates) of the projection plane 104 with respect to each projector 101 based on the image acquired by the processing of FIG. 17 and the measurement parameters stored in advance by each projector 101 ( Steps S1802 to S1804).

  Each projector 101 transmits the calculated three-dimensional position (the position of the circular pattern) and the captured calibration pattern image to other projectors 101 (steps S1805 to 1807). Each projector 101 uses the information received from the other projectors 101 to calculate the correspondence relationship between the three-dimensional positions with the adjacent projectors 101 (steps S1808 to S1810). In addition, the projectors 101-2 and 101-3 transmit the calculated information to the projector 101-1 (steps S1811 and S1812).

  The projector 101-1 uses the information received from the other projectors 101-1, 101-3, converts the three-dimensional position into the same coordinate system, and integrates them (step S1813). Further, the projector 101-1 calculates the image processing parameter of each projector using the integrated three-dimensional position (step S <b> 1814).

  The projector 101-1 transmits the calculated image processing parameters to the other projectors 101-2 and 101-3 (step S1815), and stores the image processing parameters of the own apparatus in the storage unit 401 (step S1816).

  The projectors 101-2 and 101-3 store the received image processing parameters in the storage unit 401 of their own devices (steps S1817 and 1818), and end the calibration process.

(Projection processing)
FIG. 19 is a flowchart illustrating an example of image projection processing according to the third embodiment. It is assumed that a USB memory or the like that stores a content image to be projected is connected to each projector 101 in advance.

  When the projection process is started, the projector 101-1 transmits a multi-projection instruction to the other projectors 101-2 and 101-3 (step S1901).

  Next, each of the projectors 101-1 to 101-3 reads out the image processing parameters stored in the storage unit 401 of the own apparatus (steps S1902 to S1904).

  Each of the projectors 101-1 to 101-3 enters a parameter application mode for processing an image to be projected with the read image processing parameters (steps S1905 to S1907).

  Next, the projector 101-1 transmits a projection start instruction to the other projectors 101-2 and 101-3 (step S1908).

  Each of the projectors 101-1 to 101-3 acquires one frame of content image from a USB memory or the like (steps S1909 to S1911), and applies parameters to the acquired content image (steps S1912 to S1914).

  Here, the projectors 101-1 to 101-3 synchronize the projection timing (steps S1915 to S1917). For example, each of the projectors 101-1 to 101-3 transmits and receives control signals using the synchronization unit 1606, confirms that the other projectors 101 are ready for projection, and confirms that the projections are ready. Sometimes move on to the next process.

  Next, each projector 101-1 to 101-3 updates the projection image (steps S1918 to S1920), and projects the updated image.

  The above process is repeated, for example, at 60 fps until the multi-projection is completed (step S1921). If it is determined in step S1921 that multi-projection has ended, projector 101-1 transmits a multi-projection end instruction to other projectors 101-2 and 101-3 (step S1922).

  Next, each of the projectors 101-1 to 101-3 shifts to the simple projection mode (steps S1923 to S1925) and ends the process.

  With the above processing, the image processing system 100 can realize multi-projection without using the PC 102.

  FIG. 20 is a diagram illustrating an example of a connection form of the image processing system according to the third embodiment. In the example of FIG. 20, each of the projectors 101-1 to 103-3 transmits / receives a control signal by wireless mutual communication such as wireless LAN or short-range wireless communication, for example, an external storage unit 2001 such as a USB memory. To obtain a common content image. Note that the connection form in FIG. 20 is merely an example.

[Fourth Embodiment]
In the first to third embodiments, the plurality of projectors 101 are arranged so that a part of the projection area overlaps with the adjacent projector 101, and multi-projection projects a large screen larger than the projection screen of each projector 101. The explanation was made on the assumption that

  In the present embodiment, a plurality of projectors 101 are arranged so that a part or the whole of the projection area overlaps with adjacent projectors 101, and a stack projection that projects a bright screen that is brighter than the projection screen of each projector. An example will be described.

  FIG. 21 is a diagram illustrating a configuration example of an image processing system according to the fourth embodiment. The configuration and connection form of each device are the same as in the first embodiment, but the overlapping portions of the projection areas 103-1 to 101-3 of each projector 101 are larger than those in the first embodiment. . Since a larger image can be projected as the overlapping portion of the projection area is larger, attention should be paid when each projector 101 is arranged.

  The flow of the configuration image photographing process and the flow of the calibration process may be the same as those in the first embodiment shown in FIGS.

  Further, the processing at the time of projection is basically the same as that of the first embodiment shown in FIG. 11, but in step S1108, the image transmitted by the PC 102 to each projector 101 is not horizontally long and matches the resolution of the projector 101. The difference is that content images with different ratios are transmitted. Further, the image processing parameters are calculated so that the entire image is included in the projection area, not a part of the image.

  FIG. 21 is a diagram for explaining a region where a projection image according to the fourth embodiment is generated. In this example, the parameters are calculated so that the content image is projected onto an area 2104 including the pattern 2101 projected by the projector 101-1, the pattern 2102 projected by the projector 101-2, and the pattern 2103 projected by the projector 101-3. The

  Thus, the image processing system 100 can be applied not only to multi-projection but also to stack projection by overlapping the projection areas of the projectors 101-1 to 101-3 and performing predetermined image processing. .

DESCRIPTION OF SYMBOLS 100 Image processing system 101 Projector (image projection apparatus)
101-1 projector (first image projection apparatus)
101-2 Projector (second image projection device)
101-3 Projector 102 PC
DESCRIPTION OF SYMBOLS 103 Projection area 104 Projection surface 105 Projection means 106 Imaging means 401 Storage means 402, 1301 Image processing means 404, 1605 Control means 405, 1602 Three-dimensional position measurement means (measurement means)
406, 1603 Three-dimensional position integration means (integration means)
407, 1604 Parameter calculation means

JP 2009-005044 A

Yagi Yasushi and Saito Hideo, “Computer Vision Cutting-Edge Guide 3”, Adcom Media Co., Ltd., p.5-8, December 2010 Toru Tamaki, “Attitude Estimation and Rotation Matrix”, IEICE Technical Report, IEICE Technical Report, SIP2009-48, SIS2009-23, p.59-64, September 2009

Claims (10)

An image processing system that projects a predetermined image on a projection surface using a plurality of image projection devices,
Each of the plurality of image projection devices includes:
Projection means for projecting an image onto the projection plane;
An imaging unit that is provided at a predetermined position with respect to the projection unit and that captures an image projected on the projection plane;
Have
The image processing system includes:
Control means for controlling projection by the projection means of the plurality of image projection apparatuses and photographing by the photographing means of the plurality of image projection apparatuses;
3D of the projection plane for each of the plurality of image projection devices based on the image captured by the control and the measurement unit of the projection unit and the imaging unit of the plurality of image projection devices. A measuring means for measuring the position;
Integration means for integrating the plurality of measured three-dimensional positions;
Have
An image processing system that projects the predetermined image on the projection plane based on the integrated three-dimensional position of the projection plane. Each of the plurality of image projection devices includes:
The image processing system according to claim 1, further comprising a storage unit that stores in advance measurement parameters of the three-dimensional measurement performed by the projection unit and the imaging unit of the apparatus itself. The measuring means includes
A first captured image captured by the first image projection device of an image projected by the first image projection device of the plurality of image projection devices, and a measurement parameter of the three-dimensional measurement of the first image projection device; The image processing system according to claim 1, wherein a three-dimensional position of the projection plane with respect to the first image projection apparatus is measured based on the first image projection apparatus. The measurement parameter of the three-dimensional measurement includes information corresponding to a baseline length of triangulation by the projection unit and the photographing unit,
The measuring means includes
The image processing system according to claim 3, wherein coordinates of the projection plane with respect to the first image projection device are measured based on information corresponding to the baseline length. The plurality of image projection devices include a second image projection device disposed adjacent to the first image projection device,
The integration means includes
Based on the first photographed image and a second photographed image obtained by photographing the at least part of the image projected by the projection unit of the first image projection apparatus by the photographing unit of the second image projection apparatus. 5. The image processing system according to claim 3, wherein a correspondence relationship between the coordinate system of the first image projection device and the coordinate system of the second image projection device is calculated. The integration means includes
The image processing system according to claim 5, wherein the correspondence relationship is calculated by Structure from Motion. The integration means includes
And integrating at least a three-dimensional position of the projection plane with respect to the first image projection apparatus and a three-dimensional position of the projection plane with respect to the second image projection apparatus based on the calculated correspondence. Item 7. The image processing system according to Item 5 or 6. Based on the three-dimensional position of the projection plane integrated by the integration unit, parameter calculation means for calculating image processing parameters of an image projected by each of the plurality of image projection apparatuses;
Each of the plurality of image projection devices includes:
The image processing system according to claim 1, wherein an image subjected to image processing based on the calculated image processing parameter is projected onto the projection plane. Each of the plurality of image projection devices is arranged such that at least a part of a projection region overlaps with an adjacent image projection device,
The image processing parameter is:
Image processing parameters for multi-projection for projecting a large screen larger than the projection screen of each of the plurality of image projection devices using the plurality of image projection devices, or the plurality of the plurality of image projection devices using the plurality of image projection devices The image processing system according to claim 8, wherein the image processing parameter is an image processing parameter for stack projection that projects a high-brightness screen brighter than each projection screen of the image projection apparatus. An image projection apparatus that projects a predetermined image on a projection surface in cooperation with another image projection apparatus,
Projection means for projecting an image onto the projection plane;
An imaging unit that is provided at a predetermined position with respect to the projection unit and that captures an image projected on the projection plane;
Storage means for storing in advance measurement parameters of three-dimensional measurement by the projection means and the photographing means;
Control means for controlling projection by the projection means of the own apparatus and the other image projection apparatus and photographing by the photographing means of the own apparatus and the other image projection apparatus;
Measuring means for measuring a three-dimensional position of the projection surface based on an image photographed by the control and the measurement parameter of the three-dimensional measurement stored in advance;
Integration means for integrating the measured three-dimensional position of the projection plane and the three-dimensional position of the projection plane measured by the other image projection apparatus;
Have
An image projection apparatus that projects at least a part of the predetermined image on the projection plane based on the integrated three-dimensional position of the projection plane.

Images