JP2015076657A - Projection system and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress the degree of influences of the design of a building structure on a projection image.SOLUTION: A projection system comprises: an image-capturing unit 415 for reading the state of the wall surface of a building structure; a captured image feature point extraction unit 420 for detecting, from the read wall surface image, a first feature point of the image; an image editing unit 105 for editing the image on the basis of the first feature point; a captured-image feature point information 1 storage unit 413 for storing the image-edited image and the first feature point in correlation with each other; a feature point information comparison unit 422 for comparing the positions of the first feature point stored in correlation with the image and a second feature point read by the image-capturing means again and detected by the feature point detection means; an image data adhesion unit 411 for automatically editing the image so that the first feature point of the image to be projected overlaps the second feature point detected again; and an image projection unit 128 for protecting the edited image onto the wall surface of the building structure.

Description

  The present invention relates to a projection system and method, and more particularly to a technique for projecting an image onto a wall surface of an uneven building.

  In recent years, a projection system that projects an image on a huge screen using a projector has been attracting attention. Furthermore, it has been devised to project an image using a projector on a complicated surface such as a wall of a building such as an outdoor building, instead of simply projecting onto a giant flat screen.

  As an example, there is a “geometric correction method in a multi-projection system” of Patent Document 1, for example. According to this document, a test pattern image is displayed on a screen from a projector, and the test pattern image is captured by a digital camera. Then, the projection position of the projector is calculated from the image, and a plurality of feature points in the test pattern captured image are detected by a technique such as pattern matching. A projection position parameter is calculated based on the detected feature point position, and the projection position of the projector is corrected.

  For example, when projecting an image on the wall of a building such as an outdoor building using a multi-projection system, the conventional technology displays a test pattern image on the screen from the projector and captures the test pattern image with a digital camera. To do. Next, the projection position of the projector is calculated from the captured image, and a plurality of feature points in the test pattern captured image are detected by a technique such as pattern matching. Then, a bump on the surface of the projection position is detected based on the detected position of the feature point, and the projection position of the projector is corrected.

  However, in the case of projecting an image on the wall of a building such as an outdoor building, there is a line pattern indicating the shape of various parts such as a substantial frame, a window frame, a pillar, and a joint. In the prior art described above, there is a problem that these patterns and the projected image are synthesized and the projected image becomes difficult to see.

  Patent Document 1 is considered to be related to the present application in that it detects the state of the projector projection surface and attempts to correct the projection image based on the detection result. However, the problem that the projection image is difficult to see because the line-shaped pattern indicating the shapes of various parts such as window frames, columns, and joints and the projection image cannot be solved.

  The difficulty in viewing the projected image when projecting an image on the wall of a building such as an outdoor building is considered to be due to the fact that the pattern of the building affects the projection on the image.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to suppress the influence of a building pattern on a projected image.

  In order to achieve the above object, the present invention provides, as a first aspect, an imaging unit that reads a state of a wall surface of a building, and a feature point detection unit that detects a first feature point of the image from the read wall surface image. A first image editing unit that edits an image based on the first feature point, a feature point storage unit that stores the image edited image and the first feature point in association with each other, and stores the image in association with the image. A first feature point that is read again by the imaging unit and the position of the second feature point that is detected by the feature point detection unit is compared with the first feature point of the image to be projected Comprising: a second image editing means for automatically editing the image so as to overlap the second feature point detected again; and an image projecting means for projecting the edited image on the wall surface of the building. Providing a characteristic projection system To.

  According to the present invention, it is possible to suppress the influence of the building pattern on the projected image.

It is a block diagram which shows the structure of the general projection system which can apply this invention. It is a figure for demonstrating a multi-projection system. It is a figure which shows an example of the image projected with a multi-projection system. It is a block diagram which shows the structure of the projection system which concerns on embodiment of this invention. It is a figure for demonstrating the information processing in this embodiment. It is a figure which shows an example of the image file format in this embodiment. It is a figure (the 1) for demonstrating the adsorption | suction process in this embodiment. It is a block diagram which shows the structure of the image data adsorption | suction part of this embodiment. It is FIG. (2) for demonstrating the adsorption | suction process in this embodiment. It is a flowchart which shows the flow of the projection surface feature point detection process of this embodiment. It is a flowchart which shows the flow of the projection image edit / transmission process of this embodiment. It is a flowchart which shows the flow of the received image projection process of this embodiment.

  A projection surface such as a wall surface of a building has a line-like pattern indicating the shape of various parts such as a substantial body, a window frame, a pillar, and a joint. The prior art has a problem that it becomes difficult to see the projected image if these patterns and the projected image are combined.

  The multi-projection system disclosed below detects in advance a line-shaped pattern indicating the shape of various parts such as a window frame, a pillar, or a joint of a wall surface of a building such as an outdoor building. Then, it is converted into feature point information on the projection surface, which is a wall surface. Also, when creating and editing an image to be projected, the feature point information is synthesized and displayed on the image editing screen, and lines indicating the shape of various parts such as window frames, columns, and joints on the wall surface of the building that is the projection surface Create and edit images with the pattern in mind. Further, the feature point information of the wall surface is also stored in association with the image file.

  Therefore, even if the position of each of the plurality of projectors changes at the projector projection site, the difference between the feature point information of the newly read projection plane and the feature point information stored in association with the image file is detected, The image can be corrected. Even when an image is projected onto the wall of a building such as an outdoor building in a multi-projection system, even if the projected image is combined with a line pattern indicating the shape of various parts such as window frames, pillars, and joints. The projected image can be made difficult to see.

<Schematic functional configuration of this embodiment>
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a general projection system.

  As a largely configured device, there is a PC (personal computer) 100 having a wireless function and a wired network function, and a projector 120 that can be connected to the PC 100 by wireless or wired. A screen 129 in FIG. 1 is a screen that displays an image projected from the projector 120. The PC 100 includes an image data transmission unit 102 that can compress image data and transmit the image data to the projector 120 in order to project the internal image data onto the screen 129 via the projector 120. In addition, the projector 120 includes an image data receiving unit 121 that can receive and expand the compressed image data transmitted from the PC 100.

  Next, constituent blocks of the PC 100 will be described. The HDD 110 is a hard disk drive that can store image data and the like. The HDD driver 111 is means for controlling the HDD 110 by driver software called by application software operable on the PC 100 to read / write a stored file. Files read from the HDD 110 via the HDD driver 111 are sequentially stored in the buffer memory 112, and then are expanded by the file expansion unit 103 and stored in the frame memory 104 as raster images. The frame memory 104 is a memory for creating and editing image data expanded into a raster image. Reference numeral 105 in FIG. 1 denotes an image editing unit. The image editing unit 105 is a unit that can edit the image data expanded in the frame memory 104. The image data edited by the image editing unit 105 is stored in the frame memory 104 again. The image data stored in the frame memory 104 can be stored in the HDD 110 that is a nonvolatile memory through the buffer memory 112 while adjusting the device timing by the HDD driver 111.

  In addition, the image extraction unit 113 displays an image display area of the image data from the frame size information of the image data and the displayable size of the display 116 in order to display the image data expanded in the frame memory 104 on a display as an output device. It is a means for cutting out. The clipped image data is developed in the Video RAM 114, and the developed image data is converted into an image display signal via the display driver 115 and displayed on the display 116.

  Reference numeral 102 in FIG. 1 denotes the image data transmission unit. The image data transmission unit 102 is means for transmitting the image data expanded in the Video RAM 114 to the projector via a wireless LAN or a wired LAN in order to display the display screen of the PC 100 on the projector. Reference numeral 106 in FIG. 1 denotes a data compression unit. The data compression unit 106 is a means for compressing transmission data and reducing the transmission size. As a result, it is possible to efficiently perform data transmission via a relatively low speed wireless LAN or wired LAN. Reference numeral 107 in FIG. 1 denotes a transmission buffer. The transmission buffer 107 is a temporary memory for synchronizing with the wireless transmission timing of the wireless transmission unit 108. The wireless transmission unit 108 is means for transmitting image data according to a wireless LAN or wired LAN protocol via the data transmission I / F 109. The data transmission I / F 109 is a wireless antenna element in the case of a wireless LAN and a connector jack of a LAN cable in the case of a wired LAN.

  As a flow of image data in the PC 100, first, a file stored in the HDD 110 is read and developed in the frame memory 104. The developed image data is subjected to image editing by the image editing unit 105, and the image area cut out by the image extracting unit 113 is displayed on the display 116. At the same time, the image data transmitting unit 102 performs compression processing to obtain a radio signal. It is transmitted to the projector side.

  Next, a general projector side configuration will be described. Reference numeral 121 in FIG. 1 denotes an image data receiving unit. An image data receiving unit 121 that receives transmission image data from the PC 100 includes a data reception I / F 122, a wireless reception unit 123, a reception buffer 124, and a data decompression unit 125. The data reception I / F 122 is a wireless antenna element in the case of a wireless LAN, and a connector jack of a LAN cable in the case of a wired LAN. The wireless reception unit 123 is means for receiving transmission data from the PC 100 in accordance with a communication protocol via the data reception I / F 122. A reception buffer 124 in FIG. 1 is a temporary memory that stores data received by the wireless reception unit 123. Reference numeral 125 in FIG. 1 denotes means for decompressing data that has been compressed and transmitted from the transmission side. Reference numeral 126 in FIG. 1 denotes a projection buffer, and the image stored in this buffer is the image itself projected from the projector. Reference numeral 127 in FIG. 1 denotes a projection control unit. The projection control unit 127 is a unit that controls the mechanical unit of the projector and projects the image in the projection buffer 126. 1 denotes a mechanical configuration of the projector controlled by the projection control unit 127, such as a liquid crystal panel, a lamp, a lens, and a fan. Examples of the optical mechanism of the projector include, for example, a CRT projector that enlarges and projects an image displayed on a CRT using an optical system, a liquid crystal projector that transmits light through a liquid crystal panel, and a DMD (digital micromirror). DLP projector using a device), LCOS projector using a reflective liquid crystal element, and the like.

  Transmission data (compressed image data) from the PC 100 is received by the wireless reception unit 123 via the data reception I / F 122 and stored in the reception buffer 124. The stored compressed image data is sequentially read from the reception buffer 124 and subjected to decompression processing by the data decompression unit 125. The decompressed image data is stored in the projection buffer 126, and the stored image data is projected onto the screen 129 by the projection control unit 127 via the image projection unit 128.

  Next, the multi-projection system will be described. The multi-projection system is a technique for projecting onto a huge screen using a plurality of projectors as shown in FIG. FIG. 2A shows an example in which four projectors are used. The upper left projector projects an image on the upper left area of the screen, the upper right projector projects an image on the upper right area of the screen, the lower left projector projects an image on the lower left area of the screen, and the lower right projector projects an image on the lower right area of the screen. To do. (Since FIG. 2 (1) is difficult to understand, only the projection light of the upper left projector is written, but actually the projection light is output simultaneously from four projectors). Theoretically, in a multi-projection system, if the number of projectors is increased, any number of large images can be projected. However, it is difficult to prepare a large screen for that purpose.

  In recent years, in order to project a large image as a presentation, an image is projected using the present multi-projection system on a wall surface of an outdoor building, which is not a simple screen. For example, FIG. 2B shows an example in which an image is projected on the wall of a building such as an outdoor building. In this example as well, the multi-projection is in the form of 2 × 2 using four projectors. By using the wall of an original building such as a building, it is not necessary to prepare a huge screen, so it can be projected relatively easily if there is an appropriate building.

  However, unlike the screen, the wall of a building is a window frame of a building, a pillar, or a connecting portion on each floor (for example, see FIG. 3 (1)), and their shapes. May be visually combined with the projected image, making it difficult to see. FIG. 2B is an example in which the projection planes (dotted lines) of four projectors and the pattern of the building (solid lines) are combined. FIG. 3B shows an example of a projection image that is difficult to see.

  The technical idea disclosed in the present embodiment is difficult to see in this multi-projection system when various images of a building affect the projected image as described above when projecting an image on an outdoor building. The purpose of this is to suppress the influence of the structure of the building on the projected image.

<Detailed functional configuration of this embodiment>
FIG. 4 shows a block diagram of the multi-projection system of this embodiment, and the configuration of this embodiment will be described. A description common to the description of the general multi-projection system is omitted. First, description will be given for each block added in FIG. Reference numeral 415 in FIG. 4 denotes an image capturing unit. The image capturing unit 415 is a digital camera that can optically capture a projection surface and convert it into digital data. Reference numeral 416 in FIG. 4 denotes an imaging control unit. The imaging control unit 416 is means for controlling the image imaging unit 415 to read a digital image on the projection surface.

  Reference numeral 417 in FIG. 4 denotes a captured image binarization unit. The captured image binarization unit 417 is a unit that binarizes the digital image read by the imaging control unit 416 via the image imaging unit 415. In this case, the binarization processing is processing for converting the image on the projection surface into white or black and further simplifying the image. At this time, the ratio of the white level to the black level can be adjusted by setting a threshold level for binarization.

  Reference numeral 418 in FIG. 4 denotes a captured image thinning unit. The captured image thinning unit 418 is means for thinning the image binarized by the captured image binarization unit 417. Thinning is a process of detecting a center line of a line segment image having a certain width and performing line segmentation with a width of 1 pixel. Reference numeral 419 in FIG. 4 denotes a captured image vectorization unit. The captured image vectorization unit 419 regards the image thinned by the captured image thinning unit 418 as an assembly of line segments, and sets both ends of the line segments as relative to the lower left of the projection plane as a reference point (0, 0). It is indicated by an address (x, y).

  Reference numeral 420 in FIG. 4 denotes a captured image feature point extraction unit. The captured image feature point extraction unit 420 is a unit that detects the intersection of the line segment and the line segment from the vector data of the line segment specified by the captured image vectorization unit 419, and detects the intersection of the projection frame and the line segment. The intersections of these line segments and line segments, and the intersections of the projection frame and line segments are taken as the captured image feature points on the projection plane.

  Reference numeral 413 in FIG. 4 denotes a captured image feature point information 1 storage unit. The captured image feature point information 1 storage unit 413 is a memory that stores the captured image feature points detected by the captured image feature point extraction unit 420. By storing in the memory, the feature points can be read out from the captured image feature point information 1 storage unit 413 and used at the timing when the feature points are to be read out in the subsequent processing.

  Reference numeral 414 in FIG. 4 denotes a captured image feature point information 1 transmission unit. The captured image feature point information 1 transmission unit 414 is means for transmitting the captured image feature point information of the projection plane stored in the captured image feature point information 1 storage unit 413 to the PC 100 side. As will be described later, the PC 100 can perform processing using the captured image feature point information of the projection plane by this data transmission. Reference numeral 121 in FIG. 4 denotes a data transmission / reception I / F. Although only data reception is performed in 121 of FIG. 1, data can be transmitted in 121 of FIG. 4 according to the present embodiment. Reference numeral 109 in FIG. 4 denotes a data transmission / reception I / F. Also here, only data transmission was performed in 109 of FIG. 1, but data can also be received in 109 of FIG. 4 according to the present embodiment.

  Reference numeral 402 in FIG. 4 denotes a captured image feature point information 1 receiving unit. The captured image feature point information 1 receiving unit 402 is a means for receiving the captured image feature point information of the projection plane transmitted from the captured image feature point information 1 transmitting unit 414 of the projector unit 120 via the data transmitting / receiving unit I / F 122. is there. The received captured image feature point information of the projection plane is stored in the captured image feature point information 1 storage unit 403.

  The stored captured image feature point information of the projection plane is read out by the feature point setting unit 400, and the feature point setting unit 400 combines and displays the image data developed in the frame memory 104 from the feature point information. Thus, the user who is editing the image data can edit the image data on which the feature points are synthesized and displayed while being aware of the synthesized and displayed feature points. The image file that has been edited is compressed and transmitted to the projector side in the same manner as described above with reference to FIG. 1. At this time, the feature point information 2 writing unit 401 associates the image file with the image data to be transmitted. The feature point information is added to the header portion of the image data. As a result, the image file edited in consideration of the feature points and the associated feature points are transmitted to the projector side as one file data.

  The image data transmitted from the PC 100 side and its feature point information 2 are received by the data transmission / reception I / F 122, and the image data (compressed image) is received by the compressed image wireless reception unit 123, the reception buffer as described in FIG. 124, the image data is expanded via the image data expansion unit 125 and stored in the work memory 412. On the other hand, the feature point information 2 is read by the feature point information 2 reading unit 410, stored in the feature point information 2 storage unit 421, and the feature point information comparison unit 422 compares the feature point information 1 and the feature point information 2, The comparison result is a parameter for internal processing of the image data adsorption unit 411. In the image data adsorption unit 411, the feature point information 1 of the projection plane stored in the captured image feature point information 1 storage unit 413 and the feature point associated with the projection image read by the feature point information 2 reading unit 410. The difference with the information 2 is calculated, the image data stored in the work memory 412 is subjected to reduction / enlargement processing in the main scanning direction and sub-scanning direction, and image region movement processing, and the captured image feature point information 1 storage unit 413 A process of matching the image with the feature points of the projection plane read in real time is performed. As a result, it is possible to project an image that is edited into a projection image that matches the characteristics of the projection surface and that does not appear to be strange.

<Information processing in this embodiment>
There are three processes in the multi-projection system of this embodiment. The three processes are a projection plane feature point detection process (FIG. 10), a projection image editing / transmission process (FIG. 11), and a reception image projection process (FIG. 12). Hereinafter, these three processes will be described. The first projection plane feature point detection process will be described with reference to the flowchart of FIG. The first projection surface feature point detection process is a process of detecting a feature point of the surface to be projected.

(Flow 1000)
First, the image projection unit 128 projects a white image on the entire projection surface. Projecting a white image on the entire projection surface has an effect of recognizing a projection frame region and skipping a fine pattern on the projection surface.

(Flow 1001)
In a state where the white image is projected on the entire projection surface, the image capturing unit 415 captures an area that is the entire white image by the image capturing unit 415. An example of an image read by this flow is shown in FIG. FIG. 5A is an image captured from one projector among a plurality of projectors used in the multi-projection system. (Images taken by other projectors are omitted). When the projection surface of a white image is projected, for example, on the wall surface of a building as shown in FIG. 2B, images of the window frame, pillars, seams, etc. on the wall surface are captured.

(Flow 1002)
Next, the captured image binarization unit 417 performs binarization processing on the image of the projection plane imaged by the image imaging unit 415. By this binarization, the density of the image captured by the image capturing unit 415 is eliminated, and a black and white image can be obtained. Further, an image portion having a relatively low density can be skipped to a white image by adjusting a threshold level at the time of binarization. This binarized threshold level can also be set while the photographer is viewing in order to obtain a better binarized image. An example of the binarized image is shown in FIG.

(Flow 1003)
Next, the imaged image thinning unit 418 thins the image of the binarized projection plane. By this thinning, the center line of the line segment image having a certain width is detected, and the line segment is 1 pixel wide. An example of this thinning is shown in FIG.

(Flow 1004)
Next, the image thinned by the captured image vectorization unit 419 is regarded as an assembly of line segments, and relative addresses (x, y) with both ends of the line segments as reference points (0, 0) at the lower left of the projection plane It shows with. As shown in FIG. 5 (4), addresses at both ends of the line segment are obtained. The address 9 in the lower left corner is the reference point.

(Flow 1005)
Next, the captured image feature point extraction unit 420 detects the intersection of the line segment and the line segment from the vector data (address) of the line segment specified by the captured image vectorization unit 419, and detects the intersection of the projection frame and the line segment. . The intersections of these line segments and line segments, and the intersections of the projection frame and line segments are taken as the captured image feature points on the projection plane. As shown in FIG. 5 (5), the intersection between the line segment and the line segment, and the intersection point between the projection frame and the line segment are feature points.

(Flow 1006)
Next, the captured image feature point information 1 storage unit 413 stores the captured image feature point detected by the captured image feature point extraction unit 420. The feature points are stored in the memory, and the feature points are read from the captured image feature point information 1 storage unit 413 and used at the timing when the feature points are desired to be read in the third received image projection processing.

(Flow 1007)
Next, the captured image feature point information 1 transmission unit 414 transmits the captured image feature point information of the projection plane stored in the captured image feature point information 1 storage unit 413 to the PC 100 side. The PC 100 performs image editing processing using the captured image feature point information of the projection plane by this data transmission.

  The flow up to this point is a preparation stage before actually projecting image data from the projector. With the flow so far, the feature point of the pattern on the wall surface of the building that is the projection surface can be detected by the projector, and the feature point can be transmitted to the connected PC. Next, the flow on the PC side that receives this feature point will be described with reference to the flowchart of FIG. The processing is second projection image editing / transmission processing.

(Flow 1100)
An original image file is read from the HDD 110 to edit an image file to be projected by the multi-projection system. However, when a new image file is created, it is created from the beginning without being read from the HDD 110. This read file or created file is expanded in the frame memory 104.

(Flow 1101)
Next, the feature point information of the projection plane transmitted from the projector side stored in the captured image feature point information 1 storage unit 403 is read out to the feature point setting unit 400.

(Flow 1102)
Next, the feature point setting unit 400 synthesizes and displays the feature point on the image data developed in the frame memory based on the read feature point information 1 (feature point address) (synthesizes to the actual image data). It does n’t mean it ’s composed on the display). FIG. 5 (6) is an example of an image file read from the HDD 110 and feature points synthesized and displayed thereon. (○ is a feature point).

(Flow 1103)
Next, the read image file is edited based on the feature points synthesized and displayed on it. The read image is drawn regardless of the feature points, but by editing based on the synthesized and displayed feature points, it is possible to edit the feature points so that important parts of the image do not overlap. . The fact that the important part of the image overlaps the feature point means that after the image is projected by the projector, the pattern on the projection surface and the important part of the image overlap.

(Flow 1104)
When the editing is completed, the edited image is transmitted to the projector side together with the feature point information that is displayed and synthesized in order to project the image by the projector. The image data and the feature point can be associated by writing the feature point in the image data subjected to the image compression process and the header information thereof. FIG. 6 shows an example of the image file format. SOF: StartOfFile, SOH: StartOfHeader, HD: HeaderData, EOH: EndOfHeader, SOD: StartOfData, Data: Image data, EOD: EndOfData, EOF: EndOfFile format. In the HD portion, feature point information (feature point address) associated with image data is stored.

  The flow here is a flow from transmitting image data to the projector side for image projection in a state in which the image data edited while being aware of the feature points on the projection plane is associated with the feature point information. Next, the flow on the projector side that receives this image file will be described with reference to the flowchart of FIG. The process is a third received image projection process.

(Flow 1200)
The image projection unit 128 projects a white image on the entire projection surface. Projecting a white image on the entire projection surface has an effect of recognizing a projection frame region and skipping a fine pattern on the projection surface.

(Flow 1201)
In a state where the white image is projected on the entire projection surface, the image capturing unit 415 captures an area that is the entire white image by the image capturing unit 415. An example of an image read by this flow is shown in FIG. FIG. 5A is an image captured from one projector among a plurality of projectors used in the multi-projection system. (Images taken by other projectors are omitted). When the projection surface of a white image is projected, for example, on the wall surface of a building as shown in FIG. 2B, images of the window frame, pillars, seams, etc. on the wall surface are captured.

(Flow 1202)
Next, the captured image binarization unit 417 performs binarization processing on the image of the projection plane imaged by the image imaging unit 415. By this binarization, the density of the image captured by the image capturing unit 415 is eliminated, and a black and white image can be obtained. Further, an image portion having a relatively low density can be skipped to a white image by adjusting a threshold level at the time of binarization. This binarized threshold level can also be set while the photographer is viewing in order to obtain a better binarized image. An example of the binarized image is shown in FIG.

(Flow 1203)
Next, the imaged image thinning unit 418 thins the image of the binarized projection plane. By this thinning, the center line of the line segment image having a certain width is detected, and the line segment is 1 pixel wide. An example of this thinning is shown in FIG.

(Flow 1204)
Next, the image thinned by the captured image vectorization unit 419 is regarded as an assembly of line segments, and relative addresses (x, y) with both ends of the line segments as reference points (0, 0) at the lower left of the projection plane It shows with. As shown in FIG. 5 (4), addresses at both ends of the line segment are obtained. The address 9 in the lower left corner is the reference point.

(Flow 1205)
Next, the captured image feature point extraction unit 420 detects the intersection of the line segment and the line segment from the vector data (address) of the line segment specified by the captured image vectorization unit 419, and detects the intersection of the projection frame and the line segment. . The intersections of these line segments and line segments, and the intersections of the projection frame and line segments are taken as the captured image feature points on the projection plane. As shown in FIG. 5 (5), the intersection between the line segment and the line segment, and the intersection point between the projection frame and the line segment are feature points.

(Flow 1206)
The feature points are sequentially read, and if the detected feature points remain, the process proceeds to flow 1207, and if not, the process of flow 1210 is performed.

(Flow 1207)
Next, the feature point information 1 of the projection surface extracted again up to the flow 1205 is compared with the feature point information 2 transmitted from the PC side to the projector side in the flow 1104 of the second projection image editing / transmission process. Find the difference information.

(Flow 1208)
As a result, it is determined whether or not there is a difference greater than or equal to a value defined in advance, and when there is a difference greater than or equal to a value defined in advance, and the subsequent process is switched based on the determination result. When there is no difference greater than the standard, the flow 1209 is performed, and when there is a difference greater than the standard, the flow 1206 is performed.

(Flow 1209)
This is a process when there is no difference greater than or equal to the specified value in the flow 1208. Since there is no difference greater than the standard, the feature points of the projection plane extracted again up to the flow 1205 and the feature points transmitted from the PC side to the projector side in the flow 1104 of the second projection image editing / transmission process are shown. , Which is closer than a predetermined physical distance, and is transmitted from the PC side to the projector side in the flow 1104 of the second projection image editing / transmission process to the feature points of the projection plane extracted again up to the flow 1205 Image editing is performed so as to superimpose the feature points. Specifically, an image area suction process is performed. The adsorption process will be described later.

(Flow 1210)
When the processing for all the feature points is completed in the flow 1206, the processing of the flow 1210 is performed. A flow 1210 projects the projection image on which the feature point suction processing has been performed in the previous stage from the image projection unit 128 onto the screen.

<Adsorption treatment>
Next, image adsorption processing will be described. In the present embodiment, processing for fitting an image in a certain area is called adsorption processing. As shown in FIG. 7 (1), this is a process of fitting certain images e, f, g, and h into certain regions a, b, c, and d. However, as shown by the relationship between the point a and the point e, the processing is performed so as to attract when the distance from the point a to the point e is L or less, and not when it is more than that. Similar processing is performed for point b and point f, point c and point g, and point d and point h. FIG. 7B shows the result of adsorbing the images e, f, g, and h in the areas a, b, c, and d.

  A block diagram for performing this adsorption processing is shown in FIG. The feature point calculation unit 801 is the difference between the feature point of the image data transmitted from the PC side in association with the image data and the feature point detected by image processing on the image of the projection plane actually captured by the imaging unit. Is a means for calculating and detecting. A main scanning reduction / enlargement process, a sub-scanning reduction / enlargement process, and an image movement process as shown in FIG. 8 are performed from the difference information of each feature point detected by the feature point calculation unit 801.

  In the main scanning reduction / enlargement processing 802, as shown in FIG. 9A, the difference between the main scanning length (point b-point a) and the main scanning length of the image (point f-point e) is shown. Is used to calculate a scaling factor in the main scanning direction of the image, and performs reduction / enlargement processing in the main scanning direction of the image according to the scaling factor (reduction processing if the scaling factor is less than 1, enlargement processing if the scaling factor is 1 or more) I do).

  In the sub-scanning reduction / enlargement processing 803, as shown in FIG. 9B, the difference between the sub-scanning length (point c-point a) and the main scanning length (point g-point e) of the image to be inserted. Then, a scaling factor in the sub-scanning direction of the image is calculated, and reduction / enlargement processing in the sub-scanning direction of the image is performed according to the scaling factor. In this example, since the image area is rectangular, the value of point b-point a and the value of point d-point c are the same, so either calculation may be performed.

  As shown in FIG. 9 (3), the image moving process 804 is performed in order to align the inlaid area after the main scanning reduction / enlargement process and the sub-scanning reduction / enlargement process with the position of the image having the same image size. Move to the area where the image is to be fitted so that point a and point e, point b and point f, point c and point g, point d and point h overlap.

  Even when an image is projected onto the wall surface of a building such as an outdoor building by such processing, a line pattern indicating the shape of various parts such as a window frame, a pillar, or a joint is detected in advance, and this is detected on the wall surface. When creating and editing an image to be projected, the feature point information can be synthesized and displayed on the image editing screen, and the window on the wall of the building as the projection plane It is possible to create and edit an image while conscious of a line pattern indicating the shape of various parts such as a frame, a column, and a joint.

  Furthermore, the feature point information of the wall surface can be stored in association with the image file. Even if the position of each of the plurality of projectors changes at the projector projection site, the feature point information and image file of the newly read projection surface are read again. Detects and corrects the difference with the feature point information stored in association with the projected image, and can fit the projected image into a line-shaped pattern showing the shape of various parts such as window frames, pillars, and joints on the wall of the building .

<Action and effect>
The operation and effect of the above embodiment will be described below.
The technical idea disclosed in the above embodiment can automatically fit the projected image into a characteristic pattern such as a window frame, a pillar, or a seam on the wall surface of a large-scale building. There is an effect that an easy-to-see image can be projected. That is, it is possible to suppress the influence of the building pattern on the projected image.

  Also, when reading the state of the wall surface of the building by the image pickup means, the image projection means projects a white image on the entire surface, and thus fine patterns such as cracks, stains and dirt on the wall surface of the building are blown off by exposure. There is an effect that can be.

  Further, the feature point detection means linearizes the wall pattern of the building read from the imaging means by the line segmentation means, and uses the intersection of the plurality of line segments as a feature point, thereby making the window frame of the wall surface of the building There is an effect that it is possible to finely detect characteristic features such as pillars and joints.

  In addition, the first editing means includes feature point synthesis display means for synthesizing and displaying the first feature point on the image being edited, and the image can be edited based on the feature point. The image can be edited while being visually observed, and the editing work can be greatly simplified.

  The feature point comparison means reads the first feature point indicated by the relative address in the image frame of the image file and the image point when the image file is projected and is detected by the feature point detection means. By comparing the second feature points indicated by the relative addresses in the projection plane, it is possible to edit the projection image that matches the projection plane.

  The second editing means includes an image reduction means capable of reducing a specific image area, an image enlargement means capable of enlargement processing, and an image moving means capable of movement processing. The first feature point is obtained by the feature point comparison means. When the distance between the first feature point and the second feature point is shorter than a predetermined length, the image data may be reduced, enlarged, or moved so that the first feature point overlaps the second feature point. By doing so, there is an effect that the closest feature points can be accurately superimposed.

104 frame memory 105 image editing unit 128 image projecting unit 400 feature point setting unit 403 captured image feature point information 1 storage unit (PC side)
411 Image data adsorption unit 413 Captured image feature point information 1 storage unit (projector side)
420 Captured Image Feature Point Extraction Unit 422 Feature Point Information Comparison Unit 801 Feature Point Difference Calculation Unit 802 Main Scanning Reduction / Enlargement Processing 803 Subscanning Reduction / Enlargement Processing 804 Image Movement Processing

Japanese Patent No. 4637845

Claims (7)

Imaging means for reading the state of the wall surface of the building;
Feature point detecting means for detecting a first feature point of the image from the read wall surface image;
First image editing means for editing an image based on the first feature point;
Feature point storage means for storing the image edited image and the first feature point in association with each other;
A first feature point stored in association with an image and a feature point comparison unit that compares the position of the second feature point read by the imaging unit and detected by the feature point detection unit;
Second image editing means for automatically editing the image so that the first feature point of the image to be projected overlaps the second feature point detected again;
Image projection means for projecting the edited image on the wall of the building;
A projection system characterized by comprising:   The projection system according to claim 1, wherein the image projecting unit projects a white image on the entire surface when the imaging unit reads the state of the wall surface of the building.   3. The feature point detecting means, wherein the wall surface pattern of the building read from the imaging means is line-divided by a line dividing means, and an intersection of the plurality of line segments is used as a feature point. The projection system described in 1.   The first image editing means includes feature point synthesis display means for synthesizing and displaying the first feature point on the image being edited, and the image can be edited based on the feature point. The projection system according to any one of claims 1 to 3.   The feature point comparison unit reads the first feature point indicated by the relative address in the image frame of the image file and the image point when the image file is projected and is detected by the feature point detection unit. 5. The projection system according to claim 1, wherein the second feature point indicated by the relative address in the projection plane is compared. 6.   The second image editing means includes an image reduction means for reducing a specific image area, an image enlargement means for enlargement processing, and an image movement means for movement processing, and includes a first feature point and a second feature point. When it is determined that the distance between the feature points is shorter than a predetermined length, the image data is reduced, enlarged, or moved so that the first feature points overlap the second feature points. The projection system according to claim 1, wherein the projection system is characterized in that: An imaging step for reading the state of the wall of the building;
A feature point detecting step of detecting a first feature point of the image from the read wall surface image;
A first image editing step of editing an image based on the first feature point;
A feature point storing step of storing the image edited image and the first feature point in association with each other;
A first feature point stored in association with an image, and a feature point comparison step that compares the position of the second feature point read in the imaging step and detected in the feature point detection step;
A second image editing step of automatically editing the image so that the first feature point of the image to be projected overlaps the second feature point detected again;
An image projection step of projecting the edited image on the wall of the building;
A projection method comprising:

Images