JP2010250560A - Image processing apparatus and method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform image processing in consideration of a shape of a three-dimensional object and an observation position of an image projected to the three-dimensional object. <P>SOLUTION: The shape information of the three-dimensional object to which an image is to be projected, observation position information indicating an observation position of the image to be projected to the three-dimensional object and device position information indicating the position of an image projector for projecting the image to the three-dimensional object are input (S11 to S13). Then, an observation angle indicating relation between a direction of each polygon expressing the shape information and the observation position is calculated based on the observation position information and the shape information (S14). The image to be output to the image projector is corrected based on the observation angle, the shape information and the device position information (S15 to S17). <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to image processing when an image is projected onto a three-dimensional object.

  Opportunities for computer graphics (CG) to be used to simulate interior colors and patterns have increased. If CG is used, the user can select the color and pattern and simulate the completed interior image on the monitor. However, the simulation on the monitor is different in texture from the real thing, and furthermore, the display size is limited, and it cannot be said that the image in the real space can be faithfully reproduced.

  Therefore, using an image projection device such as a projector, the image is projected onto the surface (projection plane) of a three-dimensional object having the same three-dimensional shape and size as the real object, thereby reproducing the image in real space more faithfully. The method to do was put into practical use. At that time, it is necessary to correct the image in accordance with the three-dimensional shape of the projection surface.

  As a technique for projecting an image in accordance with the shape of a projection surface having a three-dimensional shape, for example, there is a technique disclosed in Patent Document 1. In this technique, a projection image is corrected using a mesh model for a projection plane having an arbitrary three-dimensional shape, and distortion of projection light visually recognized by an observer is suppressed.

  However, depending on the position of the viewer with respect to the shape of the projection plane, the projected image may appear distorted or blurred with only correction that takes into account the shape of the projection plane. Especially, distortion and blurring of the pattern are strongly recognized. Is done.

Japanese Patent No. 4013989

  An object of the present invention is to perform image processing in consideration of the shape of a three-dimensional object and the observation position of an image projected on the three-dimensional object.

  The present invention has the following configuration as one means for achieving the above object.

  Image processing according to the present invention includes three-dimensional object shape information for projecting an image, observation position information indicating an observation position of an image projected on the three-dimensional object, and image projection for projecting an image on the three-dimensional object. The apparatus position information indicating the position of the apparatus is input, and based on the observation position information and the shape information, an observation angle indicating the relationship between the direction of each polygon representing the shape information and the observation position is calculated, and the observation angle The image output to the image projection apparatus is corrected based on the shape information and the apparatus position information.

  According to the present invention, it is possible to reduce image distortion and blurring when an image projected on a three-dimensional object is observed.

A block diagram showing a configuration example of an image processing apparatus of an embodiment, Block diagram showing a functional configuration example of an image processing device, A flowchart illustrating image processing performed by the image processing apparatus; A diagram showing an example of 3D shape information, A flowchart for explaining processing of the angle information generation unit; A diagram showing the relationship between the polygon and the observation position indicated by the observation position information, It is a flowchart explaining the process of a correction | amendment part.

  Hereinafter, image processing according to an embodiment of the present invention will be described in detail with reference to the drawings.

[Device configuration]
FIG. 1 is a block diagram illustrating a configuration example of the image processing apparatus 11 according to the embodiment.

  The CPU 401 executes a program stored in a read only memory (ROM) 402 and a hard disk drive (HDD) 408 using a random access memory (RAM) 403 as a work memory. Then, various processes including image processing described later are executed by controlling each component described later via the system bus 404.

  An input interface (I / F) 405 is a serial bus interface such as USB or IEEE1394 for connecting an input device 406 such as a keyboard, mouse, digital camera, scanner, or three-dimensional laser scanner. The CPU 401 can read data from the input device 406 via the input I / F 405.

  The HDD I / F 407 is an interface such as a serial ATA (SATA) for connecting a secondary storage device such as the HDD 408 or an optical disk drive. The CPU 401 can read data from the HDD 408 and write data to the HDD 408 via the HDD I / F 407. Furthermore, the CPU 401 can load the data stored in the HDD 408 into the RAM 403 and similarly store the data stored in the RAM 403 in the HDD 408. The CPU 401 can execute the data loaded in the RAM 403 as a program.

  The video I / F 409 is an interface for connecting the monitor 410. The CPU 401 can control the video I / F 409 to display an arbitrary character string, image, and user interface (UI) on the monitor 410.

  The output I / F 411 is a serial bus interface for connecting an output device 412 such as a printer, a plotter, a film recorder, or a projector. The CPU 401 can send data to the output device 412 via the output I / F 411 to execute printing, recording, projection, and the like. Note that when a bidirectional communication interface such as USB is used, the input I / F 405 and the output I / F 411 can be combined into one.

  A network interface card (NIC) 413 is an interface for connecting to the network 414.

[Function configuration]
FIG. 2 is a block diagram illustrating a functional configuration example of the image processing apparatus 11. Although details will be described later, a computer-executable program describing the processing procedure shown in FIG. 3 is stored in the ROM 402 or HDD 408 in advance. The CPU 401 loads the program into the RAM 403 in accordance with a user instruction, and executes the program, thereby realizing the functional configuration shown in FIG. 2 and starting the image processing shown in FIG.

  The 3D shape acquisition unit 101 acquires information (hereinafter referred to as 3D shape information) regarding the three-dimensional shape of the projection surface onto which the image is projected. The observation position acquisition unit 102 acquires information (hereinafter referred to as observation position information) related to a position where an observer who observes an image projected on the projection surface is present.

  The apparatus position acquisition unit 103 acquires information (hereinafter referred to as apparatus position information) related to the position where the image projection apparatus 12 that projects an image on the projection surface is present. The angle information generation unit 104 calculates an angle (hereinafter referred to as observation angle distribution information) formed by the observer and the three-dimensional shape at each point on the projection plane.

  The correction unit 105 corrects the input image based on the observation angle distribution information, and generates an image to be projected on the projection plane. The buffer memory 106 is a memory allocated temporarily to the RAM 403, for example, for temporarily storing calculation results during processing. The image projection device 12 is an output device 412 such as a projector connected to the image processing device 11 via an output I / F 411, for example.

[Image processing]
FIG. 3 is a flowchart illustrating image processing executed by the image processing apparatus 11.

  The CPU 401 inputs the 3D shape information of the projection plane by the 3D shape acquisition unit 101 and stores the 3D shape information in the buffer memory 106 (S11).

  FIG. 4 is a diagram showing an example of 3D shape information. The 3D shape information is expressed as a set of polygons arranged in a three-dimensional space. The 3D shape acquisition unit 101 measures the surface shape (projection plane) of a three-dimensional object on which an image is projected using a measurement device such as a known three-dimensional laser scanner, and inputs 3D shape information, or HDD408 The 3D shape information that is measured in advance is read.

  Next, the CPU 401 inputs observation position information from the observation position acquisition unit 102, and stores the observation position information in the buffer memory 106 (S12). The observation position information is input by the user via the UI, but is read from the HDD 408 or the like. The observation position information includes information such as a distance and an angle indicating a relationship between a position where an object on which an image is projected (hereinafter referred to as an object position) and a position where an observer exists (hereinafter referred to as an observation position). Have.

  Next, the CPU 401 inputs device position information of the image projection device 12 through the device position acquisition unit 103 and stores the device position information in the buffer memory 106 (S13). The device position information is input from the HDD 408 or the like by the user through the UI. Further, the observation position information includes information such as a distance and an angle indicating a relationship between the object position and a position where the image projection apparatus 12 exists (hereinafter, apparatus position).

  Next, as will be described in detail later, the CPU 401 generates observation angle distribution information based on the 3D shape information and the observation position information stored in the buffer memory 106 by the angle information generation unit 104, and the observation angle distribution information is stored in the buffer memory. It is stored in 106 (S14).

  Next, the CPU 401 inputs an image designated by the user (hereinafter referred to as an original image) from the HDD 408 or the like via the UI (S15). Then, the correction unit 105 corrects the original image based on the apparatus position information and the observation angle distribution information stored in the buffer memory 106, as will be described in detail later (S16). Then, the corrected image is output to the image projection device 12 (S17), and the corrected image is projected onto the projection plane.

[Angle information generator]
FIG. 5 is a flowchart illustrating the processing of the angle information generation unit 104.

  The angle information generation unit 104 acquires 3D shape information of the projection plane from the buffer memory 106 (S41), and acquires observation position information (S42).

  FIG. 6 is a diagram showing the relationship between the polygon 53 and the observation position 52 indicated by the observation position information.

  The angle information generation unit 104 selects one polygon from each polygon constituting the 3D shape information (S43). Then, the center of gravity of the selected polygon 53 and the observation position 52 are connected by a straight line 54, and an angle formed by the straight line 54 and the normal line 51 of the polygon 53 (hereinafter, the observation angle θ) is calculated (S44).

  Next, the angle information generation unit 104 determines whether or not the calculation of the observation angle θ in step S44 has been performed for all the polygons constituting the 3D shape information (S45), and the calculation of the observation angle θ for all the polygons has not been performed. If yes, the process returns to step S43.

  When the calculation of the observation angles θ for all the polygons is completed, the angle information generation unit 104 generates observation angle distribution information from the observation angles θ for each polygon, stores the observation angle distribution information in the buffer memory 106 (S46), The process ends.

[Correction section]
FIG. 7 is a flowchart for explaining the processing of the correction unit 105.

  The correcting unit 105 acquires observation angle distribution information from the buffer memory 106 (S51), and acquires apparatus position information (S52). Then, a polygon whose observation angle θ indicated by the observation angle distribution information is subdivided is subdivided, and a virtual projection plane is generated for each subdivided polygon (S53).

  The virtual projection plane is orthogonal to a straight line 54 that connects the center of gravity of the subdivided polygon and the observation position 52. In other words, the normal 51 of the virtual projection plane coincides with the straight line 54. Further, the predetermined threshold (observation angle) may be designated by the user via the UI, or may be calculated from the 3D shape information and the observation position information.

  The correction unit 105 calculates 3D shape information in consideration of an observation position having a projection plane whose observation angle θ is less than a predetermined threshold and a virtual projection plane obtained by subdividing a projection plane whose observation angle θ is equal to or greater than a predetermined threshold. Shape information) is generated and stored in the buffer memory 106 (S54).

  Next, the correction unit 105 sets a perspective transformation model for projecting an image from the image projection device 12 on the projection plane indicated by the corrected shape information, performs a projection simulation for projecting the original image, and is projected onto the projection plane. An image (hereinafter referred to as a projected image) is obtained (S55).

  Next, the correction unit 105 sets a perspective transformation model for projecting an image from the image projection device 12 on the projection plane indicated by the 3D shape information, and performs a back projection simulation for back projecting the projection image obtained in step S54. Then, the image projection device 12 generates an image to be projected (S56). The image to be projected by the image projection device 12 is a corrected image of the original image to be output by the image processing device 11.

  The process related to the perspective transformation is a process well known in the field of 3DCG, and high-speed processing circuits are already widely used. Accordingly, the correction process can be speeded up by using the circuit and controlling the approximation accuracy of the virtual projection plane.

  With the above image correction, it is possible to reduce image distortion and blur caused by the shape of the projection surface and the observation position when observing the image projected on the projection surface of the three-dimensional object.

[Other embodiments]
The present invention supplies a recording medium (or storage medium) recording a program for realizing the functions of the above embodiments to the apparatus (or system), and executes the program read from the recording medium by the computer (CPU or MPU) of the apparatus It is also realized by doing. In this case, the program stored in the recording medium realizes the functions of the above embodiments, and the program and the computer-readable recording medium storing the program constitute the present invention.

  Further, the above functions are not only realized by the execution of the program. That is, it includes a case where an operating system (OS) running on a computer and / or another program or the like performs part or all of actual processing or control by the instruction of the program, thereby realizing the above functions. The program may be stored in a memory of a device such as a function expansion card or unit connected to the computer. In other words, it includes the case where the CPU or the like of the device performs part or all of the actual processing according to the instructions of the program, thereby realizing the above functions. When the present invention is applied to a recording medium, the recording medium stores a program corresponding to or related to the flowchart described above.

Claims (8)

Shape information of a three-dimensional object projecting an image, observation position information indicating an observation position of an image projected on the three-dimensional object, and apparatus position information indicating a position of an image projection apparatus that projects an image on the three-dimensional object An input means for inputting
Based on the observation position information and the shape information, calculation means for calculating an observation angle indicating a relationship between the direction of each polygon representing the shape information and the observation position;
An image processing apparatus comprising: correction means for correcting an image output to the image projection apparatus based on the observation angle, the shape information, and the apparatus position information.   2. The image processing apparatus according to claim 1, wherein the shape information represents a three-dimensional shape of a projection plane that is a surface of the three-dimensional object.   3. The image processing apparatus according to claim 1, wherein the observation angle represents an angle formed by a straight line connecting the center of gravity of the polygon and the observation position with a normal line of the polygon.   The correction means subdivides a polygon having an observation angle equal to or greater than a predetermined value, so that a straight line connecting the center of gravity of the subdivided polygon and the observation position matches a normal line of the subdivided polygon. 4. The image processing apparatus according to claim 1, wherein the corrected shape information obtained by correcting the shape information is generated by setting the orientation of the converted polygonal surface.   The correction means performs a projection simulation of an image to be output to the image projection device using a perspective transformation model set based on the corrected shape information and the device position information, and a perspective set based on the shape information and the device position information. 5. The image processing apparatus according to claim 4, wherein a back projection simulation of the result of the projection simulation is performed using a conversion model, and a corrected image to be output to the image projection apparatus is generated. Shape information of a three-dimensional object projecting an image, observation position information indicating an observation position of an image projected on the three-dimensional object, and apparatus position information indicating a position of an image projection apparatus that projects an image on the three-dimensional object Enter
Based on the observation position information and the shape information, calculate the observation angle indicating the relationship between the observation position and the orientation of each polygon representing the shape information,
An image processing method, comprising: correcting an image output to the image projection device based on the observation angle, the shape information, and the device position information.   6. A non-transitory computer-readable storage medium storing a program for controlling a computer apparatus to function as each unit of the image processing apparatus according to claim 1.   8. A computer-readable recording medium on which the program according to claim 7 is recorded.

Images