JP2013192189A - Image processing device, projection system, program and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable mapping of a texture on a surface of an object projected even if the object projected changes its shape, moves or the like.SOLUTION: The image processing device 4 of an apparatus 1 used for projection mapping comprises a computer 40 that repeatedly executes: image inputting processing for inputting images including an image of an object projected 99 taken by an electronic camera 43; coordinate conversion processing for coordinate-converting an image 70 inputted by the image input process from a coordinate system of the electronic camera 43 to a coordinate system of a display screen; region extraction processing for extracting a closed region 73a of the image 73 of the object projected 99 of images 72 converted by the coordinate conversion processing; mapping processing for mapping an object 41b on the closed region 73a extracted by the region extraction processing; and picture-signal generation processing for generating a picture signal of the display screen including the object 41b mapped by the mapping processing.

Description

  The present invention relates to an image processing apparatus, a projection system, a program, and an image processing method.

  An object (hereinafter referred to as a projection object) is placed in front of the illumination device, and the illumination device irradiates the projection object with irradiation light, thereby mapping an object such as a pattern or color on the surface of the projection object. There is technology to do. The technique described in Patent Literature 1 uses an irradiation light projection unit (2) that is a projection device and an illumination control device (1) that is an image processing device, and performs image (video) by computer graphic processing by the illumination control device (1). ) And projecting the image by the irradiation light projection unit (2), the object in the image is mapped onto the surface of the projection object (20). Specifically, data obtained by modeling the projection object (20) in three dimensions is prepared in advance in the illumination device control unit (1), and an object to be mapped onto the surface of the projection object (20) and its background are displayed. When the included image is input to the illumination control device (1) as a video signal, the illumination control device (1) corrects the video signal based on the three-dimensional model data, and the image based on the corrected video signal is irradiated light. Projected by the projection unit (2). Then, the object is mapped only on the surface of the projection object, and light of the background color is projected around the projection object.

JP 2009-49007 A

However, since the three-dimensional model data of the projection object (20) is prepared in advance in the illumination control device (1), if the projection object (20) changes its shape or moves, the object becomes the projection object (20 ) Will not be mapped to the surface.
Therefore, a problem to be solved by the present invention is to enable an object to be mapped on the surface of the projection object even if the projection object changes its shape or moves.

  In order to solve the above problems, an image processing apparatus according to the present invention is acquired from an imaging unit, an image acquisition unit that acquires an image including a projection object imaged by the imaging unit, and the image acquisition unit. And a mapping means for mapping an object corresponding to the projection area to the projection area.

  The program according to the present invention includes a computer of an image processing apparatus including an imaging unit, an image acquisition unit that acquires an image including an object captured by the imaging unit, and an image acquired from the image acquisition unit. It functions as an area extracting means for extracting a projection area of a projection and a mapping means for mapping an object corresponding to the projection area to the projection area.

  An image processing method according to the present invention includes an image acquisition process in which a computer of an image processing apparatus including an imaging unit acquires an image including a projection image captured by the imaging unit, and an image acquired by the image acquisition process. And a mapping process for mapping an object corresponding to the projection area to the projection area.

  According to the present invention, an object image is mapped to a projection object even if the projection object undergoes a shape change, a movement, or the like.

It is a perspective view which shows the use condition of the projection mapping apparatus which concerns on embodiment of this invention. It is a block diagram of the projection mapping apparatus. It is the flowchart which showed the flow of the process performed by the computer of the projection mapping apparatus. It is drawing which showed typically the coordinate system of the alignment mark memorize | stored in the computer, and a display screen. It is drawing which showed the projection screen actualized on the screen. It is drawing which showed the image imaged by the imaging means. It is the figure which showed the image obtained by converting the image imaged by the imaging means into the coordinate system of a display screen. It is the figure which showed the image obtained by performing an area | region extraction process with respect to the image converted into the coordinate system of a display screen. It is the figure which showed the image obtained by performing an area | region extraction process with respect to the image converted into the coordinate system of a display screen. It is drawing which showed the change of a to-be-projected object, and the change of a texture. It is drawing which showed the change of a to-be-projected object, and the change of a texture. It is drawing which showed the change of a to-be-projected object, and the change of a texture.

  EMBODIMENT OF THE INVENTION Below, the form for implementing this invention is demonstrated using drawing. The embodiments described below are provided with various technically preferable limitations for carrying out the present invention. Therefore, the technical scope of the present invention is not limited to the following embodiments and illustrated examples.

  FIG. 1 is a perspective view showing a use state of a projection mapping apparatus 1 as a projection system. The projection mapping apparatus 1 detects a projection area 98 of the projection object 99 and projects a texture 97 such as a pattern, color, mark, symbol, etc. on the projection area 98. Even if the position, size, or range of the projection area 98 changes due to movement, rotation, direction change, displacement, movement, expansion, contraction, shape change, or other changes of the projection object 99, the texture 97 is Follow changes. The projection screen 90 is the maximum projection range that can be projected by the projection mapping apparatus 1 when projected by the projection mapping apparatus 1 onto a screen in front of the projection mapping apparatus 1. A range 96 is a shooting range of the electronic camera 43 provided in the projection mapping apparatus 1.

FIG. 2 is a block diagram of the projection mapping apparatus 1.
The projection mapping apparatus 1 includes a projection apparatus main body 2 and an image processing apparatus 4. The projection apparatus main body 2 and the image processing apparatus 4 may be integrated and incorporated in one casing, or the projection apparatus main body 2 and the image processing apparatus 4 are incorporated in separate casings, and the projection apparatus main body 2 and the image The processing device 4 may be connected by a video signal cable.

  The image processing device 4 detects the projection region 98, generates a video signal of a display screen for mapping the texture 97 to the projection region 98, and outputs the video signal to the projection device main body 2. The projection apparatus main body 2 receives the video signal output from the image processing apparatus 4 and projects a texture 97 on the projection area 98 by projecting a display screen based on the video signal forward.

  The projection apparatus main body 2 includes a control unit 20, an input device 21, a video signal conversion unit 22, a video signal processing unit 23, a display controller 24, a display element 25, a light irradiation device 26, a projection lens 27, a lens driving unit 28, and the like. .

  The input device 21 is a plurality of switches (push buttons), a remote controller, or a combination thereof. The input device 21 outputs a signal corresponding to the operation content to the control unit 20 when operated by the user.

  The video signal output by the image processing device 4 is input to the video signal converter 22. When the video signal output by the image processing device 4 is analog, the video signal conversion unit 22 converts the analog video signal input from the image processing device 4 into a digital video signal and converts the digital video signal into a predetermined format. Convert to When the video signal output by the image processing device 4 is digital, the video signal conversion unit 22 converts the digital video signal into a predetermined format. When the digital video signal output by the image processing device 4 has a predetermined format, the video signal conversion unit 22 passes the digital video signal without converting it.

  The video signal processing unit 23 performs various types of signal processing on the video signal output by the video signal conversion unit 22. For example, the video signal processing unit 23 scales the video signal output by the video signal conversion unit 22 to increase or decrease the number of pixels (resolution) of the video signal. In addition, the video signal processing unit 23 performs keystone correction and the like by performing coordinate transformation (for example, projective transformation) of the screen (image) based on the scaled video signal. The video signal processed by the video signal processing unit 23 is transferred to the display controller 24. The projection screen 90 is the maximum projection range after the keystone correction.

  The display controller 24 drives the display element 25 based on the video signal input from the video signal processing unit 23. The display controller 24 performs PWM control or PNM control for each pixel of the display element 25 in accordance with the video signal input from the video signal processing unit 23, so that light emitted to the display element 25 is modulated for each pixel by the pixel. Is done. Thereby, a screen (image) is formed by the display element 25.

  The display element 25 is a DMD (digital micromirror device), a transmissive LCD panel (liquid crystal display panel), or a reflective LCD panel having a plurality of pixels arranged in a two-dimensional array. When the display element 25 is a DMD, the pixel of the display element 25 is a movable micromirror, and when the display element 25 is an LCD panel, the pixel of the display element 25 is a liquid crystal shutter element.

  The display controller 24 controls the light irradiation device 26. The light irradiation device 26 irradiates the display element 25 with visible light. Specifically, the light irradiation device 26 is of a time division method, a color separation method, or an independent light source method.

When the light irradiation device 26 is a color separation method, the light irradiation device 26 has a white light source, a color separator, and the like, and white light emitted from the white light source is emitted by a color separator (for example, red, green, blue). ). In this case, the display element 25 is provided for each color, light of a plurality of colors is irradiated to the display element 25, and light transmitted or reflected by each display element 25 is synthesized.
When the light irradiation device 26 is a time-division method, the light irradiation device 26 sequentially irradiates the display element 25 with visible light of different colors (for example, red light, green light, and blue light) one by one. The period corresponds to a period of one frame. The time division type light irradiation device 26 is of a color filter type or a color light source type. When the light irradiation device 26 is a color filter system, the light irradiation device 26 includes a color filter, a spindle motor, a white light source, and the like. The color filter is rotated by the spindle motor, and white light emitted from the white light source is rotating. By passing through the color filter, the white light is sequentially converted into each color (red, green, blue) and converted. When the light irradiation device 26 is a color light source system, the light irradiation device 26 has a plurality of light sources having different colors (for example, a red light source, a green light source, and a blue light source), and these light sources are sequentially turned on repeatedly.
When the light irradiation device 26 is an independent light source method, the light irradiation device 26 includes a plurality of light sources that emit visible light of different colors (for example, red light, green light, and blue light). In this case, the display element 25 is provided for each color, light of a plurality of colors is irradiated to the display element 25, and light transmitted or reflected by each display element 25 is synthesized.
The light source is a semiconductor light emitting element (for example, laser diode, light emitting diode) or a lamp (halogen lamp, xenon lamp, metal halide, mercury lamp, discharge lamp), or a phosphor combined with a semiconductor light emitting element or lamp. Say things.

  The projection lens 27 projects the display screen (image) formed by the display element 25 onto a screen, and converts the display screen formed on the display element 25 into a projection screen 90 on the screen. The projection lens 27 can adjust the focusing and can adjust the focal length. The projection lens 27 is driven by the lens driving unit 28. The lens driving unit 28 performs zooming and focusing by driving the constituent lenses of the projection lens 27.

  The control unit 20 is an arithmetic processing device. The control unit 20 controls the video signal conversion unit 22, the video signal processing unit 23, the display controller 24, and the lens driving unit 28 in an integrated manner according to a program.

  The image processing apparatus 4 includes a computer 40, a storage device 41, an input device 42, an electronic camera 43, a trigger sensor 44, and the like.

  The input device 42 is a plurality of switches (for example, a keyboard), a pointing device, or a combination thereof. The input device 42 is operated by the user to output a signal corresponding to the operation content to the computer 40.

  The electronic camera (imaging means) 43 is provided in the same direction as the projection lens 27. The electronic camera 43 includes an imaging device, an imaging lens, an image processing circuit, and the like. The imaging lens forms an image on the imaging device, and an image obtained by the imaging device connected by the imaging lens is converted into an electrical signal by photoelectric conversion. The image signal picked up by the above is output to the image processing circuit, the image signal is digitized by the signal processing by the image processing circuit, and the digitized image signal is transferred to the computer 40.

  The imaging range 96 of the electronic camera 43 includes a projection screen 90 that is the maximum projection range of the projection apparatus body 2 (see FIG. 1). Accordingly, the projection object 99 in front of the projection mapping apparatus 1 is also imaged by the electronic camera 43, and the image of the projection object 99 is included in the image transferred from the electronic camera 43 to the computer 40.

  The trigger sensor 44 is provided on the projection object 99 or is provided around the projection object 99. The trigger sensor 44 detects the start or end of change of the projection object 99 or both, and outputs a detection signal to the computer 40. The change of the projection object 99 refers to movement, rotation, direction change, displacement, movement, expansion, contraction, shape change, and the like of the projection object 99.

  The trigger sensor 44 includes an acceleration sensor (acceleration switch), a limit sensor (limit switch), a photoelectric switch (photoelectric sensor), a proximity switch (proximity sensor), a level switch (level sensor), a pressure switch (pressure sensor), and a temperature switch ( Temperature sensor) Other sensors. The type of trigger sensor 44 is suitable for the projection object 99. The trigger sensor 44 may transmit a detection signal to the computer 40 wirelessly, or may output the detection signal to the computer 40 via a lead wire.

  The computer 40 has a CPU, memory, GPU, VRAM, ASIC, bus, input / output interface, disk reader, and the like. The storage device 41 is accessible to the computer 40. The storage device 41 is a hard disk drive, nonvolatile memory, ROM, or other storage device.

The storage device 41 stores a program 41a. The program 41a causes the computer 40 to realize functions, and the computer 40 performs processing according to the program 41a. The program 41a includes a data map application and a drawing application.
The storage device 41 stores a plurality of objects 41b. These objects 41b are an image (texture), an animation image (GIF animation), a moving image, a text (character string), and the like. Each object 41b has a different shape, size, color, pattern, text font, and the like.

  Next, the operation and usage method of the projection mapping apparatus 1 will be described with reference to FIG. FIG. 3 is a flowchart for explaining the function and processing flow of the computer 40 based on the program 41a.

  First, the user activates the projection mapping apparatus 1. The zoom of the lens of the electronic camera 43 and the projection lens 27 is performed so that the user points the electronic camera 43 and the projection lens 27 toward the screen and the projection screen 90 on the screen is included in the shooting range 96 of the electronic camera 43. Adjust. It is preferable to arrange the lens of the electronic camera 43 and the projection lens 27 in parallel.

  The computer 40 performs the drawing process of the alignment mark 61 as shown in FIG. 4 (step S1). FIG. 4 schematically shows a screen (frame) 60 and an alignment mark 61 stored in the VRAM, memory or the like of the computer 40.

  Specifically, as shown in FIG. 4, the computer 40 displays the XY coordinate system of the display screen 60 (the horizontal position on the display screen 60 is represented by an X coordinate value, and the vertical position is represented by a Y coordinate. And the positions of the plurality of feature points 62 to 65 of the alignment mark 61 to be displayed on the display screen 60 are set as XY coordinate values. Then, the computer 40 generates (draws) the alignment mark 61 and the feature points 62 to 65, and outputs a video signal of the display screen 60 including the alignment mark 61 and the feature points 62 to 65 to the projection apparatus main body 2. The alignment mark 61 is an image composed of line segments connecting the feature points 62 to 65. Since the positions of the feature points 62 to 65 on the display screen 60 are set by the computer 40 according to the program 41a, the positions of the feature points 62 to 65 are known to the computer 40. Hereinafter, the XY coordinate system on the display screen 60 is referred to as a screen coordinate system.

  The video signal output by the computer 40 is processed by the video signal conversion unit 22 and the video signal processing unit 23 and transferred to the display controller 24. The display controller 24 controls the display element 25 and the light irradiation device 26 according to the video signal. . Then, the light irradiation device 26 irradiates the display element 25 with light, the display screen 60 including the alignment mark 61 and the feature points 62 to 65 is displayed on the display element 25, and the display screen 60 is projected onto the screen by the projection lens 27. Is done. Then, as shown in FIG. 5, the display screen 60 becomes apparent as the projection screen 90 on the screen, and the alignment marks 91 and the feature points 92 to 95 are displayed on the projection screen 90. The reason why the alignment mark 61 shown in FIG. 4 is deformed like the alignment mark 91 shown in FIG. 5 is that trapezoidal distortion caused by the positional relationship between the projection apparatus main body 2 and the screen, and the video signal conversion unit 22 of the projection apparatus main body 2. And video signal processing performed by the video signal processing unit 23.

  Next, the computer 40 causes the electronic camera 43 to perform an imaging operation by outputting a release signal to the electronic camera 43 (step S2). Thereby, the projection screen 90 on which the alignment mark 91 and the feature points 92 to 95 are displayed is captured by the electronic camera 43, and the image captured by the electronic camera 43 is input to the computer 40.

  Next, the computer 40 detects the positions of the feature points 92 to 95 on the captured image (step S3). Specifically, the computer 40 performs feature extraction processing (for example, edge extraction processing) on the captured image input from the electronic camera 43 to extract the alignment mark 91 and the feature points 92 to 95 in the captured image. The positions of the feature points 92 to 95 are detected as XY coordinate values. And the computer 40 memorize | stores the position of the detected feature points 92-95. Hereinafter, the XY coordinate system in the image captured by the electronic camera 43 is referred to as a captured image coordinate system.

  Thereafter, the computer 40 stops processing and enters a standby state, and the user installs the projection object 99 between the projection mapping apparatus 1 and the screen. Alternatively, the user points the electronic camera 43 and the projection lens 27 toward the projection object 99 in a state where the relative positional relationship between the electronic camera 43 and the projection lens 27 is fixed.

  Next, the user operates the input device 42 to restart the processing of the computer 40. Then, the computer 40 outputs a release signal to the electronic camera 43, thereby causing the electronic camera 43 to perform an imaging operation (step S4). As a result, an image including an image of the projection object 99 is captured by the electronic camera 43, and the image captured by the electronic camera 43 is input to the computer 40. FIG. 6 shows an example of an image 70 captured by the electronic camera 43. The captured image 70 includes an image 71 of the projection object 99.

  Next, the computer 40 uses the coordinate values of the feature points 62 to 65 and the coordinate values of the feature points 92 to 95 as coordinate conversion parameters to convert the captured image 70 acquired in step S4 from the captured image coordinate system to the screen coordinate system. The coordinates are converted (step S5). For example, the computer 40 sets the coordinate conversion parameters for converting the coordinate values of the feature points 92 to 95 to the coordinate values of the feature points 62 to 65, respectively, as the coordinate values of the feature points 62 to 65 and the coordinate values of the feature points 92 to 95. The coordinate conversion of the captured image 70 is performed using the coordinate conversion parameter. At this time, the computer 40 performs coordinate conversion while interpolating between pixels of the captured image 70 before coordinate conversion (or thinning out the captured image before coordinate conversion). The coordinate transformation is projective transformation, affine transformation, perspective projection transformation, parallel movement or rotational movement, or a combination of two or more of these. Note that the computer 40 may divide the captured image 70 before conversion into a plurality of regions and perform coordinate conversion for each region, or a predetermined region (region surrounded by the feature points 92 to 95) of the captured image 70 before conversion. ) May be divided into a plurality of regions and coordinate-converted for each region, and coordinate conversion may be performed so as to extrapolate a region outside the predetermined region. FIG. 7 shows an example of an image 72 obtained by performing coordinate conversion on the captured image 70. The image 71 of the projection object 99 is also coordinate-converted into an image 73, and the shape, position, size, and the like of the image 71 of the projection object 99 change like the image 73.

  Next, as shown in FIG. 8, the computer 40 extracts a closed region 73a from the image 73 of the projection object 99 after coordinate conversion, and defines it as a drawing region (step S6). Specifically, the computer 40 extracts the contour shape of the closed region 73a by performing feature extraction such as edge extraction after performing binarization processing or monochromatic processing on the image 72 after coordinate conversion. Then, the closed region 73a inside the contour shape is defined as a drawing region. As shown in FIG. 9, when a plurality of closed areas 73a, 73b, and 73c are extracted, the computer 40 performs a labeling process to identify these closed areas 73a, 73b, and 73c as separate areas. One closed region (for example, the largest closed region 73a among the closed regions 73a, 73b, 73c) is selected and defined as a drawing region, or these closed regions 73a, 73b, 73c are not subjected to labeling processing. Is defined as one drawing area.

  Next, the computer 40 selects and reads one object 41b suitable for the drawing area from the plurality of objects 41b stored in the storage device 41 (step S7). For example, the computer 40 selects the object 41b closest to the size and / or shape of the drawing area, or searches for one object 41b by a predetermined search algorithm using the drawing area as a search condition.

  Next, the computer 40 maps the selected object 41b to the drawing area (step S8).

  Next, the computer 40 reproduces the object 41b mapped to the drawing area and performs a drawing process (video signal generation process) for drawing a screen including the object 41b (step S9). That is, the computer 40 writes a screen including the object 41b mapped in the drawing area into a VRAM or the like, generates a video signal of the screen, and outputs the video signal to the projection apparatus main body 2. The computer 40 preferably generates a video signal in which the background area other than the drawing area is black.

  The video signal output by the computer 40 is processed by the video signal conversion unit 22 and the video signal processing unit 23 and transferred to the display controller 24. The display controller 24 controls the display element 25 and the light irradiation device 26 according to the video signal. . Then, the light irradiation device 26 irradiates the display element 25 with light, a screen including the selected object 41 b is displayed on the display element 25, and the screen is projected by the projection lens 27. Therefore, as shown by a two-dot chain line in FIG. 1, the object 41b is displayed as a texture 97 in the projection area 98 (corresponding to the drawing area) of the projection object 99.

  Thereafter, the computer 40 determines whether or not an event has occurred (step S10), and continues the drawing process if no event has been detected (step S10: NO, step S9). On the other hand, when the computer 40 detects the occurrence of an event (step S10: YES), the computer 40 performs the above-described steps S4 to S10 again. Then, in step S7, the image including the newly selected object 41b is drawn (step S9). As shown by the solid line in FIG. It is displayed in area 98. That is, the display in the projection area 98 of the projection object 99 is updated by performing the above-described steps S4 to S10 again. When the computer 40 stops the drawing process at the time of the imaging process of step S4 performed again, the object 41b is not displayed on the projection area 98 of the projection object 99, and the subsequent closed area extraction process (step S6). Is done accurately.

  When an event occurs intermittently or continuously, or when an event occurs repeatedly a plurality of times, the computer 40 repeats the processing from step S4 to step S10 a plurality of times. Therefore, the display in the projection area 98 of the projection object 99 is updated intermittently, continuously, or repeatedly.

  An example of event detection is given below.

Example 1: Use of trigger sensor 44 The occurrence of an event is detected using the trigger sensor 44. The event is that the projection object 99 undergoes movement, rotation, direction change, displacement, movement, expansion, contraction, shape change, or other changes. The start and / or end of change of the projection 99 is detected by the trigger sensor 44, the detection signal output by the trigger sensor 44 is input to the computer 40, and the computer 40 detects the occurrence of an event (step S10). : YES)

Example 2: Use of the electronic camera 43 The electronic camera 43 is used to detect the occurrence of an event.
As an event, the projection object 99 may be moved, rotated, changed direction, displaced, operated, expanded, contracted, changed in shape, or other changes. A plurality of frames (images) are transferred from the electronic camera 43 to the computer 40, and the computer 40 determines whether or not the projection object 99 has changed based on these frames. For example, the computer 40 recognizes the image of the projection object 99 in a certain frame, recognizes the image of the projection object 99 in the next frame, calculates the similarity of the images of the projection object 99, and When the similarity is equal to or less than a predetermined threshold, the occurrence of an event is detected as a change in the projection object 99 has occurred (step S10: YES).
As an event, there is an event within the photographing range of the electronic camera 43 other than the change of the projection object 99. For example, the shooting range of the electronic camera 43 becomes dark by blocking the front of the electronic camera 43 with an obstacle, or the texture 97 is not displayed on the projection object 99 by blocking the front of the projection lens 27 with an obstacle, A thing other than the projection object 99 (for example, a user) enters the photographing range of the electronic camera 43, or a thing other than the projection object 99 (for example, the user) falls outside the photographing range of the electronic camera 43. . In this case, the computer 40 detects whether an event has occurred based on a plurality of frames input from the electronic camera 43. For example, the computer 40 calculates the similarity between a certain frame and the next frame, and detects the occurrence of an event that the projection object 99 has changed when the similarity is equal to or less than a predetermined threshold (step S10). : YES)

Example 3: Use of Input Device 42 The occurrence of an event is detected using the input device 42. An event is when a user operates the input device 42 to input. When the user operates the input device 42, an operation signal is input to the computer 40, and the computer 40 detects the occurrence of an event (step S10: YES).

  Subsequently, some specific examples of changes in the projection object 99 and the texture 97 will be given. As shown in FIG. 10, when the projection object 99 extends in the vertical direction, the projection area 98 expands in the vertical direction, and the texture 97 projected on the projection area 98 also changes. Specifically, the texture 97 changes so that the number of marks in the texture 97 increases. Note that the texture 97 changes even when the position, size, shape, or the like of the projection region 98 in the captured image changes due to movement or rotation of the projection object 99.

  As shown in FIG. 11, the projection object 99 is a sliding window arranged in front of the screen. By opening this sliding window, the open area which is the projection area 98 is expanded, and the texture 97 is changed so as to expand in the pulling direction of the sliding window. In this way, the continuous change of the projection object is recognized, and the texture 97 is changed accordingly.

  As described above, the computer 40 repeatedly performs the processing of step S4 to step S10, and the projection object 99 is imaged in step S4. Therefore, the texture 97 displayed in the projection area 98 of the projection object 99 is displayed. It changes following the shape change and movement of the projection object 99. Therefore, even if the projection object 99 changes shape, moves, or the like, it is not necessary to change the direction of the projection lens 27 of the projection apparatus body 2, the projection magnification, or the like, and it is not necessary to change the video content. By using the projection mapping apparatus 1, it is possible to improve the indoor article decoration effect by light, or to easily try a plan for changing the indoor / outdoor design.

[Modification 1]
In the above description, as shown in FIG. 9, when a plurality of closed regions 73a, 73b, 73c are extracted in step S6, any one of the closed regions 73a, 73b, 73c is selected as the drawing region. Alternatively, these closed areas 73a, 73b, and 73c are defined as one drawing area. On the other hand, these closed areas 73a, 73b, and 73c may be defined as separate drawing areas.

  Specifically, when a plurality of closed regions 73a, 73b, and 73c are extracted, the computer 40 assigns individual identification numbers to these closed regions 73a, 73b, and 73c by performing a labeling process. Thereby, the computer 40 identifies these closed areas 73a, 73b, and 73c as separate areas, and defines these closed areas 73a, 73b, and 73c as separate drawing areas. Further, the computer 40 counts the number of closed areas 73a, 73b, 73c (here, 3). Then, the computer 40 obtains the area, the perimeter, the maximum width (the length in the Y direction), and the maximum height (the length in the X direction) for each of the closed regions 73a, 73b, and 73c. Then, the computer 40 selects a plurality of objects 41b suitable for each of the plurality of closed regions 73a, 73b, 73c for each of the closed regions 73a, 73b, 73c from among the plurality of objects 41b stored in the storage device 41. (Step S7). The subsequent processing is as described above. In the drawing process (step S9), the computer 40 may draw the objects 41b mapped to the closed regions 73a, 73b, and 73c at the same time or may draw them sequentially.

An example of changes in the projection object 99 and the texture 97 when a plurality of closed areas are defined as separate drawing areas will be described. As shown in FIG. 12, the projection object 99 is a cone, and the projection object 99 rotates in the pitch direction as a horizontal axis orthogonal to the axis of the cone. Then, the texture 97a projected on the conical surface 98a of the projection object 99 changes, and the texture 97b projected on the bottom surface 98b changes.
For example, when a plurality of projection objects having different shapes (for example, people, human shadows, vehicles, boxes, etc.) enter the projection range of the projection lens 27 one after another, the objects according to the characteristics of the shapes of these projection objects Can be selected and projected.
Changes in the number and size of the projected closed areas due to changes in one object (for example, balloons, rotating objects, ropes, animals, animal shadows, etc.) Objects can be selected and projected according to changes in objects.
It is also possible to distinguish the display contents for one area from the display contents for two areas (see FIG. 12). Changes when dividing from one area into multiple areas, aggregation from multiple areas into one area It is possible to perform expressive drawing with changes of time.
Further, when a large number of projection objects are gathered or diffused, a more expressive drawing can be performed by creating an object according to the arrangement of the projection objects.

[Modification 2]
The computer 40 performs the imaging / image input process in step S4 again without determining whether or not an event has occurred after the drawing process (step S9). That is, the computer 40 repeatedly performs the processing from step S4 to step S9. For example, steps S4 to S9 are performed during one frame period of the video signal output by the computer 40, and the computer 40 performs the processes of steps S4 to S9 for each frame. Therefore, when the projection object 99 changes such as movement, rotation, direction change, displacement, movement, expansion, contraction, and shape change, the texture 97 projected on the projection area 98 of the projection object 99 becomes the projection area 98. Changes following changes in position, size, range, etc.

[Others]
Changes and improvements can be made from the specific examples described above without departing from the scope of the present invention. Moreover, although embodiment and the modification of this invention were demonstrated above, the technical scope of this invention is not limited to the above-mentioned embodiment or modification, and is defined based on description in a claim. . Furthermore, the technical scope of the present invention also includes an equivalent range obtained by changing modifications not related to the essence of the present invention from the description in the claims.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
Imaging means;
Image acquisition means for acquiring an image including the projection imaged by the imaging means;
Area extraction means for extracting a projection area of the projection object from the image acquired from the image acquisition means;
Mapping means for mapping an object corresponding to the projection area to the projection area;
An image processing apparatus comprising:
<Claim 2>
The area extracting means extracts a change in the projection area of the projection object,
The mapping means maps an object corresponding to a change in the projection area extracted by the area extraction means;
The image processing apparatus according to claim 1.
<Claim 3>
The area extracting means extracts a plurality of projection areas of the projection object,
The mapping means maps an object corresponding to each of a plurality of projection areas extracted by the area extraction means;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus.
<Claim 4>
A trigger sensor for detecting a change in the projection object;
The region extracting means extracts a change in the projection region when the trigger sensor detects a change,
The image processing apparatus according to claim 2, wherein the image processing apparatus is an image processing apparatus.
<Claim 5>
The image processing apparatus according to claim 2, wherein a change in the projection area is extracted based on a plurality of frames transferred from the imaging unit.
<Claim 6>
Video signal generating means for generating a video signal of a screen including the mapped object;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus.
<Claim 7>
An image processing apparatus according to claim 6;
A projection apparatus main body that projects a screen based on the video signal generated by the video signal generation means toward the projection object;
A projection system comprising:
<Claim 8>
A computer of an image processing apparatus comprising an imaging means;
Image acquisition means for acquiring an image including the projection object imaged by the imaging means;
Area extraction means for extracting a projection area of the projection object from the image acquired from the image acquisition means;
as well as,
Mapping means for mapping an object corresponding to the projection area to the projection area;
Program to function as.
<Claim 9>
A computer of an image processing apparatus provided with an imaging means,
Image acquisition processing for acquiring an image including a projection object imaged by the imaging means;
An area extraction process for extracting a projection area of the projection object from the image acquired by the image acquisition process;
And a mapping process for mapping an object corresponding to the projection area to the projection area.

DESCRIPTION OF SYMBOLS 1 Projection mapping apparatus 2 Projection apparatus main body 4 Image processing apparatus 40 Computer 41a Program 41b Object 42 Input apparatus 43 Electronic camera 44 Trigger sensor

Claims (9)

Imaging means;
Image acquisition means for acquiring an image including the projection imaged by the imaging means;
Area extraction means for extracting a projection area of the projection object from the image acquired from the image acquisition means;
Mapping means for mapping an object corresponding to the projection area to the projection area;
An image processing apparatus comprising: The area extracting means extracts a change in the projection area of the projection object,
The mapping means maps an object corresponding to a change in the projection area extracted by the area extraction means;
The image processing apparatus according to claim 1. The area extracting means extracts a plurality of projection areas of the projection object,
The mapping means maps an object corresponding to each of a plurality of projection areas extracted by the area extraction means;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. A trigger sensor for detecting a change in the projection object;
The region extracting means extracts a change in the projection region when the trigger sensor detects a change,
The image processing apparatus according to claim 2, wherein the image processing apparatus is an image processing apparatus.   The image processing apparatus according to claim 2, wherein a change in the projection area is extracted based on a plurality of frames transferred from the imaging unit. Video signal generating means for generating a video signal of a screen including the mapped object;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. An image processing apparatus according to claim 6;
A projection apparatus main body that projects a screen based on the video signal generated by the video signal generation means toward the projection object;
A projection system comprising: A computer of an image processing apparatus comprising an imaging means;
Image acquisition means for acquiring an image including the projection object imaged by the imaging means;
Area extraction means for extracting a projection area of the projection object from the image acquired from the image acquisition means;
as well as,
Mapping means for mapping an object corresponding to the projection area to the projection area;
Program to function as. A computer of an image processing apparatus provided with an imaging means,
Image acquisition processing for acquiring an image including a projection object imaged by the imaging means;
An area extraction process for extracting a projection area of the projection object from the image acquired by the image acquisition process;
And a mapping process for mapping an object corresponding to the projection area to the projection area.

Images