JP2011182026A - Image processing apparatus, image display system, and image extracting device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus for extracting an object to be extracted by a reliable and simple method, even in an environment, where the background cannot be identified easily and background information varies easily. <P>SOLUTION: The image processing apparatus 2 includes: an infrared LED 6; an infrared CCD camera 7; a visible CCD camera 8; and an image processing apparatus for processing an image based on an infrared image and a visible image. The image processing apparatus includes: a difference image creation unit for creating a difference image between an infrared image in infrared light radiation and an infrared image in infrared light non-radiation; a first mask pattern creation unit for creating a first mask pattern by performing threshold processing to the differential image; a motion image detection unit for detecting a motion image by comparing the infrared image in infrared light radiation with the infrared image in infrared light non-radiation; a second mask pattern creation unit for creating a second mask pattern created by deleting a motion image region detected by the motion image detection unit from the first mask pattern; and a composite image creation unit for synthesis of the visible image and the second mask pattern. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing device, an image display system, and an image extraction device.

  In recent years, real-time video conferencing systems via networks are becoming widespread due to the enhancement of communication environments using networks. In such a video conferencing system, only the target person is cut out for reasons such as reducing the amount of transmitted data, etc., in order to maximize the sense of presence, so as not to show the private environment to the other party. A method of transmitting the video is considered. Various techniques have been proposed to achieve this.

  For example, a technique using a chroma key or background difference is conventionally known. However, in the technique using the chroma key, it is necessary to prepare a dedicated background (wall) having a uniform color such as blue, which is very inconvenient. Further, the method using the background difference has a problem that, for example, when a person other than the target person moves in the background in an environment such as an office, the movement of the person is detected.

Therefore, a technique using infrared light has been proposed as a technique for solving the above problem.
The example shown in Patent Document 1 includes an infrared light source, first and second cameras arranged orthogonally via a half mirror, an infrared transmission filter arranged on the incident side of the first camera, and a second And an infrared cut filter disposed on the incident side of the camera. In this apparatus, a first image serving as a mask image for extracting a subject is acquired by infrared light, while a second image including a background and a subject is acquired by visible light, and based on these images. An image in which only the subject is extracted from the background can be obtained.

JP-A-11-73491

  However, there is a problem with the above-described method using infrared light. In the apparatus of Patent Document 1, if the background includes strong infrared light having the same wavelength as the light source, there is a problem that the subject and the background cannot be reliably separated, and further, natural light including infrared light is strongly reflected. When the object to be moved is moving in the background, the object is easily recognized, and the reliable separation of the subject and the background becomes even more difficult.

  The present invention has been made to solve the above-described problems, and it is possible to extract an extraction target by a reliable and simple method even in an environment where the background is difficult to specify and the background information easily fluctuates. An object of the present invention is to provide an image processing apparatus, an image display system, and an image extraction apparatus that can perform image processing.

  In order to achieve the above object, an image processing apparatus according to the present invention includes an infrared light source that irradiates infrared light toward an extraction target, an infrared imaging unit that captures an image using infrared light, and visible light. A visual imaging unit that captures an image; and an image processing unit that performs image processing based on the infrared image acquired by the infrared imaging unit and the visible image acquired by the visible imaging unit. However, the infrared image acquired during the period when the infrared light is irradiated from the infrared light source toward the extraction target, and the red acquired during the period when the infrared light is not irradiated A difference image creation unit that creates a difference image by taking a difference from an infrared image when no external light is irradiated, and a first mask pattern that creates a first mask pattern by performing threshold processing on the difference image Creation unit and infrared image during infrared light irradiation And a motion image detection unit that detects a motion image region having a motion by comparison with the infrared light non-irradiation infrared image, and a second mask pattern in which the motion image region is erased from the first mask pattern A second mask pattern creation unit for creating a composite image, and a composite image creation unit for creating a composite image including an extraction target image by combining the visible image and the second mask pattern. And

  The image processing apparatus of the present invention includes an infrared light source and an infrared imaging means, and the infrared imaging means does not irradiate infrared light during a period in which infrared light is not emitted from the infrared light source toward the extraction target. While acquiring a time infrared image, an infrared image is acquired at the time of infrared light irradiation in the period when infrared light is irradiated. At this time, the reflected infrared light is strong because the infrared reflected light from the extraction target located at a position relatively close to the infrared light source is strong, and the infrared reflected light from the object located at a remote position is weak. The intensity enables the analogy of the distance from the infrared light source to the object, and as a result, the extraction target and the background can be separated. In addition, comparing the infrared image when irradiated with infrared light and the infrared image when not irradiated with infrared light, the part of the extraction target with a high reflection intensity is significantly different, and the background part with a low reflection intensity is almost unchanged. Become. Furthermore, even if a strong infrared light portion is included in the background, the portion does not change between the infrared image when irradiated with infrared light and the infrared image when not irradiated with infrared light.

  And the difference image creation part of an image processing means takes the difference of the infrared image at the time of infrared light irradiation, and the infrared image at the time of non-irradiation of infrared light, and produces a difference image. Then, the extraction target part differs greatly between the infrared image when irradiated with infrared light and the infrared image when not irradiated with infrared light, and the background part is almost the same. Is left as an infrared image, and a difference image in a state where the background portion is erased is obtained.

  The first mask pattern creation unit creates a first mask pattern by performing threshold processing on the difference image created by the difference image creation unit. Here, for example, paying attention to the luminance value of each pixel constituting the difference image, a pixel whose luminance value is less than a predetermined threshold (corresponding to a pixel constituting the background portion) is displayed in black, and the luminance value is equal to or higher than the predetermined threshold. Is binarized such that a pixel (corresponding to a pixel constituting the extraction target portion) is displayed in white, and a first mask pattern including black display pixels is created.

  Then, the motion image detection unit detects an image that has moved in the comparison between the infrared image without infrared light irradiation and the infrared image with infrared light irradiation, and creates a motion image. Then, only the part which moved smoothly between the infrared image at the time of infrared light non-irradiation and the infrared image at the time of infrared light irradiation is detected. Sites that disappear suddenly only during irradiation are not detected. That is, in the background, an image in which only an object that moves smoothly is detected while reflecting infrared light included in natural light can be obtained.

  Then, the second mask pattern creation unit erases only the motion image detected by the motion image detection unit from the first mask pattern, thereby obtaining the second mask pattern.

  Then, the synthesized image creation unit synthesizes the visible image acquired by the visible imaging unit and the second mask pattern, thereby including the extraction target image composed of the visible image and the background portion being erased. Create an image.

  In the conventional technique, for example, when the background includes strong infrared light having the same wavelength as the light source, there is a problem that the extraction target and the background cannot be reliably separated. On the other hand, in the image processing apparatus of the present invention, a method of creating a difference image from an infrared image when irradiated with infrared light and an infrared image when not irradiating with infrared light, and creating a mask pattern from the difference image. Therefore, by appropriately setting the intensity of infrared light and the threshold value for threshold processing, an extraction target with a sufficiently strong infrared reflected light intensity from the newly provided infrared light source and a newly provided infrared light source It is possible to reliably separate the background from the sufficiently weak infrared reflected light intensity. In addition, even when an object that strongly reflects infrared light contained in natural light is moving in the background, it is possible to distinguish the moving object by motion image detection, so that infrared light contained in natural light is strongly reflected. It is possible to clearly distinguish a moving object from a moving object that strongly reflects only infrared light from a newly provided infrared light source. This makes it possible to reliably extract only the extraction target. As described above, according to the image processing apparatus of the present invention, even in an environment in which the background is difficult to specify and the background information easily fluctuates, the extraction target is extracted by a reliable and simple method, and the background portion is erased. A composite image can be created.

In the image processing apparatus of the present invention, the image acquisition cycle of the infrared imaging unit is 1 / n times (n: an integer equal to or greater than 2) the image acquisition cycle of the visible imaging unit, and is one time of the visible imaging unit. It is desirable that the infrared light source is turned on or off n times during the image acquisition cycle.
For example, if the image acquisition cycle of the infrared imaging means is ½ of the image acquisition cycle of the visible imaging means, and the infrared light source is turned on or off twice in one image acquisition cycle of the visible imaging means. Even if the background includes strong infrared light having the same wavelength as the light source, only the extraction target can be reliably extracted from the visible image acquired within any one image acquisition cycle.
In the present invention, the absolute value (time) of the image acquisition cycle of the infrared imaging means is not particularly limited and may be long. However, if the absolute value (time) of the image acquisition period of the infrared imaging means is too long, for example, when the background object contains a large infrared light having the same wavelength as the light source, the background object moves greatly. There is a possibility that a changed part other than the target may be extracted. In that respect, it is desirable that the image acquisition cycle of the infrared imaging means is short.

In the image processing apparatus according to the aspect of the invention, it is preferable that the infrared light source is turned on and off when the infrared light source is turned off.
According to this configuration, the motion image area that appears at the stage of creating the first mask pattern and the motion image area detected by the motion image detection unit are accurately detected even in the first frame when the visible image is acquired. It can be extracted and matched exactly, and the motion image can be erased accurately when the second mask pattern is created.

In the image processing apparatus of the present invention, the infrared image capturing unit and the visible image capturing unit may be configured by an infrared image capturing element and a visible image capturing element that are separated by a half mirror and arranged coaxially.
According to this configuration, it is only necessary to separately prepare an image sensor dedicated to an infrared image and an image sensor dedicated to a visible image. Therefore, all pixels of each image sensor are either an infrared image or a visible image. Can be used exclusively. Thereby, the resolution of the image obtained by each image sensor can be increased.

An image display system according to the present invention includes the image processing device according to the present invention and a display device that displays an extracted image obtained by the image processing device.
According to the image display system of the present invention, by providing the image processing apparatus of the present invention, a composite image in which the background portion is erased by a reliable and simple method can be obtained even in a situation where the background easily fluctuates. An image display system suitable for use in a video conference system or the like can be constructed.

An image extraction apparatus according to the present invention includes an infrared light source that irradiates infrared light toward an extraction target, an infrared imaging unit that captures an image using infrared light, and an infrared image acquired by the infrared imaging unit. Image processing means for performing image processing based on the infrared light irradiation infrared acquired during the period in which the infrared light is emitted from the infrared light source toward the extraction target. The difference mask image is obtained by further performing threshold processing on the difference image created by taking the difference between the image and the infrared light non-irradiation infrared image acquired during the period when the infrared light is not irradiated. A difference mask image creation unit that creates a motion image detection unit that detects a moving motion image region by comparing the infrared image when irradiated with infrared light and the infrared image when not irradiated with infrared light; Deleting the motion image area from the difference mask image; And extracting an image creation unit that creates an extracted image that, characterized in that it comprises a.
According to the image extraction device of the present invention, as in the image processing device of the present invention described above, only the extraction target can be extracted by a reliable and simple method even in a situation where the background easily changes.

1 is a schematic configuration diagram of an image display system according to a first embodiment of the present invention. It is a block diagram which shows the structure of the control apparatus of this image display system. It is a flowchart which shows the procedure of the image process of this image display system. It is a timing chart which shows operation of each image sensor and an infrared light source. It is a figure which shows the example of the image in each process of image processing. It is a figure which shows the operation | movement of a motion image detection.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described with reference to FIGS.
The image display system of the present embodiment is a configuration example of a system suitable for use in, for example, a video conference system used when a conference is performed by displaying an image of the other party on a screen.

  As shown in FIG. 1, the image display system 1 of the present embodiment displays an image processing apparatus 2 that captures images of a user A (extraction target) and a background B and performs image processing, and displays an image of the other party. Projector 3 (display device) and screen 4. In this embodiment, the other party also has the same image display system 1, and the image of the user A is transmitted to the projector 3 of the other party's image display system 1 and displayed. The display device is not limited to the projector 3, and any display device such as a television or a display of a personal computer may be used. Also, various means that are generally used can be used as means for transmitting an image to the other party.

  The image processing apparatus 2 includes an infrared LED (Light Emitting Diode) 6 (infrared light source), an infrared CCD (Charge Coupled Device) camera 7 (infrared imaging means), and a visible CCD camera 8 (visible imaging means). And a control device 9. The infrared LED 6 is for irradiating the user A with infrared light. The infrared CCD camera 7 captures an image by infrared light, and the visible CCD camera 8 captures an image by visible light. The control device 9 controls the drive of the infrared LED 6 and includes an image processing unit described later to perform image processing. The infrared CCD camera 7 and the visible CCD camera 8 are arranged coaxially via the half mirror 5.

  As shown in FIG. 2, the control device 9 includes an infrared LED drive unit 11, a synchronization signal generation unit 12, and an image processing unit 13 (image processing means). The infrared LED drive unit 11 has a function of generating a drive signal for driving the infrared LED 6 and outputting the drive signal toward the infrared LED 6. The synchronization signal generation unit 12 has a function of generating a synchronization signal for synchronizing the operations of the infrared LED 6, the infrared CCD camera 7, and the visible CCD camera 8, and outputting them to them. The image processing unit 13 performs image processing based on the infrared image acquired by the infrared CCD camera 7 and the visible image acquired by the visible CCD camera 8.

  The image processing unit 13 includes image acquisition units 15 and 16 corresponding to the infrared CCD camera 7 and the visible CCD camera 8, a first mask pattern generation unit 17, a motion image detection unit 18, and a second mask pattern generation. A unit 19 and a composite image creation unit 22 are provided. The image acquisition units 15 and 16 have a function of reading image data obtained from the infrared CCD camera 7 and the visible CCD camera 8 and include a memory for storing the read image data. The first mask pattern creating unit 17 serving as a difference mask image creating unit is configured as a first mask pattern serving as a difference mask image based on a difference between an infrared image at the time of infrared light irradiation and an infrared image at the time of no infrared light irradiation. It has a function to create. The motion image detection unit 18 has a function of detecting a region (motion image region) that has moved in comparison between an infrared image without infrared light irradiation and an infrared image with infrared light irradiation. The second mask pattern creation unit 19 creates a second mask pattern as an extracted image obtained by erasing the motion image area created by the motion image detection unit 18 from the first mask pattern created by the first mask pattern creation unit 17 It has the function to do. The composite image creation unit 22 has a function of creating a composite image by synthesizing the second mask pattern created by the second mask pattern creation unit 19 and the visible image.

  Further, the first mask pattern creation unit 17 includes a difference image creation unit 20 and a threshold processing unit 21. The difference image creation unit 20 has a function of creating a difference image between an infrared image when irradiated with infrared light and an infrared image when not irradiated with infrared light. The threshold processing unit 21 has a function of performing threshold processing on the difference image created by the difference image creating unit 20.

Hereinafter, the operation of the control device 9 will be described along the flowchart of FIG.
Note that the image examples shown in FIGS. 5A to 5H are re-created in an easy-to-understand manner based on images actually acquired by the inventor using a system equivalent to the present embodiment.
In the present embodiment, the image data obtained by the visible CCD camera 8 is read by the image acquisition unit 16, and the image acquisition unit 16 acquires a visible image at a frequency of 60 Hz and an image acquisition period of 1/60 seconds, for example ( Step S1) in FIG.

  FIG. 5A is an example of a visible image obtained from the visible CCD camera 8. This visible image is actually expressed in full color, and it can be seen that the background B appears after the image of the user A to be extracted.

  On the other hand, the image data obtained by the infrared CCD camera 7 is read by the image acquisition unit 15, and the image acquisition unit 15 acquires an image with a frequency of 120 Hz and an image acquisition period of 1/120 seconds. That is, the image acquisition unit 15 on the infrared CCD camera 7 side acquires an image at a frequency twice that of the image acquisition unit 16 on the visible CCD camera 8 side and an image acquisition cycle of ½. Further, while the infrared CCD camera 7 acquires an infrared image, a drive signal is input to the infrared LED 6 from the infrared LED drive unit 11, and the infrared LED 6 is alternately turned on or off at a frequency of 120 Hz. repeat. Here, one cycle of the infrared LED refers to a time during which one lighting or one extinguishing is performed.

  At this time, the synchronization signal output from the synchronization signal generator 12 shown in FIG. 2 is received, and as shown in FIG. 4, visible image acquisition by the visible CCD camera 8, infrared image acquisition by the infrared CCD camera 7, red The light emitting operation of the outer LED 6 is executed synchronously at a predetermined timing. That is, since the infrared LED 6 is turned off or turned on alternately at a frequency of 120 Hz, the infrared LED 6 is turned off, and the infrared CCD is synchronized with a period in which the user A is not irradiated with infrared light. The image acquisition unit 15 of the camera 7 acquires an infrared image (step S2 in FIG. 3).

  FIG. 5B is an example of an infrared image obtained from the infrared CCD camera 7 when the infrared LED 6 is turned off. As can be seen from the figure, even when the infrared light is extinguished, a portion of strong infrared light is included in the background.

Subsequently, the infrared LED 6 is turned on and the image acquisition unit 15 of the infrared CCD camera 7 acquires an infrared image in synchronization with a period in which the user A is irradiated with infrared light (step in FIG. 3). S3).
In the following, for convenience of explanation, an infrared image during a period when infrared light is not irradiated is referred to as a “first infrared image (infrared image when no infrared light is irradiated)”, and infrared light is irradiated. The infrared image during the period is referred to as “second infrared image (infrared image during infrared light irradiation)”. In this way, the infrared LED 6 is turned off or turned on once in a period in which the image acquisition unit 16 of the visible CCD camera 8 acquires a visible image once (hereinafter also referred to as a frame period). 7 image acquisition unit 15 acquires the first infrared image and the second infrared image once. As will be described later, the first infrared image and the second infrared image obtained here are used to create a mask pattern in one frame period of the visible CCD camera 8.

  FIG. 5C is an example of a second infrared image obtained from the infrared CCD camera 7 in a state where infrared light is irradiated from the infrared LED 6. As can be seen from the figure, not only the reflection from the user A but also the background contains a portion of strong infrared light. It is difficult to extract things. In addition, the face surface and the clothes part represented by the strong reflection intensity in FIG. 5C show the weak reflection intensity in FIG. 5B, but the non-moving part of the background part is shown in the figure. There is no change from the state of 5 (b).

  Next, the difference image creation unit 20 takes the difference between the second infrared image and the first infrared image and creates a difference image (step S4 in FIG. 3).

  FIG. 5D is an example of a difference image obtained from two infrared images of the second infrared image shown in FIG. 5C and the first infrared image shown in FIG. 5B. is there. Mainly, the user A's face and the front part of the body are expressed with strong reflection intensity, the periphery thereof is expressed with intermediate reflection intensity, and the other parts are expressed with weak reflection intensity. In addition, in the background B, the non-moving portion is filled with one black color, and the background B reflected after the user A disappears including the portion where strong infrared light is seen.

  The infrared LED 6 and the infrared CCD camera 7 are used for extracting an extraction target. However, since there is light including infrared light such as sunlight in the natural world, the infrared LED 6 is irradiated with infrared light. It is impossible to extract the extraction target by itself. Therefore, if an infrared image is acquired by the infrared CCD camera 7 even during a period when the infrared light is not irradiated, and a difference from the infrared image during the period when the infrared light is irradiated is taken, FIG. As shown, it is possible to separate an extraction target having a strong reflected light intensity of infrared light from an image including a background having a portion having a high reflected light intensity of infrared light, excluding a moving background.

  Next, the threshold processing unit 21 creates a first mask pattern by performing threshold processing on the difference image created by the difference image creating unit 20 (step S5 in FIG. 3). Specifically, for example, paying attention to the luminance value of each pixel constituting the difference image, pixels whose luminance value is less than a predetermined threshold (corresponding to the pixels constituting the background portion) are displayed in black, and the luminance value is predetermined. Binarization is performed such that pixels that are equal to or greater than the threshold (corresponding to the pixels constituting the extraction target portion) are displayed in white, and a temporary mask pattern C composed of pixels that display black is created.

  FIG. 5E is an example of a temporary mask pattern C obtained by performing threshold processing on the difference image in FIG. In the case of the present embodiment, in addition to the background portion, the hair portion is also part of the temporary mask pattern C.

  Next, an area in which a portion such as hair is inferred is added to the temporary mask pattern C by a method such as extracting an area having a low luminance value in the visible image shown in FIG. The shape is corrected as appropriate by image processing such as expansion / contraction to obtain a first mask pattern D as shown in FIG.

  Next, the motion image detection unit 18 performs motion image detection that detects only a motion image region that has moved in the comparison between the first infrared image and the second infrared image (step S6 in FIG. 3). Motion image detection is one of the moving object analysis methods, which is a method of analyzing a motion from luminance information in an image and representing a motion of an object by a velocity vector. As a method for performing the motion image detection, there are methods such as a gradient method and a block matching method.

  The function of the motion image detection unit 18 will be described more specifically with reference to FIG. Assume that images as shown in FIGS. 6A to 6E are acquired between adjacent frames acquired by the infrared CCD camera 7. Since the infrared LED 6 blinks between the frames, the target region OBJ1 to be finally extracted repeatedly appears and disappears between the frames. In addition, OBJ2, which is a final non-extraction region that is strongly reflected in the infrared light contained in natural light and exists in the background region and moves, appears in every frame and slightly changes its position in every frame. It is changing.

  Here, the algorithm used for motion image detection assumes that the shading pattern in the image does not change during movement, and is an algorithm that analyzes where the non-changing shading pattern has moved in the next frame. is there. Therefore, the moving object OBJ2 that moves with its position slightly changed between frames can be detected by the motion detection as shown by the hatched lines in FIGS. 6 (a) to 6 (e). However, in FIG. 6A, OBJ2 is not detected as a motion image because of the initial frame.

  On the other hand, OBJ1, which is the final extraction target, repeatedly appears and disappears between frames, and the motion becomes discontinuous between frames. As a result, even if the position is moved, estimation by motion image detection becomes difficult, the estimation condition by motion detection is not satisfied, and the region is not detected as a motion region.

  In addition, OBJ3, which is a final non-extraction region existing in the background region and does not move, strongly reflects infrared light included in natural light, and is not detected as a region with movement.

  As a result, the motion image detection unit 18 finally detects only the area of OBJ2. The region extracted by the motion image detection can be more accurately designated by appropriately performing image processing such as expansion / contraction.

  Returning to FIG. 5, the motion area detected by the motion image detector 18 is erased from the first mask pattern D created by the first mask pattern creator 17 as shown in FIG. A second mask pattern E is obtained (step S7 in FIG. 3).

  Here, the difference between the second infrared image at the time of infrared light irradiation and the first infrared image at the time of non-irradiation of infrared light is as shown in FIG. The position of the motion image detected in () matches the position of OBJ2 appearing in the difference image FIG. 6 (b ′). As shown in FIG. 4, the emission timing of the infrared LED 6 is synchronized with the timing behind the image acquisition timing of the infrared CCD camera 7, that is, IR (t) ′, IR (t + 1) ′,. By taking, it is possible to accurately match the position of OBJ2 appearing in the difference image FIG. 6B 'with the position of the motion image OBJ2 detected between FIGS. 6A and 6B. Yes, that is, in the stage of creating the second mask pattern E, the motion image can be accurately erased even in the first frame of the visible image.

  Next, the synthesized image creating unit 22 synthesizes the visible image acquired by the visible CCD camera 8 and the second mask pattern E described above. As a result, a composite image including the extraction target image made of a visible image and having the background portion erased is created (step S8 in FIG. 3) and output from the image processing unit 13 (step S9 in FIG. 3).

  FIG. 5H is an example of an extracted image obtained by combining the visible image of FIG. 5A and the mask pattern E of FIG. A composite image was obtained in which the extraction target was expressed in full color and the background portion was filled with black.

  In the image display system 1 of the present embodiment, a difference image is created from the second infrared image at the time of infrared light irradiation and the first infrared image at the time of non-irradiation of infrared light, and a mask pattern is created from the difference image. The method of creating is used. Therefore, by appropriately setting the intensity of the infrared light to be irradiated and the threshold value for threshold processing, it is possible to reliably separate the extraction target having a sufficiently high infrared reflected light intensity and the background having a sufficiently weak infrared reflected light intensity. be able to.

  In addition, the motion image detection unit 18 can detect a region where motion has occurred between the first infrared image and the second infrared image. In this embodiment, the image acquisition cycle of the infrared CCD camera 7 is set to ½ of the image acquisition cycle of the visible CCD camera 8, and the infrared LED 6 is turned off or turned on once in one frame period of the visible CCD camera 8. The configuration is such that a motion image detection unit 18 for detecting the motion of the infrared CCD camera 7 in a two-frame period is provided. For this reason, even if the background object contains strong infrared light with the same wavelength as the light source, even if the background object gradually moves, the person can be detected as a moving area, and the detected person's By erasing only the region, it is possible to reliably extract only the extraction target from the visible image acquired within any one frame period.

  As described above, according to the image display system 1 of the present embodiment, even in an environment where the background is difficult to specify and the background information easily fluctuates, the extraction target is extracted by a reliable and simple method, and the background portion is An erased composite image can be created.

  In this embodiment, the infrared CCD camera 7 and the visible CCD camera 8 arranged on the same axis are used. If a CCD camera dedicated to an infrared image and a CCD camera dedicated to a visible image are prepared separately, For this reason, the pixels of all the imaging elements of the infrared CCD camera 7 and the visible CCD camera 8 can be used exclusively for either the infrared image or the visible image. Thereby, the resolution of the image obtained by the infrared CCD camera 7 and the visible CCD camera 8 can be increased.

  In the present embodiment, when the position of the user A (extraction target) changes in the depth direction, the irradiation amount of the infrared light from the infrared LED 6 is appropriately changed, or the first mask pattern is changed. What is necessary is just to take the method of changing the setting value of the threshold value at the time of creation suitably. Further, even when a person who is not scheduled to be extracted enters the background, it is possible to perform reliable mask processing by optimizing the setting of the irradiation amount of infrared light and the setting of the threshold value. According to the image display system 1 of the present embodiment, since the visible image is only subjected to the mask process, the image of the user A can be transmitted with a natural expression that is not false.

  The technical scope of the present invention is not limited to the above embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, in the first embodiment, the visible CCD camera 8 and the infrared CCD camera 7 are arranged on the same axis. However, the visible CCD camera 8 and the infrared CCD camera 7 are generated on the basis of images acquired by the infrared CCD camera 7 arranged on different optical axes. A mask pattern suitable for the visible CCD camera 8 may be generated by performing spatial conversion on the mask pattern.

Further, in the above embodiment, the image acquisition period of the infrared CCD camera 7 is set to ½ of the image acquisition period of the visible CCD camera 8, and the first and second red colors are within one frame period of the visible CCD camera 8. Although the outside image is acquired, the image acquisition cycle of the infrared CCD camera 7 is not limited to this. When there is little possibility that the background changes, the image acquisition period of the infrared CCD camera 7 is set to m times (m: an integer of 1 or more) of the image acquisition period of the visible CCD camera 8. It is good also as a structure which acquires a 1st, 2nd infrared image within 10 frame periods. In that case, the image processing time that can be spent to create the mask pattern may be long, so the load on the image processing unit can be reduced.
In addition, the specific configuration such as the arrangement and number of components is not limited to the above-described embodiment, and can be changed as appropriate.

In the above embodiment, the image display system of the present invention is exemplified. However, if the visible CCD camera 8, the image acquisition unit 16, and the composite image creation unit 22 are excluded from the image display system 1 of the above embodiment, a difference image is obtained. Thus, the image extraction apparatus of the present invention that outputs the extracted image as an extracted image can be realized by combining the motion image detection. In addition to the image display system exemplified above, the image extraction apparatus can be used for an image recognition system, an inspection / measurement system, and the like.

  DESCRIPTION OF SYMBOLS 1 ... Image display system, 2 ... Image processing apparatus, 3 ... Projector (display apparatus), 4 ... Screen, 5 ... Half mirror, 6 ... Infrared LED (infrared light source), 7 ... Infrared CCD camera (infrared imaging) Means), 8... Visible CCD camera (visible imaging means), 13... Image processing section (image processing means), 17... First mask pattern creation section, 18. 20 ... difference image creation unit, 21 ... threshold processing unit, 22 ... composite image creation unit.

Claims (6)

An infrared light source that emits infrared light toward the extraction target;
Infrared imaging means for imaging an image by infrared light;
Visible imaging means for imaging an image by visible light;
Image processing means for performing image processing based on the infrared image acquired by the infrared imaging means and the visible image acquired by the visible imaging means;
An infrared image irradiation infrared image acquired during a period in which the image processing means is irradiated with the infrared light from the infrared light source toward the extraction target, and a period in which the infrared light is not irradiated A difference image creation unit that creates a difference image by taking a difference between the infrared image and the infrared image that has not been irradiated, and a first mask pattern that creates a first mask pattern by performing threshold processing on the difference image. 1 mask pattern creation unit;
A motion image detection unit that detects a motion image region having motion by comparing the infrared image when irradiated with infrared light and the infrared image when not irradiated with infrared light;
A second mask pattern creating unit that creates a second mask pattern in which the moving image region is erased from the first mask pattern;
An image processing apparatus comprising: a synthesized image creating unit that creates a synthesized image including an extraction target image by synthesizing the visible image and the second mask pattern.   The image acquisition cycle of the infrared imaging means is 1 / n times (n: an integer equal to or greater than 2) the image acquisition cycle of the visible imaging means, and the red is included in one image acquisition cycle of the visible imaging means. The image processing apparatus according to claim 1, wherein the external light source is turned on or off n times.   The image processing apparatus according to claim 2, wherein the infrared light source is turned on and off when the infrared light source is turned off.   The said infrared image imaging means and said visible imaging means are comprised with the infrared imaging element and visible imaging element which were arrange | positioned coaxially by isolate | separating with a half mirror, The Claim 1 thru | or 3 characterized by the above-mentioned. Image processing device.   An image display system comprising: the image processing device according to claim 1; and a display device that displays an extracted image obtained by the image processing device. An infrared light source that emits infrared light toward the extraction target;
Infrared imaging means for imaging an image by infrared light;
Image processing means for performing image processing based on an infrared image acquired by the infrared imaging means;
In the infrared light irradiation infrared image acquired in the period when the image processing means is irradiated with the infrared light from the infrared light source toward the extraction target, and during the period when the infrared light is not irradiated A difference mask image creating unit that creates a difference mask image by further performing threshold processing on the difference image created by taking the difference between the acquired infrared light non-irradiation infrared image,
A motion image detection unit that detects a motion image region having motion by comparing the infrared image when irradiated with infrared light and the infrared image when not irradiated with infrared light;
An image extraction apparatus comprising: an extraction image creation unit that creates an extraction image formed by erasing the motion image region from the difference mask image.