WO2021229804A1 - System and method for image extraction - Google Patents

System and method for image extraction Download PDF

Info

Publication number
WO2021229804A1
WO2021229804A1 PCT/JP2020/019475 JP2020019475W WO2021229804A1 WO 2021229804 A1 WO2021229804 A1 WO 2021229804A1 JP 2020019475 W JP2020019475 W JP 2020019475W WO 2021229804 A1 WO2021229804 A1 WO 2021229804A1
Authority
WO
WIPO (PCT)
Prior art keywords
extraction
image
subject
light source
reflector
Prior art date
Application number
PCT/JP2020/019475
Other languages
French (fr)
Japanese (ja)
Inventor
慈郎 長尾
誉宗 巻口
優次 森西
建 井阪
Original Assignee
日本電信電話株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日本電信電話株式会社 filed Critical 日本電信電話株式会社
Priority to PCT/JP2020/019475 priority Critical patent/WO2021229804A1/en
Publication of WO2021229804A1 publication Critical patent/WO2021229804A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/272—Means for inserting a foreground image in a background image, i.e. inlay, outlay
    • H04N5/275—Generation of keying signals

Definitions

  • the present invention relates to an image extraction system and its method.
  • the background subtraction method is known as a method of extracting an accurate area of a person or an object from an image.
  • the difference between the input image and the background image is taken, and the changed area is extracted as the subject area by performing the threshold processing.
  • Non-Patent Document 1 discloses an image extraction system using machine learning to discriminate a subject area. Further, a technique using infrared rays for the purpose of making the separation between the background and the subject more accurate and robust has been proposed (for example, Non-Patent Document 2).
  • the present invention has been made in view of this problem, and an object of the present invention is to provide an image extraction system and a method thereof, which makes it easy to distinguish between a subject area and a background area.
  • the image extraction system includes a reflector provided with a Fresnel lens-shaped reflecting layer, a first light source for extracting light that irradiates light from diagonally above the subject arranged on the reflector, and the like.
  • An image information extraction unit that extracts an image representing the subject and the shadow of the subject from the first image pickup device arranged at a position facing the first extraction light source and the first video signal output by the first image pickup device. The gist is to prepare for.
  • the image extraction system includes a reflector provided with a reflective layer formed on a sawtooth-shaped slope and arranged vertically, and the reflective layer on the sawtooth-shaped slope.
  • An image representing the area of the subject from an extraction light source that irradiates light from behind the subject, the reflector, an image pickup device arranged at a corresponding position across the subject, and an image signal output by the image pickup device.
  • the gist is to have an image information extraction unit for extracting information.
  • the image extraction method is an image extraction method performed by the above-mentioned image extraction system, in which the image extraction system includes a reflector having a reflective layer in the shape of a Fresnel lens and the reflector.
  • the first extraction light source that irradiates light from diagonally above the subject arranged above, the first image pickup device arranged at a position facing the first extraction light source, and the first image pickup device output.
  • the first extraction light source includes an image information extraction unit that extracts image information representing the subject and a shadow region of the subject from the first video signal, and the first extraction light source irradiates the subject with light and the first image pickup apparatus. Takes an image based on the light reflected by the reflector, and the image information extraction unit extracts image information representing the subject and a shadow region of the subject from the first video signal output by the first image pickup apparatus.
  • the gist is to do.
  • FIG. 1 is a diagram schematically showing a functional configuration example of the image extraction system according to the first embodiment of the present invention.
  • the image extraction system 100 shown in FIG. 1 is a system that extracts image information of a subject and its shadow region.
  • the image extraction system 100 includes a reflector 10, a first light source for extraction 21, a first image pickup device 31, and an image information extraction unit 40.
  • the plane on which the reflector 10 is arranged is defined as the xy plane, and the upward direction is defined as the z direction.
  • the reflector 10 includes a Fresnel lens-shaped reflective layer.
  • the Fresnel lens shape is similar to a lens in which the lens is divided into concentric regions to reduce the thickness, and has a sawtooth-shaped cross section.
  • the reflector 10 is arranged on an xy plane such as a floor.
  • the first extraction light source 21 irradiates light from diagonally above (z direction) the subject O arranged on the reflector 10.
  • the subject O may be a person (for example, a player) or an object.
  • the ambient light is the light of an indoor lighting device in which the reflector 10 is arranged.
  • the wavelength of the light of the first extraction light source 21 is different from the wavelength of the ambient light.
  • the first image pickup device 31 is arranged at a position facing the first extraction light source 21.
  • FIG. 1 shows an example in which the first image pickup apparatus 31 faces the first extraction light source 21 in the x direction.
  • the first image pickup device 31 and the first extraction light source 21 may face each other in the y direction.
  • the first image pickup apparatus 31 outputs the first video signal mainly based on the light of the first extraction light source 21 reflected by the reflector 10.
  • the image information extraction unit 40 deletes the light region of the first extraction light source 21 from the first video signal, and extracts image information representing the subject O and the shadow region of the subject. That is, when the chroma key is applied using the light of the first extraction light source 21 as a key signal, only the subject O and its shadow region can be extracted.
  • Chroma key is a technology that makes a part of an image transparent from a specific color component and synthesizes another image there. Also called chroma key compositing.
  • the image information extraction unit 40 can be configured by a general device such as a personal computer and a video capture board.
  • the image extraction system 100 irradiates light from diagonally above the reflector 10 having a Fresnel lens-shaped reflective layer and the subject O arranged on the reflector 10.
  • Extraction light source 21 the first image pickup device 31 arranged at a position facing the first extraction light source 21, and the light of the first extraction light source 21 from the first video signal output by the first image pickup device 31.
  • An image information extraction unit 40 for deleting a region and extracting image information representing a subject O and a shadow region of the subject O is provided.
  • the light of the first extraction light source 21 does not wrap around to the first imaging device 31 side of the subject O, so that the subject O and its shadow region can be clearly distinguished from the other regions.
  • FIG. 2 is a diagram schematically showing a cross section of the reflector 10.
  • FIG. 2 shows a cross section of the reflector 10 cut in the x direction.
  • the reflecting plate 10 includes a holding plate 10a, a Fresnel lens layer 10b, and a reflecting layer 10c from the bottom.
  • the holding plate 10a is a substrate that maintains the flatness of the reflector 10, and is composed of, for example, a plastic plate or the like.
  • a Fresnel lens layer 10b is formed on the holding plate 10a.
  • the lower surface side of the Fresnel lens layer 10b is flat, the upper surface side forms the Fresnel lens surface, and the Fresnel lens layer 10b is made of a resin such as epoxy acrylic rate.
  • the Fresnel lens layer has a sawtooth-shaped cross section.
  • the reflective layer 10c is formed on the sawtooth-shaped slope of the Fresnel lens layer 10b.
  • the reflective layer 10c is made of a metal having a high reflectance such as aluminum, silver, and nickel.
  • the reflector 10 includes a reflective layer 10c in the shape of a Fresnel lens.
  • the actual sawtooth wave shape is larger and finer than that shown in FIG.
  • the light of the first extraction light source 21 is reflected by the reflection layer 10c and condensed by the first image pickup device 31. Since the light of the first extraction light source 21 is reflected in this way, the light does not wrap around to the first image pickup device 31 side of the subject O. Therefore, the subject O and its shadow can be clearly distinguished from other areas.
  • the reflector 10 collects the light of the first extraction light source 21 at a predetermined position. Therefore, the brightness of the light of the first extraction light source 21 can be suppressed to be lower than that of the light source used for general chroma key.
  • the subject O contains many colors, it becomes difficult to use visible light as the light of the first extraction light source 21. Therefore, in that case, for example, it is preferable to use infrared light for the light of the first extraction light source 21 and use an RGB-IR camera having sensitivity to infrared light and visible light for the first image pickup device 31.
  • the subject O and its shadow region can be extracted without being affected by the color of the subject O.
  • FIG. 3 is a diagram schematically showing a functional configuration example of the image extraction system according to the second embodiment of the present invention.
  • the first extraction light source 21 and the second image pickup device 32 are arranged on one side of the subject O, and the second extraction light source 22 and the first image pickup device 31 are arranged on the other side. It is a thing.
  • the light of the first extraction light source 21 is an infrared light WG having a wavelength wg
  • the light wavelength of the light of the second extraction light source 22 is an infrared light WR having a wavelength wr. It should be noted that both lights do not have to be infrared light. Either one or both may be visible light.
  • the first image pickup apparatus 31 has a subject O that reflects the infrared light WR of the wavelength wr of the second extraction light source 22 and a background that reflects the infrared light WG of the wavelength wg of the first extraction light source 21.
  • the captured first video signal is output.
  • the second image pickup apparatus 32 has a subject O that reflects the infrared light WG of the wavelength wg of the first extraction light source 21 and a background that reflects the infrared light WR of the wavelength wr of the second extraction light source 22.
  • the captured second video signal is output.
  • the image information extraction unit 40 deletes the light region of the infrared light WG of the first extraction light source 21 from the first video signal, and reflects the infrared light WR of the second extraction light source 22 of the subject O and the subject O.
  • the image information extraction unit 40 can extract the subject O and its background with high contrast. It is not necessary to use either the first image pickup device 31 or the second image pickup device 32.
  • the subject O extracted from each of the first video signal and the second video signal and its background may be compared, and the matching area may be extracted as image information representing the subject O and its shadow. By doing so, it is possible to improve the extraction accuracy of the image information representing the subject O and its shadow.
  • the image extraction system 200 has the configuration of the image extraction system 100 (FIG. 1) and the second extraction light source 22 arranged at a position facing the first extraction light source 21. And a second image pickup device 32 arranged at a position facing the first image pickup device 31, and the wavelength of the light emitted by the second extraction light source 22 is different from the wavelength of the first extraction light source 21.
  • the extraction unit 40 also extracts image information representing the subject O and the shadow region of the subject O from both or one of the first video signal and the second video signal output by the second image pickup device 32. As a result, image information representing the subject O and its shadow can be extracted with high contrast.
  • the brightness of the light of the first extraction light source 21 and the second extraction light source 22 is suppressed to be lower than that of the light source used for a general chroma key. be able to.
  • FIG. 4 is a cross-sectional view schematically showing a display device and a transparent flat plate used in the image extraction system of the modified example of the above embodiment.
  • the display device 50 and the transparent flat plate 60 shown in FIG. 4 are used in place of the reflector 10.
  • the display device 50 is a self-luminous display device such as a backlit liquid crystal panel or an organic EL panel, and displays an arbitrary image by a video signal from a display control device (not shown).
  • the display device 50 may have, for example, a person resting on it, and has strength to withstand the load.
  • the transparent flat plate 60 is arranged on the surface of the display device 50 and includes a half mirror 61.
  • the half mirror 61 is embedded in the transparent flat plate 60 in the same shape as the reflective layer 10c, for example.
  • the image displayed on the display device 50 can be viewed from the front surface (z direction) of the display device 50 without being refracted by the transparent flat plate 60, except for the attenuation in the half mirror 61. Further, the image information representing the subject O and the shadow region thereof, which are omitted in FIG. 4, can be extracted in the same manner as in the above embodiment.
  • the first image pickup apparatus 31 can extract image information representing a subject O arranged on an arbitrary image and a shadow region of the subject O. Since the reflector 10 of the image extraction system 200 (FIG. 3) can also be replaced with the display device 50 and the transparent flat plate 60 of the present modification 1, the second image pickup device 32 is also arranged on an arbitrary image. Image information representing the subject O and the shadow region of the subject O can be extracted.
  • the image extraction system includes a display device 50 and a transparent flat plate 60 arranged on the surface of the display device 50 and provided with a half mirror 61, instead of the reflector 10. This makes it possible to extract image information representing the subject O arranged on an arbitrary image and its shadow.
  • FIG. 5 is a cross-sectional view schematically showing a modification 2 in which a diffusion plate 70 is added to the configuration shown in FIG.
  • the modification 2 shown in FIG. 5 is different from the modification 1 in that the diffusion plate 70 is provided.
  • the diffuser plate 70 diffuses the incident light from the first extraction light source 21 and the second extraction light source 22 to smooth the luminance distribution. By the action of the diffuser plate 70, the reflected image (shadow region) of the subject O can be blurred.
  • the image of subject O remains sharp. Therefore, it is possible to easily separate the image information of the subject O from the image information representing the subject O and its shadow extracted by the image information extraction unit 40. Further, even if the first image pickup device 31 or the second image pickup device 32 is slightly deviated from the ideal position (the reflected light is collected), the reflected light from the first extraction light source 21 or the second extraction light source 22 Can receive light. That is, the design of the image extraction system can be made robust.
  • the half mirror 61 has a predetermined reflection angle, either the first extraction light source 21 and the first image pickup device 31 or the second extraction light source 22 and the second image pickup device 32 are set. If is placed in an ideal position, the other pair cannot be placed in that ideal position. However, by providing the diffuser plate 70, the first image pickup device 31 or the second image pickup device 32 can receive sufficient reflected light.
  • the image extraction system according to the modified example 1 includes a diffuser plate 70 arranged on the surface of the transparent flat plate 60 in addition to the configuration of the modified example 1. This makes it easier to separate the image information of the subject O. In addition, the design of the image extraction system can be made robust.
  • FIG. 6 is a diagram schematically showing a functional configuration example of the image extraction system according to the third embodiment of the present invention.
  • the reflector 10 of the image extraction system 100 is arranged on a wall (not shown).
  • the image extraction system 300 shown in FIG. 6 includes a reflector 13, an extraction light source 23, an image pickup device 33, and an image information extraction unit 43.
  • the reflector 13 includes a reflection layer 13c formed on a sawtooth-shaped slope and is arranged vertically. In FIG. 6, the notation of the holding plate 13a and the Fresnel lens layer 13b is omitted.
  • the extraction light source 23 irradiates the reflection layer 13c on the sawtooth-shaped slope of the reflection layer 13c with light from behind the subject O.
  • the extraction light source 23 irradiates light from the front lower part of the reflector 13.
  • the extraction light source 23 irradiates light from above the front of the reflector 13.
  • the extraction light source 23 is the same as the first extraction light source 21 and the second extraction light source 22 except that the positions of the extraction light sources 23 are different.
  • the extraction light source 23 is arranged between the reflector 13 and the subject O regardless of whether the reflective layer 13c faces downward or upward. Therefore, the shadow of the subject O is not formed on the surface of the reflector 13.
  • the image pickup device 33 is arranged at a position facing the reflector 13 with the subject O in between.
  • the image pickup apparatus 33 has a different reference code from the first extraction light source 21 of the first embodiment and the second extraction light source 22 of the second embodiment, but the first extraction light source 21 and the second extraction light source 22. Is the same as.
  • the image information extraction unit 43 extracts image information representing the subject O from the video signal output by the image pickup device 23.
  • the image information extraction unit 43 extracts only the subject O because the video signal does not include the shadow region of the subject O, but is the same as the image information extraction unit 40 (FIG. 1).
  • the image extraction system 300 includes a reflector 13 provided with a reflection layer 13c formed on a sawtooth-shaped slope and vertically arranged, and a reflection layer 13c on the sawtooth-shaped slope.
  • the subject O is obtained from an extraction light source 23 that irradiates light from behind the subject O, a reflector 13, an image pickup device 33 arranged at a corresponding position across the subject O, and a video signal output by the image pickup device 33. It is provided with an image information extraction unit 43 for extracting image information representing the region of. As a result, the light of the extraction light source 23 hits the entire surface of the reflector 13, so that the shadow region of the subject O is not generated.
  • the light of the extraction light source 23 is collected at a predetermined position by the reflector 13. Therefore, the brightness of the light of the extraction light source 23 can be suppressed to be lower than that of the light source used for general chroma key.
  • Modifications 1 and 2 may be applied to the reflector 13 of the present embodiment.
  • the modification 1 is applied to the image extraction system 300, the subject O with an arbitrary image as a background can be photographed. Further, when the modification 2 is applied, the background image can be blurred.
  • FIG. 7 is an operation flowchart showing the processing steps of the image extraction method performed by the image extraction system 100.
  • the first extraction light source 21 first irradiates light from diagonally above the subject O arranged on the reflector 10 (step S1).
  • the first image pickup apparatus 31 arranged at a position facing the first extraction light source 21 images the subject O and its background mainly based on the light of the first extraction light source 21 reflected by the reflector 10. (Step S2).
  • the image information extraction unit 40 extracts image information representing the subject O and the shadow region of the subject O from the first video signal output by the first image pickup device 31 (step S3).
  • steps S1 to S3 are repeated until the image extraction system 100 stops operating.
  • the image extraction method according to the present embodiment is an image extraction method performed by the image extraction system 100, and the image extraction system 100 includes a reflector 10 having a reflective layer in the shape of a Fresnel lens and a reflector.
  • a first extraction light source 21 that irradiates light from diagonally above the subject O arranged on the ten, a first image pickup device 31 arranged at a position facing the first extraction light source 21, and a first image pickup device.
  • An image information extraction unit 40 for extracting image information representing the subject O and the shadow region of the subject O from the first video signal output by 31 is provided, and the first extraction light source 21 irradiates the subject with light (Ste S1), the first image pickup device 31 takes an image based on the light reflected by the reflector 10 (step S2), and the image information extraction unit 40 takes the subject O from the first video signal output by the first image pickup device 31. And the image information representing the shadow area of the subject O is extracted (step S3). As a result, the area of the subject O and the background area can be clearly distinguished.
  • the light emitted by the first extraction light source 21 is reflected by the reflector 10 in the direction of the first image pickup apparatus 31, and the light hitting the subject O is reflected in the direction of the first extraction light source 21. Areas and their shadow areas can be clearly extracted.
  • the basic processing procedure of the image extraction method performed by the image extraction system 200 using two sets of the extraction light source and the image pickup device and the image extraction system 300 in which the reflector 13 is vertically arranged is the image extraction system 100. It is the same as (FIG. 7).
  • chroma key using both background light and light reflected by the subject becomes possible, and improvement in accuracy and robustness of separation between subject O and its background can be expected. Further, according to the image extraction system 300, since the light of the extraction light source 23 is irradiated to the entire reflection plate 13, it is possible to avoid extracting the shadow region of the subject O when the chroma key is applied by the light. can do.
  • the image extraction systems 100, 200, and 300 according to the present embodiment can clearly distinguish the region of the subject O.
  • first extraction light source 21 and the first image pickup device 31 have been described with the example of facing each other in the x direction, the present invention is not limited to this example.
  • the first extraction light source 21 and the first image pickup device 31 may face each other in the y direction. That is, the extraction light source and the image pickup apparatus may be arranged at any position as long as they face each other with the subject O on the reflector 10 interposed therebetween.
  • the number of subjects O may be two or more. In that case, the reflection on the surface of one subject may affect the other subject (wraparound due to diffuse reflection). According to the above embodiment, since the brightness of the extraction light source can be suppressed to a low level, the influence of wraparound due to diffuse reflection can be reduced.
  • modified example 1 in which the display device 50 and the transparent flat plate 60 are used instead of the reflector 10 and the modified example 2 in which the diffuser plate 70 is added to the modified example 1 are any of the image extraction systems 100, 200, and 300, respectively. It can also be applied to.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Circuits (AREA)
  • Processing Of Color Television Signals (AREA)
  • Studio Devices (AREA)

Abstract

The present invention comprises: a reflection plate 10 provided with a Fresnel lens-shaped reflective layer; a first extraction light source 21 which emits light from obliquely above a subject O disposed on the reflection plate 10; a first imaging device 31 disposed at a position facing the first extraction light source 21; and an image information extraction unit 40 that removes, from a first image signal output from the first imaging device 31, the region of the light of the first extraction light source 21, and that extracts image information indicating the regions of the subject O and the shadow of the subject O.

Description

画像抽出システムとその方法Image extraction system and its method
　本発明は、画像抽出システムとその方法に関する。 The present invention relates to an image extraction system and its method.
　映像の中から人や物体の正確な領域を抽出する方法として背景差分法が知られている。背景差分法は、入力画像と背景画像の差分を取り、閾値処理することで変化のあった領域を被写体の領域として抽出する。 The background subtraction method is known as a method of extracting an accurate area of a person or an object from an image. In the background subtraction method, the difference between the input image and the background image is taken, and the changed area is extracted as the subject area by performing the threshold processing.
　例えば非特許文献１に、被写体領域を判別するのに機械学習を用いた画像抽出システムが開示されている。また、背景と被写体の分離をより正確かつ頑健にする目的で赤外線を利用する技術も提案されている（例えば非特許文献２）。 For example, Non-Patent Document 1 discloses an image extraction system using machine learning to discriminate a subject area. Further, a technique using infrared rays for the purpose of making the separation between the background and the subject more accurate and robust has been proposed (for example, Non-Patent Document 2).
　例えばスポーツの試合等は、選手を斜め上から撮影する場合が多く、選手の背景は床面となる。近年では、床全面がディスプレイとなる競技コートもあって、背景差分法による被写体抽出は難しい場合がある。つまり、従来の技術では、背景と被写体の色及び照明の状態によって被写体領域と背景領域との区別が困難であるという課題がある。 For example, in sports games, players are often photographed from diagonally above, and the background of the players is the floor. In recent years, there are competition courts where the entire floor is displayed, and it may be difficult to extract subjects by the background subtraction method. That is, in the conventional technique, there is a problem that it is difficult to distinguish between the subject area and the background area depending on the color of the background and the subject and the state of illumination.
　本発明は、この課題に鑑みてなされたものであり、被写体領域と背景領域との区別が容易な画像抽出システムとその方法を提供することを目的とする。 The present invention has been made in view of this problem, and an object of the present invention is to provide an image extraction system and a method thereof, which makes it easy to distinguish between a subject area and a background area.
　本発明の一態様に係る画像抽出システムは、フレネルレンズ形状の反射層を具備する反射板と、前記反射板の上に配置される被写体の斜め上から光を照射する第１抽出用光源と、前記第１抽出用光源と対向する位置に配置される第１撮像装置と、前記第１撮像装置が出力する第１映像信号から前記被写体と該被写体の影を表す画像を抽出する画像情報抽出部とを備えることを要旨とする。 The image extraction system according to one aspect of the present invention includes a reflector provided with a Fresnel lens-shaped reflecting layer, a first light source for extracting light that irradiates light from diagonally above the subject arranged on the reflector, and the like. An image information extraction unit that extracts an image representing the subject and the shadow of the subject from the first image pickup device arranged at a position facing the first extraction light source and the first video signal output by the first image pickup device. The gist is to prepare for.
　また、本発明の他の態様に係る画像抽出システムは、鋸波形状の斜面に形成される反射層を具備し垂直に配置される反射板と、前記鋸波形状の斜面の前記反射層に、被写体の背後から光を照射する抽出用光源と、前記反射板と、前記被写体を挟んで対応する位置に配置される撮像装置と、前記撮像装置が出力する映像信号から前記被写体の領域を表す画像情報を抽出する画像情報抽出部とを備えることを要旨とする。 Further, the image extraction system according to another aspect of the present invention includes a reflector provided with a reflective layer formed on a sawtooth-shaped slope and arranged vertically, and the reflective layer on the sawtooth-shaped slope. An image representing the area of the subject from an extraction light source that irradiates light from behind the subject, the reflector, an image pickup device arranged at a corresponding position across the subject, and an image signal output by the image pickup device. The gist is to have an image information extraction unit for extracting information.
　また、本発明の一態様に係る画像抽出方法は、上記の画像抽出システムが行う画像抽出方法であって、前記画像抽出システムは、フレネルレンズ形状の反射層を具備する反射板と、前記反射板の上に配置される被写体の斜め上から光を照射する第１抽出用光源と、前記第１抽出用光源と対向する位置に配置される第１撮像装置と、前記第１撮像装置が出力する第１映像信号から前記被写体と該被写体の影の領域を表す画像情報を抽出する画像情報抽出部とを備え、前記第１抽出用光源は、前記被写体に光を照射し、前記第１撮像装置は、前記反射板で反射した光に基づいて撮像し、前記画像情報抽出部は、前記第１撮像装置が出力する第１映像信号から前記被写体と該被写体の影の領域を表す画像情報を抽出することを要旨とする。 Further, the image extraction method according to one aspect of the present invention is an image extraction method performed by the above-mentioned image extraction system, in which the image extraction system includes a reflector having a reflective layer in the shape of a Fresnel lens and the reflector. The first extraction light source that irradiates light from diagonally above the subject arranged above, the first image pickup device arranged at a position facing the first extraction light source, and the first image pickup device output. The first extraction light source includes an image information extraction unit that extracts image information representing the subject and a shadow region of the subject from the first video signal, and the first extraction light source irradiates the subject with light and the first image pickup apparatus. Takes an image based on the light reflected by the reflector, and the image information extraction unit extracts image information representing the subject and a shadow region of the subject from the first video signal output by the first image pickup apparatus. The gist is to do.
　本発明によれば、被写体領域と背景領域との区別が容易な画像抽出システムとその方法を提供することができる。 According to the present invention, it is possible to provide an image extraction system and a method thereof that make it easy to distinguish between a subject area and a background area.
本発明の第１実施形態に係る画像抽出システムの機能構成例を模式的に示す図である。It is a figure which shows typically the functional structure example of the image extraction system which concerns on 1st Embodiment of this invention. 図１に示す反射板の断面を模式的に示す図である。It is a figure which shows typically the cross section of the reflector shown in FIG. 本発明の第２実施形態に係る画像抽出システムの機能構成例を模式的に示す図である。It is a figure which shows typically the functional structure example of the image extraction system which concerns on 2nd Embodiment of this invention. 図２に示す反射板の変形例の断面を示す図である。It is a figure which shows the cross section of the modification of the reflector shown in FIG. 図４に示す反射板に拡散板を付加した変形例の断面を示す図である。It is a figure which shows the cross section of the modification which added the diffusing plate to the reflector shown in FIG. 本発明の第３実施形態に係る画像抽出システムの機能構成例を模式的に示す図である。It is a figure which shows typically the functional structure example of the image extraction system which concerns on 3rd Embodiment of this invention. 図１に示す画像抽出システムの処理手順を示すフローチャートである。It is a flowchart which shows the processing procedure of the image extraction system shown in FIG.
　以下、本発明の実施形態について図面を用いて説明する。複数の図面中同一のものには同じ参照符号を付し、説明は繰り返さない。 Hereinafter, embodiments of the present invention will be described with reference to the drawings. The same reference numerals are given to the same ones in a plurality of drawings, and the description is not repeated.
　〔第１実施形態〕
　図１は、本発明の第１実施形態に係る画像抽出システムの機能構成例を模式的に示す図である。図１に示す画像抽出システム１００は、被写体とその影の領域の画像情報を抽出するシステムである。 [First Embodiment]
FIG. 1 is a diagram schematically showing a functional configuration example of the image extraction system according to the first embodiment of the present invention. The image extraction system 100 shown in FIG. 1 is a system that extracts image information of a subject and its shadow region.
　画像抽出システム１００は、反射板１０、第１抽出用光源２１、第１撮像装置３１、及び画像情報抽出部４０を備える。図１において、反射板１０が配置される平面をｘ-ｙ平面、上方向をｚ方向と定義する。 The image extraction system 100 includes a reflector 10, a first light source for extraction 21, a first image pickup device 31, and an image information extraction unit 40. In FIG. 1, the plane on which the reflector 10 is arranged is defined as the xy plane, and the upward direction is defined as the z direction.
　反射板１０は、フレネルレンズ形状の反射層を具備する。フレネルレンズ形状は、レンズを同心円状の領域に分割して厚みを減らしたレンズと同様の形状であり、鋸波形状の断面を持つ。反射板１０は、例えば床等のｘ-ｙ平面上に配置される。 The reflector 10 includes a Fresnel lens-shaped reflective layer. The Fresnel lens shape is similar to a lens in which the lens is divided into concentric regions to reduce the thickness, and has a sawtooth-shaped cross section. The reflector 10 is arranged on an xy plane such as a floor.
　第１抽出用光源２１は、反射板１０の上に配置される被写体Ｏの斜め上（z方向）から光を照射する。被写体Ｏは人（例えば選手）であっても物であってもどちらでも構わない。 The first extraction light source 21 irradiates light from diagonally above (z direction) the subject O arranged on the reflector 10. The subject O may be a person (for example, a player) or an object.
　なお、反射板１０の表面を照らす環境光はあってもよい。環境光は、反射板１０を配置した屋内の照明装置の光である。環境光（図示せず）がある場合、第１抽出用光源２１の光の波長は環境光の波長と異なると好ましい。 Note that there may be ambient light that illuminates the surface of the reflector 10. The ambient light is the light of an indoor lighting device in which the reflector 10 is arranged. When there is ambient light (not shown), it is preferable that the wavelength of the light of the first extraction light source 21 is different from the wavelength of the ambient light.
　第１撮像装置３１は、第１抽出用光源２１と対向する位置に配置される。図１は、第１撮像装置３１がｘ方向で第１抽出用光源２１と対向する例を示す。 The first image pickup device 31 is arranged at a position facing the first extraction light source 21. FIG. 1 shows an example in which the first image pickup apparatus 31 faces the first extraction light source 21 in the x direction.
　なお、第１撮像装置３１と第１抽出用光源２１はｙ方向で対向しても構わない。第１撮像装置３１は、主に反射板１０で反射する第１抽出用光源２１の光に基づいて第１映像信号を出力する。 The first image pickup device 31 and the first extraction light source 21 may face each other in the y direction. The first image pickup apparatus 31 outputs the first video signal mainly based on the light of the first extraction light source 21 reflected by the reflector 10.
　画像情報抽出部４０は、第１映像信号から第１抽出用光源２１の光の領域を削除し、被写体Ｏと該被写体の影の領域を表す画像情報を抽出する。つまり、第１抽出用光源２１の光をキー信号としてクロマキーを適用すると、被写体Ｏとその影の領域のみを抽出することができる。 The image information extraction unit 40 deletes the light region of the first extraction light source 21 from the first video signal, and extracts image information representing the subject O and the shadow region of the subject. That is, when the chroma key is applied using the light of the first extraction light source 21 as a key signal, only the subject O and its shadow region can be extracted.
　クロマキーとは、特定の色の成分から映像の一部を透明にし、そこに別の映像を合成する技術である。クロマキー合成とも称される。画像情報抽出部４０は、パーソナルコンピュータ及び映像キャプチャボード等の一般的な装置で構成することができる。 Chroma key is a technology that makes a part of an image transparent from a specific color component and synthesizes another image there. Also called chroma key compositing. The image information extraction unit 40 can be configured by a general device such as a personal computer and a video capture board.
　以上説明したように本実施形態に係る画像抽出システム１００は、フレネルレンズ形状の反射層を具備する反射板１０と、反射板１０の上に配置される被写体Ｏの斜め上から光を照射する第１抽出用光源２１と、第１抽出用光源２１と対向する位置に配置される第１撮像装置３１と、第１撮像装置３１が出力する第１映像信号から第１抽出用光源２１の光の領域を削除し、被写体Ｏと該被写体Ｏの影の領域を表す画像情報を抽出する画像情報抽出部４０とを備える。これにより、被写体Ｏの第１撮像装置３１側に第１抽出用光源２１の光が回り込まないので、被写体Ｏとその影の領域を、その他の領域から鮮明に区別することができる。 As described above, the image extraction system 100 according to the present embodiment irradiates light from diagonally above the reflector 10 having a Fresnel lens-shaped reflective layer and the subject O arranged on the reflector 10. 1 Extraction light source 21, the first image pickup device 31 arranged at a position facing the first extraction light source 21, and the light of the first extraction light source 21 from the first video signal output by the first image pickup device 31. An image information extraction unit 40 for deleting a region and extracting image information representing a subject O and a shadow region of the subject O is provided. As a result, the light of the first extraction light source 21 does not wrap around to the first imaging device 31 side of the subject O, so that the subject O and its shadow region can be clearly distinguished from the other regions.
　（反射板）
　図２は、反射板１０の断面を模式的に示す図である。図２は、反射板１０をｘ方向で切った断面を示す。図２に示すように、反射板１０は下から保持板１０a、フレネルレンズ層１０ｂ、及び反射層１０ｃを備える。 (a reflector)
FIG. 2 is a diagram schematically showing a cross section of the reflector 10. FIG. 2 shows a cross section of the reflector 10 cut in the x direction. As shown in FIG. 2, the reflecting plate 10 includes a holding plate 10a, a Fresnel lens layer 10b, and a reflecting layer 10c from the bottom.
　保持板１０ａは、反射板１０の平坦性を維持する基板であり、例えばブラスチック板等で構成される。保持板１０ａの上には、フレネルレンズ層１０ｂが形成される。 The holding plate 10a is a substrate that maintains the flatness of the reflector 10, and is composed of, for example, a plastic plate or the like. A Fresnel lens layer 10b is formed on the holding plate 10a.
　フレネルレンズ層１０ｂは下面側が平坦であり、上面側がフレネルレンズ面を形成し、エポキシアクリルレート等の樹脂で構成される。フレネルレンズ層は鋸波形状の断面を持つ。 The lower surface side of the Fresnel lens layer 10b is flat, the upper surface side forms the Fresnel lens surface, and the Fresnel lens layer 10b is made of a resin such as epoxy acrylic rate. The Fresnel lens layer has a sawtooth-shaped cross section.
　反射層１０ｃは、フレネルレンズ層１０ｂの鋸波形状の斜面に形成される。反射層１０ｃは、例えばアルミニウム、銀、ニッケル等の高反射率の金属で構成される。 The reflective layer 10c is formed on the sawtooth-shaped slope of the Fresnel lens layer 10b. The reflective layer 10c is made of a metal having a high reflectance such as aluminum, silver, and nickel.
　このように、反射板１０は、フレネルレンズ形状の反射層１０ｃを備える。なお、実際の鋸波形状は図２に示すより数も多く細かい。 As described above, the reflector 10 includes a reflective layer 10c in the shape of a Fresnel lens. The actual sawtooth wave shape is larger and finer than that shown in FIG.
　図２に示すように、第１抽出用光源２１の光は、反射層１０ｃで反射され、第１撮像装置３１で集光する。第１抽出用光源２１の光は、このように反射するので被写体Ｏの第１撮像装置３１側にその光は回り込まない。したがって、被写体Ｏとその影を、他の領域から鮮明に区別することができる。 As shown in FIG. 2, the light of the first extraction light source 21 is reflected by the reflection layer 10c and condensed by the first image pickup device 31. Since the light of the first extraction light source 21 is reflected in this way, the light does not wrap around to the first image pickup device 31 side of the subject O. Therefore, the subject O and its shadow can be clearly distinguished from other areas.
　また、反射板１０は第１抽出用光源２１の光を所定の位置に集光する。よって、第１抽出用光源２１の光は、一般的なクロマキーに使われる光源よりも明るさを低く抑えることができる。 Further, the reflector 10 collects the light of the first extraction light source 21 at a predetermined position. Therefore, the brightness of the light of the first extraction light source 21 can be suppressed to be lower than that of the light source used for general chroma key.
　なお、被写体Ｏに多くの色が含まれる場合は、第１抽出用光源２１の光に可視光を用いることが難しくなる。よって、その場合は、例えば、第１抽出用光源２１の光に赤外線光を用い、第１撮像装置３１に赤外線光と可視光に感度を持つＲＧＢ-ＩＲカメラを用いるとよい。 If the subject O contains many colors, it becomes difficult to use visible light as the light of the first extraction light source 21. Therefore, in that case, for example, it is preferable to use infrared light for the light of the first extraction light source 21 and use an RGB-IR camera having sensitivity to infrared light and visible light for the first image pickup device 31.
　赤外線光を用いることで、被写体Ｏの色に影響されることなく被写体Ｏとその影の領域を抽出することができる。 By using infrared light, the subject O and its shadow region can be extracted without being affected by the color of the subject O.
　〔第２実施形態〕
　図３は、本発明の第２実施形態に係る画像抽出システムの機能構成例を模式的に示す図である。図３に示す画像抽出システム２００は、被写体Ｏを挟んで一方に第１抽出用光源２１と第２撮像装置３２、他方に第２抽出用光源２２と第１撮像装置３１を配置するようにしたものである。 [Second Embodiment]
FIG. 3 is a diagram schematically showing a functional configuration example of the image extraction system according to the second embodiment of the present invention. In the image extraction system 200 shown in FIG. 3, the first extraction light source 21 and the second image pickup device 32 are arranged on one side of the subject O, and the second extraction light source 22 and the first image pickup device 31 are arranged on the other side. It is a thing.
　本実施形態に係る第１抽出用光源２１の光は、波長ｗｇの赤外線光ＷＧであり、第２抽出用光源２２の光の波長は波長ｗｒの赤外線光ＷＲである。なお、両方の光が赤外線光である必要はない。どちらか一方又は両方は可視光であっても構わない。 The light of the first extraction light source 21 according to the present embodiment is an infrared light WG having a wavelength wg, and the light wavelength of the light of the second extraction light source 22 is an infrared light WR having a wavelength wr. It should be noted that both lights do not have to be infrared light. Either one or both may be visible light.
　この場合、第１撮像装置３１は、第２抽出用光源２２の波長ｗｒの赤外線光ＷＲを反射する被写体Ｏと、第１抽出用光源２１の波長ｗｇの赤外線光ＷＧが反射された背景とを撮像した第１映像信号を出力する。同様に、第２撮像装置３２は、第１抽出用光源２１の波長ｗｇの赤外線光ＷＧを反射する被写体Ｏと、第２抽出用光源２２の波長ｗｒの赤外線光ＷＲが反射された背景とを撮像した第２映像信号を出力する。 In this case, the first image pickup apparatus 31 has a subject O that reflects the infrared light WR of the wavelength wr of the second extraction light source 22 and a background that reflects the infrared light WG of the wavelength wg of the first extraction light source 21. The captured first video signal is output. Similarly, the second image pickup apparatus 32 has a subject O that reflects the infrared light WG of the wavelength wg of the first extraction light source 21 and a background that reflects the infrared light WR of the wavelength wr of the second extraction light source 22. The captured second video signal is output.
　画像情報抽出部４０は、第１映像信号から第１抽出用光源２１の赤外線光ＷＧの光の領域を削除し、第２抽出用光源２２の赤外線光ＷＲを反射する被写体Ｏと該被写体Ｏの影の画像情報を抽出し、第２映像信号から第２抽出用光源２２の赤外線光ＷＲの光の領域を削除し、第１抽出用光源２１の赤外線光ＷＧを反射する被写体Ｏと該被写体Ｏの影を表す画像情報を抽出する。 The image information extraction unit 40 deletes the light region of the infrared light WG of the first extraction light source 21 from the first video signal, and reflects the infrared light WR of the second extraction light source 22 of the subject O and the subject O. The subject O and the subject O that reflect the infrared light WG of the first extraction light source 21 by extracting the image information of the shadow and deleting the light region of the infrared light WR of the second extraction light source 22 from the second video signal. Extract the image information that represents the shadow of.
　このように画像情報抽出部４０は、被写体Ｏとその背景を高いコントラストで抽出することができる。なお、第１撮像装置３１と第２撮像装置３２のどちらか一方は無くても構わない。 In this way, the image information extraction unit 40 can extract the subject O and its background with high contrast. It is not necessary to use either the first image pickup device 31 or the second image pickup device 32.
　また、第１映像信号と第２映像信号のそれぞれから抽出した被写体Ｏとその背景を比較し、一致する領域を被写体Ｏとその影を表す画像情報として抽出するようにしてもよい。そうすることで、被写体Ｏとその影を表す画像情報の抽出精度を高めることができる。 Alternatively, the subject O extracted from each of the first video signal and the second video signal and its background may be compared, and the matching area may be extracted as image information representing the subject O and its shadow. By doing so, it is possible to improve the extraction accuracy of the image information representing the subject O and its shadow.
　以上説明したように本実施形態に係る画像抽出システム２００は、画像抽出システム１００（図１）の構成に加えて、第１抽出用光源２１と対向する位置に配置される第２抽出用光源２２と、第１撮像装置３１と対向する位置に配置される第２撮像装置３２とを備え、第２抽出用光源２２が照射する光の波長は第１抽出用光源２１の波長と異なり、画像情報抽出部４０は、上記の第１映像信号と第２撮像装置３２が出力する第２映像信号の両方又は一方から被写体Ｏと該被写体Ｏの影の領域を表す画像情報も抽出する。これにより、被写体Ｏとその影を表す画像情報を高いコントラストで抽出することができる。 As described above, the image extraction system 200 according to the present embodiment has the configuration of the image extraction system 100 (FIG. 1) and the second extraction light source 22 arranged at a position facing the first extraction light source 21. And a second image pickup device 32 arranged at a position facing the first image pickup device 31, and the wavelength of the light emitted by the second extraction light source 22 is different from the wavelength of the first extraction light source 21. The extraction unit 40 also extracts image information representing the subject O and the shadow region of the subject O from both or one of the first video signal and the second video signal output by the second image pickup device 32. As a result, image information representing the subject O and its shadow can be extracted with high contrast.
　なお、第１実施形態の画像抽出システム１００と同様に、第１抽出用光源２１と第２抽出用光源２２の光の明るさは、一般的なクロマキーに使われる光源よりも明るさを低く抑えることができる。 Similar to the image extraction system 100 of the first embodiment, the brightness of the light of the first extraction light source 21 and the second extraction light source 22 is suppressed to be lower than that of the light source used for a general chroma key. be able to.
　（変形例１）
　図４は、上記の実施形態の変形例の画像抽出システムに用いる表示装置と透明平板を模式的に示す断面図である。図４に示す表示装置５０と透明平板６０は、反射板１０に替えて用いる。 (Modification 1)
FIG. 4 is a cross-sectional view schematically showing a display device and a transparent flat plate used in the image extraction system of the modified example of the above embodiment. The display device 50 and the transparent flat plate 60 shown in FIG. 4 are used in place of the reflector 10.
　表示装置５０は、バックライト付きの液晶パネル又は有機ＥＬパネル等の自発光型の表示装置であり、図示を省略している表示制御装置からの映像信号により任意の画像を表示する。表示装置５０は、その上に例えば人が載る場合もあり、その荷重に耐えられる強度を備える。 The display device 50 is a self-luminous display device such as a backlit liquid crystal panel or an organic EL panel, and displays an arbitrary image by a video signal from a display control device (not shown). The display device 50 may have, for example, a person resting on it, and has strength to withstand the load.
　透明平板６０は、表示装置５０の表面に配置され、ハーフミラー６１を具備する。ハーフミラー６１は、例えば反射層１０ｃと同じ形状で透明平板６０の中に埋め込まれる。 The transparent flat plate 60 is arranged on the surface of the display device 50 and includes a half mirror 61. The half mirror 61 is embedded in the transparent flat plate 60 in the same shape as the reflective layer 10c, for example.
　表示装置５０に表示された映像は、ハーフミラー６１での減衰を除いて表示装置５０の正面（ｚ方向）から透明平板６０で屈折することなく見ることができる。また、図４では省略している被写体Ｏ及びその影の領域を表す画像情報は、上記の実施形態と同様に抽出することができる。 The image displayed on the display device 50 can be viewed from the front surface (z direction) of the display device 50 without being refracted by the transparent flat plate 60, except for the attenuation in the half mirror 61. Further, the image information representing the subject O and the shadow region thereof, which are omitted in FIG. 4, can be extracted in the same manner as in the above embodiment.
　よって、第１撮像装置３１は、任意の画像の上に配置された被写体Ｏと該被写体Ｏの影の領域を表す画像情報を抽出することができる。なお、画像抽出システム２００（図３）の反射板１０も本変形例１の表示装置５０と透明平板６０に替えることができるので、第２撮像装置３２も、任意の画像の上に配置された被写体Ｏと該被写体Ｏの影の領域を表す画像情報を抽出することができる。 Therefore, the first image pickup apparatus 31 can extract image information representing a subject O arranged on an arbitrary image and a shadow region of the subject O. Since the reflector 10 of the image extraction system 200 (FIG. 3) can also be replaced with the display device 50 and the transparent flat plate 60 of the present modification 1, the second image pickup device 32 is also arranged on an arbitrary image. Image information representing the subject O and the shadow region of the subject O can be extracted.
　以上説明したように変形例１に係る画像抽出システムは、反射板１０に替えて、表示装置５０と、該表示装置５０の表面に配置され、ハーフミラー６１を具備する透明平板６０とを備える。これにより、任意の画像の上に配置された被写体Ｏとその影を表す画像情報を抽出することができる。 As described above, the image extraction system according to the first modification includes a display device 50 and a transparent flat plate 60 arranged on the surface of the display device 50 and provided with a half mirror 61, instead of the reflector 10. This makes it possible to extract image information representing the subject O arranged on an arbitrary image and its shadow.
　（変形例２）
　図５は、図４に示す構成に、拡散板７０を追加した変形例２を模式的に示す断面図である。図５に示す変形例２は、拡散板７０を備える点で変形例１と異なる。 (Modification 2)
FIG. 5 is a cross-sectional view schematically showing a modification 2 in which a diffusion plate 70 is added to the configuration shown in FIG. The modification 2 shown in FIG. 5 is different from the modification 1 in that the diffusion plate 70 is provided.
　拡散板７０は、第１抽出用光源２１及び第２抽出用光源２２からの入射光を拡散させ、輝度分布をなだらかにする。拡散板７０の作用により、被写体Ｏの反射像（影の領域）にぼかしを生じさせることができる。 The diffuser plate 70 diffuses the incident light from the first extraction light source 21 and the second extraction light source 22 to smooth the luminance distribution. By the action of the diffuser plate 70, the reflected image (shadow region) of the subject O can be blurred.
　一方で、被写体Ｏの映像は鮮鋭なままである。よって、画像情報抽出部４０で抽出した被写体Ｏとその影を表す画像情報から、被写体Ｏの画像情報を分離し易くすることができる。また、第１撮像装置３１又は第２撮像装置３２が理想的な（反射光が集光する）位置から少々ずれている場合でも第１抽出用光源２１又は第２抽出用光源２２からの反射光を受光できる。つまり、画像抽出システムの設計を頑健にすることができる。 On the other hand, the image of subject O remains sharp. Therefore, it is possible to easily separate the image information of the subject O from the image information representing the subject O and its shadow extracted by the image information extraction unit 40. Further, even if the first image pickup device 31 or the second image pickup device 32 is slightly deviated from the ideal position (the reflected light is collected), the reflected light from the first extraction light source 21 or the second extraction light source 22 Can receive light. That is, the design of the image extraction system can be made robust.
　また、ハーフミラー６１は、所定の反射角度をもつので、第１抽出用光源２１と第１撮像装置３１の組、又は、第２抽出用光源２２と第２撮像装置３２の組のどちらか一方を理想的な位置に配置した場合、他方の組はその理想的な位置に配置することができない。しかし、拡散板７０を備えることで第１撮像装置３１又は第２撮像装置３２は、十分な反射光を受光することが可能になる。 Further, since the half mirror 61 has a predetermined reflection angle, either the first extraction light source 21 and the first image pickup device 31 or the second extraction light source 22 and the second image pickup device 32 are set. If is placed in an ideal position, the other pair cannot be placed in that ideal position. However, by providing the diffuser plate 70, the first image pickup device 31 or the second image pickup device 32 can receive sufficient reflected light.
　以上説明したように変形例１に係る画像抽出システムは、変形例１の構成に加えて透明平板６０の表面に配置される拡散板７０を備える。これにより、被写体Ｏの画像情報を分離し易くすることができる。また、画像抽出システムの設計を頑健にすることができる。 As described above, the image extraction system according to the modified example 1 includes a diffuser plate 70 arranged on the surface of the transparent flat plate 60 in addition to the configuration of the modified example 1. This makes it easier to separate the image information of the subject O. In addition, the design of the image extraction system can be made robust.
　〔第３実施形態〕
　図６は、本発明の第３実施形態に係る画像抽出システムの機能構成例を模式的に示す図である。図６に示す画像抽出システム３００は、画像抽出システム１００の反射板１０を壁（図示せず）に配置するようにしたものである。 [Third Embodiment]
FIG. 6 is a diagram schematically showing a functional configuration example of the image extraction system according to the third embodiment of the present invention. In the image extraction system 300 shown in FIG. 6, the reflector 10 of the image extraction system 100 is arranged on a wall (not shown).
　図６に示す画像抽出システム３００は、反射板１３、抽出用光源２３、撮像装置３３、及び画像情報抽出部４３を備える。 The image extraction system 300 shown in FIG. 6 includes a reflector 13, an extraction light source 23, an image pickup device 33, and an image information extraction unit 43.
　反射板１３は、鋸波形状の斜面に形成される反射層１３ｃを具備し垂直に配置される。なお、図６において保持板１３ａとフレネルレンズ層１３ｂの表記は省略している。 The reflector 13 includes a reflection layer 13c formed on a sawtooth-shaped slope and is arranged vertically. In FIG. 6, the notation of the holding plate 13a and the Fresnel lens layer 13b is omitted.
　抽出用光源２３は、反射層１３ｃの鋸波形状の斜面の反射層１３ｃに、被写体Ｏの背後から光を照射する。反射層１３ｃが下向きの場合、抽出用光源２３は反射板１３の前方下から光を照射する。また、反射層１３ｃが上向きの場合、抽出用光源２３は反射板１３の前方上から光を照射する。抽出用光源２３は、配置される位置が異なるだけで第１抽出用光源２１及び第２抽出用光源２２と同じものである。 The extraction light source 23 irradiates the reflection layer 13c on the sawtooth-shaped slope of the reflection layer 13c with light from behind the subject O. When the reflective layer 13c faces downward, the extraction light source 23 irradiates light from the front lower part of the reflector 13. When the reflective layer 13c faces upward, the extraction light source 23 irradiates light from above the front of the reflector 13. The extraction light source 23 is the same as the first extraction light source 21 and the second extraction light source 22 except that the positions of the extraction light sources 23 are different.
　反射層１３ｃが下向き又は上向きに関わりなく、抽出用光源２３は反射板１３と被写体Ｏとの間に配置される。よって、被写体Ｏの影が反射板１３の表面に形成されることがない。 The extraction light source 23 is arranged between the reflector 13 and the subject O regardless of whether the reflective layer 13c faces downward or upward. Therefore, the shadow of the subject O is not formed on the surface of the reflector 13.
　撮像装置３３は、反射板１３と被写体Ｏを挟んで対向する位置に配置される。撮像装置３３は、第１実施形態の第１抽出用光源２１及び第２実施形態の第２抽出用光源２２と参照符号を変えているが、第１抽出用光源２１及び第２抽出用光源２２と同じものである。 The image pickup device 33 is arranged at a position facing the reflector 13 with the subject O in between. The image pickup apparatus 33 has a different reference code from the first extraction light source 21 of the first embodiment and the second extraction light source 22 of the second embodiment, but the first extraction light source 21 and the second extraction light source 22. Is the same as.
　画像情報抽出部４３は、撮像装置２３が出力する映像信号から被写体Ｏを表す画像情報を抽出する。画像情報抽出部４３は、映像信号に被写体Ｏの影の領域が含まれないので被写体Ｏのみを抽出するが、画像情報抽出部４０（図１）と同じものである。 The image information extraction unit 43 extracts image information representing the subject O from the video signal output by the image pickup device 23. The image information extraction unit 43 extracts only the subject O because the video signal does not include the shadow region of the subject O, but is the same as the image information extraction unit 40 (FIG. 1).
　以上説明したように本実施形態に係る画像抽出システム３００は、鋸波形状の斜面に形成される反射層１３ｃを具備し垂直に配置される反射板１３と、鋸波形状の斜面の反射層１３ｃに、被写体Ｏの背後から光を照射する抽出用光源２３と、反射板１３と、被写体Ｏを挟んで対応する位置に配置される撮像装置３３と、撮像装置３３が出力する映像信号から被写体Ｏの領域を表す画像情報を抽出する画像情報抽出部４３とを備える。これにより、反射板１３の全面に抽出用光源２３の光が当たるため、被写体Ｏの影の領域を生じさせない。 As described above, the image extraction system 300 according to the present embodiment includes a reflector 13 provided with a reflection layer 13c formed on a sawtooth-shaped slope and vertically arranged, and a reflection layer 13c on the sawtooth-shaped slope. The subject O is obtained from an extraction light source 23 that irradiates light from behind the subject O, a reflector 13, an image pickup device 33 arranged at a corresponding position across the subject O, and a video signal output by the image pickup device 33. It is provided with an image information extraction unit 43 for extracting image information representing the region of. As a result, the light of the extraction light source 23 hits the entire surface of the reflector 13, so that the shadow region of the subject O is not generated.
　また、抽出用光源２３の光を反射板１３で所定の位置に集光する。よって、抽出用光源２３の光は、一般的なクロマキーに使われる光源よりも明るさを低く抑えることができる。 Further, the light of the extraction light source 23 is collected at a predetermined position by the reflector 13. Therefore, the brightness of the light of the extraction light source 23 can be suppressed to be lower than that of the light source used for general chroma key.
　なお、本実施形態の反射板１３に対して変形例１，２を適用してもよい。画像抽出システム３００に変形例１を適用した場合は、任意の画像を背景にした被写体Ｏを撮影することができる。また、変形例２を適用した場合は、背景の画像にぼかしを生じさせることができる。 Modifications 1 and 2 may be applied to the reflector 13 of the present embodiment. When the modification 1 is applied to the image extraction system 300, the subject O with an arbitrary image as a background can be photographed. Further, when the modification 2 is applied, the background image can be blurred.
　（画像抽出方法）
　図７は、上記の画像抽出システム１００が行う画像抽出方法の処理ステップを示す動作フローチャートである。 (Image extraction method)
FIG. 7 is an operation flowchart showing the processing steps of the image extraction method performed by the image extraction system 100.
　画像抽出システム１００は、動作を開始すると先ず第１抽出用光源２１が、反射板１０の上に配置される被写体Ｏの斜め上から光を照射する（ステップＳ１）。 When the image extraction system 100 starts operation, the first extraction light source 21 first irradiates light from diagonally above the subject O arranged on the reflector 10 (step S1).
　次に、第１抽出用光源２１と対向する位置に配置された第１撮像装置３１は、主に反射板１０で反射する第１抽出用光源２１の光に基づいて被写体Ｏとその背景を撮像する（ステップＳ２）。 Next, the first image pickup apparatus 31 arranged at a position facing the first extraction light source 21 images the subject O and its background mainly based on the light of the first extraction light source 21 reflected by the reflector 10. (Step S2).
　次に、画像情報抽出部４０は、第１撮像装置３１が出力する第１映像信号から被写体Ｏと該被写体Ｏの影の領域を表す画像情報を抽出する（ステップＳ３）。 Next, the image information extraction unit 40 extracts image information representing the subject O and the shadow region of the subject O from the first video signal output by the first image pickup device 31 (step S3).
　図７では省略しているがステップＳ１～Ｓ３は、画像抽出システム１００が動作を停止するまで繰り返される。 Although omitted in FIG. 7, steps S1 to S3 are repeated until the image extraction system 100 stops operating.
　以上説明したように本実施形態に係る画像抽出方法は、画像抽出システム１００が行う画像抽出方法であって、画像抽出システム１００は、フレネルレンズ形状の反射層を具備する反射板１０と、反射板１０の上に配置される被写体Ｏの斜め上から光を照射する第１抽出用光源２１と、第１抽出用光源２１と対向する位置に配置される第１撮像装置３１と、第１撮像装置３１が出力する第１映像信号から被写体Ｏと該被写体Ｏの影の領域を表す画像情報を抽出する画像情報抽出部４０とを備え、第１抽出用光源２１は、被写体に光を照射し（ステップＳ１）、第１撮像装置３１は、反射板１０で反射した光に基づいて撮像し（ステップＳ２）、画像情報抽出部４０は、第１撮像装置３１が出力する第１映像信号から被写体Ｏと該被写体Ｏの影の領域を表す画像情報を抽出する（ステップＳ３）。これにより、被写体Ｏの領域と背景領域とを鮮明に区別することができる。 As described above, the image extraction method according to the present embodiment is an image extraction method performed by the image extraction system 100, and the image extraction system 100 includes a reflector 10 having a reflective layer in the shape of a Fresnel lens and a reflector. A first extraction light source 21 that irradiates light from diagonally above the subject O arranged on the ten, a first image pickup device 31 arranged at a position facing the first extraction light source 21, and a first image pickup device. An image information extraction unit 40 for extracting image information representing the subject O and the shadow region of the subject O from the first video signal output by 31 is provided, and the first extraction light source 21 irradiates the subject with light ( Step S1), the first image pickup device 31 takes an image based on the light reflected by the reflector 10 (step S2), and the image information extraction unit 40 takes the subject O from the first video signal output by the first image pickup device 31. And the image information representing the shadow area of the subject O is extracted (step S3). As a result, the area of the subject O and the background area can be clearly distinguished.
　つまり、第１抽出用光源２１が照射する光は反射板１０によって第１撮像装置３１の方向に反射され、被写体Ｏに当たる光は第１抽出用光源２１の方向へ反射されるので、被写体Ｏの領域とその影の領域を鮮明に抽出することができる。 That is, the light emitted by the first extraction light source 21 is reflected by the reflector 10 in the direction of the first image pickup apparatus 31, and the light hitting the subject O is reflected in the direction of the first extraction light source 21. Areas and their shadow areas can be clearly extracted.
　なお、抽出用光源と撮像装置の組を2つ用いる画像抽出システム２００、及び反射板１３を垂直に配置する画像抽出システム３００のそれぞれが行う画像抽出方法の基本的な処理手順は画像抽出システム１００（図７）と同じである。 The basic processing procedure of the image extraction method performed by the image extraction system 200 using two sets of the extraction light source and the image pickup device and the image extraction system 300 in which the reflector 13 is vertically arranged is the image extraction system 100. It is the same as (FIG. 7).
　画像抽出システム２００によれば、背景光と被写体で反射する光の両方を用いたクロマキーが可能となり、被写体Ｏとその背景の分離の精度向上や頑健性向上が期待できる。また、画像抽出システム３００によれば、反射板１３の全体に抽出用光源２３の光が照射されるため、その光でクロマキーを適用した際に被写体Ｏの影の領域が抽出されることを回避することができる。 According to the image extraction system 200, chroma key using both background light and light reflected by the subject becomes possible, and improvement in accuracy and robustness of separation between subject O and its background can be expected. Further, according to the image extraction system 300, since the light of the extraction light source 23 is irradiated to the entire reflection plate 13, it is possible to avoid extracting the shadow region of the subject O when the chroma key is applied by the light. can do.
　以上説明したように本実施形態に係る画像抽出システム１００，２００，３００は、被写体Ｏの領域を鮮明に区別することができる。 As described above, the image extraction systems 100, 200, and 300 according to the present embodiment can clearly distinguish the region of the subject O.
　なお、第１抽出用光源２１と第１撮像装置３１は、ｘ方向で対向する例で説明したが、本発明はこの例に限定されない。第１抽出用光源２１と第１撮像装置３１は、ｙ方向で対向してもよい。つまり、抽出用光源と撮像装置は、反射板１０上の被写体Ｏを挟んで対向する位置であればどのような位置に配置しても構わない。 Although the first extraction light source 21 and the first image pickup device 31 have been described with the example of facing each other in the x direction, the present invention is not limited to this example. The first extraction light source 21 and the first image pickup device 31 may face each other in the y direction. That is, the extraction light source and the image pickup apparatus may be arranged at any position as long as they face each other with the subject O on the reflector 10 interposed therebetween.
　なお、被写体の数は一つの例で説明したが、被写体Ｏは二つ以上の複数であっても構わない。その場合、一つの被写体の表面の反射が他方の被写体に影響（拡散反射による回り込み）する場合がある。上記の実施形態によれば、抽出用光源の明るさを低く抑えることができるので拡散反射による回り込みの影響も小さくすることができる。 Although the number of subjects has been explained in one example, the number of subjects O may be two or more. In that case, the reflection on the surface of one subject may affect the other subject (wraparound due to diffuse reflection). According to the above embodiment, since the brightness of the extraction light source can be suppressed to a low level, the influence of wraparound due to diffuse reflection can be reduced.
　また、反射板１０に替えて、表示装置５０と透明平板６０を用いる変形例１、及び変形例１に拡散板７０を追加した変形例２のそれぞれは、画像抽出システム１００，２００，３００の何れにも適用することが可能である。 Further, the modified example 1 in which the display device 50 and the transparent flat plate 60 are used instead of the reflector 10, and the modified example 2 in which the diffuser plate 70 is added to the modified example 1 are any of the image extraction systems 100, 200, and 300, respectively. It can also be applied to.
　このように、本発明はここでは記載していない様々な実施形態等を含むことは勿論である。したがって、本発明の技術的範囲は上記の説明から妥当な特許請求の範囲に係る発明特定事項によってのみ定められるものである。 As described above, it goes without saying that the present invention includes various embodiments not described here. Therefore, the technical scope of the present invention is defined only by the matters specifying the invention relating to the reasonable claims from the above description.
１０，１３：反射板
２１：第１抽出用光源
２２：第２抽出用光源
２３：抽出用光源
３１：第１撮像装置
３２：第２撮像装置
３３：撮像装置
４０，４３：画像情報抽出部
５０：表示装置
６０：透明平板
６１：ハーフミラー
７０：拡散板 10, 13: Reflector 21: First extraction light source 22: Second extraction light source 23: Extraction light source 31: First image pickup device 32: Second image pickup device 33: Image pickup device 40, 43: Image information extraction unit 50 : Display device 60: Transparent flat plate 61: Half mirror 70: Diffusing plate

Claims (8)

  1. 　フレネルレンズ形状の反射層を具備する反射板と、
    　前記反射板の上に配置される被写体の斜め上から光を照射する第１抽出用光源と、
    　前記第１抽出用光源と対向する位置に配置される第１撮像装置と、
    　前記第１撮像装置が出力する第１映像信号から前記第１抽出用光源の光の領域を削除し、前記被写体と該被写体の影の領域を表す画像情報を抽出する画像情報抽出部と
    　を備える画像抽出システム。     A reflector with a Fresnel lens-shaped reflective layer,
    A first extraction light source that irradiates light from diagonally above the subject placed on the reflector, and
    A first image pickup device arranged at a position facing the first extraction light source, and
    It is provided with an image information extraction unit that deletes the light region of the first extraction light source from the first video signal output by the first image pickup apparatus and extracts image information representing the subject and the shadow region of the subject. Image extraction system.
  2. 　前記第１抽出用光源と対向する位置に配置される第２抽出用光源と、
    　前記第１撮像装置と対向する位置に配置される第２撮像装置とを備え、
    　前記第２抽出用光源が照射する光の波長は前記第１抽出用光源の波長と異なり、前記画像情報抽出部は、前記第１映像信号と前記第２撮像装置が出力する第２映像信号の両方又は一方から前記被写体と該被写体の影の領域を表す画像情報も抽出する
    　請求項１に記載の画像抽出システム。     A second extraction light source arranged at a position facing the first extraction light source, and a second extraction light source.
    A second image pickup device arranged at a position facing the first image pickup device is provided.
    The wavelength of the light emitted by the second extraction light source is different from the wavelength of the first extraction light source, and the image information extraction unit receives the first video signal and the second video signal output by the second image pickup apparatus. The image extraction system according to claim 1, wherein image information representing the subject and a shadow region of the subject is also extracted from both or one of them.
  3. 　前記反射板に替えて、
    　表示装置と、
    　該表示装置の表面に配置され、ハーフミラーを具備する透明平板と
    　を備える請求項１又２に記載の画像抽出システム。     Instead of the reflector,
    Display device and
    The image extraction system according to claim 1 or 2, which is arranged on the surface of the display device and includes a transparent flat plate provided with a half mirror.
  4. 　前記反射板又は前記透明平板の表面に配置される拡散板
    　を備える請求項１乃至３の何れかに記載の画像抽出システム。     The image extraction system according to any one of claims 1 to 3, further comprising a diffuser plate arranged on the surface of the reflector or the transparent flat plate.
  5. 　鋸波形状の斜面に形成される反射層を具備し垂直に配置される反射板と、
    　前記鋸波形状の斜面の前記反射層に、被写体の背後から光を照射する抽出用光源と、
    　前記反射板と、前記被写体を挟んで対応する位置に配置される撮像装置と、
    　前記撮像装置が出力する映像信号から前記被写体の領域を表す画像情報を抽出する画像情報抽出部と
    　を備える画像抽出システム。     A reflector with a reflective layer formed on a sawtooth-shaped slope and arranged vertically,
    An extraction light source that irradiates the reflective layer on the sawtooth-shaped slope with light from behind the subject.
    The reflector, an image pickup device arranged at a corresponding position across the subject, and the image pickup device.
    An image extraction system including an image information extraction unit that extracts image information representing the area of the subject from a video signal output by the image pickup apparatus.
  6. 　前記反射板に替えて、
    　表示装置と、
    　該表示装置の表面に配置され、ハーフミラーを具備する透明平板と
    　を備える請求項５に記載の画像抽出システム。     Instead of the reflector,
    Display device and
    The image extraction system according to claim 5, further comprising a transparent flat plate provided on the surface of the display device and provided with a half mirror.
  7. 　前記反射板又は前記透明平板の表面に配置される拡散板
    　を備える請求項５又は６に記載の画像抽出システム。     The image extraction system according to claim 5 or 6, further comprising a diffuser plate arranged on the surface of the reflector or the transparent flat plate.
  8. 　画像抽出システムが行う画像抽出方法であって、
    　前記画像抽出システムは、フレネルレンズ形状の反射層を具備する反射板と、
    　前記反射板の上に配置される被写体の斜め上から光を照射する第１抽出用光源と、
    　前記第１抽出用光源と対向する位置に配置される第１撮像装置と、
    　前記第１撮像装置が出力する第１映像信号から前記被写体と該被写体の影の領域を表す画像情報を抽出する画像情報抽出部と
    　を備え、
    　前記第１抽出用光源は、前記被写体に光を照射し、
    　前記第１撮像装置は、前記反射板で反射した光に基づいて撮像し、
    　前記画像情報抽出部は、前記第１撮像装置が出力する第１映像信号から前記被写体と該被写体の影の領域を表す画像情報を抽出する
    　画像抽出方法。     This is an image extraction method performed by an image extraction system.
    The image extraction system includes a reflector having a Fresnel lens-shaped reflective layer and a reflector.
    A first extraction light source that irradiates light from diagonally above the subject placed on the reflector, and
    A first image pickup device arranged at a position facing the first extraction light source, and
    It is provided with an image information extraction unit that extracts image information representing the subject and the shadow region of the subject from the first video signal output by the first image pickup apparatus.
    The first extraction light source irradiates the subject with light.
    The first image pickup apparatus captures images based on the light reflected by the reflector.
    The image information extraction unit is an image extraction method for extracting image information representing a subject and a shadow region of the subject from a first video signal output by the first image pickup apparatus.
PCT/JP2020/019475 2020-05-15 2020-05-15 System and method for image extraction WO2021229804A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/019475 WO2021229804A1 (en) 2020-05-15 2020-05-15 System and method for image extraction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/019475 WO2021229804A1 (en) 2020-05-15 2020-05-15 System and method for image extraction

Publications (1)

Publication Number Publication Date
WO2021229804A1 true WO2021229804A1 (en) 2021-11-18

Family

ID=78525611

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/019475 WO2021229804A1 (en) 2020-05-15 2020-05-15 System and method for image extraction

Country Status (1)

Country Link
WO (1) WO2021229804A1 (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002118770A (en) * 2000-10-05 2002-04-19 Sony Corp Background device
JP2002118859A (en) * 2000-10-05 2002-04-19 Sony Corp Photographing device and photographing method
JP2003058873A (en) * 2001-08-13 2003-02-28 Olympus Optical Co Ltd Device and method for extracting form and segmenting image
JP2004077958A (en) * 2002-08-21 2004-03-11 Make Softwear:Kk Photographic print preparing apparatus
JP2006065164A (en) * 2004-08-30 2006-03-09 Toppan Printing Co Ltd Reflection type screen and display device
JP2010527168A (en) * 2007-04-03 2010-08-05 ライフ タッチ インク． Background replacement method and apparatus for still photography
JP2015148710A (en) * 2014-02-06 2015-08-20 日本電信電話株式会社 Screen and multi-oriented video display system

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002118770A (en) * 2000-10-05 2002-04-19 Sony Corp Background device
JP2002118859A (en) * 2000-10-05 2002-04-19 Sony Corp Photographing device and photographing method
JP2003058873A (en) * 2001-08-13 2003-02-28 Olympus Optical Co Ltd Device and method for extracting form and segmenting image
JP2004077958A (en) * 2002-08-21 2004-03-11 Make Softwear:Kk Photographic print preparing apparatus
JP2006065164A (en) * 2004-08-30 2006-03-09 Toppan Printing Co Ltd Reflection type screen and display device
JP2010527168A (en) * 2007-04-03 2010-08-05 ライフ タッチ インク． Background replacement method and apparatus for still photography
JP2015148710A (en) * 2014-02-06 2015-08-20 日本電信電話株式会社 Screen and multi-oriented video display system

Similar Documents

Publication Publication Date Title
CN209625238U (en) Shield lower optical fingerprint identification system and electronic device
CN101573988B (en) Specular reflection reduction using multiple cameras
EP1804942B1 (en) Game board, pawn, sticker and system for detecting pawns on a game board
CN101523278B (en) Interactive display using planar radiation guide
CN101836180B (en) Detecting ambient light levels in a vision system
US7394058B2 (en) Touch screen with light-enhancing layer
TWI288384B (en) Display device and information terminal device
CN101971128B (en) Interaction arrangement for interaction between a display screen and a pointer object
US10311271B2 (en) Diffuse bright field illumination system for a barcode reader
CN109154959A (en) Optical fingerprint sensor with non-contact imaging capability
US20130100082A1 (en) Touch panels with dynamic zooming and low profile bezels
RU2003137846A (en) INTERACTIVE VIDEO DISPLAY SYSTEM
JP2007514242A (en) Built-in interactive video display system
JP2007514241A (en) Built-in interactive video display system
WO2020087241A1 (en) Under-screen optical fingerprint device and handheld device having three-dimensional fingerprint anti-counterfeit sensing function
WO2021097720A1 (en) Under-screen fingerprint identification device and terminal apparatus
CN100412690C (en) Rear projection screen
CN102667689A (en) Interactive display
WO2021229804A1 (en) System and method for image extraction
US20200167542A1 (en) Electronic Device, Method For Controlling The Same, And Computer Readable Storage Medium
CN103348306B (en) Interactive display device and method for the same
TW200417390A (en) Entertainment apparatus using card
KR20210040897A (en) Fingerprint Acquisition Sensor for Mobile Device, the Mobile Device Comprising the Sensor and Fingerprint Image Acqusition Method thereof
US8079714B2 (en) Projector and method for acquiring coordinate of bright spot
TWI772826B (en) Fingerprint identification device and fingerprint identification method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20935792

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20935792

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP