WO2021193102A1 - Imaging device, video processing device, and video processing method - Google Patents
Imaging device, video processing device, and video processing method Download PDFInfo
- Publication number
- WO2021193102A1 WO2021193102A1 PCT/JP2021/009792 JP2021009792W WO2021193102A1 WO 2021193102 A1 WO2021193102 A1 WO 2021193102A1 JP 2021009792 W JP2021009792 W JP 2021009792W WO 2021193102 A1 WO2021193102 A1 WO 2021193102A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- video
- resolution
- low
- image
- imaging device
- Prior art date
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
Definitions
- the present invention relates to an imaging device, a video processing device, and a video processing method.
- CMOS Complementary Metal Oxide Semiconductor
- High-resolution images can be displayed on a large-screen display or screen with a realistic and realistic image in which jaggies do not stand out.
- Broadcast cameras and cine cameras mainly use high-resolution images for the former purpose, and industrial cameras and medical cameras mainly use high-resolution images for the latter purpose.
- Still cameras use high-resolution video for both purposes.
- display elements also tend to have more pixels and higher densities due to the spread of OLEDs (Organic Light Emitting Diodes), but they are not as advanced as image sensors.
- OLEDs Organic Light Emitting Diodes
- the number of pixels of the display element is smaller than the number of pixels of the image sensor, it is necessary to compress and scale the image according to the number of pixels of the display element in order to confirm the image of the entire image pickup area. Alternatively, it is necessary to cut out a plurality of regions according to the number of pixels of the display element from the video and switch each region for confirmation.
- Such confirmation of the image of the entire imaging region is performed at the time of inspection of dust, scratches, and pixel defects in the production process of the imaging device, the optical system such as the lens and the filter, and the display device.
- the inspection of the image pickup device / optical system is performed by imaging a white subject or a light source, and the inspection of the display device is performed by imaging the display device of the inspection target displaying the solid white color.
- Patent Document 1 discloses a technique for detecting the adhesion of foreign matter by the brightness level.
- the image of the entire imaging region is collectively displayed without compressing and scaling the high-resolution image.
- the purpose is to provide a technology that can be confirmed.
- one of the typical image pickup devices of the present invention is an image pickup device that captures a subject image from an optical system, and a high-resolution image of the entire imaging region is divided into a plurality of regions. It has a video dividing unit that obtains a plurality of low-resolution images, and a video compositing unit that superimposes a plurality of low-resolution images to obtain a single low-resolution image.
- the entire image in the imaging region can be collectively confirmed without compressing and scaling the high-resolution image. can do.
- the block diagram which shows the structural example of the image pickup apparatus of embodiment of this invention The figure which shows an example of the operation which divides and synthesizes the image which performs the dust inspection in embodiment of this invention.
- the figure which shows an example of the operation which divides and synthesizes the image which performs the dust inspection in embodiment of this invention The figure which shows an example of the inspection in embodiment of this invention.
- the figure which shows an example of the inspection in embodiment of this invention. The figure which shows an example of the inspection in embodiment of this invention.
- FIG. 1 is a block diagram showing a configuration example of an imaging device according to an embodiment of the present invention.
- the image pickup device 101 includes a lens 102, an image pickup element 103, a video signal processing unit 104, a frame memory 105, a CPU (Central Processing Unit) 106, and a video signal output unit 107.
- the incident light from the subject is imaged by the lens 102 and photoelectrically converted into an electric signal by the image sensor 103.
- the video signal processing unit 104 performs various signal processing on the video signal, and the video signal output unit 107 outputs an HD-SDI (High Definition Serial Digital Interface) signal.
- HD-SDI High Definition Serial Digital Interface
- the video signal is divided by the video dividing unit 108 and combined by the video combining unit 109, and the gamma correction unit 110 performs gamma correction, knee correction, contour correction, color correction, etc.
- the video signal is processed and output to the video signal output unit 107.
- the video signal output unit 107 generates an HD-SDI signal from the input video signal and outputs it to the outside.
- the video signal output from the video signal output unit 107 is not limited to HD-SDI, and may or may not be compressed or encrypted.
- the CPU 106 controls each part of the image pickup apparatus 101.
- the video signal is input / output to the frame memory 105 as needed.
- the video dividing unit 108 divides the video into a plurality of videos having an arbitrary number of pixels.
- the arbitrary number of pixels is appropriately set according to the number of pixels of the display element for displaying the inspection image, but the user may arbitrarily set the number of pixels. If all the divided images are combined, the divided images are divided so that all the pixels of the undivided image can be restored.
- the video compositing unit 109 superimposes the images divided into each region by the video dividing unit 108 into one, and then synthesizes the pixel having the lowest brightness level among the pixels having the same coordinates of the video in each region. Select as pixels with the same coordinates. Pixels with a low brightness level are selected by utilizing the fact that dust generally blocks light, so that the brightness level of an image is lower than that of the peripheral portion.
- the video combining unit 109 can use the information.
- FIGS. 2A to 2D are examples in which the image to be inspected for dust is divided by the image dividing unit 108 into four images having a pixel count of 1/2 in the vertical and horizontal directions.
- the image 201 before division is an image in which four dusts are present in a white subject.
- the image 206 in the upper left area has dust on the grain (brightness level: medium) 202 and the linear dust (brightness level: low) 203
- the image 207 in the upper right area has dust on the grain.
- Brightness level: medium) 204 the image 208 in the lower left region has large dust (luminance level: low) 205, and the image 209 in the lower right region has no dust.
- the image compositing unit 109 superimposes these four images and selects the pixel having the lowest brightness level among the pixels having the same coordinates as the pixel having the same coordinates after compositing.
- the linear dust (brightness level: low) 203 present in the image 206 in the upper left region and the grain dust (brightness level: medium) present in the image 207 in the upper right region.
- the dust (brightness level: medium) 202 on the grain existing in the image 206 in the upper left region does not exist in the image 210 after composition because the dust (brightness) on the large grain existing in the image 208 in the lower left region does not exist.
- the composited image is an image in which the pixel having the lowest brightness level among the four images is selected for the image in the white area having a high brightness level, but the image in the white area without dust is dust. Since the image has a clearly higher brightness level than the image in which is present, it can be confirmed that there is no dust in any area of the image selected, and the purpose of the dust inspection can be fulfilled.
- the image of dust is displayed in different colors for each area, it will help to identify in which area the dust exists. Specifically, the hue of the image is set to a predetermined hue different from each other for each region. If the video dividing unit 108 adds information for identifying an area to the divided video, the video compositing unit 109 can process the hue of the divided video using the information. By reversing the brightness of the composited image, the dust image becomes brighter and the hue can be easily discriminated.
- the image pickup apparatus can also inspect scratches and pixel defects. Of course, it is also possible to output a normal video without dividing and synthesizing the video.
- 3A and 3B are diagrams showing an example of inspection in the embodiment of the present invention.
- a white subject or a light source 301 is imaged and visually confirmed on the monitor 304.
- a black subject is imaged or an image is taken in a light-shielded state, and the pixel having the highest brightness level is selected in the image compositing unit.
- the monitor 305 When inspecting the monitor 305, the monitor 305 displaying a solid white color is imaged and visually confirmed on the monitor 304. However, when inspecting a pixel defect that always displays the white level, the monitor 305 that displays a solid black color is imaged, and the pixel having the highest brightness level is selected in the image compositing unit.
- the process applied to the pixels having the same coordinates in the video of each region is not limited to the above-mentioned process of selecting the pixel having the lowest brightness level and the process of selecting the pixel having the highest brightness level.
- it may be a process of obtaining the average value or the median value of the brightness level.
- the hues of the images By setting the hues of the images to different predetermined hues for each region, it is possible to specify in which region dust, scratches, and pixel defects are present. In addition, by reversing the brightness of the combined image, it is possible to brighten the image of dust, scratches, and pixel defects that always output and display the black level, and darken the image of pixel defects that always output and display the white level. can.
- the video processing of the present invention is carried out by an imaging device
- a display device or a video processing device such as a personal computer
- a display device or a video processing device such as a personal computer has a video dividing unit and a video compositing unit.
- the present invention can also be expressed as a video processing method.
- the display device, the image processing device, and the image processing method also have the same effect as the image pickup device.
Abstract
Provided are techniques for enabling collective confirmation of a video of an imaging region as a whole without performing compressive scaling of a high-resolution video during inspection of an object to be inspected for dust, scratch, or pixel defect, even when a low-resolution display device is used. This imaging device comprises: an imaging element for capturing a subject image from an optical system; a video dividing unit for dividing a high-resolution video of an imaging region as a whole into a plurality of regions to obtain a plurality of low-resolution videos; and a video composing unit for obtaining a single low-resolution video by superposing the plurality of low-resolution videos. The video composing unit obtains a single low-resolution video composed of pixels obtained by performing a predetermined process on the pixels of each low-resolution video having the same coordinates when the plurality of low-resolution videos are superposed. The predetermined process is a process for selecting pixels having a lowest luminance level.
Description
本発明は、撮像装置、映像処理装置及び映像処理方法に関する。
The present invention relates to an imaging device, a video processing device, and a video processing method.
近年、CMOS(Complementary Metal Oxide Semiconductor)撮像素子の生産技術の向上によりフォトダイオードの多画素化・高密度化が進んでいる。フォトダイオードが多画素化・高密度化された撮像素子を搭載した撮像装置ではより高解像度の映像を得ることができる。
In recent years, the number of pixels and the density of photodiodes have been increasing due to improvements in the production technology of CMOS (Complementary Metal Oxide Semiconductor) image sensors. An image pickup device equipped with an image sensor in which a photodiode has a large number of pixels and a high density can obtain a higher resolution image.
高解像度の映像は、大画面のディスプレイやスクリーンに大写しにしてもジャギーが目立つことがないリアルで臨場感のある映像を表示することができる。また、被写体の細部まで撮像して拡大表示することにより、精細な映像で微小な傷や異物を確認することができる。主に放送用カメラやシネカメラでは、前者の目的で高解像度の映像が用いられており、主に産業用カメラや医療用カメラでは、後者の目的で高解像度の映像が用いられている。スチルカメラでは、両方の目的で高解像度の映像が用いられている。
High-resolution images can be displayed on a large-screen display or screen with a realistic and realistic image in which jaggies do not stand out. In addition, by capturing the details of the subject and enlarging the display, it is possible to confirm minute scratches and foreign substances in a fine image. Broadcast cameras and cine cameras mainly use high-resolution images for the former purpose, and industrial cameras and medical cameras mainly use high-resolution images for the latter purpose. Still cameras use high-resolution video for both purposes.
一方で、表示素子においてもOLED(Organic Light Emitting Diode)等の普及により多画素化・高密度化の傾向があるが、撮像素子ほどは進んでいない。これには、技術的な課題もあるが、人間の目が捉えることができる単位面積当たりの画素数には限度があり、その限度を超えてまで表示素子の多画素化・高密度化を進めても意義が乏しいという事情もある。
On the other hand, display elements also tend to have more pixels and higher densities due to the spread of OLEDs (Organic Light Emitting Diodes), but they are not as advanced as image sensors. Although there are technical issues with this, there is a limit to the number of pixels per unit area that can be perceived by the human eye, and we will continue to increase the number of pixels and density of display elements beyond that limit. However, there are also circumstances in which it has little significance.
表示素子の画素数が撮像素子の画素数よりも少ない場合、撮像領域全体の映像を確認するためには、表示素子の画素数に合わせて映像を圧縮スケーリングする必要がある。あるいは、表示素子の画素数に合わせた複数の領域を映像から切り出して各領域を切り替えて確認する必要がある。
When the number of pixels of the display element is smaller than the number of pixels of the image sensor, it is necessary to compress and scale the image according to the number of pixels of the display element in order to confirm the image of the entire image pickup area. Alternatively, it is necessary to cut out a plurality of regions according to the number of pixels of the display element from the video and switch each region for confirmation.
このような撮像領域全体の映像の確認は、撮像装置、レンズやフィルタ等の光学系及び表示装置の生産工程におけるゴミ・傷・画素欠陥の検査時に行われる。撮像装置・光学系の検査は、白一色の被写体又は光源を撮像することで行い、表示装置の検査は、白一色を表示した検査対象の表示装置を撮像することで行う。
Such confirmation of the image of the entire imaging region is performed at the time of inspection of dust, scratches, and pixel defects in the production process of the imaging device, the optical system such as the lens and the filter, and the display device. The inspection of the image pickup device / optical system is performed by imaging a white subject or a light source, and the inspection of the display device is performed by imaging the display device of the inspection target displaying the solid white color.
特許文献１には、異物の付着を輝度レベルによって検出する技術が開示されている。
Patent Document 1 discloses a technique for detecting the adhesion of foreign matter by the brightness level.
ゴミ・傷・画素欠陥の検査においては、映像を圧縮スケーリングしてしまうと、数画素スケールの微細なゴミ・傷・画素欠陥が消えてしまうおそれがある。また、各領域を切り替えて確認する場合には、領域の数だけ検査の作業量が増えて、生産コストが上昇してしまう。撮像装置と同等の高解像度の表示装置を用いた場合においても、表示素子の画素密度が高く画面が小さい場合には、微細なゴミ・傷・画素欠陥を目視確認で見逃してしまうおそれがある。画面が大きい場合には、目視確認する面積が増えて各領域を切り替えて確認するのと変わらない。
In the inspection of dust, scratches, and pixel defects, if the image is compressed and scaled, there is a risk that fine dust, scratches, and pixel defects on a scale of several pixels will disappear. Further, when each area is switched and confirmed, the amount of inspection work increases by the number of areas, and the production cost increases. Even when a display device having a high resolution equivalent to that of the image pickup device is used, if the pixel density of the display element is high and the screen is small, fine dust, scratches, and pixel defects may be overlooked by visual confirmation. When the screen is large, the area for visual confirmation increases, which is the same as switching each area for confirmation.
そこで、本発明は、検査対象のゴミ・傷・画素欠陥の検査において、低解像度の表示装置を用いた場合でも、高解像度の映像を圧縮スケーリングすることなく、撮像領域全体の映像を一括して確認することができる技術を提供することを目的とする。
Therefore, according to the present invention, even when a low-resolution display device is used in the inspection of dust, scratches, and pixel defects to be inspected, the image of the entire imaging region is collectively displayed without compressing and scaling the high-resolution image. The purpose is to provide a technology that can be confirmed.
上記の課題を解決するために、代表的な本発明の撮像装置の一つは、光学系からの被写体像を撮像する撮像素子と、撮像領域全体の高解像度映像を複数の領域に分割して複数の低解像度映像を得る映像分割部と、複数の低解像度映像を重ね合わせて一の低解像度映像を得る映像合成部と、を有する。
In order to solve the above problems, one of the typical image pickup devices of the present invention is an image pickup device that captures a subject image from an optical system, and a high-resolution image of the entire imaging region is divided into a plurality of regions. It has a video dividing unit that obtains a plurality of low-resolution images, and a video compositing unit that superimposes a plurality of low-resolution images to obtain a single low-resolution image.
本発明によれば、検査対象のゴミ・傷・画素欠陥の検査において低解像度の表示装置を用いた場合でも、高解像度の映像を圧縮スケーリングすることなく、撮像領域全体の映像を一括して確認することができる。
According to the present invention, even when a low-resolution display device is used in the inspection of dust, scratches, and pixel defects to be inspected, the entire image in the imaging region can be collectively confirmed without compressing and scaling the high-resolution image. can do.
上記した以外の課題、構成および効果は、以下の実施をするための形態における説明により明らかにされる。
Issues, configurations and effects other than those mentioned above will be clarified by the explanation in the form for carrying out the following.
以下、図面を参照して、本発明の実施形態について説明する。なお、この実施形態により本発明が限定されるものではない。
Hereinafter, embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to this embodiment.
図１は本発明の実施形態の撮像装置の構成例を示すブロック図である。
FIG. 1 is a block diagram showing a configuration example of an imaging device according to an embodiment of the present invention.
図１において、撮像装置１０１は、レンズ１０２、撮像素子１０３、映像信号処理部１０４、フレームメモリ１０５、CPU(Central Processing Unit)１０６、映像信号出力部１０７で構成されている。
In FIG. 1, the image pickup device 101 includes a lens 102, an image pickup element 103, a video signal processing unit 104, a frame memory 105, a CPU (Central Processing Unit) 106, and a video signal output unit 107.
被写体からの入射光はレンズ１０２で結像され、撮像素子１０３で電気信号に光電変換される。映像信号処理部１０４では映像信号に各種の信号処理が施され、映像信号出力部１０７からHD-SDI(High Definition Serial Digital Interface)信号が出力される。
The incident light from the subject is imaged by the lens 102 and photoelectrically converted into an electric signal by the image sensor 103. The video signal processing unit 104 performs various signal processing on the video signal, and the video signal output unit 107 outputs an HD-SDI (High Definition Serial Digital Interface) signal.
映像信号処理部１０４では、映像信号に、映像分割部１０８による分割処理、映像合成部１０９による合成処理が施されるほか、ガンマ補正部１１０にてガンマ補正、ニー補正、輪郭補正、色補正等の映像信号処理が施されて、映像信号出力部１０７に出力される。
In the video signal processing unit 104, the video signal is divided by the video dividing unit 108 and combined by the video combining unit 109, and the gamma correction unit 110 performs gamma correction, knee correction, contour correction, color correction, etc. The video signal is processed and output to the video signal output unit 107.
映像信号出力部１０７は、入力した映像信号からHD-SDI信号を生成して、外部に出力する。なお、映像信号出力部１０７から出力する映像信号は、HD-SDIに限定されるものではなく、圧縮や暗号化等の有無も問わない。CPU１０６は撮像装置１０１の各部を制御する。
The video signal output unit 107 generates an HD-SDI signal from the input video signal and outputs it to the outside. The video signal output from the video signal output unit 107 is not limited to HD-SDI, and may or may not be compressed or encrypted. The CPU 106 controls each part of the image pickup apparatus 101.
映像分割部１０８による分割処理及び映像合成部１０９による合成処理では、必要に応じてフレームメモリ１０５に映像信号を入出力する。
In the division processing by the video division unit 108 and the composition processing by the video composition unit 109, the video signal is input / output to the frame memory 105 as needed.
映像分割部１０８は、映像を任意の画素数の複数の映像に分割する。ここで、任意の画素数は、検査映像を表示させる表示素子の画素数に合わせたものが適切であるが、ユーザーが任意に設定しても良い。分割後の映像を全て組み合わせれば、分割前の映像の全ての画素を復元できるように分割を行う。
The video dividing unit 108 divides the video into a plurality of videos having an arbitrary number of pixels. Here, the arbitrary number of pixels is appropriately set according to the number of pixels of the display element for displaying the inspection image, but the user may arbitrarily set the number of pixels. If all the divided images are combined, the divided images are divided so that all the pixels of the undivided image can be restored.
映像合成部１０９は、映像分割部１０８によって各領域に分割された映像を一に重ね合わせた上で、各領域の映像の同一の座標となる画素のうち最も輝度レベルが低い画素を合成後の同座標の画素として選択する。輝度レベルが低い画素を選択するのは、一般的にゴミは光を遮るため、周辺部と比較して映像の輝度レベルが低下して表示されることを利用したものである。
The video compositing unit 109 superimposes the images divided into each region by the video dividing unit 108 into one, and then synthesizes the pixel having the lowest brightness level among the pixels having the same coordinates of the video in each region. Select as pixels with the same coordinates. Pixels with a low brightness level are selected by utilizing the fact that dust generally blocks light, so that the brightness level of an image is lower than that of the peripheral portion.
映像分割部１０８が分割後の映像に領域を識別する情報を付加すれば、当該情報を映像合成部１０９が利用できる。
If the video dividing unit 108 adds information for identifying an area to the divided video, the video combining unit 109 can use the information.
図２Ａないし図２Ｄは、ゴミ検査を行う映像を映像分割部１０８で分割前の映像２０１を縦横１／２の画素数の４つの映像に分割した例である。分割前の映像２０１は白一色の被写体中にゴミが４つ存在する映像である。４つの映像に分割すると、左上領域の映像２０６には粒上のゴミ（輝度レベル：中）２０２と線状のゴミ（輝度レベル：低）２０３が、右上領域の映像２０７には粒上のゴミ（輝度レベル：中）２０４が、左下領域の映像２０８には大きい粒上のゴミ（輝度レベル：低）２０５が存在し、右下領域の映像２０９にはゴミが存在しない映像となる。映像合成部１０９は、これら４つの映像を重ね合わせ、同一の座標となる画素のうち最も輝度レベルが低い画素を合成後の同座標の画素として選択する。その結果、合成後の映像２１０には、左上領域の映像２０６に存在した線状のゴミ（輝度レベル：低）２０３と、右上領域の映像２０７に存在した粒上のゴミ（輝度レベル：中）２０４と、左下領域の映像２０８に存在した大きい粒上のゴミ（輝度レベル：低）２０５が存在し、分割前の映像２０１の縦横１／２の画素数の低解像度映像が得られる。
FIGS. 2A to 2D are examples in which the image to be inspected for dust is divided by the image dividing unit 108 into four images having a pixel count of 1/2 in the vertical and horizontal directions. The image 201 before division is an image in which four dusts are present in a white subject. When divided into four images, the image 206 in the upper left area has dust on the grain (brightness level: medium) 202 and the linear dust (brightness level: low) 203, and the image 207 in the upper right area has dust on the grain. (Brightness level: medium) 204, the image 208 in the lower left region has large dust (luminance level: low) 205, and the image 209 in the lower right region has no dust. The image compositing unit 109 superimposes these four images and selects the pixel having the lowest brightness level among the pixels having the same coordinates as the pixel having the same coordinates after compositing. As a result, in the combined image 210, the linear dust (brightness level: low) 203 present in the image 206 in the upper left region and the grain dust (brightness level: medium) present in the image 207 in the upper right region. There are 204 and large dust (luminance level: low) 205 present in the image 208 in the lower left region, and a low-resolution image having 1/2 the number of pixels in the vertical and horizontal directions of the image 201 before division can be obtained.
ここで、左上領域の映像２０６に存在した粒上のゴミ（輝度レベル：中）２０２が合成後の映像２１０中に存在しないのは、左下領域の映像２０８に存在した大きい粒上のゴミ（輝度レベル：低）２０５と座標が重なり、より輝度レベルの低い左下領域のゴミ２０５の映像に隠れたためである。これは、ゴミの個数を検査する目的としては不向きであるが、一般的にゴミの検査は、ゴミの大きさや、ゴミによる光透過への影響すなわち輝度レベルの低下を検査の判定基準としているため、輝度レベルが高い小さなゴミの映像が、輝度レベルが低い大きなゴミの映像に隠れたとしても、ゴミ検査の目的は果たすことができる。
Here, the dust (brightness level: medium) 202 on the grain existing in the image 206 in the upper left region does not exist in the image 210 after composition because the dust (brightness) on the large grain existing in the image 208 in the lower left region does not exist. This is because the coordinates overlap with 205 (level: low) and are hidden in the image of dust 205 in the lower left region where the brightness level is lower. This is not suitable for the purpose of inspecting the number of dust, but in general, dust inspection is based on the size of dust and the influence of dust on light transmission, that is, the decrease in brightness level. Even if the image of small dust with a high brightness level is hidden by the image of a large dust with a low brightness level, the purpose of the dust inspection can be achieved.
尚、合成後の映像は、輝度レベルが高い白領域の映像についても、４つの映像のうち最も輝度レベルが低い画素が選択された映像であるが、ゴミが存在しない白領域の映像は、ゴミが存在する映像と比べて明らかに輝度レベルが高い映像であるため、どの領域の映像が選択されたとしても、そこにゴミがないということを確認でき、ゴミ検査の目的は果たすことができる。
The composited image is an image in which the pixel having the lowest brightness level among the four images is selected for the image in the white area having a high brightness level, but the image in the white area without dust is dust. Since the image has a clearly higher brightness level than the image in which is present, it can be confirmed that there is no dust in any area of the image selected, and the purpose of the dust inspection can be fulfilled.
合成後の映像にゴミが存在した場合、そのゴミが分割前の映像のどの領域に存在するかは特定できない。しかしながら、一般的にゴミの検査において、検査の判定基準を満たさないゴミが１つでも存在した場合には、清掃工程に戻す等の処置がとられるため、ゴミがどの領域に存在するかを特定する必要性は低い。
If dust is present in the composited video, it is not possible to specify in which area of the video before splitting the dust is present. However, in general, in the dust inspection, if there is even one dust that does not meet the inspection criteria, measures such as returning to the cleaning process are taken, so it is possible to identify the area where the dust exists. There is little need to do it.
応用例として、領域毎にゴミの映像を色分け表示すれば、ゴミがどの領域に存在するかを特定する一助となる。具体的には、領域毎に映像の色相を互いに異なる所定の色相とする。映像分割部１０８が分割後の映像に領域を識別する情報を付加していれば、当該情報を利用して映像合成部１０９が分割後の映像の色相を処理できる。合成後の映像の明暗を反転すれば、ゴミの映像が明るくなり、色相を判別しやすい。
As an application example, if the image of dust is displayed in different colors for each area, it will help to identify in which area the dust exists. Specifically, the hue of the image is set to a predetermined hue different from each other for each region. If the video dividing unit 108 adds information for identifying an area to the divided video, the video compositing unit 109 can process the hue of the divided video using the information. By reversing the brightness of the composited image, the dust image becomes brighter and the hue can be easily discriminated.
なお、本発明の実施形態の撮像装置において、傷・画素欠陥を検査することもできる。また、映像の分割及び合成を行わない通常の映像を出力することも当然可能である。
It should be noted that the image pickup apparatus according to the embodiment of the present invention can also inspect scratches and pixel defects. Of course, it is also possible to output a normal video without dividing and synthesizing the video.
図３Ａ及び図３Ｂは、本発明の実施形態における検査の一例を示す図である。光学系３０２又は撮像装置３０３を検査する場合、白一色の被写体又は光源３０１を撮像して、モニタ３０４で目視確認する。ただし、常に白レベルを出力する画素欠陥を検査する場合は、黒一色の被写体を撮像するか、遮光した状態で撮像し、映像合成部で最も輝度レベルが高い画素を選択する。
3A and 3B are diagrams showing an example of inspection in the embodiment of the present invention. When inspecting the optical system 302 or the image pickup apparatus 303, a white subject or a light source 301 is imaged and visually confirmed on the monitor 304. However, when inspecting a pixel defect that always outputs a white level, a black subject is imaged or an image is taken in a light-shielded state, and the pixel having the highest brightness level is selected in the image compositing unit.
モニタ３０５を検査する場合、白一色を表示したモニタ３０５を撮像して、モニタ３０４で目視確認する。ただし、常に白レベルを表示する画素欠陥を検査する場合は、黒一色を表示したモニタ３０５を撮像し、映像合成部で最も輝度レベルが高い画素を選択する。
When inspecting the monitor 305, the monitor 305 displaying a solid white color is imaged and visually confirmed on the monitor 304. However, when inspecting a pixel defect that always displays the white level, the monitor 305 that displays a solid black color is imaged, and the pixel having the highest brightness level is selected in the image compositing unit.
以上、本発明の実施の形態について説明したが、本発明は、上述した実施の形態に限定されるものではなく、本発明の要旨を逸脱しない範囲において種々の変更が可能である。
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention.
各領域の映像の同一の座標となる画素に施す処理は、上述の最も輝度レベルが低い画素を選択する処理及び最も輝度レベルが高い画素を選択する処理に限らない。例えば、輝度レベルの平均値や中央値を求める処理でもよい。
The process applied to the pixels having the same coordinates in the video of each region is not limited to the above-mentioned process of selecting the pixel having the lowest brightness level and the process of selecting the pixel having the highest brightness level. For example, it may be a process of obtaining the average value or the median value of the brightness level.
各領域の映像の同一の座標となる画素に所定の処理を施すことにより、各領域の映像の同一の座標におけるゴミ・傷・画素欠陥の有無を一括して確認することができる。最も輝度レベルが低い画素を選択する処理により、ゴミ・傷・常に黒レベルを出力・表示する画素欠陥の有無を確認することができる。また、最も輝度レベルが高い画素を選択する処理により、常に白レベルを出力・表示する画素欠陥の有無を確認することができる。
By performing a predetermined process on pixels having the same coordinates of the image of each area, it is possible to collectively check the presence or absence of dust, scratches, and pixel defects at the same coordinates of the image of each area. By selecting the pixel with the lowest brightness level, it is possible to confirm the presence or absence of dust, scratches, and pixel defects that constantly output and display the black level. Further, by the process of selecting the pixel having the highest luminance level, it is possible to confirm the presence or absence of a pixel defect that always outputs and displays the white level.
領域毎に映像の色相を互いに異なる所定の色相とすることにより、ゴミ・傷・画素欠陥がどの領域に存在するかを特定することができる。また、合成後の映像の明暗を反転することにより、ゴミ・傷・常に黒レベルを出力・表示する画素欠陥の映像を明るく、常に白レベルを出力・表示する画素欠陥の映像を暗くすることができる。
By setting the hues of the images to different predetermined hues for each region, it is possible to specify in which region dust, scratches, and pixel defects are present. In addition, by reversing the brightness of the combined image, it is possible to brighten the image of dust, scratches, and pixel defects that always output and display the black level, and darken the image of pixel defects that always output and display the white level. can.
上述の実施形態では、本発明の映像処理を撮像装置で実施する例を記述したが、本発明の映像処理を表示装置や、パソコン等の映像処理装置で実施することも可能である。すなわち、表示装置や、パソコン等の映像処理装置が映像分割部及び映像合成部を有することも可能である。また、本発明は、映像処理方法としても表現し得る。表示装置、映像処理装置及び映像処理方法においても、撮像装置と同等の効果を奏する。
In the above-described embodiment, an example in which the video processing of the present invention is carried out by an imaging device has been described, but it is also possible to carry out the video processing of the present invention by a display device or a video processing device such as a personal computer. That is, it is also possible that a display device or a video processing device such as a personal computer has a video dividing unit and a video compositing unit. The present invention can also be expressed as a video processing method. The display device, the image processing device, and the image processing method also have the same effect as the image pickup device.
１０１…撮像装置、１０２…レンズ、１０３…撮像素子、１０４…映像信号処理部、１０５…フレームメモリ、１０６…CPU、１０７…映像信号出力部、１０８…映像分割部、１０９…映像合成部、１１０…ガンマ補正部、２０１…分割前の映像、２０２ないし２０５…ゴミ、２０６ないし２０９…分割後の映像、２１０…合成後の映像、３０１…被写体又は光源、３０２…光学系、３０３…撮像装置、３０４、３０５…モニタ。
101 ... Imaging device, 102 ... Lens, 103 ... Imaging element, 104 ... Video signal processing unit, 105 ... Frame memory, 106 ... CPU, 107 ... Video signal output unit, 108 ... Video dividing unit, 109 ... Video compositing unit, 110 ... Gamma correction unit, 201 ... image before division, 202 to 205 ... dust, 206 to 209 ... image after division, 210 ... image after composition, 301 ... subject or light source, 302 ... optical system, 303 ... image pickup device, 304, 305 ... Monitor.
Claims (8)
- 光学系からの被写体像を撮像する撮像素子と、
撮像領域全体の高解像度映像を複数の領域に分割して複数の低解像度映像を得る映像分割部と、
前記複数の低解像度映像を重ね合わせて一の低解像度映像を得る映像合成部と、
を有する撮像装置。 An image sensor that captures the subject image from the optical system,
An image division unit that divides the high-resolution image of the entire imaging area into a plurality of areas to obtain a plurality of low-resolution images.
A video compositing unit that obtains one low-resolution video by superimposing the plurality of low-resolution videos.
An imaging device having. - 請求項１に記載の撮像装置であって、
前記映像合成部は、前記複数の低解像度映像を重ね合わせた場合に同一の座標となる各低解像度映像の画素に所定の処理を施して得た画素により構成される一の低解像度映像を得る、
撮像装置。 The imaging device according to claim 1.
The image synthesizing unit obtains one low-resolution image composed of pixels obtained by performing predetermined processing on the pixels of each low-resolution image having the same coordinates when the plurality of low-resolution images are superimposed. ,
Imaging device. - 請求項２に記載の撮像装置であって、
前記所定の処理は、最も輝度レベルが低い画素を選択する処理である、
撮像装置。 The imaging device according to claim 2.
The predetermined process is a process of selecting a pixel having the lowest luminance level.
Imaging device. - 請求項２に記載の撮像装置であって、
前記所定の処理は、最も輝度レベルが高い画素を選択する処理である、
撮像装置。 The imaging device according to claim 2.
The predetermined process is a process of selecting a pixel having the highest luminance level.
Imaging device. - 請求項３に記載の撮像装置であって、
前記映像合成部は、前記複数の低解像度映像の色相を互いに異なる所定の色相とする、
撮像装置。 The imaging device according to claim 3.
The image compositing unit sets the hues of the plurality of low-resolution images to predetermined hues different from each other.
Imaging device. - 請求項１に記載の撮像装置であって、
前記映像合成部は、前記一の低解像度映像の明暗を反転する、
撮像装置。 The imaging device according to claim 1.
The video compositing unit reverses the brightness of the one low-resolution video.
Imaging device. - 高解像度映像を複数の領域に分割して複数の低解像度映像を得る映像分割部と、
前記複数の低解像度映像を重ね合わせて一の低解像度映像を得る映像合成部と、
を有する映像処理装置。 A video divider that divides a high-resolution video into multiple areas to obtain multiple low-resolution videos,
A video compositing unit that obtains one low-resolution video by superimposing the plurality of low-resolution videos.
Video processing device with. - 高解像度映像を複数の領域に分割して複数の低解像度映像を得る映像分割ステップと、
前記複数の低解像度映像を重ね合わせて一の低解像度映像を得る映像合成ステップと、
を有する映像処理方法。 A video division step that divides a high-resolution video into multiple areas to obtain multiple low-resolution videos,
A video compositing step of superimposing a plurality of low-resolution images to obtain one low-resolution image,
Video processing method having.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2022509909A JP7441304B2 (en) | 2020-03-24 | 2021-03-11 | Imaging device, video processing device, and video processing method |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2020051972 | 2020-03-24 | ||
| JP2020-051972 | 2020-03-24 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2021193102A1 true WO2021193102A1 (en) | 2021-09-30 |
Family
ID=77891992
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2021/009792 WO2021193102A1 (en) | 2020-03-24 | 2021-03-11 | Imaging device, video processing device, and video processing method |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP7441304B2 (en) |
| WO (1) | WO2021193102A1 (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009093280A (en) * | 2007-10-04 | 2009-04-30 | Fuji Xerox Co Ltd | Image processor and collation system |
| JP2019102929A (en) * | 2017-11-30 | 2019-06-24 | パナソニックＩｐマネジメント株式会社 | Video processing system, video processing apparatus, and video processing method |
-
2021
- 2021-03-11 WO PCT/JP2021/009792 patent/WO2021193102A1/en active Application Filing
- 2021-03-11 JP JP2022509909A patent/JP7441304B2/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009093280A (en) * | 2007-10-04 | 2009-04-30 | Fuji Xerox Co Ltd | Image processor and collation system |
| JP2019102929A (en) * | 2017-11-30 | 2019-06-24 | パナソニックＩｐマネジメント株式会社 | Video processing system, video processing apparatus, and video processing method |
Also Published As
| Publication number | Publication date |
|---|---|
| JPWO2021193102A1 (en) | 2021-09-30 |
| JP7441304B2 (en) | 2024-02-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5566133B2 (en) | Frame rate conversion processor | |
| JP2010093472A (en) | Imaging apparatus, and signal processing circuit for the same | |
| US11582401B2 (en) | Image signal processing device, imaging device, flicker check method in imaging device, and server | |
| JP2009085617A (en) | Defect detection device and method | |
| JP2005244980A (en) | Apparatus and method for processing image signal | |
| JP5708036B2 (en) | Imaging device | |
| WO2010089836A1 (en) | Image processing device | |
| TW201010429A (en) | Image processing system and method thereof | |
| JP2017050720A (en) | Information embedding device, information embedding method, and information embedding program | |
| CN110809885A (en) | Image sensor defect detection | |
| JP2015185943A (en) | Microlens with filter array and solid state image sensor | |
| US20190051270A1 (en) | Display processing device and imaging device | |
| WO2021193102A1 (en) | Imaging device, video processing device, and video processing method | |
| JP2006153914A (en) | Liquid crystal projector | |
| JP2008170288A (en) | Device for inspecting flaw, and method for inspecting flaw | |
| US9723283B2 (en) | Image processing device, image processing system, and image processing method | |
| JP2005173879A (en) | Fused image display device | |
| JP2012089918A (en) | Imaging device | |
| JP5299159B2 (en) | Imaging apparatus and program | |
| JP5387341B2 (en) | Imaging device | |
| JP2019140468A (en) | Image processing device and control method of same | |
| JP2011151454A (en) | Image processor, imaging apparatus and image processing method | |
| JP2016096551A (en) | Imaging device | |
| JP5444720B2 (en) | projector | |
| JP6695700B2 (en) | Image processing device and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 21775390 Country of ref document: EP Kind code of ref document: A1 |
|
| ENP | Entry into the national phase |
Ref document number: 2022509909 Country of ref document: JP Kind code of ref document: A |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 21775390 Country of ref document: EP Kind code of ref document: A1 |