JP2007201549A - Image display apparatus and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display apparatus for displaying an image obtained by an infrared ray camera with a simple processing in a way of providing excellent visibility. <P>SOLUTION: The image display apparatus 100 disclosed herein includes: a contour extract section for extracting a contour from the image photographed by the infrared ray camera 101; an emphasis processing section for coloring the contour extracted by the contour extract section to emphasize the contour; and a liquid crystal display panel 105 for composing and displaying the contour emphasized by the contour extract section with the image photographed by the infrared ray camera 101. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image display device and an image processing method, and more particularly to an image display device and an image processing method for displaying an image taken by an infrared camera.

  When driving a car at night, the driver relies on headlights, street lights, etc. in a low-light environment. For this reason, visibility is poor compared to daytime driving, discovery of pedestrians and obstacles is delayed, and the risk of accidents increases. Therefore, there is an increasing need for information for assisting the forward field of view in order to drive the vehicle safely, and an image HMI (Human Machine Interface) is required for more effectively transmitting this to the driver.

As a technique for providing such forward vision auxiliary information, a night vision system that captures the front of an automobile using an infrared camera and displays the obtained image has been proposed (see Non-Patent Document 1). In Non-Patent Document 1, the front of the automobile is photographed by two far-infrared cameras installed in the automobile, and the obtained image is displayed on a head-up display provided on the instrument panel. In addition, the position of the pedestrian is detected, and an emphasis frame surrounding the pedestrian is displayed in the image by the infrared camera to alert the driver.
"Honda" Intelligent Night Vision System "newly developed", [online], August 24, 2004, Honda Motor Co., Ltd., [December 28, 2005 search], Internet <URL: http: // www.honda.co.jp/news/2004/4040824a.html>

  In the invention described in Non-Patent Document 1, whether or not a person is a pedestrian is determined based on the shape, size, and the like from an image taken by an infrared camera. Such processing requires a lot of calculations and is complicated. Further, since two infrared cameras are required, there is a problem that the image display device itself becomes expensive.

  The present invention has been made in the background of such circumstances, and an object of the present invention is to provide an image display device and an image processing method for displaying an image obtained by an infrared camera with high visibility by simple processing. It is.

  An image display device according to a first aspect of the present invention includes a contour extracting unit that extracts a contour from an original image captured by an infrared camera, and an emphasis that highlights the contour by coloring the contour extracted by the contour extracting unit. It is a person having a processing unit and a display unit that displays the outline enhanced by the enhancement processing unit and an image photographed by the infrared camera. With such a configuration, it is possible to display an image from the infrared camera with high visibility by a simple process.

  The image display device according to a second aspect of the present invention is the image display device described above, further comprising a contrast adjustment unit that adjusts a contrast of an original image captured by the infrared camera and generates a contrast-adjusted image. The extraction unit extracts a contour from the contrast adjustment image. With such a configuration, it is possible to display an image from the infrared camera with high visibility by a simple process.

  An image display device according to a third aspect of the present invention is the above image display device, wherein a contrast adjustment unit that adjusts a contrast of an original image taken by the infrared camera and generates a contrast adjustment image, and the contrast adjustment image A brightness adjustment unit that adjusts the brightness of the image, and the display unit displays the contrast-adjusted image with the adjusted brightness and the outline in a synthesized manner. With such a configuration, it is possible to display an image from the infrared camera with higher visibility by a simple process.

  An image display device according to a fourth aspect of the present invention is the image display device described above, further including a binarization processing unit that binarizes the image from which the contour is extracted by the contour extraction unit, and the enhancement processing. The section is for coloring the outline of the image subjected to the binarization process. With such a configuration, it is possible to display an image from the infrared camera with higher visibility by a simple process.

  In the image display device according to the fifth aspect of the present invention, in the above image display device, the binarization processing unit performs binarization processing for each of a plurality of adjacent pixels in the image from which the contour is extracted. It is. By setting it as such a structure, the influence of the noise in image data can be suppressed, and the image by an infrared camera can be displayed with sufficient visibility.

  In the image display device according to a sixth aspect of the present invention, in the above image display device, the enhancement processing unit selects a color having a high color contrast with respect to the image combined with the contour, and the contour is selected. It will be colored. With such a configuration, an image from an infrared camera can be displayed with high visibility.

  An image processing method according to a seventh aspect of the present invention is the image processing method described above, wherein an original image is captured by an infrared camera, a contour is extracted from the captured original image, and the extracted contour is enhanced. The outline and the image taken by the infrared camera are combined and displayed. Thereby, it is possible to easily display an image by the infrared camera with good visibility.

  An image processing method according to an eighth aspect of the present invention is the image processing method described above, wherein a contrast-adjusted image is generated by adjusting a contrast of an original image captured by the infrared camera, and an outline is extracted from the contrast-adjusted image. To do. Thereby, it is possible to easily display an image from the infrared camera with higher visibility.

  An image processing method according to a ninth aspect of the present invention is the image processing method described above, wherein the contrast adjustment image is generated by adjusting the contrast of the original image captured by the infrared camera, and the brightness of the contrast adjustment image is adjusted. The contrast-adjusted image that has been adjusted and the brightness is adjusted and the contour are combined and displayed. Thereby, it is possible to easily display an image from the infrared camera with high visibility.

  An image processing method according to a tenth aspect of the present invention is the image processing method described above, wherein the image obtained by extracting the contour from the original image is binarized and the contour of the image subjected to the binarization processing is obtained. Color. Thereby, it is possible to easily display an image from the infrared camera with high visibility.

  In the image processing method according to an eleventh aspect of the present invention, in the above image processing method, the binarization processing binarizes each of a plurality of adjacent pixels in the image from which the contour is extracted. Thereby, the influence of the noise in image data can be suppressed and the image by an infrared camera can be displayed with sufficient visibility.

  The image processing method according to a twelfth aspect of the present invention is the image processing method according to the above image processing method, wherein the enhancement processing selects a color having a high color contrast with respect to the image combined with the contour, and colors the contour. To do. Thereby, it is possible to display the image by the infrared camera with better visibility.

  According to the present invention, it is possible to provide an image display device and an image processing method for displaying an image obtained by an infrared camera with high visibility by a simple process.

  Embodiments to which the present invention can be applied will be described below with reference to the drawings. The following description is to describe the embodiment of the present invention, and the present invention is not limited to the following embodiment. Moreover, those skilled in the art can easily change, add, and convert each element of the following embodiments within the scope of the present invention.

  An image display apparatus according to an embodiment of the present invention will be described with reference to FIG. Here, as an image display device according to the present embodiment, an in-vehicle TFT liquid crystal display device that displays auxiliary information on the front vision of the vehicle will be described as an example. FIG. 1 is a block diagram showing a configuration of an image display apparatus 100 according to the present embodiment. As shown in FIG. 1, the image display apparatus 100 includes an infrared camera 101, an A / D converter 102, an image processing unit 103, a timing controller 104, a liquid crystal display panel 105, and the like. The image display apparatus 100 according to the present invention displays an original image captured by the infrared camera 101 on the liquid crystal display panel 105 with high visibility by performing image processing on the image processing unit 103. A point to be noted in the present invention is the image processing unit 103, which will be described in detail later.

  The infrared camera 101 is attached toward the traveling direction (front) of the automobile. The infrared camera 101 detects an infrared ray radiated from an object, captures a scene in front of the automobile, and outputs video data of the captured original image. Here, all objects emit infrared rays having an intensity that is uniquely related to the temperature of the object itself. For this reason, the higher the temperature of the object photographed by the infrared camera 101, the higher the gradation of video data. Therefore, for example, an object that generates heat, such as a pedestrian, has relatively high gradation video data, and an oncoming vehicle has relatively low gradation video data.

  However, since such an infrared camera 101 captures an object at a substantially normal temperature based on thermal radiation (thermal radiation) having a smaller energy than natural light or irradiation light, the obtained signal level is low and the dynamic range is narrow. . For this reason, a clear image cannot be obtained as an original image obtained by the infrared camera 101. Therefore, in the present invention, the image processing unit 103 performs image processing on the original image, thereby obtaining a clearer image with better visibility.

  The A / D converter 102 converts analog video data output from the infrared camera 101 into digital image data. The image processing unit 103 performs image processing on the input digital image data and outputs display data. Note that the image processing unit 103 can be provided on the input side of the A / D converter 102. The image processing unit 103 and the image processing method will be described in detail later.

  The timing controller 104 is electrically connected to the liquid crystal display panel 105, and outputs various control signals and display data for display on the liquid crystal display panel 105 to the liquid crystal display panel 105.

  The liquid crystal display panel 105 displays an image based on display data output from the image processing unit 103. The liquid crystal display panel 105 has a display area composed of a plurality of pixels, and a liquid crystal is provided between a TFT (Thin Film Transistor) array substrate (not shown) and a counter substrate (not shown) arranged to face each other. It has a sandwiched configuration. On the TFT array substrate, gate lines (scanning lines) are formed in the horizontal direction, and source lines (signal lines) are formed in the vertical direction, and TFTs are provided near the intersections of the gate lines and the source lines. In addition, a plurality of pixel electrodes are formed in a matrix between the gate line and the source line. On the other hand, a common electrode and R (red), G (green) and B (blue) color filters are formed on the counter substrate.

  The liquid crystal display panel 105 is provided with a gate driver 106 and a source driver 107. The gate driver 106 and the source driver 107 drive the liquid crystal display panel 105 based on various control signals input from the timing controller 104 to display an image.

  The gate driver 106 and the source driver 107 are directly mounted on the surface of the TFT array substrate using COG (Chip On Glass) technology. It should be noted that the present invention is not limited to the COG technology, and a configuration in which each driver and the liquid crystal display panel are connected using other technology may be employed. For example, a TAB (Tape Automated Bonding) technique may be used to electrically connect a TCP (Tape Carrier Package) in which an IC chip is bonded on an FPC (Flexible Printed Circuit) to the liquid crystal display panel.

  In this example, the timing controller 104 and the image processing unit 103 are separately provided. However, the timing controller 104 and the image processing unit 103 may be configured as one chip. Furthermore, the gate driver 106 and the source driver 106 can also be configured as a single chip.

  Here, a driving method of the liquid crystal display panel 105 will be described. First, a voltage is supplied from an external power source to the gate driver 106, the source driver 107, and each logic circuit via a DC / DC converter. The analog video data of the original image taken by the infrared camera 101 is transmitted to the A / D converter 102 and converted into digital image data. The converted image data is input to the image processing unit 103, subjected to image processing, and output to the timing controller 104 as display data.

  For example, the timing controller 104 receives a synchronization signal including a dot clock signal, which is an input cycle of display data for one pixel, a horizontal synchronization signal Hsync, a vertical synchronization signal Vsync, and the like. The timing controller 104 processes various received signals and outputs various control signals to be supplied to the gate driver 106 and the source driver 107 at a necessary timing.

  A start pulse signal, a clock signal, an enable signal, and the like are input from the timing controller 104 to the gate driver 106. The gate signal from which the start pulse signal outputs the ON signal is selected, and the enable signal controls the output of the scanning signal, whereby the ON signal is sequentially output from each gate line. Normally, the gate driver 106 outputs a scanning signal so as to sequentially scan pixels in each row from the first row to the subsequent row.

  On the other hand, the source driver 107 supplies a display signal corresponding to the display data to each source line of the liquid crystal display panel 105 according to the timing instructed from the timing controller 104. A display signal input from the source driver 107 is supplied to the pixel electrode via the source / drain of the TFT, and a voltage corresponding to the gradation is applied to the pixel electrode. Then, an electric field is applied to the liquid crystal between the pixel electrode and the counter electrode. Thus, when each switching element is driven and the amount of charge to be stored is controlled according to the gradation, the alignment state of the liquid crystal changes for each pixel. Thus, the amount of light transmitted from a backlight (not shown) can be controlled, and the brightness can be changed for each pixel. Each pixel of the liquid crystal display panel 105 displays an image by color gradation according to the amount of transmitted light and RGB color display.

  Here, the configuration of the image processing unit 103 according to the present embodiment will be described with reference to FIG. FIG. 2 is a block diagram showing an example of the configuration of the image processing unit 103 according to the present embodiment. As illustrated in FIG. 2, the image processing unit 103 includes a contrast adjustment unit 108, a brightness adjustment unit 109, a contour extraction unit 110, a binarization processing unit 111, and a synthesis unit 112.

  A brightness adjustment unit 109 is provided on the output side of the contrast adjustment unit 108. Further, an outline extraction unit 110 is provided on the output side of the contrast adjustment unit 108. Therefore, the output from the contrast adjustment unit 108 is input to the brightness adjustment unit 109 and the contour extraction unit 110. On the output side of the contour extraction unit 110, a binarization processing unit 111 is provided. A synthesis unit 112 is provided on the output side of the brightness adjustment unit 109 and the binarization processing unit 111. Therefore, the synthesizing unit 112 receives the contrast-adjusted image adjusted in brightness generated by the brightness adjusting unit 109 and the image data of the contour that has been binarized and colored.

  Here, an image processing method in the image processing unit 103 will be described with reference to FIG. FIG. 3 is a flowchart for explaining the image processing method according to this embodiment. As shown in FIG. 3, first, image data is input from the A / D converter 102 to the contrast adjustment unit 108 (step S1). At this time, as described above, since the video data of the captured original image is obtained from infrared rays having an intensity that is uniquely related to the temperature of the object itself, the higher the temperature of the object, the higher the gradation level. It has become. For this reason, for example, pedestrians that generate heat have high gradation data, and oncoming vehicles that do not generate heat have low gradation data. However, since the video data of the original image obtained by the infrared camera 101 has a low intensity and a narrow dynamic range, there is little difference in gradation between the image data of the pedestrian and the oncoming vehicle.

  Therefore, the contrast adjustment unit 108 adjusts the contrast of the input image to generate a contrast adjusted image (step S2). For example, by expanding the range of image data of a blurred original image with a small brightness difference in 127-255 gradations to 0-255 gradations, the contrast of the image is made clear by making the image clearer and darker. Can be generated. Therefore, the difference between the pedestrian image data having a relatively high gradation and the image data gradation of the oncoming vehicle having a relatively low gradation in the input image data of the original image can be increased by adjusting the contrast. it can. Thereby, in the contour extraction unit 110 described later, a clearer pedestrian contour can be extracted, and an image can be displayed with higher visibility. Then, the contrast adjustment unit 108 outputs the image data of the contrast adjustment image to the brightness adjustment unit 109 and the contour extraction unit 110.

  Then, the brightness adjustment unit 109 adjusts the brightness of the contrast-adjusted image (Step S3). For example, the brightness can be adjusted by adding or subtracting certain gradation data to the original image data of all input pixels. Although not shown here, image data output from the brightness adjustment unit 109 and a binarization processing unit 111 (to be described later) is combined by the combining unit 112 (to be described later). A buffer for holding the image data of the contrast-adjusted image whose height is adjusted is provided.

  On the other hand, the contour extracting unit 110 extracts the contour of the subject from the contrast adjusted image whose contrast has been adjusted by the contrast adjusting unit 108 (step S4). For example, by using a 3 × 3 matrix contour extraction filter 113 as shown in FIG. 4, the contour can be extracted from the contrast adjusted image. Therefore, in this case, although not shown in FIG. 3, the contour extracting unit 110 is provided with a three-line buffer.

  Since the contour extraction unit 110 extracts the contour from the contrast adjustment image whose contrast has been adjusted by the contrast adjustment unit 108, the contour of the pedestrian can be extracted as image data with a higher gradation. On the other hand, the contour of the oncoming vehicle is extracted as low gradation image data or a discontinuous contour. When the oncoming vehicle is turned on, the light is also heated, so that the infrared camera 101 captures the original image. However, the light diffuses heat and there is no clear outline. For this reason, the contour of the light is not extracted by the contour extraction unit 110.

  The binarization processing unit 111 binarizes the image data of the contour image obtained by the contour extraction unit 110. Specifically, a certain threshold value is set, and if the pixel image data is below the threshold value, 0 gradation is determined, and if it is equal to or greater than the threshold value, 255 gradation is determined. For example, if the 127th gradation is set as a threshold value, the gradation below the 127th gradation is 0 gradation, and the 127th gradation or more is 255 gradation. By adjusting this threshold value, the contour of the pedestrian can be 255 gradations and the contour of the oncoming vehicle can be 0 gradations. This binarization processing may be performed for each pixel, but it is preferable to perform a plurality of pixels collectively. For example, if three or more pixels among adjacent four pixels are equal to or greater than the threshold value, these four pixels can be set to 255 gradations, and if two or less pixels, the gradation can be collectively set to 0 gradations. Thereby, the influence of noise can be eliminated and an image with better visibility can be displayed.

  In addition, after performing the binarization process, the binarization processing unit 111 colors the contour image and performs the enhancement process. Therefore, the binarization processing unit 111 also serves as an enhancement processing unit that enhances the contour. As described above, the contour of the pedestrian is colored and emphasized because it has 255 gradations, but the contour of the oncoming vehicle is not colored because it has 0 gradation. As the coloring process, for example, the gradations of (R (red), G (green), and B (blue)) are set to (255, 0, 255), respectively, so that a magenta color can be obtained. The coloring color is preferably a color with high visibility with respect to a contrast adjustment image in which the brightness to be combined with the colored outline image is adjusted in the combining unit 112 described later. That is, a color having a high color contrast with respect to the background image is preferable. For example, when the contrast-adjusted image whose brightness has been adjusted is entirely gray, a warm color such as red or orange can be selected as the color of the contour. Also, red or yellow may be used to alert the driver.

  Thereafter, the synthesizing unit 112 synthesizes the contrast-adjusted image whose brightness is input from the brightness adjusting unit 109 and the colored outline input from the binarization processing unit 111 (step S6). As a result, the outline of an object that emits heat, such as a pedestrian, is colored, and display data of an image with good visibility is generated.

  The synthesizer 112 outputs the display data to the timing controller 104. As described above, the display data is output from the timing controller 104 to the source driver 107 of the liquid crystal display panel 105. The liquid crystal display panel 105 displays an image corresponding to the display data. In this way, in the image displayed on the liquid crystal display panel 105, the outline of an object that generates heat, such as a pedestrian, is colored with a highly visible color. For this reason, the driver can clearly recognize a safety confirmation target such as a pedestrian by separating it from a background such as a road or an oncoming vehicle.

  In the above description, the example in which the contrast adjustment unit 108 and the brightness adjustment unit 109 are provided has been described. However, the present invention is not limited to this. For example, a configuration in which the contrast adjustment unit 108 is not provided is also possible. It is also possible to extract the contour directly from the original image captured by the infrared camera 101 in the contour extraction unit 110. Further, the binarization processing unit 111 may not be provided. The contour extracted by the contour extraction unit 110 may be directly colored. Further, both the contrast adjustment unit 108 and the binarization processing unit 111 may not be provided.

  Further, the brightness adjustment unit 109 may not be provided. The image to be combined with the contour image data in the combining unit 112 may be an original image captured by the infrared camera 101.

  Here, an example is shown in which the image processing unit 103 is configured as a circuit, but the present invention is not limited to this, and the image processing unit 103 may be configured as a program (software). The image processing method according to the present invention can be used for various image display devices such as PDP and organic EL.

It is a block diagram which shows an example of a structure of the liquid crystal display device concerning this Embodiment. It is a block diagram which shows an example of a structure of the image process part concerning this Embodiment. It is a flowchart for demonstrating the image processing method concerning this Embodiment. It is a figure which shows an example of the outline extraction filter used for the image processing concerning this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Liquid crystal display device 101 Infrared camera 102 A / D converter 103 Image processing part 104 Timing controller 105 Liquid crystal display panel 106 Gate driver 107 Source driver 108 Contrast adjustment part 109 Brightness adjustment part 110 Contour extraction part 111 Binarization process part 112 Composition Part 113 contour extraction filter

Claims (12)

A contour extraction unit that extracts a contour from an original image captured by an infrared camera;
An emphasis processing unit that colors the contour extracted by the contour extraction unit and emphasizes the contour;
A display unit that synthesizes and displays the contour enhanced by the enhancement processing unit and the image captured by the infrared camera;
An image display apparatus. A contrast adjustment unit that adjusts a contrast of the original image captured by the infrared camera and generates a contrast adjustment image;
The image display device according to claim 1, wherein the contour extraction unit extracts a contour from the contrast adjustment image. A contrast adjusting unit that adjusts a contrast of an original image captured by the infrared camera and generates a contrast adjusted image;
A brightness adjustment unit for adjusting the brightness of the contrast adjustment image;
The image display device according to claim 1, wherein the display unit displays the contrast-adjusted image with adjusted brightness and the outline. A binarization processing unit that binarizes the image from which the contour is extracted in the contour extraction unit;
The image display device according to claim 1, wherein the enhancement processing unit colors an outline of the image subjected to the binarization processing.   The image display apparatus according to claim 4, wherein the binarization processing unit performs binarization processing for each of a plurality of adjacent pixels in the image from which the contour is extracted.   The image display device according to claim 1, wherein the enhancement processing unit selects a color having a high color contrast with respect to the image combined with the contour, and colors the contour. Take an original image with an infrared camera,
Extract the outline from the captured original image,
Emphasize the extracted outline,
An image processing method for combining and displaying the contour and an image photographed by the infrared camera. Contrast adjustment of the original image taken by the infrared camera to generate a contrast adjusted image,
The image processing method according to claim 7, wherein a contour is extracted from the contrast adjusted image. Contrast adjustment of the original image taken by the infrared camera to generate a contrast adjusted image,
Adjust the brightness of the contrast adjustment image,
The image processing method according to claim 7 or 8, wherein the contrast-adjusted image with adjusted brightness and the contour are combined and displayed. Binarizing the image in which the contour is extracted from the original image,
The image processing method according to claim 7, 8 or 9, wherein an outline of the image subjected to the binarization process is colored.   The image processing method according to claim 10, wherein the binarization processing binarizes each of a plurality of adjacent pixels in the image from which the contour is extracted.   The image processing method according to claim 7, wherein the enhancement processing selects a color having a high color contrast with respect to an image combined with the contour, and colors the contour.

Images