JP2019212937A - Image display device and image display method - Google Patents

Abstract

To provide an image display device and an image display method capable of improving a recognition degree of a subject imaged by an imaging unit.SOLUTION: An image display device 1 comprises: an imaging unit 2 which images a subject and outputs an image signal; a storage unit 37 which stores set wavelength data 370 related to a predetermined wavelength; a phase offset unit 33 which, when a signal of the predetermined wavelength is included in the image signal, shifts the wavelength of the signal to another wavelength; and an image display unit 3 which displays an image on the basis of the signal outputted from the phase offset unit 33.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image display device and an image display method for displaying an image by an image display unit based on an image signal output from an imaging unit.

  An in-vehicle camera system having an in-vehicle camera mounted on a vehicle and imaging the periphery of the vehicle has been proposed and realized. Conventional vehicle-mounted camera systems have demanded color reproducibility that is faithful to the displayed image from the viewpoint of faithfully reproducing the scenery and the like that is actually viewed by the passenger.

JP 2017-47902 A

  As the age increases, the relative luminous sensitivity of the eye differs from that of the younger age group. In particular, it is known that the blue relative short-distance sensitivity is shorter than that of the younger age group.

  FIG. 4 is a diagram showing a specific luminous efficiency curve defined by the CIE (International Commission on Illumination). The relative visibility curve shown in FIG. 4 is a relative visibility curve of a person who is excellent (not inferior) in color recognition mainly like a young age group.

  FIG. 5 is a diagram illustrating an example of the relative visibility curve of an elderly person. In FIG. 5, the relative visibility curve of the elderly is indicated by a dotted line. For reference, the relative visibility curve shown in FIG. 4 is also shown by a solid line.

  As shown in FIG. 5, it can be seen that in the relative visibility curve of the elderly, the blue-based relative visibility with a short wavelength is lower than that of the younger age group. It can also be seen that the red-based relative luminous sensitivity with a long wavelength is different from that of the younger age group. Therefore, if a subject including many colors having different relative luminosity as compared with the relative luminosity sensitivity of the younger age group is picked up by a camera, a phenomenon that it is difficult for an elderly person to recognize the subject may occur.

  Therefore, if an on-vehicle camera captures a subject that includes a large number of colors with reduced specific sensitivity, a phenomenon in which it is difficult for an elderly person to recognize the subject may occur. The same phenomenon can occur not only for the elderly but also for people with reduced specific visibility for a specific wavelength range. In addition, this phenomenon is not limited to the in-vehicle camera system, and such a phenomenon can occur in an image display apparatus having a camera.

  In addition, although the thing which changes the color of the emitted white laser beam according to a driver | operator's age is proposed in a vehicle lamp (what is called a headlight) (refer patent document 1), also with this vehicle lamp The above-mentioned problem cannot be solved.

  Therefore, an object of the present invention is to provide an image display device and an image display method capable of improving the recognition degree of a subject imaged by an imaging unit.

  In order to achieve the above object, an image display device of the present invention includes an imaging unit that captures an image of a subject and outputs an image signal, and when the image signal includes a signal of a predetermined wavelength, the signal wavelength is set to another wavelength. And a phase offset unit that shifts to an image display unit, and an image display unit that displays an image based on a signal output from the phase offset unit.

  In the image display device of the present invention configured as described above, the phase offset unit shifts the wavelength of the signal to another wavelength when the image signal includes a signal having a predetermined wavelength.

  By doing in this way, it becomes possible to improve the recognition degree of the subject imaged by the imaging part.

1 is a functional block diagram illustrating a schematic configuration of an image display apparatus according to an embodiment of the present invention. It is a flowchart for demonstrating operation | movement of the image display apparatus which is embodiment. It is a figure for demonstrating the operation | movement of the phase offset by the image display apparatus which is embodiment. It is a figure which shows an example of a general specific visibility curve. It is a figure which shows an example of the relative visibility curve of an elderly person.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of an image display apparatus according to an embodiment of the present invention.

  This embodiment is an example in which an image display device is applied to an in-vehicle camera system. The image display device 1 includes a camera (imaging unit) 2, an image processing unit 3, and an image display unit 4. The camera 2 captures an image of a subject in the forward direction (left side in FIG. 1) of a vehicle (not shown) on which the image display device 1 is mounted, and outputs an image signal. The image processing unit 3 performs various signal processing including phase offset and gradation offset processing described later on the image signal output from the camera 2. The image display unit 4 displays an image based on the image signal that is the output of the image processing unit 3.

  The image processing unit 3 according to this embodiment includes a programmable logic device such as a CPU and FPGA, an arithmetic element typified by an integrated circuit such as an ASIC, a large-capacity storage medium such as a hard disk drive, and a semiconductor storage such as a ROM and RAM. It has a storage medium such as a medium and an interface circuit with an external device. Here, in the present embodiment, the image processing unit 3 is described as having an integral arithmetic element, a storage medium, and an interface circuit, but these may be configured by separate arithmetic elements or the like.

  An unillustrated control program is stored in the storage unit 37 which is a storage medium of the image processing unit 3, and this control program is executed by the arithmetic element when the image display device 1 is started up. The functional configuration as shown in FIG. 1 is provided. In particular, since the image processing unit 3 of the present embodiment performs high-speed image processing as will be described later, it is preferable that the image processing unit 3 includes a computing element capable of high-speed computation, such as an FPGA.

  The camera 2 includes an optical system 20, an optical filter 21, and a sensor 22.

  The optical system 20 is composed of optical elements such as lenses and mirrors, and forms an image of light emitted from the subject or reflected by the subject on an imaging surface (not shown) of the sensor 22 described later. When the camera 2 is used for in-vehicle applications, the optical system 20 is generally a pan-focus lens such as a narrow angle, a wide angle, or a fisheye. Further, a lens system including a zoom mechanism or an autofocus mechanism may be used, or a lens system including an aperture or a shutter may be used. In order to improve image quality and color reproducibility, a lens system including various filters such as an optical low-pass filter, a band separation filter, and a polarization filter may be used.

  The sensor 22 is composed of a plurality of pixels, and photoelectrically converts incident light based on the subject image formed on the imaging surface by the optical system 20 into an output voltage signal corresponding to the luminance. The photoelectrically converted output voltage signal is digitized through an amplifier (not shown) provided in the sensor 22 and further an AD converter (not shown) provided in the sensor 22 to generate an image signal. .

  As the sensor 22, a photoelectric conversion element such as a CMOS image sensor or a CCD image sensor having sensitivity to at least visible light is used. In the present embodiment, the sensor 22 images a subject ahead in the traveling direction of the vehicle at a predetermined frame rate (for example, 1/30 seconds, 1/60 seconds), and sends an image signal to the image processing unit 3.

  In addition, on each pixel constituting the sensor 22, three types of filters (red filter R, green filter G, and blue filter B) having different transmission wavelength bands are regularly arranged as an example, and these three types are arranged. The optical filter 21 is composed of these filters.

  The image processing unit 3 includes a region dividing unit 30, a wavelength output unit 31, a wavelength comparison unit 32, a phase offset unit 33, a gradation comparison unit 34, a gradation offset unit 35, a composite image output unit 36, and a storage unit 37.

  The area dividing unit 30 divides an image displayed by the image signal output from the imaging unit 2 into a plurality of areas, and outputs information regarding which pixels belong to each area. The size and the number of regions of the region set by the region dividing unit 30 can be arbitrarily set based on various image processes in the phase offset unit 33 and the gradation offset unit 35 described later. As an example, the area dividing unit 30 divides an image into areas of n × n (n is a natural number), for example, 3 × 3 pixels.

  The wavelength output unit 31 separates the image signal output from the camera 2 into three color signals (R, G, B) corresponding to the respective colors constituting the image signal. At this time, the output of a pixel having no signal is 0 (blank). Next, the wavelength output unit 31 linearly interpolates the pixel values expected to be observed in the blank pixels using the neighboring pixel values. Then, three color signals (R, G, B) are generated in all pixels. Further, the wavelength output unit 31 performs color signal correction processing by a known linear matrix calculation.

  In addition, the wavelength output unit 31 calculates a wavelength included in the image signal of the region unit divided by the region dividing unit 30 by a known linear matrix calculation. The output from the wavelength output unit 31 is, for example, an intensity value at each wavelength in a certain wavelength range (for example, a visible wavelength range). Although the wavelength value may be discrete, it is preferable that the wavelength increment or difference be as fine as possible. The wavelength output unit 31 may calculate the wavelength included in the image signal for each pixel.

  The wavelength comparison unit 32 compares and determines the wavelength included in the image signal such as the region unit output from the wavelength output unit 31 and the predetermined wavelength set by the set wavelength data 370 stored in the storage unit 37, The determination result is output.

  Based on the determination result of the wavelength comparison unit 32, the phase offset unit 33 shifts the wavelength of the signal to another wavelength when a signal having a predetermined wavelength is included in the image signal for each region. The term “phase offset” used herein refers to offsetting (changing) the phase of a color, that is, the hue, by shifting the wavelength to either the long wavelength side or the short wavelength side.

  More specifically, the phase offset unit 33 shifts the wavelength of the signal to the long wavelength side by a predetermined value when the image signal includes a signal having a wavelength less than the predetermined wavelength. On the other hand, when the image signal includes a signal having a wavelength exceeding a predetermined wavelength, the phase offset unit 33 shifts the wavelength of the signal to the short wavelength side by a predetermined value.

  In the image signal output from the phase offset unit 33, the gradation comparison unit 34 includes an image signal of a specific pixel (first pixel) and an image of another pixel (second pixel) adjacent to the specific pixel. The hue and saturation of the signal are compared and determined, and the determination result is output.

  The gradation offset unit 35, based on the determination result of the gradation comparison unit 34, when the image signal corresponding to the first pixel and the image signal of the second pixel have the same hue and saturation, The saturation of the signal of the pixel or the second pixel is shifted by a predetermined value.

  Based on the image signal whose wavelength has been shifted by the phase offset unit 33 and whose saturation has been shifted by the gradation offset unit 35, the composite image output unit 36 is the image signal resulting from the wavelength shift and saturation shift. Is synthesized and output. The image signal output from the composite image output unit 36 is output to the image display unit 4.

  The storage unit 37 stores an unillustrated control program for performing overall control of the image processing unit 3, data necessary for various image processing by the image processing unit 3, and set wavelength data 370.

  The set wavelength data 370 is data relating to a wavelength that is a determination criterion in the wavelength comparison unit 32. The set wavelength data 370 is also data serving as a reference for determining whether the phase offset unit 33 offsets the wavelength. The set wavelength data 370 can be arbitrarily set and is stored in the storage unit 37 in advance. In the image display device 1 according to the present embodiment, two types of set wavelength data 370 are stored in the storage unit 37 for each of the short wavelength side (blue side) and the long wavelength side (red side). .

  The image display unit 4 includes a display device such as a liquid crystal display, for example, and displays an image based on the image signal output from the image processing unit 3.

  Next, details of the operation of the image display apparatus 1 according to the present embodiment will be described with reference to the flowchart of FIG. 2 and FIGS. 4 to 6. Of the operations of the respective parts constituting the image display device 1, operations already described in detail are appropriately omitted.

  The operation shown in the flowchart of FIG. 3 starts when the image display apparatus 1 is activated. First, in step S1, the sensor 22 of the imaging unit 2 images the front of the vehicle and outputs an image signal. The image signal output from the imaging unit 2 is input to the image processing unit 3.

  In step S2, the area dividing unit 30 divides the image displayed by the image signal input to the image processing unit 3 in step S1 into a plurality of areas, and outputs information regarding which pixels belong to each area. . As an example, the region dividing unit 30 divides the image into regions each including 3 × 3 pixels.

  In step S3, the image processing unit 3 extracts one (specific) region from the plurality of regions divided in step S2.

  In step S4, based on the image signal of the pixel belonging to the region extracted in step S3, the wavelength output unit 31 calculates and outputs the wavelength included in the extracted region unit image signal.

  In steps S5 to S8, a phase offset operation is performed by the wavelength comparison unit 32 and the phase offset unit 33.

  The phase offset unit 33 uses a wavelength of this color that is different from the relative luminous sensitivity of the younger age group, and the wavelength of this color is less different from that of the younger age group. By shifting to the wavelength, the recognition degree of the subject is improved. Specifically, the phase offset unit 33 is shifted to the long wavelength side for the blue color and to the short wavelength side for the red color.

  The set wavelength data 370 described above is preferably data relating to a wavelength at which the change in the relative visibility of the younger age group and the relative visibility of the elderly is large. More specifically, the wavelength λb of the set wavelength data 370 on the short wavelength side is set as a wavelength in which the change in the relative visibility of the young age group and the relative visibility of the elderly is large in a shorter wavelength band. Similarly, the wavelength λr of the set wavelength data 370 on the long wavelength side is set as a wavelength in which the change in the relative visibility of the younger age group and the relative visibility of the elderly is large in a longer wavelength band.

  The wavelengths λb and λr of the set wavelength data 370 can be arbitrarily set because a person (such as a driver) who recognizes the image displayed by the image display device 1 has a specific luminous efficiency curve. Further, the wavelengths λb and λr of the set wavelength data 370 may differ depending on under what circumstances the driver or the like recognizes the image of the image display device 1 (for example, whether the vehicle mainly travels in a city with many street lamps). . Therefore, also from this viewpoint, the wavelengths λb and λr of the set wavelength data 370 can be arbitrarily set.

  Returning to the description of the flowchart, in step S5, the wavelength comparison unit 32 determines the magnitude relationship between the wavelength λx and the wavelength λb included in the image signal of the specific region output by the wavelength output unit 31 in step S4. As a result of the determination, if it is determined that λx <λb (YES in step S5), the program proceeds to step S6. On the other hand, if it is determined that λx ≧ λb (NO in step S5), the program proceeds to step S7.

  In step S7, similarly, the wavelength comparison unit 32 determines the magnitude relationship between the wavelength λx and the wavelength λr. If it is determined that λ <λr (YES in step S7), the program proceeds to step S9. On the other hand, if it is determined that λx ≧ λr (NO in step S7), the program proceeds to step S8.

  In step S6, the phase offset unit 33 performs a process of shifting the wavelength λx to the long wavelength side.

  Specific processing will be described with reference to FIG. FIG. 3 is an enlarged view of the specific visibility curve near λb in FIG. The phase offset unit 33 shifts the wavelength λx, which is an input wavelength, to a wavelength λy that is shifted from the wavelength λx to the long wavelength side by a predetermined value.

  The wavelength λy is determined such that Ey−Ex is a positive constant, where Ex is the specific luminous sensitivity of the wavelength λx and Ey is the specific luminous sensitivity of the wavelength λy. From the viewpoint of improving the degree of recognition of the subject, it is preferable to offset (increase sensitivity) the relative visibility to the relative visibility that can be recognized by the driver or the like. For this reason, the recognition degree of the subject can be reliably improved by setting the offset value of the relative visibility to a positive constant. Further, it is necessary to determine so that λy> λb.

  Similarly, in step S8, the phase offset unit 33 performs a process of shifting the wavelength λx to the short wavelength side. The operation in step S8 is substantially the same as that in step S6, except that the phase offset unit 33 shifts the wavelength λx to the short wavelength side. Further, the wavelength λy is determined so that λy <λr.

  Next, in step S9, the image processing unit 3 extracts one (specific) pixel, that is, the first pixel among the pixels in the specific region extracted in step S3.

  In step S10, the gradation comparison unit 34 compares and determines the hue and saturation of the image signal of the first pixel in the specific area and the image signal of the second pixel adjacent to the specific pixel. If it is determined that the hue and saturation of the image signal of the first pixel and the hue and saturation of the image signal of the second pixel are not the same as a result of the determination (there is a gradation difference in step S10), the program Proceed to step S12. On the other hand, if it is determined that the hue and saturation of the image signal of the first pixel and the hue and saturation of the image signal of the second pixel are the same (no gradation difference in step S10), the program proceeds to step S11. move on.

  In step S11, the gradation offset unit 35 shifts the saturation of the image signal of the first pixel or the second pixel by a predetermined value. The human eye is sensitive to the difference in gradation compared to the difference in hue. Therefore, the case where the hue and the saturation are the same as a result of the wavelength shift by the phase offset unit 33 is included, and the gradation difference is given by shifting the saturation.

  Specifically, the saturation shift processing by the gradation offset unit 35 may be performed as follows. That is, the magnitude relationship between the hue and saturation of the first pixel before the phase offset process and the hue and saturation of the second pixel is determined. Furthermore, the sign of the difference between the hue and saturation of the first pixel and the hue and saturation of the second pixel is determined. Next, the saturation of the first pixel or the second pixel is shifted by a predetermined value so that the above-described difference is enlarged.

  In step S12, the image processing unit 3 determines whether all the pixels in the specific area have been extracted. If it is determined that all the pixels have been extracted (YES in step S12), the program proceeds to step S14. On the other hand, if it is determined that there is a pixel that has not yet been extracted (NO in step S12), the image processing unit 3 extracts the next pixel in step S13. Thereafter, the program returns to step S10.

  In step S14, it is determined whether the image processing unit 3 has extracted all the regions. If it is determined that all regions have been extracted (YES in step S14), the program proceeds to step S16. On the other hand, if it is determined that there is an area that has not yet been extracted (NO in step S14), the image processing unit 3 extracts the next area in step S15. Thereafter, the program returns to step S3.

  In step S <b> 16, the composite image output unit 36 composites and outputs the image signal resulting from the wavelength shift and saturation shift. The image signal output from the composite image output unit 36 is output to the image display unit 4. After this, the program ends.

  In the image display device 1 according to the present embodiment configured as described above, the phase offset unit 33 shifts the wavelength of the signal to another wavelength when the image signal includes a signal having a predetermined wavelength. .

  Accordingly, for example, by appropriately setting the wavelengths λb and λr in the above-described embodiment, the wavelength of this color is changed to the relative visibility for a color that is inferior to the general relative visibility. It is possible to shift to wavelengths of colors that are not inferior. Therefore, according to the image display device 1 of the present embodiment, it is possible to improve the recognition degree of the subject imaged by the imaging unit 2.

  Thereby, when the image display apparatus 1 which is this Embodiment is applied to a vehicle-mounted camera system, for example, the recognition degree of the target object (subject) around the vehicle can be improved. As an example, when a vehicle is parked in a dark parking lot, when the color of the body of an adjacent vehicle is difficult to recognize, the hue of the color of this body can be shifted. Therefore, the recognition degree of the adjacent vehicle can be improved and parking can be performed safely.

  In particular, in the image display apparatus 1 according to the present embodiment, the wavelengths λb and λr are set based on the relative visibility curve, so the relationship between the hue and the relative visibility can be parameterized. Accordingly, the image processing operations of the image processing unit 3, particularly the wavelength comparison unit 32 and the phase offset unit 33 can be formulated and simplified.

  Further, in the image display device 1 according to the present embodiment, the gradation offset unit 35 has the same hue and color when the signal corresponding to the first pixel and the signal of the second pixel adjacent to the first pixel are the same. When it has a degree, the saturation of the signal of the first pixel or the second pixel is shifted by a predetermined value. As a result, even when the hue and saturation of adjacent pixels become the same as a result of the wavelength shift by the phase offset unit 33, the recognition degree of the subject can be further improved by providing a gradation difference between adjacent pixels. Is possible.

  The embodiment of the present invention has been described in detail above with reference to the drawings. However, the specific configuration is not limited to the embodiment and the example, and the design change is within a range not departing from the gist of the present invention. Are included in the present invention.

  As an example, in the above-described embodiment, an example in which the image display device 1 is applied to an in-vehicle camera system has been described. However, an example in which the image processing device of the present invention is applied is not limited to an in-vehicle camera system.

  In the above-described embodiment, the image display apparatus 1 may be provided with an input unit that inputs the set wavelength data 370 in advance. Further, as described above, since the relative visibility curve changes depending on the age, it is preferable to store the typical visibility curve or set wavelength data 370 for each age in the storage unit 37. In this case, an input unit for inputting the age may be provided in the image display device 1, and the relative visibility curve or the set wavelength data 370 may be selected according to the input age. Furthermore, the age may be determined by capturing the face of a person who visually recognizes the image displayed on the image display unit 4 such as a driver and recognizing the captured face.

  Furthermore, although the above-mentioned embodiment was about the image display apparatus 1 which can respond to the change of the relative visibility curve accompanying an aging, when a relative visibility curve changes by factors other than aging. Is also available. In this case, a specific visibility curve of a person who visually recognizes an image displayed on the image display unit 4 may be acquired in advance, and the set wavelength data 370 may be set based on the specific visibility curve. Alternatively, the set wavelength data 370 based on the human specific visibility curve may be stored in the storage unit 37.

1 Image display device 2 Imaging unit (camera)
3 Image processing unit 4 Image display unit 30 Region division unit 31 Wavelength output unit 32 Wavelength comparison unit 33 Phase offset unit 34 Tone comparison unit 35 Tone offset unit 37 Storage unit 370 Setting wavelength data

Claims (5)

An imaging unit that images a subject and outputs an image signal;
A phase offset unit that shifts the wavelength of the signal to another wavelength when the image signal includes a signal of a predetermined wavelength;
An image display device comprising: an image display unit configured to display an image based on the signal output from the phase offset unit.   2. The phase offset unit according to claim 1, wherein when the image signal includes the signal having a wavelength less than the predetermined wavelength, the phase offset unit shifts the wavelength of the signal to a long wavelength side by a predetermined value. Image display device.   2. The phase offset unit according to claim 1, wherein when the image signal includes the signal having a wavelength exceeding the predetermined wavelength, the phase offset unit shifts the wavelength of the signal to a short wavelength side by a predetermined value. Image display device. The imaging unit outputs the image signal in pixel units, and
As a result of the wavelength shift of the signal by the phase offset unit, the signal corresponding to the first pixel and the signal of the second pixel adjacent to the first pixel have the same hue and saturation. 4. The image display according to claim 1, further comprising: a gradation offset unit that shifts the saturation of the signal of the first pixel or the second pixel by a predetermined value when the image signal is included. 5. apparatus. An image display method by an image display device having an imaging unit that images a subject and outputs an image signal, and an image display unit that displays an image based on the image signal,
An image display method characterized by shifting the wavelength of the signal to another wavelength when the image signal includes a signal of a predetermined wavelength.