WO2022059139A1 - 画像表示装置および画像表示方法 - Google Patents

画像表示装置および画像表示方法 Download PDF

Info

Publication number
WO2022059139A1
WO2022059139A1 PCT/JP2020/035284 JP2020035284W WO2022059139A1 WO 2022059139 A1 WO2022059139 A1 WO 2022059139A1 JP 2020035284 W JP2020035284 W JP 2020035284W WO 2022059139 A1 WO2022059139 A1 WO 2022059139A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
flare
light source
region
source position
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/035284
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
勇人 菊田
俊明 久保
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to PCT/JP2020/035284 priority Critical patent/WO2022059139A1/ja
Priority to JP2022550267A priority patent/JP7355252B2/ja
Publication of WO2022059139A1 publication Critical patent/WO2022059139A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • This disclosure relates to an image display device such as an electronic mirror device and an image display method.
  • image correction that reduces the brightness value is performed so that the user's visibility is not impaired by the influence of flare generated by a strong local light source such as the headlight of a vehicle behind. It has been devised to do. Further, for example, in Patent Document 1, the brightness value of the concentric region from the center of the high-luminance region by the headlight of the rear vehicle in the image of the rear of the own vehicle is reduced, and the brightness value of the central portion is reduced. Image correction that suppresses the rate is performed to brighten only the central part of the headlights of the rear vehicle so that the driver can easily grasp the position of the rear vehicle.
  • the present disclosure has been made to solve the above-mentioned problems, and an object of the present disclosure is to provide an image display device for displaying an image with improved visibility.
  • an image pickup unit that images the outside of a vehicle and pixels in which pixels having a preset first brightness value or higher in the image pickup input image captured by the image pickup unit are gathered adjacent to each other.
  • a light source position detection unit that detects an area as a light source position, and a flare area determination unit that determines a pixel area having a preset second brightness value or more as a flare area from a pixel area around the center of the light source position.
  • a correction image generation unit that generates a correction image by reducing the brightness value of pixels of at least the input image in the flare area of the captured input image that are equal to or higher than the preset third brightness value, and at least the flare area.
  • the corrected image and the input image are combined with a composite image generation unit that generates a composite image by increasing the ratio of the corrected image as it is closer to the center of the light source position and increasing the ratio of the input image as it is farther from the center.
  • a composite image is provided in the pixel area where the image is generated, a corrected image or an input image is provided in the flare area among the pixel areas where the composite image is not generated, and an image display unit is provided outside the flare area to display the input image. It is an image.
  • the image display method includes a step of detecting as a light source position a pixel region in which pixels having a preset first brightness value or higher in an image captured by capturing the outside of a vehicle are adjacent to each other.
  • a correction image or an input image is provided in the flare area, and a display image for displaying the input image is generated outside the flare area.
  • the ratio of the input image to the corrected image is close to the center of the light source position in at least a part of the flare region determined from the center of the light source position by detecting the light source position from the brightness value of the captured input image. Since the composite image generated by increasing the proportion of the corrected image and increasing the proportion of the input image is displayed, the discontinuity of the brightness value of the displayed image can be suppressed and the image with high visibility can be displayed. ..
  • FIG. 1 is a schematic configuration diagram of the image display device of the first embodiment.
  • FIG. 2 is a schematic block diagram of the image display device of the first embodiment.
  • 3A to 3C are explanatory views illustrating control of the image processing unit of the first embodiment.
  • 4A and 4B are graphs showing the function F used in the composite image generation unit of the first embodiment.
  • 5A to 5D are examples of processed images in the image display device of the first embodiment.
  • FIG. 6 is an example of a time-by-time image capture input image for explaining the second embodiment.
  • FIG. 7 is a schematic block diagram of the image display device of the second embodiment.
  • FIG. 8 is a schematic block diagram of the image display device of the third embodiment.
  • the image display device 1 includes an image pickup unit 2 that captures an image of the outside of the own vehicle, an image processing unit 3 that processes the captured image, and an image display unit 4 that displays the processed image.
  • the image pickup unit 2 has one or more cameras that output an image pickup input image 110 that images the outside of the own vehicle.
  • an example in which the image pickup unit 2 takes an image of the situation behind the own vehicle will be described.
  • the camera for example, a two-dimensional image pickup device using a CMOS (Complementary MOSFET) or a CCD (Charge Coupled Device) element as an image sensor may be used.
  • CMOS Complementary MOSFET
  • CCD Charge Coupled Device
  • the image display unit 4 is mounted in the own vehicle and is arranged at a position suitable for the user to see the image display unit 4. For example, it is arranged near the A-pillars on the left and right sides of the front surface of the own vehicle, near the door trim in front of the side surface of the own vehicle, and the like.
  • the image display unit 4 uses a display device having a backlight composed of, for example, a liquid crystal filter such as an LCD (Liquid Crystal Display) panel, an LED (Light Emitting Diode), and a diffuser. Further, it may be a display device in which LEDs that emit organic light are arranged in pixel units.
  • the image display unit 4 displays a display image obtained by processing the image pickup input image 110 described later.
  • the image processing unit 3 is partially or wholly composed of a processing circuit.
  • the plurality of functions of the image processing unit 3 may be realized by separate processing circuits, or the functions of the plurality of parts may be collectively realized by one processing circuit.
  • the processing circuit may be configured by hardware or software, that is, a programmed computer.
  • a part may be realized by hardware and the other part may be realized by software.
  • the hardware for example, FPGA (Field-Programmable Gate Array), microprocessor, microcontroller, DSP (Digital Signal Processor) or the like may be used.
  • the software is, for example, a program that executes a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like on a processor.
  • FIG. 2 is a schematic block diagram of the image display device 1.
  • 3A to 3C are explanatory views for explaining the processing control of the image processing unit, FIG. 3A shows an image pickup input image, FIG. 3B shows a light source position information, and FIG. 3C shows an example of flare region information.
  • the image processing unit 3 includes a light source position detection unit 21, a flare area determination unit 22, a correction image generation unit 23, and a composite image generation unit 24.
  • the light source position information 31 indicating the light source position 41 detected by the light source position detection unit 21 from the image pickup input image 110 is output to the flare area determination unit 22 and the composite image generation unit 24, and the flare area determination unit 22 outputs the light source position information 31.
  • the flare area information 32 indicating the determined flare area 42 is output to the composite image generation unit 24.
  • the corrected image generation unit 23 generates a corrected image 12 in which the brightness value of the input image 11 in at least the flare region 42 of the captured input image 110 is reduced, and outputs the corrected image 12 to the composite image generation unit 24.
  • the composite image generation unit 24 generates a composite image 13 in which the composite ratio of the input image 11 and the corrected image 12 is changed based on the distance D from the center of the light source position 41 in at least a part of the flare region 42.
  • the composite image 13 is placed in the region where the composite image 13 is generated, and the corrected image 12 or the corrected image 12 is set in the flare region 42 including the end De of the flare region 42 in the region where the composite image 13 is not generated.
  • the image including the input image 11 is displayed on the image display unit 4 in the area outside the flare area 42.
  • the above-mentioned image processing is performed in each of the flare areas 42, and the input image 11 is displayed on the image display unit 4 in the areas other than the flare areas 42.
  • the light source position detection unit 21 inputs the image pickup input image 110 shown in FIG. 3A output from the image pickup unit 2, and the pixels having the first luminance value Th1 or more set in advance in the image pickup input image 110 are adjacent to each other.
  • the gathered pixel region is detected as the light source position 41, and the light source position 41 is used as the light source position information 31 shown in FIG. 3B.
  • the light source position information 31 is, for example, two-dimensional map information having the same size as the captured input image 110, and indicates the light source position 41 in which the white portion in FIG. 3B is detected.
  • the light source position detection unit 21 performs gradation processing on the pixels of the image pickup input image 110 to calculate the luminance value, and detects the high-luminance pixel whose luminance value is equal to or higher than the first luminance value Th1 as the light source position 41. do.
  • the luminance value for example, when the configuration of the image pickup element is a gradation value of three colors obtained from an RGB color filter, it is converted into a YCbCr color space composed of a luminance component and a color difference component, and the Y component is used as the luminance value. ..
  • the luminance value becomes high when strong light is projected onto the imaging device, when imaging noise or circuit noise appears locally, and the like.
  • the light source position detection unit 21 calculates the area of the area where the pixels having the first luminance value Th1 or more are adjacent to each other, and sets the area of the area equal to or more than the preset area as one light source position 41.
  • the object to be visually recognized by the user is the rear vehicle 51 behind the own vehicle, and the range in which the rear vehicle 51 exists within the angle of view of the camera is limited. Therefore, the pixel for calculating the luminance value by the light source position detection unit 21 may be limited to the range in which the rear vehicle 51 exists.
  • the flare area determination unit 22 determines the flare area 42 by using the light source position information 31 obtained from the light source position detection unit 21, and uses it as the flare area information 32.
  • the flare region information 32 is, for example, two-dimensional map information having the same size as the captured input image 110 as shown in FIG. 3C, and is information on a pixel region having high brightness around the center of the light source position 41. For example, the white portion in FIG. 3C is the determined flare region 42.
  • a method for determining the flare region 42 will be described. First, peripheral pixels are extracted in order from the center of the light source position 41 obtained in the light source position information 31, and the luminance values of the surrounding pixels are processed and calculated in the same manner as the light source position detection unit 21.
  • the center of the light source position 41 is, for example, the center of gravity of the pixel region detected as the light source position 41 by the light source position detection unit 21.
  • the pixel region in which the luminance value of the pixels around the center of the light source position 41 detected by the light source position detection unit 21 is equal to or greater than the preset second luminance value Th2 is determined as the flare region 42. ..
  • the corrected image generation unit 23 refers to at least the input image 11 in the flare region 42 of the captured input image 110, and reduces the brightness value of the pixel having the third brightness value Th3 or more set in advance. Twelve is generated.
  • the corrected image 12 may be one in which the luminance value is reduced only for the pixels having the third luminance value Th3 or more to remove the high luminance component, and both the third luminance value Th3 or more and the third luminance value Th3 or not are reduced. It may be something to make. For example, since the pixels obtained by imaging the headlight 52 of the rear vehicle 51 have a very high luminance value, the pixels around the headlight 52 have a large luminance value due to the influence of flare.
  • the third luminance value Th3 or more Flare can be suppressed by reducing the brightness value of the pixel.
  • the setting of the minimum value of the third brightness value Th3 is affected by the sensitivity of the image sensor in the image pickup unit 2 and the allocation to the digital value by HDR (High Dynamic Range) conversion, so it depends on the performance of the equipment to be used and the image pickup environment. Determined as a parameter based on.
  • the corrected image generation unit 23 may perform a process of reducing the luminance value of the pixel having the above-mentioned third luminance value Th3 or more, for example.
  • a method of preparing a table for determining an output value based on the luminance value of the captured input image 110 by a LUT (Look Up Table) and generating the corrected image 12 can be mentioned.
  • the luminance value of the input pixel is output for the luminance value smaller than the third luminance value Th3, and the luminance value of the input pixel is reduced when the luminance value is equal to or greater than the third luminance value Th3. Set the output so that it is the value that was set.
  • the luminance value increased due to the influence of flare becomes higher as the distance from the center of the light source position 41 becomes closer, so that a large reduction is required. Therefore, as the LUT parameter to be prepared becomes higher in brightness than the third luminance value Th3, for example, linear, Gaussian, and exponential function parameters may be given to reduce the luminance value.
  • the LUT parameter may be a one-dimensional input parameter using the luminance value of the pixel as an input value, or may be a multidimensional input parameter including the value of each RGB color and the chromaticity.
  • a LUT may be constructed only in the flare region 42 determined by the flare region determination unit 22 to generate the corrected image 12. Further, the corrected image 12 may be generated beyond the flare region boundary, which is the boundary between the flare region 42 and the region other than the flare region 42. That is, the correction image 12 may be generated by the correction image generation unit 23 in the entire image pickup input image 110, or not in the entire image pickup input image 110. It may be carried out at least in the pixel region for generating the composite image 13 described later
  • the composite image generation unit 24 uses the input image 11, the light source position information 31, the flare area information 32, and the corrected image 12, which are at least a part of the captured input image 110, and the closer to the center of the light source position 41, the closer the corrected image 12 is.
  • the ratio is increased, and the ratio of the captured input image 110 is increased as the ratio is increased.
  • a composite image 13 in which the corrected image 12 and the input image 11 are combined over the entire flare region 42 is generated and output to the image display unit 4.
  • the method of generating the composite image 13 in the composite image generation unit 24 will be described. For example, when the image obtained by synthesizing the input image 11 and the corrected image 12 in the flare area 42 at a constant ratio and the input image 11 outside the flare area 42 are displayed together, a discontinuity of the luminance value appears at the boundary of the flare area. It occurs and visibility is reduced. Therefore, the ratio of combining the input image 11 and the corrected image 12 is changed based on the distance D from the center of the light source position 41. That is, the closer to the center of the light source position 41, the smaller the composite ratio Ba of the input image 11, the larger the composite ratio Bb of the corrected image 12, and the farther from the center of the light source position 41, the larger the composite ratio Ba of the input image 11. The synthesis ratio Bb of 12 is reduced.
  • FIGS. 4A and 4B are graphs showing the function F (D, De, Tc) used in the composite image generation unit 24, and the composite ratio Ba of the input image 11 and the distance D from the center of the light source position 41. Shown in relation.
  • the function F (D, De, Tc) includes the distance D from the center of the light source position 41, the composition ratio Ba of the input image 11, the distance De from the center of the light source position 41 to the end of the flare region 42, and the adjustment parameter Tc. It becomes a function.
  • the composite ratio Bb of the corrected image 12 is a value obtained by subtracting the composite ratio Ba of the input image 11 from 1.
  • the distance D from the center of the light source position 41 is a variable, for example, the Euclidean distance of the line segment connecting the pixel for calculating the composition ratio and the pixel at the center of the light source position 41.
  • the distance De is the distance between the pixel at the end of the flare region 42 and the pixel at the center of the light source position 41 when the line segment is extended toward the end of the flare region 42.
  • the flare region 42 is not concentric from the center of the light source position 41, and the distance De from the center of the light source position 41 to the end of the flare region 42 varies depending on the selected line segment.
  • the composition ratio Ba may be 1, and as shown in FIG. 4B, the composition ratio Ba of the input image 11 from the center of the light source position 41 to an arbitrary distance Dn is 0, and the composition ratio Ba from Dn to the end De of the flare region 42 is arbitrary.
  • the composition ratio Ba of the input image 11 may be changed up to the distance De + ⁇ D obtained by adding ⁇ D.
  • the function F (D, De, Tc) may use an exponential function obtained by inversion of the Gaussian distribution, or may use a linear function.
  • the composite image generation unit 24 may generate the composite image 13 in the flare region 42, or may generate the composite image 13 in the region including the end of the flare region and its periphery.
  • the composition ratio may be generated by changing it for each line segment, and the composition ratio may be changed at the end of the flare region and its periphery such as the pixel region from Dn to De + ⁇ D, and the end of the flare region 42. It may include a pixel area exceeding.
  • the composition ratio Ba of the input image 11 may be 0 as shown in FIG. 4B, or the input image 11 and the correction image 12 may be added together at an arbitrary ratio.
  • FIG. 5A is an example of an image pickup input image 110 in which the rear right side of the own vehicle is imaged. Due to the influence of flare caused by the headlight 52 of the rear vehicle 51, the shape of the rear vehicle 51 in the vicinity of the headlight 52 and the road environment are difficult to visually recognize. Further, since the headlights 52 are arranged one on each side, the two flares are connected and visually recognized as one light source, making it difficult to grasp the position of the rear vehicle 51. In FIG.
  • the pixel region in which the pixels having the first luminance value Th1 or more in the image pickup input image 110 are adjacently gathered is detected as the light source position 41, and the luminance value of the pixels around the center of the light source position 41 is the second.
  • a pixel region having a luminance value of Th2 or more is determined as a flare region 42.
  • FIG. 5C is an example of the corrected image 12 in which the image pickup input image 110 is obtained by reducing the luminance value of the pixel having the third luminance value Th3 or more as a whole.
  • 5D shows the composite image 13 in the region where the composite image 13 is generated, the corrected image 12 or the composite image 13 in the flare region 42 among the regions where the composite image 13 is not generated, and the region other than the flare region 42.
  • a display image in which the input image 11 is displayed Specifically, in the flare region 42, the closer to the light source position 41, the larger the composite ratio Bb of the corrected image 12, and the farther it is, the larger the composite ratio Ba of the input image 11, and the composite of the input image 11 and the corrected image 12.
  • the image 13 is generated, and the input image 11 is displayed in a region other than the flare region 42.
  • the image display device 1 has a second brightness from the pixel region around the center of the light source position 41 detected from the brightness value of the captured input image 110.
  • a pixel region having a value Th2 or more is determined to be a flare region 42, and the brightness value of a pixel having a third brightness value Th3 or more of at least the input image 11 in the flare region 42 of the captured input image 110 is reduced and corrected.
  • Image 12 is generated.
  • the corrected image 12 and the input image 11 are combined by increasing the proportion of the corrected image 12 as it is closer to the center of the light source position 41 and increasing the proportion of the input image 11 as it is farther away.
  • Image 13 is generated.
  • the composite image 13 is placed in the region where the composite image 13 is generated, and the corrected image 12 or the corrected image 12 includes the input image 11 in the flare region 42 in the region where the composite image 13 is not generated. Since the input image 11 is displayed on the image display unit 4 outside the flare area 42, the discontinuity of the brightness value at the boundary between the flare area 42 and the non-flare area 42 is suppressed, and the image has high visibility. Can be displayed. Further, since it is not necessary to reduce the luminance value in the entire captured input image 110, it becomes easy to visually recognize notable information such as the shapes of surrounding buildings and the rear vehicle 51.
  • the influence of flare is suppressed and visual information such as the shape of the vehicle 51 behind the flare region boundary and the road environment becomes easier to visually recognize.
  • composition ratio Ba of the input image 11 and the composition ratio Bb of the corrected image 12 for each line segment, the discontinuity of the brightness value at the boundary of the flare region is suppressed even if the flare shape is complicated. , Highly visible images can be displayed.
  • the generation of the composite image 13 from the center of the light source position 41 to the end De of the flare region 42, or from Dn to De + ⁇ D described above may be processed for each pixel, and the flare region may be processed.
  • the function F (D, De, Tc) may be patterned and processed using a table based on the area of the pixel region of 42 or the light source position 41. It can be simplified by processing the pixel areas collectively.
  • the flare area determination unit 22 an example of determining a pixel area having a second luminance value Th2 or more as a flare region 42 from a pixel region around the center of the light source position 41 is shown, but the second luminance value is shown. Th2 may be changed based on the distance from the center of the light source position 41. The luminance value of the flare region 42 increases as the distance from the center of the light source position 41 increases. Therefore, for example, if the second luminance value Th2 is lowered sequentially from the center of the light source position 41, the flare region 42 can be more accurately formed. It can be determined.
  • the luminance value that determines the end portion of the flare region 42 is Th2i, which is smaller than the second luminance value Th20 set near the center of the light source position 41.
  • the second luminance value Th2 is set larger as the distance from the center of the light source position 41 is shorter, even if the pixels around the light source position 41 are affected by flare and the luminance value is increased, the luminance value is increased.
  • the flare region 42 can be appropriately determined.
  • the flare region 42 determined by the flare region determination unit 22 is a region continuous from the center of the light source position 41, if it is determined that the flare region 42 is not the flare region 42 in the above-mentioned determination of the flare region 42, the flare region 42 is determined. It can be determined that the region outside the determined pixel region is not the flare region 42. Therefore, it is better to start from the center of the light source position 41 and make a determination toward the outside.
  • the flare region 42 may be determined in a limited direction.
  • the end portion of the flare region 42 in eight directions is determined by advancing in order from the center of the light source position 41 with respect to a total of eight-direction axes of up / down / left / right and diagonal 45-degree axes.
  • the pixel region other than the on-axis pay attention to the two neighboring axes, and draw the pixels that are the ends of the flare region 42 of the two axes into a straight line or an arc centered on the center of the light source position 41.
  • the region located on the light source position 41 side of the connected line may be determined as the flare region 42.
  • the determination of the flare region 42 can be simplified, and the generation of the composite image 13 in the composite image generation unit 24 can also be simplified.
  • the processing load can be reduced by limiting the pixel region for determining the flare region 42 within the range of the distance from the center of the preset light source position 41. For example, a distance proportional to the area of the pixel region of the light source position 41 obtained from the light source position information 31 may be set, or a plurality of adjacent pixels may be averaged and set as a group.
  • the image display device 1 includes an image pickup unit 2, an image processing unit 3, and an image display unit 4 in the same manner as the schematic configuration of the first embodiment shown in FIG.
  • the image processing unit 3 holds the front frame information 33 such as the light source position information 31 and the flare area information 32 obtained in the previous frame captured in the previous time zone, which can be referred to in the image processing of the current frame captured at the present time.
  • the front frame information holding unit 25 is provided.
  • the flare area determination unit 22 determines a pixel region in which the luminance value around the center of the light source position 41 is equal to or greater than the second luminance value Th2 as the flare region 42. Since the light source position 41 is detected from the pixel region having the first luminance value Th1 or more in the image pickup input image 110, when noise appears in the input image 11, the flare region 42 in which the noise is erroneously detected as the light source position 41 is determined. There is a risk of Further, for example, when the road surface reflected light 53 of the headlight 52 of the rear vehicle 51 is projected and the pixel region having the second luminance value Th2 or more is large, it is determined to be the flare region 42, but when it is small, the flare region 42 is determined.
  • the camera used for the image pickup unit 2 that acquires the image pickup input image 110 usually takes images at intervals of 30 to 60 frames per second and recognizes noise and road surface reflected light 53. Focuses on the fact that is not all frames, and the display image is visually recognized even if noise is deleted or the flare area 42 is erroneously determined by referring to the information of the previous frame during image processing in the current frame. It does not reduce the sex.
  • FIG. 6 describes an example in which the road surface reflected light 53 is projected on the image pickup input image 110.
  • FIG. 6 is an example of the image pickup input image 110 captured in the order of time T1 to T7, and the road surface reflected light 53 suddenly appears at time T4 to T6. Further, the luminance value of the road surface reflected light 53 changes and becomes maximum at time T5. Due to the influence of such a momentary change in the luminance value, if the road surface reflected light 53 is not determined to be the flare region 42 at the time T4 and T6, and is determined to be the flare region 42 at the time T5, the input image 11 is used as it is. When the composite image 13 combined with the corrected image 12 is displayed, the display image displayed on the image display unit 4 will flicker. Therefore, the light source position information 31 and the flare area information 32 of the front frame are held in the front frame information holding unit 25, and the composite image generation unit 24 generates the composite image 13 of the current frame using these information.
  • the image pickup input image 110 or the input image 11 before the current time captured by the image pickup unit 2 is used as a front frame (for example, if the current time is time T5, the front frame of the current frame at time T5 is time T1 to One of the frames of 4). It may also refer to multiple previous frames including the frame before the previous frame.
  • FIG. 7 is a schematic block diagram of the image display device 1 of the present embodiment.
  • the image processing unit 3 includes a front frame information holding unit 25 that holds the light source position information 31 acquired by the light source position detecting unit 21 and the flare area information 32 acquired by the flare area determination unit 22 in the front frame.
  • the processing performed by the light source position detection unit 21, the flare area determination unit 22, and the correction image generation unit 23 is the same as that of the first embodiment.
  • a ring buffer having a memory area secured so that the information from the time T1 to the time T7 can be sequentially held is used.
  • the composite image generation unit 24 inputs the input image 11, the light source position information 31, the flare area information 32, the corrected image 12, and the front frame information 33 obtained from the front frame.
  • the composite image generation unit 24 synthesizes the input image 11.
  • the composite image 13 is generated by increasing the ratio Ba to, for example, 0.7 or more. In this way, when the noise of the captured input image 110, the road surface reflected light 53 appearing in a hurry, and the like are reflected in the image with reference to the front frame information 33, and the influence of flare is considered to be small, the brightness of the input image 11 is increased. By approaching the value, the flicker of the displayed image can be suppressed.
  • the front frame information holding unit 25 may hold a plurality of front frame information. For example, if the pixel at the position determined to be the flare region 42 in the time T6 is determined to be the flare region 42 only in T5 in the previous frames of the times T1 to T5 held in the front frame information 33, it is determined to be the flare region 42.
  • the percentage of unfilled frames is 4/5.
  • composition ratio Ba of the input image 11 may be determined by using a function having the ratio of the number of previous frames that is not determined to be the flare region 42 as a variable or a LUT having the number of frames as the input value.
  • the ratio of the input image 11 and the ratio of the corrected image 12 in the composite image generation unit 24 are such that the larger the number of the front frames that are not determined to be the flare region 42 in the front frame captured by the image pickup unit 2, the larger the number of the input image 11. Increase the proportion and decrease the proportion of the corrected image.
  • the front frame information holding unit 25 may hold the information from which the information of the flare area 42 determined by the area determination unit 22 is obtained.
  • the light source position 41 in the front frame is held by the front frame information holding unit 25, and the ratio of the input image 11 and the ratio of the corrected image 12 in the composite image generation unit 24 are set to the light source position in the front frame imaged by the image pickup unit 2.
  • the ratio of the input image 11 may be increased and the ratio of the corrected image 12 may be decreased so that 41 is not detected.
  • the light source position 41 is detected from the brightness value of the captured input image 110, and the input image 11 and the corrected image 12 are formed in at least a part of the flare region 42 determined from the center of the light source position 41.
  • the composite image 13 generated by increasing the proportion of the corrected image 12 as it is closer to the center of the light source position 41 and increasing the proportion of the input image 11 as it is farther away is displayed, and the input image is displayed in the area other than the flare area 42. Since 11 is displayed, it is possible to suppress the discontinuity of the brightness value of the displayed image and display an image with high visibility. Further, referring to the front frame information 33, when the influence of flare is considered to be small, the flicker of the displayed image is suppressed by bringing the brightness value closer to the input image 11, and an image with higher visibility is displayed. can.
  • the image display device 1 includes an image pickup unit 2, an image processing unit 3, and an image display unit 4 in the same manner as the schematic configuration of the first embodiment shown in FIG. As shown in 8, the front frame information holding unit 26 is provided with the composite image 13 of the front frame output from the composite image generation unit 24.
  • the front frame information holding unit 26 stores the composite image 13 output from the composite image generation unit 24 in a plurality of front frames.
  • a memory area is secured so that information from time T1 to time T7 can be sequentially held, and a ring buffer or the like is used.
  • the light source position detection unit 21 uses the captured input image 110 of the current frame and the composite image 13 of the plurality of front frames held by the front frame information holding unit 26, and presents from the transition of the light source position 41 in the composite image 13 of the front frame.
  • the light source position 41 of the frame is estimated. For example, in the case of a camera that captures 60 frames per second, the transition of the light source can be grasped because the same light source is repeatedly imaged in the composite image 13 of a plurality of previous frames. Then, by comparing the brightness values of the composite images 13 of the plurality of front frames, it is possible to find and remove high-luminance pixels that are not affected by flare such as noise and road surface reflected light 53.
  • a composite image 13 of a plurality of front frames for a preset time is taken out from the front frame information holding unit 26, and the transition of the position of the center of gravity of the high-luminance region calculated for each frame is observed.
  • the position of the center of gravity moves according to the traveling direction of the road. Can be estimated from. If the movement of the position of the center of gravity in the high-luminance region cannot be estimated in a plurality of front frames, it is determined that the region is not affected by flare caused by noise of the image pickup input image 110, road surface reflected light, or the like, and the light source position 41. Can be prevented from being detected.
  • the flare in the input image 11 of the current frame is changed from the fluctuation of the size of the flare region 42 due to the movement of the headlight 52 of the rear vehicle 51 depending on the directivity characteristic of the sensitivity of the camera.
  • the region 42 may be determined.
  • the region is not affected by the flare caused by the noise of the image pickup input image 110, the road surface reflected light 53, and the like, and the region is determined to be the flare region 42. You can avoid it.
  • the light source position 41 is detected from the brightness value of the captured input image 110, and the input image 11 and the corrected image 12 are formed in at least a part of the flare region 42 determined from the center of the light source position 41.
  • the composite image 13 generated by increasing the proportion of the corrected image 12 as it is closer to the center of the light source position 41 and increasing the proportion of the input image 11 as it is farther away is displayed, and the input image is displayed in the area other than the flare area 42. Since 11 is displayed, it is possible to suppress the discontinuity of the brightness value of the displayed image and display an image with high visibility. Then, by using the front frame information 34 with reference to the composite image 13 of the front frame, the positions of the light source position 41 and the flare region 42 can be estimated, and the processing can be simplified.
  • the captured input image 110 may be taken out as it is, or an image whose brightness, hue, etc. have been adjusted may be used.
  • the image display device 1 has been described, the image processing unit 3 constituting the image display device 1, the image display method implemented by the image display device 1, and the image processing method implemented by the image processing unit 3 are also the present invention. Form a part of. Further, a program that causes a computer to perform processing in the image processing unit 3 or the image processing method described above, and a computer-readable recording medium that records the program, such as a non-temporary recording medium, also form a part of the present invention. ..
  • 1 image display device 2 image pickup unit, 3 image processing unit, 4 image display unit, 11 input image, 12 corrected image, 13 composite image, 21 light source position detection unit, 22 flare area determination unit, 23 corrected image generation unit, 24 Composite image generation unit, 25, 26 front frame information holding unit, 31 light source position information, 32 flare area information, 33, 34 front frame information, 41 light source position, 42 flare area, 51 rear vehicle, 52 headlight, 53 road surface reflection Hikari, 110 image capture input image

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Processing (AREA)
PCT/JP2020/035284 2020-09-17 2020-09-17 画像表示装置および画像表示方法 Ceased WO2022059139A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2020/035284 WO2022059139A1 (ja) 2020-09-17 2020-09-17 画像表示装置および画像表示方法
JP2022550267A JP7355252B2 (ja) 2020-09-17 2020-09-17 画像表示装置および画像表示方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/035284 WO2022059139A1 (ja) 2020-09-17 2020-09-17 画像表示装置および画像表示方法

Publications (1)

Publication Number Publication Date
WO2022059139A1 true WO2022059139A1 (ja) 2022-03-24

Family

ID=80776560

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/035284 Ceased WO2022059139A1 (ja) 2020-09-17 2020-09-17 画像表示装置および画像表示方法

Country Status (2)

Country Link
JP (1) JP7355252B2 (https=)
WO (1) WO2022059139A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025099908A1 (ja) * 2023-11-09 2025-05-15 日立Astemo株式会社 画像処理装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004048456A (ja) * 2002-07-12 2004-02-12 Niles Co Ltd 撮像システム
JP2009141813A (ja) * 2007-12-07 2009-06-25 Panasonic Corp 撮像装置、カメラ、車両及び撮像方法
WO2009081533A1 (ja) * 2007-12-21 2009-07-02 Panasonic Corporation フレア補正装置
JP2009296224A (ja) * 2008-06-04 2009-12-17 Nippon Soken Inc 撮像手段及びプログラム
JP2010004450A (ja) * 2008-06-23 2010-01-07 Nippon Soken Inc 撮像装置及びプログラム
JP2013232880A (ja) * 2012-04-27 2013-11-14 Lg Innotek Co Ltd イメージ処理装置及びイメージ処理方法
JP2018121122A (ja) * 2017-01-23 2018-08-02 株式会社デンソー 運転支援システム、運転支援方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004048456A (ja) * 2002-07-12 2004-02-12 Niles Co Ltd 撮像システム
JP2009141813A (ja) * 2007-12-07 2009-06-25 Panasonic Corp 撮像装置、カメラ、車両及び撮像方法
WO2009081533A1 (ja) * 2007-12-21 2009-07-02 Panasonic Corporation フレア補正装置
JP2009296224A (ja) * 2008-06-04 2009-12-17 Nippon Soken Inc 撮像手段及びプログラム
JP2010004450A (ja) * 2008-06-23 2010-01-07 Nippon Soken Inc 撮像装置及びプログラム
JP2013232880A (ja) * 2012-04-27 2013-11-14 Lg Innotek Co Ltd イメージ処理装置及びイメージ処理方法
JP2018121122A (ja) * 2017-01-23 2018-08-02 株式会社デンソー 運転支援システム、運転支援方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2025099908A1 (ja) * 2023-11-09 2025-05-15 日立Astemo株式会社 画像処理装置

Also Published As

Publication number Publication date
JP7355252B2 (ja) 2023-10-03
JPWO2022059139A1 (https=) 2022-03-24

Similar Documents

Publication Publication Date Title
CN104756487B (zh) 图像处理系统和图像处理方法
JP5257695B2 (ja) 監視装置
US9123179B2 (en) Surrounding image display system and surrounding image display method for vehicle
WO2012067028A1 (ja) 画像入力装置および画像処理装置
WO2009088080A1 (ja) プロジェクタ
JP4952499B2 (ja) 画像処理装置
JP2010257282A (ja) 障害物検知装置、および当該装置を搭載した車両
JP7279613B2 (ja) 画像処理装置
CN111355880B (zh) 半导体器件、图像处理系统和方法及计算机可读存储介质
JP7355252B2 (ja) 画像表示装置および画像表示方法
JP2012008845A (ja) 画像処理装置
JPWO2015098107A1 (ja) 画像処理装置、警報装置、および画像処理方法
US11410274B2 (en) Information processing device and program
US11381757B2 (en) Imaging system and method
CN109788211B (zh) 拍摄图像显示系统、电子镜系统以及拍摄图像显示方法
JP6991664B2 (ja) 撮影画像表示システム及び電子ミラーシステム
CN107079108A (zh) 具有直方图扩展的机动车辆相机装置
JP2016076912A (ja) 撮影画像表示装置、撮影画像表示方法および撮影画像表示プログラム
JP2011251600A (ja) ブレーキ操作検知装置
JP6327388B2 (ja) 撮影画像表示装置、撮影画像表示方法および撮影画像表示プログラム
US12356103B2 (en) Imaging system and method
JP7080724B2 (ja) 配光制御装置、投光システム及び配光制御方法
JP2019067267A (ja) 画像処理装置及び画像処理方法、プログラム、記憶媒体
JP6311826B2 (ja) 撮影画像表示装置、撮影画像表示方法および撮影画像表示プログラム
JP2018037739A (ja) 画像処理装置、画像処理システム、移動体、及び画像処理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20954124

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022550267

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20954124

Country of ref document: EP

Kind code of ref document: A1