JP2018037739A - Image processing apparatus, image processing system, mobile body, and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain an image for generating a composite image which is well-balanced in color saturation and imparts a less feeling of incompatibility to a person who refers thereto.SOLUTION: An image processing apparatus 1 comprises: an acquisition part 21 which acquires a plurality of images picked up by a plurality of cameras 10 each comprising a filter transmitting visible light and selectively transmitting infrared light within a predetermined wavelength range; and a processor 22 which calculates color saturation information for each of the plurality of images acquired by the acquisition part 21, and lowers color saturation of pixels constituting at least one image among the plurality of images based upon respective pieces of color saturation information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing system, a moving body, and an image processing method.

  2. Description of the Related Art Conventionally, there has been known an image processing apparatus that creates one continuous composite image by combining images captured by a plurality of cameras. For example, the image processing apparatus described in Patent Document 1 overlooks the vehicle and the road surface around the vehicle by synthesizing images obtained by imaging the front, rear, right, and left sides of the vehicle with four cameras. A composite image representing the state is generated.

  On the other hand, imaging devices that capture images without causing glare or discomfort to the subject person by irradiating the subject with near-infrared light when the illuminance of the camera's imaging range is low, such as in the evening or at night, are known. It has been. For example, the imaging device described in Patent Document 2 transmits near-infrared light of 800 nm or more where the relative sensitivities of pixels of each color are substantially equal, and shields infrared light except near-infrared light in this wavelength range. It has a filter and images a subject irradiated with near-infrared light.

JP 2009-17020 A JP-A-10-108206

  However, an imaging apparatus having a filter that transmits near-infrared light in a predetermined wavelength range and shields infrared light other than the near-infrared light in the predetermined wavelength range, as in the above-described prior art, When imaging a subject that has been irradiated with infrared light, the relative sensitivity of the pixels of each color in the near-infrared light is almost the same, so the saturation is reduced compared to an image of a subject that has not been irradiated with near-infrared light. To do.

  Therefore, in an image processing apparatus that generates a composite image by combining a plurality of images, an image captured by an imaging apparatus having a filter that transmits near-infrared light and blocks infrared light other than near-infrared light Is used, an image relating to a subject irradiated with near infrared light and an image relating to a subject not irradiated with near infrared light have different saturations. Therefore, a synthesized image obtained by synthesizing these images has a different saturation for each region, and may become an unnatural image when viewed as a whole.

  Therefore, the object of the present invention, which has been made by paying attention to these points, is a sense of incongruity for the reference person who has balanced saturation even when a part of the subject is irradiated with near-infrared light. An image processing apparatus, an image processing system, a moving body, and an image processing method capable of obtaining an image for generating a composite image with a small amount of image data.

  An image processing apparatus that solves the above problems obtains a plurality of images captured by a plurality of cameras each having a filter that transmits visible light and selectively transmits infrared light in a predetermined wavelength range. And saturation information of each of the plurality of images acquired by the acquisition unit, and based on the respective saturation information, saturation of pixels constituting at least one image of the plurality of images And a processor for reducing the above.

  An image processing system that solves the above problems includes a plurality of cameras including filters that transmit visible light and selectively transmit infrared light in a predetermined wavelength range, and a plurality of images captured by the plurality of cameras, respectively. And obtaining saturation information of each of the plurality of images obtained by the obtaining unit, and obtaining at least one image of the plurality of images based on the respective saturation information And an image processing device including a processor for reducing the saturation of the constituent pixels.

  A moving body that solves the above problems includes a plurality of cameras having filters that transmit visible light and selectively transmit infrared light in a predetermined wavelength range, and a plurality of images captured by the plurality of cameras, respectively. An acquisition unit that acquires an image, and calculates saturation information of each of the plurality of images acquired by the acquisition unit, and configures at least one of the plurality of images based on the respective saturation information And an image processing apparatus including a processor for reducing the saturation of a pixel to be processed.

  An image processing method that solves the above problem is an image processing method executed by an image processing device, wherein the image processing device transmits visible light and selectively transmits infrared light in a predetermined wavelength range. A plurality of images captured by a plurality of cameras each having a filter to be acquired, and the image processing device calculates saturation information of each of the acquired plurality of images, and based on the respective saturation information Thus, the saturation of the pixels constituting at least one of the plurality of images is reduced.

  According to an embodiment of the present invention, even when a part of a subject is irradiated with near-infrared light, a composite image that is balanced in saturation and less discomfort for a reference person is generated. It is possible to obtain an image for

FIG. 1 is a functional block diagram of the image processing system according to the first embodiment. FIG. 2 is a top view of a moving body on which the imaging device, the image processing device, and the display device shown in FIG. 1 are mounted. FIG. 3 is a flowchart showing a processing flow of the image processing system in the first embodiment. FIG. 4 is a functional block diagram of the image processing system according to the second embodiment. FIG. 5 is a flowchart showing a processing flow of the image processing system in the second embodiment.

  A first embodiment of the present invention will be described below with reference to the drawings.

  As shown in FIG. 1, the image processing system 1 according to the first embodiment includes imaging devices (cameras) 10a, 10b, 10c, and 10d, an image processing device 20, a display device 30, and the like. The image processing device 20 is connected to the cameras 10a, 10b, 10c, 10d, and the display device 30 via a communication network to transmit and receive information. Further, the image processing device 20 may be connected to a driving support device or the like via a communication network to receive driving support information. The driving support information is information for supporting a driver's operation, for example, indicating a target parking position, an expected route, a reverse start position, and the like. The image processing device 20 may be provided in the same housing as the display device 30. In this case, the communication interfaces 24 and 31 may not be provided, and the functions executed by the processor 22 and the display processor 32 may be executed by the same processor. The display device 30 may also serve as a display unit such as a driving support device.

  As shown in FIG. 2, the cameras 10 a, 10 b, 10 c, 10 d, the image processing device 20, and the display device 30 are mounted on the moving body 4. Here, the “moving body” means, for example, a vehicle traveling on a road such as a passenger car, a truck, and a bus. In the following, any imaging device among the cameras 10a, 10b, 10c, and 10d is referred to as an “imaging device (camera) 10”.

  The camera 10 is installed so as to image the periphery of the moving body 4. If it demonstrates using the example shown in FIG. 2, camera 10a, 10b, 10c, 10d will be installed so that the front, right side, back, and left side of the moving body 4 may respectively be imaged. Each camera is installed such that a part of the imaging range of the camera is common to a part of the imaging range of the other cameras.

  The imaging range of the camera 10a is A1, A2, and A3 shown in FIG. 2, and the imaging range of the camera 10b is A3, A5, and A8. That is, A3 which is a part of the imaging range of the camera 10a is a part of the imaging range of the camera 10b.

  Similarly, the imaging range of the camera 10c is A6, A7, A8. That is, A8 which is a part of the imaging range of the camera 10b is a part of the imaging range of the camera 10c.

  The imaging range of the camera 10d is A1, A4, and A6. That is, A1 which is a part of the imaging range of the camera 10d is a part of the imaging range of the camera 10a.

  The camera 10 includes an imaging lens 11, a filter 12, an imaging element 13, a signal processing unit 14, and the like. Specifically, as shown in FIG. 1, the camera 10a includes an imaging lens 11a, a filter 12a, an imaging element 13a, a signal processing unit 14a, and the like. Similarly, the camera 10b includes an imaging lens 11b, a filter 12b, an imaging element 13b, a signal processing unit 14b, and the like. The camera 10c includes an imaging lens 11c, a filter 12c, an imaging element 13c, a signal processing unit 14c, and the like, and the camera 10d includes an imaging lens 11d, a filter 12d, an imaging element 13d, a signal processing unit 14d, and the like. Is done.

  The imaging lens 11 is a lens that collects light so that incident light forms an image on the light receiving surface of the imaging element 13. The imaging lens 11 may be configured with, for example, a fisheye lens or an ultra-wide angle lens. The imaging lens 11 may be composed of a single lens or a plurality of lenses.

  The filter 12 can be disposed on the image side of the imaging lens 11 so that the surface of the filter 12 is substantially perpendicular to the optical axis of the imaging lens 11. The filter 12 transmits visible light and selectively transmits near infrared light in a predetermined wavelength range. The predetermined wavelength range is a range of near-infrared light wavelengths in which the relative sensitivities of the image pickup devices 13 of the respective colors are equal, for example, a range including 860 nm.

  The pixels constituting the image generated by the light passing through the filter 12 being imaged on the image sensor 13 are expressed in color by visible light. As described above, near infrared light in a predetermined wavelength range reaches the image sensor 13 together with visible light. Since the signal intensity of each color pixel by near-infrared light is substantially equal, the saturation of the pixel represented by the color is lower than when the near-infrared light does not reach.

  For example, when the near-infrared light is irradiated to only a part of the range captured by each camera 10a, 10b, 10c, 10d, the above-described wavelength range only in the camera 10 that captures the range irradiated with the infrared light. Infrared light passes through the filter 12 and reaches the image sensor 13. Therefore, the first image obtained by imaging the range irradiated with near infrared light among the cameras 10a, 10b, 10c, and 10d and the range not irradiated with near infrared light among the cameras 10a, 10b, 10c, and 10d are imaged. In some cases, the saturation of the second image is so different as to be uncomfortable from the human eye. In particular, when the processor 22 generates one synthesized image by synthesizing the first image and the second image, the region related to the first image and the second image within the synthesized image. Since the saturation is different from that of the area related to, the user who visually recognizes the composite image has a sense of incongruity.

  The imaging element 13 is an imaging element that captures an image formed by the imaging lens 11. The image sensor 13 includes a charge-coupled device (CCD) image sensor and a complementary metal oxide semiconductor (CMOS) image sensor. The image sensor 13 includes a color filter for capturing a color image. As the color filter, filters of a plurality of colors such as red, green, blue, cyan, magenta, and yellow are arranged for each light receiving element. The color filters may be arranged in a Bayer arrangement.

  The signal processing unit 14 is a processor that processes an image. For example, the signal processing unit 14 is a dedicated microprocessor formed to execute a specific function or a processor that executes a specific function by reading a specific program.

  The signal processing unit 14 generates an image signal representing an image formed by the image sensor 13. The signal processing unit 14 may perform arbitrary processing such as distortion correction and gamma correction on the image.

  The communication interface 15 transmits an image signal representing an image processed by the signal processing unit 14 to the image processing device 20.

  The image processing apparatus 20 includes an acquisition unit 21, a processor 22, a memory 23, a communication interface 24, and the like.

  The acquisition unit 21 is an interface with the camera 10 that acquires an image represented by the image signal by receiving the input of the image signal generated by the signal processing unit 14.

  The processor 22 is a processor that processes the image acquired by the acquisition unit 21, and is a general-purpose processor that executes a specific function by reading a dedicated microprocessor or a specific program formed to execute the specific function, for example. CPU (Central Processing Unit).

  The processor 22 is imaged by each of the plurality of cameras 10a, 10b, 10c, and 10d, and the difference in brightness information of the plurality of images acquired by the acquisition unit 21 is equal to or less than a predetermined threshold (third threshold). adjust. Brightness information is a value indicating characteristics relating to the brightness of the entire image. The brightness information is calculated from the brightness (brightness) of each pixel constituting the image, and is, for example, a statistical value such as an average value, a median value, a mode value, and a total brightness of each pixel. Since the plurality of cameras 10a, 10b, 10c, and 10d have sensitivity to near-infrared light, near-infrared light contributes to brightness information together with visible light.

  Specifically, the processor 22 determines whether or not the difference in the brightness information of the plurality of images captured by the plurality of cameras 10a, 10b, 10c, and 10d and acquired by the acquisition unit 21 is greater than the third threshold value. Determine. If the difference in brightness information is greater than the third threshold, the brightness of the pixels constituting one or more images is changed so that the difference in brightness information for all images is less than or equal to the third threshold. To do. If the difference in brightness information is less than the third threshold, the processor 22 does not change the brightness of any pixel.

  Here, the third threshold value is a value that can be considered that the brightness of a plurality of images is comparable with the human eye when the difference in brightness information is less than or equal to this value. It is determined in advance.

  The method by which the processor 22 changes the brightness information of each image is arbitrary. For example, the processor 22 adjusts the brightness information of another image to the image having the highest brightness information among the plurality of images. The brightness of the pixels that make up the image may be increased, or the brightness of the pixels that make up the image with the highest brightness information may be reduced so that the brightness information matches the other images. Further, the processor 22 may calculate an average value of the brightness information of the plurality of images, and change the brightness of the pixels constituting each image so that the brightness information of the plurality of images becomes the average value. .

  The processor 22 calculates the saturation information of the plurality of images that are respectively captured by the plurality of cameras 10a, 10b, 10c, and 10d and whose brightness information is adjusted as described above. Saturation information is a value indicating the characteristics related to the vividness of the entire image. The saturation information is calculated from the saturation of each pixel constituting the image, and is, for example, a statistical value such as an average value, a median value, a mode value, or a sum of the saturation of each pixel. The saturation of each pixel is calculated from the RGB value of each pixel.

  Further, the processor 22 makes the difference between the saturation information of the plurality of images within a predetermined range based on the saturation information about the plurality of images captured by the plurality of cameras 10a, 10b, 10c, and 10d, respectively. In addition, the saturation of the pixels constituting at least one image is reduced. Here, a process in which the processor 22 reduces the saturation of the pixel based on the plurality of saturation information will be described in detail.

  The processor 22 calculates a difference in the saturation information of the plurality of images, and based on the difference in the saturation information between the first image and the second image different from the first image among the plurality of images. It is determined whether or not the saturation of the pixels constituting the image is reduced.

  For example, the processor 22 sets the image having the highest brightness information among the images acquired by the acquisition unit 21 as the first image, and the saturation information based on a difference from the second image different from the first image. Calculate the difference. In FIG. 2, when the area on the left side of the moving body 4 surrounded by a two-dot broken line ellipse is illuminated by a streetlight composed of a halogen light, among the plurality of cameras 10a, 10b, 10c, and 10d, the camera 10d The brightness information of the captured image Id is the highest. In addition, since the light emitted by the halogen light includes a lot of light in the near-infrared region, the saturation information of the image Id imaged by the camera 10d is lower than that when not illuminated by the halogen light. ing.

  Accordingly, the processor 22 determines that the saturation information SPd of the first image Id having the highest brightness information and the images Ia, Ib, and Ic different from the first image Id (in this example, these images are the second images, respectively). Differences between the respective saturation information SPa, SPb, and SPc (corresponding to images) are calculated. Specifically, the processor 22 calculates a difference | SPd−SPa | between the saturation information SPd of the first image Id and the saturation information SPa of the image Ia captured by the camera 10a. Further, the processor 22 captures the difference | SPd−SPb | between the saturation information SPd of the image Id and the saturation information SPb of the image Ib captured by the camera 10b, and the saturation information SPd of the image Id and the camera 10c. The difference | SPd−SPc | with respect to the saturation information SPc of the image Ic is calculated. Then, the processor 22 determines whether any of the calculated saturation information differences is larger than a predetermined threshold (first threshold). Then, the processor 22 determines that the saturation of the pixels constituting at least one of the plurality of images is reduced when any of the differences in the saturation information is greater than or equal to the first threshold. Here, the first threshold value is a value that can be considered that the vividness of a plurality of images is comparable with the human eye when the difference in saturation information is less than or equal to this value. It is determined in advance.

  The method of calculating the difference between the saturation information of a plurality of images is not limited to this, and may be calculated as follows. For example, the processor 22 calculates differences in saturation information in all combinations of arbitrary first images and second images different from the first images captured by the plurality of cameras 10a, 10b, 10c, and 10d, respectively. When it is calculated and any one of the differences is equal to or greater than the first threshold, it is determined that the saturation of the pixels constituting at least one of the plurality of images is reduced.

  Specifically, the processor 22 calculates a difference | SPa−SPb | between the saturation information SPa of the image Ia captured by the camera 10a and the saturation information SPb of the image captured by the camera 10b. Similarly, the processor 22 uses the saturation information SPc of the image captured by the camera 10c and the saturation information SPd of the image captured by the camera 10d, to | SPa-SPc |, | SPa-SPd |, | SPb-SPc |, | SPb-SPd |, and | SPc-SPd | are calculated, respectively. Then, the processor 22 determines that the saturation of the pixels constituting at least one of the plurality of images is reduced when any of these differences is larger than the first threshold.

  Here, a process in which the processor 22 reduces the saturation of the pixels constituting the image will be described in detail.

  The processor 22 reduces the saturation so that the saturation information of the second image different from the first image is matched with the first image having the lowest saturation information among the images Ia, Ib, Ic, and Id. . When the saturation information Spd of the image Id is the lowest among the images Ia, Ib, Ic and Id, the difference between the saturation information Spa, Spb and Spc of the images Ia, Ib and Ic and the saturation information Spd of the image Id The saturation of the pixels constituting the images Ia, Ib, and Ic is reduced so that becomes less than or equal to the first threshold. At this time, the saturation may be lowered so that the saturation reduction rate of each pixel becomes the same, or the saturation may be lowered based on any other algorithm.

  In addition, the processor 22 reduces the saturation of the pixels constituting at least one of the plurality of images, so that the difference in the saturation information of the plurality of images is equal to or less than the first threshold value. A composite image is generated by combining.

  In addition, the processor 22 outputs a composite image generated by combining a plurality of images to the communication interface 24. Further, the processor 22 may output, to the communication interface 24, an image obtained by superimposing driving assistance information received from the driving assistance device, for example, indicating a target parking position, an expected route, a reverse start position, and the like on the composite image.

  The communication interface 24 receives image information representing images captured by the plurality of cameras 10a, 10b, 10c, and 10d from the respective cameras 10a, 10b, 10c, and 10d. The communication interface 24 receives driving support information from the driving support device. In addition, the communication interface 24 transmits image information representing a composite image output by the processor 22 or an image in which driving support information is superimposed on the composite image to the display device. The communication interface 24 can be a physical connector, a wireless communication device, or the like. Physical connectors include electrical connectors, optical connectors, and electromagnetic connectors. The wireless communication device includes a wireless communication device that conforms to each standard including Bluetooth (registered trademark) and IEEE802.11, and an antenna.

  The display device 30 includes a communication interface 31, a display processor 32, a display panel 33, and the like.

  The communication interface 31 receives image information representing a composite image or an image obtained by superimposing driving support information on the composite image via the communication interface 24 from the image processing apparatus 20. As with the communication interface 24, the communication interface 31 can be a physical connector, a wireless communication device, or the like.

  The display processor 32 displays the image information received by the communication interface 31 on the display panel 33.

  Next, an image processing method of the image processing system 1 according to the first embodiment will be described with reference to FIG.

  First, the cameras 10a, 10b, 10c, and 10d respectively capture subjects around the moving body 4 to generate images Ia, Ib, Ic, and Id (step S11).

  When the images Ia, Ib, Ic, and Id are generated in step S11, the acquisition unit 21 of the image processing apparatus 20 displays the image information representing the images Ia, Ib, Ic, and Id, respectively, on the respective cameras 10a, 10b, 10c, It is acquired from 10d (step S12).

  When the image information is acquired in step S12, the processor 22 calculates the brightness information of each image based on the brightness of the pixels constituting the images Ia, Ib, Ic, and Id (step S13).

  When the brightness information is calculated in step S13, the processor 22 changes the brightness of the pixels constituting at least one image so that the brightness information difference between the images is equal to or smaller than the third threshold value. Brightness information is adjusted (step S14).

  When the brightness information is adjusted in step S14, the processor 22 calculates saturation information for each of the plurality of images whose brightness information has been adjusted (step S15).

  When the saturation information is calculated in step S15, it is determined whether or not the difference between the saturation information of the plurality of images is equal to or less than the first threshold (step S16).

  If it is determined in step S16 that the difference in saturation information for the plurality of images is greater than the first threshold, the saturation of the pixels constituting at least one of the plurality of images is reduced (step S17). Specifically, the second image different from the first image so that the difference from the first image having the lowest saturation information among the images Ia, Ib, Ic, and Id is not more than the first threshold value. The saturation of the pixels constituting each of them is reduced.

  When the difference in saturation information for a plurality of images is equal to or less than the first threshold value by reducing the saturation of the pixels in step S17, the processor 22 combines the plurality of images to generate a combined image. (Step S18). At this time, the processor 22 may generate an image obtained by superimposing driving support information including a target parking position, an expected route, a reverse start position, etc. received from the driving support device on the composite image.

  When the composite image or the image in which the driving support information is superimposed on the composite image is generated in step S18, the processor 22 outputs image information representing the generated image and transmits the image information to the display device 30 via the communication interface 24. (Step S19).

  When the image information is output to the display device 30 in step S19, the display device 30 displays the image (step S20).

  In the first embodiment, even when any one of the plurality of images captured by the cameras 10a, 10b, 10c, and 10d is reduced in saturation by near-infrared light, the saturation of each of the plurality of images. Since the saturation of the pixels constituting at least one image is reduced based on the information, the saturation information of all the images can be made the same level. Therefore, when a composite image is generated by combining these images, the reference can refer to the image without feeling a difference in color for each region.

  In the first embodiment, the saturation of any one of the plurality of images adjusted so that the difference in brightness information is equal to or smaller than the third threshold is reduced. For this reason, for example, when only a part of an image is brightened by a streetlight that emits light containing a lot of near-infrared light, the brightness of all the images is almost the same as the eyes of the referring person. And saturation information can be made similar. When only a part of the imaging range of the plurality of cameras 10a, 10b, 10c, and 10d is illuminated at night or in the evening by a streetlight such as a halogen light containing a lot of near infrared light, an object in the imaging range is displayed. The captured image is brighter than the other images and has a reduced saturation. Therefore, when the brightness information is adjusted as described above for a plurality of images and the saturation information is set to the same level, if a composite image is generated by combining these images, the composite image is A person who refers to the image can refer to the image without feeling a difference between the areas in terms of brightness and vividness.

  Hereinafter, a second embodiment of the present invention will be described with reference to the drawings.

  As shown in FIG. 4, the image processing system 2 according to the second embodiment is similar to the image processing system 1 according to the first embodiment, and includes cameras 10 a, 10 b, 10 c, 10 d, an image processing device 20, and a display. It includes the device 30 and the like. The image processing system 2 is mounted on the moving body 4 in the same manner as the image processing system 2 according to the first embodiment. The image processing system 2 further includes the illumination devices 40a, 40b, 40c, and 40d that irradiate the imaging ranges of the cameras 10a, 10b, 10c, and 10d with near-infrared light, respectively, according to the first embodiment. Different from the processing system 2. In the following description, an arbitrary illumination device among the illumination devices 40a, 40b, 40c, and 40d is referred to as “illumination device 40”.

  In FIG. 4, the configuration of the cameras 10b, 10c, and 10d is omitted, but the camera 10b includes an imaging lens 11b, a filter 12b, an imaging element 13b, a signal processing unit 14b, and the like, as in the first embodiment. Consists of. The camera 10c includes an imaging lens 11c, a filter 12c, an imaging element 13c, a signal processing unit 14c, and the like, and the camera 10d includes an imaging lens 11d, a filter 12d, an imaging element 13d, a signal processing unit 14d, and the like. Is done.

  The illuminating device 40 irradiates at least a part of the range captured by the camera 10 with near infrared light. The lighting device 40 includes a communication interface 41, an irradiation control unit 42, a light emitting unit 43, and the like. Specifically, the illumination device 40a includes a communication interface 41a, an irradiation control unit 42a, a light emitting unit 43a, and the like. Although not shown in FIG. 4, the illumination device 40b is similarly configured to include a communication interface 41b, an irradiation control unit 42b, a light emitting unit 43b, and the like. The illumination device 40c includes the communication interface 41c and the irradiation control unit. 42c, the light emission part 43c, etc. are comprised, and the illuminating device 40d is comprised including the communication interface 41d, the irradiation control part 42d, the light emission part 43d, etc.

  The communication interface 41 receives intensity information indicating the intensity of illumination transmitted from the image processing apparatus 20 via the communication interface 24. As with the communication interface 24, the communication interface 41 can be a physical connector, a wireless communication device, or the like.

  The irradiation control unit 42 controls the intensity of irradiation by the light emitting unit 43 based on the intensity information received by the communication interface 41.

  The light emitting unit 43 irradiates the range captured by the camera 10 with the light emission intensity controlled by the irradiation control unit 42. The light emitting unit 43 includes a near infrared light source. The near-infrared light source can be, for example, a near-infrared LED (light emitting diode).

  In the second embodiment, the processor 22 of the image processing apparatus 20 is imaged by each of the plurality of cameras 10a, 10b, 10c, and 10d, and is based on the difference in the brightness information of the plurality of images acquired by the acquisition unit 21. The intensity of the lighting devices 40a, 40b, 40c, and 40d is controlled.

  Specifically, the processor 22 determines the intensities of the lighting devices 40a, 40b, 40c, and 40d so that the difference in the brightness information of the plurality of images is equal to or less than the third threshold value. Then, the processor 22 transmits intensity information representing the intensity determined for each of the lighting devices 40a, 40b, 40c, and 40d to the respective lighting devices 40a, 40b, 40c, and 40d via the communication interface 24.

  Since other configurations and operations in the second embodiment are the same as those in the first embodiment, the same or corresponding components are denoted by the same reference numerals and description thereof is omitted.

  Next, an image processing method of the image processing system 2 according to the second embodiment will be described with reference to FIG.

  First, the cameras 10a, 10b, 10c, and 10d respectively capture subjects around the moving body 4 to generate images Ia1, Ib1, Ic1, and Id1 (step S31).

  When the images Ia1, Ib1, Ic1, and Id1 are generated in step S11, the acquisition unit 21 of the image processing device 20 displays the image information representing the images Ia1, Ib1, Ic1, and Id1, respectively, on the respective cameras 10a, 10b, 10c, 10d is acquired (step S32).

  When the image information is acquired in step S12, the processor 22 calculates the brightness information of each image based on the brightness of the pixels constituting the images Ia1, Ib1, Ic1, and Id1 (step S33).

  When the brightness information is calculated in step S33, the processor 22 determines the intensity of each of the lighting devices 40a, 40b, 40c, and 40d so that the difference in brightness information between the images is equal to or less than the third threshold value. (Step S34).

  When the intensity is determined in step S34, the processor 22 transmits intensity information indicating the intensity to the lighting devices 40a, 40b, 40c, and 40d (step S35).

  When the intensity information is transmitted in step S35, the illuminating devices 40a, 40b, 40c, and 40d, based on the intensity information, at least part of the imaging ranges of the cameras 10a, 10b, 10c, and 10d, respectively, with near infrared light. Irradiate (step S36).

  In a state where at least a part of the imaging ranges of the cameras 10a, 10b, 10c, and 10d is irradiated with near-infrared light in step S36, the cameras 10a, 10b, 10c, and 10d again detect subjects around the moving body 4. Images are taken to generate images Ia2, Ib2, Ic2, and Id2 (step S37). Due to the processing in steps S33 to S36 described above, the difference in brightness information between the images Ia2, Ib2, Ic2, and Id2 generated here is equal to or smaller than the third threshold value.

  When the images Ia2, Ib2, Ic2, and Id2 are generated in step S37, the acquisition unit 21 acquires the generated images Ia2, Ib2, Ic2, and Id2 (step S38).

  When the images Ia2, Ib2, Ic2, and Id2 are acquired, the processor 22 performs the same processing as steps S15 to S20 of the first embodiment in steps S39 to S44.

  As described above, in the second embodiment, the saturation of any of the plurality of images captured by the plurality of cameras 10a, 10b, 10c, and 10d is reduced by near infrared light. In addition, since the saturation of the pixels constituting at least one image is reduced based on the saturation information of each of the plurality of images, the saturation information of all the images can be set to the same level. Therefore, when a synthesized image is generated by synthesizing these images, the same effect as in the first embodiment can be obtained, in which the reference can refer to the image without feeling a difference in color for each region. .

  In the second embodiment, the imaging ranges of the plurality of cameras 10a, 10b, 10c, and 10d are based on the brightness information of the plurality of images captured by the plurality of cameras 10a, 10b, 10c, and 10d, respectively. The lighting devices 40a, 40b, 40c, and 40d are controlled so that the brightness is approximately the same. Therefore, at night or in the evening, when the lighting devices 40a, 40b, 40c, and 40d are irradiated with only a part of the imaging range of the plurality of cameras 10a, 10b, 10c, and 10d by street lights or the like, the lighting devices 40a, 40b , 40c and 40d are controlled, and the plurality of images taken by the plurality of cameras 10a, 10b, 10c, and 10d have the same brightness information. In such a plurality of images, if the saturation information is the same level, when the synthesized image is generated by synthesizing these images, the person who refers to the synthesized image has both brightness and vividness. An image can be referred to without feeling a difference for each region.

  Subsequently, a third embodiment of the present invention will be described.

  The image processing system 3 according to the third embodiment is mounted on the moving body 4 in the same manner as the image processing system 1 according to the first embodiment, and the imaging devices (cameras) 10a, 10b, 10c, and 10d, and the image processing device. 20 and the display device 30 and the like.

  The processor 22 in the first embodiment matches the saturation information of the second image different from the first image to the first image having the lowest saturation information among the images Ia, Ib, Ic, and Id. Reduce saturation. The processor 22 according to the third embodiment, when one or more of the saturation information Spa, Spb, Spc, and Spd of each of the plurality of images is equal to or less than a predetermined threshold (second threshold), the image Ia , Ib, Ic, and Id are reduced in saturation so that the pixels constituting each image become monochrome images. In this respect, the processor 22 in the third embodiment is different from the processor 22 in the first embodiment.

  Since other configurations and operations in the third embodiment are the same as those in the first embodiment, the same or corresponding components are denoted by the same reference numerals and description thereof is omitted.

  As described above, in the third embodiment, even when any one of a plurality of images captured by the cameras 10a, 10b, 10c, and 10d is reduced in saturation by near-infrared light. Based on the saturation information Spa, Spb, Spc, Spd of each of the plurality of images, the saturation of the pixels constituting at least one image is reduced. Therefore, the saturation information of all the images can be set to the same level. Therefore, when a synthesized image is generated by synthesizing these images, an effect similar to that of the first embodiment in which a person who refers to the synthesized image can refer to the image with less discomfort. .

  In the third embodiment, any one or more of the saturation information Spa, Spb, Spc, Spd of each of the plurality of images Ia, Ib, Ic, Id is a predetermined threshold (second threshold). In the following cases, all of the plurality of images captured by the cameras 10a, 10b, 10c, and 10d are all monochrome images. Therefore, the difference in the saturation information of a plurality of images can be further reduced. Therefore, when a synthesized image is generated by synthesizing these images, a person who refers to the synthesized image can further reduce the sense of discomfort of the referring person due to a difference in saturation.

  Although the above embodiment has been described as a representative example, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments and examples, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of constituent blocks described in the embodiments and examples can be combined into one, or one constituent block can be divided.

  In the above-described embodiment, the image processing system 1 includes the four cameras 10a, 10b, 10c, and 10d. However, the number of the cameras 10 is not limited to four, and may be two, three, or five or more. I just need it. That is, the image processing system 1 can include a plurality of cameras.

DESCRIPTION OF SYMBOLS 1 Image processing system 4 Moving body 10,10a, 10b, 10c, 10d Camera 11,11a, 11b, 11c, 11d Imaging lens 12,12a, 12b, 12c, 12d Filter 13,13a, 13b, 13c, 13d Image pick-up element 14 , 14a, 14b, 14c, 14d Signal processing unit 15, 15a, 15b, 15c, 15d Communication interface 20 Image processing device 21 Acquisition unit 22 Processor 23 Memory 24 Communication interface 30 Display device 31 Communication interface 32 Display processor 33 Display panel 40 , 40a, 40b, 40c, 40d Illumination device 41, 41a, 41b, 41c, 41d Communication interface 42, 42a, 42b, 42c, 42d Irradiation control unit 43, 43a, 43b, 43c, 43d Light emitting unit

Claims (13)

An acquisition unit for acquiring a plurality of images respectively captured by a plurality of cameras including a filter that transmits visible light and selectively transmits infrared light in a predetermined wavelength range;
The saturation information of each of the plurality of images acquired by the acquisition unit is calculated, and the saturation of pixels constituting at least one of the plurality of images is reduced based on the respective saturation information. A processor;
An image processing apparatus comprising:   The processor is configured based on a difference between saturation information of a first image of the plurality of images and saturation information of a second image different from the first image among the plurality of images. The image processing apparatus according to claim 1, wherein the saturation of the pixels constituting the image having the higher saturation information among the first image and the second image is reduced.   When the difference between the saturation information of the first image and the saturation information of the second image is greater than a first threshold having a predetermined value, the processor is configured to output the first image and the second image. The image processing apparatus according to claim 2, wherein the saturation of the pixel having the higher saturation information in the image is reduced.   The processor includes: an image obtained by lowering a saturation of a pixel constituting an image having a high saturation information among the plurality of images; and an image having a low saturation information among the plurality of images. The image processing apparatus according to claim 3, wherein the saturation of the pixels constituting the image with the high saturation information is reduced so that the difference in information is equal to or less than the first threshold.   The processor, when any one or more of the saturation information of each of the plurality of images is equal to or less than a second threshold having a predetermined value, so that all of the plurality of images become monochrome images. The image processing apparatus according to claim 1, wherein the saturation of pixels constituting each of the plurality of images is reduced.   The processor adjusts the difference between the brightness information of the plurality of images acquired by the acquisition unit to be equal to or less than a third threshold value having a predetermined value, and colors the plurality of images for which the brightness information is adjusted. The image processing apparatus according to claim 1, wherein the degree information is calculated. A plurality of cameras including a filter that transmits visible light and selectively transmits infrared light in a predetermined wavelength range;
An acquisition unit for acquiring a plurality of images respectively captured by the plurality of cameras, and calculating saturation information of each of the plurality of images acquired by the acquisition unit, and based on the respective saturation information, A processor for reducing the saturation of pixels constituting at least one of the plurality of images;
An image processing system comprising: A plurality of illumination devices that respectively illuminate the imaging ranges of the plurality of cameras;
The image processing system according to claim 7, wherein the processor calculates brightness information of each of the plurality of images acquired by the acquisition unit, and controls the plurality of lighting devices based on the brightness information.   The processor determines whether a difference in brightness information of each of the plurality of images is equal to or less than a first threshold having a predetermined value, and if the difference is greater than the first threshold, the difference is the first threshold. When at least one illumination device of the plurality of illumination devices is controlled to irradiate the imaging range so as to be less than or equal to a threshold value of 1, and when the difference is less than or equal to the first threshold value, The image processing system according to claim 8, wherein the image is controlled not to irradiate the imaging range.   The image processing system according to claim 8, wherein the illumination device emits light including infrared light in a predetermined wavelength range.   When the saturation of the pixel is decreased, the processor generates a combined image by combining the image with the decreased saturation and an image different from the at least one image among the plurality of images. The image processing system according to claim 7. A plurality of cameras having filters that transmit visible light and selectively transmit infrared light in a predetermined wavelength range;
An acquisition unit for acquiring a plurality of images respectively captured by the plurality of cameras, and calculating saturation information of each of the plurality of images acquired by the acquisition unit, and based on the respective saturation information, A moving object provided with an image processing system including an image processing device which has a processor which reduces the saturation of a pixel which constitutes at least one picture among a plurality of pictures. An image processing method executed by an image processing apparatus,
The image processing apparatus acquires a plurality of images respectively captured by a plurality of cameras including filters that transmit visible light and selectively transmit infrared light in a predetermined wavelength range;
The image processing device calculates the saturation information of each of the acquired plurality of images, and based on each of the saturation information, the saturation of pixels constituting at least one image of the plurality of images Reduce,
Image processing method.

Images