WO2015064095A1

WO2015064095A1 - Image correction parameter output device, camera system, and correction parameter output method

Info

Publication number: WO2015064095A1
Application number: PCT/JP2014/005471
Authority: WO
Inventors: 崇功金谷; 理洋森島; 貴裕岡田
Original assignee: 京セラ株式会社
Priority date: 2013-10-29
Filing date: 2014-10-29
Publication date: 2015-05-07
Also published as: EP3065390B1; EP3065390A1; EP3065390A4; US10097733B2; US20160269597A1; JPWO2015064095A1

Abstract

　The present invention outputs image correction parameters capable of improving the visibility of a composite image of the surrounding area of a moving body.　An image correction parameter output device (22) is provided with: a storage unit (24) that stores, in association with control information for a moving body (15), a correction parameter group comprising a plurality patterns for correcting a plurality of images to be captured after partially superimposing regions of the surrounding area of the moving body (15); a control information acquisition unit (23) that acquires the control information for the moving body (15); and an output unit (25) that outputs the correction parameter group associated with the acquired control information.

Description

Image correction parameter output device, camera system, and correction parameter output method

Cross-reference to related applications

This application claims the priority of Japanese Patent Application No. 2013-224508 (filed on October 29, 2013), the entire disclosure of which is incorporated herein by reference.

The present invention relates to an image correction parameter output device, a camera system, and a correction parameter output method for outputting parameters used for correcting a captured image.

2. Description of the Related Art Conventionally, there is known an around view monitor system in which a plurality of in-vehicle cameras are installed on a moving body such as an automobile, the periphery of the moving body is imaged, and a combined image overlooking the periphery of the moving body is displayed using the generated captured images. ing. In such a system, a technique is known in which visual continuity is provided at a joint between captured images (for example, Patent Document 1).

JP 2010-116196 A

In order to ensure the visibility of the image, it is common to correct the color and brightness of the image using predetermined correction parameters. However, the visibility of the combined image may be lowered depending on the environment around the moving body and the state of the moving body. For example, the value of an appropriate correction parameter that can ensure visibility for various light sources around the moving body is not necessarily constant. Thus, the combined image may not be able to ensure sufficient visibility for various states of the moving object.

An object of the present invention made in view of such circumstances is to provide an image correction parameter output device, a camera system, and a correction parameter output method for outputting an image correction parameter capable of improving the visibility of a combined image around a moving object. There is to do.

In order to solve the above-described problems, an image correction parameter output apparatus according to the present invention includes:
A storage unit for storing correction parameter groups of a plurality of patterns for correcting a plurality of images captured by partially overlapping a peripheral area of the moving object, in association with the control information of the moving object;
A control information acquisition unit for acquiring control information of the mobile body;
And an output unit that outputs a correction parameter group corresponding to the acquired control information.

In the correction parameter output apparatus according to the present invention,
The output unit preferably outputs different correction parameter groups depending on whether or not the control information includes information indicating lighting of the moving body.

In the correction parameter output apparatus according to the present invention,
The correction parameter group preferably includes a color correction parameter for correcting so as to reduce a color difference between the plurality of images.

The correction parameter output apparatus according to the present invention is
An image acquisition unit for acquiring a plurality of images corrected based on the correction parameter group output by the output unit;
One of the plurality of corrected images is defined as a reference image, and the other corrected image is adjusted based on a color signal component in an overlapping area between the reference image and another corrected image. An adjustment parameter calculation unit for calculating the color adjustment parameter of
The output unit preferably outputs the calculated color adjustment parameter.

In the correction parameter output apparatus according to the present invention,
The image acquisition unit acquires an image adjusted based on the color adjustment parameter output by the output unit,
The adjustment parameter calculation unit calculates a color adjustment parameter for adjusting the corrected image based on a color signal component in an overlapping area between the adjusted image and a corrected image different from the reference image. Calculate
The output unit preferably outputs the calculated color adjustment parameter.

In the correction parameter output apparatus according to the present invention,
The reference image is preferably determined based on the control information from among the plurality of corrected images.

In the correction parameter output apparatus according to the present invention,
The reference image is preferably determined based on information indicating a traveling direction of the moving body included in the control information from among the plurality of corrected images.

Moreover, the camera system according to the present invention includes:
A plurality of imaging units that generate a plurality of images that partially image the peripheral area of the moving body;
A storage unit that stores a plurality of pattern correction parameter groups for correcting the plurality of images in association with the control information of the moving body;
A control information acquisition unit for acquiring control information of the mobile body;
An output unit that outputs a correction parameter group corresponding to the acquired control information; an image processing unit that corrects the plurality of images based on a correction parameter output by the output unit;
And an image combining unit that combines the plurality of corrected images to generate a combined image.

The image correction parameter output method according to the present invention includes:
Storing a correction parameter group of a plurality of patterns for correcting a plurality of images captured by partially overlapping a peripheral area of the moving body in association with the control information of the moving body;
Obtaining control information of the mobile body;
And a step of outputting a correction parameter group corresponding to the acquired control information.

According to the image correction parameter output device, the camera system, and the correction parameter output method according to the present invention, it is possible to output an image correction parameter that improves the visibility of the combined image around the moving body.

It is a functional block diagram which shows schematic structure of the camera system which concerns on the 1st Embodiment of this invention. It is the schematic which shows arrangement | positioning of the component of the camera system of FIG. It is the schematic which shows the imaging range of the imaging device of FIG. It is a figure which shows the example of the correction parameter group which the memory | storage part of FIG. 1 memorize | stores. It is a flowchart explaining operation | movement of the camera system of FIG. It is a figure which shows the example of the joint image around a mobile body. It is a functional block diagram which shows schematic structure of the camera system which concerns on the 2nd Embodiment of this invention. It is a flowchart explaining operation | movement of the camera system of FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
First, the correction parameter output device and camera system according to the first embodiment of the present invention will be described. FIG. 1 is a functional block diagram showing a schematic configuration of a camera system according to the first embodiment of the present invention.

As shown in FIG. 1, the camera system 100 includes an imaging device 10 and a display device 11. The imaging device 10 includes a plurality of imaging units, in the present embodiment, for example, a front camera 12, a rear camera 13, and a side camera 14 (a left side camera 14L and a right side camera 14R).

As shown in FIG. 2, the display device 11 is disposed at a position that can be viewed from the driver's seat. The front camera 12 is arranged so as to be able to image a peripheral region in front of the moving body 15. The rear camera 13 is arranged so as to be able to image a peripheral region behind the moving body 15. The side cameras 14 are arranged so that, for example, the left and right door mirrors 16 can vertically image the peripheral regions on the sides of the moving body 15 downward. Further, the side cameras 14 are symmetrically arranged on both the left and right sides of the moving body 15.

The front camera 12, the rear camera 13, and the side camera 14 are provided with a lens having a wide angle of view such as a fisheye lens, for example, and a peripheral region of the moving body 15 can be photographed at a wide angle. As shown in FIG. 3, the imaging range of the front camera 12 includes a front area FA of the moving body 15. The imaging range of the rear camera 13 includes a rear area ReA of the moving body 15. The imaging range of the left side camera 14L and the imaging range of the right side camera 14R include a left side area LA and a right side area RA of the moving body 15, respectively.

The imaging ranges of the

cameras

12, 13, 14L, and 14R include regions around the four corners of the moving body 15 overlapping each other. Specifically, the imaging range of the front camera 12 and the imaging range of the side cameras 14 (14L, 14R) include the left front area FLA and the right front area FRA of the moving body 15 overlapping each other. Further, the imaging range of the rear camera 13 and the imaging range of the side cameras 14 (14L, 14R) include the left rear region ReLA and the right rear region ReRA of the moving body 15 overlapping each other. Hereinafter, the area around the moving body 15 where the imaging ranges of the

cameras

12, 13, 14L, and 14R overlap each other is referred to as an overlapping area (FLA, FRA, ReLA, ReRA).

Next, the configuration of the imaging unit (front camera 12, rear camera 13, side camera 14) will be described. The front camera 12 includes an optical system 17a, an imager 18a, an image processing unit 19a, a camera control unit 20a, an image combining device (image combining unit) 21, and a correction parameter output device 22 (see FIG. 1). ).

The optical system 17a includes a plurality of lenses and forms a subject image. In the present embodiment, the optical system 17a has a wide angle of view, and can form a subject image included in the peripheral region of the moving body 15 as described above.

The imager 18a is, for example, a CMOS image sensor, and generates an image obtained by capturing a subject image formed by the optical system 17a.

The image processing unit 19a performs image processing such as image conversion, color correction, gamma correction, and luminance correction on the image generated by the imager 18a. The image processing unit 19a outputs an image subjected to image processing.

The image processing unit 19a converts the wide-angle captured image generated by the imager 18a into an overhead image by image conversion. That is, a captured image generated by wide-angle imaging and generally distorted in the peripheral portion of the image is converted into an overhead image when the peripheral area of the moving body 15 is viewed from above the moving body 15 vertically downward. Specifically, the image processing unit 19a converts the image captured by the imager 18a into an overhead image in the front area FA and the overlapping areas FLA and FRA (see FIG. 3) of the moving body 15.

The image processing unit 19a corrects the color of the captured image or the overhead image by color correction. For color correction, the image processing unit 19 a acquires color correction parameters from the correction parameter output device 22. For example, the image processing unit 19a performs color correction by multiplying a specific color signal component in the captured image or the overhead image by the acquired color correction parameter.

The image processing unit 19a corrects the nonlinearity of the input signal versus the emission intensity of the display device 11 by, for example, normal gamma correction.

The image processing unit 19a corrects the brightness of the captured image or the overhead image by brightness correction. For brightness correction, the image processing unit 19 a acquires a brightness correction parameter from the correction parameter output device 22. For example, the image processing unit 19a performs luminance correction by multiplying the luminance signal component in the captured image or the overhead image by the acquired luminance correction parameter.

The camera control unit 20a (see FIG. 1) controls the operation of each part of the front camera 12. For example, the camera control unit 20a causes the imager 18a to capture the peripheral region of the moving body 15 in synchronization with the rear camera 13 and the side camera 14, and periodically generate an image at, for example, 30 fps. The camera control unit 20a transmits and receives information via the in-vehicle network 101 or a dedicated line.

The image combining device 21 combines images output from the

image processing units

19a, 19b, 19c, and 19d of the front camera 12, the rear camera 13, and the

side cameras

14L and 14R to generate a combined image. The combined image is, for example, an overhead image of the entire periphery of the moving body 15. In the bird's-eye view of the entire periphery in this embodiment, the front area FA and the overlapping areas FLA and FRA of the moving body are images of the front camera 12, the rear area ReA and the overlapping areas ReLA and ReRA are images of the rear camera 13, and the left and right sides. The images of the

side cameras

14L and 14R are used for the direction areas LA and RA, respectively (see FIG. 3). In addition, the image combining device 21 outputs the generated combined image to the display device 11.

The correction parameter output device 22 (see FIG. 1) includes a control information acquisition unit 23, a storage unit 24, an output unit 25, and a control unit 26.

The control information acquisition unit 23 acquires control information of the moving body 15. In the present embodiment, the control information includes various information regarding the time information and the state of the moving body 15. The variety of information related to the state of the moving body 15 includes, for example, information indicating whether the lighting (head lamp, tail lamp, and brake lamp) of the moving body 15 is turned on or off. The control information acquisition unit 23 can acquire control information by an arbitrary method. For example, the control information acquisition unit 23 may acquire the control information from the mobile body 15 via the in-vehicle network 101, or control information output by other components of the mobile body 15 May be acquired by wire / wireless.

The storage unit 24 stores a plurality of pattern correction parameter groups respectively associated with various control information. The correction parameter group is determined in advance through experiments or simulations.

For example, as shown in pattern 1 in FIG. 4, when the state is “daytime”, the correction parameters (color correction parameter and luminance correction parameter) of each

camera

12, 13, 14L, 14R included in the corresponding correction parameter group are as follows. Are the same. Regarding the determination of the state, if the time information included in the control information indicates “7 o'clock to 17 o'clock”, it is determined that the state is “daytime”.

In the daytime, various subjects in the shooting ranges of the

cameras

12, 13, 14L, and 14R are often illuminated by the same light source such as sunlight. Therefore, when a common value (reference value) for executing normal white balance is used as each correction parameter, the colors and luminances of the images captured by the

cameras

12, 13, 14L, and 14R are equal on average. It is adjusted so that it can be visually recognized.

On the other hand, for example, as shown in the pattern 2 of FIG. 4, when the state is “night (headlamp ON)”, the correction parameter of the front camera 12 is a reference value. In the correction parameters of the rear camera 13 and the

side cameras

14L and 14R, the color correction parameter is determined to increase the saturation of the image, and the luminance correction parameter is determined to increase the luminance of the image. Regarding the determination of the state, when the time information included in the control information indicates “17:00 to 7:00” and the control information includes information indicating the lighting of the headlamp (white), the state is “night (headlamp ON ) ”.

In the case of night, when the headlamp is irradiated only in front of the moving body 15, the front subject has higher brightness and saturation on the image than the left and right rear subjects. Therefore, by using the correction parameters as described above, adjustment is performed so that the images captured by the

cameras

12, 13, 14L, and 14R can be visually recognized as having the same color and brightness on average.

For example, as shown in pattern 3 of FIG. 4, when the state is “night (head / tail lamp ON)”, the correction parameter of the front camera 12 is a reference value. In the correction parameters of the rear camera 13, the color correction parameter is determined to reduce (weaken) the redness of the image, and the luminance correction parameter is determined to increase the luminance of the image. In the correction parameters of the

side cameras

14L and 14R, the color correction parameter is determined to increase the saturation of the image, and the luminance correction parameter is determined to increase the luminance of the image. Regarding the determination of the state, when the time information included in the control information indicates “17:00 to 7:00” and the control information includes information indicating lighting of the head lamp (white) and the tail lamp (red), the state is “ It is determined that it is “night (head / tail lamp ON)”.

In the case of night, when the red tail lamp is irradiated only to the rear of the moving body 15, the rear subject is captured as an image that is more reddish than the left and right front subjects. Therefore, by using the correction parameters as described above, adjustment is performed so that the images captured by the

cameras

For example, as shown in pattern 4 of FIG. 4, when the state is “night (head brake lamp ON)”, the correction parameter of the front camera 12 is a reference value. In the correction parameters of the rear camera 13, the color correction parameter is determined to reduce (strong) redness of the image, and the luminance correction parameter is determined to increase the luminance of the image. In the correction parameters of the

side cameras

14L and 14R, the color correction parameter is determined to increase the saturation of the image, and the luminance correction parameter is determined to increase the luminance of the image. Regarding the state determination, when the time information included in the control information indicates “17:00 to 7:00” and the control information includes information indicating lighting of the headlamp (white) and the brake lamp (red), the state is It is determined that it is “Night (head brake lamp ON)”.

In the case of night, when the red brake lamp is irradiated only behind the moving body 15, the rear subject is captured as an image that is more reddish than the left and right front subjects. In general, the brake lamp is brighter than the tail lamp and red is stronger. Therefore, by using the correction parameters as described above, the color and brightness that can be visually recognized when the images taken by the

cameras

12, 13, 14L, and 14R are equal on average are adjusted.

For example, as shown in the pattern 5 in FIG. 4, when the state is “night (lighting OFF)”, the color correction parameter is an image among the correction parameters of the front camera 12, the rear camera 13, and the

side cameras

14L and 14R. The luminance correction parameter is determined to increase the luminance of the image. Regarding the determination of the state, when the time information included in the control information indicates “17:00 to 7:00”, it is determined that the state is “night (lighting OFF)”.

In the case of night, when all the lights of the moving body 15 are turned off, subjects around the moving body 15 have lower brightness and saturation on the image than in the daytime. Therefore, by using the correction parameters as described above, adjustment is performed so that the images captured by the

cameras

The output unit 25 (see FIG. 1) converts the correction parameters included in the correction parameter group corresponding to the control information acquired by the control information acquisition unit 23 into the

image processing units

19a, 19b, and 14R of the

cameras

12, 13, 14L, and 14R. It outputs to 19c and 19d.

The control unit 26 controls the operation of each part of the correction parameter output device 22. For example, the control unit 26 causes the control information acquisition unit 23 to acquire the control information of the moving body 15 simultaneously with the image generation by the

imagers

18a, 18b, 18c, and 18d of the

cameras

12, 13, 14L, and 14R, and causes the output unit 25 to The correction parameter is output periodically. Moreover, the control part 26 transmits / receives information via the vehicle-mounted network 101 or a dedicated line.

As with the front camera 12, the rear camera 13 and the side cameras 14 (14L, 14R) (see FIG. 1) are

optical systems

17b, 17c, 17d,

imagers

18b, 18c, 18d,

image processing units

19b, 19c, 19d, And

camera control units

20b, 20c, and 20d. The functions and configurations of the

optical systems

17b, 17c, 17d,

imagers

18b, 18c, 18d,

image processing units

19b, 19c, 19d, and

camera control units

20b, 20c, 20d are the same as those of the front camera 12.

For example, the image processing unit 19b of the rear camera 13 converts an image captured by the imager 18b of the rear camera 13 into an overhead view image in the rear area ReA and the overlapping areas ReLA and ReRA. In addition, the image processing unit 19c of the left side camera 14L converts the image captured by the imager 18c of the left side camera 14L into an overhead view image in the left side area LA and the overlapping areas FLA and ReLA. In addition, the image processing unit 19d of the right side camera 14R converts the image captured by the imager 18d of the right side camera 14 into an overhead view image in the right side area RA and the overlapping areas FRA and ReRA.

The display device 11 is, for example, an LCD and can display a real-time moving image. The display device 11 acquires and displays the combined image output from the image combining device 21. The display device 11 may be configured by a touch panel, for example, and may function as an interface that receives user operations. The display device 11 can transmit and receive information via the in-vehicle network 101 or a dedicated line.

Next, processing executed by the camera system 100 according to the first embodiment will be described with reference to the flowchart of FIG. This process is started, for example, when the imaging apparatus 10 is operated, and is repeatedly executed until an end instruction is given by the user.

First, the

camera control units

20a, 20b, 20c, and 20d of the

cameras

12, 13, 14L, and 14R control the

imagers

18a, 18b, 18c, and 18d to generate an image that captures the peripheral area of the moving body 15. (Step S100).

Next, the control unit 26 of the correction parameter output device 22 controls the control information acquisition unit 23 to acquire control information when the captured images are captured by the

cameras

12, 13, 14L, and 14R (step S101). The control information includes time information and information indicating lighting of the moving body 15 (head lamp, tail lamp, and brake lamp).

Subsequently, the control unit 26 reads out a correction parameter group corresponding to the control information in step S101 from the correction parameter groups of a plurality of patterns stored in the storage unit 24 (step S102), and sets the correction parameter to each camera 12, Output to 13, 14L, 14R.

Next, the

camera control units

20a, 20b, 20c, and 20d of the

cameras

12, 13, 14L, and 14R control the image processing unit 19 to convert the images generated in step S100 into overhead images (step S103). ).

Subsequently, each of the

camera control units

20a, 20b, 20c, and 20d controls the

image processing units

19a, 19b, 19c, and 19d to correct the bird's-eye view image in step S103 based on the correction parameter in step S102 ( In step S104, the corrected overhead image is output to the image combining device 21.

Next, the image combining device 21 generates a combined image of the plurality of corrected overhead images in step S104 (step S105) and outputs the combined image to the display device 11.

Then, the display device 11 displays the combined image of step S105 (step S106).

As described above, according to the image correction parameter output apparatus of the first embodiment, the correction parameters for correcting a plurality of images according to the control information of the moving body 15 are output. For this reason, as described below, correction suitable for various states of the moving body 15 can be performed, and the visibility of the combined image can be improved.

For example, when the headlamp is turned on at night, white light from the headlamp is irradiated in front of the moving body 15, so that the subject images in front of the moving body 15 are left, right, and rear subjects. Whiter than the statue. Therefore, as shown in FIG. 6A, the left and right regions on the image FIm by the front camera 12 and the images LIm and RIm by the side camera 14 are different in average color and luminance. Similarly, when the tail lamp or the brake lamp is lit at night, the red light of the tail lamp or the brake lamp is irradiated to the rear of the moving body 15, so that the subject image behind the moving body 15 is left, right , And more reddish than the front subject image. Therefore, the entire image ReIm from the rear camera 13 and the images LIm and RIm from the side camera 14 are different in average color and luminance.

The correction parameter output device of the first embodiment reads out and outputs a correction parameter group corresponding to control information of the moving body 15 from a plurality of correction parameter groups stored in advance in the storage unit 24. In this way, the

image processing units

19a, 19b, 19c, and 19d of the

cameras

12, 13, 14L, and 14R can visually recognize that the captured images are equal on average with respect to various states of the moving body 15. Color and luminance can be corrected (see FIG. 6B).

In the first embodiment, different correction parameters are output depending on whether or not the control information includes information indicating lighting of the moving body 15. For this reason, the difference of the color and brightness | luminance of each image by the presence or absence of lighting of the moving body 15 can be reduced, and the visibility of a combined image can further be improved.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. The configuration of the camera system 100 according to the second embodiment is the same as that of the first embodiment, but the function of the image processing unit and the configuration of the correction parameter output device are different.

The

image processing units

19a, 19b, 19c, and 19d (see FIG. 7) perform image conversion, color correction, luminance correction, and the like on the images generated by the

imagers

18a, 18b, 18c, and 18d, as in the first embodiment. The image processing is performed. The

image processing units

19 a, 19 b, 19 c, and 19 d according to the present embodiment acquire adjustment parameters that will be described later from the correction parameter output device 220. For example, the

image processing units

19a, 19b, 19c, and 19d perform image adjustment by multiplying the color signal component of the image by the acquired color adjustment parameter.

The correction parameter output device 220 according to the present embodiment includes a control information acquisition unit 23, a storage unit 24, an output unit 25, a control unit 26, an image acquisition unit 270, and an adjustment parameter calculation unit 280. The configurations and functions of the control information acquisition unit 23, the storage unit 24, and the control unit 26 are the same as those in the first embodiment.

Similarly to the first embodiment, the output unit 25 uses the correction parameters included in the correction parameter group corresponding to the control information acquired by the control information acquisition unit 23 as image processing units of the

cameras

12, 13, 14L, and 14R. It outputs to 19a, 19b, 19c, 19d. Further, the output unit 25 in the present embodiment outputs the adjustment parameters calculated by the adjustment parameter calculation unit 280 to the

image processing units

19a, 19b, 19c, and 19d of the

cameras

12, 13, 14L, and 14R.

The image acquisition unit 270 acquires images from the

image processing units

19a, 19b, 19c, and 19d of the

cameras

12, 13, 14L, and 14R.

The adjustment parameter calculation unit 280 calculates an adjustment parameter for adjusting a plurality of images acquired by the image acquisition unit 270. For example, the adjustment parameter calculation unit 280 calculates the adjustment parameter by a two-stage process as described below.

First, the first stage process will be described. The adjustment parameter calculation unit 280 determines one of the plurality of images acquired by the image acquisition unit 270 as a reference image. Here, it is assumed that an image from the front camera 12 is set as a reference image.

The adjustment parameter calculation unit 280 calculates the average value of the color signal components of the image in each overlapping region (FLA, FRA) for the reference image and adjacent images (images by the

side cameras

14L and 14R) each including an overlapping region common to the reference image. Are calculated respectively.

The adjustment parameter calculation unit 280, when the difference in color signal component average value between the reference image and the adjacent image is greater than or equal to a predetermined threshold, that is, when the average color difference between the reference image and the adjacent image is large, A color adjustment parameter to be multiplied by the color signal component of the adjacent image is calculated so as to reduce the color difference. For example, the adjustment parameter calculation unit 280 calculates the color adjustment parameter of the adjacent image so that the difference between the average values is less than a predetermined threshold or the average value of the adjacent image matches the average value of the reference image. .

Then, the adjustment parameter calculation unit 280 outputs the color adjustment parameters calculated via the output unit 25 to the

side cameras

14L and 14R.

When the image acquisition unit 270 acquires images from the

side cameras

14L and 14R adjusted based on the color adjustment parameters calculated as described above, the adjustment parameter calculation unit 280 performs a second stage process described below.

The adjustment parameter calculation unit 280 performs the overlap regions (ReLA, ReRA) on the images by the

side cameras

14L and 14R and the third image (image by the rear camera 13) that includes an overlap region common to the images and is different from the reference image. The average values of the color signal components of the image at are respectively calculated.

The adjustment parameter calculation unit 280 calculates the average value A of the color signal component average value of the image by the left side camera 14L and the color signal component average value of the image by the right side camera 14R. Further, the adjustment parameter calculation unit 280 calculates an average value B of the color signal component average value of the image by the rear camera 13 in the overlap region ReLA and the color signal component average value of the image by the rear camera 13 in the overlap region ReRA. .

When the difference between the average value A and the average value B is equal to or greater than a predetermined threshold, the adjustment parameter calculation unit 280, for example, sets the average value B to the average value A so that the difference between the average values is less than the predetermined threshold. The color adjustment parameters of the image by the rear camera 13 are calculated so as to coincide with each other.

Then, the adjustment parameter calculation unit 280 outputs the color adjustment parameter calculated via the output unit 25 to the rear camera 13.

Preferably, the adjustment parameter calculation unit 280 determines which of the plurality of images acquired by the image acquisition unit 270 is determined as the above-described reference image based on the control information acquired by the control information acquisition unit 23. To do. Here, the control information includes information indicating the traveling direction of the moving body 15. The adjustment parameter calculation unit 280 determines the traveling direction of the moving body 15 based on the control information. For example, when moving forward, the front camera 12 image, when moving backward, the rear camera 13 image, when turning left or right, the side camera 14 (14L or 14R) is determined as a reference image.

Next, processing executed by the camera system 100 according to the second embodiment will be described with reference to the flowchart of FIG. This process is started, for example, when the imaging apparatus 10 is operated, and is repeatedly executed until an end instruction is given by the user.

From step S200 to step S203, processing similar to that from step S100 to step S103 in the first embodiment is performed.

Subsequently, each of the

camera control units

20a, 20b, 20c, and 20d controls the image processing unit 19 to correct the bird's-eye view image in step S203 based on the correction parameter in step S202 (step S204). The overhead image is output to the correction parameter output device 220.

Next, based on the control information in step S200, the correction parameter output device 220 determines one of the corrected overhead images in step S204 as a reference image (step S205). Here, an image obtained by the front camera 12 is defined as a reference image.

Subsequently, the correction parameter output device 220 adjusts the images by the

side cameras

14L and 14R based on the reference image and the color signal components of the images by the

side cameras

14L and 14R among the plurality of overhead images corrected in step S204. Are calculated (step S206) and output to the

side cameras

14L and 14R.

Next, the

side cameras

14L and 14R adjust the images by the

side cameras

14L and 14R corrected in step S204 based on the color adjustment parameters in step S206 (step S207), and the adjusted images are sent to the correction parameter output device 220. Output.

Next, the correction parameter output device 220 adjusts the image by the rear camera 13 based on the color signal component of the image by the

side camera

14L, 14R adjusted in step S207 and the image by the rear camera 13 corrected in step S204. Are calculated (step S208) and output to the rear camera 13.

Next, the rear camera 13 adjusts the image by the rear camera 13 corrected in step S204 based on the color adjustment parameter in step S208 (step S209).

Subsequently, each of the

cameras

12, 13, 14L, and 14R outputs the corrected or adjusted overhead image to the correction parameter output device 22 (step S210). Specifically, the image by the front camera 12 corrected in step S204, the image by the

side cameras

14L and 14R adjusted in step S207, and the image by the rear camera 13 adjusted in step S209 are output.

Next, the image combining device 21 generates a combined image based on the overhead image in step S210 (step S211) and outputs the combined image to the display device 11.

Then, the display device 11 displays the combined image in step S211 (step S212).

As described above, according to the camera system of the second embodiment, the color signal component of the reference image is corrected after correcting the images by the

cameras

12, 13, 14L, and 14R as in the camera system of the first embodiment. Since the images from the

cameras

12, 13, 14L, and 14R are adjusted with reference to, the visibility of the combined image can be improved.

In the second embodiment, for example, since the reference image is determined according to the traveling direction of the moving body 15, the color of the other image is matched with the color of the image including the peripheral area of the moving body 15 that the driver pays attention to. And the visibility of the combined image can be further improved.

Although the present invention has been described based on the drawings and embodiments, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these variations and modifications are included in the scope of the present invention.

For example, in the above-described embodiment, the imaging device 10 includes the front camera 12, the rear camera 13, and the side camera 14, but may include more cameras. For example, a configuration further including a distant camera that can capture the entire periphery of the moving body 15 may be used.

In addition, each component of the camera system of the above-described embodiment can be divided and rearranged. For example, the image combining device 21 and the correction parameter output device 22 may be divided from the front camera 12 and configured as a single device. Further, for example, a navigation system may be further provided, and the image combining device 21 and the correction parameter output device 22 may be provided in the navigation system.

In the above-described embodiment, the correction parameter may be any other parameter related to image correction. For example, in the second embodiment, the configuration in which the adjustment parameter calculation unit 280 calculates the color adjustment parameter has been described. However, the luminance adjustment parameter may be calculated by the same process. In this case, for example, the

image processing units

19a, 19b, 19c, and 19d perform image adjustment by multiplying the luminance signal component of the image by the luminance adjustment parameter.

In the above-described embodiment, the control information has been described as including time information, information indicating lighting of the moving body 15, information indicating the traveling direction of the moving body 15, and the like. Information may be included.

In the above-described embodiment, the configuration has been described in which the

image processing units

19a, 19b, 19c, and 19d correct the captured images (or overhead images) generated by the

imagers

18a, 18b, 18c, and 18d. A configuration may be adopted in which correction parameters are input to an AFE (Analog Front End) or a white balance control unit having a color, and the color and luminance are corrected when an image is generated by the

imagers

18a, 18b, 18c, and 18d.

In the above-described embodiment, the luminance correction parameter has been described as a parameter to be multiplied with the luminance signal component of the image, but may be a parameter indicating an aperture value and a shutter speed, for example. In this case, the luminance is adjusted by inputting the luminance correction parameter to the aperture and exposure time control unit. Further, as the brightness correction parameter, a parameter to be multiplied with the brightness signal component of the image and a parameter indicating the aperture value and the shutter speed may be used in combination.

In the second embodiment, the adjustment parameter calculation unit 280 calculates the correction parameter of the image by the rear camera 13 using the color signal component of the image by the left and right side cameras 14. 14 may be configured to calculate the correction parameter using the color signal component of the image of No. 14.

DESCRIPTION OF SYMBOLS 10 Imaging device 11 Display apparatus 12 Front camera 13

Rear camera

14, 14L, 14R Side camera 15 Moving body 16

Door mirror

17a, 17b, 17c,

17d Optical system

18a, 18b, 18c,

18d Imager

19a, 19b, 19c, 19d

Image processing Unit

20a, 20b, 20c, 20d Camera control unit 21

Image combination device

22, 220 Correction parameter output device 23 Control information acquisition unit 24 Storage unit 25 Output unit 26 Control unit 100 Camera system 101 In-vehicle network 270 Image acquisition unit 280 Adjustment parameter calculation Part

Claims

A storage unit for storing correction parameter groups of a plurality of patterns for correcting a plurality of images captured by partially overlapping a peripheral area of the moving object, in association with the control information of the moving object;
A control information acquisition unit for acquiring control information of the mobile body;
An image correction parameter output apparatus comprising: an output unit that outputs a correction parameter group corresponding to the acquired control information.
The correction parameter output apparatus according to claim 1, wherein the output unit outputs different correction parameter groups depending on whether or not the control information includes information indicating lighting of the moving body. A characteristic correction parameter output device.
The correction parameter output apparatus according to claim 1, wherein the correction parameter group includes a color correction parameter for correcting so as to reduce a color difference between the plurality of images. Correction parameter output device.
The correction parameter output device according to claim 1,
An image acquisition unit for acquiring a plurality of images corrected based on the correction parameter group output by the output unit;
One of the plurality of corrected images is defined as a reference image, and the other corrected image is adjusted based on a color signal component in an overlapping area between the reference image and another corrected image. An adjustment parameter calculation unit for calculating the color adjustment parameter of
The correction parameter output apparatus, wherein the output unit outputs the calculated color adjustment parameter.
The correction parameter output device according to claim 4,
The image acquisition unit acquires an image adjusted based on the color adjustment parameter output by the output unit,
The adjustment parameter calculation unit calculates a color adjustment parameter for adjusting the corrected image based on a color signal component in an overlapping area between the adjusted image and a corrected image different from the reference image. Calculate
The correction parameter output apparatus, wherein the output unit outputs the calculated color adjustment parameter.
5. The correction parameter output device according to claim 4, wherein the reference image is determined based on the control information from among the plurality of corrected images.
The correction parameter output device according to claim 6, wherein the reference image is determined based on information indicating a traveling direction of the moving body included in the control information from among the plurality of corrected images. A correction parameter output device.
A plurality of imaging units that generate a plurality of images that partially image the peripheral area of the moving body;
A storage unit that stores a plurality of pattern correction parameter groups for correcting the plurality of images in association with the control information of the moving body;
A control information acquisition unit for acquiring control information of the mobile body;
An output unit that outputs a correction parameter group corresponding to the acquired control information; an image processing unit that corrects the plurality of images based on a correction parameter output by the output unit;
A camera system comprising: an image combining unit that combines the plurality of corrected images to generate a combined image.
Storing a correction parameter group of a plurality of patterns for correcting a plurality of images captured by partially overlapping a peripheral area of the moving body in association with the control information of the moving body;
Obtaining control information of the mobile body;
And a step of outputting a correction parameter group corresponding to the acquired control information.