JP2010171530A - Imaging controller - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging controller that easily optimizes an imaging state. <P>SOLUTION: The imaging controller 10 includes a mechanism portion 22 and a signal correction portion 31 which correct image data output from an imaging element 23, a correction amount control portion 40 which sets correction amounts so that correction amounts differ for each of a plurality of corrections, a processing data generation portion 33 which acquires a plurality of image data within a predetermined time according to change in correction amount and generates a processing image for each correction amount based upon the image data, a feature quantity extraction portion 35 which extracts a plurality of feature quantities from the processing images, a reference value storage portion 37 which stores reference values of the plurality of feature quantities, a correction amount storage portion 39 which stores the correction amounts, a matching degree calculation portion 36 which calculates degrees of matching between the reference values and the feature quantities extracted by the feature quantity extraction portion 35, and a reference correction amount setting portion 38 which sets a correction amount corresponding to a reference value having the highest degree of matching as a reference correction amount. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging control apparatus.

Conventionally, for example, a plurality of feature amounts are extracted from photometric data of an image, and based on these plurality of feature amounts, each feature amount and determination condition are expressed in a plurality of exposure amount calculation expressions associated with a plurality of preset patterns. There is known a copying apparatus that performs weighting based on the degree of coincidence and calculates an exposure amount using a plurality of exposure amount calculation formulas according to these weightings (see, for example, Patent Document 1).
Conventionally, for example, a vehicle imaging in which an exposure frame is set in a captured image of an imaging device that captures a seat area in a vehicle interior and the exposure of the imaging device is adjusted based on a luminance value for each pixel in the exposure frame. An apparatus is known (see, for example, Patent Document 2).

Japanese Patent No. 2612956 JP 2006-264404 A

By the way, according to the above-described conventional copying apparatus, the exposure amount calculated by the plurality of weighted exposure amount calculation formulas is only used for copying the entire area of the original image, and can be obtained by actual copying. Whether the copied image is optimal for the original image, that is, whether the exposure amount is optimal for the original image is not taken into consideration.
In addition, according to the above-described vehicular imaging apparatus, it is difficult to optimize the imaging state in detail simply by adjusting the exposure so that the area within the exposure frame has a desired brightness. The problem arises.
The present invention has been made in view of the above circumstances, and an object thereof is to provide an imaging control apparatus that can easily optimize an imaging state.

  In order to solve the above-described problems and achieve the object, the imaging control device according to the first aspect of the present invention provides an imaging device (for example, an image sensor that outputs image information including images obtained by imaging a predetermined imaging target region (for example, The image sensor 23 in the embodiment, correction means for correcting the image information output from the image sensor (for example, the mechanism unit 22 and the signal correction unit 31 in the embodiment), and a plurality of the corrections. Correction amount setting means (for example, the correction amount control unit 40 in the embodiment) that sets the correction amount for each different, and a plurality of the image information is acquired within a predetermined time according to the change of the correction amount. A processing image generation unit (for example, a processing data generation unit 33 in the embodiment) that generates a processing image based on the image information for each correction amount, and a plurality of feature amounts from the processing image. Feature extraction means to extract (for example, , A feature amount extraction unit 35) in the embodiment, a first storage unit (for example, a reference value storage unit 37 in the embodiment) that stores reference values of the plurality of feature amounts, and the correction amount. Second storage means for storing (for example, the correction amount storage unit 39 in the embodiment), the reference value stored in the first storage means, and the feature amount extracted by the feature amount extraction means Calculation means for calculating the degree of coincidence (for example, the degree of coincidence calculation unit 36 in the embodiment) and the second storage means corresponding to the reference value having the highest degree of coincidence calculated by the calculation means. And a reference correction amount setting means (for example, a reference correction amount setting unit 38 in the embodiment) that sets the correction amount as a reference correction amount.

  Furthermore, in the imaging control device according to the second aspect of the present invention, the feature amount includes a frequency state amount related to the frequency in the frequency distribution of the luminance of the processing image.

  Furthermore, in the imaging control device according to the third aspect of the present invention, the feature amount includes a luminance state amount related to the luminance of the processing image.

  Furthermore, in the imaging control device according to the fourth aspect of the present invention, the feature amount includes a statistical state amount related to a statistical value in a luminance frequency distribution of the processing image.

  Furthermore, in the imaging control device according to the fifth aspect of the present invention, the calculation means calculates an inner product of the reference value stored in the first storage means and the feature quantity extracted by the feature quantity extraction means. Then, the reference correction amount setting means sets the correction amount stored in the second storage means corresponding to the reference value having the largest inner product as the reference correction amount.

  Furthermore, the imaging control device according to the sixth aspect of the present invention can acquire a distance distribution based on distance information in real space corresponding to the pixels of the image based on the image information output from the plurality of imaging elements. A distribution acquisition unit (for example, a distance distribution calculation unit 32 in the embodiment), and the processing image generation unit generates the processing image having the distance distribution, and changes the correction amount according to the correction amount. A distance image generation unit (for example, a distance data generation unit in the embodiment) that generates a distance image by combining a plurality of the processing images based on the amount of change in the distance distribution is provided.

  Furthermore, the imaging control apparatus according to the seventh aspect of the present invention provides distance information calculation means (for example, step S07 in the embodiment) that calculates distance information in the real space corresponding to the pixel for each processing image. A region-of-interest setting unit that sets a region of interest by excluding a pixel region corresponding to the distance information that coincides with a predetermined reference distance from the distance information calculated by the distance information calculation unit. For example, the feature amount extracting unit extracts the feature amount from the region of interest.

  Furthermore, in the imaging control device according to the eighth aspect of the present invention, the feature amount includes a distance state amount related to the distance information.

  Further, the imaging control apparatus according to the ninth aspect of the present invention provides a stopping unit (for example, in the embodiment) that stops imaging by the imaging device when the degree of coincidence calculated by the calculating unit is a predetermined value or less. Imaging control unit 41).

  According to the imaging control device of the first aspect of the present invention, a plurality of processing images are generated corresponding to a plurality of different correction amounts, a plurality of feature amounts are extracted for each processing image, and each processing is performed. By calculating the degree of coincidence between a plurality of feature amounts and a reference value for each image, a processing image having the highest degree of coincidence, that is, a correction amount can be detected. Using this correction amount as a reference correction amount, it is possible to easily and in detail optimize image information correction, for example, correction of an imaging state such as exposure or image processing correction such as gamma correction of an image obtained by imaging. .

Furthermore, according to the imaging control device according to the second aspect of the present invention, it is possible to grasp the occupation area of the imaging object by correction according to the degree of coincidence in the frequency state quantity.
Furthermore, according to the imaging control device according to the third aspect of the present invention, it is possible to accurately identify the imaging object by correction according to the degree of coincidence in the luminance state quantity.
Furthermore, according to the imaging control device according to the fourth aspect of the present invention, it is possible to detect an imaging object that at least partially matches the reference shape by correction according to the degree of matching in the statistical state quantity.

  Furthermore, according to the imaging control device according to the fifth aspect of the present invention, in a multidimensional vector space with a plurality of feature amounts, the inner product of the plurality of feature amounts extracted from the processing image and the reference value is calculated. Arithmetic processing can be configured only by addition and integration, and the time required for the arithmetic processing can be shortened and the reference correction amount can be set quickly.

  Furthermore, according to the imaging control device according to the sixth aspect of the present invention, the distance information that can be acquired may be lost by synthesizing the processing image based on the image information corrected with different correction amounts into the distance image. Even if the sensitivity and dynamic range of the image sensor are small, the area of pixels from which distance information can be acquired with a desired resolution can be expanded without degrading the reliability of distance information. . This prevents the distance information from becoming difficult to detect in the low-brightness area or the high-brightness area, compared with the case where correction is performed to increase the brightness uniformly in the entire area of the image, for example. It can be detected with high accuracy.

Furthermore, according to the imaging control device according to the seventh aspect of the present invention, since the degree of coincidence is calculated for a plurality of feature amounts extracted from the region of interest, the calculation accuracy can be improved.
Furthermore, according to the imaging control device according to the eighth aspect of the present invention, the presence / absence of the imaging object can be accurately identified by the correction according to the degree of coincidence in the distance state quantity.
Furthermore, according to the imaging control device according to the ninth aspect of the present invention, inappropriate imaging can be prevented.

1 is a configuration diagram of a vehicle occupant detection device including an imaging control device according to an embodiment of the present invention. It is a figure which shows an example of imaging object area | region PT which concerns on embodiment of this invention, image data Pa, Pb, distance distribution image LP, image TP for processing, and distance image PL. It is a figure which shows an example of the attention area | region R which concerns on embodiment of this invention. It is a figure which shows an example of the area | region images Ra and Rb which concern on embodiment of this invention. It is a figure which shows an example of the coincidence degree for every area image Ra and Rb which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the imaging control apparatus which concerns on embodiment of this invention.

Hereinafter, an imaging control apparatus according to an embodiment of the present invention will be described with reference to the accompanying drawings.
The imaging control device 10 according to the present embodiment is provided, for example, in a vehicle occupant detection device, as shown in FIG. 1, for example, in a vehicle occupant detection device, and as shown in FIG. The image pickup apparatus 11 and the light source 12 that use the seat area as an image pickup target area, and a processing device 13 are provided.

The imaging device 11 includes a plurality of (for example, two) cameras such as a stereo camera. Each camera has a lens unit 21 including a plurality of lenses and various filters, a mechanism unit 22, and an imaging element. 23.
The mechanism unit 22 includes, for example, a shutter and an aperture mechanism, and controls a shutter speed and an aperture value at the time of imaging according to a control signal output from the processing device 13.
The image sensor 23 includes, for example, a CMOS sensor that can image from the visible light region to the infrared region, and outputs a signal of image data obtained by image capturing.
The imaging device 11 is substantially simultaneously (that is, predetermined) by a plurality of (for example, two) imaging elements 23 for each imaging state (that is, shutter speed, aperture value, etc.) corresponding to the control signal output from the processing device 13. Within a period of time, images are taken from different directions, and a plurality of (for example, two) image data signals are output.
The light source 12 includes, for example, an LED that emits light ranging from the visible light region to the infrared region, and irradiates light on the imaging target region of the imaging device 11.

  The processing device 13 includes, for example, a signal correction unit 31, a distance distribution calculation unit 32, a processing data generation unit 33, a distance data generation unit 34, a feature amount extraction unit 35, a coincidence calculation unit 36, a reference A value storage unit 37, a reference correction amount setting unit 38, a correction amount storage unit 39, a correction amount control unit 40, and an imaging control unit 41 are configured.

In response to the control signal output from the correction amount control unit 40, the signal correction unit 31 performs tone curve correction, level correction, and gamma correction on a plurality of (for example, two) image data signals output from the imaging device 11. Correct image processing such as.
For example, the signal correction unit 31 performs tone correction on the signal of the image data by using a correction map created in advance, the output characteristics of the imaging device 11, the imaging state, the environment of the imaging target region, and the like and the correction map indicating the phase between the luminance correction. To do.
The distance distribution calculation unit 32 performs stereo matching based on a plurality (for example, two) of image data with substantially the same imaging time output from the signal correction unit 31, and performs imaging in real space for each pixel constituting the image data. The distance of the object is calculated to generate distance distribution data (distance distribution data) at the pixels of the image data.

For example, as shown in FIG. 2, the front seat area in the passenger compartment is set as the imaging target area PT, and the imaging state of the imaging apparatus 11 (that is, the shutter speed, the aperture value, etc.) or the image processing correction of the signal correction unit 31 is different. When two image data (for example, image data imaged from a position shifted horizontally in the horizontal direction with respect to the imaging target region) Pa and Pb are output from the imaging device 11 at each imaging time, the distance distribution The computing unit 32 generates a distance distribution image LP as distance distribution data every three imaging times.
The distance distribution image LP is, for example, three-dimensional information (that is, pixels on two-dimensional coordinates) in which the luminance of each pixel of the image data is a value corresponding to the distance of the imaging target in the real space corresponding to each pixel. The position of the image and the luminance of the pixel) are visualized image data. The distance distribution data is not limited to image data that can be visualized, and simply corresponds to the position of the pixel on the two-dimensional coordinates and the information on the distance of the imaging target in the real space corresponding to each pixel. Any attached three-dimensional data may be used.

The processing data generation unit 33 generates processing data by excluding data with low reliability of distance information from the distance distribution data output from the distance distribution calculation unit 32.
For example, as shown in FIG. 2, in the distance distribution image LP at three imaging times, the luminance of the pixel increases as the distance of the imaging target in the real space corresponding to each pixel decreases. In the set state, the processing data generation unit 33 excludes from the distance distribution image LP data that is determined to have an infinite distance because the distance variance is less than a predetermined value, and the processing image. Generate TP. Note that the processing data is not limited to image data that can be visualized, and simply corresponds to the position of the pixel on the two-dimensional coordinate and the distance information of the imaging target in the real space corresponding to each pixel. Any attached three-dimensional data may be used.

The distance data generation unit 34 changes the distribution of distances among a plurality of processing data according to the imaging state of the imaging device 11 (that is, the shutter speed and the aperture value) or the change in the image processing correction of the signal correction unit 31. Based on the amount, a plurality of pieces of processing data output from the processing data generation unit 33 are combined to generate distance data.
For example, as illustrated in FIG. 2, the distance data generation unit 34 performs imaging at every three imaging times in which the imaging state of the imaging device 11 (that is, the shutter speed and the aperture value) or the image processing correction of the signal correction unit 31 is different. Based on the processing image TP, an area in which pixel data does not exist is complemented, and in an area in which pixel data exists, the reliability of distance information depends on, for example, the degree of variation in distance information. Data is synthesized by targeting only a region having a high degree or a region having similar distance information, and a distance image PL is generated. The distance data is not limited to image data that can be visualized, but simply associates the position of a pixel on a two-dimensional coordinate with information on the distance of an imaging target in a real space corresponding to each pixel. Any three-dimensional data may be used.

The feature amount extraction unit 35 is output from the processing data generation unit 33 at each imaging time in which the imaging state of the imaging device 11 (that is, the shutter speed and the aperture value) or the image processing correction of the signal correction unit 31 is different. A plurality of feature amounts are extracted from the processing data.
The plurality of feature amounts are, for example, feature state amounts (for example, zero brightness number, 25 in the frequency distribution of the brightness of the processing image TP, such as the feature amount (f ⁱ _(MODE) ) shown in Table 1 below. % Luminance, 50% luminance, 75% luminance, saturation (up / down luminance number, etc.), luminance state quantity (for example, mode value) relating to the luminance of the processing image TP, and frequency of luminance of the processing image TP. It consists of statistical state quantities (for example, average brightness, variance, etc.) related to statistical values in the distribution.

Note that the zero luminance number is attributed to, for example, a pixel area in which the distance is determined to be infinite in the processing image TP, and there is no luminance information, or overexposure or underexposure during imaging by the imaging device 11. In the processing image TP, the number of pixels in the pixel region having the largest area among the pixel regions in which no luminance information is present.
For example, 25% luminance, 50% luminance, and 75% luminance are obtained when the frequency (that is, the number of pixels) is integrated from luminance 0% toward luminance 100% in the luminance frequency distribution of the processing image TP. This is the luminance when the integrated value sequentially reaches 25%, 50%, and 75% with respect to the total number of pixels of the processing image TP.
The saturation (up and down) luminance numbers are, for example, a frequency of 0% luminance (that is, the number of pixels) and a frequency of luminance 100% (that is, the number of pixels).
The mode value is the luminance with the highest frequency (that is, the number of pixels) in the luminance frequency distribution of the processing image TP.
The average luminance and variance are the average value and variance in the frequency distribution of the luminance of the processing image TP.

Note that the plurality of feature quantities may include a distance state quantity related to distance information in the processing image TP. The distance state quantity may be, for example, the shape or presence of a pixel area having distance information in a predetermined distance range. It is presence or absence.
Further, by setting the number of feature amounts to, for example, the number of powers of 2, it is possible to quickly perform a calculation process in the coincidence degree calculation unit 36 described later.

Note that the feature amount extraction unit 35 excludes pixel regions in the processing data where the distance of the imaging target in the real space corresponding to each pixel coincides with a predetermined reference distance or becomes a value within a predetermined reference distance range. An attention area may be set, and a plurality of feature amounts may be extracted from the attention area.
For example, as shown in FIG. 3, when only an occupant existing in the passenger compartment is set as a region of interest, the feature amount extraction unit 35 is generated by stereo matching based on the two image data Pa and Pb output from the imaging device 11. The attention area image NP is excluded by excluding the pixel area having the distance information that matches the reference distance data (that is, the distance information with respect to the interior of the passenger compartment) created in a state where no occupant is present in advance from the distance distribution image LP thus obtained. Is generated. Then, a pixel area having distance information in the attention area image NP is set as the attention area R, and the attention area R is set in the processing data.

Note that the feature quantity extraction unit 35 may divide the processing data into a plurality of area data and extract a plurality of feature quantities from each area data.
For example, as shown in FIG. 4, in the processing image TP, when only a passenger existing in the vehicle interior is set as a region of interest, and this region of interest is divided for each passenger, a plurality of region data (region images) are set. The amount extraction unit 35 divides the processing image TP into, for example, a region of interest R in which an occupant is present, a region A in which only a vehicle interior is present, and a region B in which the variation in exposure state due to ambient light is large. Further, the region of interest R where the occupant is present is divided for each occupant to set a plurality of region images Ra and Rb.

  The degree-of-match calculation unit 36 extracts feature amounts for each processing data generated by the processing data generation unit 33, and for each region data when the processing data is divided into a plurality of region data. The degree of coincidence between the plurality of feature amounts output from the unit 35 and the reference value stored in the reference value storage unit 37 is calculated.

The reference value storage unit 37 includes a plurality of predetermined region states (S1, S2,..., Sk,..., Sn) in which states of the imaging target region of the imaging device 11 (for example, the presence or absence of an occupant) are different, and the imaging device 11. Imaging device 11 in each combination with a plurality of correction modes (M1, M2,..., Mk,..., Mn) having different imaging states (that is, shutter speed, aperture value, etc.) or different image processing corrections of the signal correction unit 31. The reference values (F ⁱ _{Mk, Sk} ) for a plurality of feature amounts extracted by the feature amount extraction unit 35 are stored in the case where the imaging by is performed.

The coincidence calculation unit 36 calculates, for example, a phase (for example, inner product) between a plurality of feature amounts and a reference value in a multidimensional vector space including a plurality of feature amounts output from the feature amount extraction unit 35. In the result, for example, it is determined that the degree of coincidence increases as the inner product increases.
For example, when imaging by the imaging device 11 is executed in the predetermined correction mode M1, a plurality of (for example, eight) feature amounts (f ⁱ _M1 ) as shown in Table 1 are extracted by the feature amount extraction unit 35. , matching degree calculation unit 36, for example, as shown in table 2, a plurality (e.g., eight) area state of (S1, ..., S8) with respect to correction mode M1 set reference value for each (F ⁱ _{(M1, S1)} ,..., F ⁱ _{(M1, S8)} ) are acquired from the reference value storage unit 37, and the feature value (f ⁱ _M1 ) and the reference value (for each region state (S1,..., S8) are obtained. Calculate the inner product with F ⁱ _{(M1, S1)} ,..., F ⁱ _{(M1, S8)} ). In Table 2 below, since the attention area R of the processing image TP is divided into two area images Ra and Rb, the inner product is calculated for each of the area images Ra and Rb.

  The reference correction amount setting unit 38 extracts an area state corresponding to the reference value having the highest degree of coincidence output from the coincidence degree calculating unit 36, and calculates the correction amount of the correction mode corresponding to the area state as the correction amount storage unit 39. And the correction amount is set as a reference correction amount. When the processing data is divided into a plurality of area data and the area state corresponding to the reference value having the highest degree of coincidence is different for each area data, the degree of coincidence is compared between the plurality of area data. Set the correction amount.

The correction amount storage unit 39 stores information on an optimal correction mode and correction amount corresponding to each of a plurality of predetermined region states (S1, S2,..., Sk,..., Sn), that is, an optimal imaging state ( That is, information on the optimum correction of image processing of the signal correction unit 31 is stored.
For example, as shown in Table 3 below, the correction modes (MODE1,..., MODE2) are stored in association with the region states (S1,..., S8) shown in Table 2 above.

For example, for the region image Ra shown in Table 2 above, the reference correction amount setting unit 38 sets the feature amount for each region state (S1,..., S8) as the region state corresponding to the reference value having the highest degree of coincidence. Among the inner products of (f ⁱ _M1 ) and the reference value (F ⁱ _{(M1, S1)} ,..., F ⁱ _{(M1, S8)} ), the reference value (F ⁱ _{(M1, S2)} = 0 at which the inner product is maximum .937) is extracted. Then, the correction mode MODE2 and the correction amount that are optimum for the region state S2 are acquired from the correction amount storage unit 39. For example, for the region image Rb shown in Table 2 above, as the region state corresponding to the reference value having the highest degree of coincidence, the reference value (F ⁱ _{(M1, S3)} = 0.893) that maximizes the inner product. The region state S3 corresponding to is extracted. Then, the correction mode MODE 3 and the correction amount that are optimum for the region state S 3 are acquired from the correction amount storage unit 39.

  The reference correction amount setting unit 38 divides the processing data into a plurality of area data, and when the degree of coincidence is calculated for each area data, the area corresponding to the reference value having the highest degree of coincidence for each area data When the states are different, the degree of coincidence is compared among a plurality of area data.

  For example, in the embodiment shown in FIGS. 5A to 5D, every three imagings in which the imaging state of the imaging device 11 (that is, the shutter speed, the aperture value, etc.) or the image processing correction of the signal correction unit 31 is different. The attention area R of the processing image TP is divided into two area images Ra and Rb, and the degree of coincidence (for example, inner product) is calculated for each of the area images Ra and Rb.

  For example, as shown in FIG. 5A, the degree of coincidence for each of the two region images Ra and Rb is equal to or greater than a predetermined lower limit Lth in all of the three times of imaging power, and the degree of coincidence between the three times of power of imaging. When the highest imaging power matches, the correction amount of the correction mode corresponding to the highest imaging power (for example, imaging power = 1) is set as the reference correction amount.

  Further, for example, as shown in FIG. 5B, the imaging power having the highest matching degree among the three imaging powers does not match between the two region images Ra and Rb, and the matching degree is predetermined in some imaging powers. Is less than the lower limit value Lth, the imaging power at which the degree of coincidence between the two area images Ra and Rb is equal to or greater than the predetermined lower limit value Lth is selected, and the degree of coincidence between the two area images Ra and Rb is selected. The correction amount in the correction mode corresponding to the imaging frequency (for example, imaging frequency = 2) having the highest overall matching degree such as sum or product is set as the reference correction amount.

  For example, as shown in FIG. 5C, when the degree of coincidence of one region image Rb is less than a predetermined lower limit Lth in all three imaging frequencies, the other image other than this region image Rb is used. The correction amount in the correction mode corresponding to the imaging frequency (for example, imaging frequency = 1) having the highest matching degree in the region image Ra is set as the reference correction amount.

  Further, for example, as shown in FIG. 5D, when the degree of coincidence between any of the region images Ra and Rb is less than a predetermined lower limit Lth in all three imaging frequencies, the imaging device 11 or the imaging target It is determined that the state of the region (for example, the occupant's posture) is an abnormal state, and setting of the reference correction amount is prohibited.

The correction amount control unit 40 corrects the imaging state of the imaging apparatus 11 (that is, the shutter speed and the aperture value) or the image processing of the signal correction unit 31 according to the reference correction amount output from the reference correction amount setting unit 38. A control signal for controlling is output.
The correction amount control unit 40 outputs a control signal for controlling the irradiation state of the light source 12 according to the reference correction amount output from the reference correction amount setting unit 38.

The imaging control unit 41 controls execution and stop of imaging by the imaging device 11 according to, for example, the degree of coincidence output from the coincidence degree calculating unit 36.
For example, the imaging control unit 41 stops imaging by the imaging device 11 when all of the matching degrees output from the matching degree calculation unit 36 are less than a predetermined lower limit value Lth.

  The imaging control device 10 according to the present embodiment has the above-described configuration. Next, the operation of the imaging control device 10 will be described.

First, for example, in step S01 shown in FIG. 6, a predetermined initial imaging state set in advance or an imaging state corresponding to a control signal output from the processing device 13 at the previous imaging (that is, shutter speed, aperture value, etc.). Thus, imaging of the imaging device 11 is executed.
In step S02, the left and right image data Pa and Pb imaged from the different directions (for example, positions shifted left and right in the horizontal direction with respect to the imaging target region) almost simultaneously by the two imaging elements 23 of the imaging device 11. To get.

In step S03, image processing corrections such as tone curve correction, level correction, and gamma correction are performed on the signals of the image data Pa and Pb.
In step S04, stereo matching is performed based on the two corrected image data Pa and Pb.
In step S05, the distance of the imaging target in the real space is calculated for each pixel constituting the two image data Pa and Pb, and a distance distribution image LP is generated.

In step S06, a processing image TP is generated by excluding a pixel region having low reliability of distance information from the distance distribution image LP.
In step S07, pixel regions in which the distance of the imaging target in the real space corresponding to each pixel in the processing image TP matches the predetermined reference distance or has a value within the predetermined reference distance range are excluded. An attention area R is set.
In step S08, the attention area R is divided into a plurality of area images, for example, corresponding to each of a plurality of passengers in the passenger compartment.

In step S09, a plurality of feature amounts are extracted for each of the plurality of region images.
In step S10, in a multi-dimensional vector space with a plurality of feature amounts, the correlation between the plurality of feature amounts and the reference value is calculated by calculating the phase (for example, inner product) between the plurality of feature amounts and the reference value. Calculate the degree.
In step S11, the degree of coincidence is compared between the plurality of region images.
In step S12, the reference correction amount is set according to the comparison result of the degree of coincidence between the plurality of area images.

In step S13, it is determined whether or not it is instructed to change the image processing correction of the signal correction unit 31 according to the reference correction amount and acquire a new processing image TP.
If this determination is “YES”, the flow proceeds to step S 03 described above.
On the other hand, if this determination is “NO”, the flow proceeds to step S14.
In step S14, whether or not it is instructed to change the imaging state (that is, shutter speed, aperture value, etc.) of the imaging device 11 according to the reference correction amount and acquire a new processing image TP. Determine.
If this determination is “YES”, the flow proceeds to step S 01 described above.
On the other hand, if this determination is “NO”, the flow proceeds to step S15.
In step S15, a plurality of processing images TP are combined to generate a distance image PL, and the process proceeds to the end.

  As described above, according to the imaging control device 10 according to the present embodiment, the imaging state of the imaging device 11 (that is, the shutter speed, the aperture value, etc.) or the processing for a plurality of processings with different image processing corrections of the signal correction unit 31 is different. Processing data with the highest degree of matching by generating data, extracting multiple feature quantities for each processing data, and calculating the degree of matching between multiple feature quantities and reference values for each processing data That is, the correction amount can be detected. With this correction amount as a reference correction amount, image data correction, for example, correction of image processing such as exposure or gamma correction of an image obtained by imaging can be optimized easily and in detail. .

Furthermore, the occupation area of the imaging target can be grasped by correction according to the degree of coincidence in the frequency state quantity.
Furthermore, the object to be imaged can be accurately identified by the correction according to the degree of coincidence in the luminance state quantity.
Furthermore, it is possible to detect an imaging object that at least partially matches a predetermined reference shape by correction according to the degree of coincidence in the statistical state quantity.

  Furthermore, in a multidimensional vector space with a plurality of feature quantities, the inner product of the plurality of feature quantities extracted from the processing data and the reference value is computed, so that arithmetic processing can be configured only by addition and integration. The time required for processing can be shortened and the reference correction amount can be set quickly.

  Furthermore, by synthesizing the processing data based on the image data corrected with different correction amounts into the distance data, the distance information that can be acquired is lost, or the reliability of the distance information is not reduced. Even if the sensitivity and dynamic range of the image sensor 23 are small, the data area in which the distance information can be acquired with a desired resolution can be expanded. This prevents the distance information from becoming difficult to detect in the low-brightness area or the high-brightness area, compared with the case where correction is performed to increase the brightness uniformly in the entire area of the image, for example. It can be detected with high accuracy.

Furthermore, since the degree of coincidence is calculated for a plurality of feature amounts extracted from the attention area R, the calculation accuracy can be improved.
Furthermore, the presence / absence of the imaging target can be accurately identified by correction according to the degree of coincidence in the distance state quantity.
Further, when the degree of coincidence is less than the predetermined lower limit Lth, it is determined that the state of the imaging device 11 or the imaging target region (for example, the posture of the occupant) is an abnormal state, and imaging is stopped, Inappropriate imaging can be prevented.

10 Imaging control device 22 Mechanism (correction means)
23 Image sensor 31 Signal correction part (correction means)
32 Distance distribution calculation unit (distance distribution acquisition means)
33 processing data generation unit (processing image generation means)
34 Distance data generation unit (distance image generation means)
35 Feature Extraction Unit (Feature Extraction Unit)
36 Matching degree calculation part (calculation means)
37 Reference value storage unit (first storage means)
38 Reference correction amount setting unit (reference correction amount setting means)
39 Correction amount storage section (second storage means)
40 Correction amount control unit (correction amount setting means)
41 Imaging control unit (stopping means)
Step S07 Distance information calculation means, attention area setting means

Claims (9)

An image sensor that outputs image information including an image obtained by imaging a predetermined imaging target region;
Correction means for correcting the image information output from the image sensor;
A correction amount setting means for setting a plurality of correction amounts for different correction amounts;
Processing image generation means for acquiring a plurality of the image information within a predetermined time according to a change in the correction amount, and generating a processing image based on the image information for each of the correction amounts;
Feature quantity extraction means for extracting a plurality of feature quantities from the processing image;
First storage means for storing reference values of the plurality of feature quantities;
Second storage means for storing the correction amount;
Computing means for computing the degree of coincidence between the reference value stored in the first storage means and the feature quantity extracted by the feature quantity extraction means;
Reference correction amount setting means for setting the correction amount stored in the second storage means corresponding to the reference value having the highest degree of coincidence calculated by the calculation means as a reference correction amount. An imaging control device. The imaging control apparatus according to claim 1, wherein the feature amount includes a frequency state amount related to a frequency in a luminance frequency distribution of the processing image. The imaging control apparatus according to claim 1, wherein the feature amount includes a luminance state amount related to a luminance of the processing image. The imaging control apparatus according to claim 1, wherein the feature amount includes a statistical state amount related to a statistical value in a luminance frequency distribution of the processing image. The calculation means calculates an inner product of the reference value stored in the first storage means and the feature amount extracted by the feature amount extraction means,
The reference correction amount setting unit sets the correction amount stored in the second storage unit corresponding to the reference value having the largest inner product as the reference correction amount. Imaging control device. A distance distribution acquisition unit capable of acquiring a distance distribution based on distance information in a real space corresponding to the pixels of the image based on the image information output from the plurality of imaging elements;
The processing image generation means generates the processing image having the distance distribution,
6. A distance image generating unit that generates a distance image by combining a plurality of the processing images based on a change amount of the distance distribution according to the change of the correction amount. The imaging control device according to any one of the above. Distance information calculating means for calculating distance information in the real space corresponding to the pixel for each processing image;
Attention area setting means for setting an attention area by excluding a pixel area corresponding to the distance information that coincides with a predetermined reference distance from the distance information calculated by the distance information calculation means. ,
The imaging control apparatus according to claim 1, wherein the feature amount extraction unit extracts the feature amount from the region of interest. The imaging control apparatus according to claim 6, wherein the feature amount includes a distance state amount related to the distance information. 9. The apparatus according to claim 1, further comprising a stopping unit that stops imaging by the imaging element when the degree of coincidence calculated by the calculating unit is equal to or less than a predetermined value. Imaging control device.

Images