JP2009177250A - Onboard image recognition device, vehicle condition decision device, and onboard image recognition method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress processing loads and to secure a high dynamic range of a camera in an onboard image recognition device. <P>SOLUTION: The onboard image processor images an object in the image updating cycle of 100[ms]. Three image frames of about 33[ms] are acquired within one image updating cycle. A first image frame is obtained by imaging the object with normal exposure time within one image updating cycle first, then a second image frame is obtained by imaging the same object with the exposure time shorter than the normal exposure time within the same cycle, and a third image frame is obtained by imaging the same object with the exposure time longer than the normal exposure time within the same cycle. Then, the contrast of each of the acquired first image frame, second image frame and third image frame is checked, and an image is recognized on the basis of an image composed of selected partial images having the most excellent contrast among the respective image frames. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention is based on an in-vehicle image recognition device and an in-vehicle image recognition method for recognizing an image based on an image around the vehicle imaged by an imaging device provided in the vehicle, and an image recognition processing result of the in-vehicle image recognition device. In particular, the present invention relates to a vehicle situation determination apparatus that determines a situation in which the vehicle is placed, and in particular, takes a plurality of images with different exposure times, and synthesizes the plurality of images into a single image so as to improve image recognition processing accuracy. The vehicle-mounted image recognition device and the vehicle-mounted image recognition method capable of ensuring the high dynamic range of the camera of the vehicle-mounted image recognition device, and the presence of objects existing around the vehicle. The present invention relates to a vehicle situation determination device that automatically avoids a collision.

  Conventionally, it is provided in the vicinity of a room mirror inside the vehicle and recognizes other vehicles, pedestrians, road lane lines (white lines and yellow lines on the road), etc., by recognizing an image obtained by photographing the front of the vehicle with a camera. An in-vehicle image processing apparatus is known. In-vehicle image processing devices saturate the bright part of the image because the camera's dynamic range is insufficient when driving at night, when driving near the entrance of a tunnel, or when driving under backlighting. Or the dark part of the image becomes completely black (this is called “blackout”), and the image recognition process cannot be performed normally, adversely affecting the recognition accuracy of other vehicles, pedestrians, road lane markings, etc. There was a problem of affecting.

  In order to solve this problem, conventionally, the output with respect to the luminance value of the shooting target changes logarithmically (that is, when the luminance value of the shooting target is low, the output is emphasized, and conversely, There has been proposed a high dynamic range camera that suppresses output when the luminance value is high.

  However, although a camera with a high dynamic range can suppress overexposure and underexposure of an image, in exchange for this, the contrast of the entire image is reduced. Since the image recognition process is performed by extracting a part having a feature in the contrast difference of the image, the decrease in contrast also adversely affects the recognition accuracy of the image recognition process.

  There has also been proposed a camera capable of photographing an image obtained by combining images obtained by a plurality of different shutter times so as to remove overexposure and underexposure. However, in this case, a boundary line is generated by combining a plurality of images, but the boundary line may adversely affect the recognition accuracy of the image recognition process. Further, even if the image correction process is performed to remove the boundary line, there is a problem that the processing load increases due to the image correction process.

  Therefore, in order to solve such a problem, for example, as disclosed in Patent Document 1, a plurality of images shot at different shutter speeds are extracted, and an image having an optimal pixel value is extracted for each of a plurality of areas of the image. Thus, an imaging apparatus that can capture an image corresponding to a high dynamic range by combining the images into one image has been proposed.

  Further, as disclosed in Patent Document 2, a plurality of images with different exposure amounts are captured, each of the plurality of images with different exposure amounts is expanded in the vertical direction, and the signal level of each image after the expansion is averaged. There has been proposed an imaging system capable of capturing a clearer image by combining the image into a single image after conversion.

  Further, as disclosed in Patent Document 3, an image captured by a plurality of cameras with different exposure times provided in a vehicle is combined into a single image, so that an object to be imaged can be obtained even under high contrast conditions. An in-vehicle camera system that can capture a clear image has been proposed.

  In addition, as disclosed in Patent Document 4, a plurality of images are taken with different exposure times, and the images are divided into regions with different subject movements. In each region, the exposure time increases as the subject motion increases. When a shorter image is selected, an image with a longer exposure time is selected as the subject's movement is smaller, and the images are taken from a moving vehicle or the like by combining the areas and combining one image. Even in such a case, an imaging apparatus that can obtain a high-quality image has been proposed.

JP 2003-250094 A JP 2004-48345 A JP 2000-217100 A JP 2006-245909 A

  However, in the conventional techniques represented by the above-mentioned Patent Documents 1 to 3, it is a camera that always captures a plurality of images captured at different shutter speeds and synthesizes the plurality of images into one image. High dynamic range can be secured. However, in particular, in the in-vehicle image recognition device, the foreground of the vehicle changes rapidly according to the movement of the vehicle, changes in time, weather, and the like. For this reason, in-vehicle image recognition devices are not always in a situation where a high dynamic range is required, and in situations where a high dynamic range is not required, wasteful processing is performed, and the processing load is a burden. There was a problem of becoming.

  Note that the conventional technology represented by the above-mentioned Patent Document 4 ensures a high dynamic range of the camera even when an image is taken from a moving vehicle or the like, even if a high-quality image can be obtained. It does not contribute to the improvement of the recognition accuracy of the image recognition process using the photographed image.

  The present invention has been made to solve the above problems (problems), and in an in-vehicle image recognition apparatus, a plurality of images with different exposure times are taken, and these images have an image recognition processing accuracy. An in-vehicle image recognition apparatus, a vehicle state determination apparatus, and an in-vehicle image recognition method capable of performing a process of combining with a single image so as to increase the processing load and securing a high dynamic range of the camera of the in-vehicle image recognition apparatus The purpose is to provide.

  In order to solve the above-described problems and achieve the object, the present invention is an in-vehicle image recognition device that performs image recognition based on an image around a vehicle imaged by an imaging device provided in the vehicle. An imaging processing unit that controls the imaging device to capture a plurality of images with different exposure times within one update period, and each of the plurality of images captured by the imaging processing unit being controlled by the imaging processing unit. Contrast determination means for determining whether or not the contrast is a value within a predetermined range; and for each of the plurality of images for which the contrast of the plurality of images is determined to be a value within the predetermined range by the contrast determination means Partial image selection that extracts partial images of the same part and selects the partial images with the highest contrast from the partial images of the same part of the plurality of images. An image synthesizing unit that combines the partial images selected by the partial image selecting unit into one image, and an image that performs image recognition processing based on the one image synthesized by the image synthesizing unit. And a recognition processing means.

  In addition, the present invention provides the weighting coefficient according to the contrast of the partial images of the plurality of images when the partial images of the plurality of images are combined into one image by the image combining means. Weighting coefficient storage means for storing, and the image composition means multiplies the pixel value of each partial image by a weighting coefficient corresponding to the contrast of each partial image selected by the partial image selection means. The method is characterized in that partial images of a plurality of images are combined into one image.

  In addition, the present invention is an in-vehicle image recognition device that performs image recognition based on an image around a vehicle imaged by an imaging device provided in the vehicle, and has different exposure times within one cycle of an image update cycle. An imaging processing unit that controls the imaging device to capture a plurality of images, and whether or not the contrast of each of the plurality of images captured by controlling the imaging device by the imaging processing unit is a value within a predetermined range. Contrast determination means for determining whether or not, and image recognition processing means for performing image recognition processing based on an image picked up by the image pickup device, the image pickup processing means within one cycle of the image update cycle, The predetermined exposure until the image for which the contrast is determined by the contrast determination means to be within a predetermined range is captured. The image pickup apparatus is controlled so as to take an image with an exposure time different from the time, and the image recognition processing means has a contrast within a predetermined range by the contrast determination means within one period of the image update period. An image recognition process is performed based on an image determined for the first time.

  Further, according to the present invention, in the above invention, based on information acquired from an in-vehicle device provided in the vehicle, the imaging device captures a plurality of images with the different exposure times within one cycle of the image update cycle. A plurality of image capturing necessity determining means for determining whether or not there is a necessity, wherein the image capturing processing means determines the different exposure times within one cycle of the image update period by the plurality of image capturing necessity determining means; Only when it is determined that there is a need to capture a plurality of images, the imaging apparatus is controlled to capture a plurality of images with the different exposure times within one period of the image update period. And

  In addition, the present invention is mounted on the vehicle, which determines a situation in which the vehicle is placed based on a result of image recognition based on an image around the vehicle imaged by an imaging device provided in the vehicle. An imaging process for controlling the imaging apparatus to capture a plurality of images with different exposure times within one period of an image update period, which is a vehicle status determination apparatus capable of communicating with a vehicle control apparatus that is connected via an in-vehicle network Means for determining whether or not the contrast of each of the plurality of images captured by the imaging device by the imaging processing unit is a value within a predetermined range, and each of the plurality of the plurality of images by the contrast determination unit Extracting partial images of the same part of each of the plurality of images determined to have a contrast within a predetermined range, and extracting the same part of the plurality of images A partial image selecting means for selecting a partial image having the highest contrast from the partial images, an image combining means for combining the partial images selected by the partial image selecting means into one image, and the image combining means. An image recognition processing means for performing image recognition processing based on the synthesized one image, and determining a situation around the vehicle based on a result of the image recognition processing performed by the image recognition processing means; Vehicle condition determination processing means for outputting a braking control signal to the vehicle control device when braking control of the vehicle is required in accordance with the result.

  In addition, the present invention is mounted on the vehicle, which determines a situation in which the vehicle is placed based on a result of image recognition based on an image around the vehicle imaged by an imaging device provided in the vehicle. An imaging process for controlling the imaging apparatus to capture a plurality of images with different exposure times within one period of an image update period, which is a vehicle status determination apparatus capable of communicating with a vehicle control apparatus that is connected via an in-vehicle network Means for determining whether or not the contrast of each of the plurality of images picked up by the control of the image pickup device by the image pickup processing means is a value within a predetermined range, and picked up by the image pickup device Image recognition processing means for performing image recognition processing based on the image, and based on the result of the image recognition processing performed by the image recognition processing means. Vehicle situation determination processing means for determining the situation of both surroundings and outputting a braking control signal to the vehicle control device when braking control of the vehicle is required according to the result of the determination, the imaging The processing means includes the predetermined exposure time until an image in which the contrast determination means determines that the contrast of the predetermined image is a value within a predetermined range within one cycle of the image update period, respectively. The imaging apparatus is controlled to capture images with different exposure times, and the image recognition processing means is the first time that the contrast is determined by the contrast determination means within a predetermined range within one period of the image update period. An image recognition process is performed based on the determined image.

  Further, the present invention is an in-vehicle image recognition method in which an in-vehicle image recognition device performs image recognition based on an image around a vehicle imaged by an imaging device provided in the vehicle, and the in-vehicle device provided in the vehicle Multiple image imaging necessity determination step for determining whether or not it is necessary to capture a plurality of images with different exposure times within one period of the image update period based on the information acquired from When it is determined by the necessity determination step that it is necessary to capture a plurality of images at the different exposure times within one cycle of the image update cycle, the different exposures within one cycle of the image update cycle. An imaging processing step of controlling the imaging device so as to capture a plurality of images in time, and imaging by controlling the imaging device by the imaging processing step. A contrast determination step for determining whether or not the contrast of each of the plurality of images is a value within a predetermined range; and the contrast determination step determines that the contrast of each of the plurality of images is a value within a predetermined range. Extracting a partial image of the same portion of each of the plurality of images and selecting a partial image having the highest contrast from the partial images of the same portion of the plurality of images, and selecting by the partial image selection step The partial images of each of the plurality of images are multiplied by the pixel values of the partial images by a weighting coefficient corresponding to the contrast of the partial images of each of the plurality of images. And an image recognition process based on the one image synthesized by the image synthesis step. Characterized in that it comprises a Cormorant image recognition processing step.

  According to the present invention, a partial image having the highest contrast is selected from the partial images in which the contrast of a plurality of images captured at different exposure times within a predetermined range of the image update cycle is within a predetermined range, and is combined into one image. Since the image recognition process is performed based on the image, the gradation of the intermediate colors may be lost, resulting in a coarse image. The image with a large contrast and the gradation of the dark part and the bright part are averaged. An image having a low contrast is excluded from the processing target, and the processing load is reduced and the accuracy of image recognition is improved.

  Further, according to the present invention, partial images in which the contrast of a plurality of images captured at different exposure times within one period of the image update cycle is within a predetermined range, the weighting coefficient corresponding to the contrast of each partial image has a pixel value Is multiplied to be combined into one image, so that the contrast of the combined image is improved, and an image more suitable for image recognition can be obtained.

  In addition, according to the present invention, within one period of the image update period, the exposure time is increased until the contrast becomes a value within a predetermined range and a contrast image suitable for image recognition processing is captured within a predetermined number of times. When the image is picked up and a contrast image suitable for the image recognition processing is captured, the image recognition processing is performed based on the contrast image suitable for the image recognition processing without image capturing thereafter. Thus, it is possible to reduce the processing load and obtain an image more suitable for image recognition.

  In addition, according to the present invention, only when there is a need to capture a plurality of images with different exposure times within one cycle of the image update cycle, a plurality of images with the different exposure times within one cycle of the image update cycle. Therefore, there is an effect that the processing conditions are limited and unnecessary processing is omitted. Specifically, for example, based on the current position information of the vehicle and the current time, the vehicle is located in the tunnel or near the tunnel entrance, or the current time is in the evening time zone that is susceptible to backlighting Based on the above, it is determined that it is necessary to capture a plurality of images with different exposure times within one cycle of the image update cycle, and a plurality of images are captured with the different exposure times within one cycle of the image update cycle. Therefore, there is an effect that the processing conditions are limited and unnecessary processing is omitted.

  In addition, according to the present invention, the situation around the vehicle is determined based on the result of the image recognition process, and when the braking control of the vehicle is necessary according to the determination result, the braking control signal is sent to the vehicle control device. Therefore, the vehicle control device automatically performs braking control, and it is possible to automatically avoid a collision with an object around the vehicle and prevent an accident from occurring. There is an effect.

  Embodiment 1 and Embodiment 2 according to the in-vehicle image recognition device, the vehicle state determination device, and the in-vehicle image recognition method of the present invention will be described in detail below with reference to the accompanying drawings. Note that the in-vehicle image recognition apparatus and the in-vehicle image recognition method shown in the following first and second embodiments are image recognition apparatuses mounted on a vehicle, but the present invention is not limited thereto. The present invention can be widely applied to various image recognition apparatuses and in-vehicle image recognition methods.

  Prior to the description of the first and second embodiments, the outline and features of the present invention will be described. FIG. 1 is a diagram showing an outline and features of the present invention. As shown in the figure, for example, the in-vehicle image processing apparatus captures an image of the imaging target with an image update period of 100 [ms]. This image update period is divided into three equal parts, and three image frames of about 33 [ms] are acquired within one period of the image update period.

  First, the in-vehicle image processing apparatus acquires a first image frame in which an imaging target is imaged with a normal exposure time within one period of an image update period. Subsequently, the in-vehicle image processing apparatus acquires a second image frame obtained by imaging the same imaging object with an exposure time shorter than the normal exposure time within the same period of the above-described image update period. Finally, the in-vehicle image processing apparatus acquires a third image frame obtained by imaging the same imaging object with an exposure time longer than the normal exposure time within the same period of the above-described image update period. Note that acquiring a plurality of image frames while changing the exposure time is referred to as “variable exposure time image frame acquisition”.

  Then, the in-vehicle image processing apparatus checks the contrast of each of the acquired first image frame, second image frame, and third image frame, and selects and selects a partial image having the best contrast in each image frame. Image recognition is performed based on a synthesized image obtained by synthesizing the respective partial images, or an image frame having the best contrast is selected for image recognition.

  In this way, by performing image recognition processing based on a plurality of image frames acquired by changing the exposure time, image recognition processing based on an image having good contrast without “whiteout” or “blackout” And the accuracy of image recognition can be improved.

  In the following first and second embodiments, the image update cycle is set to 100 [ms], the image update cycle is divided into three equal parts, and three image frames are acquired within one cycle of the image update cycle. Indicates when to do. However, the present invention is not limited to this, and the setting may be arbitrarily changed.

  The configurations of the in-vehicle image processing device and the vehicle situation determination device according to the first embodiment will be described below. FIG. 2 is a functional block diagram of the configuration of the in-vehicle image processing device and the vehicle state determination device according to the first embodiment. As illustrated in FIG. 1, an in-vehicle image recognition device 10a and a vehicle situation determination device 20a according to the first embodiment are provided in a vehicle 1 via an in-vehicle network 100 such as a CAN (Controller Area Network). A car navigation device 20 that delivers car navigation information, date information, time information, and the like to 10a and a vehicle control ECU (Electronic Control Unit) device 30 that controls an electronic device included in the vehicle 1 are communicably connected.

  The vehicle control ECU device 30 controls the alarm device 30a, the brake control device 30b, the motor control device 30c, and the seat belt control device 30d. The alarm device 30 a outputs an alarm sound toward the outside of the vehicle 1 and / or the driver of the vehicle 1 in accordance with an instruction from the vehicle control ECU device 30.

  In response to an instruction from the vehicle control ECU device 30, the brake control device 30b increases the brake hydraulic pressure and controls the brake control device of the vehicle 1 to brake the travel of the vehicle 1. The motor control device 30 c suppresses supply of fuel or energy to the motor of the vehicle 1 in accordance with an instruction from the vehicle control ECU device 30. The seat belt control device 30d locks the seat belt in use of the vehicle 1 in accordance with an instruction from the vehicle control ECU device 30.

  An image sensor unit 17 that is an imaging device is connected to the in-vehicle image recognition device 10a. The image sensor unit 17 includes an optical lens 17a, a shutter unit 17b, and a CMOS sensor 17c. The CMOS sensor 17c of the image sensor unit 17 acquires an image of the imaging target via the optical lens 17a and the shutter unit 17b, converts it into image data, and transfers it to the in-vehicle image recognition device 10a. Further, a display unit 18 such as a display device is connected to the in-vehicle image recognition device 10a. The shutter unit 17b controls the exposure time for the CMOS sensor 17c under the control of the image frame acquisition processing unit 11d.

  The in-vehicle image recognition device 10a includes a control unit 11, a storage unit 12, an image sensor interface unit 13 that is an interface for transferring image data from the image sensor unit 17, and an image to be displayed on the display unit 18. And a display output interface unit 14 which is an interface for transferring data to the display unit 18.

  The control unit 11 is a control device such as a microcomputer that controls the entire on-vehicle image recognition device 10a. In particular, as a component closely related to the first embodiment, an imaging processing unit 11a and a time determination process are included. Unit 11b, own vehicle position determination processing unit 11c, image frame acquisition processing unit 11d, image contrast determination processing unit 11e, partial image selection processing unit 11f, partial image composition processing unit 11g, and image recognition processing unit 11h. And have.

  The storage unit 12 is configured to store an image buffer 12a that is an image memory that develops image data of an image frame captured by the image sensor unit 17, and a setting value that is a parameter that defines the operation of the in-vehicle image recognition device 10a. A composite weighting coefficient for storing a composite weighting coefficient for multiplying the pixel values of each partial image when the value storage area 12b and the partial image composition processing unit 11g compose a partial image to generate one image as will be described later. The storage area 12c includes an image recognition pattern data storage area 12d for storing image recognition pattern data used when the image recognition processing unit 11h performs image recognition as will be described later.

  Note that the set value storage table shown in FIG. 3 is stored in the set value storage area 12b. As shown in the figure, the setting value storage table has a “setting item” column and a “setting value” column. “Setting item” includes “Image update cycle”, “Image update cycle division number”, “Normal exposure time”, “Short exposure time”, “Long exposure time”, “Image contrast upper limit”, “Image contrast” There is a lower limit.

  The “image update period” is a time required for the image sensor unit 17 to capture one image. In FIG. 3, “100 [ms]” is stored as the “set value” of the “image update cycle”. The “image update cycle division number” is the number of divisions for equally dividing the image update cycle. In FIG. 3, “3” is stored as “setting value” of “number of image update period divisions”.

  “Normal exposure time” is a standard exposure time when the image sensor unit 17 captures an image. In FIG. 3, “x [s]” is stored as “set value” of “normal exposure time”. The “short exposure time” is an exposure time shorter than the “standard exposure time” when the image sensor unit 17 captures an image. In FIG. 3, “y [s]” is stored as the “set value” of “short exposure time”. The “long exposure time” is an exposure time longer than the “standard exposure time” when the image sensor unit 17 captures an image. In FIG. 3, “z [s]” is stored as the “set value” of “long exposure time”. Naturally, the magnitude relationship of y [s] <x [s] <z [s] holds.

The “image contrast upper limit value” is an upper limit value of the contrast of an image allowed in the image recognition process. Contrast is the difference between the maximum and minimum pixel values (or luminance values) in the same image. Therefore, the higher the contrast, the brighter the part is brighter and the black part is expressed more black. It becomes. On the other hand, the gradation of the intermediate color is lost, and there is a possibility that the image becomes rough. For this reason, an upper limit value of the contrast of the image allowed in the image recognition process is set. In FIG. 3, “a ₀ ” is stored as “setting value” of “image contrast upper limit value”.

The “image contrast lower limit value” is a lower limit value of the contrast of an image allowed in the image recognition process. If the contrast is too small, the gradations of the dark part and the bright part are averaged, which may result in an unclear image. For this reason, a lower limit value of the contrast of the image allowed in the image recognition process is set. In FIG. 3, “b” is stored as “setting value” of “image contrast lower limit value”. Naturally, the magnitude relationship b <a ₀ holds.

  Each set value stored in the set value storage table can be customized according to the performance of the image sensor unit 17 and the in-vehicle image recognition device 10a. By customizing each set value, it may be possible to maximize the image recognition performance of the in-vehicle image recognition device 10a.

  Further, the composite weighting coefficient storage table shown in FIG. 4 is stored in the composite weighting coefficient storage area 12c. As shown in the figure, the composite weighting coefficient storage table has columns of “contrast upper limit value”, “contrast lower limit value”, and “weighting coefficient”. In the composite weighting coefficient storage table, one “weighting coefficient” is associated with a contrast range determined by “contrast upper limit value” and “contrast lower limit value”.

For example, when “contrast upper limit value” and “contrast lower limit value” are “a ₀ ” and “a ₁ ”, “α ₀ ” is associated as “weighting coefficient”. Similarly, the combinations of ““ contrast upper limit value ”,“ contrast lower limit value ”, and“ weighting coefficient ”are“ “a ₁ ”, “a ₂ ”, “α ₁ ” ”,“ “a ₂ ”, “ a ₃ ”,“ α ₂ ””,..., ““ a _n ”,“ b ”,“ α _n ””. Naturally, the magnitude relationship of b <a _n <... <A ₃ <a ₂ <a ₁ <a ₀ holds. a ₀ and b are the same as the “set value” of “image contrast upper limit value” and “image contrast lower limit value” set in the set value storage table.

For example, the contrast of an image frame having a partial image with the highest contrast among the same partial images of a plurality of image frames acquired by the image frame acquisition processing unit 11d is a (where a ₁ ≦ a <a ₂ ). If the, when all the partial images by the partial image synthesizing processing unit 11g is synthesized into one image, one image in terms of contrast is multiplied by the alpha ₁ in all of the pixel values of the partial image is a Is synthesized. In this way, it becomes possible to synthesize a sharp and clear image.

Note that “α ₀ ”, “α ₁ ”, “α ₂ ”,..., “Α _n ” are predetermined adaptation values for synthesizing an image optimal for image recognition processing. “Α ₀ ”, “α ₁ ”, “α ₂ ”,..., “Α _n ” can be appropriately changed in order to synthesize an optimal image by image recognition processing. _Further, b <insofar as to maintain the magnitude relation of _{a n <··· <a 3 <} a 2 <a 1 <a 0, a 1, a 2, a 3, ···, a n is the image recognition In order to synthesize an optimal image for processing, it can be changed as appropriate.

  The image recognition pattern data storage area 12d stores an image recognition pattern data table shown in FIG. As shown in the figure, the image recognition pattern data table has columns of “image recognition target name” and “image recognition pattern data”. “Image recognition pattern data” is associated with “image recognition target name”. For example, when the “image recognition target name” is “white line”, “white line image recognition pattern data” is associated as “image recognition pattern data”.

  Returning to the description of the control unit 11, the imaging processing unit 11 a controls the image sensor unit 17 to capture a plurality of images with different exposure times within one cycle of the image update cycle.

  The time determination processing unit 11b indicates that the time information acquired from the car navigation device 20 or other devices or the system time of the vehicle-mounted image recognition device 10a (hereinafter, the time information and the system time are referred to as the current time) is “evening. It is determined whether or not it is a time zone in which the accuracy of image recognition by the in-vehicle image recognition device 10a is expected to decrease, such as “night”.

  For example, in the case of “evening”, the quality of the image captured by the image sensor unit 17 due to backlight is lowered, and the accuracy of image recognition is lowered. In addition, if it is “night”, “blackout” is likely to occur in the image captured by the image sensor unit 17, and the accuracy of image recognition similarly decreases. When the time determination processing unit 11b determines that the current time is in a time zone in which the accuracy of image recognition by the in-vehicle image recognition device 10a is expected to decrease, the time determination processing unit 11b gives the image frame acquisition processing unit 11d a variable exposure time image. A variable exposure time image frame acquisition signal for instructing frame acquisition is output.

  The own vehicle position determination processing unit 11c determines whether or not the current position of the vehicle 1 acquired from the car navigation device 20 is in a position where the periphery of the vehicle 1 is dark, such as in the tunnel or near the tunnel entrance. When the vehicle position determination processing unit 11c determines that the current position of the vehicle 1 is in a position where the surroundings are dark, variable exposure that instructs the image frame acquisition processing unit 11d to acquire a variable exposure time image frame. Output temporal image frame acquisition signal.

  The image frame acquisition processing unit 11d has “image update cycle”, “image update cycle division number”, “normal exposure time”, “short exposure” set in the set value storage table stored in the set value storage area 12b. Based on “time” and “long exposure time”, an image frame based on “normal exposure time”, an image frame based on “short exposure time”, and “long exposure time” within one image update period from the image sensor unit 17. The image data of the image frame based on “” is acquired and developed in the image buffer 12 a of the storage unit 12.

  The image contrast determination processing unit 11e is acquired by the image frame acquisition processing unit 11d and developed in the image buffer 12a. The image frame based on the “normal exposure time”, the image frame based on the “short exposure time”, and the “long exposure time” And the contrast of the image data of the image frame based on “” is determined, and whether or not the contrast is within a range determined by the “image contrast upper limit value” and the “image contrast lower limit value” is determined.

  The partial image selection processing unit 11f is the same partial image within a range determined by the “image contrast upper limit value” and the “image contrast lower limit value” among the plurality of image frames, and the “normal exposure time”. The partial image having the highest contrast among the partial image of the image frame based on the above, the partial image of the image frame based on the “short exposure time”, and the partial image of the image frame based on the “long exposure time” is selected.

  The partial image synthesis processing unit 11g performs a process of synthesizing the partial images selected by the partial image selection processing unit 11f into one image. The partial image synthesis processing unit 11g sets the weighting coefficient corresponding to each contrast in the synthesis weighting coefficient storage table stored in the synthesis weighting coefficient storage area 12c to the pixel value of each partial image selected by the partial image selection processing unit 11f. Multiply them into a single image.

  As described above, the image synthesized by the partial image synthesis processing unit 11g becomes an image having an appropriate contrast without “overexposure” or “blackout”, and the recognition accuracy of the image recognition processing based on the image is improved. . Further, by multiplying the pixel value of each partial image selected by the partial image selection processing unit 11f by a weighting coefficient corresponding to each contrast, it becomes possible to synthesize a sharp and clear image.

  The image recognition processing unit 11h is based on the image synthesized by the partial image synthesis processing unit 11g and the image recognition pattern data stored in the image recognition pattern data table stored in the image recognition pattern data storage area 12d. Image recognition processing is performed for other vehicles traveling in front of 1, road marking lines, pedestrians, and the like.

  Specifically, pattern matching is performed between the image data of the object projected on at least a part of the image synthesized by the partial image synthesis processing unit 11g and the image recognition pattern data, and the matching rate is a predetermined value (for example, 80%) or more, it identifies what the object is. Furthermore, the image recognition processing unit 11 h displays the image recognition processing result on the display unit 18 via the display output interface unit 14. Further, the image recognition processing result is transferred to a vehicle situation determination processing unit 15 described later.

  Furthermore, the vehicle situation determination device 20a is an in-vehicle network input / output that is an interface for communication with the above-described in-vehicle image recognition device 10a, the vehicle situation determination processing unit 15, and other in-vehicle devices via the in-vehicle network 100. And an interface unit 16.

  The vehicle situation determination processing unit 15 is a control device such as a microcomputer. Based on the image recognition processing result delivered from the image recognition processing unit 11h of the in-vehicle image recognition device 10a, the vehicle situation determination processing unit 15 is connected to other vehicles with high collision risk at a short distance in front of the vehicle 1. It is determined whether there is a pedestrian. If it is determined that another vehicle or pedestrian with high risk of collision exists at a close distance in front of the vehicle 1, an alarm is given to the vehicle control ECU device 30 via the in-vehicle network input / output interface unit 16. A safety ensuring signal is output to instruct output control of the device 30a, braking control of the brake control device 30b, fuel or energy supply suppression control of the motor control device 30c, and seat belt lock control of the seat belt control device 30d.

  When the vehicle state determination processing unit 15 of the vehicle state determination device 20 a outputs a safety ensuring signal to the vehicle control ECU device 30, the vehicle control ECU device 30 automatically performs braking control of the vehicle 1. Thus, it is possible to automatically avoid a collision with another vehicle or a pedestrian that exists at a close distance in front of the vehicle 1 and has a high risk of collision, thereby preventing an accident from occurring.

  Next, an image recognition process executed by the in-vehicle image recognition apparatus 10a according to the first embodiment illustrated in FIG. 2 will be described. FIG. 6 is a flowchart of an image recognition processing procedure executed by the in-vehicle image recognition apparatus 10a according to the first embodiment.

  As shown in the figure, first, the time determination processing unit 11b determines whether or not the time is a time zone that is susceptible to backlight such as evening or a dark time zone such as night (step S101). If it is determined that the time is a time zone that is susceptible to backlight such as evening or a dark time zone such as night (Yes in step S101), the process proceeds to step S102, and the time is a time zone or night that is susceptible to backlight such as evening. When it is not determined that it is a dark time zone such as (No at step S101), the process proceeds to step S111.

  In step S102, the host vehicle position determination processing unit 11c determines whether or not the position information of the host vehicle acquired from the car navigation device 20 is a dark position such as a tunnel. Since the car navigation apparatus 20 has a map database, it is possible to determine whether or not the position information of the vehicle is a dark position such as a tunnel based on the map information stored in the map database. Is possible.

  When it is determined that the position information of the own vehicle acquired from the car navigation device 20 is a dark position such as a tunnel (Yes in step S102), the process proceeds to step S103 and the vehicle information acquired from the car navigation device 20 is determined. If the position information is not determined to be a dark position such as a tunnel (No at Step S102), the process proceeds to Step S111.

  Subsequently, in step S103, the image frame acquisition processing unit 11d acquires the first image frame at “normal exposure time”. Subsequently, the image contrast determination processing unit 11e extracts a partial image whose contrast is within a predetermined range (that is, within a range determined by the “image contrast upper limit value” and the “image contrast lower limit value”) from the image of the first image frame. At the same time, the contrast of the partial image is calculated (step S104).

  Subsequently, the image frame acquisition processing unit 11d acquires the second image frame with “short exposure time” (step S105). Subsequently, the image contrast determination processing unit 11e extracts a partial image whose contrast is within a predetermined range (that is, within a range determined by the “image contrast upper limit value” and the “image contrast lower limit value”) from the image of the second image frame. At the same time, the contrast of the partial image is calculated (step S106).

  Subsequently, the image frame acquisition processing unit 11d acquires the third image frame with “long exposure time” (step S107). Subsequently, the image contrast determination processing unit 11e extracts a partial image whose contrast is within a predetermined range (that is, within a range determined by the “image contrast upper limit value” and the “image contrast lower limit value”) from the image of the third image frame. At the same time, the contrast of the partial image is calculated (step S108).

  Subsequently, the partial image selection processing unit 11f selects a partial image having the highest contrast among the same partial images extracted in the processes of step S104, step S106, and step S108. Then, for all partial images in the image, the partial image having the highest contrast is selected from the partial images extracted in the processes of step S104, step S106, and step S108. Then, the partial image composition processing unit 11g synthesizes the partial images selected from the images of the first image frame, the second image frame, and the third image frame by a predetermined weighting based on the contrast value (the above steps). S109).

  Subsequently, the image recognition processing unit 11h performs image recognition processing using the image synthesized by the partial image synthesis processing unit 11g (step S110). When this process ends, the image recognition process of the first embodiment ends. Then, the image recognition processing unit 11h displays the image recognition processing result on the display unit 18 via the display output interface unit 14, and transfers it to the vehicle situation determination processing unit 15 of the vehicle situation determination device 20a.

  On the other hand, in step S111, as in step S103, the image frame acquisition processing unit 11d acquires the first image frame with “normal exposure time”. Subsequently, the image recognition processing unit 11h performs image recognition processing using the image of the first image frame (step S112). When this process ends, the image recognition process of the first embodiment ends.

  According to the image recognition processing of the first embodiment described above, when the time is a time zone that is susceptible to backlight such as evening or a dark time zone such as night, or the position information of the own vehicle is dark such as in a tunnel. Since the variable exposure time image frame acquisition is not performed when the position or tunnel entrance / exit is a position where “whiteout” or “blackout” is likely to occur, unnecessary processing is omitted and the processing load of the in-vehicle image recognition device 10a is reduced. Can be reduced.

  Next, an example of an image when variable exposure time image frame acquisition is performed by the in-vehicle image recognition apparatus according to the first embodiment will be described. FIG. 7A to FIG. 7D are diagrams of examples of images when variable exposure time image frames are acquired by the in-vehicle image recognition apparatus according to the first embodiment.

  First, FIG. 7A is a diagram illustrating an example of an image in the vicinity of the tunnel exit captured by [normal exposure time] by the in-vehicle image recognition apparatus 10a. As shown in the figure, the illumination in the tunnel and the white line in the tunnel are clearly imaged and are at a level where the image can be recognized. However, the front vehicle A and the front vehicle B are only images of the tail lamp, and are not at a level where image recognition is possible. Further, “out-of-white” occurs in the portion outside the tunnel exit, and image recognition is impossible. Therefore, in the image shown in FIG. 7A, the contrast is excessive outside the tunnel exit, and there is a portion where the contrast is insufficient inside the tunnel, so that the image is inappropriate for the image recognition processing.

  Next, the in-vehicle image recognition apparatus 10a captures an image near the tunnel exit in [short exposure time]. FIG. 7B is a diagram illustrating an example of an image in the vicinity of the tunnel exit captured with [short exposure time] by the in-vehicle image recognition apparatus 10a. As shown in the figure, in the portion outside the tunnel exit, “whiteout” does not occur, the forward vehicle C and the white line outside the tunnel are clearly imaged, and are at a level where image recognition is possible. However, “blackout” occurs in the tunnel, and image recognition is impossible. Therefore, the image shown in FIG. 7-2 is inappropriate for the image recognition process because the contrast inside the tunnel is insufficient.

  Next, the in-vehicle image recognition apparatus 10a captures an image near the tunnel exit in [long exposure time]. FIG. 7C is a diagram illustrating an example of an image in the vicinity of the tunnel exit that is captured with [long exposure time] by the in-vehicle image recognition apparatus 10a. As shown in the figure, the white line in the tunnel, the forward vehicle A and the forward vehicle B inside the tunnel are clearly imaged and are at a level where image recognition is possible. However, tunnel illumination is unsuitable for image recognition due to excessive contrast. Further, “out-of-white” occurs in the portion outside the tunnel exit, and image recognition is impossible. Therefore, the image shown in FIG. 7-3 is also inappropriate for the image recognition process because the contrast outside the tunnel interior and the tunnel exit is excessive.

  Therefore, the in-vehicle image recognition apparatus 10a according to the first embodiment selects a partial image of the tunnel illumination from the image near the tunnel exit imaged in [normal exposure time] in FIG. A partial image of the portion outside the tunnel exit is selected from the images near the tunnel exit taken at [Short exposure time], and the white line inside the tunnel is taken from the image near the tunnel exit taken at [Long exposure time] in FIG. The partial images of the forward vehicle A and the forward vehicle B inside the tunnel are selected, and these partial images are combined. It should be noted that the partial image of the white line in the tunnel was selected from the images near the tunnel exit taken at [long exposure time] because the contrast of the partial image of the white line in the tunnel was picked up at [normal exposure time]. This is because the contrast of the partial image of the white line in the tunnel of the nearby image is larger.

  The image synthesized in this way is as shown in FIG. According to the image of FIG. 7-4, all of the lighting in the tunnel, the white line in the tunnel, the forward vehicle A and the forward vehicle B inside the tunnel, the forward vehicle C outside the tunnel exit and the white line outside the tunnel are suitable for the image recognition process. The image is clearly displayed due to the contrast, and the accuracy of the image recognition process can be improved.

  The configurations of the in-vehicle image processing apparatus and the vehicle state determination apparatus according to the second embodiment will be described below. FIG. 8 is a functional block diagram of the configuration of the in-vehicle image processing device and the vehicle state determination device according to the second embodiment. As shown in the figure, the in-vehicle image recognition device 10b and the vehicle situation determination device 20b according to the second embodiment are similar to the in-vehicle image recognition device 10a and the vehicle situation determination device 20a according to the first embodiment. The car navigation device 20 that passes car navigation information, date information, time information, and the like to the in-vehicle image recognition device 10a via the in-vehicle network 100 such as CAN (Controller Area Network), and the electronic device included in the vehicle 1 are controlled. The vehicle control ECU (Electronic Control Unit) 30 is connected to be communicable.

  In the following description of the second embodiment, only differences from the first embodiment will be described. The configuration, function, and processing that are not specified are the same as those of the in-vehicle image recognition device 10a and the vehicle situation determination device 20a according to the first embodiment.

  The in-vehicle image recognition apparatus 10b according to the second embodiment has a configuration in which the partial image selection processing unit 11f, the partial image synthesis processing unit 11g, and the synthesis weighting coefficient storage area 12c are omitted from the in-vehicle image recognition apparatus 10a according to the first embodiment. It is.

  The image frame acquisition processing unit 11d acquires an image frame while changing the exposure time until it is determined that the contrast of the acquired image frame is a value within a predetermined range within one period of the image update period. In addition, the image recognition processing unit 11h performs image recognition processing based on the image for which the contrast is determined to be a value within a predetermined range for the first time by the image contrast determination processing unit 11e within one cycle of the image update cycle described above. .

  In addition, the image contrast determination processing unit 11e determines the presence or absence of “whiteout” and “blackout” portions of the image. “White skip” of the image is a case where the contrast of the image exceeds the “image contrast upper limit value”, and “blackout” of the image is a case of the image contrast being lower than the “image contrast lower limit value”.

  The vehicle situation determination device 20b according to the second embodiment is an interface for communication with the in-vehicle image recognition device 10b according to the second embodiment, the vehicle situation determination processing unit 15, and other in-vehicle devices via the in-vehicle network 100. And an in-vehicle network input / output interface unit 16. The configuration, function, and processing of the vehicle situation determination processing unit 15 of the vehicle-mounted image recognition device 10b according to the second embodiment are the same as those of the vehicle situation determination processing unit 15 of the vehicle-mounted image recognition device 10a according to the first embodiment.

  Next, image recognition processing executed by the in-vehicle image recognition apparatus according to the second embodiment will be described. FIG. 9 is a flowchart of an image recognition processing procedure executed by the in-vehicle image recognition apparatus according to the second embodiment.

  As shown in the figure, first, the time determination processing unit 11b determines whether or not the time is a time zone in which backlight is easily received such as evening or a dark time zone such as night (step S201). If it is determined that the time is a time zone that is susceptible to backlight such as evening or a dark time zone such as night (Yes in step S201), the process proceeds to step S202, and the time is a time zone or night that is susceptible to backlight such as evening. If it is not determined that it is a dark time zone such as (No at step S201), the process proceeds to step S215.

  In step S202, the own vehicle position determination processing unit 11c determines whether or not the position information of the own vehicle acquired from the car navigation device 20 is a dark position such as a tunnel. Since the car navigation apparatus 20 has a map database, it is possible to determine whether or not the position information of the vehicle is a dark position such as a tunnel based on the map information stored in the map database. Is possible.

  When it is determined that the position information of the own vehicle acquired from the car navigation device 20 is a dark position such as a tunnel (Yes in step S202), the process proceeds to step S203, and the own vehicle acquired from the car navigation device 20 is acquired. When the position information is not determined to be a dark position such as a tunnel (No at Step S202), the process proceeds to Step S215.

  Subsequently, in step S203, the image frame acquisition processing unit 11d acquires the first image frame at “normal exposure time”. Subsequently, the image contrast determination processing unit 11e determines whether or not there is a “overexposed” portion in the image of the first image frame (that is, whether or not there is a portion where the contrast of the image exceeds the “image contrast upper limit value”). ) Is determined (step S204).

  If it is determined that there is a “whiteout” portion in the image of the first image frame (Yes in step S204), the process proceeds to step S205, and it is not determined that there is a “whiteout” portion in the image of the first image frame. If so (No at step S204), the process proceeds to step S210.

  In step S205, the image frame acquisition processing unit 11d acquires the second image frame with “short exposure time”. Subsequently, the image contrast determination processing unit 11e determines whether or not there is a “blackout” portion in the image of the second image frame (that is, whether or not there is a portion where the image contrast is less than the “image contrast lower limit value”). Is determined (step S206).

  When it is determined that there is a “blackout” portion in the image of the second image frame (Yes in step S206), the process proceeds to step S207, and it is not determined that there is a “blackout” portion in the image of the second image frame. If (No at step S206), the process proceeds to step S209.

  In step S207, the image frame acquisition processing unit 11d acquires the third image frame with “long exposure time”. Subsequently, the image recognition processing unit 11h performs image recognition using the third image frame acquired in step S207 (step S208). When this process ends, the image recognition process of the second embodiment ends.

  On the other hand, in step S209, the image recognition processing unit 11h performs image recognition using the second image frame. When this process ends, the image recognition process of the second embodiment ends.

  On the other hand, in step S210, the image contrast determination processing unit 11e determines whether or not there is a “blackout” portion in the image of the first image frame (that is, whether there is a portion where the image contrast is less than the “image contrast lower limit value”). Or not).

  When it is determined that there is a “blackout” portion in the image of the first image frame (Yes in step S210), the process proceeds to step S211 and it is not determined that there is a “blackout” portion in the image of the second image frame. If so (No at step S210), the process proceeds to step S216.

  In step S211, the image frame acquisition processing unit 11d acquires the second image frame with “long exposure time”. Subsequently, the image contrast determination processing unit 11e determines whether or not there is a “overexposed” portion in the image of the second image frame (that is, whether or not there is a portion where the contrast of the image exceeds the “image contrast upper limit value”). ) Is determined (step S212).

  When it is determined that the “whiteout” portion is present in the image of the second image frame (Yes in step S212), the process proceeds to step S213, and it is not determined that the “whiteout” portion is present in the image of the second image frame. If this is the case (No at step S212), the process proceeds to step S209.

  In step S213, the image frame acquisition processing unit 11d acquires the third image frame with “short exposure time”. Subsequently, the image recognition processing unit 11h performs image recognition using the third image frame acquired in step S213 (step S214). When this process ends, the image recognition process of the second embodiment ends.

  In step S215, as in step S203, the image frame acquisition processing unit 11d acquires the first image frame with “normal exposure time”. Subsequently, the image recognition processing unit 11h performs image recognition using the first image frame acquired in step S215 (step S216). When this process ends, the image recognition process of the second embodiment ends.

  According to the image recognition processing of the second embodiment described above, when an image frame having no “whiteout” or “blackout” and having a contrast within a predetermined range is acquired, subsequent variable exposure time image frame acquisition is performed. Since this is not performed, unnecessary processing can be omitted, and the processing load of the in-vehicle image recognition device 10b can be reduced.

  As mentioned above, although Example 1 and Example 2 of this invention were demonstrated, this invention is not restricted to this, Furthermore, various different Example is within the range of the technical idea described in the claim. May be implemented. Moreover, the effect described in Example 1 and Example 2 is not limited to this.

  In addition, among the processes described in the first and second embodiments, all or a part of the processes described as being automatically performed can be manually performed, or manually performed. All or a part of the processing described as can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, information including various data and parameters shown in the above embodiment can be arbitrarily changed unless otherwise specified.

  Each component of each illustrated device is functionally conceptual and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured.

  Furthermore, each or all of the processing functions performed in each device are entirely or partially a CPU (Central Processing Unit) (or a microcomputer such as an MPU (Micro Processing Unit) or MCU (Micro Controller Unit)) and It may be realized by a program that is analyzed and executed by the CPU (or a microcomputer such as MPU or MCU), or may be realized as hardware by wired logic.

  The present invention performs a process of capturing a plurality of images with different exposure times in an in-vehicle image recognition apparatus and combining the plurality of images into a single image so as to increase the accuracy of image recognition processing with a reduced processing load. This is useful when it is desired to secure a high dynamic range of the camera of the in-vehicle image recognition apparatus.

It is a figure which shows the outline | summary and characteristic of this invention. 1 is a functional block diagram illustrating configurations of an in-vehicle image recognition device and a vehicle situation determination device according to a first embodiment. It is a figure which shows the example of a setting value storage table. It is a figure which shows the example of a synthetic | combination weighting coefficient storage table. It is a figure which shows the example of an image recognition pattern data table. 3 is a flowchart illustrating an image recognition processing procedure executed by the in-vehicle image recognition apparatus according to the first embodiment. It is a figure which shows the example of the image of the tunnel exit vicinity imaged with the vehicle-mounted image recognition apparatus concerning Example 1 by normal exposure time. It is a figure which shows the example of the image of the tunnel exit vicinity imaged with the vehicle-mounted image recognition apparatus concerning Example 1 with short exposure time. It is a figure which shows the example of the image of the tunnel exit vicinity imaged with the vehicle-mounted image recognition apparatus concerning Example 1 with long exposure time. It is a figure which shows the example of the image which synthesize | combined the image of the tunnel exit vicinity imaged with the vehicle-mounted image recognition apparatus concerning Example 1 with normal exposure time, short exposure time, and long exposure time. It is a functional block diagram which shows the structure of the vehicle-mounted image recognition apparatus and vehicle condition determination apparatus concerning Example 2. FIG. 12 is a flowchart illustrating an image recognition processing procedure executed by the in-vehicle image recognition apparatus according to the second embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle 10a, 10b Car-mounted image recognition apparatus 11 Control part 11a Imaging process part 11b Time determination process part 11c Own vehicle position determination process part 11d Image frame acquisition process part 11e Image contrast determination process part 11f Partial image selection process part 11g Partial image Composition processing unit 11h Image recognition processing unit 12 Storage unit 12a Image buffer 12b Setting value storage area 12c Composition weighting coefficient storage area 12d Image recognition pattern data storage area 13 Image sensor interface unit 14 Display output interface unit 15 Vehicle condition determination processing unit 16 In-vehicle Network input / output interface unit 17 Image sensor unit 17a Optical lens 17b Shutter unit 17c CMOS sensor 18 Display unit 20 Car navigation device 20a, 20b Vehicle situation determination device 30 Vehicle control ECU device 30a Alarm device 30b Brake control device 30c Engine control device 30d Seat belt control device 100 In-vehicle network

Claims (7)

An in-vehicle image recognition device that performs image recognition based on an image around a vehicle imaged by an imaging device provided in the vehicle,
Imaging processing means for controlling the imaging apparatus so as to capture a plurality of images with different exposure times within one period of an image update period;
Contrast determination means for determining whether or not the contrast of each of the plurality of images captured by the imaging device is controlled by the imaging processing means is a value within a predetermined range;
Extracting a partial image of the same part of each of the plurality of images determined by the contrast determination means to have a contrast within a predetermined range of the contrast of each of the plurality of images, from the partial image of the same part of the plurality of images Partial image selection means for selecting each partial image with the highest contrast,
Image combining means for combining each partial image selected by the partial image selecting means into one image;
An in-vehicle image recognition apparatus comprising: an image recognition processing unit that performs image recognition processing based on the one image combined by the image combining unit. And further comprising weighting coefficient storage means for storing a weighting coefficient corresponding to the contrast of the partial images of the plurality of images when the image combining means combines the partial images of the plurality of images into one image. ,
The image synthesizing unit multiplies the pixel value of each partial image by a weighting coefficient corresponding to the contrast of each partial image selected by the partial image selecting unit to convert the partial images of the plurality of images into one image. The vehicle-mounted image recognition device according to claim 1, wherein An in-vehicle image recognition device that performs image recognition based on an image around a vehicle imaged by an imaging device provided in the vehicle,
Imaging processing means for controlling the imaging apparatus so as to capture a plurality of images with different exposure times within one period of an image update period;
Contrast determination means for determining whether or not the contrast of each of the plurality of images captured by the imaging device is controlled by the imaging processing means is a value within a predetermined range;
Image recognition processing means for performing image recognition processing based on an image captured by the imaging device;
The imaging processing unit is configured to perform the predetermined exposure time until an image in which the contrast determination unit determines that the contrast of the predetermined image is a value within a predetermined range within one cycle of the image update cycle. And controlling the imaging device to capture images with different exposure times,
The image recognition processing means performs image recognition processing based on an image that is determined for the first time that the contrast is a value within a predetermined range by the contrast determination means within one period of the image update period. In-vehicle image recognition device. Based on information acquired from an in-vehicle device provided in the vehicle, it is determined whether or not the imaging device needs to capture a plurality of images with the different exposure times within one cycle of the image update cycle. A plurality of image capturing necessity determination means;
The imaging processing means updates the image only when it is determined by the multiple image imaging necessity determination means that there is a need to capture a plurality of images within the different period of the image update period. The in-vehicle image recognition apparatus according to claim 1, wherein the imaging apparatus is controlled so as to capture a plurality of images with the different exposure times within one period. A vehicle control device mounted on the vehicle for determining a situation where the vehicle is placed based on a result of image recognition based on an image around the vehicle imaged by an imaging device provided in the vehicle; A vehicle status determination device capable of communicating via an in-vehicle network,
Imaging processing means for controlling the imaging apparatus so as to capture a plurality of images with different exposure times within one period of an image update period;
Contrast determination means for determining whether or not the contrast of each of the plurality of images captured by the imaging device by the imaging processing means is a value within a predetermined range;
Extracting a partial image of the same portion of each of the plurality of images, in which the contrast of the plurality of images is determined to be a value within a predetermined range by the contrast determination unit, and extracting from the partial image of the same portion of the plurality of images Partial image selection means for selecting each partial image with the highest contrast,
Image combining means for combining the partial images selected by the partial image selecting means into one image;
Image recognition processing means for performing image recognition processing based on the one image synthesized by the image synthesis means;
A situation around the vehicle is determined based on a result of the image recognition processing performed by the image recognition processing means, and when the braking control of the vehicle is required according to the determination result, the vehicle control device is referred to. Vehicle status determination processing means for outputting a brake control signal. A vehicle control device mounted on the vehicle for determining a situation where the vehicle is placed based on a result of image recognition based on an image around the vehicle imaged by an imaging device provided in the vehicle; A vehicle status determination device capable of communicating via an in-vehicle network,
Imaging processing means for controlling the imaging apparatus so as to capture a plurality of images with different exposure times within one period of an image update period;
Contrast determination means for determining whether or not the contrast of each of the plurality of images captured by the imaging device is controlled by the imaging processing means is a value within a predetermined range;
Image recognition processing means for performing image recognition processing based on an image captured by the imaging device;
A situation around the vehicle is determined based on the result of the image recognition processing performed by the image recognition processing means, and when the braking control of the vehicle is necessary according to the result of the determination, the vehicle control device is referred to. Vehicle condition determination processing means for outputting a braking control signal,
The imaging processing unit is configured to perform the predetermined exposure time until an image in which the contrast determination unit determines that the contrast of the predetermined image is a value within a predetermined range within one cycle of the image update cycle. And controlling the imaging device to capture images with different exposure times,
The image recognition processing means performs image recognition processing based on an image that is determined for the first time that the contrast is a value within a predetermined range by the contrast determination means within one period of the image update period. Vehicle status determination device. An in-vehicle image recognition method in which an in-vehicle image recognition device performs image recognition based on an image around a vehicle imaged by an imaging device provided in the vehicle,
Multi-image imaging necessity determination that determines whether or not there is a need to capture a plurality of images with different exposure times within one period of an image update period based on information acquired from an in-vehicle device provided in the vehicle Steps,
If it is determined by the multiple image imaging necessity determination step that there is a need to capture a plurality of images at the different exposure times within one cycle of the image update cycle, within one cycle of the image update cycle An imaging processing step of controlling the imaging device so as to capture a plurality of images at the different exposure times,
A contrast determination step for determining whether or not the contrast of each of the plurality of images captured by the control of the imaging device in the imaging processing step is a value within a predetermined range;
Extracting a partial image of the same part of each of the plurality of images, in which the contrast of each of the plurality of images is determined to be a value within a predetermined range by the contrast determination step, from the partial image of the same part of the plurality of images A partial image selection step for selecting each partial image with the highest contrast,
The partial images of the plurality of images selected by the partial image selection step are multiplied by the pixel values of the partial images by a weighting coefficient corresponding to the contrast of the partial images of the plurality of images. An image synthesizing step for synthesizing the partial images into one image;
An in-vehicle image recognition method comprising: an image recognition process step of performing an image recognition process based on the one image synthesized by the image synthesis step.

Images