JP2011152865A - On-vehicle image pickup device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an on-vehicle image pickup device which prevents an invisible area of a picked-up image from obstructing a vision auxiliary function of the on-vehicle image pickup device. <P>SOLUTION: The on-vehicle image pickup device includes: an image pickup means 1 mounted on a vehicle; a position detection means 5, which detects the position of the vehicle; a memory means 17, which associates an image obtained by the image pickup means 1 with the position information detected by the position detection means 5 to memorize them; an invisible area judgment means 13, which judges whether or not the image picked up by the image pickup means 1 has an invisible area; and an image composition means 14, which, when it is judged by the invisible area judgment means 13 that an invisible exists, extracts a past image matching the position of a vehicle detected by the position detection means 5 from the memory means 17 and combines the past image into the present image picked up by the image pickup device. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an in-vehicle image pickup apparatus that picks up an image of the periphery of a host vehicle using image pickup means mounted on the vehicle.

Conventionally, an in-vehicle imaging device has been put into practical use in which an imaging unit is mounted on a vehicle and an image captured by the imaging unit is displayed on an in-vehicle monitor to improve safety and operability when the vehicle is running or parked. Has been.
As one specific function of the in-vehicle imaging device, there is a function of assisting the driver's field of view by displaying a real-time actual scene image captured by the imaging means in the vehicle. Furthermore, in recent years, a technique has been proposed in which a captured image is stored in a storage unit and this stored image is used in a navigation device or the like.

  For example, Patent Document 1 (Japanese Patent Application Laid-Open No. 2004-151063) discloses a configuration in which a vehicle having a navigation function for displaying a route to a destination displays a real scene image actually captured in addition to map information. Has been. Further, Patent Document 1 stores a captured image and additional information including position information, weather, and time zone in association with each other, extracts an image that most closely matches the current position and the traveling direction, and passes this image to the route. In addition to the above, it has a configuration to display.

  Further, in Patent Document 2 (Japanese Patent Laid-Open No. 11-304500), an image is captured and stored in correspondence with the travel position, and when performing guidance in the vicinity of the travel position, the stored image is read out and displayed on the guidance screen. A configuration to be displayed in the above is disclosed.

JP 2004-151063 A JP-A-11-304500

  However, conventional in-vehicle imaging devices that provide visual assistance by displaying captured images in the vehicle in real time are not visible in the captured images at night or when the imaging range is partially obstructed by an obstacle. There may be a region that becomes. When an invisible region exists in the captured image, there is a problem that the visual assistance function of the in-vehicle imaging device, such as improvement in driving safety and operability, cannot be sufficiently exhibited.

  On the other hand, Patent Document 1 aims to display a real scene image in addition to the route display of the navigation function and to guide the driver effectively, and does not mention the invisible region of the captured image. . Patent Document 2 displays a real scene image stored in accordance with vehicle position information and indicates a travel position including visual information. Like Patent Document 1, there is no mention of an invisible region. Even using the conventional technology, it was impossible to solve the problem caused by the invisible region of the captured image.

  Therefore, an object of the present invention is to provide a vehicle-mounted imaging device that can prevent the visual assistance function of the vehicle-mounted imaging device from being hindered by an invisible region of a captured image.

  In order to solve the above-described problems, an in-vehicle imaging device according to the present invention includes an imaging unit mounted on a vehicle, a position detection unit that detects a position of the vehicle, an image obtained by the imaging unit, and the position detection unit. Storage means for associating and storing the positional information detected by the imaging means, invisible area determination means for determining whether or not an image captured by the imaging means exists, and the image by the invisible area determination means When it is determined that there is an invisible region, a past image that matches the position of the vehicle detected by the position detection unit is extracted from the storage unit, and the past image is converted into a current image captured by the imaging unit. And image synthesizing means for synthesizing.

  According to the present invention, when it is determined that an invisible area exists in the captured image, the invisible area of the current image is interpolated with the past image, so that the visual assistance function is reliably realized regardless of the situation around the vehicle. it can. Therefore, driving safety and operability can be further improved by using the in-vehicle imaging device of the present invention. Note that the past image to be combined with the current image may be a partial region of the image.

In addition, the invisible area determination means includes the illuminance sensor attached to the vehicle and the invisible area when the illuminance around the vehicle detected by the illuminance sensor is lower than a preset threshold value. It is preferable that a determination means for determining is included.
As described above, the presence or absence of the invisible region can be easily and reliably determined by detecting that the surroundings of the vehicle are dark due to nighttime or bad weather using the illuminance sensor. The illuminance sensor may be an image sensor.

Further, the invisible area determination means extracts a preset interpolation target from the past image, and determines whether or not the invisible area exists based on a difference between the current image and the interpolation target of the past image. It is preferable to include a judging means for judging.
Thus, by comparing the past image with the current image, it is possible to accurately determine the presence or absence of an invisible region, particularly when an invisible region exists due to the presence of an obstacle.

Furthermore, it is preferable that the past image to be combined with the current image is at least one of a sign, a white line, a guardrail, and a fixed installation in a parking lot.
Signs, white lines, guardrails, and fixed installations in parking lots must be reliably recognized when the vehicle is running or parked. In this configuration, when these are invisible in the captured image, they are extracted from the past image and combined with the current image, so that the user can recognize them reliably and improve driving safety. It becomes.

The apparatus further includes a communication unit that communicates with a database in which the past image and the position information are associated and stored, and the communication unit acquires the past image corresponding to the position of the vehicle from the database together with the position information. It is preferable to do.
Thereby, it becomes possible to acquire a past image even in a place where the vehicle is not traveling.

  Furthermore, it is preferable to further include display means for displaying the interpolated image synthesized by the image synthesizing means, so that the user in the vehicle can easily confirm the interpolated image.

  According to the present invention, when it is determined that an invisible area exists in the captured image, the invisible area of the current image is interpolated with the past image, so that the visual assistance function is reliably realized regardless of the situation around the vehicle. it can. Therefore, driving safety and operability can be further improved by using the in-vehicle imaging device of the present invention.

1 is an overall configuration diagram of an in-vehicle imaging device according to an embodiment of the present invention. It is a flowchart which shows operation | movement of the vehicle-mounted imaging device which concerns on embodiment of this invention. It is a flowchart which shows the interpolation process of an image. It is a figure which shows an example of the past image imaged in the daytime. It is a figure which shows an example of the present image imaged at night. It is a figure which shows an example of an interpolation image. It is a flowchart which shows the recording process of an image. It is a figure which shows the database table of image recording information. It is a whole block diagram of the vehicle-mounted imaging device which concerns on the modification of this embodiment. It is a flowchart which shows operation | movement of the vehicle-mounted imaging device which concerns on the modification of this embodiment.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the shape of the component described in this example is not intended to limit the scope of the present invention, but is merely an illustrative example.

FIG. 1 is an overall configuration diagram of an in-vehicle imaging device according to an embodiment of the present invention.
The in-vehicle imaging device mainly includes a camera module 1 that is an imaging unit, a position detection device 5 that detects the position of the vehicle, a storage unit 17, a recording processing unit 12, an invisible region determination unit 13, and an image synthesis unit 14. And a processing apparatus 10 having a calculation means 11 including the same. More preferably, an input device 21 and a display device 22 are provided.

  The camera module 1 is mounted on a vehicle and images the surroundings of the vehicle. The imaging range of the camera module 1 is not particularly limited. For the camera module 1, for example, a front view camera that images the front of the vehicle, a rear view camera that images the rear of the vehicle, and the like are used. Specifically, the camera module 1 includes a lens 2, an image sensor 3, and driving means 4. The lens 2 is preferably a fish-eye lens. The driving unit 4 drives devices constituting the camera module 1 such as the lens 2. The image captured by the camera module 1 may be a still image or a moving image.

  The position detection device 5 detects the current position of the vehicle. The position detection device 5 is, for example, a device that has a GPS receiver and acquires positioning information by a GPS system, a device that has a gyro sensor, an acceleration sensor, or the like and detects the current position of the vehicle by an autonomous navigation system, or A device that detects the current position of a vehicle using both a GPS system and an autonomous navigation system is used.

The processing device 10 is mounted on a vehicle and includes a calculation unit 11 including hardware such as an MPU and a CPU, and a storage unit 17 that stores captured images and the like. The calculation means 11 functionally includes a recording processing means 12, an invisible area determination means 13, and an image composition means 14, and reads out and executes a software program for realizing each of these means from a program memory (not shown). Thereby, each means of the calculation means 11 is implement | achieved.
A captured image captured by the camera module 1 is input to the processing device 10.

  The recording processing unit 12 performs a recording process in which the captured image captured by the camera module 1 is stored in the storage unit 17 in association with the vehicle position information detected by the position detection device 5. Further, in the captured image, incidental information other than the position information, for example, information such as imaging date and time, weather, season, etc. may be stored in association with the captured image.

  The invisible area determination unit 13 determines whether or not an invisible area exists in a captured image around the current vehicle (hereinafter referred to as a current image) captured by the camera module 1. An invisible area is not captured in the captured image for whatever reason, such as an area where the surroundings of the vehicle are dark and the image is not clear at night, or an area that does not appear in the image due to an obstacle. An area.

Preferably, the invisible area determination means 13 includes an illuminance sensor (not shown) attached to the vehicle and an invisible area when the illuminance around the vehicle detected by the illuminance sensor is lower than a preset threshold value. Then, it is configured to include determination means (not shown) for determining. The illuminance sensor is installed at a position different from the illumination range of the vehicle light. The illuminance sensor may be an image sensor.
As described above, the presence or absence of the invisible region can be easily and reliably determined by detecting that the surroundings of the vehicle are dark due to nighttime or bad weather using the illuminance sensor.

The invisible region determination unit 13 includes a determination unit (not shown) that extracts a preset interpolation target from the past image and determines the presence or absence of the invisible region from the difference between the current image and the interpolation target of the past image. It is good.
The preset interpolation target is preferably at least one of a sign, a white line, a guardrail, and a fixed installation in the parking lot. Signs, white lines, guardrails, and fixed installations in parking lots must be reliably recognized when the vehicle is running or parked. In this configuration, when these are invisible in the captured image, they are extracted from the past image and combined with the current image, so that the user can recognize them reliably and improve driving safety. It becomes.

  Since the signs, white lines, width, length, color, etc. are determined, they can be extracted from past images based on their characteristics. Regarding the guardrail and the fixed installation in the parking lot, it is preferable that features such as these shapes are registered in advance together with the installation position information, and extracted from the past image matching the installation position information based on the features. Thus, by comparing the past image with the current image, it is possible to accurately determine the presence or absence of an invisible region, particularly when an invisible region exists due to the presence of an obstacle.

The image synthesizing unit 14 performs a process of synthesizing the past image on the invisible region of the current image and interpolating the current image.
The interpolation processing flow executed by the invisible area determination unit 13 and the image composition unit 14 will be described later.
The storage unit 17 stores image recording information in which a captured image and position information are associated with each other. The captured image stored in the storage unit 17 is referred to as a past image.

Furthermore, the vehicle-mounted imaging device may have an input device 21. The input device 21 inputs a command to the processing device 10 by key input including a touch panel or voice input.
Furthermore, it is preferable that the vehicle-mounted imaging device has a display device 22. Examples of the display device 22 include a monitor installed in the vehicle.

Next, the operation of the in-vehicle imaging device according to the present embodiment will be described with reference to the flowchart of FIG.
When the vehicle starts running (S1), the camera module 1 captures an image of the surroundings of the vehicle (S2). The processing device 10 determines whether or not the recording processing and the interpolation processing are necessary (S3). If neither processing is necessary, the captured current image is displayed as it is on the display device 22 (S4).
When the recording process is necessary, the image recording process is executed (S5). This recording processing flow will be described in detail with reference to FIG.

On the other hand, when the interpolation process is necessary, the invisible area determination means 13 determines whether or not an invisible area exists in the current image (S6). If it is determined that there is no invisible area, the current image is displayed as it is on the display device 22 without performing interpolation processing on the current image, and the presence / absence of the invisible area is determined again from the current captured image. If it is determined that an invisible region exists, the past image stored in the storage unit 17 is combined with the current image to create an interpolated image in which the invisible region is interpolated (S7), and the interpolated image is displayed on the display device 22. Display (S8).
Then, it is determined whether or not the traveling is finished (S9), and when finished, the processing flow is stopped. If not completed, the steps after (S2) are repeated.
Note that the determination of the recording process and the interpolation process described above is performed in conjunction with, for example, ON / OFF of a power supply device that supplies power to the camera module 1, the processing device 10, or the like. The processing may be executed, and the processing may be terminated when the processing is turned off, or an ON / OFF circuit may be provided separately so that the user can select ON / OFF using the input device 21. Good. Further, according to the ON / OFF, only one of the recording process and the interpolation process may be executed, or both processes may be executed in parallel. As a suitable processing flow, when the power supply device is turned on, the processing device 10 determines that the interpolation processing is necessary and executes the interpolation processing. When the invisible region determination means 13 determines that there is no invisible region, the processing device 10 determines that the recording process is necessary and executes the recording process flow.

  As described above, according to the present embodiment, when it is determined that there is an invisible area in the captured image, the invisible area of the current image is interpolated with the past image, so that the visual image can be reliably viewed regardless of the situation around the vehicle. Auxiliary functions can be realized. Therefore, driving safety and operability can be further improved by using the in-vehicle imaging device of the present embodiment.

Here, the interpolation processing flow will be described in detail with reference to FIG.
When the interpolation process is necessary, the position information of the vehicle is acquired by the position detection device 5, and a past image at the same position as the current position of the vehicle is selected from the storage means 17 (S11). As an example, FIG. 4 shows a past image 50A captured in the daytime and stored in the storage means 17, and FIG. 5 shows a real-time current image 50B captured in the nighttime. The past image 50A clearly shows the road 51, the white lines such as the side line 52, the center line 53, the regulatory sign 54, and the signs 55 and 56, but the current image 50B is other than the illumination area 60 by the headlight. Is dark and there is an invisible region 61.

  A preset interpolation target portion is extracted from the past image and stored as interpolation target image information (S12). The object to be interpolated is set in the processing apparatus 10 by the input device 21 in advance. Here, as an example, the interpolation target is set to a white line, a sign, and a fixed installation in the parking lot (hereinafter abbreviated as an installation), the image information of the white line is A1, the image information of the sign is B1, and the image information of the installation is Let C1.

  The difference between the current image and the image information A1, B1, and C1 to be interpolated is calculated, and the portion of the white line, sign, and installation that is not displayed in the current image, that is, the image information to be interpolated in the current image is extracted. (S13). The image information to be interpolated among the white line image information A1 is A2, the image information to be interpolated among the sign image information B1 is B2, and the image information to be interpolated among the image information C1 of the installation is C2.

  Then, the interpolated image information B2 of the sign is synthesized with the current image, and the sign is interpolated (S14). Further, the illuminance sensor detects the illuminance around the vehicle and determines whether the illuminance is lower than a preset threshold value (S15). If the illuminance is equal to or greater than the threshold value, only the sign is interpolated. The interpolated image is displayed on the display device 22 (S19). If the illuminance is lower than the threshold, the white line interpolation image information A2 is combined with the current image, and the white line is interpolated into the current image (S16). FIG. 6 shows an interpolated image 50C in which the signs 55 and 56 and the white lines 52, 53 and 54 are interpolated.

If the vehicle is parked, it is determined whether or not the vehicle is back parked (S17). If the vehicle is not back parked, an interpolated image obtained by interpolating only the sign and the white line is displayed on the display device 22 (S19). In the case of back parking, the interpolation image information C2 of the installation object is combined with the current image, the installation object is interpolated into the current image (S18), and then the current image in which the sign, the white line, and the installation object are interpolated is displayed 22 (S19). The determination as to whether or not the vehicle is back parked may be made by detecting the shift range by the shift operation means of the vehicle and determining that the vehicle is back parked when the reverse mode is set, or has an autonomous navigation function. In this case, the traveling direction of the vehicle may be detected and it may be determined that the vehicle is back parked.
Thereafter, it is detected whether or not the power is OFF (S20). If it is OFF, the process is terminated, and if it is ON, the steps after (S11) are repeated.

The above-described interpolation processing flow is an example. For the sign, the sign image information extracted from the past image is compared with the current image so that the sign is unconditionally interpolated when the sign is not displayed in the current image. ing. On the other hand, the white line is interpolated when the white line is not currently displayed in the image and the illuminance is lower than the threshold value. This is because when the surroundings of the vehicle are bright, it is possible to drive safely with other indicators such as the road width and other vehicles even if the white line is not displayed. In addition, the fixed installation in the parking lot can be safely parked by interpolating only when the back parking is being performed.
In this way, a plurality of invisible area determination means 13 may be combined, or the invisible area determination means 13 and an interpolation target to be combined with the current image may be appropriately combined, and only a necessary interpolation target may be combined. As a result, the processing time can be shortened compared to the case where all of the interpolation targets are combined.

Furthermore, it is preferable to store the past image in the storage unit 17 after removing the dynamic object from the past image by filtering.
An example of a method for detecting a dynamic object from a captured image is shown. First, the two capture images are combined and the absolute value of the difference is used as a detection image. At this point, the same design cancels out to become a black bit, and only the different design emerges with a value. The difference image has lightness of 256 gradations × RGB 3 colors for each pixel (from black of 0 to white of RGB for each of RGB). The dynamic object can be detected by counting the lightness for determination in the histogram table of the difference image, that is, the bit more than the threshold value for each color.
Light source, vegetation movement, noise, etc. may be optimized and excluded by adjusting the threshold and sensitivity.

Further, the presence of an object can be detected by detecting a pixel in which a color difference is extremely seen between adjacent pixels, and the accuracy of dynamic object detection can be improved.
When the host vehicle is traveling, it is a collection of still images taken continuously, so that images at substantially the same position taken at a frame rate of about 1/200 second can be acquired. Furthermore, when sharing past images via a network as will be described later, from the captured images of other vehicles that have traveled on the same route, the same place that differs only in the situation of dynamic objects such as humans, fallen leaves, and traveling vehicles. A large number of images of a landscape can be acquired, and the detection accuracy of a dynamic object can be further increased by using them.

Furthermore, when the past image is combined with the current image, it is preferable to perform a correction process for matching the past image and the visible region of the current image to align the images.
Thereby, it becomes possible to synthesize the past image and the current image with high accuracy.

The recording process flow will be described in detail with reference to FIG. Here, as an example, it is assumed that the position detection device 5 includes a GPS receiver and an autonomous navigation device.
When the recording process is necessary, the GPS receiver starts receiving positioning information (S21). At this time, the GPS receiver determines whether or not the radio wave from the GPS satellite can be received (S22). If the GPS receiver can receive the radio wave, positioning information of the host vehicle is received (S23). On the other hand, if the vehicle is located in a tunnel, underground, or between high-rise buildings that block radio waves and cannot receive radio waves from GPS satellites, vehicle position information is acquired by an autonomous navigation device attached to the vehicle. The acceleration information of the vehicle is obtained by the acceleration sensor (S24), the direction information of the vehicle is obtained by the magnetic sensor (S25), the inclination information is obtained by the inclination sensor (S26), and the angular velocity information is obtained by the gyro sensor. Then (S27), the current vehicle position is calculated by adding the vehicle movement direction and distance calculated from the vehicle information obtained by each sensor to the previously received GPS positioning information. Note that the above-described autonomous navigation device is an example, and any device configuration may be used as long as a device that can acquire the moving direction and distance of the vehicle is provided.

When the vehicle position information is detected in this way, the current position information stored in the vehicle is updated (S28), and the position information is associated with the captured image and stored in the storage means 17 (S29).
Thereafter, it is detected whether or not the power is OFF (S30). If it is OFF, the process is terminated, and if it is ON (S22), the subsequent steps are repeated.

  Note that the positioning method using the GPS described above is based on a single positioning method using a GPS receiver and four GPS satellites, and positioning information obtained using the GPS receiver and four GPS satellites. There is a relative positioning method in which the error is corrected by relative positioning and the accuracy is improved, and there is no particular limitation on which one is used, but in this embodiment, it is more preferable to use the relative positioning method.

Relative positioning methods include D-GPS and RTK-GPS. D-GPS analyzes the code information sent from GPS satellites, finds the distance between the satellite and the receiver from the time when the satellite emits the radio wave and the time when the receiver receives the radio wave. The position of the receiver is acquired from the positional relationship. Further, the positioning information of the vehicle is corrected based on the error obtained from the position information of the reference station installed at the fixed point and the positioning information of the reference station by GPS. RTK-GPS calculates the error in the same way as D-GPS, and corrects the positioning information of the vehicle using the error. This is not the code information but the number and phase of the carrier when measuring the distance to the satellite. The accuracy is higher.
Further, an Internet GPS that performs GPS correction using the Internet as a transmission path may be used.
By using these positioning methods, it is possible to acquire position information with high accuracy.

  Further, in the recording process, it is preferable to store incidental information such as imaging date / time, weather, season, etc. in addition to the position information in the storage unit 17 in association with the captured image. FIG. 8 is a diagram illustrating an example of a database table of image recording information. The image identification number is assigned to the captured image, and the position information including the latitude and longitude of the captured image corresponding to the captured image, the date and time of image capture, and the weather are stored in the image recording information stored for each image identification number. Create a database table. When performing the interpolation process, an appropriate past image is selected based on incidental information such as date and weather, and the current image is interpolated by the selected past image. For example, since a past image captured at night may not have clear signs or white lines, a past image captured at daytime at the same position is selected.

  Next, a modification of the present embodiment will be described with reference to FIGS. This modification is configured to share past images with other vehicles via a network. Detailed descriptions of the same configurations as those of the embodiment shown in FIGS. 1 to 8 are omitted.

FIG. 9 is an overall configuration diagram of an in-vehicle imaging device according to a modification of the present embodiment.
The in-vehicle imaging device mainly includes a camera module 1, a GPS receiver 5 a that is a position detection device, a storage unit 17, a recording processing unit 12, an invisible region determining unit 13, and an image synthesizing unit 14. And a processing unit 10 connected to a network 40. More preferably, an input device 21 and a display device 22 are provided. Here, as an example, a GPS receiver 5a is used as a position detection device, and a case in which navigation means 19 for searching and displaying a route to a destination is further shown is shown.
This in-vehicle imaging device can be connected to the management server 30 via the network 40.

The management server 30 mainly includes a communication device 31 connected to the network 40, a management processing device 32 that performs various arithmetic processes, and a database 33.
The management server 30 receives the image recording information in which the past image and the positional information are associated with each other from the in-vehicle imaging device via the network 40 by the communication device 31 and accumulates the image recording information in the database 33. For example, the database table shown in FIG. 8 is registered in the database 33. Further, the management server 30 similarly receives the image recording information from the other vehicles 35 and 36 and accumulates it in the database 33.

FIG. 10 is a flowchart showing the operation of the in-vehicle imaging device according to the modification of the present embodiment.
First, when the destination is input by the input device 21, the current position is acquired by the GPS receiver 5a, and the route from the current position to the destination is searched and set by the navigation means 19 (S31). Then, the communication means 18 searches the database 33 of the management server 30 via the network 40 for image recording information whose position information matches the set route, downloads it, and stores it in the storage means 17 (S32).

When the vehicle starts running (S33), the camera module 1 captures an image of the vehicle periphery (S34). The processing device 10 determines whether or not the recording processing and the interpolation processing are necessary (S35). If neither processing is necessary, the captured current image or route image is displayed on the display device 22 (S36).
When the recording process is necessary, an image recording process is executed (S37). In this recording process, the image recording information in which the captured image is associated with the current vehicle position information detected by the position detection device 5 is temporarily stored in the storage unit 17 and managed appropriately via the network 40 by the communication unit 18. To the server 30. In the management server 30, the received image recording information is stored in the database 33.

On the other hand, when interpolation processing is necessary, the invisible area determination means 13 determines whether or not an invisible area exists in the current image (S38). If it is determined that there is no invisible area, the current image is displayed as it is on the display device 22 without performing interpolation processing on the current image, and the presence / absence of the invisible area is determined again from the current captured image. If it is determined that an invisible area exists, the past image downloaded to the storage means 17 is combined with the current image to create an interpolated image in which the invisible area is interpolated (S39), and the interpolated image is displayed on the display device 22. Display (S40).
Then, it is determined whether or not the traveling is finished (S41), and when finished, the processing flow is stopped. If not completed, the steps after (S34) are repeated.
As described above, the past image can be acquired even when the vehicle is not traveling by adopting a configuration in which the past image can be received via the network 40.

As an example of use of this embodiment and the modified example, for example, in a situation where visibility is poor such as at night or in bad weather, interpolation targets such as white lines and signs existing in an invisible region of the current image are interpolated into the current image and displayed on the monitor. Assist driving.
Also, if a large vehicle such as a truck stops in front of the vehicle at an intersection, or if a sign or a landmark to the destination is hidden by an obstacle on the road, the truck or obstacle ahead The image is displayed on the monitor so that the sign and the mark to the destination can be confirmed.
Also, if you park in a place where you park frequently, such as at home or at work, visibility will be worse at night compared to daytime, so if there are obstacles in the parking place, you will notice the presence of obstacles in the daytime, but at night It also comes out if you don't notice. Therefore, in order to inform the driver of the presence of obstacles in areas with poor visibility when parking at night from the past image of the parking place taken in the daytime, an interpolated image interpolated with the obstacles is displayed on the monitor during back parking. To do.

DESCRIPTION OF SYMBOLS 1 Camera module 2 Lens 3 Image pick-up element 4 Drive means 5 Position detection apparatus 10 Processing apparatus 11 Calculation means 12 Recording processing means 13 Invisible area judgment means 14 Image synthesis means 17 Storage means 21 Input device 22 Display apparatus 30 Management server 31 Communication apparatus 32 Management processing device 33 Database 35, 36 Other vehicle 40 Network

Claims (6)

Imaging means mounted on the vehicle;
Position detecting means for detecting the position of the vehicle;
Storage means for storing the image obtained by the imaging means and the position information detected by the position detection means in association with each other;
An invisible area determining means for determining whether or not an invisible area exists in the image captured by the imaging means;
When the invisible area determining means determines that an invisible area exists in the image, a past image that matches the position of the vehicle detected by the position detecting means is extracted from the storage means, and the past image is captured. An in-vehicle imaging device comprising: an image synthesis unit that synthesizes the current image captured by the unit. The invisible region determining means includes
An illuminance sensor attached to the vehicle;
The in-vehicle device according to claim 1, further comprising: a determination unit that determines that the invisible region exists when an illuminance around the vehicle detected by the illuminance sensor is lower than a preset threshold value. Imaging device. The invisible region determining means includes
It includes a judging unit that extracts a preset interpolation target from the past image and judges whether or not the invisible region exists based on a difference between the current image and the interpolation target of the past image. The in-vehicle imaging device according to claim 1.   The in-vehicle imaging according to any one of claims 1 to 3, wherein the past image combined with the current image is at least one of a sign, a white line, a guardrail, and a fixed installation in a parking lot. apparatus. A communication unit that communicates with a database in which the past image and the position information are associated and stored;
The in-vehicle imaging device according to claim 1, wherein the past image corresponding to the position of the vehicle is acquired from the database together with the position information by the communication unit.   The in-vehicle imaging device according to claim 1, further comprising display means for displaying the interpolated image synthesized by the image synthesizing means.

Images