JP2010183281A - In-vehicle photographing device, and safety measure method for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an in-vehicle photographing device which achieves a photographing device (digital camera) for simultaneously photographing images of a central perspective and a peripheral perspective by applying a feature that recognition of a dynamic object in the peripheral perspective of human eyes is excellent, and effectively uses the image obtained by simultaneously photographing the central perspective and the peripheral perspective as application of the photographing device. <P>SOLUTION: The photographing device divides a photographing area into a central part and a peripheral part, and sets the pixel density of photographed image data of the central part high, and the pixel density of photographed image data of the peripheral part low. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle-mounted imaging device that is mounted on a vehicle and is used for assisting driver's visual recognition, monitoring of a person or an obstacle, lane recognition, and the like, and a safety measure method for a vehicle equipped with the vehicle-mounted imaging device.

  In recent years, in the field of vehicles, a driver's visual assistance system that uses a camera, an obstacle monitoring system that recognizes people and obstacles and warns of danger, and a lane keep that recognizes lanes and maintains the lane of the vehicle. System etc.) have been proposed and partially put into practical use. For example, Patent Document 1 discloses a road position recognition technique using a vehicle-mounted camera for a lane keeping system or the like.

JP 2005-056128 A

  As an imaging means used in the above-mentioned patent documents and the like, an array of pixels of an imaging device (for example, a charge-coupled device (CCD)) used in a normal digital camera (imaging unit) is photographed 1 It is even at all positions on the screen.

On the other hand, human eye image processing has the following characteristics.
The human eye has two fields of view, the center and the periphery.
The central visual field refers to an area of about 20 degrees around the line of sight.
The peripheral visual field is an area of 130 degrees up and down and 180 degrees from the central visual field.
In this peripheral vision, it is visible but the shape of the object is vague, and the color cannot be accurately recognized. However, a force superior to the central visual field can be exerted on a moving object or an object (dynamic object) that changes with time such as blinking light.

An object of the present invention is to apply a feature that the recognition of a dynamic object in the peripheral visual field of the human eye is excellent to the photographing apparatus, and to be capable of photographing the central visual field and the peripheral visual field simultaneously (digital camera). ) And an in-vehicle imaging device that effectively uses images obtained by simultaneously capturing images of a central visual field and a peripheral visual field as an application of the imaging device.
Another object of the present invention is to realize a safety measure method for a vehicle equipped with the in-vehicle imaging device.

In order to achieve the above object, the photographing apparatus of the present application separates the photographing area into a central part and a peripheral part, and increases the pixel density of the photographed image data in the central part, and increases the pixel density of the photographed image data in the peripheral part. It is characterized by being set low.
Further, the image sensor of the central image capturing unit and the image sensor of the peripheral image capturing unit have different configurations.
In addition, the central image capturing unit and the peripheral image capturing unit are configured with a common optical system and a common image sensor, and the pixel density of the image sensor is different between the central unit and the peripheral unit. Features.

  In order to achieve the above object, the vehicle photographing apparatus of the present application separates the photographing area into a central part and a peripheral part, and the pixel density of the photographed image data in the central part is high, and the pixels of the photographed image data in the peripheral part An imaging device having a low density is mounted on a vehicle, and the imaging device further includes a pixel data processing unit that individually processes pixels of captured image data in the central portion and the peripheral portion, and the pixel data processing unit Then, at least the data processing of the dynamic object included in the image data of the peripheral image sensor is mainly executed.

The dynamic object is a dynamic object that enters a peripheral imaging area, and is extracted by a difference between a plurality of peripheral image data that are continuously captured.
Further, the determination of the presence or absence of a dynamic object entering the peripheral image capturing area may be performed by determining whether the dynamic object extracted by a difference between a plurality of peripheral image data continuously captured within a predetermined time or It is determined by the moving speed.
In addition, when the dynamic object is detected, an alarm or a protection operation for the dynamic object is executed by the vehicle.
In addition, the dynamic object is a person or a bicycle entering in front of the vehicle in the traveling direction.

  In order to achieve the above object, the vehicle safety measures method of the present application separates the shooting area into a central part and a peripheral part, increases the pixel density of the centrally-captured image data, A safety measure method for a vehicle equipped with a photographing device including a pixel data processing unit that sets a pixel density low and individually processes pixels of photographed image data in a central part and a peripheral part. A vehicle safety measure for the dynamic object is executed when the included dynamic object is detected.

According to the imaging device, the in-vehicle imaging device, and the vehicle safety countermeasure method of the present invention, the feature that the recognition of the dynamic object in the peripheral visual field of the human eye is excellent is applied to the imaging device, An imaging device (digital camera) capable of simultaneously capturing images of the peripheral vision can be realized.
In addition, as an application of the imaging apparatus, an in-vehicle imaging apparatus that effectively uses images obtained by simultaneously capturing images of the central visual field and the peripheral visual field can be realized.
In addition, it is possible to realize a safety measure method for a vehicle equipped with the in-vehicle imaging device.

It is a figure explaining the relationship of the visual field of the high resolution telephoto camera for center visual field photography, and the low-resolution wide-angle camera for peripheral visual field photography. It is a schematic diagram of a charge coupled device CCD into which image data photographed by the photographing apparatus of the present invention is input. It is a figure which shows the relationship between CCD center visual field and peripheral visual field of the imaging device mounted in the vehicle. It is a flowchart explaining operation | movement at the time of detecting that the person and the bicycle approached in front of the vehicle with the imaging device for vehicles of this invention. It is a figure which shows the 1st hardware structural example of the imaging device of this invention. It is a figure which shows the 2nd hardware structural example of the imaging device of this invention.

  In the present invention, in the first embodiment of the photographing apparatus for solving the above-described problems, a high-resolution telephoto camera for central field photography and a low-resolution wide-angle camera for peripheral vision photography are used.

The relationship between the field of view of a high-resolution telephoto camera for central field imaging and a low-resolution wide-angle camera for peripheral field imaging will be described with reference to FIG.
In FIG. 1, reference numeral 1 denotes a photographing apparatus, and a high-resolution telephoto camera 1a for central field photographing and a low-resolution wide-angle camera 1b for photographing peripheral vision are integrally formed in the photographing apparatus.
A is a central visual field area photographed by the high-resolution telephoto camera for central visual field photography, and B is a peripheral visual field area photographed by a low-resolution wide-angle camera for peripheral visual field photography.

  The spread of the central visual field area A in FIG. 1 is an area of about 20 degrees centered in the photographing direction (line of sight), similar to the human eye, and the peripheral visual field area B is 130 degrees above and below the central visual field and 180 degrees left and right However, this value is not necessarily required, and the range of the central visual field area and the peripheral visual field area can be set according to the application.

  In FIG. 1, the central-view camera 1a is a single camera. However, two high-resolution telephoto cameras for central-view imaging are arranged at a predetermined distance so that the focus of both is set on the subject. In addition, the color, shape position (distance) relationship and movement of the object to be imaged may be detected as with two human eyes.

  FIG. 2 is a schematic diagram of an image sensor (for example, a charge-coupled device (CCD)) to which image data captured by the image capturing apparatus 1 of FIG. 1 is input. A photographed image of a high-resolution telephoto camera 1a for photographing is input, and a photographed image of a low-resolution wide-angle camera 1b for photographing peripheral vision is input to B-1.

In the following description, an example in which a CCD image sensor (hereinafter referred to as CCD) is used as the image sensor is described. However, a CMOS image sensor may be used as the image sensor.
In FIG. 2, the pixel density of A-1 is high and suitable for high-resolution imaging, and the pixel density of B-1 is low and suitable for low-resolution imaging.
In the above description, FIG. 2 is described as a CCD, but FIG. 2 may be understood as a display screen composed of a liquid crystal or the like for displaying image data captured by the CCD.
In FIG. 2, A-1 and B-1 are each shown as a rectangle, but the shape of the CCD is not necessarily a rectangle, and it is needless to say that an arbitrary shape such as a concentric circle can be selected.

  As shown in FIG. 2, since the pixel density of the CCD for the image data of the peripheral visual field is low, the pixel density is small even in the peripheral visual field in a far wider range than the central visual field. Data processing that includes a dynamic object that enters the peripheral visual field area, especially the difference between image data of multiple peripheral visual fields that are captured continuously. it can.

As an example of processing for determining whether or not there is a dynamic object entering the peripheral visual field area, a dynamic object is extracted (recognized) based on a difference between a plurality of peripheral visual field image data that are continuously captured.
Furthermore, the calculation of the moving distance or moving speed within a predetermined time within the peripheral visual field of the extracted dynamic object is executed in a short time.

As an example of extracting a dynamic object based on a difference in image data, the following is known as described in JP-A-9-252467.
In order to detect a moving object based on the digitally encoded data captured by the camera, an image signal from the surveillance television camera is converted into digital image compression encoded data by an encoding unit. The encoding unit encodes pixel information of the difference between the image on the previous screen and the subsequent screen, and information on a motion vector indicating the magnitude and direction of the motion on the two screens in units of a plurality of blocks. When decoding the encoded data, a motion vector for a moving object is extracted by a moving object detection / determination unit. In this determination unit, a motion vector is extracted, an object block is extracted based on the motion index, its statistic is obtained, and the target moving body is determined by comparison with a reference value.

  Extraction (recognition) of a dynamic object based on a difference between a plurality of peripheral visual field image data continuously photographed in the present invention is not limited to the above recognition method, and a known recognition method at the time of filing the present application is used. Needless to say, this is applicable.

A hardware configuration of the photographing apparatus of the present invention will be described with reference to FIGS.
FIG. 5 is a diagram showing a first hardware configuration example of the photographing apparatus of the present invention.
The photographing apparatus 1 is provided with an optical system and a CCD for photographing a central visual field individually, and includes a central visual field optical system 1-1, a central visual field CCD 1-1a, a peripheral visual field optical system 1-2, respectively. This is a peripheral visual field CCD1-2a.
Reference numeral 1-3 denotes a processing device (CPU), which individually processes image data of individual CCDs photographed by individual optical systems, and processes the processed data via the interface 1-4. Signals are exchanged with the control device (CPU) of the mounted vehicle.
The center visual field CCD 1-1a has a high pixel density and is suitable for high resolution photography, and the peripheral visual field CCD 1-2a has a low pixel density and is suitable for low resolution photography.

FIG. 6 is a diagram illustrating a second hardware configuration example of the photographing apparatus of the present invention.
The photographing apparatus 1 includes a common optical system for photographing the central and peripheral visual fields and a common CCD, and includes a common optical system 1-5 and a common CCD 1-5a.
1-3 is a processing unit (CPU), which processes image data of a common CCD imaged by a common optical system individually for each of the central visual field and the peripheral visual field, and also through the interface 1-4. Signals are exchanged with the control device (CPU) of the vehicle on which the imaging device is mounted.
Further, the pixel density of the central visual field A-1 of the common CCD 1-5a is set high, and the pixel density of the peripheral visual field B-1 is set low.

Next, an application example will be described in which the photographing apparatus shown in FIGS.
In FIG. 3, A-1 and B-1 indicate the central visual field area and peripheral visual field area of the same CCD (or display device) as in FIG.
The image of the central visual field area A-1 is an obstacle monitoring system that recognizes obstacles and warns danger by always shooting road lanes 3 and vehicles 2 traveling ahead, etc., and recognizes lanes. It is used to maintain the vehicle lane.

The photographed image of the peripheral visual field area B-1 unique to the present invention is a photograph of the object 4 in the peripheral visual field at a distance (for example, immediately before the vehicle) shorter than the distance between the vehicle traveling ahead and the own vehicle. There is a feature in doing.
In the present invention, instead of simply photographing an object in the peripheral vision area, a plurality of dynamic objects 4 entering immediately before the vehicle, such as a person or a bicycle entering the peripheral vision area, are continuously photographed. Extraction (recognition) is performed based on the difference between the peripheral image data.

The determination of the presence or absence of a dynamic object that enters the peripheral imaging area is performed by moving a dynamic object extracted within a predetermined time by a difference between a plurality of peripheral image data (for each screen) that are continuously captured. Judged by distance or moving speed.
Depending on the case, the moving direction or moving speed of the dynamic object extracted by the difference between a plurality of peripheral image data (for each screen) that are continuously captured may be added to the moving distance or moving speed within a predetermined time. Thus, it may be determined as a dynamic object moving in a predetermined direction.

  With the above configuration, when it is detected that a person or bicycle has entered just before the vehicle, an alarm is issued to the driver and the dynamic object, or an emergency such as a pedestrian protection airbag provided in the vehicle. The protective device can be activated.

Next, an operation when it is detected that a person or a bicycle has entered immediately before the vehicle will be described with reference to the flowchart of FIG.
The CCD image data of the captured image of the central visual field area A-1 and the captured image of the peripheral visual field area B-1 are individually processed by the processing device (CPU) 1-3. (Step S1)
Here, the processing result of the photographing data of the central visual field area is data obtained by constantly photographing a normal road lane or a vehicle traveling in front, so an obstacle monitoring system that recognizes an obstacle and warns of danger, It recognizes the lane and is used for maintaining the driving lane of the vehicle, but the details are omitted here.
In the photographing data analysis of the peripheral visual field area, a difference for each photographed screen is extracted, and it is determined whether or not there is a change (dynamic object) in the peripheral visual field area. (Step S2)
If the determination in step S2 is Yes, it is determined whether the detected dynamic object is moving at a predetermined speed or more or whether it has been moved more than a predetermined distance. (Step S3)
Here, whether the detected dynamic object is moving at a predetermined speed or more, or whether it has moved more than a predetermined distance is because, for example, the vehicle is traveling, This is to remove the influence of the scenery as it changes.
・ Determine whether the vehicle is traveling at a predetermined speed or higher. (Step S4)
Safe operation is not required when the vehicle is stopped or below a predetermined speed.
If the determination in step S4 is Yes, an alarm is issued to the driver and the dynamic object, or an emergency protection device such as a pedestrian protection airbag provided in the vehicle is activated. (Step S5)
If the determination in step S2, step S3, and step S4 is No, the process returns to step S1 and repeats the process.

1: Shooting device
1a: Central vision camera
1b: Peripheral vision camera
A: Central vision area
B: Peripheral vision area

Claims (9)

  An imaging apparatus, wherein the imaging area is divided into a central portion and a peripheral portion, and the pixel density of the captured image data in the central portion is set high and the pixel density of the captured image data in the peripheral portion is set low.   The imaging apparatus according to claim 1, wherein the imaging device of the central image capturing unit and the imaging device of the peripheral image capturing unit have different configurations.   The central image photographing unit and the peripheral image photographing unit are configured by a common optical system and a common image sensor, and the pixel density of the image sensor is different between the central part and the peripheral part. The photographing apparatus according to claim 1. The shooting area is separated into a central part and a peripheral part, and the imaging device in which the pixel density of the captured image data in the central part is set high and the pixel density of the captured image data in the peripheral part is set low is mounted on the vehicle.
The photographing apparatus further includes a pixel data processing unit that individually processes the pixels of the photographed image data in the central part and the peripheral part,
The in-vehicle imaging device characterized in that the pixel data processing unit mainly executes at least data processing of a dynamic object included in image data of a peripheral image sensor.   The dynamic object is a dynamic object that enters a peripheral shooting area, and is extracted by a difference between a plurality of peripheral image data that are continuously captured. In-vehicle imaging device.   The determination of the presence or absence of a dynamic object entering the peripheral shooting area is based on the movement distance or moving speed within a predetermined time of the dynamic object extracted by the difference between a plurality of peripheral image data captured continuously. 6. The in-vehicle photographing device according to claim 5, wherein   The in-vehicle use according to claim 6, wherein when the dynamic object is detected and the vehicle is traveling at a predetermined speed, an alarm or a protection operation for the dynamic object is executed by the vehicle. Shooting device.   The in-vehicle imaging device according to claim 7, wherein the dynamic object is a person or a bicycle entering in front of the vehicle in the traveling direction. The shooting area is separated into a central portion and a peripheral portion, the pixel density of the captured image data in the central portion is set high, the pixel density of the captured image data in the peripheral portion is set low, and the captured image data of the central portion and the peripheral portion is set. A vehicle safety measure method equipped with a photographing device equipped with a pixel data processing unit that individually processes pixels,
When a dynamic object included in the peripheral image data is detected and the vehicle is traveling at a predetermined speed or more, a vehicle safety measure for the dynamic object is executed. Safety measures method.

Images