JP5728929B2 - Vehicle periphery monitoring device - Google Patents

Description

  The present invention relates to a vehicle periphery monitoring device.

  2. Description of the Related Art Conventionally, a technique is known in which an image pickup device equipped with a CCD type solid-state image pickup device installed in a vehicle picks up an image around the vehicle and displays the obtained picked-up image on a display device in a vehicle interior (Patent Document). 1).

JP 2010-89716 A

  However, in the case of using an imaging device including a CCD type solid-state imaging device as in the prior art, smear occurs when a high-luminance subject exists in the captured image, and the captured image has a smear. In some cases, whitening occurs due to the occurrence of smear, making it difficult to see the display image.

  The problem to be solved by the present invention is to reduce the influence of smear on a display image even when smear occurs due to the influence of a subject with high brightness when displaying an image around the host vehicle.

The present invention includes a plurality of imaging means for imaging the surroundings of the vehicle, based on a plurality of images captured by the plurality of image pickup means, as well as viewpoint conversion a plurality of captured images to an image looked down from the vehicle upper side Image conversion means for connecting and generating a combined overhead image; display means for simultaneously displaying an image of the surroundings of the host vehicle imaged by one of the plurality of imaging means and the synthesized overhead image; In the vehicle periphery monitoring apparatus, a CCD type solid-state image pickup device in which the smear generation direction is suppressed to one direction is used as the plurality of image pickup means , and the image is picked up by the CCD type solid-state image pickup device. , generating direction of the smear, with respect to the vertical downward direction of the vehicle, so that more than 90 ° angle, by arranging fixed the CCD type solid-state imaging device, the Division To resolve.

  According to the present invention, since the CCD type solid-state imaging device is arranged so that the smear generation direction has an angle of 90 ° or more with respect to the downward direction of the host vehicle, the smear in the imaging screen is arranged. The generation area of the vehicle can be outside the range of the area used for monitoring the vehicle periphery, thereby reducing the influence of smear on the display image for vehicle periphery monitoring.

FIG. 1 is a configuration diagram of a vehicle periphery monitoring system according to the present embodiment. FIG. 2 is a diagram illustrating an example of an installation position and an imaging area of the in-vehicle camera according to the present embodiment. FIG. 3 is a diagram illustrating an example of the synthesized overhead image. 4A shows the position of the in-vehicle camera, the position of the virtual viewpoint, and the imaging area, FIG. 4B shows the captured image before the viewpoint conversion, and FIG. 4C shows the viewpoint converted. It is a figure which shows the synthesized bird's-eye view image. FIG. 5 is a diagram illustrating an example of the boundary image. FIG. 6 is a diagram illustrating a display example of a monitoring image. FIG. 7 is a diagram showing the arrangement of the light receiving elements constituting the CCD type solid-state imaging device used in the present embodiment. FIG. 8 is a diagram for explaining a smear phenomenon of a CCD type solid-state imaging device. FIG. 9 is a diagram for explaining the smear phenomenon in the CCD type solid-state imaging device in which the first smear phenomenon is prevented, which is used in the present embodiment. FIG. 10 illustrates the smear phenomenon in the case where the CCD solid-state image pickup device in which the first smear phenomenon is prevented and which is used in the present embodiment is arranged so that the smear generation direction is on the upper side in the vehicle vertical direction. FIG. FIG. 11 is a diagram illustrating a display state of a monitoring image when a smear phenomenon occurs when the in-vehicle camera according to the present embodiment is used. FIG. 12 is a diagram illustrating a display state of a monitoring image when a smear phenomenon occurs when the in-vehicle camera according to the present embodiment is not used.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block configuration diagram of a vehicle periphery monitoring system 100 according to the present embodiment. As shown in FIG. 1, the vehicle periphery monitoring system 100 according to the present embodiment includes a control device 10, four in-vehicle cameras 1 a to 1 d fixed outside the host vehicle, an ignition switch 2, an imaging switch 3, and a reverse position switch. 4, an image changeover switch 5 and a display 6 are provided. Each of these devices is connected by a CAN (Controller Area Network) or other in-vehicle LAN, and can exchange information with each other.

  As shown in FIG. 1, the control device 10 includes a ROM (Read Only Memory) 12 in which various programs are stored, a CPU (Central Processing Unit) 11 as an operation circuit that executes the programs stored in the ROM 12, And a RAM (Random Access Memory) 13 that functions as an accessible storage device.

  And the control apparatus 10 is provided with the captured image acquisition function, the image conversion function, and the display control function in order to produce | generate the monitoring image for monitoring the vehicle periphery, and to display the produced | generated monitoring image on the display 6. FIG. ing. The control device 10 can execute each function by cooperation of software for realizing the above functions and the hardware described above.

  Below, each function which the control apparatus 10 of the image display apparatus 100 implement | achieves is each demonstrated.

  First, the captured image acquisition function of the control device 10 will be described. When an input operation is performed on the ignition switch 2, the control device 10 according to the present embodiment causes the vehicle-mounted cameras 1a to 1d to image the surroundings of the vehicle at a predetermined period. When the user turns on the imaging switch 3, or when the reverse position switch 4 outputs an on signal (reverse is selected as the shift position), the screen of the display 3 divided into two as shown in FIG. A monitoring image S is displayed on the left side of M, and a captured image P of the in-vehicle camera 1 is displayed on the right side. Details will be described later.

  FIG. 2 is a diagram illustrating an arrangement example when the in-vehicle cameras 1a to 1d are attached to the vehicle. The in-vehicle cameras 1a to 1d are installed at different positions outside the vehicle, and respectively capture images in four directions around the vehicle. For example, as shown in FIG. 2, the in-vehicle camera 1a installed at a predetermined position in front of the vehicle such as near the front grill picks up an image of a predetermined imaging area SP1 in front of the vehicle (hereinafter referred to as a front view image). The in-vehicle camera 1b installed at a predetermined position on the left side of the vehicle such as a left side mirror captures an image of a predetermined imaging area SP2 on the left side of the vehicle (hereinafter referred to as a left side view image). The in-vehicle camera 1c installed at a predetermined position on the rear side of the vehicle such as a roof spoiler picks up an image of a predetermined imaging area SP3 on the rear side of the vehicle (hereinafter referred to as a rear view image). The in-vehicle camera 1d installed at a predetermined position on the right side of the vehicle such as a right side mirror captures an image of the predetermined imaging area SP4 on the right side of the vehicle (hereinafter referred to as a right side view image). These four in-vehicle cameras 1a to 1d send captured images to the image display device 100 at a predetermined cycle. The control apparatus 10 acquires captured images from the in-vehicle cameras 1a to 1d. The specific configuration of the in-vehicle cameras 1a to 1d will be described later.

  Next, the image conversion function of the control device 10 will be described. Based on the acquired captured images of the in-vehicle cameras 1a to 1d, the control device 10 of the present embodiment converts the virtual viewpoint above the host vehicle into an overhead view image looking down at the host vehicle and the surroundings of the host vehicle, and connects them. In addition, one composite overhead image is generated. Specifically, the control device 10 refers to the conversion table indicating the correspondence between the pixel address of each captured image and the address in the synthesized overhead image for each captured image acquired by the in-vehicle cameras 1a to 1d at different positions. To convert it to the coordinates of the synthesized overhead image. And the control apparatus 10 connects each captured image by which the coordinate conversion was carried out, and produces | generates one synthetic | combination overhead image which shows the mode of the own vehicle and the surroundings of the own vehicle.

  FIG. 3 is a diagram illustrating an example of the composite overhead image K. 3 includes a bird's-eye view image A1 obtained by performing viewpoint conversion on the front view image, the left side view image, the rear view image, and the right side view image captured by the vehicle-mounted cameras 1a to 1d illustrated in FIG. A2, A3 and A4 are connected. Further, the hatched portion shown in the synthesized overhead image K shown in FIG. 3 is a portion where the host vehicle is present. By superimposing an image of the vehicle viewed from the upper virtual viewpoint on this portion, it is possible to generate a composite overhead image K as if looking down the host vehicle and the surroundings of the host vehicle from the upper virtual viewpoint.

  In the present embodiment, as a specific method for generating the composite overhead image K, for example, a captured image obtained by imaging the area SP with the installation position of the in-vehicle camera 1c in FIG. 4A as a viewpoint (FIG. 4B). )) Is converted into a bird's-eye view image (FIG. 4 (c)) captured at the virtual viewpoint Q in the sky shown in FIG. 4 (a). This overhead image is an image of the area SP looking down from the virtual viewpoint Q directly above the center of the vehicle and the surroundings of the vehicle. Incidentally, the correspondence relationship between the coordinate values of the captured images before and after conversion (FIG. 4B) and the coordinate values of the overhead image (FIG. 4C) is uniquely converted by the lens characteristics and the mounting angle of the in-vehicle camera 1c. 131 can be defined. Further, in this method, when generating the overhead view image shown in FIG. 4C, a part of the captured image captured by the in-vehicle camera 1c shown in FIG. 4B, more specifically, The overhead image shown in FIG. 4C is generated using only a part of the lower part of the captured image.

  Next, the display control function of the control device 10 will be described. The control device 10 according to the present embodiment superimposes the boundary region image L indicating the boundary regions R of the overhead images A1, A2, A3, and A4 constituting the above-described combined overhead image K on the synthesized overhead image K. And the generated monitoring image S is displayed in a predetermined manner via the display 6.

  FIG. 5 is a diagram illustrating an example of the boundary region image L superimposed on the synthesized overhead image K. As illustrated in FIG. The boundary area image L includes each boundary area R of a plurality of captured images constituting the composite overhead image K. The control device 10 preliminarily displays one or more boundary region images L according to the positions of the in-vehicle cameras 1a to 1d, the imaging ranges of the in-vehicle cameras 1a to 1d, the position of the virtual viewpoint, and the display range of the combined overhead image K. I remember it. The reason why a plurality of boundary area images L are stored is that, for example, when the vehicle is in contact with a wall or the like and the mounting position of the in-vehicle camera 1 is deviated from the time of shipment, the boundary area image in the monitoring image This is because, in order to prevent the L position from being shifted and displayed, the boundary region image L is changed so that the so-called calibration for preventing the shift can be executed.

  A vehicle image V shown in FIG. 5 is a CG (computer graphics) image that is a plan view of the vehicle looking down from a virtual viewpoint in order to make the vehicle occupant recognize the position of the host vehicle in the monitoring image. The vehicle image V may be stored as a part of the boundary region image L, or may be stored as image data separate from the boundary region image L.

  Since the control device 10 of the present embodiment performs coordinate conversion on captured images captured at different positions and connects them to generate one composite overhead image K, the boundary between regions combined from different captured images ( In the bird's-eye view images A1, A2, A3, and A4 shown in FIG. 3, there is a problem in that the degree of image distortion increases and the image continuity is impaired. In particular, when a three-dimensional object on the road is reflected at the joint between the overhead images A1, A2, A3, and A4 constituting each composite overhead image K, it is difficult to recognize the three-dimensional object due to the discontinuity of the image.

  In order to solve this point, the control device 10 of the present embodiment superimposes the boundary region image L indicating the boundary region R of the overhead images A1, A2, A3, and A4 synthesized from the captured images on the synthesized overhead image K. And displayed as a monitoring image S. As described above, the boundary region image L is shown superimposed on the composite overhead image K, so that the composite overhead image K is composed of a plurality of captured images and the image is not accurate near the boundary region R. Can be recognized by the passenger.

  And the control apparatus 10 of this embodiment produces | generates the monitoring image S by superimposing the boundary region image L which shows such a boundary region R, and the vehicle image V on the synthetic | combination overhead image K, and the produced | generated monitoring image S is displayed on the in-vehicle display 6.

  FIG. 6 shows a display example of the monitoring image S of the present embodiment. In the present embodiment, as shown in FIG. 6, the monitoring image S is displayed on the left side of the screen M of the display 6 divided into two, and the captured image P of the in-vehicle camera 1a or 1c is displayed on the right side. In FIG. 6, an example in which the captured image of the in-vehicle camera 1 a is displayed as the captured image P is shown.

  The captured image P shown in FIG. 6 is an image captured by either the in-vehicle camera 1a or 1c, that is, a front view image or a rear view image, and the reverse position switch 4 is on, that is, the shift position is reverse. In this case, the rear view image captured by the in-vehicle camera 1c installed at the rear of the vehicle is displayed, and when the reverse position switch 4 is OFF, the front view captured by the in-vehicle camera 1a installed at the front of the vehicle. An image is displayed. The display layout of the screen M is not particularly limited and can be set arbitrarily. Further, the image displayed on the left side of the screen M can be switched by turning on the image switch 5. For example, from the left side view image captured by the in-vehicle camera 1b, a captured image near the front wheel on the passenger seat side (left side of the vehicle in left-hand traffic) that becomes the blind spot of the driver is cut out (referred to as a side monitor image). The side monitor image and the monitoring image S can be switched by turning on the image switching switch 5. In FIG. 6, when the image displayed in the left display area is switched from the route guidance image or the like, the monitoring image S is first displayed. In this case, when the image changeover switch 5 operated by the user is turned on, the image is switched to the side monitor image, and when the image changeover switch 5 is turned on again, the image is switched to the monitoring image S.

  Next, the in-vehicle cameras 1a to 1d in the present embodiment will be described. In the present embodiment, as the in-vehicle cameras 1a to 1d, those having a CCD type solid-state imaging device are used. FIG. 7 is a diagram showing the arrangement of the light receiving elements 20 constituting the CCD type solid-state imaging element used in the present embodiment. As shown in FIG. 7, the CCD type solid-state imaging device used in the present embodiment includes a plurality of light receiving elements 20 arranged in a matrix, and the amount of light incident on each light receiving element 20 is set as a charge amount. It is an element capable of obtaining a captured image by performing photoelectric conversion to be converted and reading out an electric signal obtained thereby. In FIG. 7, the configuration in which the number of light receiving elements 20 is 18 × 14 (i = 0 to 17, j = 0 to 13) is illustrated, but the number of light receiving elements 20 is not particularly limited.

  Next, a smear phenomenon that is a phenomenon peculiar to a CCD type solid-state imaging device will be described. FIG. 8 is a diagram for explaining a smear phenomenon of a CCD type solid-state imaging device. As shown in FIG. 8, in a CCD type solid-state imaging device, for example, the light receiving element 20a (i = 15, j = 4) receives light from a high-luminance subject, and as a result, can be received by the light receiving element. When the light intensity exceeds the light intensity, the signal intensity of the light receiving element 20a is saturated, the signal intensity at the light receiving element 20a becomes the maximum value, and is detected as white on the obtained imaging screen. It becomes. In addition to this, the light receiving elements located in the same column as the light receiving elements 20a due to the influence of the light receiving elements 20a exceeding the intensity of light that can be received in this way, specifically, (i = 15, j As for the light receiving elements of = 0 to 3, 5 to 13), the signal intensity becomes the maximum value, and is similarly detected as white on the obtained imaging screen. Therefore, as a result, a streaky whiteout portion is generated at a position corresponding to the light receiving element (i = 15, j = 0 to 13) on the obtained imaging screen. This is the smear phenomenon of a CCD type solid-state imaging device. In FIG. 8, the light receiving element 20a (i = 15, j = 4), that is, the light receiving element that receives light from a high-luminance subject is shown in black, and white spots occur due to smearing. The light receiving element is shown in gray.

Such a smear phenomenon can be divided into those that occur mainly due to the following two causes.
That is, first, when the signal intensity of the light receiving element 20a is saturated, the light received by photoelectric conversion by the light receiving element 20a overflows in the light receiving elements located in the same column, thereby receiving light received in the same column. Also in the element, a smear phenomenon caused by saturation of the signal intensity (hereinafter referred to as a first smear phenomenon).
Second, in order to convert the electric charges accumulated in the light receiving elements located in the same column into electric signals, the electric charges photoelectrically converted by the light receiving elements 20a are added when sequentially transferred in the column direction. As a result, even in the light receiving elements located in the same column, the smear phenomenon caused by the saturation of the signal intensity (hereinafter referred to as a second smear phenomenon).

  Of these first and second smear phenomena, the first smear phenomenon (that is, the smear phenomenon caused by the overflow of the charge photoelectrically converted by the light receiving element 20a) is, for example, the light receiving element 20a. This can be prevented by adopting a configuration in which the exposure can be completed before the photoelectrically converted charges overflow. In this case, as shown in FIG. 9, the smear phenomenon does not occur for the light receiving elements (i = 15, j = 0 to 3), and in these light receiving elements, on the imaging screen, It is possible to prevent whiteout from occurring. In the present embodiment, a CCD solid-state imaging device constituting the in-vehicle cameras 1a to 1d is used that includes a CCD solid-state imaging device in which such a first smear phenomenon is prevented. That is, in this embodiment, as a CCD solid-state imaging device, the first smear phenomenon is prevented, and when the signal intensity of the light receiving element 20a of (i = 15, j = 4) is saturated, the same column is used. In the light receiving elements (i = 15, j = 0 to 3) that are located and located below the light receiving elements 20a, the smear phenomenon does not occur, and the light receiving elements 20a are located in the same row. Only the light receiving element (i = 15, j = 5 to 13) located on the lower side, in which a smear phenomenon occurs, is used. As the CCD type solid-state imaging device in which such a first smear phenomenon is prevented, a device capable of imaging control in a short time, such as a high-speed CCD, can be used. Not.

  In the CCD solid-state imaging device in which the first smear phenomenon is prevented, the transfer direction when the charges accumulated in the light receiving elements located in the same column are sequentially transferred in the column direction is usually Therefore, when such a CCD solid-state imaging device is used in the positive direction, the smear phenomenon occurs in the lower direction of the captured image as shown in FIG. It will be headed.

  On the other hand, in the CCD type solid-state imaging device in which the first smear phenomenon is prevented, the direction in which the smear phenomenon occurs is the downward direction of the own vehicle, that is, the own vehicle is in a horizontal position. In some cases, the CCD type solid-state imaging device is arranged so as to be in a direction opposite to the direction from the host vehicle toward the ground. Specifically, the CCD solid-state imaging device in which the first smear phenomenon is prevented is arranged in the upside down direction, that is, rotated by 180 °. FIG. 10 shows an arrangement example of a CCD type solid-state imaging device in which the first smear phenomenon is prevented in this embodiment. In FIG. 10, compared with the case of FIG. 9, the state is rotated by 180 °, and in FIG. 10, a high brightness subject is imaged at the same position as in FIGS. This shows the manner of occurrence of smear phenomenon in the case where it is applied. In the case shown in FIG. 10, compared with the case of FIG. 9, the light receiving element 20 a that receives light from the high-luminance subject is (i = 2, j = The light receiving element (9) that causes the smear phenomenon is the light receiving element (i = 2, j = 10 to 13).

  The in-vehicle cameras 1a to 1d used in the present embodiment and the CCD type solid-state imaging device constituting the same are configured as described above.

  As described above, in the present embodiment, as the in-vehicle cameras 1a to 1d, the first smear phenomenon is prevented and a CCD type solid-state image pickup device is used. That is, they are arranged so that the direction in which the smear phenomenon occurs after turning 180 ° is the direction of improvement in the up and down direction of the host vehicle. Therefore, for example, in the screen M of the display 6 shown in FIG. 6, as shown in FIG. 11, in the captured image (front view image) of the in-vehicle camera 1a displayed on the right side of the screen M of the display 6, smear is present. Even if it occurs, the smear generation direction can be the upper direction of the captured image, and the smear generation area can be outside the use area range in the overhead image. As a result, in the present embodiment, in the monitoring image S displayed on the left side of the screen M of the display 6, it is possible to effectively prevent the influence of smear generated in the captured image of the in-vehicle camera 1 a. can do. In FIG. 11, for easy understanding, the captured image of the in-vehicle camera 1 a displayed on the right side of the screen M is shown together with the arrangement pattern of the light receiving elements and the use area in the overhead image. However, these are not displayed in the actual captured image (the same applies to FIG. 12 described later).

  On the other hand, as shown in FIG. 12, although the CCD solid-state imaging device in which the first smear phenomenon is prevented is used as the CCD solid-state imaging device constituting the in-vehicle cameras 1a to 1d, When such a CCD type solid-state imaging device is used in the forward direction and the direction in which the smear phenomenon occurs is arranged in the direction of upward and downward improvement of the host vehicle, the smear generation region is an overhead image. It will be within the range of the use area. As a result, in the monitoring image S displayed on the left side of the screen M of the display 6, there is an effect of smear generated in the captured image of the in-vehicle camera 1a. Although not specifically shown, CCD type solid-state image sensors other than the CCD type solid-state image sensor in which the first smear phenomenon is prevented are used as CCD type solid-state image sensors constituting the in-vehicle cameras 1a to 1d. When used, the same phenomenon occurs. On the other hand, according to this embodiment, the first smear phenomenon is prevented and a CCD solid-state image sensor is used, and the direction in which the smear phenomenon occurs in such a CCD solid-state image sensor is Such a problem can be effectively solved by arranging the vehicle so as to be directed upward and downward.

  Further, according to the present embodiment, since the smear generation direction can be the upper direction of the captured image, for example, even if a smear phenomenon occurs in the rear view image captured by the in-vehicle camera 1c, Since it occurs toward the upper side, it is possible to prevent smear from occurring in a relatively lower part of the captured image, which is necessary when white line recognition or rear vehicle recognition is performed. Thus, according to the present embodiment, it is possible to reduce the influence of smear when white line recognition or rear vehicle recognition is performed using the rear view image captured by the in-vehicle camera 1c.

  Furthermore, according to the present embodiment, as the in-vehicle cameras 1a to 1d, the first smear phenomenon is prevented and a CCD type solid-state image pickup device is used. In other words, since it is used after being rotated by 180 °, it is only necessary to perform processing such as upside down when converting an image picked up by a CCD type solid-state image pickup device. Can be simplified.

  In the above-described embodiment, the in-vehicle cameras 1a to 1d correspond to the imaging unit of the present invention, the display 6 corresponds to the display unit of the present invention, and the image conversion function of the control device 10 corresponds to the image conversion unit of the present invention. .

  As mentioned above, although embodiment of this invention was described, these embodiment was described in order to make an understanding of this invention easy, and was not described in order to limit this invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  For example, in the above-described embodiment, the first smear phenomenon is prevented and the CCD type solid-state imaging device is used rotated by 180 °, and the direction in which the smear phenomenon occurs is the direction in which the vehicle is improved upward and downward. However, the configuration is not particularly limited, and at least the vehicle is rotated by 90 ° or more with respect to the downward direction in the vertical direction, and the direction in which the smear phenomenon occurs is determined by the vertical direction of the vehicle. What is necessary is just to set it as the structure which has an angle of 90 degrees or more with respect to a direction downward direction. In this case, it may be rotated 90 ° or more on the left side, or may be rotated 90 ° or more on the right side.

DESCRIPTION OF SYMBOLS 100 ... Vehicle periphery monitoring system 1a-1d ... Car-mounted camera 10 ... Control apparatus 2 ... Ignition switch 3 ... Imaging switch 4 ... Reverse position switch 5 ... Image change-over switch 6 ... Display P ... Captured image S ... Monitoring image R ... Boundary area V ... own vehicle

Claims (2)

A plurality of imaging means for imaging the surroundings of the host vehicle;
Based on the plurality of captured images captured by the plurality of image capturing means, the plurality of captured images are looked down from above the own vehicle using a part of the lower part of the captured images captured by the respective image capturing means. Image conversion means for converting a viewpoint to an image and connecting the images to generate a composite overhead image;
In a vehicle periphery monitoring device having display means for simultaneously displaying an image around the host vehicle imaged by one of the plurality of imaging means and the synthesized overhead image ,
The plurality of imaging means are CCD type solid-state imaging devices that receive incident light by a plurality of light receiving devices arranged in a matrix,
In the CCD solid-state imaging device, when the intensity of light incident on the light receiving element is saturated, smear is generated only in a predetermined direction in the column direction from the light receiving element in which the intensity of the incident light is saturated. However, smear does not occur in the other direction and the row direction of the column direction,
The CCD type solid-state imaging device has an angle of 90 ° or more with respect to a direction in which the smear is generated in an up-down direction of the host vehicle on an imaging screen imaged by the CCD-type solid-state imaging device. A vehicle periphery monitoring device, wherein the vehicle periphery monitoring device is arranged so as to be fixed . In the vehicle periphery monitoring device according to claim 1,
The CCD type solid-state imaging device is fixed and arranged on the imaging screen imaged by the CCD type solid-state imaging device so that the smear generation direction is the upward and downward improvement direction of the host vehicle. A vehicle periphery monitoring device characterized by comprising: