JP2004328240A - Peripheral image display equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate a solid object existing in an image that an image pickup means photographs as a solid object. <P>SOLUTION: The rear part of a vehicle 1 is photographed by a camera 4 and is displayed on a monitor 3. Infrared projectors 5a and 5b irradiate a photographing area A with slit light L, and it is photographed by the camera 4. A controller 2 operates differential image data from a slit light irradiation image and a slit light non-irradiation image. If a pattern of the image of slit light differs from a pattern in a state where the solid object does not exist in the photographing area A when image data of slit light are obtained, that the solid object exists in the part is estimated and is displayed on the monitor 3. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a peripheral image display device that is mounted on a moving body such as a vehicle, captures the surroundings, and displays the images on a display unit.
[0002]
[Prior art]
For example, there is a device that can photograph the rear of a car with a camera and check the rear with a monitor provided in a driver's seat. This is excellent in usability because it is possible for a vehicle that is difficult to see behind or an unfamiliar driver to see by seeing the image of the back on the monitor.
[0003]
As such, for example, in addition to displaying the image taken by the camera as it is, as shown in the image on the left side of FIG. 9, an image viewed from directly above as shown in the image on the right side of FIG. What has been provided which is easy to grasp the position of the own vehicle by being converted into a bird's-eye view and displayed (for example, see Patent Documents 1 to 4).
[0004]
[Patent Document 1]
JP-A-2002-087160
[Patent Document 2]
JP 2002-262280 A
[Patent Document 3]
JP-A-2002-262281
[Patent Document 4]
JP-A-2002-283913
[Problems to be solved by the invention]
In the above-mentioned prior art, the image taken by the camera attached to the rear of the vehicle was converted into an image viewed from directly above based on the image taken as a bird's-eye view. On the rear screen, there are three-dimensional objects such as other vehicles and obstacles. Since these three-dimensional objects are pasted as a pattern on a plane when shown in the bird's-eye view, the distortion of the image as a bird's-eye view increases as shown in FIG. Sometimes misunderstood the positional relationship.
[0009]
The present invention has been made in view of the above circumstances, and an object of the present invention is to estimate a three-dimensional object as a three-dimensional object in an image captured by an imaging unit, and thereby to determine a user from the image. It is an object of the present invention to provide a peripheral image display device capable of performing display in which a three-dimensional object can be easily grasped.
[0010]
[Means for Solving the Problems]
According to the first aspect of the present invention, the imaging unit captures a slit light irradiation image when the projection area irradiates the slit light with the slit light and a slit light non-irradiation image when the irradiation area is not irradiated. Then, a difference between pixel components of both images is calculated to create difference image data. Since the difference data described above corresponds to the difference in the presence or absence of slit light irradiation, data of only the slit light can be obtained.
[0011]
When the image pattern of the slit light is different from the pattern in a state where no three-dimensional object exists in the photographing area, it can be estimated that a three-dimensional object exists in that part. The control means causes the display means to display the result of estimating the presence of the three-dimensional object in this manner. This makes it possible to simultaneously display the presence of a three-dimensional object with respect to the image in the imaging area captured by the imaging unit, so that even if the image of the area to be captured is distorted, the three-dimensional object Can be surely recognized.
[0012]
According to the second aspect of the present invention, in the above invention, the control unit converts the image of the photographing area photographed by the imaging unit into a bird's-eye view and displays it on the display unit together with the estimated three-dimensional object. Even if the captured image of the three-dimensional object located at is obtained as a small and distorted image, the three-dimensional object can be reliably recognized without being mistaken due to the distortion when converted to a bird's-eye view. become.
[0013]
According to the third aspect of the present invention, in each of the above inventions, the projection unit is configured to irradiate infrared slit light as slit light. It is possible to perform a three-dimensional object estimation process without feeling the effect.
[0014]
According to the fourth aspect of the present invention, in each of the above aspects, the control means sets the projection area of the slit light emitted by the light projection means when the three-dimensional object does not exist in the photographing area as a mask area, and sets difference image data. The presence of a three-dimensional object is determined by determining the presence or absence of slit light along the mask area, so the presence of a three-dimensional object can be quickly performed by performing simple image processing on pixel data near the mask area. Can be estimated.
[0015]
According to the invention of claim 5, in the invention of claim 4, when the slit light irradiation image and the slit light non-irradiation image become images at different positions due to the movement of the moving body, the control means: Since it is assumed that a three-dimensional object exists at a place where the difference value of the difference image data is large and a place where the difference value of the mask area is small, it is possible to easily and quickly determine the data of the difference value within the mask area. Will be able to do it.
[0016]
According to the sixth aspect of the present invention, since the three-dimensional object existing in the side photographing area can be estimated by providing the side image capturing means and the side light projecting means, in a moving object such as a vehicle, Sometimes, an obstacle or the like present on the side can be reliably recognized by visually recognizing the display means. This makes it possible to prevent accidents such as accidentally moving and colliding sideways.
[0017]
BEST MODE FOR CARRYING OUT THE INVENTION
(1st Embodiment)
Hereinafter, a first embodiment in which the present invention is applied to a back view monitor of an automobile will be described with reference to FIGS.
FIG. 1 schematically shows a state where the present apparatus is mounted on an automobile 1, and includes a controller 2, a monitor 3, a camera 4, and infrared projectors 5a and 5b. The monitor 3 as a display means uses a display device such as a liquid crystal and is provided on a dashboard of the automobile 1 or the like, and is installed so that a driver can see it.
[0018]
The camera 4 as an imaging means is capable of photographing light in a range from visible light to infrared light. The camera 4 is disposed at the rear of the automobile 1 and photographs a photographing area A behind. The infrared projectors 5a and 5b are provided, for example, in a tail lamp portion of the automobile 1, and project the slit light L radially into the photographing area A as described later.
[0019]
The control device 2 as a control means processes a photographed image and displays it on the monitor 2 as described later, and is disposed under a seat of the automobile 1 or the like. The control device 2 includes a control circuit 6, a light emission control circuit 7, a synchronization circuit 8, a memory 9, and the like, as schematically shown in FIG.
[0020]
The control circuit 6 mainly includes a CPU, a ROM, a RAM, and the like, and is configured to perform a display operation described later by a display control program. The light emitting control circuit 7 controls the light emitting operation of the infrared light projectors 5a and 5b, and emits the slit light L to the photographing area A. Thereby, as shown in FIG. 1, the slit light L is projected radially from each of the infrared projectors 5a and 5b.
[0021]
The synchronizing circuit 8 is for synchronizing and operating the light emitting control circuit 7 and the camera 4, and as described later, a slit for photographing with the camera 4 in a state where the slit light L is projected by the infrared light projectors 5 a and 5 b. In order to obtain a light irradiation image and a slit light non-irradiation image photographed by the camera 4 in a state where the slit light L is not projected, the timing of both is controlled.
[0022]
Next, the operation of the present embodiment will be described with reference to FIGS.
FIG. 3 shows a principle for estimating a three-dimensional object by this device, and is a diagram showing an assumption that an image is taken from the position of the camera 4. The infrared light projectors 5a and 5b irradiate the slit light L into the photographing area A as shown in FIG. 4B, so that when there is no three-dimensional object in the area, the slit light L causes the slit light L to reach the ground. A straight line is projected like a line S.
[0023]
On the other hand, when the three-dimensional object P exists so as to cross the slit light L, the projected image formed by the slit light L is bent into a shape along the three-dimensional object P as a line Sp. When this is photographed by the camera 4, a polygonal line Sp is photographed as shown in the figure.
[0024]
Therefore, by comparing the image of the line S that should be obtained when the three-dimensional object P does not exist with the line Sp of the image obtained by actually photographing, the three-dimensional object P exists in a non-overlapping part. Can be estimated.
[0025]
By the way, in this embodiment, the slit light L is projected as infrared rays, and the camera 4 is provided so as to be able to photograph wavelengths in the infrared region. This is to eliminate a sense of incongruity when projecting visible slit light, and there is no inevitable reason to use infrared light.
[0026]
Then, in order to obtain projection images irradiated by the infrared projectors 5a and 5b, a difference between a slit light irradiation image when the slit light L is irradiated and a slit light non-irradiation image when the slit light L is not irradiated is calculated. Then, it is obtained as difference image data.
[0027]
Next, a processing operation for actually estimating the three-dimensional object P in the photographing area A will be described with reference to FIGS. Now, for example, a description will be given on the assumption that the car 1 is parked in a parking space PA of a parking lot. In this parking space PA, blocks P1 and P2 of wheel stoppers are arranged, which are to be estimated as three-dimensional objects P. In the parking space PA, in order to park the cars 1 one by one in a state where the cars 1 are aligned, a section serving as a guide is displayed by a white line WL so as to be easily recognized on the concrete floor surface.
[0028]
FIG. 4 shows an image when the driver puts the car 1 in the parking space PA in the back, and in this example, a case where the car 1 is stopped at this position and photographed is shown. The control device 2 controls the synchronizing circuit 8 and the light emission control circuit 7 to cause the infrared light projectors 5a and 5b to perform the light emission operation and to cause the camera 4 to shoot.
[0029]
FIG. 7A shows a slit light irradiation image, and FIG. 7B shows a slit light non-irradiation image. Here, for the sake of simplicity, an image obtained by irradiating only the slit light L of the infrared projector 5a is shown, but actually, the area of the photographing area A is covered by irradiating the slit light L of the infrared projector 5b. Is what you do. Further, the control device 2 controls the synchronization circuit 8 so as to obtain such a slit light irradiation image and a slit light non-irradiation image alternately.
[0030]
As shown in FIG. 3A, the slit light L is bent at the wheel stoppers P1 and P2, and the line S of the slit light L projected on the floor is photographed on the screen as a broken line Sp. Become. When the wheel stoppers P1 and P2 do not exist, the image is captured as a straight line S which is radial as shown by a broken line.
[0031]
The control device 2 calculates the difference between the two obtained images to obtain difference image data as shown in FIG. In the difference image data, the image of the parking space PA is canceled, and only the lines S and Sp of the slit light L emitted from the infrared projector 5a remain.
[0032]
When recognizing the line Sp that is a polygonal line from the obtained difference image data, the control device 2 estimates that the three-dimensional object P exists in that part. In this case, as shown in FIG. 5B, the control device 2 converts the area of the line S, which is a projection image of the slit light L obtained when the three-dimensional object P does not exist, as shown in FIG. Is set as a mask area MA, and it is checked whether or not the difference value is large along the mask area MA. Then, in a portion where the difference value is small, that is, in a portion where the line S is bent and becomes the line Sp, since the image of the line S does not exist in the mask area MA, it is determined that the three-dimensional object P exists in this portion. Can be estimated.
[0033]
Next, the control device 2 converts the image captured by the camera 4 into an image viewed from above, that is, a bird's-eye view, and combines the image with the estimated area PE of the estimated three-dimensional object P and displays the image as shown in FIG. I will let you. Accordingly, the user (driver) can make the vehicle enter the parking position at a predetermined position while viewing the parking space PA displayed on the monitor 3 and the position of the vehicle 1 as a bird's-eye view. At this time, since the estimated area PE is displayed on the image as the three-dimensional object P, the steering wheel operation can be performed with care.
[0034]
Next, the case where the above-described estimation operation is performed while the vehicle 1 is moving will be briefly described. In this case, since the camera 4 captures the slit light irradiation image and the slit light non-irradiation image while moving, a slight shift occurs when calculating the difference between the two.
[0035]
Here, for example, assuming that the car 1 is put in the garage at the back, the moving speed can be assumed to be about 10 km / h. In general, one image is photographed at a speed of about 1/30 second. As a result, the displacement distance between the moving images is about 10 cm. With such a movement, there is no significant change on the image, and the position of the slit light is constant regardless of the movement.
[0036]
FIG. 7 shows images corresponding to the slit light irradiation image, the slit light non-irradiation image, and the difference image data of FIG. 5 described above. Here, the position of the camera 4 in the slit light non-irradiated image in FIG. 7B is slightly moved as described above with respect to the slit light irradiated image in FIG. In addition to the lines S and Sp of the light L, a region Sw having a large difference occurs near a boundary such as a white line WL.
[0037]
However, when this difference is calculated in the same manner as in the calculation process in the case of stopping, a large difference occurs near the boundary between the slit light L and the white line WL, but the three-dimensional object P exists. On the other hand, the difference can be reduced because the difference is small. In addition, in the determination while moving, a slight error occurs near the boundary of the object, but by extracting a region having the same texture as the three-dimensional object P, the three-dimensional object P can be almost reliably estimated. .
[0038]
According to the first embodiment, an image corresponding to the presence or absence of the slit light L from the infrared light projectors 5a and 5b is captured by the camera 4, and differential image data thereof is created to create a three-dimensional object P. Since the presence is estimated, it is possible to combine the presence of a three-dimensional object that is difficult to recognize only with the image captured by the camera 4 and display the combined three-dimensional object on the monitor 3. As a result, for example, when the image is converted into a bird's-eye view and displayed, misinterpretation is prevented as much as possible, and the image can be reliably recognized.
[0039]
(Second embodiment)
FIG. 8 shows a second embodiment of the present invention, in which a three-dimensional object on the side can be estimated in addition to the configuration of the first embodiment.
As described above, when converted to a bird's-eye view, the car 1a parked next to it is also processed as a floor pattern, so that it appears that the car 1a is located farther than the actual distance. As a result, the user may lose the sense of distance, cause an illusion, and cause a contact accident or the like.
[0040]
In this embodiment, in order to cope with such a case, the configuration corresponding to the camera 4 and the infrared projectors 5a and 5b provided on the rear side is provided on both sides. Here, cameras 10 and 11 are provided as side image pickup means, respectively, and infrared light projectors 12a and 12b and 13a and 13b are provided as side light projecting means. These are connected to the control device 2, and the estimated area of the three-dimensional object P, which is the processing result, is displayed on the monitor 3.
[0041]
Thus, for example, even when the car 1a that did not exist when the car 1 was parked in the parking space PA is parked at the time of departure, the left and right shooting areas AL and AR are photographed by the cameras 10 and 11. By performing the process of estimating the existence of the three-dimensional object, the three-dimensional object can be recognized and displayed on the monitor 3 before departure.
[0042]
In this embodiment, the left and right photographing areas AL and AR are recognized. However, the left and right photographing areas AL and AR may be configured to detect only the hard-to-see side opposite to the driver's seat.
[0043]
(Other embodiments)
The present invention is not limited to the embodiment described above and shown in the drawings, but can be modified as follows.
In the above-described embodiment, the infrared projectors 5a and 5b are provided on the left and right sides of the camera 4, respectively. However, only one infrared projector 5a or 5b may be provided, or three or more infrared projectors 5a and 5b may be provided corresponding to the detection area. You can also.
A configuration in which a plurality of cameras are provided for estimating a three-dimensional object in the shooting area A may be employed.
[0044]
Although the infrared projectors 5a and 5b are used as the projector, a projector that emits visible light, ultraviolet light, or the like may be used. As the light projecting means, for example, a halogen lamp, a normal lamp, an LED, a laser or the like can be used. In this case, when a lamp is used, a mechanical shutter is used or a strobe is used to irradiate slit light in synchronization with the camera. When laser light is used, radial slit light can be realized by using a scanning mechanism or the like.
[0045]
In addition to being provided in the tail lamp of the automobile 1, the light projecting means may be configured to be provided exclusively for another portion.
[Brief description of the drawings]
FIG. 1 is a schematic configuration diagram showing a first embodiment of the present invention. FIG. 2 is an electrical block configuration diagram. FIG. 3 is an operation explanatory diagram showing a principle of estimating a three-dimensional object. FIG. 5 is a diagram illustrating a display example of a captured image. FIG. 5 is a diagram illustrating a mask area. FIG. 6 is a diagram illustrating a display example of a bird's-eye view. FIG. 7 is a diagram corresponding to FIG. FIG. 9 is a diagram showing an embodiment of the present invention. FIG. 9 is a diagram showing an example of a display screen according to the related art.
1 is an automobile (vehicle), 2 is a control device (control means), 3 is a monitor (display means), 4 is a camera (imaging means), 5a and 5b are infrared light emitters (light emitting means), 6 is a control circuit, 7 Is a light emission control circuit, 8 is a synchronization circuit, 9 is a memory, 10 and 11 are cameras (side image pickup means), 12a, 12b, 13a and 13b are infrared light projectors (side light emission means), L is slit light, A is a shooting area, AL and AR are side shooting areas, P is a three-dimensional object, WL is a white line, PA is a parking space, and P1 and P2 are wheel stoppers (three-dimensional objects).

Claims (6)

In a peripheral image display device mounted on a moving body, the periphery of which is photographed by an imaging unit and displayed on a display unit,
Light emitting means for irradiating a slit light to a shooting area;
The difference image data obtained by capturing the slit light irradiation image when the slit light is irradiated to the light projecting means and the slit light non-irradiation image when the slit light is not irradiated and calculating the difference between the pixel components is obtained. A peripheral image display device comprising: a control unit that generates the three-dimensional object from the difference image data and causes the display unit to display the three-dimensional object. The peripheral image display device according to claim 1,
A peripheral image display device, wherein the control means converts an image of a photographing area photographed by the imaging means into a bird's-eye view and displays the image together with the estimated three-dimensional object on the display means. The peripheral image display device according to claim 1 or 2,
The peripheral image display device, wherein the light projecting unit is configured to irradiate infrared slit light as the slit light. The peripheral image display device according to any one of claims 1 to 3,
The control means sets, as a mask area, a projection portion of the slit light emitted by the light projecting means when the three-dimensional object does not exist in the shooting area, and sets the slit light along the mask area for the difference image data. A peripheral image display device configured to judge the presence or absence of the three-dimensional object by determining the presence or absence of the three-dimensional object. The peripheral image display device according to claim 4,
When the slit light irradiation image and the slit light non-irradiation image become images at different positions with the movement of the moving body,
The peripheral image display device, wherein the control unit estimates that a three-dimensional object exists at a place where the difference value of the difference image data is large and a place where the difference value of the mask area is small. The peripheral image display device according to any one of claims 1 to 5,
Along with a side imaging unit set as a shooting area on the side with respect to the moving direction of the moving body,
Equipped with a side light projecting means for irradiating slit light in the side photographing area,
The peripheral image display device, wherein the control unit is configured to estimate the presence of a three-dimensional object also in a side shooting area.

Images