JP2016168877A - Visual recognition device for vehicle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily and visually recognize a necessary region without moving a photographing section and replacing a lens.SOLUTION: A visual recognition device for a vehicle acquires a rear side photographed image obtained by photographing by a rear side camera 12 and a peripheral photographed image (photographed image including at least a part of the rear side photographed image and having a wider photographing range than that of the rear side photographed image) obtained by photographing by a peripheral camera 14. The visual recognition device includes a control device 18 to perform control of compounding the rear side photographed image and the peripheral photographed image by aligning the position and size of each subject, forming a display image and displaying the image on a monitor 16, when a retreat switch is turned on.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a vehicular visual recognition device for visually recognizing a vehicle periphery by photographing the vehicle periphery and displaying a captured image.

  For example, techniques disclosed in Patent Documents 1 and 2 have been proposed as a vehicular visual recognition device that displays captured images of the vehicle periphery and visually recognizes the vehicle periphery.

  In patent document 1, when there is no obstacle around the vehicle, an overhead image centered on the vehicle is displayed, and when there is an obstacle around the vehicle, a rear captured image is displayed. Yes.

  Japanese Patent Application Laid-Open No. 2004-133620 proposes that the surroundings of a vehicle are visually recognized by photographing a landscape behind the vehicle corresponding to left and right door mirrors with a left and right camera and displaying it on one display device.

  Further, in the case of a backward movement, in the case of a backward movement, in conjunction with a shift operation to a backward position, etc., a technique for moving the mirror surface of the door mirror to the lower side direction of the vehicle and changing the obstacle to an easy-to-see obstacle on the side of the vehicle (so-called Reverse interlock control) is known.

JP 2006-27556 A JP 2011-217318 A

  In the technique of visually recognizing the periphery of a vehicle by displaying captured images as in Patent Documents 1 and 2, when using an alternative to an outer mirror, a capturing unit such as a camera requires a clear image. However, since the sharpness of the image and the angle of view are contradictory, it is necessary to narrow the angle of view of the camera in order to capture a clear image.

  In order to incorporate the control for changing the viewing range, such as the reverse interlocking control as described above and the turning interlocking control for changing the viewing range in conjunction with the turning, a method of moving the photographing unit or switching the lens is considered. However, the structure becomes complicated.

  The present invention has been made in consideration of the above-described facts, and an object of the present invention is to make it easy to visually recognize a necessary area without moving an imaging unit or exchanging lenses.

  In order to achieve the above object, the invention according to claim 1 is a rear side photographed image including a rear side photographed image obtained by photographing a rear side of a vehicle and at least a part of the rear side photographed image. An acquisition unit that acquires a peripheral captured image obtained by capturing the periphery of a vehicle having a wider imaging range than the image, and the rear side captured image acquired by the acquisition unit when a predetermined trigger signal is received, and the And a generation unit that generates a display image by combining the peripheral captured image with the position and size of each subject.

  According to the first aspect of the present invention, the acquisition unit acquires the rear side captured image and the peripheral captured image.

  Then, when the predetermined trigger signal is received, the generation unit combines the rear side captured image acquired by the acquisition unit and the peripheral captured image in accordance with the position and size of each subject to generate a display image. Generated. By displaying the display image generated by the generation unit in this manner, it becomes possible to visually recognize an area that cannot be viewed with only the rear side captured image, and it is easy without moving the imaging unit or exchanging lenses. It is possible to make a necessary area visible.

  The generation unit may generate a display image by cutting out a predetermined necessary area when generating the display image. In this case, when the generation unit receives a signal representing the reverse of the vehicle as a trigger signal, the generation unit generates a display image by cutting out a region including a region corresponding to the lower side of the rear side captured image as a necessary region. May be. Alternatively, when the generation unit receives a signal representing the turning of the vehicle as a trigger signal, the generation unit may generate a display image by cutting out a region including a region corresponding to the vehicle outside of the rear side captured image as a necessary region. Good. Here, for example, a signal such as a switch for detecting that the shift device has been operated to the reverse position can be applied as the signal representing the reverse, and a signal from the direction indicator or the like can be used as the signal representing the turning. Can be applied.

  Moreover, it is good also as a structure further equipped with the rear side imaging | photography part which image | photographs the vehicle rear side, and obtains a rear side captured image, and the periphery imaging | photography part which image | photographs the vehicle periphery and obtains a periphery captured image. In this case, the rear side photographing unit and the peripheral photographing unit may be arranged in the same housing.

  Further, the image display apparatus further includes a display unit that displays an image, and when the generation unit has not received a predetermined trigger signal, the rear side captured image is displayed, and when the predetermined trigger signal is received, the display is displayed. The display unit may be further controlled to display an image.

  As described above, according to the present invention, there is an effect that a necessary area can be easily visually recognized without moving the photographing unit or exchanging lenses.

It is a figure which shows schematic structure of the visual recognition apparatus for vehicles which concerns on embodiment of this invention. It is a block diagram which shows the structure of the control system of the visual recognition apparatus for vehicles which concerns on this embodiment. It is a figure which shows each imaging | photography range of a rear side camera and a peripheral camera. (A) is a figure which shows an example of the picked-up image of a peripheral camera, (B) is an example of the image which extracted the area | region corresponding to the picked-up image of a rear side camera from the picked-up image of a peripheral camera, and performed distortion correction. (C) is a figure which shows the image after the mirror image conversion of (B). (A) is a figure which shows an example of the picked-up image of a rear side camera, (B) is a figure which shows the image after the mirror image conversion of (A). It is a figure which shows the example which synthesize | combined the picked-up image of a peripheral camera, and the picked-up image of a rear side camera. It is a flowchart which shows an example of the flow of the reverse interlocking control performed with the control apparatus of the visual recognition apparatus for vehicles which concerns on this embodiment. It is a flowchart which shows an example of the flow of the process performed with a center monitor apparatus.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of a vehicular visual recognition apparatus according to an embodiment of the present invention.

  The vehicle visual recognition device according to the present embodiment includes a rear side camera 12 as a rear side photographing unit, a peripheral camera 14 as a vehicle peripheral photographing unit, a monitor 16 as a display unit, and an acquisition unit and a generation unit. Control devices 18 are provided corresponding to the left and right sides of the vehicle, respectively. Further, a center monitor device 20 as an overhead image display unit is provided at the center of the instrument panel.

  The rear side camera 12 is installed outside at the front side end of the vehicle in the vertical direction middle part of the side door (front side door, not shown) of the vehicle and photographs the rear side of the vehicle. The rear side camera 12 is provided in a substantially rectangular parallelepiped box-shaped housing 13 as a support, and a lens is arranged toward the rear side of the vehicle to photograph the rear side of the vehicle. The vehicle width direction inner side end part of the housing | casing 13 is attached to the side door, and the housing | casing 13 is supported by the side door (vehicle body side) so that rotation in the vehicle front-back direction is possible.

  The peripheral camera 14 is provided in the same housing 13 as the rear side camera 12 and images the periphery of the vehicle. The peripheral camera 14 is provided with a fisheye lens and is arranged in the vehicle lower direction. That is, the peripheral camera 14 captures the front-rear direction and the lateral direction of the vehicle that is the periphery of the vehicle with the fisheye lens. In addition, the peripheral captured image of the peripheral camera 14 includes at least a part of the rear side captured image of the rear side camera 12 and is a captured image having a wider capturing range than the rear side captured image. .

  The monitor 16 is provided in the vicinity of the lower end of the front pillar, and mainly displays the captured image of the rear side camera 12. That is, the rear side of the vehicle can be visually recognized by checking the monitor 16 instead of the outer mirror.

  The control device 18 is provided corresponding to each of the left and right, and performs display control of the captured image of each of the left and right rear side cameras 12 and also performs reverse interlock control described later. Note that the reverse interlock control is a control that makes it easier to visually recognize an obstacle on the vehicle side lower side by changing the visual recognition range on the vehicle rear side when reversing.

  The center monitor device 20 can generate and display a bird's-eye view image centered on the vehicle from the captured image of the peripheral camera 14, and can display an operation panel of an in-vehicle device (for example, navigation device, air conditioner, audio). Has been.

  Then, the structure of the control system of the visual recognition apparatus 10 for vehicles which concerns on this embodiment is demonstrated. FIG. 2 is a block diagram illustrating a configuration of a control system of the vehicle visual recognition device 10 according to the present embodiment.

  The control device 18 includes a microcomputer in which a CPU 18A, a ROM 18B, a RAM 18C, and an I / O (input / output interface) 18D are connected to a bus 18E.

  The ROM 18B stores various programs such as display control for displaying the rear side photographed image of the rear side camera 12 on the monitor 16 and a reverse interlock control program for performing the reverse interlock control described above. The program stored in the ROM 18B is expanded in the RAM 18C and executed by the CPU 18A, whereby display control on the monitor 16 is performed.

  The I / O 18D is connected with a backward switch 22, a rear side camera 12, a peripheral camera 14, a monitor 16, and a center monitor device 20.

  The reverse switch 22 is provided in the shift device or the like, detects whether or not the shift device is operated at the reverse position, and outputs the detection result to the control device 18. Instead of the reverse switch 22, a shift position sensor that detects the shift position may be applied.

  The rear side camera 12 obtains a rear side photographed image by photographing the rear side of the vehicle. The rear side photographed image obtained by photographing is output to the control device 18 as a photographing result of the rear side camera 12.

  The peripheral camera 14 obtains a peripheral photographed image by photographing the front-rear direction and the lateral direction of the vehicle that is around the vehicle. The peripheral photographed image obtained by photographing is output to the control device 18 as a photographing result of the peripheral camera 14. The peripheral photographed image is a photographed image around the vehicle having a wider photographing range than the rear side photographed image. Note that the photographing result of the peripheral camera 14 is also output to the center monitor device 20.

  The monitor 16 displays a rear side photographed image photographed by the rear side camera 12 during normal driving, and is generated by reverse interlock control when retreating (when the retreat switch 22 is turned on). Display the display image. In the reverse interlocking control of the present embodiment, a display image including a region below the normal captured image is obtained by supplementing the region outside the capturing range of the rear side camera 12 with the peripheral captured image of the peripheral camera 14 when reversing. Is generated and displayed.

  As described above, the center monitor device 20 generates and displays a bird's-eye view image from the peripheral captured image of the peripheral camera 14. Further, as described above, the center monitor device 20 displays an operation panel or the like of an in-vehicle device (for example, an air conditioner, a navigation device, audio, etc.). Although not shown, the center monitor device 20 includes a microcomputer in which a CPU, a ROM, a RAM, and an I / O are connected to a bus, as in the control device 18.

  Since a pair of control devices 18 are provided corresponding to the left and right sides of the vehicle, the other control device 18 is connected to each I / O 18D so that they can communicate with each other.

  Here, the above-described reverse interlock control performed by the vehicle visual recognition device 10 according to the present embodiment will be described in detail.

  In the present embodiment, the rear side camera 12 uses a lens that can capture a clearer image than the peripheral camera 14. Therefore, the angle of view of the rear side camera 12 is narrower than that of the peripheral camera 14. Since the rear side camera 12 has a high resolution and a narrow angle of view, a necessary area cannot be displayed when reverse interlock control is performed in which the display range is changed in conjunction with the backward movement.

  Therefore, in the present embodiment, a necessary area is extracted from the peripheral captured image of the peripheral camera 14, and the rear side captured image of the rear side camera 12 and the extracted peripheral captured image area are defined as the position of each subject and Necessary areas are displayed by combining them according to size.

  Specifically, the shooting range of the peripheral camera 14 is an area indicated by the screen tone shown in FIG. 3, and for example, a hemispherical area as shown in FIG. 4A is obtained as a peripheral shot image.

  On the other hand, the shooting range of the rear side camera 12 is an area indicated by hatching shown in FIG. 3. For example, as shown in FIG. 5A, an image of the vehicle rear side area is obtained as a rear side shot image. It is done.

  In the reverse interlocking control, when the backward switch 22 detects the backward movement, an image of an area including at least a part of the rear side captured image of the rear side camera 12 is cut out from the peripheral captured image of the peripheral camera 14. Then, the rear side photographed image of the rear side camera 12 and the clipped image are combined and displayed with the position and size of each subject being matched. As a result, an area outside the shooting range of the rear side camera 12 can be supplemented and displayed with the peripheral shot image of the peripheral camera 14, so that it is possible to visually recognize an obstacle on the vehicle side lower side when reversing. .

  More specifically, the area of the peripheral captured image of the peripheral camera 14 including at least a part of the rear lateral captured image of the rear lateral camera 12 in FIG. 5A (dotted line frame in FIG. 4A) is controlled. The device 18 cuts out. Further, the control device 18 corrects the distortion aberration caused by the fisheye lens by performing lens aberration correction or the like on the cut image, and generates an image as shown in FIG.

  The image shown in FIG. 4 (B) is a right rear photographed image, but when viewed as a substitute for the outer mirror, it is a horizontally reversed image. Therefore, in order to replace the monitor 16 with the outer mirror, the control device 18 performs mirror image conversion and inverts the left and right to generate the image shown in FIG.

  On the other hand, with respect to the captured image of the rear side camera 12, the rear side captured image shown in FIG. 5A is a captured image on the right rear side, but when viewed as a substitute for the outer mirror, the image is reversed left and right. Yes. Therefore, similarly to the above, the control device 18 performs mirror image conversion and inverts the left and right to generate the image shown in FIG.

  Subsequently, as shown in FIG. 6, the control device 18 combines the rear side captured image and the image cut out from the peripheral captured image in accordance with the position and size of each subject, and a necessary range from the combined image. A display image to be cut out and displayed on the monitor 16 is generated (dotted line frame in FIG. 6). Here, in the synthesis, in the present embodiment, the rear side captured image is used as a reference, and the lower vehicle area of the rear side image is combined with an image cut out from the peripheral captured image. Further, when the sizes of the subjects of the image extracted from the peripheral captured image of the peripheral camera 14 and the rear side captured image of the rear side camera 12 are matched, for example, the rear side captured image is used as a reference. Then, enlargement / reduction processing or the like is performed in accordance with the size of the image extracted from the peripheral captured image.

  Then, control is performed so that the display image generated by the control device 18 is displayed on the monitor 16. That is, a display image obtained by combining a part of the photographed image of the rear side camera 12 and a part of the photographed image of the peripheral camera 14 in accordance with the position and size of each subject is displayed on the monitor 16. The area will be changed. As a result, it is possible to display an area that is not visible in the rear side photographed image.

  Thus, by performing reverse interlocking control, a necessary area can be easily visually recognized without moving the camera or exchanging lenses. In the present embodiment, if the vehicle is equipped with the peripheral camera 14, the obstacles on the side can be confirmed when the vehicle moves backward by displaying the necessary area using the peripheral captured image of the peripheral camera 14.

  Note that the synthesis of the rear side image and the peripheral captured image is not limited to the above, and the outer region of the rear side captured image may be combined with an image cut out from the peripheral captured image.

  Further, in the present embodiment, a necessary range (dotted line frame in FIG. 6) including a region below the photographing range of the rear side photographed image is cut out from the synthesized image and displayed, but the necessary region is not cut out. The entire image synthesized in the above may be converted into a size that can be displayed on the monitor 16 and displayed.

  Then, the specific process performed with the visual recognition apparatus 10 for vehicles which concerns on this embodiment comprised as mentioned above is demonstrated.

  FIG. 7 is a flowchart showing an example of the flow of reverse interlock control performed by the control device 18 of the vehicle visual recognition device 10 according to the present embodiment. The reverse interlock control in FIG. 7 is started by executing a reverse interlock control program as an example of a vehicle visual control program stored in advance in the ROM 18B when an ignition switch (IG) (not shown) is turned on. .

  In step 100, the control device 18 controls the rear side camera 12 to start photographing with the rear side camera 12, and the process proceeds to step 102.

  In step 102, the control device 18 acquires the imaging result of the rear side camera 12, performs mirror image conversion on the acquired rear side captured image, starts displaying on the monitor 16, and proceeds to step 104. That is, a captured image as shown in FIG. 5B is displayed on the monitor 16. Since the monitor 16 is provided in the vicinity of the lower end of the front pillar, it is possible to visually recognize the rear side by looking at the monitor 16 instead of the outer mirror.

  In step 104, the control device 18 determines whether or not the vehicle is moving backward. In this determination, it is determined whether or not the control device 18 is reverse by determining whether or not the reverse switch 22 is turned on. If the determination is affirmative, the process proceeds to step 106 and is negative. If so, go to Step 120.

  In step 106, the control device 18 acquires the photographing result of the peripheral camera 14 and proceeds to step 108.

  In step 108, the control device 18 cuts out an image of a predetermined area of the peripheral captured image, and proceeds to step 110. For example, the predetermined region to be cut out is obtained in advance as a region including at least a part of the rear side photographed image of the rear side camera 12 in the peripheral photographed image of the peripheral camera 14, and a dotted frame shown in FIG. As described above, a predetermined region in the captured image of the peripheral camera 14 is cut out. It should be noted that an area corresponding to the rear side photographed image is identified from the peripheral photographed image by performing image matching or the like on the photographed images of both the rear side camera 12 and the peripheral camera 14, and includes the identified area. A region wider than the side shot image may be extracted.

  In step 110, the control device 18 performs correction such as distortion correction on the peripheral captured image of the cut-out area, thereby generating an image as shown in FIG. 4B and proceeds to step 112.

  In step 112, the control device 18 performs mirror image conversion to generate an image as shown in FIG.

  In step 114, the control device 18 uses the rear side captured image of the rear side camera 12 after the mirror image conversion and the image cut out from the peripheral captured image of the peripheral camera 14 after the mirror image conversion for the position and size of each subject. Are combined and the process proceeds to step 116. Thereby, a composite image as shown in FIG. 6 can be obtained.

  In step 116, the control device 18 performs control so that a predetermined area is cut out and displayed on the monitor 16, and the process proceeds to step 118. For example, the control device 18 cuts out an area indicated by a dotted line frame in FIG. 6 and displays it on the monitor 16, thereby displaying an area outside the imaging range of the rear side camera 12 when reversing and checking an obstacle. It becomes possible.

  In step 118, the control device 18 determines whether or not the backward movement is finished. In this determination, the control device 18 determines whether or not the reverse switch 22 is turned off. If the determination is affirmative, the process proceeds to step 120. If the determination is negative, the process returns to step 106 to perform the above-described processing. Is repeated.

  In step 120, the control device 18 performs control so that the imaging result of the rear side camera 12 is mirror-image converted and displayed on the monitor 16, and the process proceeds to step 122. That is, the reverse interlock control is terminated, and switching is performed so that only the captured image of the rear side camera 12 is displayed on the monitor 16.

  In step 122, the control device 18 determines whether or not the ignition switch (IG) is turned off. If the determination is negative, the process returns to step 104 and the above processing is repeated, and the determination is affirmed. A series of reverse interlocking control ends.

  On the other hand, the center monitor device 20 displays a bird's-eye view image when retreating. FIG. 8 is a flowchart illustrating an example of the flow of processing performed by the center monitor device 20. Note that the processing of FIG. 8 may be started when the reverse switch 22 is turned on. Or you may start when the display of a bird's-eye view image is instruct | indicated by the passenger | crew. Alternatively, it may be started when the ignition is turned on.

  In step 200, the center monitor device 20 acquires the photographing result of the peripheral camera 14, and the process proceeds to step 202. That is, the center monitor device 20 acquires the peripheral captured images of the left and right peripheral cameras 14.

  In step 202, distortion correction is performed on the peripheral captured image acquired by the center monitor device 20, and the process proceeds to step 204.

  In step 204, the center monitor device 20 performs a bird's-eye conversion on the distortion-corrected peripheral captured image to generate a bird's-eye view image, and the process proceeds to step 206. Thereby, an overhead image centering on the vehicle is generated.

  In step 206, the overhead image generated by the center monitor device 20 is displayed, and the process proceeds to step 208.

  In step 208, the center monitor device 20 determines whether or not to end the display of the overhead image. In this determination, for example, it may be determined whether the reverse switch 22 is turned off, or it may be determined whether display of another image is instructed. Alternatively, it may be determined whether or not the ignition switch is turned off. If the determination is negative, the process returns to step 200 and the above-described processing is repeated. When the determination is positive, the series of processing ends.

  Thus, by displaying the bird's-eye view image on the center monitor device 20, the reverse interlock control displays the necessary area (rear area on the lower side of the rear side) on the monitor 16 while checking the obstacle, By displaying a bird's-eye view image on the monitor device 20, the entire positional relationship can be viewed and retreated.

  In the above-described embodiment, the example in which the area displayed on the monitor 16 is changed using the signal of the reverse switch 22 as a trigger (in conjunction with the reverse) has been described, but the present invention is not limited to this. For example, the present invention may be applied to turning interlock control, and the area to be displayed on the monitor 16 may be changed using the signal from the direction indicator as a trigger (in conjunction with turning). Thereby, it becomes possible to display a necessary area on the monitor 16 at the time of confirmation of entrainment at the time of turning or at the time of rearward confirmation at the time of lane change. In the case of turn-linked control, a region that is cut out from the peripheral shot image is cut out from a region that is different from the reverse-linked control (for example, a region that includes a region corresponding to the outside of the vehicle in the rear side shot image). What is necessary is just to synthesize | combine with a picked-up image. Moreover, you may make it perform both reverse interlocking control and turning interlocking control. For example, by connecting a direction indicator to the I / O 18D with respect to the above embodiment and enabling the control device 18 to detect the operation of the direction indicator, at least one signal of the backward switch 22 and the direction indicator As a trigger signal, both reverse interlocking control and turning interlocking control can be realized.

  Moreover, although the process performed by the control apparatus 18 and the center monitor apparatus 20 in said embodiment was demonstrated as a software process, it is not restricted to this. For example, the processing may be performed by hardware, or may be processing that combines both hardware and software.

  Further, the processing performed by the control device 18 and the center monitor device 20 in the above embodiment may be stored as a program in a storage medium and distributed.

  Furthermore, the present invention is not limited to the above, and it goes without saying that various modifications can be made without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 10 Vehicle visual recognition apparatus 12 Rear side camera 14 Peripheral camera 16 Monitor 18 Control apparatus 20 Center monitor apparatus 22 Reverse switch

Claims (7)

A rear side photographed image obtained by photographing the rear side of the vehicle, and a periphery obtained by photographing a vehicle periphery having a wider photographing range than the rear side photographed image including at least a part of the rear side photographed image An acquisition unit for acquiring a captured image;
Generation of generating a display image by synthesizing the rear side captured image acquired by the acquiring unit and the peripheral captured image in accordance with the position and size of each subject when a predetermined trigger signal is received And
A vehicular visual recognition device.   The vehicular visual recognition device according to claim 1, wherein the generation unit generates the display image by cutting out a predetermined necessary area when generating the display image.   When the generation unit receives a signal indicating the reverse of the vehicle as the trigger signal, the generation unit cuts out a region including a region corresponding to the vehicle lower side of the rear side captured image as the necessary region, and displays the display image. The vehicular visual recognition device according to claim 2 to be generated.   When the generation unit receives a signal indicating turning of the vehicle as the trigger signal, the generation unit cuts out an area including an area corresponding to the outside of the rear side photographed image as the necessary area, and generates the display image. The vehicular visual recognition device according to claim 2 or 3.   5. A rear side photographing unit for photographing a rear side of the vehicle to obtain the rear side photographed image, and a peripheral photographing unit for photographing the periphery of the vehicle to obtain the peripheral photographed image. The vehicle visual recognition device according to any one of the above.   The vehicle visual recognition device according to claim 5, wherein the rear side photographing unit and the peripheral photographing unit are arranged in the same housing. A display unit for displaying an image;
The generator further displays the rear side captured image when the trigger signal is not received and further controls the display unit to display the display image when the trigger signal is received. The vehicle visual recognition device according to any one of 1 to 6.