JP2015026989A - Vehicle periphery monitoring device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle periphery monitoring device that is able to further reduce discomfort and feeling of pressure in a display image in a vehicle cabin.SOLUTION: The control section of a vehicle periphery monitoring device uses, in an original image photographed by a camera, camera mount parameters that can calculate the end edge position (bumper position) of an own vehicle and a horizontal position for the vehicle. By using the parameters, the control section makes an image correction including a predetermined coordinate conversion such that the proportions of three block areas separated vertically by the bumper position and the horizontal position are close to preset target proportions. Thereby, a virtual coordinate conversion image based on the original image is formed. Thus, using the three corresponding block areas in the coordinate conversion image as a bumper area, which is an area lower than the bumper position, a road surface area, which is an area between the end edge position and the horizontal position, and an empty area, which is an area above the horizontal area, a photographed screen in which the bumper area, road surface area, and empty area are distributed in predetermined proportions, in a balanced manner is displayed.

Description

  According to the present invention, the vehicle driver can monitor the road surface condition and the like from the passenger compartment by photographing the periphery including at least one of the front and rear road surfaces of the host vehicle and displaying the image in the passenger compartment. The vehicle periphery monitoring apparatus configured as described above and a program.

  Conventionally, as a vehicle periphery monitoring device of this type, a rear camera is installed at the rear of the vehicle, and a virtual bird's-eye image is generated by processing an original image of the rear of the vehicle photographed by the rear camera. Devices that display on a display device in a passenger compartment are known.

  In such a vehicle periphery monitoring device, for example, when performing coordinate conversion from an original image to a bird's-eye view image, coordinate conversion using an external parameter indicating the position and orientation of the back camera is performed. In this case, if the position and orientation of the back camera fluctuate due to the mounting error of the back camera or the shake of the vehicle, the bird's-eye view image may not be generated correctly due to the coordinate conversion. It has been proposed to correct the external parameter by detecting the angle and calculating the mounting angle of the back camera based on the detected bumper position (see, for example, Patent Document 1).

JP 2004-64441 A

  However, in the conventional vehicle periphery monitoring device, for example, only the road surface state may be displayed as a bird's-eye view image on a display device in the vehicle interior. Information is difficult to be transmitted to the vehicle driver, and there is a risk of giving the vehicle driver a sense of discomfort and pressure.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a vehicle periphery monitoring device and program capable of further suppressing a sense of discomfort and pressure of a vehicle driver in a display image in a vehicle interior. There is to do.

  In order to achieve the above object, a vehicle periphery monitoring device of the present invention includes a photographing unit, an image processing unit, and a display unit. The photographing means is attached to the host vehicle and configured to be capable of photographing the periphery including at least one road surface in front of and behind the host vehicle. The display means is configured to display a photographing screen based on the coordinate conversion image generated by the image processing means in a predetermined display area in the vehicle interior.

  Here, in the present invention, the image processing means uses the parameters that can calculate the edge position of the own vehicle and the horizontal line position relative to the own vehicle in the original image taken by the photographing means. A virtual coordinate conversion image based on the original image is generated by performing image correction including predetermined coordinate conversion so that the ratio of each of the three divided areas divided in the vertical direction by the above approaches a preset target ratio To do.

  In such a configuration, for example, each of the three divided areas in the coordinate conversion image is divided into an end area which is an area below the end position, and a road surface area which is an area between the end position and the horizontal position. Assuming that the sky area is an area above the horizontal line position, it is possible to display a photographing screen in which the end area, the road surface area, and the sky area are distributed at a predetermined ratio in a balanced manner.

  In other words, in the configuration of the present invention, not only the state of the road surface as a coordinate conversion image on the display device in the passenger compartment, but also the end position of the host vehicle and the horizontal line position with respect to the host vehicle can be easily shown to the vehicle driver. The vehicle driver can intuitively recognize the positional relationship between the host vehicle and the road surface and information on the height direction from such a coordinate conversion image.

  Therefore, according to the present invention, in the display image in the passenger compartment, the sense of discomfort that the positional relationship between the host vehicle and the road surface cannot be intuitively recognized, or the pressure caused by the fact that information on a position higher than the road surface cannot be obtained. That is, it is possible to further suppress a sense of discomfort and pressure on the vehicle driver.

  Note that the above parameters include external parameters indicating the position and orientation of the photographing means, and the image processing means performs, for example, camera calibration using this parameter so that the vehicle on which the photographing means is mounted (own vehicle) ), The edge position of the host vehicle in the original image and the horizontal line position with respect to the host vehicle can be calculated in advance. That is, even when the type (vehicle type) of the vehicle on which the photographing unit is mounted is different, information on the edge position of the own vehicle and the horizontal line position with respect to the own vehicle in the original image can be calculated in advance.

  In addition, the present invention can be distributed on the market as a program. Specifically, it is a program for causing a computer connected to the photographing means and the display means to function as the image processing means.

  By incorporating this program into one or a plurality of computers, it is possible to obtain the same effect as that produced by the vehicle periphery monitoring device of the present invention. The program of the present invention may be stored in a ROM, flash memory, or the like incorporated in a computer and loaded from the ROM, flash memory, or the like into the computer, or loaded into the computer via a network. May be.

  Further, the above program may be used by being recorded on a recording medium in any form readable by a computer. Examples of the recording medium include a portable semiconductor memory (for example, a USB memory or a memory card (registered trademark)).

It is a block diagram which illustrates the whole structure of a vehicle periphery monitoring apparatus. It is explanatory drawing which illustrates the attachment aspect of the camera 2 to the own vehicle. It is explanatory drawing which illustrates each division area in an image. It is a flowchart which illustrates the content of the image process which a vehicle periphery monitoring apparatus (control part) performs. It is explanatory drawing which illustrates the synthesis | combination of the simulated image (bumper image and sky image) in image processing.

Embodiments of the present invention will be described below with reference to the drawings.
The present invention is not construed as being limited in any way by the following embodiments. Moreover, the aspect which abbreviate | omitted a part of following embodiment as long as the subject could be solved is also embodiment of this invention. Moreover, all the aspects which can be considered in the limit which does not deviate from the essence of the invention specified only by the wording described in the claims are embodiments of the present invention. Further, the reference numerals used in the description of the following embodiments are also used in the claims as appropriate, but they are used for the purpose of facilitating the understanding of the invention according to each claim, and the invention according to each claim. It is not intended to limit the technical scope of

<Overall configuration>
As shown in FIG. 1, a vehicle periphery monitoring device 1 according to the present embodiment is attached to a vehicle as photographing means and includes a periphery including at least one road surface in front of and behind the vehicle (hereinafter referred to as “own vehicle”). , A display unit 4 for displaying an image in a predetermined display area in the passenger compartment of the host vehicle as display means, and image correction including predetermined coordinate transformation (hereinafter referred to as “image processing” as image processing means). ”) And a storage unit 8 for storing various information.

  The control unit 6 is a well-known electronic control device configured around a microcomputer, and each unit of the vehicle periphery monitoring device 1 is controlled by the control unit 6. The control unit 6 may be prepared exclusively for controlling the vehicle periphery monitoring device 1 or may be a general-purpose device that performs control other than the vehicle periphery monitoring device 1. Moreover, you may comprise by the single control part 6, and the some control part 6 may function in cooperation.

  The camera 2 has a wide area around the vehicle including a bumper as a rear end portion of the own vehicle and a landscape at a position higher than the road surface along with the road surface behind the own vehicle by a plurality of fisheye lenses installed at the rear of the own vehicle. You can shoot. Further, the camera 2 is configured to include a control device, and when an instruction for a cut-out angle of view is given from the control unit 6, a part of the original image is cut out from the original image at the angle of view according to the instruction. It has a function to provide. In the present embodiment, the control unit 6 gives an instruction to the camera 2 to cut out the central part with less distortion of the original image, and the camera 2 cuts out the central part from the original image according to this instruction (hereinafter “ It is configured to provide a “camera image”) to the control unit 6.

  The display unit 4 is a center display installed in or near the dashboard in the cabin of the host vehicle. The center display is generated by performing image processing on the camera image acquired from the camera 2 by the control unit 6. A shooting screen based on the obtained image (details will be described later) is displayed.

  The storage unit 8 includes, in addition to programs defining image processing performed by the control unit 6 and internal parameters specific to the camera 2 (ie, lens focal length, angle of view, number of pixels, etc.) in the world coordinate system. This is a non-volatile memory that stores external parameters indicating the position and orientation of the camera 2 (hereinafter referred to as “camera 2 mounting parameters”) and the like. The storage unit 8 stores information indicating the bumper position of the host vehicle and the horizontal line position with respect to the host vehicle (hereinafter referred to as “bumper / horizon line position information”) in the original image.

  Specifically, as shown in FIG. 2, the bumper / horizontal line position information has, as viewed from the camera 2, the edge located at the end of the bumper at the rear of the vehicle (side extending in the vehicle width direction) as an edge, Bumper position information indicating in coordinates where each point constituting this edge position (ie, bumper position) is projected, and similarly a position indicating the horizontal direction when the camera 2 is the starting point ( That is, it is composed of horizontal line position information indicating in coordinates where the horizontal line position is projected in the original image.

  In addition, the camera 2 attachment parameters include position information representing the attachment position of the camera 2 in three dimensions (X, Y, Z) with respect to the host vehicle in the world coordinate system, and the attachment angle of the camera 2 by roll, pitch, The control unit 6 (or the control device of the camera 2) includes, for example, a camera calibration using the attachment parameters (and internal parameters) of the camera 2 to provide a bumper. -Horizontal line position information can be calculated in advance.

  For this reason, the bumper / horizontal position information (and thus the mounting parameters of the camera 2 and the like) can be calculated in advance according to the shape of the vehicle (the host vehicle) on which the camera 2 is installed. Even when the type (vehicle type) of the vehicle is different, the bumper position of the own vehicle and the horizontal line position with respect to the own vehicle in the original image can be calculated in advance.

  In the following, an original image (or camera image) that can be acquired from the camera 2 and an image (a coordinate conversion image or a corrected image described later) generated by performing image processing by the control unit 6 are shown in FIG. As shown, the area below the bumper position is an area that mainly indicates the bumper at the rear of the vehicle and is therefore referred to as a bumper area. The area between the bumper position and the horizontal line position mainly indicates the road surface state. Since it becomes an area, it will be referred to as a road surface area, and the area above the horizontal line position will be referred to as an empty area because it will mainly be an area showing an empty landscape when there is no obstacle.

<Image processing>
Next, image processing performed by the control unit 6 will be described with reference to the flowchart of FIG. This process is started when the shift range is changed to R by detecting the shift range based on detection information supplied from a shift position sensor (not shown), for example, when the engine is started. In addition, the control unit 6 executes this process based on a program stored in the storage unit 8.

When this process is started, first, the control unit 6 reads bumper / horizontal position information from the storage unit 8 in S110, and acquires an original image from the camera 2 in subsequent S120.
In subsequent S130, based on the bumper / horizontal line position information read in S110 and the original image acquired in S120, each of the three partitioned areas (bumpers) partitioned in the vertical direction by the bumper position and the horizontal line position in the original image. A coordinate group (collection of a plurality of coordinates) indicating each of the area, the road surface area, and the sky area is specified.

  In subsequent S140, based on the original image acquired in S120, an instruction for cutting out the central portion of the original image with less distortion from the original image is supplied to the camera 2. Upon receiving this instruction from the control unit 6, the camera 2 provides the above-described camera image to the control unit 6.

  In the subsequent S150, the ratio of each partition area is set in advance in the camera image based on the camera image acquired from the camera 2 in S140 and the coordinate group indicating each of the three partition areas specified in S130. By performing image correction including predetermined coordinate conversion so as to approach the target ratio, a virtual coordinate conversion image based on the original image acquired in S110 is generated. In the present embodiment, in the camera image, image correction is performed so that at least the size of the bumper area and the sky area of each partitioned area approaches the target ratio within a range that does not exceed the size based on the target ratio. It has become.

  The target ratio is determined in advance by a sensory test or the like so that each divided area (bumper area, road area, sky area) is visually distributed in a coordinate conversion image (and a corrected image described later). It is a predetermined ratio. In the coordinate conversion, the mounting parameters of the camera 2 (that is, position information and angle information related to the mounting of the camera 2) are used so that the road surface state can be easily recognized from the actual viewpoint from the camera 2. A known viewpoint conversion process is performed to convert the viewpoint to the selected viewpoint. Further, in the image correction, in addition to the viewpoint conversion process, the aspect ratio of the image is changed as necessary.

  In subsequent S160, it is determined whether or not the ratio of each divided area is equal to the target ratio for the coordinate conversion image generated in S150. If an affirmative determination is made here, the process proceeds to S170, and a negative determination is made. Shifts to S180.

In S170, it is permitted to display on the display unit 4 a shooting screen based on the coordinate conversion image determined in S160 that the ratio of each divided area is equal to the target ratio, and this process is terminated.
On the other hand, in S180, whether or not the bumper area is smaller than the target ratio for the coordinate conversion image determined in S160 that the ratio of each divided area is not equal to the target ratio (compared to the size based on the target ratio). If the bumper area is small, the process proceeds to S190. In addition, when a negative determination is made in S180, that is, when the bumper area is a size based on the target ratio, the empty area is small (the empty area is smaller than the size based on the target ratio). The process proceeds to S210.

  In S190, for the coordinate transformation image generated in S150, an image (bumper image) simulating the rear end (bumper) of the host vehicle is used so that the bumper area has a size based on the target ratio. (Refer to FIG. 5A), and a shooting screen based on an image (hereinafter referred to as a “first corrected image”) generated by performing a combining process for combining the bumper image with the coordinate conversion image in this way is displayed. The display on the unit 4 is permitted, and this processing is terminated. In this synthesis process, a bumper image of a corresponding size may be added to an area that is not large enough based on the target ratio in the bumper area, or all areas of the bumper area based on the target ratio are supported. You may join a bumper image of size. The bumper image may be any image as long as it simulates the rear end portion (bumper) of the host vehicle. For example, a simple image may be a black-colored image.

  In subsequent S200, for the coordinate conversion image determined in S160 that the ratio of each divided area is not equal to the target ratio, whether or not the empty area is smaller than the target ratio (the empty area compared to the size based on the target ratio). If the empty area is small, the process proceeds to S210. If an affirmative determination is made here, that is, if the empty area is a size based on the target ratio, this process is performed. finish.

  Finally, in S210, for the coordinate conversion image generated in S150 (or the first corrected image generated in S190), an image (sky image) simulating the sky landscape so that the sky area has a size based on the target ratio. ) Is combined with at least a part of the sky region (see FIG. 5B), and an image generated by performing a combining process for combining the sky image with the coordinate conversion image (or the first corrected image) in this way. Allowing the display unit 4 to display a shooting screen based on (second corrected image) is permitted, and the present process ends. In this synthesis process, a sky image of a corresponding size may be added to an area that is not large enough based on the target ratio in the sky area, or all areas of the sky area based on the target ratio are supported. You may join a sky image of size. The sky image may be any image as long as it simulates the sky landscape. For example, a simple image may be an image filled with a blue color.

<Effect>
As described above, the vehicle periphery monitoring device 1 includes the camera 2, the control unit 6, and the display unit 4. The camera 2 is attached to the host vehicle and is configured to be able to photograph the periphery of the vehicle including the road surface behind the host vehicle. The display unit 4 is configured to display a shooting screen (including a shooting screen based on the corrected image) based on the coordinate conversion image generated by the control unit 6 in a predetermined display area in the vehicle interior.

  Here, the control unit uses the mounting parameters of the camera 2 that can calculate the edge position (bumper position) of the host vehicle and the horizontal line position with respect to the host vehicle in the original image taken by the camera 2, and the bumper position and A virtual coordinate conversion image based on the original image by performing image correction including predetermined coordinate conversion so that the ratio of each of the three divided areas divided in the vertical direction by the horizontal line position approaches a preset target ratio. Is generated.

  In such a configuration, each of the three partitioned areas in the coordinate conversion image is divided into a bumper area that is a lower area than the bumper position, a road surface area that is an area between the edge position and the horizontal line position, and a horizontal line position. As the sky area that is the upper area, it is possible to display a shooting screen in which the bumper area, the road surface area, and the sky area are distributed at a predetermined ratio in a balanced manner.

  That is, in the vehicle periphery monitoring device 1, not only the state of the road surface but also the bumper position of the own vehicle and the horizontal line position with respect to the own vehicle can be easily shown to the vehicle driver as a coordinate conversion image on the display unit 4 in the vehicle interior. Thus, the vehicle driver can intuitively recognize the positional relationship between the host vehicle and the road surface and information on the height direction from such a coordinate conversion image.

  Therefore, according to the vehicle periphery monitoring device 1, in the display image in the passenger compartment, the discomfort that the positional relationship between the host vehicle and the road surface cannot be intuitively recognized, and information on a position higher than the road surface cannot be obtained. It is possible to further suppress a feeling of pressure, that is, a sense of discomfort and a feeling of pressure of the vehicle driver, and it is possible to display a shooting screen that is easy to understand and good for the vehicle driver.

  Further, in the vehicle periphery monitoring device 1, the control unit 6 permits the display unit 4 to display a shooting screen when the ratio of each of the three partitioned areas in the coordinate conversion image is equal to the target ratio. According to such a configuration, in the display image in the vehicle interior, the bumper position of the host vehicle and the horizontal line position with respect to the host vehicle are always kept constant, so that the vehicle driver is not made more uncomfortable. Can do.

  In the vehicle periphery monitoring apparatus 1, the control unit 6 synthesizes an image simulating the rear end (bumper) of the host vehicle with a coordinate conversion image when the bumper area is small with respect to the target ratio. In other words, in the coordinate conversion image, when the bumper position of the host vehicle is shifted downward with respect to the predetermined reference position, or when the end side position has disappeared, the end of the host vehicle is increased in order to improve the visibility. By adding a simulated image showing the part, the bumper position of the host vehicle can be adjusted to a predetermined reference position.

  As a result, it is possible to easily cope with a case where a display image in which the bumper area is distributed with a desired balance cannot be obtained only by image correction including coordinate conversion, which is preferable for the vehicle driver. Can be suppressed. In this case, in the positional relationship between the host vehicle and the road surface, there is a possibility of giving the vehicle driver a feeling that the bumper position of the host vehicle protrudes forward or backward than actually, but if that happens However, it is considered that there is no problem in terms of safety because the vehicle operates in a safe direction for driving the vehicle (it is easy to drive the vehicle so as to avoid collision with an obstacle at an early stage).

  Further, in the vehicle periphery monitoring device 1, the control unit 6 is configured to synthesize an image simulating the sky landscape with the coordinate conversion image when the sky area is small with respect to the target ratio. That is, in the coordinate conversion image, when the horizontal line position with respect to the host vehicle is shifted upward with respect to a predetermined reference position, or when the horizontal line position has disappeared, an empty landscape is shown to improve visibility. By joining the simulated image, the horizontal line position with respect to the host vehicle can be adjusted to a predetermined reference position. As a result, it is possible to easily cope with a case where a display image in which a sky area is distributed with a desired balance cannot be obtained only by image correction including coordinate conversion, so that the vehicle driver can feel a sense of pressure. It can suppress suitably.

<Other embodiments>
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, In the range which does not deviate from the summary of this invention, it is possible to implement in various aspects.

  For example, in the vehicle periphery monitoring device 1 of the above embodiment, the display unit 4 is configured by the center display of the host vehicle. However, the display unit 4 is not limited to this, and various displays such as a meter display and a head-up display. It can be constituted by a device.

  Moreover, in the vehicle periphery monitoring apparatus 1 of the said embodiment, although the camera 2 is installed in the rear part of the own vehicle and is comprised by the rear view camera which can image | photograph the vehicle periphery including the road surface behind the own vehicle, the camera 2 is comprised. However, the present invention is not limited to this, and it may be configured by a front view camera that is installed in the front part of the host vehicle and can capture the vehicle periphery including the road surface in front of the host vehicle.

  Further, in the image processing of the above embodiment, when the ratio of each divided region is equal to the target ratio for the coordinate conversion image generated in S150 (S160; YES), a shooting screen based on this coordinate conversion image is displayed on the display unit 4. Although it is permitted to display (S170), the present invention is not limited to this. For example, when the ratio of each divided area is within an allowable range set in advance based on the target ratio, it is based on the coordinate conversion image. It may be permitted to display the shooting screen on the display unit 4. In addition, for example, when the bumper area is larger than the size based on the target ratio and the sky area is larger than the size based on the target ratio among the divided areas, the shooting screen based on the coordinate conversion image is displayed. The display on the unit 4 may be permitted.

  DESCRIPTION OF SYMBOLS 1 ... Vehicle periphery monitoring apparatus, 2 ... Camera, 4 ... Display part, 6 ... Control part, 8 ... Memory | storage part.

Claims (5)

An imaging means (2) attached to the host vehicle and capable of shooting a periphery including at least one road surface of the front and rear of the host vehicle;
The original image taken by the photographing means is partitioned in the vertical direction by the edge position and the horizontal line position using parameters that can calculate the edge position of the own vehicle and the horizontal line position with respect to the own vehicle. Image processing means (6) for generating a virtual coordinate conversion image based on the original image by performing image correction including predetermined coordinate conversion so that the ratio of each of the divided areas approaches a preset target ratio When,
Display means (4) for displaying a photographing screen based on the coordinate conversion image generated by the image processing means in a predetermined display area in the vehicle interior;
A vehicle periphery monitoring device comprising:   The image processing unit permits display of the photographing screen by the display unit when the ratio of the three divided areas in the coordinate conversion image is equal to the target ratio. Vehicle periphery monitoring device. The three partitioned areas in the coordinate-converted image are divided into an end area that is an area below the edge position, a road area that is an area between the edge position and the horizontal line position, and the horizontal line, respectively. An empty area that is the area above the position,
3. The image processing means synthesizes an image simulating an end of the host vehicle with the coordinate conversion image when the end area is small with respect to the target ratio. The vehicle periphery monitoring device according to claim 1. The three partitioned areas in the coordinate-converted image are divided into an end area that is an area below the edge position, a road area that is an area between the edge position and the horizontal line position, and the horizontal line, respectively. An empty area that is the area above the position,
4. The image processing unit according to claim 1, wherein when the sky area is small with respect to the target ratio, the image processing unit synthesizes an image simulating a sky landscape with the coordinate conversion image. The vehicle periphery monitoring device according to the item.   A program for causing a computer connected to the photographing unit and the display unit according to any one of claims 1 to 4 to function as the image processing unit.