JP5294562B2 - Vehicle periphery monitoring device and display method thereof - Google Patents

Description

  The present invention relates to a technique for monitoring the periphery of a vehicle.

  There is an apparatus that enables monitoring of the periphery of a vehicle by acquiring an image around the vehicle with a camera mounted on the vehicle and displaying the acquired image on a display device (see Patent Document 1, for example).

JP 2006-298217 A

  However, simply displaying a captured image does not always display an obstacle close to the vehicle, and the user may not be aware of its presence.

  The present invention has been made to solve the above problems, and an object of the present invention is to provide a technique for allowing a user to more reliably recognize the presence of an object around a vehicle.

  In order to solve the above-described problems, the present invention detects an entering object based on a captured image and performs a display notifying the detection.

For example, the present invention is a vehicle periphery monitoring device, and includes a captured image acquisition unit that acquires captured images in a plurality of imaging ranges around the vehicle from a plurality of cameras mounted on the vehicle, and a time-lapse for each captured image. On the basis of the change, an intruding object determining means for determining the presence of an intruding object in the imaging range of the captured image, a first display area on the screen of the display device, and an area around the first display area. The second display area is provided, and in the first display area, captured images in the plurality of imaging ranges are displayed at corresponding positions around a position corresponding to a vehicle position, In the display area 2, when it is determined that the entry object is present by the entry object determination means, the captured image in which the existence of the entry object is determined is displayed, and the entry object in the first display area is displayed. The second according to the position The position in the shown area, and a display means for displaying the presence of the intruding object.

  An embodiment of the present invention will be described below with reference to the drawings.

  FIG. 1 is a schematic configuration diagram of a vehicle periphery monitoring device 1 to which an embodiment of the present invention is applied. As shown in the figure, the vehicle periphery monitoring device 1 mounted on a vehicle includes a control device 10, a camera 20, an input device 30, a display device 40, an audio output device 50, and a current position calculation device 60. Composed.

  The camera 20 is an imaging device for obtaining an image around (near) the vehicle. 2 (a), (b), (c), (d), and (e) show the vehicle as viewed from the front, right side, top, left side, and back. As illustrated, the camera 20 includes four small cameras 20F, 20B, 20L, and 20R mounted on the front, rear, left, and right of the vehicle 500. Each of the cameras 20F, 20B, 20L, and 20R faces slightly downward, and images the ground surface in the front, rear, left side, and right side of the vehicle. The camera 20 can change the imaging range by rotating the pedestal. The rotation of the pedestal is performed by a motor and controlled by a camera control unit 12 described later.

  Returning to FIG. The input device 30 is a device for receiving an instruction from a user, and includes a touch panel, a hard switch, and the like.

  The display device 40 includes a liquid crystal display device and the like, and is a unit that displays graphics information generated by the control device 10.

  The audio output device 50 is a unit that includes a speaker or the like and outputs audio information generated by the control device 10.

  The current position calculating device 60 is a device for calculating the current position, and includes a vehicle speed sensor, a gyro sensor, a GPS (Global Positioning System) receiving device, and the like.

  The control device 10 is a device for controlling other devices (camera 20, input device 30, display device 40, audio output device 50, current position calculation device 60, and the like). The control device 10 includes a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), other devices (camera 20, input device 30, display device 40, audio output device 50, current position calculation device). 60), and a general-purpose computer system including a bus for connecting them.

  The control device 10 includes, as its functional units, a main control unit 11, a camera control unit 12, a camera image processing unit 13, an input content analysis unit 14, a display processing unit 15, a voice processing unit 16, and a vehicle state. An acquisition unit 17. These functional units are achieved by executing a predetermined program loaded into the memory by the CPU. Therefore, a program for performing each process is stored in the ROM.

  The main control unit 11 is a central unit that performs various processes. For example, the camera control unit 12 is instructed to control the camera 20. In addition, the camera image processing unit 13 is instructed to acquire an image captured by the camera 20, or the acquired image is converted to an image projected on the ground surface (referred to as a “ground projection image”). Or In addition, a request from the user is received via the input content analysis unit 14, and each functional unit is controlled so that processing corresponding to the received request is executed. In addition, an image is displayed on the display device 40 via the display processing unit 15. In addition, audio is output from the audio output device 50 via the audio processing unit 16.

  The camera control unit 12 controls the operation of each camera 20. For example, the start / end timing of imaging of each camera 20 and the imaging range are set. In addition, transmission of the captured image to the camera image processing unit 13 is controlled. Further, the viewpoint position and the imaging direction of each camera 20 are sent to the camera image processing unit 13 as information for specifying the imaging range of each camera 20. The camera control unit 12 stores in advance the installation positions of the cameras 20 with respect to the vehicle. When the orientation of the camera 20 is changed, the viewpoint position and the imaging direction are recalculated based on the changed angle.

  The camera image processing unit 13 acquires an image captured by each camera 20 as image data. Further, the acquired image is converted into a ground projection image by a known method based on information (viewpoint position, imaging direction) specifying the imaging range of each camera 20.

  FIG. 3 is a diagram for explaining a method of generating a ground projection image. The camera image processing unit 13 acquires the position of the viewpoint P of each camera 20 (coordinate position in a three-dimensional space with a predetermined position in the vehicle as the origin) and the line-of-sight direction K from the camera control unit 12. The imaging direction K intersects the center of the captured image 510 perpendicularly. The distance from the camera viewpoint P to the captured image 510 is determined in advance. The camera image processing unit 13 projects the captured image 510 on the ground surface 520 from the position of the viewpoint P of the camera 20 in the imaging direction K, and generates a ground projection image 530. The ground projection image 530 generated in this way is an image that looks like a bird's-eye view of the vehicle periphery from above the vehicle.

  Returning to FIG. The input content analysis unit 14 analyzes a request from the user input via the input device 30 and sends the result to the main control unit 11.

  The display processing unit 15 displays on the display device 40 a menu for receiving instructions from the user and a ground projection image around the vehicle generated by the camera image processing unit 13. As will be described later, the display processing unit 15 determines whether there is an object approaching the vehicle based on the captured image. When there is an approaching object, a display indicating that there is an approaching object is performed on the outer frame of the ground projection image around the vehicle.

  The voice processing unit 16 outputs a warning sound or a message to the user through the voice output device 50 in accordance with an instruction from the main control unit 11.

  The vehicle state acquisition unit 17 acquires the vehicle speed of the vehicle via the vehicle speed sensor, and determines the traveling state of the vehicle, such as whether the vehicle is traveling or stopped. The stop of the vehicle may be determined from the output of the gyro sensor, the GPS receiver, or the like. Further, the stop of the vehicle may be determined by detecting the state of the side brake and the parking state of the shift lever with a sensor.

  [Description of Operation] Next, the characteristic operation of the vehicle periphery monitoring apparatus 1 of this embodiment will be described.

  FIG. 4 is a flowchart of processing for displaying an image around the vehicle on the display device 40 using an image captured by the camera 20.

  This flow is performed when the user requests to display an image around the vehicle via the input device 30.

  First, the camera image processing unit 13 acquires an image (referred to as “camera image”) obtained by imaging from each of the cameras 20F, 20B, 20L, and 20R (S11).

  Next, the camera image processing unit 13 generates a ground projection image based on each acquired camera image as described above (S12).

  Next, the main control unit 11 instructs the display processing unit 15 to display an image around the vehicle.

  Receiving this, the display processing unit 15 displays an image showing the periphery of the vehicle on the display screen of the display device 40 (S13 in FIG. 4).

  FIG. 5 is a diagram illustrating an example of a display screen.

  As shown in the figure, the display processing unit 15 displays an icon 401 (referred to as a “vehicle icon”) 401 indicating the vehicle in the approximate center of the display screen 400. In addition, an area for displaying ground projection images (referred to as “image display area”) 410 and warning icons 421 and 422 indicating warnings are displayed in order from the vehicle icon 401 toward the outside. (Referred to as “warning display area”) 420.

  The ground projection image 210 is generated from images of the four cameras 20F, 20B, 20L, and 20R. Therefore, the ground projection image 210 around the vehicle is based on four images (a front area image 210F, a rear area image 210B, a left area image 210L, and a right area image 210R) corresponding to an area obtained by dividing the periphery of the vehicle 501 into four. Become.

  Therefore, the display processing unit 15 displays the ground projection images 211F, 211B, 211L, and 211R in the image display area 410 in association with the positions with respect to the vehicle 401, respectively.

  Next, the display processing unit 15 determines whether or not the vehicle has stopped via the vehicle state acquisition unit 17 (S14 in FIG. 4). The vehicle state acquisition unit 17 acquires the movement speed from the vehicle speed sensor, and determines that the vehicle is stopped when the movement speed is equal to or less than a predetermined speed (for example, 1 m / hour). In addition, the vehicle state acquisition part 17 may determine the stop of a vehicle by detecting the state of a side brake and the parking state of a shift lever with a sensor.

  If it is determined that the vehicle is traveling (N in S14), the display processing unit 15 returns to S11 to continue the process.

  On the other hand, when it is determined that the vehicle is stopped (Y in S14), the display processing unit 15 may be the ground projection image 210 (captured image before projection conversion) displayed at that time as the initial image of the vehicle stop. ) Is stored in a predetermined storage area on the memory (S15).

  Then, similarly to the processes of S11 to S12, the camera image processing unit 13 acquires a camera image (S16), and generates a ground projection image (S17). Similarly to S13, the display processing unit 15 displays an image around the vehicle based on the ground projection image (S18).

  Next, the display processing unit 15 determines whether or not there is an object that has entered the vicinity of the vehicle (an entering object) based on the ground projection image 210 (which may be a captured image before projection conversion) (S19). Specifically, the display processing unit 15 compares the ground projection image 210 at the initial stop of the vehicle stored in S15 with the ground projection image 210 generated in S17 this time, and each corresponding dot. Then, the difference between the color tone values is obtained. Then, for each of the ground projection images 212F, 212B, 212L, and 212R, a position of a dot having a difference greater than or equal to a predetermined value is extracted. Further, it is checked whether or not the extracted dots exist as a lump in a predetermined range or more. If it exists, it is determined that there is an entering object. The display processing unit 15 may detect two or more entering objects.

  If no entry object is detected (N in S19), the display processing unit 15 instructs the camera image processing unit 13 to return to S16 and continue the processing.

  On the other hand, if there is an entering object in any of the ground projection images 212F, 212B, 212L, and 212R (however, it may be two or more) (Y in S19), the display processing unit 15 A predetermined warning is displayed in the warning area 430 (430F, 430B, 430L, 430R) corresponding to the projection images 212F, 212B, 212L, 212R (S20).

  Specifically, the display processing unit 15 displays a warning in the warning areas 430F, 230B, 230R, and 230L in the directions of the ground projection images 210F, 210B, 210R, and 210L where there is an approaching object.

  At this time, a warning icon indicating the presence of the entering object is displayed at a position in the warning area 410 corresponding to the position of the entering object in the captured image. Specifically, the position of the approaching object on the ground projection images 210F, 210B, 210R, and 210L (if the approaching object occupies a certain area, the image center position of the area) is the ground projection image 210, A warning icon is displayed at the position moved in the direction of 210B, 210R, 210L as it is.

  For example, as shown in FIG. 5, when there is an approaching object Q1 in the left area image 210L, the display processing unit 15 displays a warning icon 421 in a blinking manner in the left warning area 430L with respect to the vehicle. At this time, a warning icon 421 is displayed at a position where the position (center position) of the approaching object Q1 on the left area image 210L is moved to the left side (downward in FIG. 5) as it is with respect to the vehicle. When the approaching object Q2 is present in the rear area image 210B, the display processing unit 15 blinks and displays the warning icon 422 in the warning area 430B on the rear side with respect to the vehicle. At this time, a warning icon is displayed at the position where the position (center position) of the approaching object Q2 on the rear area image 210B is moved rearward (rightward in FIG. 5) with respect to the vehicle.

  When displaying a warning icon, the main control unit 11 may output a sound effect or message indicating a warning to the audio output device 50 via the audio processing unit 16.

  Next, the display processing unit 15 determines whether or not the vehicle is stopped via the vehicle state acquisition unit 17 (S21 in FIG. 4). If it is stopped (Y in S21), the process returns to S16 to continue the process.

  On the other hand, when it is not stopped (N in S21), the display processing unit 15 returns to S11 and instructs the camera image processing unit to continue the processing.

  The flow of FIG. 4 has been described above.

  Note that the order of the processes shown in FIG. 4 can be changed as appropriate within a range in which the object of the present invention can be achieved.

  According to the present embodiment, when there is an object that has entered a region around the vehicle, it is possible not to simply display the captured image but to display the presence of the entry object outside the frame of the captured image. Therefore, the user can be surely aware of obstacles around the vehicle.

  In addition, since the presence of the entering object is displayed in correspondence with the existing direction, it is possible to easily notify the user of the direction in which attention should be paid.

  The present invention is not limited to the above embodiment. The above embodiment can be variously modified.

  For example, as shown in FIG. 6, the warning area 420 may indicate the moving direction of the entering object. In such a case, the display processing unit 15 obtains the moving direction of the approaching object based on the captured images (ground projection images) continuously captured within a predetermined period. For example, the display processing unit 15 obtains the size of the area occupied by the lump of dots indicating the entering object in the camera image, and determines that the entering object is approaching when the size increases with time. To do. On the contrary, when it is small, it is determined that the approaching object is moving away. Then, as shown in FIG. 6, the moving directions 423 and 324 of the entering objects Q <b> 1 and Q <b> 2 are displayed in the warning area 420.

  In this way, the user can easily understand whether the approaching object approaches or moves away from the vehicle.

  Further, when the display processing unit 15 once detects an entering object and displays a warning icon, even if the entering object does not exist on the ground projection image, the display processing unit 15 displays a warning for a predetermined period (for example, 10 seconds). Can be continued. In this way, when there is an obstacle (for example, a cat) that may enter, the user can be alerted to the direction. In such a case, the display processing unit 15 can change the display mode of the warning icon of the approaching object currently captured and the display mode of the warning icon of the approaching object that has once disappeared, such as by changing the color.

  The present invention can be applied to a vehicle-mounted navigation device that performs route search and route guidance.

  Alternatively, the captured image may be transmitted to an external device via a communication network, and the captured image may be displayed on the external device by the method described above.

FIG. 1 is a schematic configuration diagram of a vehicle periphery monitoring device to which an embodiment of the present invention is applied. FIG. 2 is a diagram illustrating a mounting position of the camera. FIG. 3 is a diagram illustrating a state in which a captured image is projected onto the ground surface. FIG. 4 is a flowchart of the display process. FIG. 5 is a diagram illustrating a screen display example. FIG. 6 is a diagram illustrating a screen display example.

Explanation of symbols

1 ... Vehicle periphery monitoring device,
DESCRIPTION OF SYMBOLS 10 ... Control apparatus, 11 ... Main control part, 12 ... Camera control part, 13 ... Camera image processing part, 14 ... Input content analysis part, 15 ... Display processing part, 16 ... Sound processing part, 17 ... Vehicle state acquisition part 20 …camera,
30 ... Input device,
40. Display device,
50 ... Audio output device 60 ... Current position calculation device

Claims (6)

A vehicle periphery monitoring device,
Captured image acquisition means for acquiring captured images in a plurality of imaging ranges around the vehicle from a plurality of cameras mounted on the vehicle;
Based on the change over time for each captured image, an entry object determination means for determining the presence of an entry object in the imaging range of the captured image;
On the screen of the display device, a first display area and a second display area arranged around the first display area are provided.
In the first display area, a captured image in the plurality of imaging ranges is displayed at a corresponding position around a position corresponding to the vehicle position,
In the second display area, the entry in the first display area that displays a captured image in which the presence of the entry is determined when the entry determination unit determines that the entry is present. A vehicle periphery monitoring device, comprising: a display means for displaying the presence of an approaching object at a position in the second display area corresponding to the position of the object. The vehicle periphery monitoring device according to claim 1,
The first display area is
A rectangular area centered on the vehicle position;
The vehicle periphery monitoring device, wherein the second display area is an outer frame area of the first display area. The vehicle periphery monitoring device according to claim 1 ,
A vehicle periphery monitoring device, comprising: audio output means for outputting a sound notifying the presence of an entering object when the display means displays the presence of the entering object. The vehicle periphery monitoring device according to claim 1 ,
A moving direction determining means for determining a moving direction of the entering object,
The display means includes
When displaying the presence of the entry object,
A vehicle periphery monitoring device that displays the moving direction determined by the moving direction determining means. The vehicle periphery monitoring device according to claim 1 ,
Based on the viewpoint position and imaging direction of the camera, comprising image conversion means for converting the captured image of the camera into a ground projection image projected on the ground surface,
The display means includes
The vehicle periphery monitoring apparatus, wherein the image converted by the image conversion means is displayed in the first display area. A display method for a vehicle periphery monitoring device,
A captured image acquisition step of acquiring captured images in a plurality of imaging ranges around the vehicle from a plurality of cameras mounted on the vehicle;
Based on the change over time for each captured image, an entering object determination step for determining the presence of an entering object in the imaging range of the captured image;
On the screen of the display device, a first display area and a second display area arranged around the first display area are provided.
In the first display area, a captured image in the plurality of imaging ranges is displayed at a corresponding position around a position corresponding to the vehicle position,
In the second display area, the entry in the first display area that displays a captured image in which the presence of the entry is determined when the entry determination unit determines that the entry is present. A display method for a vehicle periphery monitoring device, comprising: a display step of displaying the presence of an approaching object at a position in the second display area corresponding to the position of the object.

Images