JP5134608B2 - Vehicle periphery display device, vehicle periphery display method and program - Google Patents

Description

  The present invention relates to a vehicle periphery display device that captures and displays a state around a vehicle for driving support of the vehicle.

  2. Description of the Related Art Conventionally, a system that supports driving by providing an image of a situation around a vehicle to a driver has been known. Patent document 1 is disclosing the invention of the apparatus which displays the monitoring image of the side of a vehicle. In the invention described in Patent Document 1, a place where the side monitoring image needs to be displayed is registered in advance, and the side monitoring image is displayed on the display when the vehicle reaches the place.

  Patent Document 2 discloses an invention of an apparatus that switches and displays captured images around a vehicle captured by a camera. The invention described in Patent Document 2 switches the display screen based on the configuration of the road. In Patent Document 2, “road configuration” is information on the road position, the number of lanes, the width, and the connection relationship between roads such as road inflow, branching, and intersection.

  Patent Document 3 discloses an invention of an apparatus that captures and displays a situation in a direction that a driver wants to see at a planned photographing position. In the invention described in Patent Document 3, when the current position of the vehicle is at or near the shooting position, it is determined whether or not the current situation matches the shooting conditions. Take a picture with the specified camera. Note that, in Japanese Patent Application Laid-Open No. H10-228707, “imaging conditions” are conditions such as a traveling direction, a transmission shift position, and a vehicle speed.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a mode in which the scheduled shooting position is automatically registered. That is, when a driver manually instructs shooting at a point where the driver wanted to take a picture while the vehicle is running, an image of that point is taken and displayed on the display device, and the position is registered as a planned shooting position. . Further, Patent Document 3 discloses that an obstacle or the like is detected from a photographed image, and when the vehicle body is likely to rub the obstacle, the position is automatically registered as a photographing scheduled position. .

JP-A-8-293098 JP 2005-309812 A JP 2005-318519 A

  However, in the invention described in Patent Document 1, it is necessary to register in advance a place where the side monitoring image needs to be displayed, and it cannot be said that the driver is easy to use. Since the invention described in Patent Document 2 captures and displays an image around the vehicle based on the road configuration, it is not necessary to register in advance. However, whether or not to view an image around the vehicle depends on the skill level of the driver, the degree of familiarity with the road on which the vehicle is traveling, and the scene in which the user is not good. For example, some drivers are not good at heavy rain, while others are not good at night. When a peripheral image of a vehicle is displayed in a scene that is not good, the image display itself may be bothersome.

  As described in Patent Document 3, according to the method of registering a point where the driver has instructed image display in the past, it is possible to display an image at a point based on the driver's request. However, this method can be applied only to roads that have traveled in the past, and cannot be applied to travel on new roads.

  In view of the above background, an object of the present invention is to provide a vehicle periphery display device that appropriately displays a peripheral image in a place where the driver wants to display an image of the vehicle periphery.

  A vehicle periphery display device according to the present invention is a device that displays a peripheral image captured by a camera mounted on a vehicle, and includes a peripheral image displayed in the past and road data relating to a road at a point where the peripheral image is displayed. A display history storage unit, a peripheral image generation unit that acquires image data being captured by the camera and generates a peripheral image, and a current position data acquisition unit that acquires data on the current position of the vehicle And an image search unit that acquires road data corresponding to the current position of the vehicle from a map storage unit that stores map data including road data, and searches the display history storage unit for peripheral images associated with the acquired road data And a similarity calculator that calculates the similarity between the searched peripheral image and the peripheral image generated by the peripheral image generator when the peripheral image is searched by the image search unit, If serial similarity is not smaller than the predetermined threshold value, and a display control unit for displaying the peripheral image generated by said peripheral image generating unit on the display.

  Thus, when the similarity between the current peripheral image and the peripheral image displayed in the past stored in the display history storage unit is equal to or greater than a predetermined threshold, the current peripheral image is displayed by the configuration in which the current peripheral image is displayed. In the case of the same scene as the peripheral image displayed in the past, the peripheral image can be displayed. Thus, the surrounding image is not displayed unnecessarily simply because the road data is the same, and the image display can be prevented from being bothered. In addition, when a surrounding image that matches the road data is searched, the similarity calculation of the surrounding image is unlikely to be performed by the configuration that calculates the similarity of the surrounding image so that the road data is unlikely to be similar. it can.

  The vehicle periphery display device of the present invention includes a vehicle data acquisition unit that acquires vehicle data indicating a state of the vehicle from a sensor mounted on the vehicle, and the display history storage unit stores vehicle data at the time of displaying the surrounding image. The display history storage unit stores the peripheral image associated with the road image corresponding to the current position of the vehicle and the vehicle data acquired by the vehicle data acquisition unit. You may search from.

  Thus, by searching for a peripheral image associated with the vehicle data acquired by the vehicle data acquisition unit, it is possible to efficiently search for a peripheral image that may be similar.

  The vehicle periphery display device of the present invention acquires the road data on the route from the route data acquisition unit that acquires the data of the route from the current position to the destination, and the map data stored in the map storage unit, and acquires The road data stored in the display history storage unit is compared, and a point having the same road data as the road data stored in the display history storage unit is used as a display candidate site for the surrounding image on the route. The image search unit includes a display plan part that is obtained from the current position data acquisition part, and the vehicle includes the display candidate place based on the current position data acquired by the current position data acquisition part. When it is determined that the vehicle falls within a predetermined range, a peripheral image associated with the road data may be searched from the display history storage unit.

  In this way, by obtaining a display candidate site having the same road data as the road data stored in the display history storage unit based on the route data, a peripheral image is displayed when the vehicle reaches the display candidate site. It is sufficient to perform a process for determining whether or not. Accordingly, it is not always necessary to determine whether or not a peripheral image with matching road data exists in the display history storage unit during traveling, and the processing burden during traveling can be reduced.

  In the vehicle periphery display device of the present invention, the display history storage unit stores data of a periphery image when display of the periphery image is stopped, and the similarity calculation unit is a periphery image when display is stopped When the similarity is equal to or higher than a predetermined threshold, the display control unit may stop displaying the peripheral image on the display. Good.

  With this configuration, the image display can be stopped at an appropriate timing based on the past display history.

  In the vehicle periphery display device of the present invention, the image search unit uses a part of the acquired road data as a search key, searches for a peripheral image associated with the part of the road data, When there are a predetermined number or more of images associated with the part of road data, the number of road data serving as search keys may be increased to search for surrounding images.

  By using a part of road data as a search key in this way, even if the road data excluded from the search key does not match, the degree of similarity between the peripheral image stored in the display history storage unit and the current peripheral image Calculate Accordingly, since the peripheral images that are candidates for comparison are widely searched, it is possible to reduce the possibility that the processing ends at the stage before comparison with the past peripheral images and a similar peripheral image is overlooked. Also, if there are more than a predetermined number of images associated with some road data used as search keys, increasing the search keys reduces the associated images and reduces the processing load required for similarity calculation. it can. In addition, since the search key (a combination of road data) associated with a predetermined number or more of peripheral images is frequent road data, even if the search key is increased, there is a possibility that a similar peripheral image may be overlooked. It is thought that there are few.

  In the vehicle peripheral display device of the present invention, the display history storage unit stores direction specifying data for specifying which direction of the peripheral image is displayed, and the peripheral image generated by the peripheral image generation unit is stored in the display history storage unit. It may be used to generate a peripheral image specified by the direction specifying data.

  By having the direction specifying data in this way, it is possible to appropriately calculate the similarity of the peripheral images in the same direction.

  In the vehicle periphery display device of the present invention, the display history storage unit stores an image of an object on a road detected from a peripheral image displayed in the past, and the similarity calculation unit calculates the similarity from the peripheral image. The degree of similarity between the detected image of the object on the road and the image of the object stored in the display history storage unit may be calculated.

  With this configuration, the amount of data stored in the display history storage unit can be reduced, and images of objects on the road that have a large influence on judgments during driving of the driver are compared. Can be determined appropriately.

  The vehicle periphery display device according to the present invention includes an actual road width calculation unit that calculates an object width from the image of the object and calculates an actual road width based on the calculated object width and the road width included in the road data. The display history storage unit stores data of an actual road width when a surrounding image has been displayed in the past, and the similarity calculation unit stores the actual road width calculated from the surrounding image and the display history storage. The display control unit may display a peripheral image when the actual road width matches the actual road width stored in the unit.

  With this configuration, it is possible to appropriately determine whether or not to display the surrounding image in consideration of the actual road width that requires different driving assistance depending on the vehicle type and the skill level of the driver.

  A vehicle periphery display device according to the present invention is a device that displays a peripheral image captured by a camera mounted on a vehicle, and acquires vehicle data indicating vehicle status from a sensor mounted on the vehicle. A display history storage unit that associates and stores a peripheral image displayed in the past and vehicle data at the time of display of the peripheral image, and a peripheral image associated with the vehicle data acquired by the vehicle data acquisition unit. When an image search unit that searches from the display history storage unit, a peripheral image generation unit that acquires image data being captured by the camera and generates a peripheral image, and a peripheral image is searched by the image search unit A similarity calculation unit that calculates a similarity between the searched peripheral image and the peripheral image generated by the peripheral image generation unit; and the peripheral image generation when the similarity is equal to or greater than a predetermined threshold value. A peripheral image generated by and a display control unit for displaying on the display.

  Thus, when the similarity between the current peripheral image and the peripheral image displayed in the past stored in the display history storage unit is equal to or greater than a predetermined threshold, the current peripheral image is displayed by the configuration in which the current peripheral image is displayed. In the case of the same scene as the peripheral image displayed in the past, the peripheral image can be displayed. Accordingly, the peripheral image is not displayed unnecessarily simply because the contents of the operation are the same, and the image display can be prevented from being bothered. In addition, when a peripheral image that matches the vehicle data is searched, the calculation of the similarity of the peripheral image is not performed by the configuration that calculates the similarity of the peripheral image so that the vehicle data is different and the possibility that they are not similar is high. it can.

  The vehicle periphery display method of the present invention is a method of displaying a peripheral image captured by a camera mounted on a vehicle, and acquires a peripheral image generated by acquiring image data being captured by the camera. The road data corresponding to the current position of the vehicle is acquired and acquired from the generation step, the current position data acquisition step of acquiring data of the current position of the vehicle, and the map storage unit storing the map data including road data regarding the road. An image search step of searching for a peripheral image associated with road data from a display history storage unit that stores a peripheral image displayed in the past and road data of a point where the peripheral image is displayed in association with each other, and the image search step. A similarity meter that calculates the similarity between the searched peripheral image and the peripheral image generated in the peripheral image generation step when the peripheral image is searched A step, when the similarity is equal to or greater than a predetermined threshold value, and a display control step of displaying the peripheral image generated by said peripheral image generating step on the display.

  The program according to the present invention is a program for displaying a peripheral image captured by a camera mounted on a vehicle, and generates a peripheral image by acquiring image data being captured by the camera in a computer. Obtaining road data corresponding to the current position of the vehicle from the surrounding image generation step, a current position data acquisition step of acquiring data of the current position of the vehicle, and a map storage unit storing map data including road data relating to the road; An image search step of searching for a peripheral image associated with the acquired road data from a display history storage unit that stores the peripheral image displayed in the past and the road data of the point where the peripheral image is displayed in association with each other, and the image search When the peripheral image is searched in the step, the type of the searched peripheral image and the peripheral image generated in the peripheral image generation step A similarity calculation step of calculating a degree, the similarity in the case where more than a predetermined threshold value, to execute a display control step of displaying the peripheral image generated by said peripheral image generating step on the display.

  With this configuration, the vehicle periphery display method and program according to the present invention, like the vehicle periphery display device described above, simply displays the same road data, so that no peripheral image is displayed unnecessarily, and the image display is troublesome. You can avoid it. Various configurations of the vehicle periphery display device of the present invention can be applied to the vehicle periphery display method and program of the present invention.

  The present invention displays the surrounding image in the case where the current surrounding image is the same scene as the surrounding image displayed in the past, so that the surrounding image is not displayed unnecessarily simply because the road data is the same, The image display can be prevented from being bothered.

It is a figure which shows the structure of the vehicle periphery display apparatus of 1st Embodiment. (A) It is a figure which shows the example of the data at the time of the display start of the periphery image memorize | stored in display history DB. (B) It is a figure which shows the example of the data at the time of the display stop of the periphery image memorize | stored in display history DB. It is a figure explaining the calculation process of (a)-(e) image feature-value. (A) It is a figure for demonstrating calculation of a gradient value. (B) It is a figure which shows the example of the coefficient used for calculation of a gradient value. It is a figure which shows the hardware constitutions of a vehicle periphery display apparatus. (A) It is a figure which shows the operation | movement which accumulate | stores data in display history DB of the vehicle periphery display apparatus of 1st Embodiment. (B) It is a figure which shows the detailed operation | movement of display log | history memory. It is a figure which shows the operation | movement which the vehicle periphery display apparatus of 1st Embodiment displays a periphery image. It is a figure which shows the structure of the vehicle periphery display apparatus of 2nd Embodiment. It is a figure which shows the operation | movement in which the vehicle periphery display apparatus of 2nd Embodiment plans a display candidate site. It is a figure which shows the operation | movement which displays a periphery image by the vehicle periphery display apparatus of 2nd Embodiment. It is a figure for demonstrating the change of the number of search keys.

  Hereinafter, the configuration of the vehicle periphery display device according to the embodiment of the present invention will be described. Hereinafter, a case where the vehicle periphery display device is separate from the car navigation device (hereinafter referred to as “car navigation”) will be described as an example. However, the vehicle periphery display device may be incorporated in the car navigation system.

  FIG. 1 is a diagram illustrating a configuration of a vehicle periphery display device 1 according to the first embodiment. A car navigation system 30, a GPS receiver 35, a sensor group 36, a camera group 37, and a display 38 are connected to the vehicle periphery display device 1. As the display 38, the display 33 of the car navigation system 30 may be used.

  The car navigation system 30 includes a route search unit 31, a route guidance unit 32, a display 33, and a map database (hereinafter referred to as “map DB”) 34, similarly to the normal car navigation system 30. The map DB 34 stores road data indicating road types, road link types, and the like. The GPS receiver 35 receives a signal from a GPS satellite and detects the current position of the vehicle. The GPS receiver 35 transmits current position data to the vehicle periphery display device 1. In this embodiment, only the GPS receiver 35 is shown as a configuration for detecting the current position of the vehicle. However, as a sensor for detecting the current position, for example, a geomagnetic sensor for detecting the absolute azimuth of the vehicle. It is also possible to use a gyroscope for detecting the relative direction of the vehicle, a distance sensor for detecting the travel distance of the vehicle, a steering rotation sensor, a vehicle speed sensor for each rolling wheel, and the like. By using these sensors, the detection accuracy of the current position can be increased.

  The sensor group 36 is a variety of sensors mounted on the vehicle and detects the state of the vehicle. The sensor group 36 includes sensors that detect states of speed, gear shift position, blinker, headlight, wiper, handle, brake, air conditioner, window, audio, and the like. The sensor group 36 transmits the detected vehicle data to the vehicle periphery display device 1.

  The camera group 37 is a plurality of cameras mounted on the vehicle in order to photograph the periphery of the vehicle. In the present embodiment, the camera group 37 includes two cameras that respectively capture the left and right sides of the vehicle, a camera that captures the rear of the vehicle, and a camera that captures a blind spot in front of the vehicle. The camera group 37 transmits image data obtained by photographing to the vehicle periphery display device 1. In this embodiment, an example in which a plurality of cameras are used has been described. However, a plurality of cameras are not necessarily required, and for example, only a camera that captures the rear side may be mounted.

  The vehicle periphery display device 1 includes a current position data acquisition unit 10, an image search unit 11, a surrounding image generation unit 12, an image feature amount calculation unit 13, a similarity calculation unit 14, an image display determination unit 15, The display control unit 16 and a display history database (hereinafter referred to as “display history DB”) 17 are provided.

  The peripheral image generation unit 12 acquires image data captured by four cameras, and combines the acquired images to generate a peripheral image. In the present embodiment, “blind corner image”, “top view image”, “rear wide view image”, “back guide image”, “pass-through image”, and “electronic mirror image” are generated as peripheral images.

  The “blind corner image” is an image in front of the right side or front side of the left which is a blind spot by the bumper of the vehicle. It is generated based on the front camera image. The “top view image” is an image obtained by looking down at the vehicle from above. It is generated by combining four camera images. The “rear wide view image” is an image in which the rear of the vehicle is captured at a wide angle. It is generated by combining the rear and two side camera images. The “back guide image” is an image displaying a relatively close rear of the vehicle. It is generated based on the rear camera image. The “pass-through image” is an image that supports slip-through and passing on a narrow road. It is generated based on the two side camera images. The “electronic mirror image” is an image obtained by capturing an image in a direction that can be confirmed by the side mirror at a wider angle than the side mirror. It is generated based on the side camera image. Note that the types of peripheral images shown here are examples, and peripheral images other than those described above may be generated and displayed.

  The current position data acquisition unit 10 acquires current position data from the GPS receiver 35. The current position data acquisition unit 10 passes the current position data to the image search unit 11. The image search unit 11 searches the display history DB 17 for feature amounts of peripheral images displayed in the past.

  FIG. 2A and FIG. 2B are diagrams illustrating examples of data stored in the display history DB 17. FIG. 2A shows the peripheral image and its related data when the display of the peripheral image is started, and FIG. 2B shows the peripheral image and its related data when the display of the peripheral image is stopped. For convenience of explanation, it is described separately in two figures, FIG. 2A and FIG. 2B, but the data shown in FIG. 2A and FIG. 2B are both stored in the display history DB 17. Has been.

  As shown in FIG. 2A, the display history DB 17 stores a position when display of the peripheral image is started, road data, vehicle data, and image data of the peripheral image in association with each other. Further, as shown in FIG. 2B, the position when the display of the peripheral image is stopped, the road data, the vehicle data, and the image data of the peripheral image are also stored in the same manner. In FIG. 2A and FIG. 2B, “No.” corresponds to each other, and display start and display stop data of a peripheral image of the same “No.” is a set. For example, the peripheral image whose display is stopped at “No. 1” shown in FIG. 2B is the peripheral image whose display is started at “No. 1” shown in FIG.

  Next, each data stored in the display history DB 17 will be described. Various data stored in association with the peripheral image data when the display of the peripheral image is started (see FIG. 2A) and various data stored in association with the peripheral image data when the display of the peripheral image is stopped (FIG. 2). (B) is the same. Below, it demonstrates centering on Fig.2 (a). The position is data expressed by latitude and longitude. When the display of the peripheral image is started, the data acquired by the current position data acquisition unit 10 is stored.

  As road data, “type” and “link type” of the road are stored. The “type” of the road is, for example, “national road”, “prefectural road”, “other general roads”, “highway road”, “urban highway”, “parking lot”, and the like. The “link type” of the road is, for example, “intersection”, “ramp”, “rotary”, “tunnel”, or the like. The road data listed here is only an example, and other data that characterizes the road configuration and the like can also be used. For example, “branch”, “branch attribute”, “signal”, “number of lanes”, “width”, “one-way”, “statutory speed”, etc. may be used as road data. The “branch attribute” is data indicating the reason why the road is branched, such as “no branch”, “IC”, “SA, PA”, “JCT”.

  As the vehicle data, data acquired from the sensor group 36 when a peripheral image is displayed is stored. Specifically, the vehicle data is data indicating the states of “speed”, “gear”, “winker”, “headlight”, and “wiper”. The vehicle data listed here is an example, and other vehicle data may be used. When “ON” data is stored in the “headlight”, data indicating “normally lit” or “high beam” is also stored. In “wiper”, data “0” indicates that the wiper is stopped, and data “1-3” indicates the operation speed of the wiper. The larger the value, the faster the wiper is operating.

  As the image data, “direction” data for specifying the direction of the peripheral image and “image feature amount” data of the peripheral image are stored. The “direction” data specifies any of the peripheral images of “blind corner image”, “top view image”, “rear wide view image”, “rear guide image”, “pass-through image”, and “electronic mirror image”. The “image feature amount” is data for identifying an image. By using the image feature amount instead of the image itself, the data amount of the display history DB 17 can be suppressed. In addition, when the vehicle periphery display device 1 has a large-capacity memory, the image itself may be stored.

  Next, functions of the image search unit 11 will be described. When the current peripheral image is similar to the past image displayed in the past, the vehicle peripheral display device 1 according to the present embodiment determines that the scene is similar to the scene in which the driver displayed the peripheral image in the past. And display the current surrounding image. Here, if the current peripheral image is compared with all the peripheral images stored in the display history DB 17, the processing load is large. Therefore, the function of the image search unit 11 is to search for peripheral images that are candidates for comparison.

  The image search unit 11 reads road data corresponding to the current position data passed from the current position data acquisition unit 10 from the map DB 34. Further, the image search unit 11 receives the vehicle data transmitted from the sensor group 36. The image search unit 11 uses the road data read from the map DB 34 and the vehicle data received from the sensor group 36 as search keys, and searches the display history DB 17 for image feature amounts of peripheral images that match the search keys. For example, when the road data is “national road” and “tunnel”, and the vehicle data is “speed: 30 km / h”, “gear: D”, “winker: OFF”, searching for surrounding images using these as search keys , The image feature amount of the image of “No. 1” is searched. In addition, as a search key, all the road data and vehicle data shown to Fig.2 (a) may be used, and a part of road data and vehicle data may be used.

  The image search unit 11 outputs an image feature amount when a peripheral image matching the search key road data and vehicle data is searched. If there is no peripheral image that matches the search key, the image search unit 11 outputs that the peripheral image was not found.

  The similarity calculation unit 14 calculates the similarity between the past peripheral image searched by the image search unit 11 and the current peripheral image. The similarity calculation unit 14 instructs the image feature amount calculation unit 13 to transmit the image feature amount data of the peripheral image in the same direction as the peripheral image searched by the image search unit 11. For example, when “No. 1” data shown in FIG. 2A is searched by the image search unit 11, the similarity calculation unit 14 stores the image feature amount data of the “rear wide view image”. Instruct to send.

  The peripheral image generation unit 12 combines the camera images acquired from the camera group 37 and generates a specified peripheral image. The peripheral image generation unit 12 passes the generated peripheral image to the image feature amount calculation unit 13. The image feature amount calculation unit 13 calculates the image feature amount of the peripheral image.

  FIG. 3A to FIG. 3E are diagrams illustrating the image feature amount calculation processing. As shown in FIG. 3A, the image feature amount calculation unit 13 sets a region A that is a target of feature amount calculation for a peripheral image. For example, by setting the visual field that the driver gazes at the time of driving as the target area A for calculating the feature amount, the calculation accuracy of the feature amount can be increased. Next, the image feature quantity calculation unit 13 sets a plurality of small areas B for the target area A as shown in FIG. In addition, when setting the small area | region B, each small area | region B may overlap partially.

  Subsequently, the image feature amount calculation unit 13 divides the small region B into 16 parts as shown in FIG. The image feature quantity calculation unit 13 obtains gradient values in eight directions in each calculation unit region C as shown in FIG.

  4A and 4B are diagrams for explaining the calculation of the gradient value. As shown in FIG. 4A, the gradient value is calculated by multiplying each pixel value of the calculation unit region C composed of 3 × 3 pixels by a specific coefficient determined by the direction and summing them up. FIG. 4B is a diagram illustrating an example of coefficients used for calculation of the gradient value.

  The image feature amount calculation unit 13 adds the gradient values in the eight directions of all 16 calculation units included in the small region B, and obtains a histogram as shown in FIG. This histogram is a feature amount in the small area B of the image. The above image feature calculation processing is described in detail in the Institute of Electronics, Information and Communication Engineers, Mitsuru Abe, Junichi Yoshida, “Expansion of VisualRank to Multiple Classes: Automatic Classification and Ranking of Similar Images Using Image Features” .

  The similarity calculation unit 14 calculates the similarity between both images based on the past peripheral image searched by the image search unit 11 and the image feature amount of the current peripheral image calculated by the image feature amount calculation unit 13. calculate. The similarity calculation unit 14 compares the feature quantities of the small regions B at the corresponding positions in the comparison target image to obtain the similarity, and sums the comparison results to calculate the overall similarity. The similarity calculation unit 14 passes the similarity value obtained by the calculation to the image display determination unit 15.

  The image display determination unit 15 determines whether the similarity is greater than or equal to a predetermined threshold. When it is determined that the similarity is equal to or greater than a predetermined threshold, the image display determination unit 15 determines to display the current peripheral image, and instructs the display control unit 16 to display the current peripheral image. put out. Upon receiving an instruction from the image display determination unit 15, the display control unit 16 transmits the peripheral image data generated by the peripheral image generation unit 12 to the display 38 and causes the display 38 to display the peripheral image.

  Further, the similarity calculation unit 14 starts displaying the peripheral image (that is, the display of the past peripheral image searched by the image search unit 11 after the display control unit 16 starts displaying the peripheral image (that is, the peripheral image). The similarity between the peripheral image immediately before the image display is stopped) and the current peripheral image is calculated, and the similarity value obtained by the calculation is passed to the image display determination unit 15. The image display determination unit 15 determines whether or not the similarity is equal to or higher than a predetermined threshold, and determines that the display of the surrounding images is stopped when it is determined that the similarity is higher than or equal to the predetermined threshold. When receiving an instruction to stop image display from the image display determination unit 15, the display control unit 16 transmits a control signal instructing the display 38 to stop displaying peripheral images.

  FIG. 5 is a diagram illustrating a hardware configuration of the vehicle periphery display device 1 described above. The vehicle periphery display device 1 is configured by a computer in which an ECU 40, a RAM 41, a ROM 42, a communication interface 44, a hard disk 45, and an operation unit 46 are connected by a data bus 47. The ECU 40 executes the arithmetic processing according to the program 43 written in the ROM 42, thereby realizing the functions of the vehicle periphery display device 1 described above. Such a program 43 is included in the scope of the present invention.

  FIGS. 6A and 6B are diagrams illustrating an operation of accumulating data in the display history DB 17 of the vehicle periphery display device 1 according to the first embodiment. As shown in FIG. 6A, the vehicle periphery display device 1 determines whether or not there is an instruction to display a peripheral image from the driver while the vehicle is traveling (S10). If there is no display instruction (NO in S10), this determination is repeated until there is a display instruction.

  When there is an instruction to display a peripheral image from the driver (YES in S10), the vehicle peripheral display device 1 issues an instruction to the camera group 37 to transmit the image being shot. The peripheral image display instruction includes data indicating in which direction the peripheral image the driver wants to display. For example, eight switches for instructing display of peripheral images are prepared, and depending on which switch is pressed, the “blind corner image”, “top view image”, “rear wide view image”, “back guide image”, “back guide image”, “ It may be possible to identify whether one wants to display a “slip-through image” or a left and right “electronic mirror image”. When receiving the image data being photographed from the camera group 37, the vehicle periphery display device 1 combines the received camera images to generate a periphery image (S12), and displays the generated periphery image on the display 38 (S14). . Further, the vehicle periphery display device 1 stores the image feature amount data of the periphery image when display is started in the display history DB 17 (S16).

  FIG. 6B is a diagram showing a detailed operation of the display history storage (S16). As shown in FIG. 6 (b), the vehicle periphery display device 1 acquires data on the current position of the vehicle when the driver gives an instruction to display a surrounding image in the current position data acquisition unit 10 (S30). . The vehicle periphery display device 1 reads road data corresponding to the current position from the map DB 34 (S32). Moreover, the vehicle periphery display apparatus 1 acquires the vehicle data when the driver gives an instruction to display a surrounding image from the sensor group 36 (S34). The vehicle periphery display device 1 calculates the image feature amount of the surrounding image generated from the camera image when the driver gives an instruction to display the surrounding image (S36). The road data, vehicle data, and image data of surrounding images obtained as described above are associated and stored in the display history DB 17 (S38).

  The vehicle periphery display device 1 determines whether or not there is an instruction to stop displaying the surrounding image from the driver during the display of the surrounding image (S18). If there is no instruction to stop the peripheral image display (NO in S18), this determination is repeated while displaying the peripheral image until there is a stop instruction. If there is an instruction to stop displaying the surrounding image from the driver (YES in S18), the display control unit 16 of the vehicle surrounding display device 1 stops displaying the surrounding image (S20). Moreover, the vehicle periphery display apparatus 1 memorize | stores the data of the periphery image in the display log | history DB17 when the display of a periphery image is stopped, ie, just before stopping a display (S22). The detailed operation for stopping the display of the peripheral image data is the same as the operation described with reference to FIG.

  FIG. 7 is a diagram illustrating an operation in which the vehicle periphery display device 1 according to the first embodiment displays a surrounding image. The vehicle periphery display device 1 acquires current position data from the GPS receiver 35 while the vehicle is traveling (S40). The vehicle periphery display device 1 reads road data corresponding to the current position from the map DB 34 (S42). Moreover, the vehicle periphery display apparatus 1 acquires vehicle data from the sensor group 36 (S44). In addition, the order of acquisition of present position data (S40) and acquisition of vehicle data (S44) is not limited to the order shown in FIG. 7, and may be reversed, for example.

  The vehicle periphery display device 1 searches the display history DB 17 for a past peripheral image associated with the same road data and vehicle data as the acquired road data and vehicle data (S46). If a surrounding image having the same road data and vehicle data is not found (NO in S48), the vehicle surrounding display device 1 returns to the process (S40) of acquiring the current position data without displaying the surrounding image. .

  When a peripheral image having the same road data and vehicle data is found (YES in S48), the vehicle peripheral display device 1 generates image data acquired from the camera group 37 to generate a peripheral image (S50), The feature amount of the generated peripheral image is calculated (S52). Subsequently, the vehicle peripheral display device 1 calculates the similarity between the image feature amount read from the display history DB 17 and the image feature amount of the current peripheral image (S52). When the similarity is less than the predetermined threshold (NO in S54), the vehicle periphery display device 1 returns to the process (S40) of acquiring the current position data without displaying the surrounding image.

  When the degree of similarity is equal to or greater than the predetermined threshold (YES in S54), the vehicle periphery display device 1 starts displaying the peripheral image being captured by the camera group 37 on the display 38 (S56). During the display of the peripheral image, the vehicle peripheral display device 1 determines whether or not the stop condition for stopping the display of the peripheral image is satisfied (S58), and when it is determined that the stop condition is not satisfied (NO in S58). ) And repeatedly determining whether or not the stop condition is satisfied while continuing to display the peripheral image.

  When it is determined that the stop condition is satisfied (YES in S58), the vehicle periphery display device 1 stops the display of the surrounding image (S60). In this embodiment, the peripheral image display stop condition is that the similarity between the current peripheral image and the peripheral image when the display of the peripheral image stored in the display history DB 17 is equal to or greater than a predetermined threshold value. Is the time. Therefore, the vehicle periphery display device 1 compares the image feature amount of the camera image being shot with the image feature amount at the time of display stop (see FIG. 2B) stored in the display history DB 17 to determine the similarity. Calculate to determine whether the similarity is greater than or equal to a predetermined threshold. The configuration and operation of the vehicle periphery display device 1 according to the first embodiment have been described above.

  The vehicle periphery display device 1 according to the first embodiment calculates the similarity between the current surrounding image and the previously displayed surrounding image stored in the display history DB 17, and the similarity is equal to or greater than a predetermined threshold. Since the peripheral image is displayed, the peripheral image can be displayed in the same or similar scene as when the peripheral image was displayed in the past. Thereby, in a scene that the driver is not good at, a peripheral image is automatically displayed, and driving can be appropriately supported. Even if the road data and the vehicle data are the same, if the scenes are different, the peripheral image is not displayed, so the driver is less likely to feel bothered.

  The vehicle periphery display device 1 according to the first embodiment selects past peripheral image candidates to be compared with the current peripheral image based on road data and vehicle data. It is possible to reduce the peripheral image and reduce the processing load for calculating the similarity between the peripheral images.

  The vehicle periphery display device 1 according to the first embodiment calculates the similarity between the peripheral image when the display of the peripheral image was stopped in the past and the current peripheral image, and the periphery when the similarity is equal to or greater than a predetermined threshold. Since the display of the image is stopped, the display of the peripheral image can be stopped at the timing when the driver determines that the display of the peripheral image is not necessary in the past.

  The vehicle periphery display device 1 according to the first embodiment calculates the similarity between the surrounding images generated by the surrounding image generation unit 12. Since the object shown in the peripheral image generated to assist driving is roughly determined, similarity calculation between the peripheral images can be performed with high accuracy.

(Second Embodiment)
FIG. 8 is a diagram showing a configuration of the vehicle periphery display device 2 according to the second embodiment of the present invention. The basic configuration of the vehicle periphery display device 2 of the second embodiment is the same as that of the vehicle periphery display device 1 of the first embodiment. In addition to the configuration of the vehicle periphery display device 1 according to the first embodiment, the vehicle periphery display device 2 according to the second embodiment includes a route data acquisition unit 18, an image display plan unit 19, and a display candidate location database ( (Hereinafter referred to as “display candidate DB”) 20. The route data acquisition unit 18 acquires route data from the departure point to the destination searched by the car navigation system 30. The route data acquisition unit 18 passes the acquired route data to the image display plan unit 19.

  The image display planning unit 19 plans a display candidate site that is a candidate for displaying a peripheral image on the route of the route data received from the car navigation system 30. Specifically, the image display planning unit 19 acquires road data on the route from the map data stored in the map DB 34. The image display planning unit 19 compares the road data on the route with the road data stored in the display history DB 17 and obtains a point having the same road data as a display candidate site for the surrounding image. The image display planning unit 19 stores the obtained display candidate place in the display candidate place DB 20.

  The image search unit 11 basically searches for past peripheral images to be compared with the camera image being shot, as with the image search unit 11 in the first embodiment. However, unlike the first embodiment, the image search unit 11 performs an image search when the vehicle approaches a display candidate location stored in the display candidate location DB 20.

  FIG. 9 is a diagram illustrating an operation in which the vehicle periphery display device 2 according to the second embodiment plans a display candidate site. The car navigation system 30 receives a destination setting from the driver (S70), and searches for a route from the current position to the destination (S72). The car navigation system 30 transmits the searched route data to the vehicle periphery display device 2 (S74). When receiving the route data transmitted from the car navigation system 30 (S76), the vehicle periphery display device 2 reads the road data of the route specified by the received route data from the map DB 34 (S78).

  The vehicle periphery display device 2 compares the road data on the route with the road data stored in the display history DB 17, and determines whether or not the same road data as the road data on the route is stored in the display history DB 17. (S80). If there is the same road data (YES in S80), the point related to the road data is set as a display candidate place and written in the display candidate place DB 20 (S82). The above determination is performed for all road data on the route, and the display candidate sites of the surrounding images on the route are stored in the display candidate site DB 20. The car navigation system 30 starts route guidance when the display candidate site DB 20 is ready (S84).

  FIG. 10 is a diagram illustrating an operation of displaying a peripheral image by the vehicle peripheral display device 2 according to the second embodiment. The vehicle periphery display device 2 according to the second embodiment acquires current position data while the vehicle is traveling (S90), and based on the current position data and display candidate place data stored in the display candidate place DB 20. Then, it is determined whether or not the vehicle has reached the display candidate site (S92). Here, whether or not the display candidate place has been reached is determined based on whether or not the vehicle has entered a predetermined range including the display candidate place (for example, a range having a radius of 100 m centered on the display candidate place). . When the vehicle has not reached the display candidate site (NO in S92), the vehicle periphery display device 2 returns to the process of acquiring current position data (S90). Note that the size of the predetermined range for determining whether or not the candidate display area has been reached may be changed according to the road data. For example, when the road type is “National road”, the radius may be set to 100 m, and when the road type is “Parking”, the radius may be set to 10 m.

  If it is determined that the vehicle has reached the candidate display location (YES in S92), the vehicle periphery display device 2 acquires vehicle data from the sensor group 36 (S94). The vehicle periphery display device 2 reads the peripheral image data and the related data at the display candidate site from the display history DB 17, and determines whether the vehicle data included in the read data matches the vehicle data received from the sensor group 36. Determination is made (S96). If the vehicle data do not match (NO in S96), the vehicle periphery display device 2 returns to the process of acquiring the current position data without displaying the periphery image at the display candidate site (S90).

  When the vehicle data match (YES in S96), the vehicle periphery display device 2 generates a peripheral image from the image data acquired from the camera group 37, and obtains the feature amount of the generated image. Subsequently, the vehicle peripheral display device 2 compares the image feature amount of the current peripheral image with the image feature amount read from the display history DB 17 and calculates the similarity between the two images (S100). If the similarity is less than the predetermined threshold (NO in S102), the vehicle periphery display device 2 returns to the process of acquiring current position data (S90) without displaying the periphery image.

  When the similarity is equal to or greater than the predetermined threshold (YES in S102), the vehicle periphery display device 2 starts displaying the current surrounding image on the display 38 (S104). During the display of the peripheral image, the vehicle peripheral display device 2 determines whether or not the stop condition for stopping the display of the peripheral image is satisfied (S106), and when it is determined that the stop condition is not satisfied (NO in S106) ) And repeatedly determining whether or not the stop condition is satisfied while continuing to display the peripheral image.

  When it is determined that the stop condition is satisfied (YES in S106), the vehicle periphery display device 1 stops the display of the surrounding image (S108). In the present embodiment, the peripheral image display stop condition is when the similarity between the current peripheral image and the peripheral image when the display stored in the display history DB 17 is equal to or greater than a predetermined threshold. . Therefore, the vehicle periphery display device 2 compares the image feature amount of the camera image being photographed with the image feature amount at the time of display stop (see FIG. 2B) stored in the display history DB 17 to determine the similarity. Calculate to determine whether the similarity is greater than or equal to a predetermined threshold. The configuration and operation of the vehicle periphery display device 2 according to the second embodiment have been described above.

  Similar to the first embodiment, the vehicle periphery display device 2 according to the second embodiment can display a surrounding image in a scene that is the same as or similar to the case where the surrounding image is displayed in the past. . In addition, even if the road data and the vehicle data are the same, if the scene is different, the peripheral image is not displayed, so that the driver is less annoyed.

  Further, the vehicle periphery display device 2 according to the second embodiment obtains a display candidate site that is a candidate for displaying a peripheral image based on the route data set by the car navigation system 30, and the vehicle is a display candidate site. Since it is determined whether or not a peripheral image is to be displayed when arriving at, it is possible to reduce the processing load compared to a case where it is always determined whether or not to display a peripheral image.

  The configuration and operation of the vehicle periphery display device and the vehicle periphery display method of the present invention have been described in detail above with reference to the embodiment, but the present invention is not limited to the above-described embodiment.

  In the above-described embodiment, an example in which image search is performed using a part of road data and vehicle data as a search key has been described. However, parameters used as search keys depend on the number of surrounding images associated with the combination of the parameters. May be changed.

  FIG. 11 is a diagram illustrating an example of the display history DB for explaining the change in the number of search keys. When the peripheral image is searched from the display history DB shown in FIG. 11 using “road type”, “speed”, “gear”, and “winker” as search keys, the data of “No. 1” to “No. 3” Since all have the same search key value, three peripheral images “image07” to “image09” are searched as peripheral images. Here, when the processing load for calculating the degree of similarity with each of the three surrounding images is large, “headlight” and “wiper” are added to the search key to search for the surrounding images. In this case, since the values of “headlight” and “wiper” of “No. 1” to “No. 3” are different, any peripheral image of “image07” to “image09” is searched. Therefore, since the vehicle periphery display device only has to calculate the similarity with any one of the surrounding images, the calculation processing load can be reduced. How many surrounding images are searched and how many search keys are increased can be appropriately set according to the processing capability of the ECU constituting the vehicle periphery display device.

  In the above-described embodiment, as shown in FIG. 3A, the image feature amount calculation unit 13 sets a region A that is a target of feature amount calculation for a peripheral image, and calculates a feature amount in the region. Although an example of calculation has been described, the shape of the feature amount calculation target region is not necessarily rectangular. For example, the image feature quantity calculation unit 13 may detect an object on a road and set the detected object as a feature quantity calculation target. As a result, the area for calculating the feature amount can be reduced, and the feature amount data can also be reduced.

  In addition, an actual road width calculation unit that obtains an actual road width that can be actually used as a road based on the detected object on the road and the road width included in the road data may be provided. The actual road width calculation unit obtains the longer one of the distances from both sides of the object to the road edge based on the object width and the position where the object is located. In addition, you may perform the process which excludes a front vehicle among the objects on a road. Whether or not to display the surrounding image based on the actual road width excluding the part that is not likely to actually pass due to obstacles placed on the road by providing such an actual road width calculation unit Can be judged. Store the actual road width data in the display history storage unit, compare the actual road width data when the surrounding image was displayed in the past with the actual road width data calculated from the current surrounding image, It is also possible to display a peripheral image when the two match.

  In the above-described embodiment, an example of searching for a peripheral image in which both road data and vehicle data match from the display history DB 17 has been described. However, a past peripheral image search using either road data or vehicle data. May be performed.

  In the above-described embodiment, an example in which the display of the peripheral image is stopped when the similarity between the peripheral image immediately before the display stored in the display history DB 17 is stopped and the camera image being photographed is equal to or greater than a predetermined threshold value. However, another condition may be used as a condition for stopping the display. For example, the image display may be stopped when the display time of the peripheral image has passed a predetermined threshold, or the image display may be stopped when the vehicle has moved a predetermined distance from the start point of the image display. . The image display may be stopped when the feature amount of the image changes by a predetermined threshold from the image at the start of display. In addition, for example, during a traffic jam, it is assumed that the vehicle does not travel much even if time passes, so the travel distance is set as a stop condition. It is also possible to change the stop condition according to the scene, such as a stop condition.

  In the above embodiment, examples of road data and vehicle data are given in FIGS. 2A and 2B. However, road data and vehicle data are not limited to the examples given in the above embodiment. Road data and vehicle data have a role of narrowing down the peripheral images that are candidates for comparison as a preliminary step to obtain the similarity to the past peripheral images, so that the peripheral images that are related to the drive scene and are likely to be similar Any data can be selected. For example, vehicle direction data and time data may be used as vehicle data. When the peripheral image is displayed during the day, the direction of the sun is obtained from the time, and the direction in which the vehicle is facing the sun is determined by the direction of the vehicle. Therefore, the direction data of the vehicle is related to whether the driver feels dazzling, and is related to the drive scene. Further, “day of the week” data may be used as data other than vehicle data and road data. Since the day of the week may affect the congestion of the road, it is related to the drive scene.

  The present invention has an effect of determining whether or not a scene around a current vehicle is similar to a scene in which a surrounding image has been displayed in the past, and can display the surrounding image at an appropriate timing, in order to support driving It is useful as a vehicle periphery display device.

DESCRIPTION OF SYMBOLS 1, 2 Vehicle periphery display apparatus 10 Present position data acquisition part 11 Image search part 12 Peripheral image generation part 13 Image feature-value calculation part 14 Similarity calculation part 15 Image display determination part 16 Display control part 17 Display history DB
18 Route data acquisition unit 19 Image display plan unit 20 Display candidate site DB
30 Car navigation device 31 Route search unit 32 Route guide unit 33 Display 34 Map DB
35 GPS receiver 36 sensor group 37 camera group 38 display 40 ECU
41 RAM
42 ROM
43 Program 44 Communication I / F
45 Hard disk 46 Operation unit

Claims (11)

A vehicle periphery display device that displays a peripheral image captured by a camera mounted on a vehicle,
A display history storage unit that associates and stores a peripheral image displayed in the past and road data relating to the road at the point where the peripheral image is displayed;
A peripheral image generation unit that acquires image data being captured by the camera and generates a peripheral image;
A current position data acquisition unit for acquiring data of the current position of the vehicle;
An image search unit that acquires road data corresponding to the current position of the vehicle from a map storage unit that stores map data including road data, and searches the display history storage unit for peripheral images associated with the acquired road data;
A similarity calculator that calculates the similarity between the searched peripheral image and the peripheral image generated by the peripheral image generator when the peripheral image is searched by the image search unit;
A display control unit that displays a peripheral image generated by the peripheral image generation unit on a display when the similarity is equal to or greater than a predetermined threshold;
A vehicle periphery display device comprising: A vehicle data acquisition unit that acquires vehicle data indicating a vehicle state from a sensor mounted on the vehicle;
The display history storage unit stores vehicle data at the time of displaying the surrounding image in association with the surrounding image,
2. The vehicle periphery according to claim 1, wherein the image search unit searches the display history storage unit for a peripheral image associated with road data corresponding to a current position of the vehicle and vehicle data acquired by the vehicle data acquisition unit. Display device. A route data acquisition unit that acquires data of a route from the current position to the destination;
The road data on the route is acquired from the map data stored in the map storage unit, the acquired road data is compared with the road data stored in the display history storage unit, and stored in the display history storage unit A display planning unit for obtaining a point having the same road data as the road data obtained from the route as a display candidate site of the peripheral image and storing it in the display candidate site storage unit;
With
The image search unit associates with the road data when it is determined that the vehicle has entered a predetermined range including the display candidate site based on the data of the current position acquired by the current position data acquisition unit. The vehicle periphery display device according to claim 1, wherein the displayed periphery image is searched from the display history storage unit. The display history storage unit stores data of peripheral images when display of peripheral images is stopped,
The similarity calculation unit calculates the similarity between the peripheral image when the display is stopped and the peripheral image generated by the peripheral image generation unit,
The vehicle display device according to any one of claims 1 to 3, wherein the display control unit stops displaying a peripheral image on a display when the similarity is equal to or greater than a predetermined threshold.   The image search unit uses a part of the acquired road data as a search key, searches for a peripheral image associated with the part of the road data, and associates with the part of the road data. 5. The vehicle periphery display device according to claim 1, wherein when a predetermined number or more of images are present, the number of road data serving as search keys is increased to search for surrounding images. In the display history storage unit, direction specifying data for specifying which direction of surrounding image is displayed is stored,
The vehicle periphery display device according to claim 1, wherein a periphery image specified by the direction specifying data is generated using the periphery image generated by the periphery image generation unit. The display history storage unit stores an image of an object on a road detected from a peripheral image displayed in the past,
The said similarity calculation part calculates the similarity of the image of the object on the road detected from the said surrounding image, and the image of the object memorize | stored in the said display log | history memory | storage part. The vehicle periphery display apparatus of description. An actual road width calculation unit that calculates the width of the object from the image of the object and calculates the actual road width based on the calculated object width and the road width included in the road data,
The display history storage unit stores data of actual road widths when surrounding images are displayed in the past,
The similarity calculation unit compares the actual road width calculated from the surrounding image and the actual road width stored in the display history storage unit,
The vehicle display device according to claim 7, wherein the display control unit displays a peripheral image when the actual road width matches. A vehicle periphery display device that displays a peripheral image captured by a camera mounted on a vehicle,
A vehicle data acquisition unit for acquiring vehicle data indicating a state of the vehicle from a sensor mounted on the vehicle;
A display history storage unit that associates and stores a peripheral image displayed in the past and vehicle data at the time of display of the peripheral image;
An image search unit that searches the display history storage unit for a peripheral image associated with the vehicle data acquired by the vehicle data acquisition unit;
A peripheral image generation unit that acquires image data being captured by the camera and generates a peripheral image;
A similarity calculator that calculates the similarity between the searched peripheral image and the peripheral image generated by the peripheral image generator when the peripheral image is searched by the image search unit;
A display control unit that displays a peripheral image generated by the peripheral image generation unit on a display when the similarity is equal to or greater than a predetermined threshold;
A vehicle periphery display device comprising: A vehicle periphery display method for displaying a periphery image captured by a camera mounted on a vehicle,
A peripheral image generation step of acquiring image data being captured by the camera and generating a peripheral image;
A current position data acquisition step for acquiring data of the current position of the vehicle;
The road image corresponding to the current position of the vehicle is acquired from the map storage unit storing the map data including road data related to the road, and the peripheral image associated with the acquired road data is displayed in the past and the peripheral image An image search step for searching from a display history storage unit that stores the road data of the point displaying the
When a peripheral image is searched in the image search step, a similarity calculation step for calculating the similarity between the searched peripheral image and the peripheral image generated in the peripheral image generation step;
A display control step of displaying the peripheral image generated in the peripheral image generation step on a display when the similarity is equal to or greater than a predetermined threshold;
A vehicle periphery display method comprising: A program for displaying a peripheral image taken by a camera mounted on a vehicle,
A peripheral image generation step of acquiring image data being captured by the camera and generating a peripheral image;
A current position data acquisition step for acquiring data of the current position of the vehicle;
The road image corresponding to the current position of the vehicle is acquired from the map storage unit storing the map data including road data related to the road, and the peripheral image associated with the acquired road data is displayed in the past and the peripheral image An image search step for searching from a display history storage unit that stores the road data of the point displaying the
When a peripheral image is searched in the image search step, a similarity calculation step for calculating the similarity between the searched peripheral image and the peripheral image generated in the peripheral image generation step;
A display control step of displaying the peripheral image generated in the peripheral image generation step on a display when the similarity is equal to or greater than a predetermined threshold;
A program that executes