JP2012025327A - Image display system, image processing apparatus, and image display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique to show to a user situations of a plurality of peripheral areas comparatively largely within a limited screen.SOLUTION: With an image display system, a plurality of peripheral areas to be shown around the same time to a user are selected by an area selection part out of peripheral areas of a vehicle as attention areas. Then the plurality of vehicle peripheral images which show the plurality of peripheral areas selected as attention areas are switched one by one and are displayed in the same display area PA in the screen D1 of a display. Consequently size of each vehicle peripheral image to be displayed can be enlarged comparatively, since there is no need to simultaneously display the plurality of these vehicle peripheral images side by side in the screen of the display. As a result, situations of each peripheral area can be comparatively largely shown to the user in a limited display screen.

Description

  The present invention relates to a technique for displaying an image in a vehicle.

  2. Description of the Related Art Conventionally, there is known an image display system that is mounted on a vehicle such as an automobile and displays an image obtained by photographing the periphery of the vehicle with an in-vehicle camera on a display in the vehicle interior. By using this image display system, a user (typically a driver) can grasp the situation around the vehicle almost in real time.

  For example, the outer region of the front fender that is on the opposite side of the driver's seat is likely to be a blind spot from the driver's seat, and it is difficult for the user to grasp the clearance between the vehicle body and the obstacle. On the other hand, if the image display system is used, an image showing the outer region of the front fender is acquired by photographing with the in-vehicle camera, and the image is displayed on the display in the vehicle interior. As a result, the user can easily confirm the clearance between the vehicle body on the opposite side of the driver's seat and the obstacle, for example, when the vehicle is brought close.

  There has also been proposed an image display system capable of enlarging and displaying a part of the peripheral area around the vehicle using a plurality of captured images obtained by a plurality of in-vehicle cameras arranged at different positions of the vehicle. (For example, refer to Patent Document 1). By using such an image display system, the user can confirm in detail the surrounding area of the vehicle to be noted.

JP 2009-93485 A

  By the way, in the technique of the above-mentioned Patent Document 1, there is only one peripheral area that can be enlarged and displayed at the same time. However, during the actual traveling of the vehicle, there is a scene where the user should confirm the situation of the plurality of surrounding areas at the same time. For example, there are obstacles in both the front peripheral area and the rear peripheral area of the vehicle.

  In order to cope with this, for example, it is conceivable that a plurality of images respectively indicating a plurality of peripheral regions to be confirmed by the user are arranged in the screen of the display device and displayed simultaneously. However, in this case, the display size of each of the plurality of images is reduced, and it may be difficult for the user to confirm the state of the surrounding area in detail.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a technique capable of showing a user a relatively large state of a plurality of peripheral areas within a limited screen.

  In order to solve the above-described problem, the invention of claim 1 is an image display system mounted on a vehicle, wherein a peripheral region of a part of the periphery of the vehicle is obtained from a captured image obtained by a camera disposed on the vehicle. Generating means for generating a vehicle peripheral image showing the vehicle, selection means for selecting a plurality of peripheral areas to be shown to the user of the vehicle at the same time, and a plurality of vehicle peripheral images showing each of the plurality of peripheral areas Display means for switching and displaying in the same display area in the screen.

  According to a second aspect of the present invention, in the image display system according to the first aspect, the display means repeatedly displays the plurality of vehicle surrounding images.

  According to a third aspect of the present invention, in the image display system according to the first or second aspect, the generating means indicates the periphery of the vehicle indicating the peripheral area viewed from an arbitrary viewpoint from captured images obtained by a plurality of cameras. An image can be generated.

  According to a fourth aspect of the present invention, in the image display system according to the third aspect, when the display unit switches the vehicle peripheral image, the display unit after switching from the viewpoint position of the vehicle peripheral image before switching. An animation expression in which the position of the viewpoint moves to the position of the viewpoint of the vehicle peripheral image is performed.

  Further, in the image display system according to claim 4, when the position of the viewpoint moves, the moving speed of the viewpoint is set to be either the starting point or the ending point of the movement. The closer the position is, the slower the position of the viewpoint is, and the faster the position is away from the starting and end points of the movement.

  According to a sixth aspect of the present invention, in the image display system according to any of the first to fifth aspects, the plurality of vehicle peripheral images have the same angle of view.

  The invention according to claim 7 is the image display system according to any one of claims 1 to 6, further comprising acquisition means for acquiring a detection result obtained by detecting an object existing around the vehicle, wherein the selection is performed. The means selects the plurality of peripheral regions based on the position of the object indicated by the detection result.

  The invention according to claim 8 is the image display system according to claim 7, wherein the detection result includes a distance of the object from the vehicle, and the plurality of vehicle peripheral images of each of the plurality of peripheral regions. Setting means for setting a display priority based on the distance of the object.

  According to a ninth aspect of the present invention, in the image display system according to the eighth aspect, the setting means increases the priority of the peripheral area as the distance of the object existing in the peripheral area is shorter.

  The invention according to claim 10 is the image display system according to any one of claims 1 to 6, wherein the discriminating means for discriminating the type of an object existing around the vehicle, and each of the plurality of surrounding areas Setting means for setting a priority in displaying a plurality of vehicle periphery images based on the type of the object.

  The invention according to claim 11 is the image display system according to any one of claims 1 to 6, further comprising switching means for switching a plurality of display modes suitable for each of a plurality of driving states of the vehicle, wherein the selection is performed. The means selects the plurality of peripheral regions according to the current display mode.

  According to a twelfth aspect of the present invention, there is provided an image processing device for generating an image to be displayed on a display device mounted on a vehicle, wherein one of the surroundings of the vehicle is obtained from a photographed image obtained by a camera disposed on the vehicle. Generating means for generating a vehicle peripheral image indicating a peripheral area of the unit, selecting means for selecting a plurality of peripheral areas to be shown to the user of the vehicle at the same time, and a plurality of vehicle peripheral images indicating each of the plurality of peripheral areas Are output to the display device, and are sequentially switched and displayed on the same display area in the screen of the display device.

  The invention according to claim 13 is an image display method for displaying an image in a vehicle, the vehicle periphery showing a part of the peripheral region around the vehicle from a photographed image obtained by a camera arranged in the vehicle The step of generating an image, the step of selecting a plurality of peripheral regions to be shown to the user of the vehicle at the same time, and the plurality of vehicle peripheral images showing each of the plurality of peripheral regions are sequentially switched to be the same in the screen And displaying in the display area.

  According to the first to thirteenth aspects, a plurality of vehicle peripheral images showing a plurality of peripheral regions to be shown to the user at the same time are sequentially switched and displayed in the same display region on the screen. Thereby, the state of a plurality of peripheral areas can be shown relatively large to the user within a limited screen.

  In particular, according to the invention of claim 2, the user can grasp the change in the state of the plurality of peripheral regions.

  In particular, according to the invention of claim 3, it is possible to show the user a peripheral region viewed from a viewpoint that is easy to grasp.

  In particular, according to the invention of claim 4, the vehicle after the switching is performed in order to perform animation expression in which the position of the viewpoint moves from the position of the viewpoint of the vehicle peripheral image before the switching to the position of the viewpoint of the vehicle peripheral image after the switching. The user can intuitively grasp which peripheral area the peripheral image indicates.

  In particular, according to the fifth aspect of the present invention, the user can easily grasp the starting and ending positions of the viewpoint movement, and the user's attention can be directed to the peripheral area to be shown to the user.

  In particular, according to the sixth aspect of the invention, since the angle of view of the plurality of vehicle peripheral images displayed in the same display area is the same, the user can determine the size of the peripheral area indicated by the vehicle peripheral image after switching. Intuitive grasp.

  Further, according to the seventh aspect of the present invention, since the peripheral area is selected based on the positions of the objects existing around the vehicle, it is possible to indicate to the user the state of the object that the user should be aware of.

  In addition, according to the invention of claim 8 in particular, since the priority is set based on the distance of the object, it is possible to preferentially display the vehicle peripheral image indicating the object that the user should be aware of.

  In particular, according to the ninth aspect of the present invention, it is possible to preferentially display a vehicle peripheral image showing an object close to the vehicle that the user should be aware of.

  In particular, according to the tenth aspect of the present invention, it is possible to preferentially display a vehicle peripheral image indicating a type of object that the user should be aware of.

  In particular, according to the invention of claim 11, it is possible to indicate to the user a peripheral area suitable for each of a plurality of traveling states.

FIG. 1 is a diagram illustrating a configuration of an image display system according to the first embodiment. FIG. 2 is a diagram illustrating a position where the in-vehicle camera is arranged in the vehicle. FIG. 3 is a diagram mainly showing the configuration of the sonar system. FIG. 4 is a diagram showing a position where the clearance sonar is arranged in the vehicle. FIG. 5 is a diagram for explaining a method of generating a composite image. FIG. 6 is a diagram showing a screen of a display that displays the whole overhead view image. FIG. 7 is a diagram illustrating an example of the movement of the vehicle when the rear parking is performed. FIG. 8 is a diagram illustrating an example of the movement of the vehicle when performing parallel parking. FIG. 9 is a diagram showing a screen of a display that displays two vehicle peripheral images. FIG. 10 is a diagram showing a screen of a display that displays three vehicle peripheral images. FIG. 11 is a diagram illustrating an operation flow in the peripheral display mode according to the first embodiment. FIG. 12 is a diagram showing a detailed flow of attention area display processing. FIG. 13 is a diagram illustrating a state of the vehicle when parallel parking is performed. FIG. 14 is a diagram illustrating an image displayed when the vehicle periphery image is switched. FIG. 15 is a diagram illustrating changes in the speed coefficient. FIG. 16 is a diagram illustrating a configuration of an image display system according to the second embodiment. FIG. 17 is a diagram illustrating a state in which two objects exist around the vehicle. FIG. 18 is a diagram illustrating a flow of processing for setting priorities according to the second embodiment. FIG. 19 is a diagram illustrating a configuration of an image display system according to the third embodiment. FIG. 20 is a diagram illustrating transition of display modes of the image display system. FIG. 21 is a diagram illustrating an operation flow in the peripheral display mode according to the third embodiment. FIG. 22 is a diagram illustrating a peripheral area selected in the rear parking mode. FIG. 23 is a diagram illustrating a peripheral area selected in the parallel parking mode. FIG. 24 is a diagram illustrating a peripheral region selected in the passing mode. FIG. 25 is a diagram showing a display screen in the passing mode. FIG. 26 is a diagram illustrating an example in which an overall overhead view image and a vehicle peripheral image are displayed simultaneously.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1. First Embodiment>
<1-1. System configuration>
FIG. 1 is a block diagram illustrating a configuration of an image display system 120 according to the first embodiment. This image display system 120 is mounted on a vehicle (in this embodiment, an automobile), and has a function of photographing the periphery of the vehicle, generating an image, and displaying the image in the passenger compartment. By using this image display system 120, a user (typically a driver) of the image display system 120 can grasp the state around the vehicle in almost real time.

  As shown in FIG. 1, the image display system 120 mainly includes an image processing device 100 that generates an image showing a state around a vehicle, and a navigation device 20 that displays various types of information to a user who gets on the vehicle. I have. An image generated by the image processing apparatus 100 is displayed on the navigation apparatus 20.

  The navigation device 20 provides navigation guidance to the user, and controls the display 21 such as a liquid crystal provided with a touch panel function, an operation unit 22 including a hard switch that is operated by the user, and the entire device. And a control unit 23. The navigation device 20 is installed on an instrument panel or the like of the vehicle so that the screen of the display 21 is visible from the user. Various user instructions are received by the operation unit 22 and the display 21 as a touch panel. The control unit 23 is configured by a computer including a CPU, a RAM, a ROM, and the like, and various functions including a navigation function are realized by the CPU performing arithmetic processing according to a predetermined program.

  The navigation apparatus 20 is communicably connected to the image processing apparatus 100, and can transmit and receive various control signals to and from the image processing apparatus 100 and receive images generated by the image processing apparatus 100. An image based on the function of the navigation device 20 alone is normally displayed on the display 21 under the control of the control unit 23, but the state around the vehicle generated by the image processing device 100 by changing the operation mode is displayed. The image shown is displayed. Thereby, the navigation device 20 also functions as a display device that receives and displays the image generated by the image processing device 100.

  The image processing apparatus 100 is configured as an ECU (Electronic Control Unit) whose main body 10 has a function of generating an image, and is arranged at a predetermined position of the vehicle. The image processing apparatus 100 includes an imaging unit 5 that captures the periphery of the vehicle. The image generation unit 5 generates a composite image viewed from a virtual viewpoint based on a captured image obtained by capturing the periphery of the vehicle. Functions as a device.

  The photographing unit 5 is electrically connected to the main body unit 10 and operates based on a signal from the main body unit 10. The photographing unit 5 includes a front camera 51, a back camera 52, and a side camera 53 that are in-vehicle cameras. Each of the in-vehicle cameras 51 to 53 includes a lens and an image sensor and electronically acquires an image.

  The plurality of in-vehicle cameras 51 to 53 are respectively arranged at different positions of the vehicle. FIG. 2 is a diagram illustrating positions where the in-vehicle cameras 51 to 53 are arranged on the vehicle 9.

  As shown in FIG. 2, the front camera 51 is provided in the vicinity of the license plate mounting position at the front end of the vehicle 9, and its optical axis 51 a is directed in the straight traveling direction of the vehicle 9. The back camera 52 is provided in the vicinity of the license plate mounting position at the rear end of the vehicle 9, and its optical axis 52 a is directed in the direction opposite to the straight traveling direction of the vehicle 9. The mounting position of the front camera 51 and the back camera 52 is preferably at the substantially right and left center, but may be slightly shifted from the left and right center in the left and right direction. The side cameras 53 are provided on the left and right door mirrors 93, respectively, and the optical axis 53a thereof is directed to the outside of the vehicle 9 along the left-right direction of the vehicle 9 (direction orthogonal to the straight traveling direction).

  A fish-eye lens or the like is adopted as the lens of these in-vehicle cameras 51 to 53, and the in-vehicle cameras 51 to 53 have an angle of view θ of 180 degrees or more. For this reason, the surroundings of the vehicle 9 can be image | photographed by utilizing the four vehicle-mounted cameras 51-53.

  Returning to FIG. 1, the main body unit 10 of the image processing apparatus 100 includes a control unit 1 that controls the entire apparatus, an image generation unit 3 that processes a captured image acquired by the imaging unit 5 and generates an image for display. The navigation communication unit 42 that communicates with the navigation device 20 is mainly provided.

  Various instructions from the user received by the operation unit 22 or the display 21 of the navigation device 20 are received by the navigation communication unit 42 as control signals and input to the control unit 1. As a result, the image processing apparatus 100 can operate in response to a user operation on the navigation apparatus 20.

  The image generation unit 3 is configured as a hardware circuit capable of various image processing, and includes a captured image adjustment unit 31 and a composite image generation unit 32 as main functions.

  The captured image adjustment unit 31 performs adjustment for the captured image acquired by the imaging unit 5. Specifically, the captured image adjustment unit 31 performs image processing such as brightness correction on the captured image.

  The composite image generation unit 32 is a vehicle viewed from an arbitrary virtual viewpoint around the vehicle based on the plurality of captured images acquired by the plurality of in-vehicle cameras 51 to 53 of the imaging unit 5 and adjusted by the captured image adjustment unit 31. A composite image showing the state of the surrounding area is generated. A method by which the composite image generation unit 32 generates a composite image will be described later.

  The composite image generated by the composite image generation unit 32 is output to the navigation device 20 by the navigation communication unit 42. Thereby, a composite image showing at least a part of the area around the vehicle is displayed on the display 21 of the navigation device 20.

  The control unit 1 is configured as a computer including a CPU, a RAM, a ROM, and the like, and various control functions are realized by the CPU performing arithmetic processing according to a predetermined program. The image control unit 11, the display control unit 12, the region selection unit 13, and the priority setting unit 14 shown in the drawing show some of the functions of the control unit 1 realized in this way.

  The image control unit 11 performs control related to image processing executed by the image generation unit 3. For example, the image control unit 11 instructs various parameters necessary for generating a composite image generated by the composite image generation unit 32.

  The display control unit 12 performs control related to display contents displayed on the display 21 of the navigation device 20. Moreover, the area | region selection part 13 selects the surrounding area which should be shown to a user from the several surrounding areas set around the vehicle. Moreover, when there are a plurality of peripheral areas to be shown to the user, the priority setting unit 14 sets those priorities. Specific processing contents of the display control unit 12, the region selection unit 13, and the priority setting unit 14 will be described later.

  The main body 10 of the image processing apparatus 100 further includes a nonvolatile memory 40, a card reading unit 44, and a signal input unit 41, which are connected to the control unit 1.

  The nonvolatile memory 40 is configured by a flash memory or the like that can maintain stored contents even when the power is turned off. The nonvolatile memory 40 stores viewpoint data 4a. The viewpoint data 4a is used when determining the virtual viewpoint of the composite image.

  The card reading unit 44 reads a memory card MC that is a portable recording medium. The card reading unit 44 includes a card slot in which the memory card MC can be attached and detached, and reads data recorded on the memory card MC installed in the card slot. Data read by the card reading unit 44 is input to the control unit 1.

  The memory card MC is configured by a flash memory or the like capable of storing various data, and the image processing apparatus 100 can use various data stored in the memory card MC. For example, a program (firmware) that realizes the function of the control unit 1 can be updated by storing a program in the memory card MC and reading the program.

  Moreover, the signal input part 41 inputs the signal from the various apparatuses provided in the vehicle. A signal from the outside of the image display system 120 is input to the control unit 1 via the signal input unit 41. In the present embodiment, a signal from the sonar system 7 that detects an object existing around the vehicle is input to the control unit 1 via the signal input unit 41.

<1-2. Sonar system>
FIG. 3 is a diagram mainly showing the configuration of the sonar system 7. As shown in the figure, the sonar system 7 includes a sonar control unit 70 that controls the entire system, a plurality of clearance sonars 72, and a buzzer 71 that emits a warning sound in the passenger compartment.

  The clearance sonar 72 detects an object existing around the vehicle by emitting an ultrasonic wave and receiving a reflected wave reflected by the object. The clearance sonar 72 can detect the distance of the object based on the time from when the ultrasonic wave is emitted until it returns. The detection result of each clearance sonar 72 is input to the sonar control unit 70, and a warning sound is output from the buzzer 71 according to the distance to the object. Thereby, the user can grasp that an object exists around the vehicle.

  FIG. 4 is a diagram illustrating positions where a plurality of clearance sonars 72 are arranged on the vehicle 9. The plurality of clearance sonars 72 are respectively provided at the left and right end portions in front of the vehicle 9 and the left and right end portions in the rear of the vehicle 9.

  Each clearance sonar 72 transmits an ultrasonic wave toward some peripheral areas A1 to A4 around the vehicle 9. Specifically, the clearance sonar 72 provided at the front left end of the vehicle 9 transmits ultrasonic waves toward a peripheral area (hereinafter referred to as “front left area”) A1 set on the front left side of the vehicle 9. To do. Further, the clearance sonar 72 provided at the right end portion in front of the vehicle 9 transmits an ultrasonic wave toward a peripheral area (hereinafter referred to as “front right area”) A <b> 2 set on the front right side of the vehicle 9. In addition, a clearance sonar 72 provided at the rear left end of the vehicle 9 transmits an ultrasonic wave toward a peripheral area (hereinafter referred to as “rear left area”) A3 set on the rear left side of the vehicle 9. Further, the clearance sonar 72 provided at the right end of the rear of the vehicle 9 transmits an ultrasonic wave toward a peripheral area (hereinafter referred to as “rear right area”) A4 set on the rear right side of the vehicle 9.

  These four peripheral areas A1 to A4 are fixed relative to the vehicle 9 and set in advance. By arranging the clearance sonar 72 as described above, the sonar system 7 can detect objects existing in the four peripheral areas A1 to A4. The sonar system 7 can grasp which of the four peripheral areas A1 to A4 is the position of the detected object based on the position of the clearance sonar 72 that has detected the object. Further, the sonar system 7 can grasp the distance of the detected object.

  The detection result of the sonar system 7 including the position of the object and the distance of the object is input from the sonar control unit 70 to the control unit 1 of the image processing apparatus 100 via the signal input unit 41 as shown in FIG. Is done. The detection result of the sonar system 7 is used for the area selection unit 13 of the control unit 1.

<1-3. Image composition processing>
Next, a method in which the composite image generation unit 32 of the image generation unit 3 generates a composite image that shows how the periphery of the vehicle 9 is viewed from an arbitrary virtual viewpoint based on a plurality of captured images obtained by the imaging unit 5. explain. FIG. 5 is a diagram for explaining a method of generating a composite image.

  When the front camera 51, the back camera 52, and the side camera 53 of the photographing unit 5 are simultaneously photographed, four photographed images P1 to P4 that respectively indicate the front, rear, left side, and right side of the vehicle 9 are acquired. The That is, the four captured images P1 to P4 acquired by the imaging unit 5 include information indicating the entire periphery of the vehicle 9 at the time of shooting.

  Next, each pixel of the four captured images P1 to P4 is projected onto a three-dimensional curved surface SP in a virtual three-dimensional space. The three-dimensional curved surface SP has, for example, a substantially hemispherical shape (a bowl shape), and a center portion (a bottom portion of the bowl) is determined as a position where the vehicle 9 exists. The correspondence between the positions of the pixels included in the captured images P1 to P4 and the positions of the pixels of the solid curved surface SP is determined in advance. For this reason, the value of each pixel of the solid curved surface SP can be determined based on this correspondence and the value of each pixel included in the captured images P1 to P4.

  The correspondence between the positions of the pixels of the captured images P1 to P4 and the positions of the pixels of the three-dimensional curved surface SP is the arrangement of the four in-vehicle cameras 51 to 53 in the vehicle 9 (inter-distance, ground height, optical axis angle, etc. ). Table data indicating the correspondence is stored in advance in the nonvolatile memory 40.

  Further, data indicating the shape and size of the vehicle body stored in advance in the nonvolatile memory 40 is used, and a polygon model indicating the three-dimensional shape of the vehicle 9 is virtually configured. The configured model of the vehicle 9 is arranged in a substantially hemispherical central portion determined as the position of the vehicle 9 in the three-dimensional space in which the three-dimensional curved surface SP is set.

  Further, the virtual viewpoint VP is set by the control unit 1 for the three-dimensional space where the solid curved surface SP exists. The virtual viewpoint VP is defined by the viewpoint position and the visual field direction, and is set to an arbitrary visual field position corresponding to the periphery of the vehicle 9 in this three-dimensional space toward an arbitrary visual field direction.

  Then, according to the set virtual viewpoint VP, a necessary area on the three-dimensional curved surface SP is cut out as an image. The relationship between the virtual viewpoint VP and a necessary area in the three-dimensional curved surface SP is determined in advance, and is stored in advance in the nonvolatile memory 40 or the like as table data. On the other hand, rendering is performed on a polygon model according to the set virtual viewpoint VP, and the resulting two-dimensional vehicle image is superimposed on the cut-out image. Thereby, the composite image which shows a mode that the vehicle 9 and the periphery of the vehicle 9 were seen from arbitrary virtual viewpoints VP will be produced | generated.

  For example, when the virtual viewpoint VPa is set such that the viewpoint position is a position just above the center of the vehicle 9 and the viewing direction is a direction immediately below, the vehicle 9 and the vehicle 9 A composite image CPa that indicates an area around the vehicle 9 is generated. Also, as shown in the figure, when a virtual viewpoint VPb in which the viewpoint position is the left rear of the position of the vehicle 9 and the visual field direction is substantially in front of the vehicle 9 is set, the entire periphery from the left rear of the vehicle 9 is set. As seen, a composite image CPb indicating the vehicle 9 and the area around the vehicle 9 is generated.

  In the case of actually generating a composite image, it is not necessary to determine the values of all the pixels of the three-dimensional curved surface SP, and only the values of the pixels in the area necessary corresponding to the set virtual viewpoint VP are photographed. By determining based on the images P1 to P4, the processing speed can be improved.

  In the image display system 120, by using such a function of the composite image generation unit 32, only the composite image showing the vehicle 9 and the entire periphery of the vehicle 9 as an overhead view, or only a part of the peripheral region around the vehicle 9. Can be generated and displayed on the display 21.

<1-4. Peripheral display mode>
The image display system 120 has two operation modes, a navigation mode and a peripheral display mode. These operation modes are switched by the control of the control unit 1 based on a user operation via the operation unit 22.

  The navigation mode is an operation mode in which a map image for navigation guidance is displayed on the display 21 by the function of the navigation device 20. On the other hand, the peripheral display mode is an operation mode in which a composite image is displayed on the display 21 using the function of the image processing apparatus 100. In the peripheral display mode, the state of the periphery of the vehicle 9 is shown to the user almost in real time by displaying the composite image.

  In the peripheral display mode, normally, as shown in FIG. 6, a composite image (hereinafter referred to as an “overall bird's-eye view image”) BP showing a bird's-eye view of the vehicle 9 and the entire periphery of the vehicle 9 is displayed on the display 21. However, under specific conditions, one or more of the four peripheral areas A1 to A4 in FIG. 4 are selected, and a composite image showing only the selected peripheral area (hereinafter referred to as “vehicle peripheral image”). ) Is displayed on the display 21.

  The viewpoint position of the virtual viewpoint of the vehicle peripheral image indicating such a peripheral region is set substantially directly above the center of the peripheral region, and the visual field direction is directed substantially downward. Thereby, in the vehicle peripheral image, the state of the peripheral region is shown as if looking down from above. In each vehicle peripheral image, a part of the vehicle body of the vehicle 9 is shown. The viewpoint position of the virtual viewpoint corresponding to each of the peripheral areas A1 to A4 is indicated by the viewpoint data 4a stored in the nonvolatile memory 40.

  As described above, the surrounding area to be shown to the user as the vehicle surrounding image changes at any time according to the traveling state of the vehicle 9. Further, depending on the traveling state of the vehicle 9, there may be a plurality of peripheral areas to be shown to the user at the same time. Hereinafter, the surrounding area which should be shown to a user is demonstrated to the example when performing back parking and performing parallel parking. Hereinafter, the peripheral area to be shown to the user is referred to as “attention area”.

  FIG. 7 is a diagram illustrating an example of the movement of the vehicle 9 when the rear parking is performed. In the example of FIG. 7, the state of the vehicle 9 changes from the first state ST11 to the third state ST13 through the second state ST12.

  In the first state ST <b> 11, the rear left end portion of the vehicle 9 is close to another vehicle 99. For this reason, in this case, the rear left area A3 is the attention area.

  In the second state ST <b> 12, the left end portion behind the vehicle 9 and the right end portion behind the vehicle 9 are close to the other vehicle 99. Therefore, in this case, the rear left area A3 and the rear right area A4 are the attention areas.

  Further, in the third state ST13, the left end portion behind the vehicle 9 and the right end portion behind the vehicle 9 are close to other vehicles 99 and the walls of the parking space. For this reason, also in this case, the rear left region A3 and the rear right region A4 are attention regions.

  Thus, in the case where the rear parking is performed, the attention area changes depending on the state of the vehicle 9. In the second state ST12 and the third state ST13, there are two regions of interest at the same time.

  FIG. 8 is a diagram illustrating an example of the movement of the vehicle 9 when performing parallel parking. In the example of FIG. 8, the state of the vehicle 9 changes from the first state ST21 to the second state ST22 and finally to the third state ST23.

  In the first state ST21, the rear left end of the vehicle 9 is close to another vehicle 99. For this reason, in this case, the rear left area A3 is the attention area.

  Moreover, in 2nd state ST22, the left end part of the front of the vehicle 9 and the left end part of the back of the vehicle 9 are adjoining to the wall of the other vehicle 99 or a parking space. For this reason, in this case, the front left region A1 and the rear left region A3 are attention regions.

  Further, in the third state ST23, the left end portion in front of the vehicle 9, the left end portion in the rear of the vehicle 9, and the right end portion in the rear of the vehicle 9 are close to other vehicles 99 and the walls of the parking space. For this reason, in this case, the front left region A1, the rear left region A3, and the rear right region A4 are attention regions.

  Even when parallel parking is performed in this manner, the attention area changes depending on the state of the vehicle 9. In the second state ST22, there are two regions of interest at the same time, and in the third state ST13, there are three regions of interest at the same time.

  Such a region of interest is selected by the region selector 13. Based on the position of the object indicated by the detection result of the sonar system 7, the area selection unit 13 selects a peripheral area where an object (including another vehicle 99 or a parking space wall) is present as the attention area. Thus, an appropriate peripheral area is selected as the attention area according to the traveling state of the vehicle. The area selection unit 13 can select a plurality of peripheral areas to be shown to the user of the vehicle at the same time as the attention area.

  When a plurality of peripheral areas are selected as the attention area, a plurality of vehicle peripheral images indicating each of the plurality of peripheral areas are sequentially displayed on the display 21 under the control of the display control unit 12.

  FIG. 9 is a diagram showing an example of a screen displayed on the display 21 in the second state ST22 shown in FIG. In the second state ST22, two peripheral regions, the front left region A1 and the rear left region A3, are selected by the region selection unit 13 as attention regions. For this reason, on the screen D1 of the display 21, the vehicle peripheral image BP1 indicating the front left region A1 and the vehicle peripheral image BP3 indicating the rear left region A3 are repeatedly displayed while being alternately switched. The vehicle surrounding image BP1 and the vehicle surrounding image BP3 are displayed in the same display area PA of the screen D1. Further, the field angles of the vehicle peripheral image BP1 and the vehicle peripheral image BP3 are the same.

  FIG. 10 is a diagram showing an example of a screen displayed on the display 21 in the third state ST23 shown in FIG. In the third state ST23, the three peripheral regions of the front left region A1, the rear left region A3, and the rear right region A4 are selected by the region selection unit 13 as attention regions. Therefore, on the screen D1 of the display 21, a vehicle peripheral image BP1 showing the front left region A1, a vehicle peripheral image BP3 showing the rear left region A3, and a vehicle peripheral image BP4 showing the rear right region A4 are sequentially switched. Repeatedly displayed. These three vehicle peripheral images BP1, BP3, and BP4 are displayed in the same display area PA of the screen D1. Further, the three vehicle peripheral images BP1, BP3, and BP4 have the same angle of view.

  As described above, in the image display system 120, a plurality of vehicle peripheral images indicating the plurality of peripheral regions selected as the attention region are sequentially switched and displayed on the same display region PA in the screen D1 of the display 21. For this reason, since it is not necessary to display these several vehicle periphery images side by side simultaneously on the screen of the display 21, the size of each vehicle periphery image to display can be made comparatively large. As a result, the state of each peripheral area can be shown to the user relatively large in the limited screen of the display 21.

  In addition, since a plurality of vehicle peripheral images indicating the plurality of peripheral areas selected as the attention area are repeatedly displayed on the display 21, the user can appropriately grasp the change in the state of each peripheral area. is there.

  The angle of view of the plurality of vehicle periphery images to be switched is the same. Therefore, the peripheral areas included in the plurality of vehicle peripheral images as subject images are the same. For this reason, even if a plurality of vehicle peripheral images are switched, the size of the peripheral region shown in the vehicle peripheral image does not change, so the user intuitively grasps the size of the peripheral region indicated by the vehicle peripheral image after switching. can do.

<1-5. Flow of operation>
Next, an operation flow in the peripheral display mode of the image display system 120 will be described. FIG. 11 is a diagram showing an operation flow in the peripheral display mode.

  First, the region selection unit 13 selects a region of interest based on the detection result of the sonar system 7. As described above, the region selection unit 13 selects, from among the four peripheral regions A1 to A4, the peripheral region where the object exists as the attention region based on the position of the object indicated by the detection result of the sonar system 7. . However, if there is no surrounding area where the object exists, no surrounding area is selected as the attention area (step S11).

  If no peripheral region is selected as a region of interest by the region selection unit 13, there is no peripheral region that should be enlarged and shown to the user. Therefore, in this case (No in step S12), the default display as shown in FIG. 6 is performed under the control of the display control unit 12. That is, the whole overhead view image BP is displayed on the screen D1 of the display 21 (step S16).

  On the other hand, when the peripheral area is selected as the attention area by the area selection unit 13 (Yes in step S12), it is next determined whether or not there are a plurality of attention areas (step S13). If there is only one region of interest (No in step S13), the process proceeds to step S15 as it is, and region-of-interest display processing for displaying a vehicle peripheral image indicating the region of interest on the display 21 is performed under the control of the display control unit 12. The

  On the other hand, when there are a plurality of attention areas (No in step S13), the priority setting unit 14 sets the priority of each of the plurality of attention areas. This priority prescribes the order in which the vehicle peripheral images indicating the region of interest are displayed, and is expressed by the order number (1, 2, 3,...). In the present specification, when the expression “high priority” is used, the value of the order number itself is small.

  The priority setting unit 14 sets the priority based on the distance of the object indicated by the detection result of the sonar system 7. Specifically, the distance of the object detected in each attention area is referred to, and the priority is increased in order from the attention area where the distance of the object is closest. That is, the closer the distance between the objects existing in the attention area is, the higher the priority of the attention area is (step S14). When the priority is set in this way, the process proceeds to step S15, and attention area display processing for displaying a vehicle peripheral image indicating the attention area on the display 21 is performed under the control of the display control unit 12.

  When this attention area display processing is performed, the processing returns to step S11, and the same processing as described above is repeated. As a result, an appropriate peripheral area is always selected as the attention area in accordance with an object existing around the vehicle at that time, and a vehicle peripheral image indicating the attention area is displayed on the display 21. Therefore, an appropriate peripheral area is shown to the user according to the traveling state of the vehicle 9.

  Next, attention area display processing (step S15) will be described. FIG. 12 is a diagram showing a detailed flow of attention area display processing.

  First, 1 is assigned to the variable P (P = 1) (step S21). Next, timing is started by the timer function provided in the control unit 1 (step S22).

  Next, the viewpoint position of the virtual viewpoint of the attention area having the priority “P” is acquired based on the viewpoint data 4a. In this case, since P = 1, the viewpoint position of the attention area with the highest priority is acquired (step S23).

  Then, the composite image generation unit 32 generates a vehicle peripheral image indicating the attention area having the priority “P” as viewed from the virtual viewpoint, and displays it on the display 21. That is, the vehicle peripheral image indicating the attention area with the highest priority is displayed on the display 21 (step S24).

  Then, until a predetermined time (for example, 2 seconds) elapses from the start of the time measurement in step S22 (No in step S26), the vehicle surroundings of the attention area having a priority of “P” at a predetermined frame rate (for example, 30 fps) An image is repeatedly generated and displayed on the display 21 (step S24). As a result, the state of the attention area having the priority “P” is displayed to the user in substantially real time for a predetermined time (for example, 2 seconds). When the predetermined time has elapsed (Yes in step S26), the process proceeds to step S28.

  Further, until a predetermined time (for example, 2 seconds) elapses from the start of time measurement in Step S22 (No in Step S26), the user performs a predetermined operation via the operation unit 22 and gives a “stop instruction”. Can do. When this “stop instruction” is given (Yes in step S25), the vehicle peripheral image of the attention area with the priority “P” is repeatedly generated and displayed on the display 21 regardless of the time keeping time (step 21). S24).

  The display of the vehicle peripheral image after such a “stop instruction” is continued until a “stop release instruction” is issued from the user via the operation unit 22 (No in step S27). As a result, the state of the attention area having the priority “P” for a predetermined time or more is shown to the user in substantially real time. Therefore, the user can sufficiently confirm the state of the desired peripheral area. If a “stop cancellation instruction” is issued (Yes in step S27), the process proceeds to step S28.

  In step S28, it is determined whether or not the variable P is smaller than a variable N indicating the number of selected attention areas. When the variable P is smaller than the variable N (P <N) (Yes in step S28), there is a region of interest to be switched and displayed. Therefore, in this case, the vehicle peripheral image of the attention area having the next highest priority (that is, the attention area having the priority “P + 1”) is switched and displayed on the display 21.

  When switching the vehicle peripheral image, first, the viewpoint position of the attention area having the priority “P + 1” is acquired based on the viewpoint data 4a (step S29). An animation expression that gradually moves the viewpoint position of the virtual viewpoint of the vehicle periphery image to be displayed from the viewpoint position of the attention area with the priority “P” to the viewpoint position of the attention area with the priority “P + 1”. This is performed on the display 21 (step S30). Details of this animation expression will be described later.

  Thereafter, the variable P is incremented (P = P + 1) (step S31), and the process returns to step S22. As a result, the vehicle peripheral image of the attention area having the next highest priority is displayed on the display 21, and the state of the attention area is shown to the user almost in real time for a predetermined time (for example, 2 seconds).

  Such a process is repeated for all selected regions of interest. As a result, the vehicle periphery images of all the attention areas are sequentially switched and displayed on the display 21. When the vehicle peripheral images of all the attention areas are displayed, in step S28, the variable P matches the variable N (P = N) (No in step S28), and the attention area display processing ends.

  Thus, in the image display system 120, the priority is set based on the distance of the object. The priority of the attention area is increased as the object distance is shorter, and the vehicle peripheral images are displayed on the display 21 in order from the attention area having the higher priority. For this reason, the vehicle periphery image which shows the object close | similar to the vehicle which a user should pay attention to can be displayed preferentially.

<1-6. Moving the viewpoint position>
When the vehicle periphery image to be displayed is switched (step S30), an animation expression for moving the viewpoint position of the virtual viewpoint in stages is performed. Hereinafter, this animation expression will be described taking the case of the second state ST22 shown in FIG. 8 as an example.

  FIG. 13 is an enlarged view of the second state ST22. As described above, in the second state ST22, the front left area A1 and the rear left area A3 are selected as the attention areas. On the display 21, as shown in FIG. 9, a vehicle peripheral image BP1 showing the front left region A1 and a vehicle peripheral image BP3 showing the rear left region A3 are displayed while being switched alternately.

  When the vehicle peripheral image to be displayed is switched from the vehicle peripheral image BP1 to the vehicle peripheral image BP3, the viewpoint position does not instantly switch from the center position Vpp of the front left region A1 to the center position Vpf of the rear left region A3. As shown in FIG. 13, the viewpoint position is moved stepwise and continuously from the position Vpp to the position Vpf. In FIG. 13, only two viewpoint positions in the middle of movement are illustrated, but actually there are many viewpoint positions in the middle of movement. Then, a plurality of vehicle peripheral images are generated based on the virtual viewpoint in which the viewpoint position moves stepwise in this way and displayed on the display 21.

  As a result, as shown in FIG. 14, a plurality of vehicle peripheral images BPa generated during movement of the viewpoint position are displayed on the display 21 in succession between the vehicle peripheral image BP1 and the vehicle peripheral image BP3. Is done. As a result, on the display 21, an animation expression in which the area around the vehicle 9 shown in the vehicle surrounding image changes step by step is performed. By performing such animation expression, the user can intuitively grasp which peripheral region the vehicle peripheral image after switching indicates.

  Such movement of the viewpoint position is performed using a plurality of frames over a predetermined time (for example, 1 second). Hereinafter, a method for determining the viewpoint position of the virtual viewpoint of the vehicle peripheral image of each frame will be described.

  In this description, in order to express the viewpoint position of the virtual viewpoint, coordinates of a two-dimensional orthogonal coordinate system (XY coordinate system) fixed relative to the vehicle 9 as shown in FIG. 13 are adopted. The coordinates of the position Vpp (center of the front left area A1) as the starting point of the movement of the virtual viewpoint are (Xp, Yp), and the coordinates of the position Vfp (center of the rear left area A3) as the end point of the movement of the virtual viewpoint are ( Xf, Yf). The viewpoint position in the middle of movement is set to Vpm. Further, the time taken to move the viewpoint position is T (s), and the frame rate is F (fps).

  The movement distance Dx in the X-axis direction and the movement distance Dy in the Y-axis direction of the virtual viewpoint can be derived from the following expressions (1) and (2), respectively.

Dx = Xf−Xp (1)
Dy = Yf−Yp (2)
Further, the average movement amount mx per frame in the X-axis direction and the average movement amount my per frame in the Y-axis direction can be derived from the following equations (3) and (4), respectively.

mx = Dx / (F · T) (3)
my = Dy / (F · T) (4)
Then, the X coordinate X _n and the Y coordinate Y _n of the viewpoint position Vpm in the middle of movement can be derived by the following equations (5) and (6).

_Xn = _Xn-1 + [alpha] .mx (5)
Y _n = Y _n-1 + α · my (6)
X _n−1 and Y _n−1 in the equations (5) and (6) indicate the X coordinate and the Y coordinate of the viewpoint position Vpm one frame before, respectively. Α is a speed coefficient that defines the moving speed of the virtual viewpoint. This speed coefficient α is set to be 1 on average in the movement of the viewpoint position.

  FIG. 15 is a diagram showing changes in the speed coefficient α. In the figure, the horizontal axis indicates the time from the start of movement of the virtual viewpoint, and the vertical axis indicates the speed coefficient α. As shown in the figure, the speed coefficient α is low immediately after the start of the movement of the virtual viewpoint, but is highest at an intermediate time point (T / 2 time point) in the movement, and then decreases. Therefore, the moving speed of the virtual viewpoint becomes slower as the viewpoint position is closer to either the movement start point Vpp or the end point Vfp. The moving speed of the virtual viewpoint increases as the viewpoint position moves away from the movement start point Vpp and the end point Vfp, and is highest at an intermediate position between the movement start point Vpp and the end point Vfp.

  By changing the moving speed of the virtual viewpoint in this way, it becomes easier for the user to grasp the starting point and the ending point of the moving virtual viewpoint, and the user's attention is directed to the attention area that is the starting point or the ending point of the moving virtual viewpoint. Can do. Moreover, the area | region of the periphery of the vehicle 9 with a low necessity shown to users other than an attention area is not displayed unnecessarily long.

  As described above, in the image display system 120, from the four peripheral areas A1 to A4, a plurality of peripheral areas that should be shown to the vehicle user at the same time are selected by the area selecting unit 13 as the attention area. Then, a plurality of vehicle peripheral images indicating the plurality of peripheral regions selected as the attention region are sequentially switched and displayed on the same display region PA in the screen D1 of the display 21. For this reason, since it is not necessary to display these several vehicle periphery images side by side simultaneously on the screen of the display 21, the size of each vehicle periphery image to display can be made comparatively large. As a result, the state of each peripheral area can be shown to the user relatively large in the limited screen of the display 21.

<2. Second Embodiment>
Next, a second embodiment will be described. In the first embodiment, the priority of the attention area is set based only on the distance of the object. On the other hand, in the second embodiment, the type of the object is determined, and the priority of the attention area is set in consideration of the type of the object. Since the configuration and operation of the image display system 120 of the present embodiment are substantially the same as those of the first embodiment, the following description will focus on differences from the first embodiment.

  FIG. 16 is a block diagram illustrating a configuration of the image display system 120 according to the second embodiment. The image display system 120 according to the second embodiment includes a type determination unit 15 as a function realized when the CPU of the control unit 1 performs arithmetic processing according to a program. The nonvolatile memory 40 stores type determination data 4b used by the type determination unit 15. The other configuration of the image display system 120 of the second embodiment is the same as that of the image display system 120 of the first embodiment shown in FIG.

  When the sonar system 7 detects an object in any of the surrounding areas A1 to A4, the type determination unit 15 determines whether the type of the object is a person. More specifically, the type determination unit 15 recognizes an image of an object included in the vehicle peripheral image using the vehicle peripheral image of the peripheral region where the object is detected. Then, the type determination unit 15 determines whether or not the type of the object is a person by a known technique such as pattern matching based on the image of the object. Type determination data 4b is used as data used for this pattern matching.

  For example, as shown in FIG. 17, it is assumed that there are two objects Sb1 and Sb2 around the vehicle 9, one object Sb1 is a tricycle, and the other object Sb2 is a person. In this example, the object (tricycle) Sb1 exists in the front right region A2, and the object (person) Sb2 exists in the rear left region A3.

  In this example, the sonar system 7 detects that an object is present in the two peripheral areas A2 and A3. Then, the type determination unit 15 determines the type of each of the two surrounding areas A2 and A3, and determines that the object Sb1 in the surrounding area A2 is not a person and the object Sb2 in the surrounding area A3 is a person.

  When there is a person around the vehicle 9, for safety reasons, the user needs to drive the vehicle 9 with maximum attention to the person. Therefore, in the image display system 120 according to the present embodiment, when it is determined that a person exists in the peripheral area, the priority of the peripheral area (attention area) in which the person exists is determined regardless of the distance of the object. Highest set. Therefore, in the example of FIG. 17, the priority of the rear left region A3 is set to the highest.

  FIG. 18 is a diagram showing a flow of processing (step S14 in FIG. 11) for setting the priority according to the present embodiment. At the start of this process, a plurality of peripheral regions are selected as a region of interest by the region selection unit 13.

  First, similarly to the first embodiment, the priority setting unit 14 sets the priority of the attention area based on the distance of the object. That is, the closer the distance between objects in the attention area is, the higher the priority of the attention area is (step S41).

  Next, a vehicle peripheral image for each of the plurality of attention areas is generated. Then, the type determination unit 15 determines whether or not the type of the object included in the vehicle peripheral image of each region of interest is a person (step S42).

  If there is a region of interest in which it is determined by this determination that the object is a person (Yes in step S43), the priority of the region of interest is set to the highest regardless of the priority set in step S41. (Priority = 1) (step S44). In addition, when there are a plurality of attention areas determined that the object is a person, the priority of the attention area may be set in consideration of the distance of the object.

  Also in the present embodiment, the attention area display process is performed in the same manner as the process of the first embodiment shown in FIG. Accordingly, the vehicle peripheral images are displayed on the display 21 in order from the attention area having the highest priority. For this reason, in the image display system 120 of 2nd Embodiment, the vehicle periphery image which shows the person whom a user should pay attention can be displayed preferentially.

<3. Third Embodiment>
Next, a third embodiment will be described. In the first embodiment, the peripheral area where the object exists is selected as the attention area. On the other hand, in the third embodiment, there are a plurality of display modes suitable for each of a plurality of traveling states of the vehicle, and a region of interest is selected according to the display mode. Since the configuration and operation of the image display system 120 of the present embodiment are also substantially the same as those of the first embodiment, the following description will focus on differences from the first embodiment.

  FIG. 19 is a block diagram illustrating a configuration of an image display system 120 according to the third embodiment. In the image display system 120 according to the third embodiment, the image processing apparatus 100 includes a changeover switch 45. The changeover switch 45 is a button type switch that can be operated by the user, and is arranged at an appropriate position of the vehicle separately from the main body unit 10 so that the user can easily operate. The operation content of the changeover switch is input to the control unit 1 as a signal.

  The image display system 120 also includes a mode switching unit 16 as a function realized by the CPU of the control unit 1 performing arithmetic processing according to a program. Further, the nonvolatile memory 40 stores mode data 4c indicating a region of interest corresponding to the display mode.

  Further, since the image display system 120 of the third embodiment does not use the signal from the sonar system 7, the signal input unit 41 is omitted. Other configurations of the image display system 120 of the third embodiment are the same as those of the image display system 120 of the first embodiment shown in FIG.

  The mode switching unit 16 switches the display mode in the peripheral display mode of the image display system 120. FIG. 20 is a diagram illustrating display mode transition in the peripheral display mode. The image display system 120 of the present embodiment has three display modes, a rear parking mode M11, a parallel parking mode M12, and a passing mode M13. In the peripheral display mode, each time the user presses the changeover switch 45, these display modes are changed.

  Each of the three display modes is optimized for a special driving condition of the vehicle. The rear parking mode M11 is a display mode suitable for performing rear parking, and the vertical parking mode M12 is a display mode suitable for performing parallel parking. Further, the passing mode M13 is a display mode suitable for passing the oncoming vehicle on a relatively narrow road. In these display modes, the peripheral area selected as the attention area is different.

  FIG. 21 is a diagram illustrating an operation flow in the peripheral display mode according to the third embodiment. First, the current display mode is acquired from the mode switching unit 16 to the region selection unit 13 (step S51). Next, the region selection unit 13 selects a region of interest with reference to the mode data 4c based on the current display mode. That is, in the present embodiment, the peripheral area selected as the attention area is defined only by the display mode regardless of the presence of the object (step S52). Then, similarly to the first embodiment, attention area display processing is performed (step S53).

  FIG. 22 is a diagram illustrating a peripheral region selected as a region of interest in the rear parking mode M11. As shown in FIG. 22, in the rear parking mode M11, the rear left area A3 and the rear right area A4 are selected as the attention areas. In this case, the visual field direction of the virtual viewpoint of the vehicle peripheral image is set to a substantially direct downward direction. The viewpoint position of the virtual viewpoint is moved between the center Vp3 of the rear left region A3 and the center Vp4 of the rear right region A4.

  As a result, in the rear parking mode M11, the vehicle peripheral images respectively showing the two peripheral areas A3 and A4 are alternately switched and displayed on the display 21.

  FIG. 23 is a diagram illustrating a peripheral region selected as a region of interest in the parallel parking mode M12. As shown in FIG. 23, in the parallel parking mode M12, the front left area A1, the rear left area A3, and the rear right area A4 are selected as the attention areas. In this case as well, the visual field direction of the virtual viewpoint of the vehicle peripheral image is set to a substantially direct downward direction. The viewpoint position of the virtual viewpoint is moved between the center Vp1 of the front left region A1, the center Vp3 of the rear left region A3, and the center Vp4 of the rear right region A4.

  As a result, in the parallel parking mode M12, the vehicle peripheral images respectively indicating the three peripheral regions A1, A3, and A4 are sequentially switched and displayed on the display 21.

  FIG. 24 is a diagram illustrating a peripheral region selected as a region of interest in the passing mode M13. As shown in FIG. 24, in the parallel parking mode M12, the left side area A5 set on the left side of the vehicle 9 and the right side area A6 set on the right side of the vehicle are selected as the attention area. The

  In this case, the visual field direction of the virtual viewpoint of the vehicle peripheral image is set to the forward direction. Then, the viewpoint position of the virtual viewpoint is moved between the rear center Vp5 of the left side region A5 and the rear center Vp6 of the right side region A6.

  Thus, in the passing mode M13, as shown in FIG. 25, the vehicle peripheral image BP5 showing the state of the left side region A5 viewed from the rear of the vehicle 9, and the vehicle showing the state of the right side region A6 viewed from the rear of the vehicle 9. The peripheral image BP6 is alternately switched and displayed on the same display area PA of the screen D1 of the display 21. With such a display, the user can easily confirm the clearance on the side of the vehicle 9 when passing the oncoming vehicle.

  As described above, in the image display system 120 according to the third embodiment, since the attention area is selected according to the display mode at that time, the peripheral area suitable for the driving state at that time can be accurately shown to the user. it can. Further, in the image display system 120 according to the third embodiment, cooperation with the sonar system 7 or the like is not necessary, and a function of showing an appropriate peripheral area to the user can be realized at a relatively low cost.

<4. Modification>
Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible. Below, such a modification is demonstrated. All forms including those described in the above embodiment and those described below can be combined as appropriate.

  In the above embodiment, the area selection unit 13 selects the attention area. However, the attention area may be selected based on a user operation. In this case, the user can confirm a desired peripheral area on the display 21.

  Moreover, in the said embodiment, only the vehicle periphery image which shows an attention area was displayed on the screen D1 of the display 21. FIG. On the other hand, as shown in FIG. 26, the whole overhead view image BP may be displayed on the screen D2 of the display 21 together with the vehicle peripheral image. In this case, the whole bird's-eye view image BP is always displayed in the first display area PA1, and a plurality of vehicle peripheral images are sequentially switched and displayed in the second display area PA2. In this case, as shown in FIG. 26, the position of the peripheral area indicated by the vehicle peripheral image displayed in the second display area PA2 is displayed in the overall overhead image BP displayed in the first display area PA1. It may be clearly indicated by a frame C or the like. In the example of FIG. 26, the vehicle peripheral image BP1 showing the front left area A1 is displayed in the second display area PA2. Therefore, a frame C is displayed at a position corresponding to the front left area A1 in the subject image in the overall overhead view image BP.

  In the first and second embodiments, the sonar system 7 detects an object existing around the vehicle. On the other hand, for example, an object existing around the vehicle may be detected by other methods such as object detection by a radar device or object recognition based on an image.

  In the first and second embodiments, the peripheral area where an object exists around the vehicle is selected as the attention area. On the other hand, you may make it select the surrounding area in which the white line which shows a parking space exists as an attention area. In this case, a known white line recognition method for recognizing the white line shown in the image may be employed.

  Moreover, in the said embodiment, the front side of the vehicle 9 was shown by the image upper side in both the vehicle periphery image which shows the surrounding area ahead of a vehicle, and the vehicle surrounding image which shows the surrounding area behind the vehicle. On the other hand, in the vehicle peripheral image showing the peripheral region behind the vehicle, the rear side of the vehicle 9 may be shown on the upper side of the image. In this case, when switching from the vehicle peripheral image indicating the front peripheral region of the vehicle to the vehicle peripheral image indicating the peripheral region behind the vehicle, the direction of the virtual viewpoint may be rotated by 180 degrees.

  In the second embodiment, when determining the type of the object, only the person is determined. However, other types of the object may be determined. For example, a person, vehicle, animal, or the like may be determined. Moreover, you may make it determine whether an object is an approaching object. Whether or not the object is an approaching object can be determined based on whether or not the distance of the object decreases with time. Since such an approaching object is highly likely to come into contact with the vehicle, it is desirable to set a high priority for the surrounding area (attention area) where the approaching object exists.

  Moreover, in the said embodiment, the vehicle periphery image was displayed on the display 21 in an order from the attention area with a high priority. On the other hand, the higher the priority of the attention area, the longer the display time of the vehicle peripheral image indicating the attention area.

  In the above-described embodiment, the vehicle peripheral image indicating the peripheral region is described as a composite image, but may be a captured image that has not been subjected to image composite processing.

  In the above embodiment, the image processing apparatus 100 and the navigation apparatus 20 are described as separate apparatuses. However, the image processing apparatus 100 and the navigation apparatus 20 are arranged in the same casing and are integrated. It may be configured as.

  In the above embodiment, the display device that displays the image generated by the image processing device 100 is described as the navigation device 20, but a general display device that does not have a special function such as a navigation function. It may be.

  In the above embodiment, some of the functions described as being realized by the control unit 1 of the image processing apparatus 100 may be realized by the control unit 23 of the navigation apparatus 20.

  The signal from the sonar system 7 may be input to the navigation device 20. In this case, the signal may be input to the control unit 1 of the image processing apparatus 100 via the navigation communication unit 42.

  Further, in the above-described embodiment, it has been described that various functions are realized in software by the arithmetic processing of the CPU according to the program. However, some of these functions are realized by an electrical hardware circuit. Also good. Conversely, some of the functions realized by the hardware circuit may be realized by software.

7 sonar system 9 vehicle 12 display control unit 13 region selection unit 14 priority setting unit 15 type determination unit 21 display A1 front left region A2 front right region A3 rear left region A4 rear right region 120 image display system

Claims (13)

An image display system mounted on a vehicle,
Generating means for generating a vehicle peripheral image indicating a peripheral region of a part of the periphery of the vehicle from a captured image obtained by a camera disposed in the vehicle;
Selecting means for selecting a plurality of surrounding areas to be shown to the user of the vehicle at the same time;
Display means for sequentially switching a plurality of vehicle peripheral images showing the plurality of peripheral areas and displaying them in the same display area in the screen;
An image display system comprising: The image display system according to claim 1,
The image display system, wherein the display means repeatedly displays the plurality of vehicle surrounding images. The image display system according to claim 1 or 2,
The image display system characterized in that the generation means can generate the vehicle peripheral image indicating the peripheral region viewed from an arbitrary viewpoint from captured images obtained by a plurality of cameras. The image display system according to claim 3,
When the vehicle surrounding image is switched, the display means performs an animation expression in which the viewpoint position moves from the viewpoint position of the vehicle surrounding image before switching to the viewpoint position of the vehicle surrounding image after switching. An image display system characterized by the above. The image display system according to claim 4,
The moving speed of the viewpoint when the position of the viewpoint moves is
The closer the viewpoint is to either the start or end of movement, the slower
An image display system characterized in that the position of the viewpoint is faster as it moves away from the starting and end points of movement. The image display system according to any one of claims 1 to 5,
The image display system characterized in that the plurality of vehicle peripheral images have the same angle of view. The image display system according to any one of claims 1 to 6,
Obtaining means for obtaining a detection result of detecting an object existing around the vehicle;
Further comprising
The image display system, wherein the selection unit selects the plurality of peripheral regions based on the position of the object indicated by the detection result. The image display system according to claim 7,
The detection result includes a distance of the object from the vehicle,
Setting means for setting the priority in displaying the plurality of vehicle peripheral images of each of the plurality of peripheral regions based on the distance of the object;
An image display system comprising: The image display system according to claim 8,
The image display system according to claim 1, wherein the setting unit increases the priority of the peripheral area as the distance between the objects existing in the peripheral area is shorter. The image display system according to any one of claims 1 to 6,
Discriminating means for discriminating the types of objects existing around the vehicle;
Setting means for setting a priority in displaying the plurality of vehicle peripheral images of each of the plurality of peripheral regions based on the type of the object;
An image display system further comprising: The image display system according to any one of claims 1 to 6,
Switching means for switching a plurality of display modes suitable for each of a plurality of driving states of the vehicle;
With
The image display system, wherein the selection unit selects the plurality of peripheral areas according to the current display mode. An image processing device for generating an image to be displayed on a display device mounted on a vehicle,
Generating means for generating a vehicle peripheral image indicating a peripheral region of a part of the periphery of the vehicle from a captured image obtained by a camera disposed in the vehicle;
Selecting means for selecting a plurality of surrounding areas to be shown to the user of the vehicle at the same time;
Output means for outputting a plurality of vehicle peripheral images showing the plurality of peripheral areas to the display device, and sequentially switching them to be displayed on the same display region in the screen of the display device;
An image processing apparatus comprising: An image display method for displaying an image in a vehicle,
Generating a vehicle peripheral image indicating a peripheral region of a part of the periphery of the vehicle from a captured image obtained by a camera disposed in the vehicle;
Selecting a plurality of surrounding areas to be shown to the user of the vehicle at the same time;
A step of sequentially switching a plurality of vehicle peripheral images showing each of the plurality of peripheral regions and displaying them in the same display region in the screen;
An image display method comprising:

Images