JP2017092766A - Image processing device, information display system, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device, an information display system, an image processing method and a program that cause a driver to perceive a display area of a display unit functioning as an electronic mirror as an electronic mirror without feeling of strangeness.SOLUTION: An image processing device 10 includes a rearview mirror image generator 11b, a background image generator 11c, and an image composer 11d. The rearview mirror image generator 11b performs mirror-image inversion on a rear-side image of a vehicle captured by an imaging unit 20 for imaging the outside of the vehicle to generate a rearview mirror image. The rearview mirror image generator 11b generates a background image that streams at the speed corresponding to the vehicle speed of the vehicle and in an arbitrary stream direction different from the rearview mirror image. The image composer 11d combines the background image and the rearview mirror image so that the rearview mirror image is arranged inside the background image.SELECTED DRAWING: Figure 2

Description

  Embodiments disclosed herein relate to an image processing apparatus, an information display system, an image processing method, and a program.

  2. Description of the Related Art Conventionally, an electronic mirror system that displays an electronic mirror image generated from a captured image obtained by photographing the rear of a vehicle in a predetermined display area of a display unit disposed in a vehicle interior is known as a kind of information display system for vehicles. (For example, refer to Patent Document 1).

JP 2010-188903 A

  However, when the above-described conventional technology is used, the electronic mirror image is uncomfortable as compared with the case where the driver visually recognizes the actual rearview mirror, and is an electronic mirror image corresponding to the rearview mirror for the driver. There was a problem that it was difficult to recognize immediately.

  One embodiment of the present invention has been made in view of the above, and an image processing apparatus and an information display system that allow a driver to recognize a display area of a display unit functioning as an electronic mirror as an electronic mirror without a sense of incongruity An object of the present invention is to provide an image processing method and a program.

  An image processing apparatus according to an aspect of an embodiment includes a first image generation unit, a second image generation unit, and an image composition unit. The first image generation unit generates a rear-view mirror image obtained by inverting the mirror image of the rear image of the vehicle captured by an imaging unit that captures the outside of the vehicle. The second image generation unit generates a background image that flows in an arbitrary flow direction different from the rear-view mirror image at a speed according to the vehicle speed of the vehicle. The image synthesis unit synthesizes the background image and the rearview mirror image so that the rearview mirror image is arranged inside the background image.

  According to one aspect of the embodiment, the display area of the display unit functioning as an electronic mirror can be recognized by the driver as an electronic mirror without a sense of incongruity.

FIG. 1A is a diagram (part 1) illustrating an overview of an image processing method according to the embodiment. FIG. 1B is a diagram (part 2) illustrating an overview of an image processing method according to the embodiment. FIG. 1C is a diagram (part 3) illustrating an overview of the image processing method according to the embodiment. FIG. 1D is a diagram (part 4) illustrating an overview of the image processing method according to the embodiment. FIG. 1E is a diagram (part 5) illustrating an overview of the image processing method according to the embodiment. FIG. 2 is a block diagram of the information display system according to the first embodiment. FIG. 3A is a diagram illustrating an example of positions of selectable vanishing points included in the background image information. FIG. 3B is a diagram (No. 1) illustrating a method for determining a flow direction of a background image in a background image generation unit. FIG. 3C is a diagram (part 2) illustrating the method of determining the flow direction of the background image in the background image generation unit. FIG. 4A is a diagram (part 1) illustrating an example of a selectable geometric pattern included in background image information. FIG. 4B is a diagram (part 2) illustrating an example of a selectable geometric pattern included in the background image information. FIG. 4C is a diagram (part 3) illustrating an example of a selectable geometric pattern included in the background image information. FIG. 5 is a schematic diagram of each process in the image processing apparatus. FIG. 6 is a flowchart illustrating a processing procedure executed by the image processing apparatus according to the first embodiment. FIG. 7A is a block diagram of an information display system according to the second embodiment. FIG. 7B is a block diagram of a background image generation unit according to the second embodiment. FIG. 8 is a flowchart illustrating a processing procedure executed by the image processing apparatus according to the second embodiment. FIG. 9 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information display system according to the embodiment.

  Hereinafter, embodiments of an image processing apparatus, an information display system, an image processing method, and a program disclosed in the present application will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by embodiment shown below.

  In the following, after an overview of the image processing method according to the present embodiment is described with reference to FIGS. 1A to 1E, an image processing apparatus 10 to which the image processing method is applied and an information display system 1 including the image processing device 10 will be described. It will be described with reference to FIGS. In the description using FIGS. 1A to 6, the first embodiment will be described, and in the description using FIGS. 7A to 8, the second embodiment will be described.

  In the following, the information display system 1 generates a rear image of the vehicle that is normally reflected as a mirror image on the rearview mirror (rear mirror) based on the captured image of the in-vehicle camera, and displays it on a display unit such as a display. The description will be given with a main example of providing a so-called “electronic mirror” function. Note that the rearview mirror includes any of a room mirror, a door mirror, a fender mirror, and a side under mirror. Of these rearview mirrors (rear mirrors), door mirrors, fender mirrors, side under mirrors, and the like attached to the outside of the vehicle body are also referred to as “rear mirrors outside the vehicle body”.

  First, an outline of the image processing method according to the present embodiment will be described with reference to FIGS. 1A to 1E. 1A to 1E are views (No. 1) to (No. 5) showing an overview of an image processing method according to the embodiment.

  As shown in FIG. 1A, the information display system 1 according to the present embodiment is configured to display driving support information of a vehicle C, for example. Specifically, the information display system 1 includes a display unit 4 and an image processing device 10.

  The display unit 4 is a device that functions as a display area for displaying driving support information to the driver of the vehicle C. For example, as shown in FIG. 1A, information directly on the driver's view in front of the driver's seat of the vehicle C is displayed. It is configured as a head-up display that projects In the present embodiment, the description will be given below assuming that the display unit 4 is a head-up display.

  In addition, the display part 4 may be comprised as a center display etc. of the vehicle room front center of the vehicle C, for example.

  The image processing apparatus 10 is arranged, for example, inside the center of the instrument panel, and performs an image generation process as driving support information to be output to the display unit 4. Such an image is generated, for example, as an electronic mirror image corresponding to each of the door mirrors MR, ML, and as an image indicating operation information such as the speed of the vehicle C, for example.

  Specifically, as shown in FIG. 1B, the information display system 1 according to the present embodiment arranges a display area D1 of an electronic mirror image corresponding to the door mirror MR, for example, on the lower right side of the display unit 4. For example, the display area D2 of the electronic mirror image corresponding to the door mirror ML is arranged on the lower left side of the display unit 4. Further, for example, a speed display area D <b> 3 as operation information of the vehicle C is arranged in the upper center of the display unit 4.

  By the way, normally, when the driver captures the actual rearview mirror in the field of view, the driver sees the mirror image reflected on the mirror itself and the background around the mirror. Specifically, as shown in FIG. 1C, when the driver captures the actual door mirror MR in the field of view, the driver displays a mirror image M1 inside the door mirror MR and a background B1 outside the door mirror MR. Visually check.

  In such a case, if the vehicle C is moving straight ahead, the “flow direction” in which the mirror image M1 flows is, for example, the direction toward the vanishing point of the mirror image M1 (flow direction dm1). On the other hand, the “flow direction” flowing in the background B1 is the movement direction of the scenery (flow direction db1) flowing backward relative to the movement direction of the vehicle C when the driver views the door mirror MR in the field of view. ).

  That is, it can be seen that the “flow direction” of the mirror image M1 visually recognized by the driver is different from the “flow direction” of the background B1. In other words, a mirror image imitating the mirror image M1 is arranged inside the background image imitating the background B1, and is different from the mirror image M1 according to the moving speed of the vehicle C at a speed corresponding to the vehicle speed. If it is made to flow in each “flow direction”, it means that the driver can easily accept the electronic mirror image without a sense of incongruity.

  Returning to the description of FIG. Therefore, in the information display system 1 according to the present embodiment, a mirror image that simulates the mirror image M1 is arranged inside the background image that simulates the background B1 for the display areas D1 and D2 that function as electronic mirrors, and the display area D1 corresponds to the vehicle speed. It is assumed that the flow is made in a “flow direction” that is different from the mirror image M1 corresponding to the moving direction of the vehicle C at a speed.

  In the following description, the display area D1 corresponding to the door mirror MR will be mainly exemplified and described. The display area D2 corresponding to the door mirror ML is different from the display area D1 in that it is bilaterally symmetric. Note that the background images Bi1 and Bi2 of the display areas D1 and D2 to be described below are collectively referred to as a background image Bi for convenience. Similarly, when the rearview mirror images Mi1 and Mi2 in the display areas D1 and D2 are collectively referred to for convenience, they may be referred to as rearview mirror images Mi.

  Specifically, as shown in FIG. 1B, in this embodiment, the rearview mirror image (post-mirror image) Mi1 is arranged inside the background image Bi1 for the display area D1. The rearview mirror image Mi1 is generated by reversing a rear image of the vehicle C captured by an imaging unit (described later) of the vehicle C. That is, since the rearview mirror image Mi1 is based on an actual captured image, the flow velocity and the flow direction dm1 may remain captured.

  On the other hand, the background image Bi1 is generated so as to flow at a speed corresponding to the vehicle speed of the vehicle C and in a flow direction db1 different from the rearview mirror image Mi1. At this time, the background image Bi1 does not necessarily have to be generated based on the actual captured image.

  That is, the flow direction db1 of the background image Bi1 only needs to remind the driver of at least the moving direction of the scenery that flows relative to the moving direction of the vehicle C, and exactly matches the flowing direction of the captured image or the like. There is no need to let them. Therefore, the flow direction db1 of the background image Bi1 may be an arbitrary direction different from the flow direction dm1 of the rearview mirror image Mi1.

  One specific example is shown in FIG. 1D. As shown in FIG. 1D, in the information display system 1, for example, the flow direction db1 of the background image Bi1 in the display area D1 when the vehicle C travels straight, and the midpoint of the display areas D1 and D2 is the vanishing point P. The direction is along each straight line extending radially from the vanishing point P.

  In this case, the flow direction db1 of the display area D1 does not coincide with the moving direction of the scenery that flows relative to the moving direction of the vehicle C in the actual visual field from the driver's seat of the vehicle C. However, by driving the background image in a direction that diffuses or converges radially from the center of the display areas D1 and D2 that function as the left and right electronic mirrors, at least driving the moving direction of the scenery flowing relative to the vehicle C traveling straight ahead Can be recalled.

  Note that elements such as rotation based on the steering angle of the vehicle C are added to the flow direction of the background images Bi1 and Bi2, which will be described later in the description using FIGS. 3A to 3C. Further, other examples of the vanishing point P will be described later in the description using FIGS. 3A to 3C.

  Further, the background image Bi1 does not have to be based on the actual captured image, and it is only necessary to remind the driver of the moving direction of the scenery flowing relative to the traveling direction of the vehicle C. The image may move along the flow direction db1.

  One specific example is shown in FIG. 1E. As shown in FIG. 1E, in the information display system 1, for example, a background image Bi1 including a geometric pattern aP1 is generated, and the geometric pattern aP1 is moved along the flow direction db1 at a speed according to the vehicle speed. Here, the geometric pattern aP1 preferably has a shape (for example, an arrow) indicating the flow direction db1.

  As described above, the background image Bi1 in which the geometric pattern aP1 having the shape indicating the flow direction db1 is moved at a speed corresponding to the vehicle speed also indicates at least the movement direction of the scenery flowing relative to the traveling direction of the vehicle C. Can be recalled. Other examples of the geometric pattern will be described later with reference to FIGS. 4A to 4C.

  And by arrange | positioning the above-mentioned back mirror image Mi1 inside such background image Bi1, the display area D1 which functions as an electronic mirror can be made to recognize a driver as an electronic mirror without a sense of incongruity.

  Thus, in the present embodiment, a rearview mirror image Mi is generated by inverting the rear image of the vehicle C captured by the imaging unit of the vehicle C, and the rearview mirror is at a speed corresponding to the vehicle speed of the vehicle C. A background image Bi that flows in an arbitrary flow direction different from the image Mi is generated, and is synthesized so that the rearview mirror image Mi is arranged inside the background image Bi.

  Therefore, according to the present embodiment, the display areas D1 and D2 that function as electronic mirrors can be recognized by the driver as electronic mirrors without a sense of incongruity.

  Hereinafter, the image processing apparatus 10 that performs image processing by the above-described image processing method and the information display system 1 including the image processing apparatus 10 will be described more specifically.

(First embodiment)
FIG. 2 is a block diagram of the information display system 1 according to the first embodiment. In FIG. 2, only components necessary for explaining the features of the present embodiment are represented by functional blocks, and descriptions of general components are omitted.

  In other words, each component illustrated in FIG. 2 is functionally conceptual and does not necessarily need to be physically configured as illustrated. For example, the specific form of distribution / integration of each functional block is not limited to the one shown in the figure, and all or a part thereof is functionally or physically distributed in arbitrary units according to various loads or usage conditions.・ It can be integrated and configured.

  As shown in FIG. 2, the information display system 1 includes an operation unit 2, a display control unit 3, a display unit 4, and an image processing device 10. The information display system 1 exchanges information with various devices such as the imaging unit 20, the vehicle speed sensor 30, the rudder angle sensor 40, and the road surface sensor 50 mounted on the vehicle C.

  The information display system 1 will be described first from components other than the image processing apparatus 10. The operation unit 2 is an input device that receives an input operation from a user such as a driver. It may be an H / W device or an S / W device. The user performs an input operation of information related to generation of the background images Bi1 and Bi2, such as an operation of specifying the position of the vanishing point P described above and an operation of selecting the geometric pattern described above, for example, via the operation unit 2. Can be done.

  The display control unit 3 receives an electronic mirror image that is generated and output by the image processing apparatus 10 to be described later, and causes the display unit 4 to display it. The display unit 4 is, for example, a head-up display as described above, and displays an electronic mirror image as driving support information output from the display control unit 3 on the display areas D1 and D2.

  Next, the image processing apparatus 10 will be described. The image processing apparatus 10 includes a control unit 11 and a storage unit 12. The control unit 11 further includes a setting unit 11a, a rearview mirror image generation unit 11b, a background image generation unit 11c, and an image composition unit 11d.

  The storage unit 12 is a storage device such as a hard disk drive, a nonvolatile memory, or a register, and stores background image information 12a.

  The control unit 11 controls each unit of the image processing apparatus 10. The setting unit 11a accepts a user input operation via the operation unit 2, and sets the background image information 12a according to the content of the input operation.

  The background image information 12a is a group of information related to the generation of the background images Bi1 and Bi2, and includes a plurality of patterns such as the position of the vanishing point P and the geometric pattern described above that can be selected, for example.

  The rearview mirror image generation unit 11b generates rearview mirror images Mi1 and Mi2 obtained by inverting the rear image of the vehicle C captured by the rear camera 21 mounted for rearward imaging of the vehicle C in the imaging unit 20. . Further, the rearview mirror image generation unit 11b outputs the generated rearview mirror images Mi1 and Mi2 to the image composition unit 11d.

  The background image generation unit 11c generates background images Bi1 and Bi2 based on detection values of the vehicle speed sensor 30, the steering angle sensor 40, the road surface sensor 50, and the like and the background image information 12a. The vehicle speed sensor 30 detects the vehicle speed of the vehicle C. The steering angle sensor 40 detects the steering angle of the vehicle C. The road surface sensor 50 detects the undulation of the road surface on which the vehicle C is moving.

  Based on these detection values and background image information 12a, the background image generation unit 11c has a speed according to the vehicle speed of the vehicle C and in an arbitrary flow direction different from the rearview mirror images Mi1 and Mi2, as described above. Flowing background images Bi1 and Bi2 are generated. The background image generation unit 11c outputs the generated background images Bi1 and Bi2 to the image composition unit 11d.

  Here, an example of the background image information 12a and the processing of the background image generation unit 11c based on this will be described with reference to FIGS. 3A to 4C. FIG. 3A is a diagram illustrating an example of a position of a selectable vanishing point P included in the background image information 12a.

  3B and 3C are diagrams (No. 1) and (No. 2) illustrating a method of determining the flow direction of the background images Bi1, Bi2 in the background image generation unit 11c. 4A to 4C are diagrams (part 1) to (part 3) showing examples of selectable geometric patterns included in the background image information 12a.

  As shown in FIG. 3A, the background image information 12a includes, for example, a vanishing point P1 that is the position of the middle point of the display areas D1 and D2 of the display unit 4 and a vanishing point P2 that is the position of the center of the windshield of the vehicle C. And are included. The vanishing point P1 corresponds to the vanishing point P already described with reference to FIG. 1D.

  Then, when the vanishing point P1 is selected, as shown in FIG. 3B, the background image generation unit 11c sets the detected value of the rudder angle sensor 40 in a direction along each straight line extending radially from the vanishing point P1, for example. The flow directions db1 and db2 in the background images Bi1 and Bi2 are determined in consideration of the rotation angle dr and the traveling direction of the vehicle C.

  When the vanishing point P2 is selected, as shown in FIG. 3C, the background image generating unit 11c causes the straight lines extending radially from the vanishing point P2 to overlap the door mirrors MR and ML, for example, the rotation angle dr. In addition, taking into account the traveling direction of the vehicle C, the flow directions db1 and db2 in the background images Bi1 and Bi2 are determined.

  Further, the background image information 12a includes, for example, a geometric pattern aP1 as shown in FIG. 4A. The geometric pattern aP1 has an arrow as a shape indicating the flow directions db1 and db2. 4B, the background image information 12a may include a geometric pattern aP2 that shows the flow directions db1 and db2 and has a shape like a shooting star with a tail, for example.

  Further, as shown in FIG. 4C, a geometric pattern aP3 including a curve may be included in the background image information 12a although it is an arrow indicating the flow directions db1 and db2. The geometric patterns illustrated in FIGS. 4A to 4C include a shape (arrow) indicating the flow direction, but a circular shape, a rectangular shape, a polygon shape, or the like that does not include such a shape is used as the geometric pattern of the background image information 12a. It may be adopted. Further, a predetermined mark or the like may be adopted as the geometric pattern of the background image information 12a.

  Then, the background image generation unit 11c uses, for example, one selected from the geometric patterns aP1 to aP3 along the flow directions db1 and db2 determined by the geometric patterns aP1 to aP3 and the vehicle speed. Background images Bi1 and Bi2 that move at a speed according to the detection value of the sensor 30 are generated.

  For example, when the geometric pattern aP3 shown in FIG. 4C is selected, the background image generation unit 11c represents the undulation of the road surface based on the detection value from the road surface sensor 50 by the curved shape of the geometric pattern aP3. Alternatively, the background images Bi1 and Bi2 may be generated.

  Returning to the description of FIG. 2, the image composition unit 11d will be described. The image composition unit 11d synthesizes the background image Bi and the rearview mirror image Mi so that the rearview mirror image Mi is arranged inside the background image Bi.

  Here, in order to make the above description easier to understand, the processes in the rearview mirror image generation unit 11b, the background image generation unit 11c, and the image composition unit 11d are schematically summarized in FIG. FIG. 5 is a schematic diagram of each process in the image processing apparatus 10.

  That is, as illustrated in FIG. 5, the rearview mirror image generation unit 11b generates a rearview mirror image Mi obtained by inverting the rear image of the vehicle C captured by the rear camera 21 of the imaging unit 20 (step S1).

  In addition, the background image generation unit 11c generates a background image Bi that flows in an arbitrary flow direction different from the rearview mirror image Mi at a speed according to the vehicle speed of the vehicle C (step S2).

  Then, the image composition unit 11d synthesizes the back mirror image Mi and the background image Bi so that the back mirror image Mi is arranged inside the background image Bi (step S3).

  Next, a processing procedure executed by the image processing apparatus 10 according to the present embodiment will be described with reference to FIG. FIG. 6 is a flowchart illustrating a processing procedure executed by the image processing apparatus 10 according to the first embodiment.

  As shown in FIG. 6, first, the setting unit 11a performs initial setting of the background image information 12a based on an input operation via the user's operation unit 2 (step S101). If the function of the electronic mirror is ON (step S102, Yes), the rearview mirror image generation unit 11b acquires the rear image of the vehicle C from the imaging unit 20 (step S103), and based on this, the rearview mirror An image Mi is generated (step S104).

  The background image generation unit 11c acquires the vehicle speed of the vehicle C from the vehicle speed sensor 30 (step S105) and determines the flow direction of the background image Bi (step S106). Then, the background image generation unit 11c generates a background image Bi based on the processing results of steps S105 and S106 (step S107).

  Then, the image synthesis unit 11d synthesizes the rearview mirror image Mi and the background image Bi (step S108), outputs the resultant image to the display control unit 3, and then repeats the processing from step S102.

  If the function of the electronic mirror is not ON (step S102, No), the process is terminated.

  As described above, the image processing apparatus according to the first embodiment includes a rearview mirror image generation unit (corresponding to an example of a “first image generation unit”) and a background image generation unit (“second image generation unit”). And an image composition unit.

  The rearview mirror image generation unit generates a rearview mirror image (corresponding to an example of a “rearview mirror image”) obtained by inverting the rear image of the vehicle captured by the imaging unit that captures the outside of the vehicle. The background image generation unit generates a background image that flows at an arbitrary flow direction different from the rearview mirror image at a speed corresponding to the vehicle speed of the vehicle. The image composition unit synthesizes the background image and the rearview mirror image so that the rearview mirror image is arranged inside the background image.

  Therefore, according to the image processing apparatus which concerns on 1st Embodiment, a driver | operator can be made to recognize a display area of the display part which functions as an electronic mirror as an electronic mirror without a sense of incongruity.

  By the way, the case where the background image is generated without being based on the captured image of the imaging unit has been described as an example, but the background image may be generated based on the captured image. Such a case will be described below as a second embodiment.

(Second Embodiment)
FIG. 7A is a block diagram of an information display system 1 ′ according to the second embodiment. Since FIG. 7A corresponds to FIG. 2, portions different from FIG. 2 will be mainly described. FIG. 7B is a block diagram of the background image generation unit 11c ′ according to the second embodiment.

  In the information display system 1 ′ according to the second embodiment, the background image generation unit 11c ′ of the imaging unit 20 includes, for example, the vehicle C imaged by the front camera 22 mounted for imaging the vehicle C forward. A background image Bi is generated based on the front image.

  Specifically, the background image generation unit 11c ′ generates a viewpoint conversion image obtained by converting the above-described front image into an arbitrary viewpoint position as the background image Bi. More specifically, as illustrated in FIG. 7B, the background image generation unit 11c 'further includes a distortion correction unit 11ca and a viewpoint conversion calculation unit 11cb. It can be said that the viewpoint conversion image used for the background image Bi is an image obtained by viewing an arbitrary visual field direction from an arbitrary viewpoint position.

  The distortion correction unit 11ca performs correction processing of lens distortion of the front image acquired from the front camera 22 by calculation. Further, the distortion correction unit 11ca outputs the corrected forward image to the viewpoint conversion calculation unit 11cb.

  The viewpoint conversion calculation unit 11cb performs calculation processing for converting the front image after the correction processing acquired from the distortion correction unit 11ca into, for example, an arbitrary viewpoint position included in the background image information 12a. In addition, the viewpoint conversion calculation unit 11cb outputs the viewpoint conversion image after the calculation process to the image composition unit 11d as the background image Bi.

  An arbitrary viewpoint position and an arbitrary visual field direction can be set in the background image information 12a in advance by a user input operation via the operation unit 2. Here, the viewpoint position is preferably a predetermined position (hereinafter referred to as “driver viewpoint position”) corresponding to the position of the eyes of the driver when the driver is seated in the driver's seat of the vehicle. Further, the visual field direction is preferably a predetermined direction (hereinafter referred to as “driver visual field direction”) corresponding to a direction in which the driver naturally sits on the driver's seat and directs his / her line of sight to the vehicle mirrors MR and ML.

  However, as described in the first embodiment, the background image Bi only needs to remind the driver of the moving direction of the scenery that flows relative to the moving direction of the vehicle C. For this reason, the viewpoint position and the visual field direction need not exactly coincide with the driver viewpoint position and the driver visual field direction, and may be any position and direction. Further, only one of the viewpoint position and the visual field direction may be matched with the driver viewpoint position and the driver visual field direction. For example, the flow direction db of the background image Bi can be made more natural by matching only the visual field direction with the driver visual field direction. If the viewpoint conversion calculation is performed according to the arbitrary viewpoint position and the viewing direction, the flow direction db of the background image Bi in the viewpoint conversion image after the calculation process is automatically set according to the selected viewpoint position and the viewing direction. Will be determined.

  As the arithmetic processing for correcting lens distortion and the arithmetic processing for viewpoint conversion, a well-known method in the image processing technology can be used, and thus detailed description thereof is omitted here.

  Next, a processing procedure executed by the image processing apparatus 10 'according to the present embodiment will be described with reference to FIG. FIG. 8 is a flowchart illustrating a processing procedure executed by the image processing apparatus 10 ′ according to the second embodiment.

  As shown in FIG. 8, first, the setting unit 11a performs initial setting of the background image information 12a based on an input operation via the user's operation unit 2 (step S201). In the present embodiment, the above-described arbitrary viewpoint position and viewing direction are set.

  If the function of the electronic mirror is ON (step S202, Yes), the rearview mirror image generation unit 11b acquires the rear image of the vehicle C from the imaging unit 20 (step S203), and the rearview mirror is based on this. An image Mi is generated (step S204).

  Further, the background image generation unit 11c ′ acquires a front image of the vehicle C from the imaging unit 20 (step S205), and performs a viewpoint conversion calculation on the front image (step S206). Then, the background image generation unit 11c ′ generates the viewpoint converted image after the viewpoint conversion as the background image Bi (step S207).

  Then, the image composition unit 11d synthesizes the rearview mirror image Mi and the background image Bi (step S208), and outputs them to the display control unit 3, and then repeats the processing from step S202.

  If the function of the electronic mirror is not ON (step S202, No), the process ends.

  As described above, the image processing apparatus according to the second embodiment includes a rearview mirror image generation unit (corresponding to an example of a “first image generation unit”) and a background image generation unit (“second image generation unit”). And an image composition unit.

  The rearview mirror image generation unit generates a rearview mirror image (corresponding to an example of a “rearview mirror image”) obtained by inverting the rear image of the vehicle captured by the imaging unit that captures the outside of the vehicle. The background image generation unit generates a background image that flows at an arbitrary flow direction different from the rearview mirror image at a speed corresponding to the vehicle speed of the vehicle. The image composition unit synthesizes the background image and the rearview mirror image so that the rearview mirror image is arranged inside the background image.

  In the image processing apparatus according to the second embodiment, the background image generation unit uses the viewpoint converted image obtained by converting the front image of the vehicle imaged by the imaging unit into an arbitrary viewpoint position as the background image.

  Therefore, according to the image processing apparatus which concerns on 2nd Embodiment, a driver | operator can be made to recognize a display area of the display part which functions as an electronic mirror as an electronic mirror without a sense of incongruity.

  By the way, the information display systems 1 and 1 ′ according to the first or second embodiment can be realized by a computer 600 having a configuration shown as an example in FIG. 9. FIG. 9 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information display systems 1 and 1 ′ according to the embodiment.

  The computer 600 includes a CPU (Central Processing Unit) 610, a ROM (Read Only Memory) 620, a RAM (Random Access Memory) 630, and an HDD (Hard Disk Drive) 640. The computer 600 also includes a media interface (I / F) 650, a communication interface (I / F) 660, and an input / output interface (I / F) 670.

  The computer 600 may include an SSD (Solid State Drive), and the SSD may execute a part or all of the functions of the HDD 640. Further, an SSD may be provided instead of the HDD 640.

  The CPU 610 operates based on a program stored in at least one of the ROM 620 and the HDD 640 and controls each unit. The ROM 620 stores a boot program executed by the CPU 610 when the computer 600 starts up, a program depending on the hardware of the computer 600, and the like. The HDD 640 stores a program executed by the CPU 610, data used by the program, and the like.

  The media I / F 650 reads programs and data stored in the storage medium 680 and provides them to the CPU 610 via the RAM 630. The CPU 610 loads such a program from the storage medium 680 onto the RAM 630 via the media I / F 650, and executes the loaded program. Alternatively, the CPU 610 executes a program using such data. The storage medium 680 is, for example, a magneto-optical recording medium such as a DVD (Digital Versatile Disc), an SD card, or a USB memory.

  The communication I / F 660 receives data from other devices via the network 690 and sends the data to the CPU 610, and transmits the data generated by the CPU 610 to other devices via the network 690. Alternatively, the communication I / F 660 receives a program from another device via the network 690, sends the program to the CPU 610, and the CPU 610 executes the program.

  The CPU 610 controls an output unit such as the display unit 4 such as a head-up display and an input unit such as the operation unit 2 such as a keyboard, a mouse, and a button via the input / output I / F 670. The CPU 610 acquires data from the input unit via the input / output I / F 670. Further, the CPU 610 outputs the generated data to the display unit 4 and the output unit via the input / output I / F 670.

  For example, when the computer 600 functions as the information display system 1, the CPU 610 of the computer 600 executes a program loaded on the RAM 630, thereby setting the setting unit 11 a, the rearview mirror image generation unit 11 b, and the background image generation unit. The functions of the control units 11 and 11 ′ and the display control unit 3 of the image processing apparatuses 10 and 10 ′ including 11c and 11c ′ and the image synthesis unit 11d are realized.

  For example, the CPU 610 of the computer 600 reads these programs from the storage medium 680 and executes them, but as another example, these programs may be acquired from other devices via the network 690. Further, the HDD 640 can store information stored in the storage unit 12.

  By the way, in the above-described embodiment, the case where the outer frame image imitating the actual outer frame of the rearview mirror is not included in the electronic mirror image is exemplified, but the outer frame image may be included.

  In such a case, the rearview mirror image generation unit further generates an outer frame image imitating the outer frame of the actual rearview mirror, surrounds the periphery of the rearview mirror image by the outer frame image, and the image composition unit The background image, the outer frame image, and the rearview mirror image may be combined so that the outer frame image and the rearview mirror image are arranged on the screen.

  In the above-described embodiment, the case where the door mirror is mainly an electronic mirror has been described as an example. However, the present embodiment can also be applied to a case where a side under mirror or a room mirror is an electronic mirror.

  In the above-described embodiment, the case where the flow direction of the background image reminds at least the driver of the flow direction of the scenery flowing relative to the moving direction of the vehicle is taken as an example. However, when the driver sees the actual rearview mirror in the field of view, regardless of the moving direction of the vehicle, even if there is a background that flows in a different flow direction around the mirror image, only the mirror image is visible. There is a high possibility that the driver can recognize that it is a rearview mirror.

  Therefore, the background image may be a random pattern whose flow direction is random regardless of the moving direction of the vehicle, and for example, a geometric pattern may be turbulent around the rearview mirror image.

  Further effects and modifications can be easily derived by those skilled in the art. Thus, the broader aspects of the present invention are not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications can be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

DESCRIPTION OF SYMBOLS 1,1 'Information display system 2 Operation part 3 Display control part 4 Display part 10, 10' Image processing apparatus 11, 11 'Control part 11a Setting part 11b Rearview mirror image generation part 11c, 11c' Background image generation part 11ca Distortion correction Unit 11cb Viewpoint conversion calculation unit 11d Image composition unit 12 Storage unit 12a Background image information 20 Imaging unit 21 Rear camera 22 Front camera 30 Vehicle speed sensor 40 Rudder angle sensor 50 Road surface sensor Bi, Bi1, Bi2 Background image C Vehicle D1-D3 Display area ML, MR Door mirror Mi, Mi1, Mi2 Rearview mirror image P, P1, P2 Vanishing point aP1-aP3 Geometric pattern db, db1, db2 Flow direction

Claims (10)

A first image generation unit that generates a rearview mirror image obtained by inverting the rear image of the vehicle imaged by an imaging unit that images the outside of the vehicle;
A second image generation unit that generates a background image that flows at an arbitrary flow direction different from the rear-view mirror image at a speed according to the vehicle speed of the vehicle;
An image processing apparatus comprising: an image synthesis unit that synthesizes the background image and the rearview mirror image so that the rearview mirror image is arranged inside the background image. The second image generator is
The flow direction is determined so as to be a direction reminiscent of at least a moving direction of scenery flowing relative to the moving direction of the vehicle for the driver of the vehicle. Image processing device. The second image generator is
The image processing apparatus according to claim 1, wherein the background image including a geometric pattern is generated, and the geometric pattern is moved at a speed corresponding to the vehicle speed and in the flow direction. The second image generator is
The image processing apparatus according to claim 3, wherein the geometric pattern having a shape indicating the flow direction is used. The second image generator is
The image according to any one of claims 1 to 4, wherein a viewpoint conversion image obtained by converting a front image of the vehicle imaged by the imaging unit to an arbitrary viewpoint position is used as the background image. Processing equipment. The first image generator is
Further generating an outer frame image imitating the outer frame of the actual rearview mirror, surrounding the rearview mirror image by the outer frame image,
The image composition unit
The background image, the outer frame image, and the rearview mirror image are synthesized so that the outer frame image and the rearview mirror image are arranged inside the background image. The image processing apparatus according to any one of the above. The first image generator is
The image processing apparatus according to claim 1, wherein the rearview mirror image corresponding to a rearview mirror outside the vehicle body is generated. The image processing apparatus according to any one of claims 1 to 7,
An information display system comprising: a display unit that displays an image processed by the image processing device as driving support information. A first image generation step of generating a rearview mirror image obtained by inverting the rear image of the vehicle imaged by an imaging unit that images the outside of the vehicle;
A second image generation step of generating a background image flowing at an arbitrary flow direction different from the rear-view mirror image at a speed according to the vehicle speed of the vehicle;
And an image synthesis step of synthesizing the background image and the rearview mirror image so that the rearview mirror image is arranged inside the background image. A first image generation step of generating a rearview mirror image obtained by inverting the rear image of the vehicle imaged by an imaging unit that images the outside of the vehicle;
A second image generation step of generating a background image flowing at an arbitrary flow direction different from the rear-view mirror image at a speed according to the vehicle speed of the vehicle;
A program causing a computer to execute an image synthesis step of synthesizing the background image and the rearview mirror image so that the rearview mirror image is arranged inside the background image.

Images